| Message ID | cover.1692834182.git.philip@philipmcgrath.com |
|---|---|
| Headers | show |
| Series | gnu: racket: Update to 8.10. | expand |
Hi Philip, Sorry for the late reply. I can confirm that Racket 8.10 builds flawlessly on aarch64 (tested on a Raspberry Pi 4B+ 8GB, took quite a while to build). I rebased the patches on commit b51e45d3aa, i.e. I refreshed the Guix sources yesterday. Thank you for all your work. - Tim
tags 65482 + security quit On 8/23/23 20:05, Philip McGrath wrote: > Hi, > > In addition to updating Racket to 8.10, this patch series backports fixes > merged upstream for rktboot on architectures other than x86_64 and removes > a corresponding workaround from the Guix packaging. > > Efraim and Tim, I'm CC'ing you because of your recent patches for rktboot on > aarch64 and riscv64: it would be great if you could confirm that this series > works on those architectures. It would also be useful to test powerpc64le, > especially since it is supported via 'pbarch', which takes some different > branches. > Apparently Racket 8.10 fixes a notable security vulnerability related to module path parsing. There's an initial post at <https://github.com/racket/racket/issues/4731>, but they're not publishing the details of how to exploit the vulnerability until more people have had a chance to upgrade. (I don't think I fully understand the implications of the issue myself.) Also, Tim, thanks for testing! I seem not to have gotten your mail, but I saw it on the tracker just now. Philip
On Sat, Sep 02, 2023 at 09:59:23PM -0400, Philip McGrath wrote: > tags 65482 + security > quit > > On 8/23/23 20:05, Philip McGrath wrote: > > Hi, > > > > In addition to updating Racket to 8.10, this patch series backports fixes > > merged upstream for rktboot on architectures other than x86_64 and removes > > a corresponding workaround from the Guix packaging. > > > > Efraim and Tim, I'm CC'ing you because of your recent patches for rktboot on > > aarch64 and riscv64: it would be great if you could confirm that this series > > works on those architectures. It would also be useful to test powerpc64le, > > especially since it is supported via 'pbarch', which takes some different > > branches. > > > > Apparently Racket 8.10 fixes a notable security vulnerability related to > module path parsing. There's an initial post at > <https://github.com/racket/racket/issues/4731>, but they're not publishing > the details of how to exploit the vulnerability until more people have had a > chance to upgrade. (I don't think I fully understand the implications of the > issue myself.) > > Also, Tim, thanks for testing! I seem not to have gotten your mail, but I > saw it on the tracker just now. Sorry for just getting to this now. As far as it working on riscv64, the test suite for racket didn't pass before, so there's no real possibility of regression on Guix's end. Currently it fails while building chez-scheme-for-racket-9.9.9-pre-release.17, but if upstream didn't notice then that's something else. starting phase `configure' source directory: "/tmp/guix-build-chez-scheme-for-racket-9.9.9-pre-release.17.drv-0/source/racket/src/ChezScheme" (relative from build: "../ChezScheme") build directory: "/tmp/guix-build-chez-scheme-for-racket-9.9.9-pre-release.17.drv-0/source/racket/src/build" configure flags: ("--disable-x11" "--threads" "-m=trv64le" "--installcsug=/gnu/store/c66pkyb1kvbi0jn1shanxrzbjvfqjmqf-chez-scheme-for-racket-9.9.9-pre-release.17-doc/share/doc/chez-scheme-for-racket-9.9.9-pre-release.17/csug" "--installreleasenotes=/gnu/store/c66pkyb1kvbi0jn1shanxrzbjvfqjmqf-chez-scheme-for-racket-9.9.9-pre-release.17-doc/share/doc/chez-scheme-for-racket-9.9.9-pre-release.17/release_notes" "--installprefix=/gnu/store/bqjwn04ix8xd9bwdni861244yza75qrf-chez-scheme-for-racket-9.9.9-pre-release.17" "ZLIB=-lz" "LZ4=-llz4" "--libkernel" "--nogzip-man-pages") No suitable machine type found in "../ChezScheme/boot". Available machine types: tpb64l See "../ChezScheme/BUILDING" for ways of getting boot files. I'll see about fixing the missing files or configure options. Don't let it not building on riscv64 delay this update though.
Hi, On 9/4/23 10:17, Efraim Flashner wrote: > On Sat, Sep 02, 2023 at 09:59:23PM -0400, Philip McGrath wrote: >> tags 65482 + security >> quit >> >> On 8/23/23 20:05, Philip McGrath wrote: >>> Hi, >>> >>> In addition to updating Racket to 8.10, this patch series backports fixes >>> merged upstream for rktboot on architectures other than x86_64 and removes >>> a corresponding workaround from the Guix packaging. >>> >>> [...] > > Sorry for just getting to this now. As far as it working on riscv64, the > test suite for racket didn't pass before, so there's no real possibility > of regression on Guix's end. Currently it fails while building > chez-scheme-for-racket-9.9.9-pre-release.17, but if upstream didn't > notice then that's something else. > > starting phase `configure' > source directory: "/tmp/guix-build-chez-scheme-for-racket-9.9.9-pre-release.17.drv-0/source/racket/src/ChezScheme" (relative from build: "../ChezScheme") > build directory: "/tmp/guix-build-chez-scheme-for-racket-9.9.9-pre-release.17.drv-0/source/racket/src/build" > configure flags: ("--disable-x11" "--threads" "-m=trv64le" "--installcsug=/gnu/store/c66pkyb1kvbi0jn1shanxrzbjvfqjmqf-chez-scheme-for-racket-9.9.9-pre-release.17-doc/share/doc/chez-scheme-for-racket-9.9.9-pre-release.17/csug" "--installreleasenotes=/gnu/store/c66pkyb1kvbi0jn1shanxrzbjvfqjmqf-chez-scheme-for-racket-9.9.9-pre-release.17-doc/share/doc/chez-scheme-for-racket-9.9.9-pre-release.17/release_notes" "--installprefix=/gnu/store/bqjwn04ix8xd9bwdni861244yza75qrf-chez-scheme-for-racket-9.9.9-pre-release.17" "ZLIB=-lz" "LZ4=-llz4" "--libkernel" "--nogzip-man-pages") > No suitable machine type found in "../ChezScheme/boot". > > Available machine types: > tpb64l > > See "../ChezScheme/BUILDING" for ways of getting boot files. > > I'll see about fixing the missing files or configure options. Don't let > it not building on riscv64 delay this update though. > Thanks for this report! I would have expected that to work, and it's tricky to test without hardware. Before getting into the weeds, I agree with you that it shouldn't block the update, especially if it was already broken. I'm not a Guix committer, but as far as I'm concerned this series is ready to merge. As far as riscv64, it looks like chez-scheme-for-racket-bootstrap-bootfiles created "portable bytecode" bootfiles ("tpb64l") instead of native riscv64 ones. You can confirm if that is the problem (or at least *a* problem) by checking if the lib/chez-scheme-bootfiles directory in the bootstrap package's output contains a directory named "tpb64l" instead of "trv64le". If that is indeed the problem, most likely either there is a bug in my change to rktboot's auto-detection or there were additional auto-detection bugs I didn't find. One way things could have gone wrong is if Racket BC returned something unexpected from (system-library-subpath #f). It would help to confirm the results of that, (system-type 'os*), and (system-type 'arch). In principle, if the problem is only with rktboot's auto-detection, it should work to just keep supplying the explicit --machine flag for now, i.e. drop patch 3/3 from this series. Racket doesn't have CI on riscv64 or distribute builds for it, but Matthew Flatt did share a nice screenshot earlier this summer of DrRacket running on a STAR64 :) Philip
On Mon, Sep 04, 2023 at 05:21:54PM -0400, Philip McGrath wrote: > Hi, > > On 9/4/23 10:17, Efraim Flashner wrote: > > On Sat, Sep 02, 2023 at 09:59:23PM -0400, Philip McGrath wrote: > > > tags 65482 + security > > > quit > > > > > > On 8/23/23 20:05, Philip McGrath wrote: > > > > Hi, > > > > > > > > In addition to updating Racket to 8.10, this patch series backports fixes > > > > merged upstream for rktboot on architectures other than x86_64 and removes > > > > a corresponding workaround from the Guix packaging. > > > > > > > > [...] > > > > Sorry for just getting to this now. As far as it working on riscv64, the > > test suite for racket didn't pass before, so there's no real possibility > > of regression on Guix's end. Currently it fails while building > > chez-scheme-for-racket-9.9.9-pre-release.17, but if upstream didn't > > notice then that's something else. > > > > starting phase `configure' > > source directory: "/tmp/guix-build-chez-scheme-for-racket-9.9.9-pre-release.17.drv-0/source/racket/src/ChezScheme" (relative from build: "../ChezScheme") > > build directory: "/tmp/guix-build-chez-scheme-for-racket-9.9.9-pre-release.17.drv-0/source/racket/src/build" > > configure flags: ("--disable-x11" "--threads" "-m=trv64le" "--installcsug=/gnu/store/c66pkyb1kvbi0jn1shanxrzbjvfqjmqf-chez-scheme-for-racket-9.9.9-pre-release.17-doc/share/doc/chez-scheme-for-racket-9.9.9-pre-release.17/csug" "--installreleasenotes=/gnu/store/c66pkyb1kvbi0jn1shanxrzbjvfqjmqf-chez-scheme-for-racket-9.9.9-pre-release.17-doc/share/doc/chez-scheme-for-racket-9.9.9-pre-release.17/release_notes" "--installprefix=/gnu/store/bqjwn04ix8xd9bwdni861244yza75qrf-chez-scheme-for-racket-9.9.9-pre-release.17" "ZLIB=-lz" "LZ4=-llz4" "--libkernel" "--nogzip-man-pages") > > No suitable machine type found in "../ChezScheme/boot". > > > > Available machine types: > > tpb64l > > > > See "../ChezScheme/BUILDING" for ways of getting boot files. > > > > I'll see about fixing the missing files or configure options. Don't let > > it not building on riscv64 delay this update though. > > > > Thanks for this report! I would have expected that to work, and it's tricky > to test without hardware. Ah, yeah, QEMU is really good but there are definitely times it isn't enough, and without real hardware it definitely falls into a "you want it, you keep it working" category. > Before getting into the weeds, I agree with you that it shouldn't block the > update, especially if it was already broken. I'm not a Guix committer, but > as far as I'm concerned this series is ready to merge. > > As far as riscv64, it looks like chez-scheme-for-racket-bootstrap-bootfiles > created "portable bytecode" bootfiles ("tpb64l") instead of native riscv64 > ones. You can confirm if that is the problem (or at least *a* problem) by > checking if the lib/chez-scheme-bootfiles directory in the bootstrap > package's output contains a directory named "tpb64l" instead of "trv64le". That's what I saw when I looked. I've been poking a bunch of packages recently so I don't remember exactly, but I think I tried building with the tpb64l bytecode and there were some issues later on which made that not work. > If that is indeed the problem, most likely either there is a bug in my > change to rktboot's auto-detection or there were additional auto-detection > bugs I didn't find. > > One way things could have gone wrong is if Racket BC returned something > unexpected from (system-library-subpath #f). It would help to confirm the > results of that, (system-type 'os*), and (system-type 'arch). > > In principle, if the problem is only with rktboot's auto-detection, it > should work to just keep supplying the explicit --machine flag for now, i.e. > drop patch 3/3 from this series. I've thought about it both ways, and since all the testing has been with the third patch included I'm going to push all three patches and then continue working on the riscv64 build. > Racket doesn't have CI on riscv64 or distribute builds for it, but Matthew > Flatt did share a nice screenshot earlier this summer of DrRacket running on > a STAR64 :) Patches pushed!
Hi, On 9/8/23 02:12, Efraim Flashner wrote: >>> build directory: "/tmp/guix-build-chez-scheme-for-racket-9.9.9-pre-release.17.drv-0/source/racket/src/build" >>> configure flags: ("--disable-x11" "--threads" "-m=trv64le" "--installcsug=/gnu/store/c66pkyb1kvbi0jn1shanxrzbjvfqjmqf-chez-scheme-for-racket-9.9.9-pre-release.17-doc/share/doc/chez-scheme-for-racket-9.9.9-pre-release.17/csug" "--installreleasenotes=/gnu/store/c66pkyb1kvbi0jn1shanxrzbjvfqjmqf-chez-scheme-for-racket-9.9.9-pre-release.17-doc/share/doc/chez-scheme-for-racket-9.9.9-pre-release.17/release_notes" "--installprefix=/gnu/store/bqjwn04ix8xd9bwdni861244yza75qrf-chez-scheme-for-racket-9.9.9-pre-release.17" "ZLIB=-lz" "LZ4=-llz4" "--libkernel" "--nogzip-man-pages") >>> No suitable machine type found in "../ChezScheme/boot". >>> >>> Available machine types: >>> tpb64l >>> >>> See "../ChezScheme/BUILDING" for ways of getting boot files. >>> >>> I'll see about fixing the missing files or configure options. Don't let >>> it not building on riscv64 delay this update though. >>> >> >> Thanks for this report! I would have expected that to work, and it's tricky >> to test without hardware. > > Ah, yeah, QEMU is really good but there are definitely times it isn't > enough, and without real hardware it definitely falls into a "you want > it, you keep it working" category. > In one of my early attempts to test for ppc64, I thought everything was working, but I'd actually just been producing x64 binaries. Then QEMU was crashing when running Racket BC, perhaps because of some complicated tricks it plays with the stack. >> Before getting into the weeds, I agree with you that it shouldn't block the >> update, especially if it was already broken. I'm not a Guix committer, but >> as far as I'm concerned this series is ready to merge. >> >> As far as riscv64, it looks like chez-scheme-for-racket-bootstrap-bootfiles >> created "portable bytecode" bootfiles ("tpb64l") instead of native riscv64 >> ones. You can confirm if that is the problem (or at least *a* problem) by >> checking if the lib/chez-scheme-bootfiles directory in the bootstrap >> package's output contains a directory named "tpb64l" instead of "trv64le". > > That's what I saw when I looked. Ok, at least it's clear why this state doesn't work, even if still I don't know why we fall into this state. (Note the "-m=trv64le" in #:configure-flags for chez-scheme-for-racket, which correctly specifies that we want the native backend, even though rktboot produced the wrong set of bootfiles. > I've been poking a bunch of packages > recently so I don't remember exactly, but I think I tried building with > the tpb64l bytecode and there were some issues later on which made that > not work. > A tpb64l build is supposed to work, but there are probably some rough edges. (In contrast, plain pb and tpb are not enough to be able to run Racket.) One thing I remember hearing is that the C compiler tends to take a long time on bootfiles compiled from bytecode to C, I guess because the C is large and strange-looking. Overall, I think you will have a better time getting the native backend to build. >> If that is indeed the problem, most likely either there is a bug in my >> change to rktboot's auto-detection or there were additional auto-detection >> bugs I didn't find. >> >> One way things could have gone wrong is if Racket BC returned something >> unexpected from (system-library-subpath #f). It would help to confirm the >> results of that, (system-type 'os*), and (system-type 'arch). >> Here's a way to test that in one long line: guix shell -e "(@ (gnu packages racket) racket-vm-bc)" -- sh -c "\${GUIX_ENVIRONMENT}/opt/racket-vm/bin/racket -e \"(list (path->string (system-library-subpath #f)) (system-type 'arch) (system-type 'os*))\"" On my machine, this prints '("x86_64-linux" x86_64 linux). On riscv64, I would expect it to produce '("riscv64-linux" riscv64 linux). If it produces something else, then my part of racket-backport-8.10-rktboot.patch definitely wouldn't work. On the other hand, if the result is as expected, then presumably there's some other bug elsewhere in rktboot to be patched. >> In principle, if the problem is only with rktboot's auto-detection, it >> should work to just keep supplying the explicit --machine flag for now, i.e. >> drop patch 3/3 from this series. > > I've thought about it both ways, and since all the testing has been with > the third patch included I'm going to push all three patches and then > continue working on the riscv64 build. > >> Racket doesn't have CI on riscv64 or distribute builds for it, but Matthew >> Flatt did share a nice screenshot earlier this summer of DrRacket running on >> a STAR64 :) > > Patches pushed! > Thanks! Philip