| Message ID | 87v8wg40dl.fsf@ngyro.com |
|---|---|
| Headers | show |
| Series | Remove old GNU utilities from early bootstrap | expand |
Hello! Timothy Sample <samplet@ngyro.com> skribis: > This is culmination of a lot of work, so I’m excited to be submitting > it! The main thrust of this series is to update Gash and Gash-Utils, > and then remove most of the old GNU utilities we use during early > bootstrap. To elaborate, the current situation is that we climb a > ladder through time: first we build Bash 2, then we build Bash 4, and > finally we build Bash 5. This is true for most GNU utilities: Sed, > Gawk, Coreutils, etc. The reason? Until now, our Scheme > implementations of those utilities (Gash and Gash-Utils) were very > limited. Bash 2 used to be a lot more useful then Gash, for example. > Now, with recent releases of Gash and Gash-Utils, the Scheme utilities > are, in general, capable of powering the builds of modern GNU software. Woohoo! Shortening the ladder, and doing it “the nice way”, is a much welcome improvement. > To be clear, we still climb the time-ladder in a few cases. The main > ones are GCC, Glibc, and Binutils. For example, we have to use GCC 2 to > transition from TCC To GCC 4 (which is the last non-C++ GCC [1]). > Fixing this would require quite a bit of TCC hacking, I imagine. There > are others, though. We still use old versions of Gzip, Make, patch, and > Gawk. The fact that Gawk is still there disappoints me quite a bit, but > ‘glibc-mesboot’ fails in a way that I just can’t figure out when > building with Gash-Utils. Gzip is not strictly necessary, but also > pretty easy to replace. I’m not sure, but I think patch is only there > to avoid using patches in ‘origin’ records during bootstrap. We now > have a way to do that, so it may no longer be necessary. Make will > likely need to be rewritten in Scheme, but we have a head start: Potato > Make [2]. > > [1] https://bootstrappable.org/projects.html > [2] https://github.com/spk121/potato-make Yes, the C compiler situation is tricky; Make looks like a fun target. [...] > The next three are updates. The bootar update optimistically assumes > that someone with access will upload the source file to the Guix mirror > URL: “mirror://gnu/guix/...” as was done with previous versions. Sure. > 5 files changed, 192 insertions(+), 788 deletions(-) Yay! Ludo’.
Timothy Sample <samplet@ngyro.com> skribis: > The next two patches spruce up the ‘gnu-bootstrap’ module. One fixes an > omission, and the other gets it ready to handle Gash-Utils 0.2.0, which > unfortunately has two module directories. > > The next three are updates. The bootar update optimistically assumes > that someone with access will upload the source file to the Guix mirror > URL: “mirror://gnu/guix/...” as was done with previous versions. > > The rest hopefully speak for themselves. Sadly, the last one is > something of a jumble. Originally I wanted to remove the utilities one > at a time, but it turns out there are interference effects. I can’t > remember the exact details now, but to get an idea, removing both old > Bash and old Grep might work, but removing one and then the other (in > either order) might not. Hence, many utilities get updated in one > commit. The patches all LGTM! I think you can push them to ‘core-updates’ if there are no objections. Then Maxim, Ricardo, or myself (I think we’re the three people who can do that currently) can upload bootar to ftp.gnu.org. Let us know. It’s quite something to see these intermediate versions of Bash, Gawk, tar, grep, coreutils, and sed go away in the last patch, and that tells something about the magnitude of the work that has gone in this Gash-Utils release. Thumbs up, and big thanks! Ludo’.
Hello, Ludovic Courtès <ludo@gnu.org> writes: > Timothy Sample <samplet@ngyro.com> skribis: > >> The next two patches spruce up the ‘gnu-bootstrap’ module. One fixes an >> omission, and the other gets it ready to handle Gash-Utils 0.2.0, which >> unfortunately has two module directories. >> >> The next three are updates. The bootar update optimistically assumes >> that someone with access will upload the source file to the Guix mirror >> URL: “mirror://gnu/guix/...” as was done with previous versions. >> >> The rest hopefully speak for themselves. Sadly, the last one is >> something of a jumble. Originally I wanted to remove the utilities one >> at a time, but it turns out there are interference effects. I can’t >> remember the exact details now, but to get an idea, removing both old >> Bash and old Grep might work, but removing one and then the other (in >> either order) might not. Hence, many utilities get updated in one >> commit. > > The patches all LGTM! I think you can push them to ‘core-updates’ if > there are no objections. Pushed! (Maybe I didn’t leave enough time for objections – apologies if it ends up requiring a revert! I’ll be around to work through any issues this might cause.) > Then Maxim, Ricardo, or myself (I think we’re the three people who can > do that currently) can upload bootar to ftp.gnu.org. Let us know. Yes, please! > It’s quite something to see these intermediate versions of Bash, Gawk, > tar, grep, coreutils, and sed go away in the last patch, and that tells > something about the magnitude of the work that has gone in this > Gash-Utils release. Thumbs up, and big thanks! When the Scheme-only bootstrap landed, it was very exciting, but it made a real mess of ‘commencement.scm’. Since I played a part in creating that mess, I figured I ought to work to fix it. :) I’m hoping our Scheme-heavy approach to bootstrapping will result in something clean and comprehensible, so taking an explicit step in that direction makes me happy. -- Tim
Timothy Sample writes: Hello! > This is culmination of a lot of work, so I’m excited to be submitting > it! The main thrust of this series is to update Gash and Gash-Utils, > and then remove most of the old GNU utilities we use during early > bootstrap. To elaborate, the current situation is that we climb a > ladder through time: first we build Bash 2, then we build Bash 4, and > finally we build Bash 5. This is true for most GNU utilities: Sed, > Gawk, Coreutils, etc. The reason? Until now, our Scheme > implementations of those utilities (Gash and Gash-Utils) were very > limited. Bash 2 used to be a lot more useful then Gash, for example. > Now, with recent releases of Gash and Gash-Utils, the Scheme utilities > are, in general, capable of powering the builds of modern GNU software. What an amazing piece of work. Not only adding support for this in Gash and Gash-Utils, also adding the "follow-up" for the Guix bootstrap. Wow! Not meaning to create more unwanted work for you, but I believe this could do with a blog post. I very much like your "climb a ladder through time" description. > To be clear, we still climb the time-ladder in a few cases. The main > ones are GCC, Glibc, and Binutils. For example, we have to use GCC 2 to > transition from TCC To GCC 4 (which is the last non-C++ GCC [1]). > Fixing this would require quite a bit of TCC hacking Yes, that needs work on MesCC, especially the Mes C Library, and TCC. Possibly some work on mpz/gmp. We definately want to jump from TCC to GCC 4.6 (the last modular distribution) in the (near) future. > We still use old versions of Gzip, Make, patch, and > Gawk. The fact that Gawk is still there disappoints me quite a bit, but > ‘glibc-mesboot’ fails in a way that I just can’t figure out when > building with Gash-Utils. Yeah, I can imagine, especially after your (2nd?) rewrite of AWK. > Gzip is not strictly necessary, but also pretty easy to replace. and replacing possibly has quite a speed penalty. > I’m not sure, but I think patch is only there > to avoid using patches in ‘origin’ records during bootstrap. We now > have a way to do that, so it may no longer be necessary. Yes, I believe patch is needed for the manual "apply-boot-patch" stages to avoid adding a dependency on xz, created by adding patches to "origin" records. > Make will likely need to be rewritten in Scheme, but we have a head > start: Potato Make [2]. > > [2] https://github.com/spk121/potato-make Interesting! POSIX Makefile Parser Recipes can contain the following parser function (parse ...) reads a standard Makefile and creates rules based on its contents. Building glibc without GNU Make may be pretty difficult, though. > Here’s some detailed info for reviewers. > Ludovic Courtès writes: > > The patches all LGTM! I think you can push them to ‘core-updates’ if > > there are no objections. > Pushed! (Maybe I didn’t leave enough time for objections – apologies if > it ends up requiring a revert! I’ll be around to work through any > issues this might cause.) Patches LGTM, and thanks for pushing. > The rest hopefully speak for themselves. Sadly, the last one is > something of a jumble. Originally I wanted to remove the utilities one > at a time, but it turns out there are interference effects. I can’t > remember the exact details now, but to get an idea, removing both old > Bash and old Grep might work, but removing one and then the other (in > either order) might not. Hence, many utilities get updated in one > commit. That's quite understandable, and certainly acceptable. Much of this time-ladder was created by me, it was an enormous effort to find old (and some ancient) versions of the tools that worked well together. There are undocumented build dependencies all around. Some newer packages cannot be used to build older versions of other packages, etc. So, while this kludge was a necessary evil, I'm extremely happy we can now do without it. We should work towards a bootstrap that depends mainly on current packages, and strive to keep current packages bootstrappable. To summarize what you removed: bzip2 (not an old version) sed-1.18 sed-4.0.6 binutils-2.14 (really great!) bash-2.05b bash-4.4 gawk-3.0.0 tar-1.22 grep-2.0 coreutils-5.0 xz-5.0.0 (wow, that was difficult to build) that's 11 old and ancient packages, if have counted correctly. Greetings, Janneke
Hi, Timothy Sample <samplet@ngyro.com> skribis: >> Then Maxim, Ricardo, or myself (I think we’re the three people who can >> do that currently) can upload bootar to ftp.gnu.org. Let us know. > > Yes, please! Done! https://ftp.gnu.org/gnu/guix/mirror/ Ludo’.