mbox series

[bug#53878,v5,00/22] Update Racket to 8.4. Adjust Chez Scheme packages.

Message ID 20220226130254.1301333-1-philip@philipmcgrath.com
Headers show
Series Update Racket to 8.4. Adjust Chez Scheme packages. | expand

Message

Philip McGrath Feb. 26, 2022, 1:02 p.m. UTC
Hi,

I've been ruminating for a while on Liliana's comment
from <https://issues.guix.gnu.org/53878#118>:

On Wednesday, February 23, 2022 3:31:34 PM EST Liliana Marie Prikler wrote:
> [...] there are some things that still
> don't feel right for me – for instance the fact, that seemingly
> unrelated modules now have to pull in racket bootstrap sounds like a
> recipe for trouble.  The final patch in the series also still does too
> much for me to wrap my head around, which makes it difficult to audit.
> 
> Therefore, one question I have w.r.t. updating Racket is whether we
> could theoretically bump the version while keeping the old bootstrap,
> and then adjust the bootstrap by adding all the packages you've made.
> It does seem to be an all or nothing deal when doing the bootstrap
> first, but that need not necessarily hold for bootstrap second.
> 
> Also, accepting for a moment that we might have to move chez-scheme and
> other important things into chez-scheme-and-racket-bootstrap (even
> though I'm not really content with it), I still wonder if we could
> introduce chez-scheme-for-system first (defined as simply chez-scheme
> initially) and adjust the callers, then move chez-scheme while keeping
> the function in chez.scm and finally do the magic with making it either
> chez or racket.
> 

It struck me that the core of the dependency cycle (at least, from one
perspective) is that 'chez-scheme-for-racket' wants to 'inherit' from
'chez-scheme' but use '%racket-origin' for its 'source', and neither of those
are thunked fields.

I realized that, if we just pass the origin some other way than as the
'source' field, we can avoid adding the "chez-and-racket-bootstrap.scm" file
altogether: patch v5 10/22 does the core of that.

I also managed to split up the update to Racket 8.4 (patch v4 15/15) into a
number of smaller steps (or, more precisely, rewrite it now that I knew what
the end result would be). I now have the 'racket-minimal*' packages gradually
evolve into the corresponding 'racket-vm-*' packages (rather than adding the
'racket-vm-*' stack in parallel), then split the new 'racket-minimal' package
out of 'racket'. Hopefully this might be somewhat easier to review. The
downside is there are now 22 patches, rather than 15.

So, here is v5!

 -Philip

P.S.: I've also posted this series
at <https://gitlab.com/philip1/guix-patches/-/tags/guix-issue-53878-v5>.

Philip McGrath (22):
  gnu: Use license prefix in (gnu packages racket).
  gnu: racket: Update to 8.4.
  gnu: racket: Use Git origins for Racket packages.
  gnu: racket-minimal: Use new package style.
  gnu: racket-minimal: Don't configure non-existant catalogs.
  gnu: racket-minimal: Change inheritance to follow bootstrapping.
  gnu: racket-minimal: Add "debug" output.
  gnu: make-installation-layer.rkt: Adjust indentation.
  gnu: racket-minimal: Separate from the Racket VM.
  gnu: racket: Move Chez bootfiles to (gnu packages chez).
  gnu: chez: Add utilities for Chez machine types.
  gnu: racket: Add 'racket-vm-for-system'.
  gnu: chez-scheme: Use "lib/chez-scheme" for search path.
  gnu: chez-scheme: Use shared zlib and lz4.
  gnu: chez-scheme: Use new package style.
  gnu: Add stex.
  gnu: Add chez-nanopass.
  gnu: chez-scheme: Explicitly package bootstrap bootfiles.
  gnu: chez-mit: Support chez-scheme-for-racket.
  gnu: Add chez-scheme-for-racket.
  gnu: chez: Add 'chez-scheme-for-system'.
  gnu: racket-vm-cs: Build with "--enable-scheme".

 gnu/local.mk                                  |    4 +-
 gnu/packages/chez.scm                         | 1264 +++++++----
 gnu/packages/emacs-xyz.scm                    |    2 +-
 gnu/packages/loko.scm                         |    2 +-
 .../racket-enable-scheme-backport.patch       |  465 ++++
 ...acket-gui-tethered-launcher-backport.patch |   26 +
 gnu/packages/racket.scm                       | 1866 ++++++++++++-----
 7 files changed, 2774 insertions(+), 855 deletions(-)
 create mode 100644 gnu/packages/patches/racket-enable-scheme-backport.patch
 create mode 100644 gnu/packages/patches/racket-gui-tethered-launcher-backport.patch

Comments

Liliana Marie Prikler Feb. 26, 2022, 3:08 p.m. UTC | #1
Hi,

Am Samstag, dem 26.02.2022 um 08:02 -0500 schrieb Philip McGrath:
> Hi,
> 
> I've been ruminating for a while on Liliana's comment
> from <https://issues.guix.gnu.org/53878#118>:
> 
> On Wednesday, February 23, 2022 3:31:34 PM EST Liliana Marie Prikler
> wrote:
> > [...] there are some things that still
> > don't feel right for me – for instance the fact, that seemingly
> > unrelated modules now have to pull in racket bootstrap sounds like
> > a recipe for trouble.  The final patch in the series also still
> > does too much for me to wrap my head around, which makes it
> > difficult to audit.
> > 
> > Therefore, one question I have w.r.t. updating Racket is whether we
> > could theoretically bump the version while keeping the old
> > bootstrap, and then adjust the bootstrap by adding all the packages
> > you've made.  It does seem to be an all or nothing deal when doing
> > the bootstrap first, but that need not necessarily hold for
> > bootstrap second.
> > 
> > Also, accepting for a moment that we might have to move chez-scheme
> > and other important things into chez-scheme-and-racket-bootstrap
> > (even though I'm not really content with it), I still wonder if we
> > could introduce chez-scheme-for-system first (defined as simply
> > chez-scheme initially) and adjust the callers, then move chez-
> > scheme while keeping the function in chez.scm and finally do the
> > magic with making it either chez or racket.
> > 
> 
> It struck me that the core of the dependency cycle (at least, from
> one perspective) is that 'chez-scheme-for-racket' wants to 'inherit'
> from 'chez-scheme' but use '%racket-origin' for its 'source', and
> neither of those are thunked fields.
I did realize that too, but perhaps I failed to mention that or didn't
get the point across before this, so pardon me for the extra work you
had to put in that I could have helped avoid.

> I realized that, if we just pass the origin some other way than as
> the 'source' field, we can avoid adding the
> "chez-and-racket-bootstrap.scm" file
> altogether: patch v5 10/22 does the core of that.
I did miss that nugget when I skimmed it first; is there a reason to
prefer overloading unpack and redirecting it to (package-source racket-
vm-bc) over doing the same, but using simply #$%racket-origin?

> I also managed to split up the update to Racket 8.4 (patch v4 15/15)
> into a number of smaller steps (or, more precisely, rewrite it now
> that I knew what the end result would be). I now have the 'racket-
> minimal*' packages gradually evolve into the corresponding 'racket-
> vm-*' packages (rather than adding the 'racket-vm-*' stack in
> parallel), then split the new 'racket-minimal' package
> out of 'racket'. Hopefully this might be somewhat easier to review.
> The downside is there are now 22 patches, rather than 15.
In general, smaller patches = more better.  I really like this series
so far, there's only some cosmetic nitpicks, although for the record I
do have to say that I skipped over many things that felt familiar from
earlier series.

BTW for the record, if you're dropping one of my mails from the CCs,
please make sure to include the gmail account rather than my institute
mail.  This one is technically supposed to be for work and I'm using a
rather loose interpretation of "ensuring that software is up-to-date"
as part of my work when I do comment on Guix issues from it.

I'll now attempt to build racket with this patch and hopefully
encounter no error as I do.

Cheers
Philip McGrath Feb. 26, 2022, 3:36 p.m. UTC | #2
Hi,

On Saturday, February 26, 2022 10:08:30 AM EST Liliana Marie Prikler wrote:
> Am Samstag, dem 26.02.2022 um 08:02 -0500 schrieb Philip McGrath:
> > I realized that, if we just pass the origin some other way than as
> > the 'source' field, we can avoid adding the
> > "chez-and-racket-bootstrap.scm" file
> > altogether: patch v5 10/22 does the core of that.
> 
> I did miss that nugget when I skimmed it first; is there a reason to
> prefer overloading unpack and redirecting it to (package-source racket-
> vm-bc) over doing the same, but using simply #$%racket-origin?
> 

I like this:

> > +            (replace 'unpack
> > +              (lambda args
> > +                (unpack #:source #$(or (package-source this-package)
> > +                                       (package-source racket-vm-bc)))))

rather than:

    (unpack #:source #$(package-source racket-vm-bc))

to make it easier for a user to provide an alternate source.

My concern with:

    (unpack #:source #$(or (package-source this-package) %racket-origin))

is less strong, but if `(gnu packages racket)` exports `%racket-origin`, it 
seems like it would be very tempting to put it in a `source` field, but of 
course that would cause problems. My hope was that having to write
`(package-source racket-vm-bc)` might prompt a little more thought.

> > I also managed to split up the update to Racket 8.4 (patch v4 15/15)
> > into a number of smaller steps (or, more precisely, rewrite it now
> > that I knew what the end result would be). I now have the 'racket-
> > minimal*' packages gradually evolve into the corresponding 'racket-
> > vm-*' packages (rather than adding the 'racket-vm-*' stack in
> > parallel), then split the new 'racket-minimal' package
> > out of 'racket'. Hopefully this might be somewhat easier to review.
> > The downside is there are now 22 patches, rather than 15.
> 
> In general, smaller patches = more better.  I really like this series
> so far, there's only some cosmetic nitpicks, although for the record I
> do have to say that I skipped over many things that felt familiar from
> earlier series.

Thanks!

> 
> BTW for the record, if you're dropping one of my mails from the CCs,
> please make sure to include the gmail account rather than my institute
> mail.  This one is technically supposed to be for work and I'm using a
> rather loose interpretation of "ensuring that software is up-to-date"
> as part of my work when I do comment on Guix issues from it.

Will do, sorry! (I've been experimenting with MUAs recently and not getting 
everything right—you may have noticed I sent mail earlier from an address I 
wasn't intending to use.)

> 
> I'll now attempt to build racket with this patch and hopefully
> encounter no error as I do.
> 
> Cheers

-Philip
Liliana Marie Prikler Feb. 26, 2022, 4:14 p.m. UTC | #3
Hi,

Am Samstag, dem 26.02.2022 um 10:36 -0500 schrieb Philip McGrath:
> Hi,
> 
> On Saturday, February 26, 2022 10:08:30 AM EST Liliana Marie Prikler
> wrote:
> > Am Samstag, dem 26.02.2022 um 08:02 -0500 schrieb Philip McGrath:
> > > I realized that, if we just pass the origin some other way than
> > > as the 'source' field, we can avoid adding the
> > > "chez-and-racket-bootstrap.scm" file
> > > altogether: patch v5 10/22 does the core of that.
> > 
> > I did miss that nugget when I skimmed it first; is there a reason
> > to prefer overloading unpack and redirecting it to (package-source
> > racket-vm-bc) over doing the same, but using simply #$%racket-
> > origin?
> > 
> 
> I like this:
> 
> > > +            (replace 'unpack
> > > +              (lambda args
> > > +                (unpack #:source #$(or (package-source this-
> > > package)
> > > +                                       (package-source racket-
> > > vm-bc)))))
> 
> rather than:
> 
>     (unpack #:source #$(package-source racket-vm-bc))
> 
> to make it easier for a user to provide an alternate source.
Hmm, to be fair this would probably respect --with-source, which I was
missing, but I'm not sure how deep into the bootstrap we could inject
this.  Sadly, we can't make chez-scheme-for-racket itself a
transformation.

> My concern with:
> 
>     (unpack #:source #$(or (package-source this-package) %racket-
> origin))
> 
> is less strong, but if `(gnu packages racket)` exports `%racket-
> origin`, it 
> seems like it would be very tempting to put it in a `source` field,
> but of course that would cause problems. My hope was that having to
> write `(package-source racket-vm-bc)` might prompt a little more
> thought.
Hmm.  And what if we go my earlier route of defining this as a
procedure in chez and instantiating it inside racket.scm?  We should
not get a cycle triggered by chez-scheme-for-system if we do, but we
might get one if we attempt to bind it to a variable in racket.scm.

That aside, there is also the possibility of using a gexp promise as
source, as is done for example in the case of linux-libre.  If we wrap
the chez side in a promise, that ought to resolve the cycle as well.

> > 
> > BTW for the record, if you're dropping one of my mails from the
> > CCs, please make sure to include the gmail account rather than my
> > institute mail.  This one is technically supposed to be for work
> > and I'm using a rather loose interpretation of "ensuring that
> > software is up-to-date" as part of my work when I do comment on
> > Guix issues from it.
> 
> Will do, sorry! (I've been experimenting with MUAs recently and not
> getting everything right—you may have noticed I sent mail earlier
> from an address I wasn't intending to use.)
I haven't noticed, but thanks for pointing this out.  If a commit from
the wrong address does make it into our git history, please do point
that out; I'll be happy to adjust the mailmap for you.

> 
Cheers
Philip McGrath Feb. 26, 2022, 5:31 p.m. UTC | #4
Hi,

On Saturday, February 26, 2022 11:14:38 AM EST Liliana Marie Prikler wrote:
> Hi,
> 
> Am Samstag, dem 26.02.2022 um 10:36 -0500 schrieb Philip McGrath:
> > Hi,
> > 
> > On Saturday, February 26, 2022 10:08:30 AM EST Liliana Marie Prikler
> > 
> > wrote:
> > > Am Samstag, dem 26.02.2022 um 08:02 -0500 schrieb Philip McGrath:
> > > > I realized that, if we just pass the origin some other way than
> > > > as the 'source' field, we can avoid adding the
> > > > "chez-and-racket-bootstrap.scm" file
> > > > altogether: patch v5 10/22 does the core of that.
> > > 
> > > I did miss that nugget when I skimmed it first; is there a reason
> > > to prefer overloading unpack and redirecting it to (package-source
> > > racket-vm-bc) over doing the same, but using simply #$%racket-
> > > origin?
> > 
> > I like this:
> > > > +            (replace 'unpack
> > > > +              (lambda args
> > > > +                (unpack #:source #$(or (package-source this-
> > > > package)
> > > > +                                       (package-source racket-
> > > > vm-bc)))))
> > 
> > rather than:
> > 
> >     (unpack #:source #$(package-source racket-vm-bc))
> > 
> > to make it easier for a user to provide an alternate source.
> 
> Hmm, to be fair this would probably respect --with-source, which I was
> missing, but I'm not sure how deep into the bootstrap we could inject
> this.  Sadly, we can't make chez-scheme-for-racket itself a
> transformation.
> 
> > My concern with:
> > 
> >     (unpack #:source #$(or (package-source this-package) %racket-
> > origin))
> > 
> > is less strong, but if `(gnu packages racket)` exports `%racket-
> > origin`, it
> > seems like it would be very tempting to put it in a `source` field,
> > but of course that would cause problems. My hope was that having to
> > write `(package-source racket-vm-bc)` might prompt a little more
> > thought.
> 
> Hmm.  And what if we go my earlier route of defining this as a
> procedure in chez and instantiating it inside racket.scm?  We should
> not get a cycle triggered by chez-scheme-for-system if we do, but we
> might get one if we attempt to bind it to a variable in racket.scm.
> 
> That aside, there is also the possibility of using a gexp promise as
> source, as is done for example in the case of linux-libre.  If we wrap
> the chez side in a promise, that ought to resolve the cycle as well.
> 

Interesting ... I'll have to take a look at that.

In the medium term, for Racket packages, I think the 'racket-build-system' 
ought to provide some support for using a different Racket. For Chez Scheme 
packages it's probably less complicated. (At least, I've had no problems with 
the things I've tried.)

For now, mostly I thought it would be very surprising if `unpack` silently 
ignored the package's `source` and unpacked something else, instead.

> > > BTW for the record, if you're dropping one of my mails from the
> > > CCs, please make sure to include the gmail account rather than my
> > > institute mail.  This one is technically supposed to be for work
> > > and I'm using a rather loose interpretation of "ensuring that
> > > software is up-to-date" as part of my work when I do comment on
> > > Guix issues from it.
> > 
> > Will do, sorry! (I've been experimenting with MUAs recently and not
> > getting everything right—you may have noticed I sent mail earlier
> > from an address I wasn't intending to use.)
> 
> I haven't noticed, but thanks for pointing this out.  If a commit from
> the wrong address does make it into our git history, please do point
> that out; I'll be happy to adjust the mailmap for you.
> 

Thankfully git and git-send-email have been fine; only a couple prose messages 
came from the wrong address.

-Philip