mbox series

[bug#53878,RFC,0/9] Update Racket to 8.4. Adjust Chez Scheme packages.

Message ID 20220208151316.1897345-1-philip@philipmcgrath.com
Headers show
Series Update Racket to 8.4. Adjust Chez Scheme packages. | expand

Message

Philip McGrath Feb. 8, 2022, 3:13 p.m. UTC
Racket 8.4 hasn't been released yet (expect it in a matter of days), but I
wanted to solicit feedback on these changes early.

On the Racket side of things, discussions upstream and with other distribution
packagers have clarified what a ``minimal Racket'' package ought to consist
of, which in turn suggested a nice new boundary between the lowest-level
Racket VM packages, which are concerned with bootstrapping and platform
support, and a more uniform use of layered and tethered installations for both
the 'racket-minimal' and 'racket' packages. I've also started getting the
sources for the other main-distribution Racket packages directly from their
respective repositories, rather than extracting them from the bundled
tarballs. All in all, it's a few steps closer to the goal of a build system
for Racket packages.

For Chez Scheme, I set out to package Racket's variant of Chez Scheme---which
supports architectures not yet supported upstream, e.g. aarch64---and to share
more between the upstream and Racket packages. I also tweaked various things I
found along the way: hopefully some are clear improvements, though there are
some prime opportunities for bikeshedding about naming things.

Let me know what you think!

 -Philip

Philip McGrath (9):
  gnu: chez-scheme: Move to (gnu packages chez-and-racket-bootstrap).
  gnu: chez-scheme: Use "lib/chez-scheme" for search path.
  gnu: chez-scheme: Use shared zlib and lz4.
  gnu: chez-and-racket-bootstrap: Add utilities for Chez machine types.
  gnu: Add stex.
  gnu: Add chez-nanopass.
  gnu: chez-scheme: Explicitly package bootstrap bootfiles.
  gnu: Add chez-scheme-racket-variant.
  gnu: racket: Update to 8.3.900.

 gnu/local.mk                                  |    5 +-
 gnu/packages/chez-and-racket-bootstrap.scm    | 1108 ++++++++++++
 gnu/packages/chez.scm                         |  628 +++----
 gnu/packages/emacs-xyz.scm                    |    4 +-
 gnu/packages/loko.scm                         |    4 +-
 .../racket-enable-scheme-backport.patch       |  465 +++++
 ...acket-gui-tethered-launcher-backport.patch |   26 +
 gnu/packages/racket.scm                       | 1552 +++++++++++------
 8 files changed, 2850 insertions(+), 942 deletions(-)
 create mode 100644 gnu/packages/chez-and-racket-bootstrap.scm
 create mode 100644 gnu/packages/patches/racket-enable-scheme-backport.patch
 create mode 100644 gnu/packages/patches/racket-gui-tethered-launcher-backport.patch

Comments

Philip McGrath Feb. 19, 2022, 10:34 p.m. UTC | #1
(sorry, missed "reply all" ...)

----------  Forwarded Message  ----------

Subject: Re: [PATCH v3 14/15] gnu: chez-and-racket-bootstrap: Add 'chez-
scheme-for-system'.
Date: Saturday, February 19, 2022, 5:29:05 PM EST
From: Philip McGrath <philip@philipmcgrath.com>
To: Liliana Marie Prikler <liliana.prikler@gmail.com>

Hi,

On Saturday, February 19, 2022 3:00:37 PM EST you wrote:
> Am Samstag, dem 19.02.2022 um 01:42 -0500 schrieb Philip McGrath:
> > +(define* (chez-scheme-for-system #:optional
> > +                                 (system (or (%current-target-
> > system)
> > +                                             (%current-system))))
> > +  "Return 'chez-scheme' unless only 'chez-scheme-for-racket'
> > supports SYSTEM,
> > +including support for native threads."
> > +  (if (and (nix-system->chez-machine system)
> > +           (not (and=> (chez-upstream-features-for-system system)
> > +                       (cut memq 'threads <>))))
> > +      chez-scheme-for-racket
> > +      chez-scheme))
> 
> Given your previous explanation this series looks clean enough so far,
> but this looks like a bug.  You probably want
> (if (and (nix-system->chez-machine system)
>     (and=> (chez-upstream-features-for-system system)
>            (cut memq 'threads <>)))
>     chez-scheme
>     chez-scheme-for-racket)
> This variant would in particular use chez-scheme-for-racket if nix-
> system->chez-machine returns #f.
> 
> I can make that adjustment for you assuming I interpreted this
> correctly.

Thanks for catching this! I must have gotten myself confused while changing to 
chez-upstream-features-for-system from the chez-machine->upstream-restriction 
version.

-Philip

-----------------------------------------
Philip McGrath Feb. 19, 2022, 10:35 p.m. UTC | #2
(... on this one, too)

----------  Forwarded Message  ----------

Subject: Re: [PATCH v3 09/15] gnu: Add racket-vm-cgc.
Date: Saturday, February 19, 2022, 5:27:39 PM EST
From: Philip McGrath <philip@philipmcgrath.com>
To: Liliana Marie Prikler <liliana.prikler@gmail.com>

Hi,

On Saturday, February 19, 2022 3:46:47 PM EST you wrote:
> Am Samstag, dem 19.02.2022 um 01:42 -0500 schrieb Philip McGrath:
> > * gnu/packages/patches/racket-enable-scheme-backport.patch: New
> > patch.
> > * gnu/local.mk (dist_patch_DATA): Add it.
> > * gnu/packages/chez-and-racket-bootstrap.scm (unbundle-chez-
> > submodules,
> > %racket-version, %racket-origin, racket-vm-cgc): New variables.
> > (chez-scheme)[source]<snippet>: Use 'unbundle-chez-submodules'.
> 
> Something weird happened to me just now trying to build this series.
> While compiling emacs-xyz, an error was raised regarding %racket-
> version not existing and and import being missing.  Assuming that you
> didn't mess up an include somewhere, there could be a cycle meaning
> we'd have to pass the version and origin by function as I originally
> said in my reply to v2.
> 
> I'll clean up my work tree and try to reapply it, but that will take
> some time.  Since you mentioned re-exports causing issues, I'd like to
> ask if you've made a similar experience, but the results got somehow
> hidden after the right incantations.

I haven't seen errors from emacs-xyz, but I have gotten errors about %racket-
version not existing: at the time I thought it was just a problem with 
incremental rebuilds while moving back and forward through history, but, 
having just refreshed my memory on more details of the cyclic issues, I think 
it may be related. I'll send another email presently with details once I've 
gathered references.

For now, I found that `rm gnu/packages/*.go` was enough to get `make` to 
succeed again.

-Philip
-----------------------------------------
Philip McGrath Feb. 19, 2022, 11:24 p.m. UTC | #3
Hi,

On Saturday, February 19, 2022 5:35:22 PM EST Philip McGrath wrote:
> On Saturday, February 19, 2022 3:46:47 PM EST you wrote:
> > Am Samstag, dem 19.02.2022 um 01:42 -0500 schrieb Philip McGrath:
> > > * gnu/packages/patches/racket-enable-scheme-backport.patch: New
> > > patch.
> > > * gnu/local.mk (dist_patch_DATA): Add it.
> > > * gnu/packages/chez-and-racket-bootstrap.scm (unbundle-chez-
> > > submodules,
> > > %racket-version, %racket-origin, racket-vm-cgc): New variables.
> > > (chez-scheme)[source]<snippet>: Use 'unbundle-chez-submodules'.
> > 
> > Something weird happened to me just now trying to build this series.
> > While compiling emacs-xyz, an error was raised regarding %racket-
> > version not existing and and import being missing.  Assuming that you
> > didn't mess up an include somewhere, there could be a cycle meaning
> > we'd have to pass the version and origin by function as I originally
> > said in my reply to v2.
> > 
> > I'll clean up my work tree and try to reapply it, but that will take
> > some time.  Since you mentioned re-exports causing issues, I'd like to
> > ask if you've made a similar experience, but the results got somehow
> > hidden after the right incantations.
> 
> I haven't seen errors from emacs-xyz, but I have gotten errors about
> %racket- version not existing: at the time I thought it was just a problem
> with incremental rebuilds while moving back and forward through history,
> but, having just refreshed my memory on more details of the cyclic issues,
> I think it may be related. I'll send another email presently with details
> once I've gathered references.
> 
> For now, I found that `rm gnu/packages/*.go` was enough to get `make` to
> succeed again.

This reminded me of one of the commits I mentioned yesterday in <https://
issues.guix.gnu.org/53878#66>:

> commit 96db2ff145ecbd962206eae815b065bda7ed3d9f
> Author: Ludovic Courtès <ludo@gnu.org>
> Date: Tue Sep 7 15:11:46 2021 +0200
>
> gnu: racket-minimal: Remove top-level reference to 'chez-scheme'.
>
> This could cause build errors; for instance, doing:
> make && touch gnu/packages/chez.scm && make
>
> would trigger a "chez-scheme: unbound variable" error.
>
> * gnu/packages/racket.scm (racket-minimal)[source]: Add 'modules'
> field. In 'snippet', remove top-level reference to CHEZ-SCHEME, which
> could cause build errors. Simplify snippet.

That symptom of an unbound variable error on re-making sounds like what's 
going on.

In <https://issues.guix.gnu.org/48682#7>, Ludo’ explained:

On Monday, May 31, 2021 12:23:27 PM EST Ludovic Courtès wrote:
> Philip McGrath <philip@philipmcgrath.com> skribis:
> > On 5/29/21 4:15 PM, Ludovic Courtès wrote:
> >> In general we cannot use #:select for (gnu packages …) modules because
> >> that doesn’t play well with circular module dependencies.
> > 
> > Ah, interesting, I'll keep that in mind. I'm used to Racket, where all
> > cyclic module dependencies cause errors at compile time.
> 
> Yeah, in hindsight, that’s probably safer…
> 
> > Do you have any advice on what would be good practice?
> 
> For package modules, the main things are:
> 
>   1. Don’t use #:select or #:autoload for (gnu packages …) modules in a
>      (gnu packages …) module.
> 
>   2. At the top level of a module, only refer to variables within that
>      module.  For instance, the following would be wrong:
> 
>        (define-module (gnu packages racket)
>          #:use-module (gnu packages chez)
>          …)
> 
>        (define whatever
>          ;; Wrong because ‘chez-scheme’ is defined in another module,
>          ;; which might be part of a cycle with this one.
>          (package (inherit chez-scheme) …))
> 
>        (define something
>          (package
>            ;; …
>            (license (package-license chez-scheme))))  ;likewise
> 
> Note that references from ‘inputs’ and ‘arguments’ fields are perfectly
> fine (fortunately!) because those fields are “thunked” (their value is
> wrapped in a thunk).
> 

My understanding of the semantics of Guile modules is strictly less than the 
content of that thread. I'm most familiar with Racket modules, and I have some 
understanding of how those semantics differ from R6RS library semantics.

I think I had seized in my memory of № 2 on  "at the top level of a module", 
and I'd filled in my usual (Racket) understanding of "at the top level of a 
module" as a less formal way of saying "in a module context", in the sense of 
"expansion contexts".[1]

But  Ludo’'s examples show that's wrong: those uses of `chez scheme` are in 
what the "expansion contexts" model would call "expression contexts".

Instead, I think rule № 2 prohibits any reference to a variable imported from 
another (gnu packages ...) module that will be evaluated when the (gnu 
packages ...) modules are—visited? instantiated? [2][3]—IDK when exactly, but, 
for practical purposes, any variable reference that is not underneath a lambda 
abstraction.

If that's right, IIUC, it would mean that:

    (define chez-scheme-for-racket
      (make-chez-scheme-for-racket ...))

would also be prohibited.

On the other hand, uses of `(racket-vm-for-system)` and `(chez-scheme-for-
system)` in an `imports` field should still be fine, thanks to the implicit 
thunks.

I think I can can make that work relatively well. The one thing I don't know 
how to avoid is defining `%racket-version` in both "chez-and-racket-
bootstrap.scm" and "racket.scm", but I'll leave a cautionary comment.

I'll try to make a v4 with the approach I have in mind. I'd be glad to hear 
better ideas, though! Especially if anyone knows more about what Guile's 
module semantics really are.

-Philip

[1]: https://docs.racket-lang.org/reference/syntax-model.html#%28part._expand-context-model%29
[2]: http://www.r6rs.org/final/html/r6rs/r6rs-Z-H-10.html#node_sec_7.2
[4]: https://docs.racket-lang.org/reference/syntax-model.html#%28part._mod-parse%29