| Message ID | 20211002102116.27726-1-ludo@gnu.org |
|---|---|
| Headers | show |
| Series | Add 'guix shell' to subsume 'guix environment' | expand |
Ludovic Courtès <ludo@gnu.org> writes: > Hello Guix! > > Here comes ‘guix shell’, a proposed replacement for ‘guix environment’! Suddenly thousands(/dozens?) of shell scripts fear for their continued existence :-) > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ > from the current directory or one of its ancestors. <snip> > 4. ‘guix shell’ without arguments maintains a cache, such that, the > second time you run it, it runs in ~0.1s (it does not even need to > connect to the daemon). > > If you run ‘guix pull’ and run again ‘guix shell’, it recomputes > the environment, as is currently the case with ‘guix environment’. <snip> > Thoughts? Are there other changes people would like to see? > > If there’s rough consensus I can work on v2 with documentation. Please > let’s keep the discussion focused. :-) At the risk of doing exactly not that; since it will(/might) already provide some automagic conveniences, perhaps it makes sense to additionally load a guix-channels.scm. I understand that there would be some duplication of functionality w.r.t. ‘guix time-machine -C guix-channels.scm -- shell ...’, but if we already go the DWIW-route, why not go all the way? I think this should only apply when running ‘guix shell’ without arguments, if that was unclear. Love the proposal - Jelle
guix shell is very interesting, but: On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote: > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ > from the current directory or one of its ancestors. This however is concerning. Users will not expect guix to execute arbitrary code. Maybe print a suggestion to maybe --file the file instead. Maybe I should not have such expectations but IMHO Guix is not like `haunt build`, which is expected to load haunt.scm. Regards, Florian
Ludovic Courtès <ludo@gnu.org> writes: > Hello Guix! > > Here comes ‘guix shell’, a proposed replacement for ‘guix environment’! > ‘guix environment’ would stay around though, at least for some time, > probably for a long time. > > The differences to ‘guix environment’ are: > > 1. ‘--ad-hoc’ is the default. > > ‘guix shell hello’ ≍ ‘guix environment --ad-hoc hello’ > ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’ > > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ > from the current directory or one of its ancestors. > > 3. ‘--load’/‘-l’ is not ‘-f’/‘--install-from-file’ for consistency with > ‘guix package’. > > 4. ‘guix shell’ without arguments maintains a cache, such that, the > second time you run it, it runs in ~0.1s (it does not even need to > connect to the daemon). > > If you run ‘guix pull’ and run again ‘guix shell’, it recomputes > the environment, as is currently the case with ‘guix environment’. > > Here’s a summary of previous proposals: > > - Dave Thompson: https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html > - [X] --ad-hoc is the default > - [X] caching > - [X] behavior with no arguments > - [ ] --load accepts <environment> > - [ ] Shepherd services > - [ ] 'guix environment --update' to explicitly update > - make --ad-hoc the default: https://issues.guix.gnu.org/38529 > - [X] https://issues.guix.gnu.org/38529#17: proposal for a new subcommand > deprecation of ‘guix environment’ > > I think <environment> records and Shepherd services could come later. > As for ‘--update’, I prefer the behavior implemented here because it’s > stateless and thus more predictable. > > Thoughts? Are there other changes people would like to see? > > If there’s rough consensus I can work on v2 with documentation. Please > let’s keep the discussion focused. :-) You have my massive thumbs up. Exciting stuff. > As for deprecation, I think there’s no rush. I imagine there could be > several phases, like: initially we only mention deprecation in the manual, > later on ‘guix environment’ starts emitting a warning, and later (I guess > at least two years later, probably more) we ask ourselves whether to > remove ‘guix environment’. At this point keeping it doesn’t cost us much. > > Thanks, > Ludo’. > > Ludovic Courtès (10): > packages: Add 'package-development-inputs'. > profiles: Add 'package->development-manifest'. > DRAFT Add 'guix shell'. > DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' > file. > environment: Add tests for '--profile'. > environment: Skip derivation computation when '--profile' is used. > environment: Do not connect to the daemon when '--profile' is used. > environment: Autoload some modules. > cache: Gracefully handle non-existent cache. > shell: Maintain a profile cache. > > Makefile.am | 2 + > doc/guix.texi | 52 ++++++ > guix/cache.scm | 10 +- > guix/packages.scm | 10 ++ > guix/profiles.scm | 19 ++ > guix/scripts/environment.scm | 260 +++++++++++++++------------- > guix/scripts/shell.scm | 254 +++++++++++++++++++++++++++ > po/guix/POTFILES.in | 1 + > tests/guix-environment-container.sh | 8 + > tests/guix-environment.sh | 7 + > tests/guix-shell.sh | 70 ++++++++ > tests/packages.scm | 14 ++ > tests/profiles.scm | 7 + > 13 files changed, 594 insertions(+), 120 deletions(-) > create mode 100644 guix/scripts/shell.scm > create mode 100644 tests/guix-shell.sh
Hi, "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis: > On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote: >> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ >> from the current directory or one of its ancestors. > > This however is concerning. Users will not expect guix to execute > arbitrary code. Maybe print a suggestion to maybe --file the file > instead. > > Maybe I should not have such expectations but IMHO Guix is not like > `haunt build`, which is expected to load haunt.scm. I think it’s fine as long as, as in the case of ‘haunt build’ or ‘make’ or ‘git’, it’s properly documented. Also, ‘guix shell’ unconditionally writes a message. Ludo’.
Hi! Jelle Licht <jlicht@fsfe.org> skribis: > At the risk of doing exactly not that; since it will(/might) already > provide some automagic conveniences, perhaps it makes sense to > additionally load a guix-channels.scm. > > I understand that there would be some duplication of functionality > w.r.t. ‘guix time-machine -C guix-channels.scm -- shell ...’, but if we > already go the DWIW-route, why not go all the way? I think this should > only apply when running ‘guix shell’ without arguments, if that was > unclear. Oh, it’s tempting yeah, but… (1) only ‘time-machine’ and ‘pull’ deal with channels currently, so it’d feel kind of weird to take care of that here, and (2) pulling a specific channel is resource-intensive as you know, much more than anything else, so I’d rather not have that happen automatically. But yeah, I agree that it could be useful. Maybe a first step we could make is have ‘time-machine’ load ‘channels.scm’, such that those who want it can type: guix time-machine -- shell ? Ludo’.
Ludovic Courtès <ludo@gnu.org> skribis: > The differences to ‘guix environment’ are: > > 1. ‘--ad-hoc’ is the default. > > ‘guix shell hello’ ≍ ‘guix environment --ad-hoc hello’ > ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’ > > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ > from the current directory or one of its ancestors. > > 3. ‘--load’/‘-l’ is not ‘-f’/‘--install-from-file’ for consistency with > ‘guix package’. > > 4. ‘guix shell’ without arguments maintains a cache, such that, the > second time you run it, it runs in ~0.1s (it does not even need to > connect to the daemon). > > If you run ‘guix pull’ and run again ‘guix shell’, it recomputes > the environment, as is currently the case with ‘guix environment’. Oh, I had another goal in mind, which was to make it easy to use ‘guix shell’ in shebangs. Problem is, since it’s called ‘guix shell’ and not ‘guix-shell’, we have to use ‘/usr/bin/env -S’ anyway as the shebang so it splits arguments. And with ‘-S’, we can already do pretty much everything we want. Here’s a shell script: --8<---------------cut here---------------start------------->8--- $ cat t.sh #!/usr/bin/env -S guix shell hello bash -- sh type -P hello hello --8<---------------cut here---------------end--------------->8--- Should we simply document it or should we do something more? For the record, ‘nix-shell’ has shebang support where it can interpret “special” lines as arguments: https://nixos.org/manual/nix/stable/#sec-nix-shell Ludo’.
On Sat, Oct 02, 2021 at 03:40:00PM +0200, Ludovic Courtès wrote: > "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis: > > On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote: > >> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ > >> from the current directory or one of its ancestors. > > This however is concerning. Users will not expect guix to execute > > arbitrary code. Maybe print a suggestion to maybe --file the file > > instead. > I think it’s fine as long as, as in the case of ‘haunt build’ or ‘make’ > or ‘git’, it’s properly documented. Also, ‘guix shell’ unconditionally > writes a message. Let’s say I have downloaded undesirable code to a file /home/florian/Downloads/guix.scm and am hacking on source code in /home/florian/Downloads/something/ where I run `guix shell`, but /home/florian/Downloads/something/ does not in fact contain a guix.scm file. Now I’d have accidentally run the other guix.scm. Also `make` is typically used without arguments, but a novice `guix shell` user might know `guix shell program-a program-b` but is surprised when running `guix shell` without arguments in an untrusted directory. But yes, git hooks are dangerous too. Regards, Florian
On 2021-10-02, Ludovic Courtès wrote: > Here comes ‘guix shell’, a proposed replacement for ‘guix environment’! Yay! > ‘guix environment’ would stay around though, at least for some time, > probably for a long time. > > The differences to ‘guix environment’ are: ... > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ > from the current directory or one of its ancestors. This sounds a little scary to me, just implicitly importing whatever happens to be lying around doesn't sound very guixy... Wouldn't it be better to: guix shell guix.scm or guix shell ./guix.scm or guix shell --some-argument guix.scm Or maybe I'm not understanding the idea all that well... live well, vagrant
Hi! Vagrant Cascadian <vagrant@debian.org> writes: > On 2021-10-02, Ludovic Courtès wrote: >> Here comes ‘guix shell’, a proposed replacement for ‘guix environment’! > > Yay! > >> ‘guix environment’ would stay around though, at least for some time, >> probably for a long time. >> >> The differences to ‘guix environment’ are: > ... >> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ >> from the current directory or one of its ancestors. > > This sounds a little scary to me, just implicitly importing whatever > happens to be lying around doesn't sound very guixy... > [...] What about doing something like what direnv[fn:1] does? Quoting the website: "direnv checks for the existence of a .envrc file in the current and parent directories. If the file exists (and is authorized), it is loaded into a bash sub-shell and all exported variables are then captured by direnv and then made available to the current shell." The difference between direnv and the current approach is that if the file has never been "authorized", before it being imported you need to run a command (direnv allow) to authorize it. There's the ~/.config/direnv/allow dir which stores files named with the hash of the content of the config, and whose content is just the path of the file (don't know why this is needed). This allows for automatic environment ({manifest,guix}.scm) file selection AND a it's a bit more secure (it won't run arbitrary code residing anywhere in the directory structure). Except for this, I'd love to see guix shell merged, it will be a major improvement over guix environment for my use cases. Thanks! Nicolò [fn:1] direnv.net
Ludovic Courtès <ludo@gnu.org> writes: > Oh, I had another goal in mind, which was to make it easy to use ‘guix > shell’ in shebangs. > > Problem is, since it’s called ‘guix shell’ and not ‘guix-shell’, we have > to use ‘/usr/bin/env -S’ anyway as the shebang so it splits arguments. > And with ‘-S’, we can already do pretty much everything we want. Here’s > a shell script: > > --8<---------------cut here---------------start------------->8--- > $ cat t.sh > #!/usr/bin/env -S guix shell hello bash -- sh > type -P hello > hello > --8<---------------cut here---------------end--------------->8--- I was already very excited about this patch series, and now I see this! I wanted to point out that I don't think Guix's shebang substitution phases know how to deal with ~env -S~, and I wish they would (I recently ran into this). I can't really think of any use-cases for a Guix package which when built contains a ~guix shell~ script, but I wanted to raise the possibility in case someone else can.
Hi, All looks good to me; from the surface, I have not looked at the details. On Sat, 02 Oct 2021 at 12:21, Ludovic Courtès <ludo@gnu.org> wrote: > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ > from the current directory or one of its ancestors. Personally, I do not like this default behaviour because explicit is better than implicit and in the face of ambiguity, refuse the temptation to guess; as any good Zen says. :-) Idem with the proposal ’channels.scm’ and time-machine. > As for deprecation, I think there’s no rush. I imagine there could be > several phases, like: initially we only mention deprecation in the manual, > later on ‘guix environment’ starts emitting a warning, and later (I guess > at least two years later, probably more) we ask ourselves whether to > remove ‘guix environment’. At this point keeping it doesn’t cost us much. Concretely, I propose this plan: - v1.4: mention deprecation in the manual and remove from “guix help” - v1.5: emit a warning - v1.6: remove the command Well, I do not see why the removal should be an issue, because there is “guix time-machine”. To me, the real issue is to let people knowing such change. Moreover, I am doubtful that the exact same command-line from v1.0 is producing the same result with v1.3 for instance. Not because of Guix but because of the underlying packages. For example, <http://issues.guix.gnu.org/47097>. All the best, simon
Hi, "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis: > On Sat, Oct 02, 2021 at 03:40:00PM +0200, Ludovic Courtès wrote: >> "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis: >> > On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote: >> >> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ >> >> from the current directory or one of its ancestors. >> > This however is concerning. Users will not expect guix to execute >> > arbitrary code. Maybe print a suggestion to maybe --file the file >> > instead. >> I think it’s fine as long as, as in the case of ‘haunt build’ or ‘make’ >> or ‘git’, it’s properly documented. Also, ‘guix shell’ unconditionally >> writes a message. > > Let’s say I have downloaded undesirable code to a file > /home/florian/Downloads/guix.scm and am hacking on source code in > /home/florian/Downloads/something/ where I run `guix shell`, but > /home/florian/Downloads/something/ does not in fact contain a > guix.scm file. Now I’d have accidentally run the other guix.scm. Sure, but it’s all under your control; it’s not very different from someone knowingly running “guix build -f guix.scm” on an untrusted file, is it? > Also `make` is typically used without arguments, but a novice `guix > shell` user might know `guix shell program-a program-b` but is > surprised when running `guix shell` without arguments in an untrusted > directory. We have the advantage that ‘guix shell’ is a new command, so we can document it from the start as behaving this way without arguments. WDYT? Thanks, Ludo’.
Hello! Vagrant Cascadian <vagrant@debian.org> skribis: > On 2021-10-02, Ludovic Courtès wrote: [...] >> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ >> from the current directory or one of its ancestors. > > This sounds a little scary to me, just implicitly importing whatever > happens to be lying around doesn't sound very guixy... Right, it would be the first command that does that. I became quite convinced that conventions and, thus, implicit arguments can occasionally improve usability. We use tools that operate this way daily: ‘make’, ‘git’, etc. Dave nicely argued about it: https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html But yeah, it is a departure from what we’ve done so far, so we should take the time to think through it. Thanks, Ludo’.
Hi, zimoun <zimon.toutoune@gmail.com> skribis: > On Sat, 02 Oct 2021 at 12:21, Ludovic Courtès <ludo@gnu.org> wrote: > >> 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ >> from the current directory or one of its ancestors. > > Personally, I do not like this default behaviour because explicit is > better than implicit and in the face of ambiguity, refuse the temptation > to guess; as any good Zen says. :-) It should be clear from the code that I generally prefer explicit over implicit :-), but I think Dave has a point when talking about conventions for this kind of developer tool. >> As for deprecation, I think there’s no rush. I imagine there could be >> several phases, like: initially we only mention deprecation in the manual, >> later on ‘guix environment’ starts emitting a warning, and later (I guess >> at least two years later, probably more) we ask ourselves whether to >> remove ‘guix environment’. At this point keeping it doesn’t cost us much. > > Concretely, I propose this plan: > > - v1.4: mention deprecation in the manual and remove from “guix help” > - v1.5: emit a warning > - v1.6: remove the command Could be like this. I guess we could also slow down the plan if we observe that ‘guix environment’ sticks around longer than we thought. > Well, I do not see why the removal should be an issue, because there is > “guix time-machine”. To me, the real issue is to let people knowing > such change. As discussed in <https://issues.guix.gnu.org/38529#17>, removal is an issue because of existing scripts but also because of learning material around: MOOCs, articles, tutorials, etc. These won’t be updated overnight, and we owe our users stability. Ludo’.
On Mon, Oct 04, 2021 at 10:22:54AM +0200, Ludovic Courtès wrote: > "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis: > > Let’s say I have downloaded undesirable code to a file > > /home/florian/Downloads/guix.scm and am hacking on source code in > > /home/florian/Downloads/something/ where I run `guix shell`, but > > /home/florian/Downloads/something/ does not in fact contain a > > guix.scm file. Now I’d have accidentally run the other guix.scm. > > Sure, but it’s all under your control; it’s not very different from > someone knowingly running “guix build -f guix.scm” on an untrusted file, > is it? What I meant is that I may wrongly expect a guix.scm file in /home/florian/Downloads/something/, but it is not there, so things go awry. `guix shell` loading files by default would mean one would have to pay attention to what one is doing, unlike `guix environment`. For example, not save unrelated (not even malicious) code by the name guix.scm, and not run guix commands without inspecting what they’d do. This I don’t like. > We have the advantage that ‘guix shell’ is a new command, so we can > document it from the start as behaving this way without arguments. Many people don’t read manuals. The probability of an accident is low, but it feels not robust. I can live with either (and am very happy you and others keep improving Guix), I just don’t think loading by default is a good idea. Regards, Florian
Hi, On Mon, 04 Oct 2021 at 10:39, Ludovic Courtès <ludo@gnu.org> wrote: > It should be clear from the code that I generally prefer explicit over > implicit :-), but I think Dave has a point when talking about > conventions for this kind of developer tool. Well, if you speak about this thread [1], the point is others are doing so and it is an usability improvement. :-) I am fine since somehow it is already the case with ’make’ or all the config files or many of us seem to simplify their workflow using direnv or etc. so yes it is probably handy to have conventions and automatic load by default. :-) 1: <https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html> >>> As for deprecation, I think there’s no rush. I imagine there could be >>> several phases, like: initially we only mention deprecation in the manual, >>> later on ‘guix environment’ starts emitting a warning, and later (I guess >>> at least two years later, probably more) we ask ourselves whether to >>> remove ‘guix environment’. At this point keeping it doesn’t cost us much. >> >> Concretely, I propose this plan: >> >> - v1.4: mention deprecation in the manual and remove from “guix help” >> - v1.5: emit a warning >> - v1.6: remove the command > > Could be like this. I guess we could also slow down the plan if we > observe that ‘guix environment’ sticks around longer than we thought. Bah, let reconsider this when preparing v1.6. ;-) At the current release rate, probably 2024. :-) From my point of view, we should clearly document at v1.4 that this command will be removed soon or later. >> Well, I do not see why the removal should be an issue, because there is >> “guix time-machine”. To me, the real issue is to let people knowing >> such change. > > As discussed in <https://issues.guix.gnu.org/38529#17>, removal is an > issue because of existing scripts but also because of learning material > around: MOOCs, articles, tutorials, etc. These won’t be updated > overnight, and we owe our users stability. For sure, I agree «we owe our users stability». That’s I translate into «the real issue is to let people knowing such change». :-) As I am trying to explain, this scenario, guix environment <options> <packages> … wait several months or years … guix pull guix environment <options> <packages> will break because <packages>. Therefore, one will have to fix these very same scripts. Even us, we are not able to maintain working scripts for different points in time. The example using “guix environment” is broken for the manual released at v1.3, as pointed here [2]. However, “guix environment” will still work using “guix time-machine”. Nothing is really broken, IMHO. About learning materials, we could remove the command “guix environment” but keep a section in the manual explaining how to switch to the new “guix shell”. Well, let discuss all that when preparing v1.5 or v1.6. Something as one or two years from now. ;-) Which i 2: <http://issues.guix.gnu.org/47097> Cheers, simon
zimoun <zimon.toutoune@gmail.com> skribis: > On Mon, 04 Oct 2021 at 10:39, Ludovic Courtès <ludo@gnu.org> wrote: [...] >> As discussed in <https://issues.guix.gnu.org/38529#17>, removal is an >> issue because of existing scripts but also because of learning material >> around: MOOCs, articles, tutorials, etc. These won’t be updated >> overnight, and we owe our users stability. > > For sure, I agree «we owe our users stability». That’s I translate into > «the real issue is to let people knowing such change». :-) No no. The problem is that there *will* be material out there explaining Guix in terms of ‘guix environment’, people *will* stumble on it, and it’s important to keep it working long enough so people can update their material. Ludo’.
On Mon, 4 Oct 2021 at 14:23, Ludovic Courtès <ludo@gnu.org> wrote: > zimoun <zimon.toutoune@gmail.com> skribis: > > For sure, I agree «we owe our users stability». That’s I translate into > > «the real issue is to let people knowing such change». :-) > > No no. The problem is that there *will* be material out there > explaining Guix in terms of ‘guix environment’, people *will* stumble on > it, and it’s important to keep it working long enough so people can > update their material. We are saying the same, no? Except that for me "people" and "material" are vague so I propose to add a section to deal with this vagueness at removal time (maybe 2023 or 2024?) explaining how to switch from "guix enviornment" to "guix shell". Therefore, people will stumble on this not-updated material, then they will be able to run it with "guix shell" using the proposed section. One could image: $ guix enviornment <old> <example> <how> <to> -- <use> <it> error: environment had be removed by commit: xxxxx. hint: Try `guix time-machine --commit=yyyyy -- environment <old> <example> <how> <to> -- <use> <it>' Please read section "guix environment" in the manual. where yyyy is that last commit right before the removal. Or something along this idea. Well, we agree to speak about that when preparing v1.5 or v1.6 and we are far from that. ;-) Cheers, simon
Ludovic Courtès schreef op ma 04-10-2021 om 10:22 [+0200]: > Hi, > > "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis: > > > On Sat, Oct 02, 2021 at 03:40:00PM +0200, Ludovic Courtès wrote: > > > "pelzflorian (Florian Pelz)" <pelzflorian@pelzflorian.de> skribis: > > > > On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote: > > > > > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ > > > > > from the current directory or one of its ancestors. > > > > This however is concerning. Users will not expect guix to execute > > > > arbitrary code. Maybe print a suggestion to maybe --file the file > > > > instead. > > > I think it’s fine as long as, as in the case of ‘haunt build’ or ‘make’ > > > or ‘git’, it’s properly documented. Also, ‘guix shell’ unconditionally > > > writes a message. > > > > Let’s say I have downloaded undesirable code to a file > > /home/florian/Downloads/guix.scm and am hacking on source code in > > /home/florian/Downloads/something/ where I run `guix shell`, but > > /home/florian/Downloads/something/ does not in fact contain a > > guix.scm file. Now I’d have accidentally run the other guix.scm. > > Sure, but it’s all under your control; it’s not very different from > someone knowingly running “guix build -f guix.scm” on an untrusted file, > is it? Consider the following situation: 1. I browse the web and find some rando's website. It has a link to a "guix.scm" to download. 2. I'd like to know how people are using guix, so I tell IceCat to download it. IceCat downloads it to ~/Downloads/guix.scm. 3. I forget about the guix.scm and didn't look at it. 4. I download some tarball, verify it (with gpg or something), unpack it, and run "guix shell" without arguments from within the directory (e.g. ~/Downloads/some-source-code). 5. It turns out the tarball didn't actually have a guix.scm, so the ~/Downloads/guix.scm from the rando is loaded. 6. It turns out the rando's guix.scm uploads my secret keys, passwords, all e-mails, installs a keylogger ... Oops! > > Also `make` is typically used without arguments, but a novice `guix > > shell` user might know `guix shell program-a program-b` but is > > surprised when running `guix shell` without arguments in an untrusted > > directory. > > We have the advantage that ‘guix shell’ is a new command, so we can > document it from the start as behaving this way without arguments. Sure, this behaviour can be documented, but it's very easy to forget a piece of documentation, especially if the behaviour is inconsistent between "guix environment" and "guix shell", and an attacker only needs an attack to function once. I'd prefer not be constantly kept on my toes, so if "guix shell" will automatically load guix.scm in the current directory or parent directories, I think I'll keep using "guix environment" to avoid any opportunities for fatal mistakes. Greetings, Maximes.
Ludovic Courtès schreef op ma 04-10-2021 om 10:34 [+0200]: > Hello! > > Vagrant Cascadian <vagrant@debian.org> skribis: > > > On 2021-10-02, Ludovic Courtès wrote: > > [...] > > > > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ > > > from the current directory or one of its ancestors. > > > > This sounds a little scary to me, just implicitly importing whatever > > happens to be lying around doesn't sound very guixy... > > Right, it would be the first command that does that. > > I became quite convinced that conventions and, thus, implicit arguments > can occasionally improve usability. We use tools that operate this way > daily: ‘make’, ‘git’, etc. Dave nicely argued about it: 'git' doesn't run binaries in the repository, unless configured otherwise (in .git/config I think). ‘make’ and ‘bundle’ are verbs and are for building source code, which needs to be checked for backdoors anyway, so those programs implicitely reading code from the current directory seems acceptable. "guix sh" seems to be useful outside software development. E.g. I sometimes do "guix environment --pure --ad-hoc minetest various-minetest-mods-... -- minetest", which would become "guix shell --pure minetest various-minetest-mods-... -- minetest". I could very easily accidentally press the enter key after typing "shell" (I write from personal experience), and this could easily happen from within, say, a ~/Downloads directory with an untrusted guix.scm (e.g. downloaded from some rando's site to look at later). Conventions are nice, but loading arbitrary code from the current directory by default is an exploit waiting to happen. This situation seem like including "." in PATH by default to me. Greetings, Maxime
On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote: > Here comes ‘guix shell’, a proposed replacement for ‘guix environment’! > ‘guix environment’ would stay around though, at least for some time, > probably for a long time. Very nice! > ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’ Is my understanding correct, that arguments to `guix environment` are positional, whereas with `guix shell` arguments are not positional?
Hey Ludo, First of all: Thanks for picking up where I left off all those years ago. You remember mailing list posts I wrote 4 years ago much better than I do. ;) On Sat, Oct 2, 2021 at 6:22 AM Ludovic Courtès <ludo@gnu.org> wrote: > > Hello Guix! > > Here comes ‘guix shell’, a proposed replacement for ‘guix environment’! > ‘guix environment’ would stay around though, at least for some time, > probably for a long time. > > The differences to ‘guix environment’ are: > > 1. ‘--ad-hoc’ is the default. > > ‘guix shell hello’ ≍ ‘guix environment --ad-hoc hello’ > ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’ > > 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ > from the current directory or one of its ancestors. > > 3. ‘--load’/‘-l’ is not ‘-f’/‘--install-from-file’ for consistency with > ‘guix package’. > > 4. ‘guix shell’ without arguments maintains a cache, such that, the > second time you run it, it runs in ~0.1s (it does not even need to > connect to the daemon). > > If you run ‘guix pull’ and run again ‘guix shell’, it recomputes > the environment, as is currently the case with ‘guix environment’. > > Here’s a summary of previous proposals: > > - Dave Thompson: https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html > - [X] --ad-hoc is the default > - [X] caching > - [X] behavior with no arguments > - [ ] --load accepts <environment> > - [ ] Shepherd services > - [ ] 'guix environment --update' to explicitly update > - make --ad-hoc the default: https://issues.guix.gnu.org/38529 > - [X] https://issues.guix.gnu.org/38529#17: proposal for a new subcommand > deprecation of ‘guix environment’ > > I think <environment> records and Shepherd services could come later. > As for ‘--update’, I prefer the behavior implemented here because it’s > stateless and thus more predictable. > > Thoughts? Are there other changes people would like to see? A few thoughts: 1) I can't be the only one that thinks it's a pain to rebuild the whole environment just because I updated the Guix client. It's especially frustrating when there is a regression that breaks the project, or the substitute servers are having a bad day. 2) One important use-case for tools like Bundler, npm, etc. is to exactly (within their limited ability to do so, of course) reproduce the environment for all developers working on the project. With 'guix environment', and now with this proposed 'guix shell', the resulting development environment depends upon the version of Guix being used. So, to make 'guix shell' a true universal replacement for these language-specific tools, there should be a way to "lock" to a specific version of Guix. Bundler has Gemfile.lock, npm has package-lock.json, etc. Guix has good support for using older versions of Guix already, so it seems feasible. I spend a non-trivial amount of time sorting out development environment issues for a couple dozen devs, so a tool that took most of the variables out of the equation would be really great. 3) I haven't looked at the patches to see if it has already been implemented, but when running 'guix shell' with no arguments, I think the search for the manifest file should follow the established conventions of Git, Bundler, etc. and search not only the current directory, but the parent directory and so on until it finds it or gets to / and gives up. This will make the tool much more usable when working inside a subdirectory within a project. Just some things to think about. I just don't want to see Guix commit to an interface that slams the door on future improvement due to compatibility reasons. I mean, what would you even name the next tool? guix... biome? Anyway, great work! Back to lurking. - Dave
Hi everyone,
I just discovered this discussion today... a lot to read...
First of all, I very much like this proposal. It provides many
improvements to today's "guix environment" for which I have immediate
use cases.
Looking at the proposal and then the discussion, I wonder if the main
issue with "guix shell" is that it's two very different things wrapped
into one:
1) A convenience utility for interactive work, in particular software
development, implemented as "guix shell" with no arguments.
2) A modernized "guix environment" for both interactive and scripting
use.
The behavior of "guix shell" with and without arguments is sufficiently
different, which will make it a challenge to document, and then a
challenge for users to understand. And part of the criticism related to
the tacit execution of files is about the convenience for 1) vs. the
risk in 2).
So... how about making this two different commands? They could of course
share most of the implementation.
I'd use "guix shell" for the scenario that actually starts a shell, and
something different, perhaps "guix process", for the infrastructure
command for use in scripts.
Cheers,
Konrad
Hi Konrad, Konrad Hinsen <konrad.hinsen@fastmail.net> skribis: > Looking at the proposal and then the discussion, I wonder if the main > issue with "guix shell" is that it's two very different things wrapped > into one: > > 1) A convenience utility for interactive work, in particular software > development, implemented as "guix shell" with no arguments. > > 2) A modernized "guix environment" for both interactive and scripting > use. To me it’s the same as ‘guix environment’, but with a slightly different command-line interface; ‘guix environment’ was already serves these two use cases. > The behavior of "guix shell" with and without arguments is sufficiently > different, which will make it a challenge to document, and then a > challenge for users to understand. And part of the criticism related to > the tacit execution of files is about the convenience for 1) vs. the > risk in 2). ‘guix shell’ without arguments is equivalent to ‘guix environment -f guix.scm’ or ‘guix environment -m manifest.scm’. > So... how about making this two different commands? They could of course > share most of the implementation. > > I'd use "guix shell" for the scenario that actually starts a shell, and > something different, perhaps "guix process", for the infrastructure > command for use in scripts. I don’t think there’s enough to warrant two different commands (and perhaps we could leave out the auto-detection of ‘guix.scm’ or ‘manifest.scm’ is that proves to be too controversial). Maybe you’re hinting at the name and the fact that it suggests it’s starting a shell. I’m not fond of it, notably for that reason. For the record, it came out as the only proposal that seemed viable to me: https://issues.guix.gnu.org/38529#17 Thoughts? Thanks for your feedback, Ludo’.
Hi Ludo, Am Donnerstag, den 07.10.2021, 10:34 +0200 schrieb Ludovic Courtès: > > The behavior of "guix shell" with and without arguments is > > sufficiently different, which will make it a challenge to document, > > and then a challenge for users to understand. And part of the > > criticism related to the tacit execution of files is about the > > convenience for 1) vs. the risk in 2). > > ‘guix shell’ without arguments is equivalent to ‘guix environment -f > guix.scm’ or ‘guix environment -m manifest.scm’. > > > So... how about making this two different commands? They could of > > course share most of the implementation. > > > > I'd use "guix shell" for the scenario that actually starts a shell, > > and something different, perhaps "guix process", for the > > infrastructure command for use in scripts. > > I don’t think there’s enough to warrant two different commands (and > perhaps we could leave out the auto-detection of ‘guix.scm’ or > ‘manifest.scm’ is that proves to be too controversial). Leaving out the auto-detection of ‘guix.scm’ and ‘manifest.scm’ for now is a good idea, given that Guix itself would be an edge-case for that. Perhaps to avoid this trouble, guix shell could read a file that simply specifies which arguments to add one line at a time e.g. ;; .guix-shell-rc --manifest=manifest.scm or ;; .guix-shell-rc --load=guix.scm Similar behaviour is used in other (GNU) tools with lots of available arguments, which operate on a directory basis, e.g. stow. > Maybe you’re hinting at the name and the fact that it suggests it’s > starting a shell. I’m not fond of it, notably for that reason. For > the record, it came out as the only proposal that seemed viable to > me: > > https://issues.guix.gnu.org/38529#17 > > Thoughts? I think “guix shell” is fine as-is. People typically don't complain that “sh somefile.sh” doesn't launch a shell, so the same reasoning can be applied to Guix tools in my opinion. We would still have the always non-intuitive naming choices of “do”, “start” and “run” at our disposal if we ever need to discard the shell. Regards, Liliana
Hi, Leo Famulari <leo@famulari.name> skribis: > On Sat, Oct 02, 2021 at 12:21:16PM +0200, Ludovic Courtès wrote: [...] >> ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’ > > Is my understanding correct, that arguments to `guix environment` are > positional, whereas with `guix shell` arguments are not positional? No, it’s the same. However, ‘-D’ here affects only the immediately following package, whereas ‘--ad-hoc’ would affect every package that follows. Ludo’.
Hi Ludo and Liliana, > To me it’s the same as ‘guix environment’, but with a slightly different > command-line interface; ‘guix environment’ was already serves these two > use cases. Indeed, but the new proposal goes much further with supporting interactive convenience, in particular via the controversial reading of guix.scm and/or manifest.scm. That's a convenience I'd like to have interactively, but not something I expect from an infrastructure tool that ends up being used in lengthy shell script. For the latter, the "Zen of Python wisdom" of "explicit is better than implicit" matters a lot. On the command line, it makes life harder, in particular for bad typists such as myself. > Perhaps to avoid this trouble, guix shell could read a file that simply > specifies which arguments to add one line at a time e.g. > > ;; .guix-shell-rc > --manifest=manifest.scm That's a nice idea, but again something I wouldn't want an infrastructure tool to do. When I use "guix environment" or "guix shell" in a script or workflow, I want whatever it does to be as independent as possible from my machine or account, except when I explicitly say the contrary. Cheers, Konrad
On Fri, Oct 08, 2021 at 09:43:19AM +0200, Ludovic Courtès wrote: > No, it’s the same. However, ‘-D’ here affects only the immediately > following package, whereas ‘--ad-hoc’ would affect every package that > follows. I must not know the correct terminology; that's what I was trying to ask. Can we keep the old behavior? Or do we think this new way of doing it is better?
Am Freitag, den 08.10.2021, 17:45 +0200 schrieb Konrad Hinsen: > Hi Ludo and Liliana, > > > To me it’s the same as ‘guix environment’, but with a slightly > > different command-line interface; ‘guix environment’ was already > > serves these two use cases. > > Indeed, but the new proposal goes much further with supporting > interactive convenience, in particular via the controversial reading > of guix.scm and/or manifest.scm. That's a convenience I'd like to > have interactively, but not something I expect from an infrastructure > tool that ends up being used in lengthy shell script. For the latter, > the "Zen of Python wisdom" of "explicit is better than implicit" > matters a lot. On the command line, it makes life harder, in > particular for bad typists such as myself. > > > Perhaps to avoid this trouble, guix shell could read a file that > > simply specifies which arguments to add one line at a time e.g. > > > > ;; .guix-shell-rc > > --manifest=manifest.scm > > That's a nice idea, but again something I wouldn't want an > infrastructure tool to do. When I use "guix environment" or "guix > shell" in a script or workflow, I want whatever it does to be as > independent as possible from my machine or account, except when I > explicitly say the contrary. People do use “emacs -Q” so there's no hard “one way or the other” requirement. All we would need to do is offer a similar flag to prevent reading the rc file, no? Cheers, Liliana
Hi! How about a directory argument to load some standard file? And using a special ... to search all parent directories? guix shell . ;# ./guix.scm guix shell .. ;# ../guix.scm guix shell ... ;# ./(../)*guix.scm Bye Stefan
Hi, Leo Famulari <leo@famulari.name> skribis: > On Fri, Oct 08, 2021 at 09:43:19AM +0200, Ludovic Courtès wrote: >> No, it’s the same. However, ‘-D’ here affects only the immediately >> following package, whereas ‘--ad-hoc’ would affect every package that >> follows. > > I must not know the correct terminology; that's what I was trying to > ask. > > Can we keep the old behavior? Or do we think this new way of doing it > is better? What do you mean by “old behavior”? I think there’s large consensus that ‘--ad-hoc’ should be the default, and that’s the main motivation behind this patch series. As for ‘-D’: we could arrange so that it has effect on all the following packages. However, if we did that, we’d need to reintroduce ‘--ad-hoc’ to cancel the effect of ‘-D’. More importantly, I think having ‘-D’ affect only the immediately following package is more consistent with typical usage—I don’t think I’ve ever run ‘guix environment foo bar baz’ to get the merged development environments of all three packages. Thanks, Ludo’.
On Sat, Oct 09, 2021 at 03:38:55PM +0200, Ludovic Courtès wrote: > What do you mean by “old behavior”? I mean that all the packages listed after the option are affected by it. > More importantly, I think having ‘-D’ affect only the immediately > following package is more consistent with typical usage—I don’t think > I’ve ever run ‘guix environment foo bar baz’ to get the merged > development environments of all three packages. I see what you mean: that is a weird behaviour. Let's disregard the question I brought up.
Hi, Konrad Hinsen <konrad.hinsen@fastmail.net> skribis: >> To me it’s the same as ‘guix environment’, but with a slightly different >> command-line interface; ‘guix environment’ was already serves these two >> use cases. > > Indeed, but the new proposal goes much further with supporting > interactive convenience, in particular via the controversial reading of > guix.scm and/or manifest.scm. That's a convenience I'd like to have > interactively, but not something I expect from an infrastructure tool > that ends up being used in lengthy shell script. For the latter, the > "Zen of Python wisdom" of "explicit is better than implicit" matters a > lot. On the command line, it makes life harder, in particular for bad > typists such as myself. Agreed. The automatic reading of guix.scm/manifest.scm, if we keep it, should only happen in interactive use; I’ll double-check and make sure this is the case. Thanks, Ludo’.
Hello Guix!
Here’s a v2 with the following changes:
• ‘guix.scm’ and ‘manifest.scm’ only loaded in interactive mode,
as suggested by Konrad.
• Auto-loading happens if and only if the containing directory
is listed in ~/.config/guix/shell-authorized-directories, as
suggested by Florian. When a file is found but not authorized,
a hint is displayed explaining what to do.
• ‘-q’ inhibits auto-loading.
• Auto-detection of ‘guix.scm’ and ‘manifest.scm’ uses lexical
“..” resolution and does not cross device boundaries.
• Long version of ‘-f’ is now ‘--file’, like for ‘guix build’.
• Caching also works with files explicitly specified with ‘-m’
or ‘-f’. That way, ‘guix shell -f guix.scm -- make’ benefits
from it.
• Cache key does not include file name, to avoid cache misses
given that the file name is not canonicalized (see discussion
with Maxime).
• Documentation, including a tentative EOL date for ‘guix
environment’ (“supported until at least May 1st, 2023”). The
intention is to give an incentive to switch while reassuring
that it won’t disappear overnight.
How does that sound? :-)
Ludo’.
Ludovic Courtès (11):
packages: Add 'package-development-inputs'.
profiles: Add 'package->development-manifest'.
Add 'guix shell'.
DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm'
file.
DRAFT shell: Honor in ~/.config/guix/shell-authorized-directories.
environment: Add tests for '--profile'.
environment: Skip derivation computation when '--profile' is used.
environment: Do not connect to the daemon when '--profile' is used.
environment: Autoload some modules.
cache: Gracefully handle non-existent cache.
shell: Maintain a profile cache.
Makefile.am | 2 +
doc/contributing.texi | 8 +-
doc/guix.texi | 443 ++++++++++++++++++++++++++--
guix/cache.scm | 10 +-
guix/packages.scm | 10 +
guix/profiles.scm | 19 ++
guix/scripts/environment.scm | 260 ++++++++--------
guix/scripts/shell.scm | 355 ++++++++++++++++++++++
po/guix/POTFILES.in | 1 +
tests/guix-environment-container.sh | 8 +
tests/guix-environment.sh | 7 +
tests/guix-shell.sh | 100 +++++++
tests/packages.scm | 14 +
tests/profiles.scm | 7 +
14 files changed, 1098 insertions(+), 146 deletions(-)
create mode 100644 guix/scripts/shell.scm
create mode 100644 tests/guix-shell.sh
Hello Guix! Here comes ‘guix shell’, a proposed replacement for ‘guix environment’! ‘guix environment’ would stay around though, at least for some time, probably for a long time. The differences to ‘guix environment’ are: 1. ‘--ad-hoc’ is the default. ‘guix shell hello’ ≍ ‘guix environment --ad-hoc hello’ ‘guix shell -D hello git’ ≍ ‘guix environment hello --ad-hoc git’ 2. ‘guix shell’, without arguments, loads ‘guix.scm’ or ‘manifest.scm’ from the current directory or one of its ancestors. 3. ‘--load’/‘-l’ is not ‘-f’/‘--install-from-file’ for consistency with ‘guix package’. 4. ‘guix shell’ without arguments maintains a cache, such that, the second time you run it, it runs in ~0.1s (it does not even need to connect to the daemon). If you run ‘guix pull’ and run again ‘guix shell’, it recomputes the environment, as is currently the case with ‘guix environment’. Here’s a summary of previous proposals: - Dave Thompson: https://lists.gnu.org/archive/html/guix-devel/2017-08/msg00300.html - [X] --ad-hoc is the default - [X] caching - [X] behavior with no arguments - [ ] --load accepts <environment> - [ ] Shepherd services - [ ] 'guix environment --update' to explicitly update - make --ad-hoc the default: https://issues.guix.gnu.org/38529 - [X] https://issues.guix.gnu.org/38529#17: proposal for a new subcommand deprecation of ‘guix environment’ I think <environment> records and Shepherd services could come later. As for ‘--update’, I prefer the behavior implemented here because it’s stateless and thus more predictable. Thoughts? Are there other changes people would like to see? If there’s rough consensus I can work on v2 with documentation. Please let’s keep the discussion focused. :-) As for deprecation, I think there’s no rush. I imagine there could be several phases, like: initially we only mention deprecation in the manual, later on ‘guix environment’ starts emitting a warning, and later (I guess at least two years later, probably more) we ask ourselves whether to remove ‘guix environment’. At this point keeping it doesn’t cost us much. Thanks, Ludo’. Ludovic Courtès (10): packages: Add 'package-development-inputs'. profiles: Add 'package->development-manifest'. DRAFT Add 'guix shell'. DRAFT shell: By default load the local 'guix.scm' or 'manifest.scm' file. environment: Add tests for '--profile'. environment: Skip derivation computation when '--profile' is used. environment: Do not connect to the daemon when '--profile' is used. environment: Autoload some modules. cache: Gracefully handle non-existent cache. shell: Maintain a profile cache. Makefile.am | 2 + doc/guix.texi | 52 ++++++ guix/cache.scm | 10 +- guix/packages.scm | 10 ++ guix/profiles.scm | 19 ++ guix/scripts/environment.scm | 260 +++++++++++++++------------- guix/scripts/shell.scm | 254 +++++++++++++++++++++++++++ po/guix/POTFILES.in | 1 + tests/guix-environment-container.sh | 8 + tests/guix-environment.sh | 7 + tests/guix-shell.sh | 70 ++++++++ tests/packages.scm | 14 ++ tests/profiles.scm | 7 + 13 files changed, 594 insertions(+), 120 deletions(-) create mode 100644 guix/scripts/shell.scm create mode 100644 tests/guix-shell.sh