Message ID | 20230107150457.4446-1-ludo@gnu.org |
---|---|
Headers | show |
Series | Add '--with-version' package transformation | expand |
Am 07.01.23 um 16:04 schrieb Ludovic Courtès: > Now that updaters can (to some extent) import any upstream version, we > can provide a ‘--with-version’ package transformation option that picks > the given upstream version. Cool! (Can't judge on the code, though)
Hi, The code LGTM. Minor the comment below. :-) On sam., 07 janv. 2023 at 16:04, Ludovic Courtès <ludo@gnu.org> wrote: > Now that updaters can (to some extent) import any upstream version, we > can provide a ‘--with-version’ package transformation option that picks > the given upstream version. Well, transformation bypasses the checksum control and so it is not really a fixed-output, right? Therefore, the result depends on the state of the world and thus it is not necessary reproducible. While --with-latest (or other transformation) makes this potential unreproducibility clear, that’s not the case with --with-version. For instance, Alice runs guix shell python python-scipy --with-version=python-numpy=1.22.4 and the result depends on what upstream provides for Numpy@1.22.4. Even if Alice provide a channels.scm file pinning Guix, then Bob does not get necessary the same as Alice with: guix time-machine -C channels.scm \ -- shell python python-scipy --with-version=python-numpy=1.22.4 Moreover, with my hat dealing with scientific folk, I already see their confusion. For instance, they will write some manifest such that, --8<---------------cut here---------------start------------->8--- (use-modules (guix transformations)) (define transform1 (options->transformation '((with-version . "python-numpy=1.22.4")))) (packages->manifest (list (transform1 (specification->package "python")) (transform1 (specification->package "python-scipy")))) --8<---------------cut here---------------end--------------->8--- which looks very similar to some requirements.txt or other YAML variants consumed by well-known other package managers. ;-) It is confusing because the common motto is «Guix is reproducible by design» but then here it is not. And what happens if the upstream content pointed by --with-version=python-numpy=1.22.4 is not available? It is not possible to fallback. All fall downss. Well, my concerns are also valid for other transformations but their names are clear enough to ring some bell for reproducing. The name ’with-version’ appears to me a trap because it eases incorrect description of the computational environment while thinking doing it right because using Guix. The issue is the connotation of the term “version” (*). (*) term “version”: people are often confused with the difference between source code version and produced binary version containing compilation options and dependencies; or also confused with the difference between extrinsic reference as label or tag and intrinsic reference as checksum. All that said, this option is cool to have. :-) It is missing some guards, IMHO. Maybe: 1. Wording some warning for the manual 2. Add comments about reproducibility when exporting manifest with transformations; for example, (define transform1 ;; WARNING: some transformation might lead to non-reproducible profile. ;; For more details, please see the Package Transformation Options section. (options->transformation '((with-version . "python-numpy=1.22.4") (with-c-toolchain . "python-scipy=gcc-toolchain@7")))) 3. Add a warning. Similarly, to this: warning: cannot authenticate source of 'python-numpy', version 1.22.4 For instance, warning: transformation for 'python-numpy' with version 1.22.4 might be non-reproducible Cheers, simon
Hello! Simon Tournier <zimon.toutoune@gmail.com> skribis: > On sam., 07 janv. 2023 at 16:04, Ludovic Courtès <ludo@gnu.org> wrote: > >> Now that updaters can (to some extent) import any upstream version, we >> can provide a ‘--with-version’ package transformation option that picks >> the given upstream version. > > Well, transformation bypasses the checksum control and so it is not > really a fixed-output, right? It’s not fixed-output at all because we don’t know in advance what we’ll get from that download. > Therefore, the result depends on the state of the world and thus it is > not necessary reproducible. Absolutely. [...] > All that said, this option is cool to have. :-) It is missing some > guards, IMHO. Maybe: > > 1. Wording some warning for the manual Will do, that makes a lot of sense to me, I share your concerns. > 2. Add comments about reproducibility when exporting manifest with > transformations; for example, > > (define transform1 > ;; WARNING: some transformation might lead to non-reproducible profile. > ;; For more details, please see the Package Transformation Options section. > (options->transformation > '((with-version . "python-numpy=1.22.4") > (with-c-toolchain . "python-scipy=gcc-toolchain@7")))) This one’s a bit more involved technically because we don’t use (guix read-print) yet here, so I prefer to do that separately. > 3. Add a warning. Similarly, to this: > > warning: cannot authenticate source of 'python-numpy', version 1.22.4 > > For instance, > > warning: transformation for 'python-numpy' with version 1.22.4 might be non-reproducible I wonder about this one because it would be displayed unconditionally, there’s no “I understand the risks” button one can click to make it disappear. I’d lean towards making a prominent warning in the manual and not have a warning message every time here, dunno. Thoughts? Thanks for your feedback! Ludo’.
Hi Ludo, On ven., 13 janv. 2023 at 12:12, Ludovic Courtès <ludo@gnu.org> wrote: > I wonder about this one because it would be displayed unconditionally, > there’s no “I understand the risks” button one can click to make it > disappear. I’d lean towards making a prominent warning in the manual > and not have a warning message every time here, dunno. Yeah, I agree it will be visually polluting. :-) Well, this patch set exposes the concern about reproducibility and some package transformation. Therefore, it could be addressed and discussed separately since it is unique to this --with-version. A first idea. :-) Maybe, this warning could appears the first time the transformation is called; similarly as “guix shell --check”. Or if we want to be really stringent, the first time of transformation calling, we simply raise ad message explaining it will not be reproducible, ask an Yes/No “I understand the risk”, store the result in a cache. This makes sure the information is spread. However, it could be annoying when launching in some non-interactive mode. The trick would be to set this cache but that’s not super handy. Bah, it needs more thoughts. :-) Cheers, simon
Hello, Pushed! 2e652e687e master origin/master news: Add entry for '--with-version'. 20c923c40d transformations: Let users know when '--with-latest' has no effect. 137b91f03b transformations: Add '--with-version'. I added a warning in the manual as you suggested, Simon; I also added a warning in ‘package-with-upstream-version’ about downgrades. I included Florian’s translation of the news entry too. Thanks everyone! Ludo’.