| Message ID | 20201203105353.149482-1-othacehe@gnu.org |
|---|---|
| Headers | show |
Hi Mathieu, On Thu, 03 Dec 2020 at 11:53, Mathieu Othacehe <othacehe@gnu.org> wrote: > Finally, as discussed with Danny on IRC, it could make sense to change the > default image type depending on the current system: efi-raw on x86_64-linux > and i686-linux, arm32-raw on armhf-linux and so on. The “less astonishment” default sounds good to me. :-) Cheers, simon
On Thu, Dec 03, 2020 at 11:53:51AM +0100, Mathieu Othacehe wrote: > Hello, > > Here's a small patchset to improve the creation of disk-images on non-Intel > systems. Currently, when selecting "arm32-raw" or "arm64-raw" image types, > "guix system" will try to cross-compile to the relevant architectures, > regardless of the current system architecture. > > This adds a "system" field to the image definition that indicates the > appropriate system. Then, if we are already running on this system, > "system-image" will build the image natively instead of using > cross-compilation. The image type "raw" is also renamed to "efi-raw" which is > more accurate. > > Finally, as discussed with Danny on IRC, it could make sense to change the > default image type depending on the current system: efi-raw on x86_64-linux > and i686-linux, arm32-raw on armhf-linux and so on. > > WDYT? I'm not sure about i686-linux being an efi-raw type. I always assumed they were all not EFI. > Thanks, > > Mathieu > > Mathieu Othacehe (2): > image: Add system field. > image: Rename "raw" image-type to "efi-raw". > > doc/guix.texi | 10 +++++----- > gnu/image.scm | 3 +++ > gnu/system/image.scm | 18 ++++++++++++++---- > guix/scripts/system.scm | 2 +- > 4 files changed, 23 insertions(+), 10 deletions(-) > > -- > 2.29.2 > > > >
Hello Efraim, > I'm not sure about i686-linux being an efi-raw type. I always assumed > they were all not EFI. The "efi-raw" image type is actually an hybrid image type supporting EFI and BIOS legacy boot methods. Maybe we should then rename "raw" to "bios-efi-raw" or "bios-efi-hybrid-raw", what do you think? Thanks, Mathieu
On Wed, Dec 09, 2020 at 11:15:32AM +0100, Mathieu Othacehe wrote: > > Hello Efraim, > > > I'm not sure about i686-linux being an efi-raw type. I always assumed > > they were all not EFI. > > The "efi-raw" image type is actually an hybrid image type supporting EFI > and BIOS legacy boot methods. > > Maybe we should then rename "raw" to "bios-efi-raw" or > "bios-efi-hybrid-raw", what do you think? My hope is to eventually produce an image for mips64el again, which would need an ext2 formatted /boot partition. I'm hoping to make it not too hard to hack that in in the future :) Perhaps it makes sense to name them according to the architecture more? we have arm32-raw and aarch64-raw, how about i686-raw and x86_64-raw? Make it clear exactly what architecures they target. Then we can continue with the hybrid bios/efi images anyway. > Thanks, > > Mathieu
Hi! Mathieu Othacehe <othacehe@gnu.org> skribis: > Here's a small patchset to improve the creation of disk-images on non-Intel > systems. Currently, when selecting "arm32-raw" or "arm64-raw" image types, > "guix system" will try to cross-compile to the relevant architectures, > regardless of the current system architecture. > > This adds a "system" field to the image definition that indicates the > appropriate system. Then, if we are already running on this system, > "system-image" will build the image natively instead of using > cross-compilation. The image type "raw" is also renamed to "efi-raw" which is > more accurate. > > Finally, as discussed with Danny on IRC, it could make sense to change the > default image type depending on the current system: efi-raw on x86_64-linux > and i686-linux, arm32-raw on armhf-linux and so on. I understand the need for an easier way to create images. However, I feel like <image> is the wrong place for ‘system’ and ‘target’: the image format, conceptually, has nothing to do with whether we’re cross-compiling, compiling for a specific system, etc. It also seems wrong to me that ‘--image-type’ would, in some cases (but not all?), override ‘-s’ and ‘--target’. I feel like we’re missing an abstraction that would build on top of images, but I’m not sure what that would look like. Thoughts? Ludo’.
Hey, > I understand the need for an easier way to create images. However, I > feel like <image> is the wrong place for ‘system’ and ‘target’: the > image format, conceptually, has nothing to do with whether we’re > cross-compiling, compiling for a specific system, etc. On the one hand, I agree that adding "system" and "target" to <image>, so that they can override the corresponding arguments doesn't feel nice. On the other hand, I think that dealing with system/target is too low level for most users. When using Yocto, Buildroot or even OpenWrt, you say "build me an image for that board/machine" and not, "build me an image for that board by cross-compiling to this mysterious triplet". If the user selects the image type "pine64" or "novena", it's obvious that the image has to be built for ARM, so I think it makes sense to hardcode it somewhere. The <image> record might not be the right location for that information but I cannot think of another one. Maybe someone else? Thanks, Mathieu
Hi, On Sat, 12 Dec 2020 at 09:30, Mathieu Othacehe <othacehe@gnu.org> wrote: > When using Yocto, Buildroot or even OpenWrt, you say "build me an image > for that board/machine" and not, "build me an image for that board by > cross-compiling to this mysterious triplet". I confirm that the triplet is still mysterious to me. Since I do not do that often, each time I am trying, I need to browse the doc, when I am not asking again and again on IRC. Therefore, I do not know if the record is the correct abstraction but somehow a mapping helper is welcome. :-) Maybe via a command-line option displaying the “board“ and the “triplet”. All the best, simon
Hi! Mathieu Othacehe <othacehe@gnu.org> skribis: >> I understand the need for an easier way to create images. However, I >> feel like <image> is the wrong place for ‘system’ and ‘target’: the >> image format, conceptually, has nothing to do with whether we’re >> cross-compiling, compiling for a specific system, etc. > > On the one hand, I agree that adding "system" and "target" to <image>, > so that they can override the corresponding arguments doesn't feel > nice. On the other hand, I think that dealing with system/target is too > low level for most users. > > When using Yocto, Buildroot or even OpenWrt, you say "build me an image > for that board/machine" and not, "build me an image for that board by > cross-compiling to this mysterious triplet". Agreed; I understand that this is a desirable level of abstraction. > If the user selects the image type "pine64" or "novena", it's obvious > that the image has to be built for ARM, so I think it makes sense to > hardcode it somewhere. The <image> record might not be the right > location for that information but I cannot think of another one. OTOH, I might want to cross-build a Novena image from x86_64, or I might want to build it natively. Perhaps what could be said is that a Novena image can either be built natively on armhf-linux, or cross-built to arm-linux-gnueabihf. Perhaps we should encode this constraint rather than a specific ‘system’ or ‘target’? (I’m thinking out loud…) Regarding ARM boards, do you think some additional abstraction is needed to encode cross-cutting concerns that affect not just the partition layout and choice of a bootloader, but also kernel build options, and maybe options for some userland packages (are there examples of that, though?)? Maybe the best course of action is to add all this info to <image> until we have a better idea, after all. I guess I’m contributing more questions that answers. :-) Thanks, Ludo’.
Hi! zimoun <zimon.toutoune@gmail.com> skribis: > On Sat, 12 Dec 2020 at 09:30, Mathieu Othacehe <othacehe@gnu.org> wrote: > >> When using Yocto, Buildroot or even OpenWrt, you say "build me an image >> for that board/machine" and not, "build me an image for that board by >> cross-compiling to this mysterious triplet". > > I confirm that the triplet is still mysterious to me. Since I do not do > that often, each time I am trying, I need to browse the doc, when I am > not asking again and again on IRC. The triplet is a stringy DSL to designate a CPU architecture, hardware vendor, and operating system; nowadays people often piggy-back information to distinguish “OS” from “kernel”, to specify the ABI, etc. (info "(autoconf) Specifying Target Triplets"). It was designed at a time where things were quite different (nowadays the “vendor” part is almost always useless), and it’s primarily for userland software. It’s well-documented though, no mystery. ;-) Now, a triplet does not capture all the things we’re interested in, like details of the boot-up sequence of the ARM board, preferred bootloader, Binutils tweaks, etc. From a Guix System viewpoint, we could define an abstract architecture/platform/target as something that embodies the info contained in triplets and more, say: --8<---------------cut here---------------start------------->8--- ;; Description of a platform supported by the GNU system. (define-record-type* <platform> platform make-platform platform? (triplet platform-triplet) ;"x86_64-linux-gnu" (system-type platform-system-type) ;"x86_64-linux" (linux-architecture platform-linux-architecture) ;"amd64" (kernel platform-kernel) ;<package> (ld.so platform-ld.so) ;"ld-linux-x86-64.so.2" (gcc platform-gcc) ;<package> (binutils platform-binutils) ;<package> (libc platform-transform-libc)) ;<package> --8<---------------cut here---------------end--------------->8--- Currently that info is scattered in various pieces in Guix: in base.scm, cross-base.scm, linux.scm, bootstrap.scm, etc. Having all that in a single place would be an improvement. Of course this is going beyond what was originally discussed in this thread and I’m not claiming this is the solution to work on right now. It might be a general direction to follow longer-term, though. Thoughts? Ludo’.
Hi, I want to note that there's a difference between cross-compiling things and natively compiling things (even if only qemu transparent emulation). The resulting images between cross vs non-cross could be (and probably are) different. So I think there should be a command line parameter to the image script that specifies whether to cross-compile or not--there's no way around it that I can see. Also, this image generation thing is mostly for bootstrapping Guix, so it is fine if that only supports configurations we actually tested. Ludo said: > > However, I > > feel like <image> is the wrong place for ‘system’ and ‘target’: the > > image format, conceptually, has nothing to do with whether we’re > > cross-compiling, compiling for a specific system, etc. It depends on what you mean. How the word "image format" is colloquially used in the VM world, it very much has to with what guest system (and even which emulator) this image is for, and that's not at all variable. But I agree that there are other things that could be variable per image target system, like the kernel version that actually works, the u-boot that actually works, the partition layout that actually works, the initrd modules, weird system packages and/or activation scripts that are required for booting etc. See buildroot. Mathieu said: > On the one hand, I agree that adding "system" and "target" to <image>, > so that they can override the corresponding arguments doesn't feel > nice. On the other hand, I think that dealing with system/target is too > low level for most users. I agree. Also, I want to stress that if we do this kind of image generation, it has to be for Guix images we actually tested. So the general case with specifying a random system and target we never saw before cannot be supported anyway (and will likely not work), especially since bootloaders are anything but portable in general. > When using Yocto, Buildroot or even OpenWrt, you say "build me an image > for that board/machine" and not, "build me an image for that board by > cross-compiling to this mysterious triplet". I agree that we should not ask the user to specify a triplet to build a guix system image. It's obvious what the triplet is per image, since we tested it anyway (riiiight?). > If the user selects the image type "pine64" or "novena", it's obvious > that the image has to be built for ARM, so I think it makes sense to > hardcode it somewhere. I agree.
Hi, zimoun <zimon.toutoune@gmail.com> skribis: >> --8<---------------cut here---------------start------------->8--- >> ;; Description of a platform supported by the GNU system. >> (define-record-type* <platform> platform make-platform >> platform? >> (triplet platform-triplet) ;"x86_64-linux-gnu" >> (system-type platform-system-type) ;"x86_64-linux" >> (linux-architecture platform-linux-architecture) ;"amd64" >> (kernel platform-kernel) ;<package> >> (ld.so platform-ld.so) ;"ld-linux-x86-64.so.2" >> (gcc platform-gcc) ;<package> >> (binutils platform-binutils) ;<package> >> (libc platform-transform-libc)) ;<package> >> --8<---------------cut here---------------end--------------->8--- > > Naively and what confuse me is the redundancy of the information. For > example, is it possible to do something else than “gnu”? Or when one > thing is fixed, other parameters are also fixed, for instance does it > make sense > > "x86_64-linux-gnu" > "i686-hurd" > "arm" > "ld-hurd-arm.so.2" > > ? For a given platform, say “GNU/Hurd on i586”, all the parameters are fixed, with some degrees of liberty on the toolchain, though. However, currently that information is scattered across different places, so the goal here would be to gather it all in one place, which should facilitate porting to new platforms. Ludo’.
Hello, Mathieu Othacehe <othacehe@gnu.org> writes: > Hey, > >> OTOH, I might want to cross-build a Novena image from x86_64, or I might >> want to build it natively. Perhaps what could be said is that a Novena >> image can either be built natively on armhf-linux, or cross-built to >> arm-linux-gnueabihf. Perhaps we should encode this constraint rather >> than a specific ‘system’ or ‘target’? (I’m thinking out loud…) > > Maybe the next step would be to define a <machine> record that encodes > the "system" and "target" constraints for a specific board/machine. The > kernel build options and userland packages options you are mentioning > above could also be part of this record. > > As Danny is proposing, we could also have a "--native" argument to "guix > system" that would force native build instead of cross-compiling. > >> Regarding ARM boards, do you think some additional abstraction is needed >> to encode cross-cutting concerns that affect not just the partition >> layout and choice of a bootloader, but also kernel build options, and >> maybe options for some userland packages (are there examples of that, >> though?)? >> >> Maybe the best course of action is to add all this info to <image> until >> we have a better idea, after all. > > Sure, I agree. > >> I guess I’m contributing more questions that answers. :-) > > It helps a lot anyway :) I see this hasn't landed to the repo yet. Are you still refining it, or has it fallen into cracks? Just checking :-). Thank you, Maxim