Message ID | 9399f4d210641ee73f9a620f56d20bd66aef65e8.1725895378.git.arunisaac@systemreboot.net |
---|---|
State | New |
Headers | show |
Series | [bug#72766,v2,1/2] gnu: Add libjwt. | expand |
Hello Arun! Arun Isaac <arunisaac@systemreboot.net> skribis: > * gnu/packages/parallel.scm (slurm)[inputs]: Add dbus, http-parser, libjwt, > librdkafka, libtaml and (mariadb "dev"). > [arguments]: Add --enable-slurmrestd, --with-bpf, --with-http-parser, > --with-rdkafka and --with-yaml to configure flags. > > Change-Id: Icddaa8f883754129d975e82fccd732a47a467517 [...] > + (inputs (list dbus > + freeipmi > + http-parser > `(,hwloc-2 "lib") > json-c > + libjwt > + librdkafka > + libyaml > linux-pam > + `(,mariadb "dev") Could you check how this impacts ‘guix size slurm’? The reason I’m asking is that we at work often end up with slurm in packs that we carry to supercomputers, so it would be best if it wasn’t too big: --8<---------------cut here---------------start------------->8--- $ guix size slurm |tail -1 total: 359.8 MiB $ guix size slurm mariadb |tail -1 total: 663.4 MiB --8<---------------cut here---------------end--------------->8--- I wonder if we should have a ‘slurm-minimal’ variant or something? (That could probably come as a patch after this series.) Apart from that, both patches look good to me! Thanks, Ludo’.
Hi Ludo, > Could you check how this impacts ‘guix size slurm’? The size goes up from 141 MiB to 407 MiB. > The reason I’m asking is that we at work often end up with slurm in > packs that we carry to supercomputers, so it would be best if it wasn’t > too big: > > --8<---------------cut here---------------start------------->8--- > $ guix size slurm |tail -1 > total: 359.8 MiB > $ guix size slurm mariadb |tail -1 > total: 663.4 MiB > --8<---------------cut here---------------end--------------->8--- > > I wonder if we should have a ‘slurm-minimal’ variant or something? > > (That could probably come as a patch after this series.) Yes, I could add a slurm-minimal. Would you like to have a slurm-minimal for every version of slurm or just the latest? Regards, Arun
Hello! Arun Isaac <arunisaac@systemreboot.net> skribis: >> Could you check how this impacts ‘guix size slurm’? > > The size goes up from 141 MiB to 407 MiB. D’oh! >> I wonder if we should have a ‘slurm-minimal’ variant or something? >> >> (That could probably come as a patch after this series.) > > Yes, I could add a slurm-minimal. Would you like to have a slurm-minimal > for every version of slurm or just the latest? Hmm! The reason we keep different versions of slurm is because one always needs a slurm client compatible with the version of slurmd running on the target cluster. And so yes, we’d need ‘slurm-minimal’ for each version. But that doesn’t sound great so I’m not sure. Another possibility would be to do the opposite: keep ‘slurm’ without MariaDB & co., and have a ‘slurm-rest’ (?) variant with all the bells and whistles? Thanks, Ludo’.
Hi Ludo, > Another possibility would be to do the opposite: keep ‘slurm’ without > MariaDB & co., and have a ‘slurm-rest’ (?) variant with all the bells > and whistles? The problem with that approach is that I'm sure there will be more optional features to add to slurm. Then, we would have a combinatorial explosion of packages---slurm-rest, slurm-rest+feature1, slurm-rest+feature2, slurm-feature1+feature2, etc. I am ok with having a slurm-minimal for all versions. That's the lesser evil. Regards, Arun
Hi, Arun Isaac <arunisaac@systemreboot.net> skribis: >> Another possibility would be to do the opposite: keep ‘slurm’ without >> MariaDB & co., and have a ‘slurm-rest’ (?) variant with all the bells >> and whistles? > > The problem with that approach is that I'm sure there will be more > optional features to add to slurm. Then, we would have a combinatorial > explosion of packages---slurm-rest, slurm-rest+feature1, > slurm-rest+feature2, slurm-feature1+feature2, etc. Not necessarily: that second variant could be maximal, where additional features would be added. > I am ok with having a slurm-minimal for all versions. That's the > lesser evil. OK. There might be a third option: moving server-side code to a separate output. That way, perhaps client-side code (things like ‘openmpi’) would not have MariaDB & co. in their closure? Ludo’.
>> I am ok with having a slurm-minimal for all versions. That's the >> lesser evil. > > OK. I'll send a v3 patchset with the minimal variants soon.. > There might be a third option: moving server-side code to a separate > output. That way, perhaps client-side code (things like ‘openmpi’) > would not have MariaDB & co. in their closure? I presume you might need to carry both the server-side and the client-side code in a guix pack. So, how would this option help?
Morning! Arun Isaac <arunisaac@systemreboot.net> skribis: >> There might be a third option: moving server-side code to a separate >> output. That way, perhaps client-side code (things like ‘openmpi’) >> would not have MariaDB & co. in their closure? > > I presume you might need to carry both the server-side and the > client-side code in a guix pack. So, how would this option help? No, to run code on a supercomputer, you only need MPI and things like ‘salloc’, ‘sbatch’, etc. There’s definitely no need for ‘slurmd’ or anything like that in that case because it’s already running on the machine. Ludo’.
>> I presume you might need to carry both the server-side and the >> client-side code in a guix pack. So, how would this option help? > > No, to run code on a supercomputer, you only need MPI and things like > ‘salloc’, ‘sbatch’, etc. There’s definitely no need for ‘slurmd’ or > anything like that in that case because it’s already running on the > machine. Ah, ok, the server-side is maintained by a different admin team. I was thinking you were maintaining the server-side as well. That makes sense, thank you! In that case, I'll split slurm into two outputs---a server and a client-side. I'll send a patch in the coming days.
diff --git a/gnu/packages/parallel.scm b/gnu/packages/parallel.scm index d5a55a8641..5d7544783a 100644 --- a/gnu/packages/parallel.scm +++ b/gnu/packages/parallel.scm @@ -15,6 +15,7 @@ ;;; Copyright © 2024 Zheng Junjie <873216071@qq.com> ;;; Copyright © 2024 David Elsing <david.elsing@posteo.net> ;;; Copyright © 2024 Romain Garbage <romain.garbage@inria.fr> +;;; Copyright © 2024 Arun Isaac <arunisaac@systemreboot.net> ;;; ;;; This file is part of GNU Guix. ;;; @@ -49,13 +50,16 @@ (define-module (gnu packages parallel) #:use-module (gnu packages base) #:use-module (gnu packages bash) #:use-module (gnu packages check) + #:use-module (gnu packages databases) #:use-module (gnu packages documentation) #:use-module (gnu packages flex) #:use-module (gnu packages freeipmi) + #:use-module (gnu packages glib) #:use-module (gnu packages libevent) #:use-module (gnu packages linux) #:use-module (gnu packages maths) #:use-module (gnu packages mpi) + #:use-module (gnu packages networking) #:use-module (gnu packages perl) #:use-module (gnu packages pkg-config) #:use-module (gnu packages python) @@ -63,6 +67,7 @@ (define-module (gnu packages parallel) #:use-module (gnu packages python-science) #:use-module (gnu packages python-xyz) #:use-module (gnu packages readline) + #:use-module (gnu packages serialization) #:use-module (gnu packages tcl) #:use-module (gnu packages tls) #:use-module (gnu packages web)) @@ -224,12 +229,17 @@ (define-public slurm (mkdir "contribs") (rename-file "tmp-pmi" "contribs/pmi") (rename-file "tmp-pmi2" "contribs/pmi2"))))) - ;; FIXME: More optional inputs could be added, - ;; in particular mysql and gtk+. - (inputs (list freeipmi + ;; FIXME: More optional inputs (for example, gtk+) could be added. + (inputs (list dbus + freeipmi + http-parser `(,hwloc-2 "lib") json-c + libjwt + librdkafka + libyaml linux-pam + `(,mariadb "dev") openpmix munge numactl @@ -239,13 +249,22 @@ (define-public slurm (build-system gnu-build-system) (arguments (list #:configure-flags - #~(list "--enable-pam" "--sysconfdir=/etc/slurm" + #~(list "--enable-pam" + "--enable-slurmrestd" + "--sysconfdir=/etc/slurm" "--disable-static" + (string-append "--with-bpf=" + (dirname (dirname (search-input-directory + %build-inputs "include/linux")))) (string-append "--with-freeipmi=" #$(this-package-input "freeipmi")) + (string-append "--with-http-parser=" + #$(this-package-input "http-parser")) (string-append "--with-hwloc=" (ungexp (this-package-input "hwloc") "lib")) (string-append "--with-json=" #$(this-package-input "json-c")) (string-append "--with-munge=" #$(this-package-input "munge")) + (string-append "--with-rdkafka=" #$(this-package-input "librdkafka")) + (string-append "--with-yaml=" #$(this-package-input "libyaml")) ;; Use PMIx bundled with Open MPI (this is required for Open MPI 5.x). ;; Note: Older versions that inherit from this package lack the