[bug#64448,0/2] Fix hdf5-1.14 wrappers and remove generated source files from all hdf5 versions.
Message ID | cover.1688421399.git.david.elsing@posteo.net |
---|---|
Headers |
Return-Path: <guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org> X-Original-To: patchwork@mira.cbaines.net Delivered-To: patchwork@mira.cbaines.net Received: by mira.cbaines.net (Postfix, from userid 113) id 19E7927BBEA; Mon, 3 Jul 2023 22:59:30 +0100 (BST) X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on mira.cbaines.net X-Spam-Level: X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIM_INVALID, DKIM_SIGNED,MAILING_LIST_MULTI,SPF_HELO_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) by mira.cbaines.net (Postfix) with ESMTPS id 5D02327BBE2 for <patchwork@mira.cbaines.net>; Mon, 3 Jul 2023 22:59:26 +0100 (BST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from <guix-patches-bounces@gnu.org>) id 1qGRZb-0002Cm-HE; Mon, 03 Jul 2023 17:59:07 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1qGRZZ-0002Cd-Rb for guix-patches@gnu.org; Mon, 03 Jul 2023 17:59:05 -0400 Received: from debbugs.gnu.org ([209.51.188.43]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1qGRZZ-0003V0-Iy for guix-patches@gnu.org; Mon, 03 Jul 2023 17:59:05 -0400 Received: from Debian-debbugs by debbugs.gnu.org with local (Exim 4.84_2) (envelope-from <Debian-debbugs@debbugs.gnu.org>) id 1qGRZW-00075r-DJ; Mon, 03 Jul 2023 17:59:02 -0400 X-Loop: help-debbugs@gnu.org Subject: [bug#64448] [PATCH 0/2] Fix hdf5-1.14 wrappers and remove generated source files from all hdf5 versions. Resent-From: David Elsing <david.elsing@posteo.net> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces@debbugs.gnu.org> Resent-CC: andreas@enge.fr, efraim@flashner.co.il, bavier@posteo.net, guix-patches@gnu.org Resent-Date: Mon, 03 Jul 2023 21:59:02 +0000 Resent-Message-ID: <handler.64448.B.168842154127258@debbugs.gnu.org> Resent-Sender: help-debbugs@gnu.org X-GNU-PR-Message: report 64448 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 64448@debbugs.gnu.org Cc: David Elsing <david.elsing@posteo.net>, andreas@enge.fr, efraim@flashner.co.il, bavier@posteo.net X-Debbugs-Original-To: guix-patches@gnu.org X-Debbugs-Original-Xcc: andreas@enge.fr, efraim@flashner.co.il, bavier@posteo.net Received: via spool by submit@debbugs.gnu.org id=B.168842154127258 (code B ref -1); Mon, 03 Jul 2023 21:59:02 +0000 Received: (at submit) by debbugs.gnu.org; 3 Jul 2023 21:59:01 +0000 Received: from localhost ([127.0.0.1]:34581 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces@debbugs.gnu.org>) id 1qGRZU-00075a-Mq for submit@debbugs.gnu.org; Mon, 03 Jul 2023 17:59:01 -0400 Received: from lists.gnu.org ([209.51.188.17]:46676) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <david.elsing@posteo.net>) id 1qGRZQ-00075O-BN for submit@debbugs.gnu.org; Mon, 03 Jul 2023 17:58:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <david.elsing@posteo.net>) id 1qGRZQ-0002Ab-1Z for guix-patches@gnu.org; Mon, 03 Jul 2023 17:58:56 -0400 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <david.elsing@posteo.net>) id 1qGRZN-0003Qv-QO for guix-patches@gnu.org; Mon, 03 Jul 2023 17:58:55 -0400 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 0CCE5240101 for <guix-patches@gnu.org>; Mon, 3 Jul 2023 23:58:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1688421530; bh=XRasyfLG5emYUI48C2zO9uiZ41iRvjuh5loJJFBeMXI=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version: Content-Transfer-Encoding:From; b=CqnhD/1WRCR179v2FteuyvJ8TbjKQT/rUb0XCIpWbNlw7o3jCc0vls6LXD+wFB8jW g9/JOG0cu9vNCg/gTj3j+p8DDljAmC4FCEka11nejHemXwZAS0PkRiUv5iLshV0RTv Qs4Es3em7azr+vkL7cLiKJRilsIUC/0+856kcyiXU0c8jqayIC/HO4sGz1PjR/ghHf 1LiORvs0lEIqEr+uZCXP9xez49EFzDNXgFblfTtr6JxtsmMsjhsBBaWtETLlkYGFZk +8mge7Li41lDaVIzY2AC7EmiKsDpmx9mGdknjRSb+gFUrGe/NYyCB3SiOQDP2jemNa lYy8g7Uz0uj5w== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Qw0Dn3zdKz6tvt; Mon, 3 Jul 2023 23:58:49 +0200 (CEST) From: David Elsing <david.elsing@posteo.net> Date: Mon, 3 Jul 2023 21:58:45 +0000 Message-Id: <cover.1688421399.git.david.elsing@posteo.net> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=185.67.36.66; envelope-from=david.elsing@posteo.net; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H5=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list X-BeenThere: guix-patches@gnu.org List-Id: <guix-patches.gnu.org> List-Unsubscribe: <https://lists.gnu.org/mailman/options/guix-patches>, <mailto:guix-patches-request@gnu.org?subject=unsubscribe> List-Archive: <https://lists.gnu.org/archive/html/guix-patches> List-Post: <mailto:guix-patches@gnu.org> List-Help: <mailto:guix-patches-request@gnu.org?subject=help> List-Subscribe: <https://lists.gnu.org/mailman/listinfo/guix-patches>, <mailto:guix-patches-request@gnu.org?subject=subscribe> Errors-To: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org Sender: guix-patches-bounces+patchwork=mira.cbaines.net@gnu.org X-getmail-retrieved-from-mailbox: Patches |
Series |
Fix hdf5-1.14 wrappers and remove generated source files from all hdf5 versions.
|
|
Message
David Elsing
July 3, 2023, 9:58 p.m. UTC
When using hdf5 with CMake, I noticed that it does not work for the 1.14 version, because the h5cc and h5c++ wrappers contain the flag "-I/tmp/guix-build-hdf5-1.14.0.drv-0/hdf5-1.14.0/src/H5FDsubfiling". It is required in the 'configure phase, but can be removed afterwards. Additionally, I noticed that the hdf5 source tarballs contain many autogenerated files, so I also removed them. Somehow, the 1.8.23 release is missing two m4 scripts (which are included in later releases, but different from this one), so they are copied from the 1.8.21 tarball (the latest earlier release which includes them). The genparser script is missing as well, which is copied from the 1.10.9 release. David Elsing (2): gnu: hdf5-1.8: Remove generated files from source gnu: hdf5-1.14: Remove spurious include directories. gnu/packages/maths.scm | 234 +++++++++++++++----- gnu/packages/patches/hdf5-config-date.patch | 14 +- 2 files changed, 188 insertions(+), 60 deletions(-) base-commit: 76e39909e886bb51cf9b2fa03a1523545faca880
Comments
I saw that the Git repository is also available, with the releases as tags: https://github.com/HDFGroup/hdf5. Would it be preferred to use them instead and make the snippet shorter? The missing scripts are also included for the 1.8.23 release.
Hello, adding Gerd Heber in cc, who seems to be the last person having worked on these packages. I am not competent at all about them, so commenting more as an outsider. Using "guix package -A hdf": hdf4 4.2.14 out gnu/packages/maths.scm:1289:2 hdf4-alt 4.2.14 out gnu/packages/maths.scm:1372:2 hdf5 1.8.23 out,fortran gnu/packages/maths.scm:1382:2 hdf5 1.14.0 out,fortran gnu/packages/maths.scm:1543:2 hdf5 1.12.2 out,fortran gnu/packages/maths.scm:1523:2 hdf5 1.10.9 out,fortran gnu/packages/maths.scm:1503:2 hdf5-blosc 1.0.0 out gnu/packages/maths.scm:1807:2 hdf5-parallel-openmpi 1.10.9 out,fortran gnu/packages/maths.scm:1775:2 and a lot of packages in other languages, for instance: cl-hdf5-cffi 1.8.18-1.5b5c88f out gnu/packages/lisp-xyz.scm:12238:4 ecl-hdf5-cffi 1.8.18-1.5b5c88f out gnu/packages/lisp-xyz.scm:12238:4 (apparently based on 1.8.18? but it has hdf5-1.10 as input). So there are lots of versions, and in the middle of them, (define-public hdf5 ;; Default version of HDF5. hdf5-1.10) And hdf5-parallel-openmpi 1.10.9 out,fortran gnu/packages/maths.scm:1775:2 which inherits like this: (package/inherit hdf5-1.10 ;use the latest but takes inputs like this: (inputs `(("mpi" ,openmpi) ,@(package-inputs hdf5))) which, I suppose, works by chance since currently hdf5-1.10 and hdf5 are the same (but not "the latest"). Are all of these needed? If only the latest version 1.14.0 could be kept, this would make the patch for 1.8.23 obsolete, for instance. Andreas
Am Mon, Jul 24, 2023 at 03:49:49PM +0200 schrieb Andreas Enge:
> Are all of these needed?
Partial answer at
https://portal.hdfgroup.org/display/support/HDF5+1.8.23 :
"Future of HDF5-1.8
Please be aware that this will be the last release for HDF5 1.8.
We encourage users to move to HDF5 1.10 or HDF5 1.14. Please refer to
the release schedule for future releases of these versions."
The schedule at
https://github.com/HDFGroup/hdf5#release-schedule
shows that 1.10 and 1.12 should see their last release this autumn,
so probably we should really move to 1.14.
$ guix refresh -l hdf5@1.8
Building the following 9 packages would ensure 16 dependent packages are rebuilt: pigx-sars-cov-2@0.0.9 pigx@0.0.3 nip2@8.7.1 python-pyvips@2.2.1 gnss-sdr@0.0.17 hdf-eos5@1.15 scilab@5.5.0 hdf-java@3.3.2 h5check@2.0.1
$ guix refresh -l hdf5@1.10
Building the following 210 packages would ensure 411 dependent packages are rebuilt:
...
$ guix refresh -l hdf5@1.12
No dependents other than itself: hdf5@1.12.2
So this could probably be dropped immediately.
$ guix refresh -l hdf5@1.14
Building the following 2 packages would ensure 2 dependent packages are rebuilt: ecl-hdf5-cffi@1.8.18-1.5b5c88f cl-hdf5-cffi@1.8.18-1.5b5c88f
I suppose that the many dependencies on 1.10 come from the definition of
the variable hd5 as hdf5-10; it is quite possible we could simply upgrade
this to hdf5-14. Someone™ should give it a try...
Andreas
Only HDF5 1.14.x (currently, 1.14.2) should be kept. However, hdf5 and hdf5-parallel should be kept separate. Aside from the MPI dependence, the parallel build is NOT a superset of the sequential version. There could be even two versions of hdf5: a non-threadsafe build (hdf5) and a threadsafe build (hdf5-ts? hdf5-mt?). For HDF4, 4.2.16-2 is the latest HDF4 release. We are maintaining that indefinitely (for NASA). Best, G. On Mon, Jul 24, 2023 at 8:49 AM Andreas Enge <andreas@enge.fr> wrote: > Hello, > > adding Gerd Heber in cc, who seems to be the last person having worked on > these packages. > > I am not competent at all about them, so commenting more as an outsider. > Using "guix package -A hdf": > hdf4 4.2.14 out > gnu/packages/maths.scm:1289:2 > hdf4-alt 4.2.14 out > gnu/packages/maths.scm:1372:2 > hdf5 1.8.23 out,fortran > gnu/packages/maths.scm:1382:2 > hdf5 1.14.0 out,fortran > gnu/packages/maths.scm:1543:2 > hdf5 1.12.2 out,fortran > gnu/packages/maths.scm:1523:2 > hdf5 1.10.9 out,fortran > gnu/packages/maths.scm:1503:2 > hdf5-blosc 1.0.0 out > gnu/packages/maths.scm:1807:2 > hdf5-parallel-openmpi 1.10.9 out,fortran > gnu/packages/maths.scm:1775:2 > > and a lot of packages in other languages, for instance: > cl-hdf5-cffi 1.8.18-1.5b5c88f out > gnu/packages/lisp-xyz.scm:12238:4 > ecl-hdf5-cffi 1.8.18-1.5b5c88f out > gnu/packages/lisp-xyz.scm:12238:4 > (apparently based on 1.8.18? but it has hdf5-1.10 as input). > > So there are lots of versions, and in the middle of them, > (define-public hdf5 > ;; Default version of HDF5. > hdf5-1.10) > > And > hdf5-parallel-openmpi 1.10.9 out,fortran > gnu/packages/maths.scm:1775:2 > which inherits like this: > (package/inherit hdf5-1.10 ;use the latest > but takes inputs like this: > (inputs > `(("mpi" ,openmpi) > ,@(package-inputs hdf5))) > which, I suppose, works by chance since currently hdf5-1.10 and hdf5 are > the same (but not "the latest"). > > Are all of these needed? If only the latest version 1.14.0 could be kept, > this would make the patch for 1.8.23 obsolete, for instance. > > Andreas > >
Am Sun, Aug 20, 2023 at 07:36:50PM -0500 schrieb Gerd Heber: > Only HDF5 1.14.x (currently, 1.14.2) should be kept. However, hdf5 and > hdf5-parallel should be kept separate. Aside from the MPI dependence, > the parallel build is NOT a superset of the sequential version. There could be > even two versions of hdf5: a non-threadsafe build (hdf5) and a threadsafe build > (hdf5-ts? hdf5-mt?). > > For HDF4, 4.2.16-2 is the latest HDF4 release. We are maintaining that > indefinitely (for NASA). Thanks for the info, which helps us set a target! I am trying to update hdf4, see https://issues.guix.gnu.org/65443 Andreas
And I have removed hdf5@1.12, which had no dependent packages. Andreas
Hello, > $ guix refresh -l hdf5@1.8 > Building the following 9 packages would ensure 16 dependent packages are rebuilt: pigx-sars-cov-2@0.0.9 pigx@0.0.3 nip2@8.7.1 python-pyvips@2.2.1 gnss-sdr@0.0.17 hdf-eos5@1.15 scilab@5.5.0 hdf-java@3.3.2 h5check@2.0.1 Currently, 5 packages depend directly on hdf5@1.8 (from guix graph -t reverse-package): -scilab, for which the latest version can be built with hdf5@1.14: https://issues.guix.gnu.org/66201 -hdf5-eos, for which the latest version can also be built with hdf5@1.14: https://issues.guix.gnu.org/66204 -java-cisd-jhdf5, where newer versions seem to be built only with Gradle: https://sissource.ethz.ch/sispub/jhdf5. It has fastqc and pigx(-*) as dependencies. -hdf-java, where the latest version is from 2017. Maybe it can be built with a newer HDF5 version? No other packages depend on it. -h5check, where the latest version is from 2014. When I tried to compile it with hdf5@1.14, some tests failed because the file format is not accepted anymore. No other packages depend on it. Cheers, David
Hello, I have submitted two new issues with the goal of getting closer to removing hdf5@1.8 and hdf5@1.10, see https://issues.guix.gnu.org/77285 https://issues.guix.gnu.org/77287 Ricardo, since you seem to be interested in this package, your input there would be very welcome. And here also; after the proposed removal of h5check (which, I just see, does not even build any more), the only remaining direct dependency of hdf5@1.8 is java-cisd-jhdf5 and "guix refresh -l hdf5@1.8" lists pigx-sars-cov-2@0.0.9 trinityrnaseq@2.13.2 pigx@0.0.3 as leaf packages. Andreas
Hi Andreas, > I have submitted two new issues with the goal of getting closer > to > removing hdf5@1.8 and hdf5@1.10, see > https://issues.guix.gnu.org/77285 > https://issues.guix.gnu.org/77287 > > Ricardo, since you seem to be interested in this package, your > input > there would be very welcome. And here also; after the proposed > removal > of h5check (which, I just see, does not even build any more), > the only > remaining direct dependency of hdf5@1.8 is > java-cisd-jhdf5 > and "guix refresh -l hdf5@1.8" lists > pigx-sars-cov-2@0.0.9 trinityrnaseq@2.13.2 pigx@0.0.3 > as leaf packages. On the python-team branch the only user of java-cisd-jhdf5 is fastqc, which should be updated to a more recent version.
Hello Ricardo, Am Wed, Mar 26, 2025 at 06:29:39PM +0100 schrieb Ricardo Wurmus: > On the python-team branch the only user of java-cisd-jhdf5 is fastqc, which > should be updated to a more recent version. looking at its git repository, fastqc still requires java-cisd-jhdf5 (there is an eight year old jar file), so it does not look as if updating fastqc will help us get rid of java-cisd-jhdf5 and then hdf5@1.8. Andreas
Hello David, with a lot of work on hdf5 unrelated to your original issue, this bug report has become a bit unwieldy (sorry for that, but I think we made good progress around this package complex!). So I wonder whether you would be okay closing this issue and opening new ones with updated patches? Useful remaining work would be updating hdf5@1.14 to its latest version; working towards getting rid of hdf5@1.8 and hdf5@1.10; and your original issue about generated files, which maybe could be combined with an update to hdf5@1.14 (I think there is no point in spending much work on the outdated versions that we would prefer to remove). For testing updates, it might make sense to wait until the non-building dependencies have been removed in about a month. What do you think? Andreas
Hi Andreas, Andreas Enge <andreas@enge.fr> writes: > So I wonder whether you would be okay closing this issue and opening > new ones with updated patches? Sure, I'm closing it now. > Useful remaining work would be updating hdf5@1.14 to its latest version; > working towards getting rid of hdf5@1.8 and hdf5@1.10; > and your original issue about generated files, which maybe could be > combined with an update to hdf5@1.14 (I think there is no point in > spending much work on the outdated versions that we would prefer to > remove). Yes, that's a good suggestion, I made a new patch series which only changes hdf5@1.14 and hdf-java: https://issues.guix.gnu.org/77590 Thanks, David