[bug#50813] gnu: python-pandas: Update to 1.3.3.

* gnu/packages/python-science.scm (python-pandas): Update to 1.3.3.
---
 gnu/packages/python-science.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

Hi Felix,

Felix Gruber <felgru@posteo.net> skribis:

> * gnu/packages/python-science.scm (python-pandas): Update to 1.3.3.

The upgrade appears to cause build failures for python-dask and
pigx-rnaseq (via snakemake) among other things.

Could you take a look?

For the record, I tried to build everything reported by ‘guix refresh -l
python-pandas’ though some of them (ocaml4.07-re, etc.) would already
fail to build before the upgrade.

Thanks in advance!

Ludo’.

Hi Ludo’,

On 10/7/21 11:07 AM, Ludovic Courtès wrote:
> 
> Felix Gruber <felgru@posteo.net> skribis:
> 
>> * gnu/packages/python-science.scm (python-pandas): Update to 1.3.3.
> 
> The upgrade appears to cause build failures for python-dask and
> pigx-rnaseq (via snakemake) among other things.
> 
> Could you take a look?

I could not reproduce the build failure in pigx-rnaseq. For python-dask
the build failure could be fixed by upgrading it to 2021.9.1, whose
changelog mentioned that they've deactivated a failing test when
building against Pandas >= 1.3.3. After updating dask, its dependent
packages python-distributed and python-eliot failed to build.  I've
fixed both by updating them to their latest releases and disabling
failing tests.
I've also took the oppurtunity to update the synapse package which was
the only package depending on python-eliot.

> For the record, I tried to build everything reported by ‘guix refresh -l
> python-pandas’ though some of them (ocaml4.07-re, etc.) would already
> fail to build before the upgrade.

Of the remaining packages reported by `guix refresh -l python-pandas`,
I've identified the following packages with build failures:

* clipper@2.0
  sometimes fails due to nondeterministic test
  `test_get_FDR_cutoff_mean`.  The docstring of this test already says
  "Difficult to test because of random sampling" followed by a TODO.
  I've thus deactivated this test, to make it build deterministically.
* pplacer@1.1.alpha19
  already failed on master due to unbuildable ocaml4.07-re@1.10.3.
* python-scikit-rebate@0.6
  Already failed on master in test `skrebate`:
  ModuleNotFoundError: No module named 'sklearn.externals.joblib'
  (A guix refresh -u python-scikit-rebate would update it to 0.62 which
   was buildable, but this version seemed to come without any tests, so
   I'm not sure if this is the right solution.)
* orange@3.28.0
  Already failed to build on mater, because its dependency
  python-orange-canvas-core.
  I've looked into updating both to their latest version, but that
  would require updating python-pygments to >= 2.8.0 which would entail
  rebuilding 561 dependent packages.
* imp@2.13.0
  I was running out of memory while building imp, which in the end
  killed the build process. It seems that my 8GB of RAM are not
  sufficient to build imp, so I cannot tell if the update of
  python-pandas might break imp.
* tadbit@1.0.1
  depends on imp, so I couldn't build it either.

I've prepared patches for those packages that I was able to fix.

Best,
Felix

Felix Gruber (6):
  gnu: python-pandas: Update to 1.3.3.
  gnu: python-dask: Update to 2021.9.1.
  gnu: python-distributed: Update to 2021.09.1.
  gnu: python-eliot: Update to 1.13.0.
  gnu: synapse: Update to 1.44.0.
  gnu: clipper: Disable non-deterministic test.

 gnu/packages/bioinformatics.scm |  9 ++++++++-
 gnu/packages/matrix.scm         |  5 +++--
 gnu/packages/python-science.scm | 15 +++++++++------
 gnu/packages/python-xyz.scm     | 19 +++++++++++++++----
 4 files changed, 35 insertions(+), 13 deletions(-)

#48703 was an attempt in May to update numpy, scipy, and pandas. If and
when this patchset is accepted first I can then update #48703 for the more
recent numpy and scipy releases.

On Sun, Oct 10, 2021 at 11:06 AM Felix Gruber <felgru@posteo.net> wrote:

> Hi Ludo’,
>
> On 10/7/21 11:07 AM, Ludovic Courtès wrote:
> >
> > Felix Gruber <felgru@posteo.net> skribis:
> >
> >> * gnu/packages/python-science.scm (python-pandas): Update to 1.3.3.
> >
> > The upgrade appears to cause build failures for python-dask and
> > pigx-rnaseq (via snakemake) among other things.
> >
> > Could you take a look?
>
> I could not reproduce the build failure in pigx-rnaseq. For python-dask
> the build failure could be fixed by upgrading it to 2021.9.1, whose
> changelog mentioned that they've deactivated a failing test when
> building against Pandas >= 1.3.3. After updating dask, its dependent
> packages python-distributed and python-eliot failed to build.  I've
> fixed both by updating them to their latest releases and disabling
> failing tests.
> I've also took the oppurtunity to update the synapse package which was
> the only package depending on python-eliot.
>
> > For the record, I tried to build everything reported by ‘guix refresh -l
> > python-pandas’ though some of them (ocaml4.07-re, etc.) would already
> > fail to build before the upgrade.
>
> Of the remaining packages reported by `guix refresh -l python-pandas`,
> I've identified the following packages with build failures:
>
> * clipper@2.0
>   sometimes fails due to nondeterministic test
>   `test_get_FDR_cutoff_mean`.  The docstring of this test already says
>   "Difficult to test because of random sampling" followed by a TODO.
>   I've thus deactivated this test, to make it build deterministically.
> * pplacer@1.1.alpha19
>   already failed on master due to unbuildable ocaml4.07-re@1.10.3.
> * python-scikit-rebate@0.6
>   Already failed on master in test `skrebate`:
>   ModuleNotFoundError: No module named 'sklearn.externals.joblib'
>   (A guix refresh -u python-scikit-rebate would update it to 0.62 which
>    was buildable, but this version seemed to come without any tests, so
>    I'm not sure if this is the right solution.)
> * orange@3.28.0
>   Already failed to build on mater, because its dependency
>   python-orange-canvas-core.
>   I've looked into updating both to their latest version, but that
>   would require updating python-pygments to >= 2.8.0 which would entail
>   rebuilding 561 dependent packages.
> * imp@2.13.0
>   I was running out of memory while building imp, which in the end
>   killed the build process. It seems that my 8GB of RAM are not
>   sufficient to build imp, so I cannot tell if the update of
>   python-pandas might break imp.
> * tadbit@1.0.1
>   depends on imp, so I couldn't build it either.
>
> I've prepared patches for those packages that I was able to fix.
>
> Best,
> Felix
>
> Felix Gruber (6):
>   gnu: python-pandas: Update to 1.3.3.
>   gnu: python-dask: Update to 2021.9.1.
>   gnu: python-distributed: Update to 2021.09.1.
>   gnu: python-eliot: Update to 1.13.0.
>   gnu: synapse: Update to 1.44.0.
>   gnu: clipper: Disable non-deterministic test.
>
>  gnu/packages/bioinformatics.scm |  9 ++++++++-
>  gnu/packages/matrix.scm         |  5 +++--
>  gnu/packages/python-science.scm | 15 +++++++++------
>  gnu/packages/python-xyz.scm     | 19 +++++++++++++++----
>  4 files changed, 35 insertions(+), 13 deletions(-)
>
> --
> 2.30.2
>
>
>
>
>

Hi Felix,

Felix Gruber <felgru@posteo.net> skribis:

> On 10/7/21 11:07 AM, Ludovic Courtès wrote:
>> 
>> Felix Gruber <felgru@posteo.net> skribis:
>> 
>>> * gnu/packages/python-science.scm (python-pandas): Update to 1.3.3.
>> 
>> The upgrade appears to cause build failures for python-dask and
>> pigx-rnaseq (via snakemake) among other things.
>> 
>> Could you take a look?
>
> I could not reproduce the build failure in pigx-rnaseq. For python-dask
> the build failure could be fixed by upgrading it to 2021.9.1, whose
> changelog mentioned that they've deactivated a failing test when
> building against Pandas >= 1.3.3. After updating dask, its dependent
> packages python-distributed and python-eliot failed to build.  I've
> fixed both by updating them to their latest releases and disabling
> failing tests.
> I've also took the oppurtunity to update the synapse package which was
> the only package depending on python-eliot.

Neat.

>> For the record, I tried to build everything reported by ‘guix refresh -l
>> python-pandas’ though some of them (ocaml4.07-re, etc.) would already
>> fail to build before the upgrade.
>
> Of the remaining packages reported by `guix refresh -l python-pandas`,
> I've identified the following packages with build failures:
>
> * clipper@2.0
>   sometimes fails due to nondeterministic test
>   `test_get_FDR_cutoff_mean`.  The docstring of this test already says
>   "Difficult to test because of random sampling" followed by a TODO.
>   I've thus deactivated this test, to make it build deterministically.
> * pplacer@1.1.alpha19
>   already failed on master due to unbuildable ocaml4.07-re@1.10.3.
> * python-scikit-rebate@0.6
>   Already failed on master in test `skrebate`:
>   ModuleNotFoundError: No module named 'sklearn.externals.joblib'
>   (A guix refresh -u python-scikit-rebate would update it to 0.62 which
>    was buildable, but this version seemed to come without any tests, so
>    I'm not sure if this is the right solution.)
> * orange@3.28.0
>   Already failed to build on mater, because its dependency
>   python-orange-canvas-core.
>   I've looked into updating both to their latest version, but that
>   would require updating python-pygments to >= 2.8.0 which would entail
>   rebuilding 561 dependent packages.
> * imp@2.13.0
>   I was running out of memory while building imp, which in the end
>   killed the build process. It seems that my 8GB of RAM are not
>   sufficient to build imp, so I cannot tell if the update of
>   python-pandas might break imp.
> * tadbit@1.0.1
>   depends on imp, so I couldn't build it either.
>
> I've prepared patches for those packages that I was able to fix.

Awesome.

>   gnu: python-pandas: Update to 1.3.3.
>   gnu: python-dask: Update to 2021.9.1.
>   gnu: python-distributed: Update to 2021.09.1.
>   gnu: python-eliot: Update to 1.13.0.
>   gnu: synapse: Update to 1.44.0.
>   gnu: clipper: Disable non-deterministic test.

python-dask and pigx-rnaseq build fine for me with this patch series,
along with everything in between.

Applied, thank you!

Ludo’.