[bug#36623] gnu: gdal: add python support

Arne Babenhauserheide <arne_bab@web.de> writes:

> Hello Ludo`,
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
>>> validating RUNPATH of 6 binaries in "/gnu/store/q9sb0wv41ig429f1m1xspg22xm8pwpwh-gdal-2.2.4/lib"...
>>> /gnu/store/q9sb0wv41ig429f1m1xspg22xm8pwpwh-gdal-2.2.4/lib/python3.7/site-packages/osgeo/_gdal.cpython-37m-x86_64-linux-gnu.so:
>>> error: depends on 'libgdal.so.20', which cannot be found in RUNPATH
>>> ("/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib"
>>> "/gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib"
>>> "/gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib"
>>> "/gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../..")
>>
>> I’m not familiar with Cython so I don’t know how this was handled
>> before.  However, to me, it indicates that the resulting binaries are
>> unlikely to work.
>>
>> Namely, Python would dlopen “_gdal.cython*.so”, and that would fail to
>> find ‘libgdal.so’.

>>
>> Thoughts?
>
> Yes: It does work. But I don’t know why.

This is because the _gdal.cython*.so don't have rpath to the "lib"
directory, where libgdal.so will going.

>
> Cython runs at compile-time to generate c-code that acts as interface
> for Python. Given the paths in here, this needs gdal already installed
> in the runpath where it seems to be missing during validation.
>
> How can we fix that?

I think cython (which compile .py files to .c files) is not here..  It
has python C libraries generated by swig.
>
> Best wishes,
> Arne

In the end the python bindings for gdal can be build seperated (which
handle the missing rpath to libgdal nicely), and I prefer this way:
I test it with "~/src/guix/pre-inst-env guix environment --ad-hoc python
python-gdal", and in it I can run "python3 swig/python/samples/rel.py"
etc.

Does it work for you, Arne?  If so, I'll push my patch, thank you!

宋文武 <iyzsong@member.fsf.org> writes:

> Arne Babenhauserheide <arne_bab@web.de> writes:
>
>> Hello Ludo`,
>>
>> Ludovic Courtès <ludo@gnu.org> writes:
>>
>>>> validating RUNPATH of 6 binaries in "/gnu/store/q9sb0wv41ig429f1m1xspg22xm8pwpwh-gdal-2.2.4/lib"...
>>>> /gnu/store/q9sb0wv41ig429f1m1xspg22xm8pwpwh-gdal-2.2.4/lib/python3.7/site-packages/osgeo/_gdal.cpython-37m-x86_64-linux-gnu.so:
>>>> error: depends on 'libgdal.so.20', which cannot be found in RUNPATH
>>>> ("/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib"
>>>> "/gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib"
>>>> "/gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib"
>>>> "/gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../..")
>>>
>>> I’m not familiar with Cython so I don’t know how this was handled
>>> before.  However, to me, it indicates that the resulting binaries are
>>> unlikely to work.
>>>
>>> Namely, Python would dlopen “_gdal.cython*.so”, and that would fail to
>>> find ‘libgdal.so’.
>
>>>
>>> Thoughts?
>>
>> Yes: It does work. But I don’t know why.
>
> This is because the _gdal.cython*.so don't have rpath to the "lib"
> directory, where libgdal.so will going.
>
>>
>> Cython runs at compile-time to generate c-code that acts as interface
>> for Python. Given the paths in here, this needs gdal already installed
>> in the runpath where it seems to be missing during validation.
>>
>> How can we fix that?
>
> I think cython (which compile .py files to .c files) is not here..  It
> has python C libraries generated by swig.
>>
>> Best wishes,
>> Arne
>
> In the end the python bindings for gdal can be build seperated (which
> handle the missing rpath to libgdal nicely), and I prefer this way:
>
> From 09254f646b948a5603b2b3dcb99a7f2f6ad0c8de Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>
> Date: Sat, 20 Jul 2019 22:39:33 +0800
> Subject: [PATCH] gnu: Add python-gdal.
>
> For <https://issues.guix.gnu.org/issue/36623>, thank to Arne Babenhauserheide
> for the original patch.
>
> * gnu/packages/geo.scm (python-gdal): New package.
> ---
>  gnu/packages/geo.scm | 19 +++++++++++++++++++
>  1 file changed, 19 insertions(+)
>
> diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
> index e4b6a262c7..8005c46129 100644
> --- a/gnu/packages/geo.scm
> +++ b/gnu/packages/geo.scm
> @@ -604,6 +604,25 @@ utilities for data translation and processing.")
>                 ;; frmts/mrf/libLERC
>                 license:asl2.0))))
>  
> +(define-public python-gdal
> +  (package (inherit gdal)
> +    (name "python-gdal")
> +    (build-system python-build-system)
> +    (arguments
> +     '(#:tests? #f                      ; no tests
> +       #:phases
> +       (modify-phases %standard-phases
> +         (add-before 'build 'chdir
> +           (lambda _
> +             (chdir "swig/python")
> +             #t)))))
> +    (native-inputs '())
> +    (propagated-inputs
> +     `(("python-numpy" ,python-numpy)))
> +    (inputs
> +     `(("gdal" ,gdal)))
> +    (synopsis "GDAL (Geospatial Data Abstraction Library) python bindings")))
> +
>  (define-public postgis
>    (package
>      (name "postgis")

Wow, that looks pretty good!

Thank you for fixing up my patch!

Best wishes,
Arne

Arne Babenhauserheide <arne_bab@web.de> writes:

> 宋文武 <iyzsong@member.fsf.org> writes:
>
>> Arne Babenhauserheide <arne_bab@web.de> writes:
>>
>>> Hello Ludo`,
>>>
>>> Ludovic Courtès <ludo@gnu.org> writes:
>>>
>>>>> validating RUNPATH of 6 binaries in "/gnu/store/q9sb0wv41ig429f1m1xspg22xm8pwpwh-gdal-2.2.4/lib"...
>>>>> /gnu/store/q9sb0wv41ig429f1m1xspg22xm8pwpwh-gdal-2.2.4/lib/python3.7/site-packages/osgeo/_gdal.cpython-37m-x86_64-linux-gnu.so:
>>>>> error: depends on 'libgdal.so.20', which cannot be found in RUNPATH
>>>>> ("/gnu/store/b7fqhszxl02g6pfm3vw6b3cjz472qrly-python-3.7.0/lib"
>>>>> "/gnu/store/h90vnqw0nwd0hhm1l5dgxsdrigddfmq4-glibc-2.28/lib"
>>>>> "/gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib"
>>>>> "/gnu/store/4sqps8dczv3g7rwbdibfz6rf5jlk7w90-gcc-5.5.0-lib/lib/gcc/x86_64-unknown-linux-gnu/5.5.0/../../..")
>>>>
>>>> I’m not familiar with Cython so I don’t know how this was handled
>>>> before.  However, to me, it indicates that the resulting binaries are
>>>> unlikely to work.
>>>>
>>>> Namely, Python would dlopen “_gdal.cython*.so”, and that would fail to
>>>> find ‘libgdal.so’.
>>
>>>>
>>>> Thoughts?
>>>
>>> Yes: It does work. But I don’t know why.
>>
>> This is because the _gdal.cython*.so don't have rpath to the "lib"
>> directory, where libgdal.so will going.
>>
>>>
>>> Cython runs at compile-time to generate c-code that acts as interface
>>> for Python. Given the paths in here, this needs gdal already installed
>>> in the runpath where it seems to be missing during validation.
>>>
>>> How can we fix that?
>>
>> I think cython (which compile .py files to .c files) is not here..  It
>> has python C libraries generated by swig.
>>>
>>> Best wishes,
>>> Arne
>>
>> In the end the python bindings for gdal can be build seperated (which
>> handle the missing rpath to libgdal nicely), and I prefer this way:
>>
>> From 09254f646b948a5603b2b3dcb99a7f2f6ad0c8de Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>
>> Date: Sat, 20 Jul 2019 22:39:33 +0800
>> Subject: [PATCH] gnu: Add python-gdal.
>>
>> For <https://issues.guix.gnu.org/issue/36623>, thank to Arne Babenhauserheide
>> for the original patch.
>>
>> * gnu/packages/geo.scm (python-gdal): New package.
>> ---
>>  gnu/packages/geo.scm | 19 +++++++++++++++++++
>>  1 file changed, 19 insertions(+)
>>
>> diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
>> index e4b6a262c7..8005c46129 100644
>> --- a/gnu/packages/geo.scm
>> +++ b/gnu/packages/geo.scm
>> @@ -604,6 +604,25 @@ utilities for data translation and processing.")
>>                 ;; frmts/mrf/libLERC
>>                 license:asl2.0))))
>>  
>> +(define-public python-gdal
>> +  (package (inherit gdal)
>> +    (name "python-gdal")
>> +    (build-system python-build-system)
>> +    (arguments
>> +     '(#:tests? #f                      ; no tests
>> +       #:phases
>> +       (modify-phases %standard-phases
>> +         (add-before 'build 'chdir
>> +           (lambda _
>> +             (chdir "swig/python")
>> +             #t)))))
>> +    (native-inputs '())
>> +    (propagated-inputs
>> +     `(("python-numpy" ,python-numpy)))
>> +    (inputs
>> +     `(("gdal" ,gdal)))
>> +    (synopsis "GDAL (Geospatial Data Abstraction Library) python bindings")))
>> +
>>  (define-public postgis
>>    (package
>>      (name "postgis")
>
> Wow, that looks pretty good!
>
> Thank you for fixing up my patch!
>

Sure, pushed!

Hello,

Arne Babenhauserheide <arne_bab@web.de> skribis:

> 宋文武 <iyzsong@member.fsf.org> writes:

[...]

>> From 09254f646b948a5603b2b3dcb99a7f2f6ad0c8de Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?=E5=AE=8B=E6=96=87=E6=AD=A6?= <iyzsong@member.fsf.org>
>> Date: Sat, 20 Jul 2019 22:39:33 +0800
>> Subject: [PATCH] gnu: Add python-gdal.
>>
>> For <https://issues.guix.gnu.org/issue/36623>, thank to Arne Babenhauserheide
>> for the original patch.
>>
>> * gnu/packages/geo.scm (python-gdal): New package.

[...]

> Wow, that looks pretty good!
>
> Thank you for fixing up my patch!

Great that it works.  Please push and close this issue, 宋文武!

Thanks to both of you,
Ludo’.