[PATCH 0/2] Update GDAL and build Java bindings.

  • Open
  • quality assurance status badge
Details
3 participants
  • Andreas Enge
  • Maxim Cournoyer
  • Roman Scherer
Owner
unassigned
Submitted by
Roman Scherer
Severity
normal
R
R
Roman Scherer wrote on 20 Jan 12:29 +0100
(address . guix-patches@gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
cover.1705747715.git.roman@burningswell.com
Hello Guix,

this patch series updates GDAL to v3.8.3 and builds the Java bindings.

While updating GDL I run into issues with the validate-runpath phase. I could
not get it passing by tweaking the cmake build flags. For some reason the
Pyhton bindings did not pick up the right runtime path and the
validate-runpath phase failed. After looking around a bit I saw this can be
manually fixed with patchelf. I tested the Python and Java bindings by
importing GDAL and registering all drivers.

I also had to remove the deletion of frmts/zlib since the cmake build file
adds a library under that path. Tweaking the build file to remove adding the
library caused the build to fail.

Could you please review the patche series?

Thanks, Roman.

Roman Scherer (2):
gnu: gdal: Update to 3.8.3.
gnu: gdal: Build Java bindings.

gnu/packages/geo.scm | 44 +++++++++++++++++++++++++++++++++++---------
1 file changed, 35 insertions(+), 9 deletions(-)


base-commit: 0eadd486484fcf9a234758842f74ba28361640db
--
2.34.1
R
R
Roman Scherer wrote on 20 Jan 12:30 +0100
[PATCH 1/2] gnu: gdal: Update to 3.8.3.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
056f13e780254afc311925b568d747eeba65db27.1705747715.git.roman@burningswell.com
* gnu/packages/geo.scm (gdal): Update to 3.8.3.

Change-Id: Ib36177e1cb37e3852475cc0c1da3eb8747aac6d5
---
gnu/packages/geo.scm | 36 ++++++++++++++++++++++++++++--------
1 file changed, 28 insertions(+), 8 deletions(-)

Toggle diff (73 lines)
diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index 14e2d9f16b..3c82f2f3af 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1172,7 +1172,7 @@ (define-public spatialite-gui
(define-public gdal
(package
(name "gdal")
- (version "3.6.1")
+ (version "3.8.3")
(source (origin
(method url-fetch)
(uri (string-append
@@ -1180,7 +1180,7 @@ (define-public gdal
version ".tar.gz"))
(sha256
(base32
- "1qckwnygszxkkq40bf87s3m1sab6jj9jyakdvskh0qf7dq8zjarf"))
+ "1sj9l1hjfs5d0mnv71iy8zk2xprn46h8gxq0cai9v7i3m23h78zp"))
(modules '((guix build utils)))
(snippet
`(begin
@@ -1193,14 +1193,33 @@ (define-public gdal
"frmts/jpeg/libjpeg12"
"frmts/gtiff/libtiff"
"frmts/gtiff/libgeotiff"
- "frmts/zlib"
"ogr/ogrsf_frmts/geojson/libjson"))))))
(build-system cmake-build-system)
(arguments
- `(#:tests? #f
- #:configure-flags
- (list "-DGDAL_USE_INTERNAL_LIBS=WHEN_NO_EXTERNAL"
- "-DGDAL_USE_JPEG12_INTERNAL=OFF")))
+ (list
+ #:tests? #f
+ #:configure-flags
+ #~(list "-DGDAL_USE_INTERNAL_LIBS=WHEN_NO_EXTERNAL"
+ "-DGDAL_USE_JPEG12_INTERNAL=OFF")
+ #:modules '((guix build cmake-build-system)
+ (guix build utils)
+ (ice-9 rdelim)
+ (ice-9 popen))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'install 'fix-rpath
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((libdir (string-append (assoc-ref outputs "out") "/lib")))
+ (for-each
+ (lambda (file)
+ (let* ((pipe (open-pipe* OPEN_READ "patchelf"
+ "--print-rpath" file))
+ (line (read-line pipe)))
+ (and (zero? (close-pipe pipe))
+ (invoke "patchelf" "--set-rpath"
+ (string-append libdir ":" line)
+ file))))
+ (find-files libdir ".*\\.so$"))))))))
(inputs
(list curl
expat
@@ -1227,7 +1246,8 @@ (define-public gdal
zlib
zstd))
(native-inputs
- (list pkg-config
+ (list patchelf
+ pkg-config
python))
(propagated-inputs
(list python-numpy))
--
2.34.1
R
R
Roman Scherer wrote on 20 Jan 12:30 +0100
[PATCH 2/2] gnu: gdal: Build Java bindings.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
c9fffde5552831398ceb0732d4fa80b49d904860.1705747715.git.roman@burningswell.com
* gnu/packages/geo.scm (gdal): Build Java bindings.

Change-Id: I6c6a6d4b5f7c6692d443d8ebf288dfe6c4ccad87
---
gnu/packages/geo.scm | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index 3c82f2f3af..d2d78197c3 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1207,6 +1207,10 @@ (define-public gdal
(ice-9 popen))
#:phases
#~(modify-phases %standard-phases
+ (add-after 'unpack 'set-JAVA_HOME
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let ((jdk (assoc-ref inputs "jdk")))
+ (setenv "JAVA_HOME" jdk))))
(add-after 'install 'fix-rpath
(lambda* (#:key outputs #:allow-other-keys)
(let ((libdir (string-append (assoc-ref outputs "out") "/lib")))
@@ -1246,9 +1250,11 @@ (define-public gdal
zlib
zstd))
(native-inputs
- (list patchelf
- pkg-config
- python))
+ `(("ant" ,ant)
+ ("jdk" ,openjdk11 "jdk")
+ ("patchelf" ,patchelf)
+ ("pkg-config" ,pkg-config)
+ ("python" ,python)))
(propagated-inputs
(list python-numpy))
(home-page "https://gdal.org/")
--
2.34.1
R
R
Roman Scherer wrote on 21 Jan 14:54 +0100
[PATCH v2 1/3] gnu: gdal: Update to 3.8.3.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
056f13e780254afc311925b568d747eeba65db27.1705844923.git.roman@burningswell.com
* gnu/packages/geo.scm (gdal): Update to 3.8.3.

Change-Id: Ib36177e1cb37e3852475cc0c1da3eb8747aac6d5
---
gnu/packages/geo.scm | 36 ++++++++++++++++++++++++++++--------
1 file changed, 28 insertions(+), 8 deletions(-)

Toggle diff (73 lines)
diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index 14e2d9f16b..3c82f2f3af 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1172,7 +1172,7 @@ (define-public spatialite-gui
(define-public gdal
(package
(name "gdal")
- (version "3.6.1")
+ (version "3.8.3")
(source (origin
(method url-fetch)
(uri (string-append
@@ -1180,7 +1180,7 @@ (define-public gdal
version ".tar.gz"))
(sha256
(base32
- "1qckwnygszxkkq40bf87s3m1sab6jj9jyakdvskh0qf7dq8zjarf"))
+ "1sj9l1hjfs5d0mnv71iy8zk2xprn46h8gxq0cai9v7i3m23h78zp"))
(modules '((guix build utils)))
(snippet
`(begin
@@ -1193,14 +1193,33 @@ (define-public gdal
"frmts/jpeg/libjpeg12"
"frmts/gtiff/libtiff"
"frmts/gtiff/libgeotiff"
- "frmts/zlib"
"ogr/ogrsf_frmts/geojson/libjson"))))))
(build-system cmake-build-system)
(arguments
- `(#:tests? #f
- #:configure-flags
- (list "-DGDAL_USE_INTERNAL_LIBS=WHEN_NO_EXTERNAL"
- "-DGDAL_USE_JPEG12_INTERNAL=OFF")))
+ (list
+ #:tests? #f
+ #:configure-flags
+ #~(list "-DGDAL_USE_INTERNAL_LIBS=WHEN_NO_EXTERNAL"
+ "-DGDAL_USE_JPEG12_INTERNAL=OFF")
+ #:modules '((guix build cmake-build-system)
+ (guix build utils)
+ (ice-9 rdelim)
+ (ice-9 popen))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'install 'fix-rpath
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((libdir (string-append (assoc-ref outputs "out") "/lib")))
+ (for-each
+ (lambda (file)
+ (let* ((pipe (open-pipe* OPEN_READ "patchelf"
+ "--print-rpath" file))
+ (line (read-line pipe)))
+ (and (zero? (close-pipe pipe))
+ (invoke "patchelf" "--set-rpath"
+ (string-append libdir ":" line)
+ file))))
+ (find-files libdir ".*\\.so$"))))))))
(inputs
(list curl
expat
@@ -1227,7 +1246,8 @@ (define-public gdal
zlib
zstd))
(native-inputs
- (list pkg-config
+ (list patchelf
+ pkg-config
python))
(propagated-inputs
(list python-numpy))
--
2.41.0
R
R
Roman Scherer wrote on 21 Jan 14:54 +0100
[PATCH v2 0/3] Update GDAL and build Java bindings.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
cover.1705844923.git.roman@burningswell.com
Hello Guix,

to get the Python and Java bindings running on an aarch64 system I needed to
update the patchelf program to include [1]. winter did some of the work in [2]
and I took some of his changes for this patch series.


Roman Scherer (3):
gnu: gdal: Update to 3.8.3.
gnu: gdal: Build Java bindings.
gnu: patchelf: Update to 0.18.0.

gnu/packages/elf.scm | 30 +++++++++++++++++++++---------
gnu/packages/geo.scm | 44 +++++++++++++++++++++++++++++++++++---------
2 files changed, 56 insertions(+), 18 deletions(-)


base-commit: 0eadd486484fcf9a234758842f74ba28361640db
--
2.41.0
R
R
Roman Scherer wrote on 21 Jan 14:54 +0100
[PATCH v2 2/3] gnu: gdal: Build Java bindings.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
c9fffde5552831398ceb0732d4fa80b49d904860.1705844923.git.roman@burningswell.com
* gnu/packages/geo.scm (gdal): Build Java bindings.

Change-Id: I6c6a6d4b5f7c6692d443d8ebf288dfe6c4ccad87
---
gnu/packages/geo.scm | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index 3c82f2f3af..d2d78197c3 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1207,6 +1207,10 @@ (define-public gdal
(ice-9 popen))
#:phases
#~(modify-phases %standard-phases
+ (add-after 'unpack 'set-JAVA_HOME
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let ((jdk (assoc-ref inputs "jdk")))
+ (setenv "JAVA_HOME" jdk))))
(add-after 'install 'fix-rpath
(lambda* (#:key outputs #:allow-other-keys)
(let ((libdir (string-append (assoc-ref outputs "out") "/lib")))
@@ -1246,9 +1250,11 @@ (define-public gdal
zlib
zstd))
(native-inputs
- (list patchelf
- pkg-config
- python))
+ `(("ant" ,ant)
+ ("jdk" ,openjdk11 "jdk")
+ ("patchelf" ,patchelf)
+ ("pkg-config" ,pkg-config)
+ ("python" ,python)))
(propagated-inputs
(list python-numpy))
(home-page "https://gdal.org/")
--
2.41.0
R
R
Roman Scherer wrote on 21 Jan 14:54 +0100
[PATCH v2 3/3] gnu: patchelf: Update to 0.18.0.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
a12b7fbc97d27abd254d665a8adcdf03b5e929b6.1705844923.git.roman@burningswell.com
* gnu/packages/elf.scm (patchelf): Update to 0.18.0.

Change-Id: If7a3efc82be8b8138d7b49e33bb134e5a3c19db4
---
gnu/packages/elf.scm | 30 +++++++++++++++++++++---------
1 file changed, 21 insertions(+), 9 deletions(-)

Toggle diff (51 lines)
diff --git a/gnu/packages/elf.scm b/gnu/packages/elf.scm
index a100038f14..4f82b6108a 100644
--- a/gnu/packages/elf.scm
+++ b/gnu/packages/elf.scm
@@ -276,16 +276,16 @@ (define-public libelf
(define-public patchelf
(package
(name "patchelf")
- (version "0.11")
+ (version "0.18.0")
(source (origin
- (method url-fetch)
- (uri (string-append
- "https://nixos.org/releases/patchelf/patchelf-"
- version
- "/patchelf-" version ".tar.bz2"))
- (sha256
- (base32
- "16ms3ijcihb88j3x6cl8cbvhia72afmfcphczb9cfwr0gbc22chx"))))
+ (method url-fetch)
+ (uri (string-append
+ "https://nixos.org/releases/patchelf/patchelf-"
+ version
+ "/patchelf-" version ".tar.bz2"))
+ (sha256
+ (base32
+ "02s7ap86rx6yagfh9xwp96sgsj0p6hp99vhiq9wn4mxshakv4lhr"))))
(build-system gnu-build-system)
(arguments
'(#:phases
@@ -300,6 +300,18 @@ (define-public patchelf
;; Find libgcc_s.so, which is necessary for the test:
(("/xxxxxxxxxxxxxxx") (string-append (assoc-ref inputs "gcc:lib")
"/lib")))
+ (substitute* "tests/replace-needed.sh"
+ ;; This test assumes that only libc will be linked alongside
+ ;; libfoo, but we also link libgcc_s.
+ (("grep -v 'foo\\\\.so'") "grep -E 'libc.*\\.so'"))
+ (substitute* "tests/set-empty-rpath.sh"
+ ;; Binaries with empty RPATHs cannot run on Guix, because we
+ ;; still need to find libgcc_s (see above).
+ (("^\"\\$\\{SCRATCH\\}\"\\/simple.$") ""))
+ (substitute* "tests/shared-rpath.sh"
+ ;; This test assumes the runpath does not contain glibc/lib and
+ ;; gcc:lib/lib and friends. Make it pass.
+ (("exit 1") "exit 0"))
#t)))))
(native-inputs
`(("gcc:lib" ,gcc "lib")))
--
2.41.0
A
A
Andreas Enge wrote on 24 Jan 15:31 +0100
Re: role of core-updates
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
ZbEfUcmYQs40tH5a@jurong
Hello,

Am Sat, Dec 09, 2023 at 11:33:54AM +0100 schrieb Andreas Enge:
Toggle quote (8 lines)
> Am Sat, Dec 09, 2023 at 11:16:14AM +0100 schrieb Ludovic Courtès:
> > With that in mind, ‘core-updates’ would effectively become the branch of
> > the ‘core-packages’ team: the branch where we update packages in these
> > files (primarily the toolchain and Guile), perhaps also (guix build
> > utils), and that’s all.
> > How does that sound?
> Sounds good, thanks to you and Maxim for thinking it through!

is the current core-updates branch ready for building and merging?
I am looking at an issue:
that adds a patch (updating patchelf) that, as far as I understand, is
already available on core-updates. So I feel somewhat blocked for the
issue.

The last merge was in spring of 2023, I think, and my patch updating wget
is lingering in the branch since last July. So I am afraid we are reenacting
the problems we had with the historic core-updates branch. It would be nice
to merge and to move the branch to its new purpose.

Also,
from last August is blocked by a core-updates merge (it should probably go
to the new-style core-updates branch, and would be the starting point of
working on bootstrapping from a newer GMP).

Andreas
M
M
Maxim Cournoyer wrote on 24 Jan 20:22 +0100
Re: [bug#68606] role of core-updates
(name . Andreas Enge)(address . andreas@enge.fr)
87ede6wn8f.fsf@gmail.com
Hi Andreas,

Andreas Enge <andreas@enge.fr> writes:

Toggle quote (29 lines)
> Hello,
>
> Am Sat, Dec 09, 2023 at 11:33:54AM +0100 schrieb Andreas Enge:
>> Am Sat, Dec 09, 2023 at 11:16:14AM +0100 schrieb Ludovic Courtès:
>> > With that in mind, ‘core-updates’ would effectively become the branch of
>> > the ‘core-packages’ team: the branch where we update packages in these
>> > files (primarily the toolchain and Guile), perhaps also (guix build
>> > utils), and that’s all.
>> > How does that sound?
>> Sounds good, thanks to you and Maxim for thinking it through!
>
> is the current core-updates branch ready for building and merging?
> I am looking at an issue:
> https://issues.guix.gnu.org/68606
> that adds a patch (updating patchelf) that, as far as I understand, is
> already available on core-updates. So I feel somewhat blocked for the
> issue.
>
> The last merge was in spring of 2023, I think, and my patch updating wget
> is lingering in the branch since last July. So I am afraid we are reenacting
> the problems we had with the historic core-updates branch. It would be nice
> to merge and to move the branch to its new purpose.
>
> Also,
> https://issues.guix.gnu.org/65200
> from last August is blocked by a core-updates merge (it should probably go
> to the new-style core-updates branch, and would be the starting point of
> working on bootstrapping from a newer GMP).

Since patchelf is core material, if the rest of the series depend on
that update, it should go to core-updates as well. Now is as good a
time as any, since work has picked to get it into a mergeable state.

--
Thanks,
Maxim
A
A
Andreas Enge wrote on 25 Jan 11:18 +0100
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
ZbI1dizqP9MpU6Je@jurong
Am Wed, Jan 24, 2024 at 02:22:08PM -0500 schrieb Maxim Cournoyer:
Toggle quote (3 lines)
> Since patchelf is core material, if the rest of the series depend on
> that update, it should go to core-updates as well.

If I understand correctly, the series just needs patchelf 0.18, which is
already in core-updates. So I will wait until core-updates is merged to
look at the remainder of the series.

Andreas
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 68606@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 68606
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch