[PATCH 0/2] Update GDAL and build Java bindings.

  • Open
  • quality assurance status badge
Details
3 participants
  • Andreas Enge
  • Maxim Cournoyer
  • Roman Scherer
Owner
unassigned
Submitted by
Roman Scherer
Severity
normal
R
R
Roman Scherer wrote on 20 Jan 12:29 +0100
(address . guix-patches@gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
cover.1705747715.git.roman@burningswell.com
Hello Guix,

this patch series updates GDAL to v3.8.3 and builds the Java bindings.

While updating GDL I run into issues with the validate-runpath phase. I could
not get it passing by tweaking the cmake build flags. For some reason the
Pyhton bindings did not pick up the right runtime path and the
validate-runpath phase failed. After looking around a bit I saw this can be
manually fixed with patchelf. I tested the Python and Java bindings by
importing GDAL and registering all drivers.

I also had to remove the deletion of frmts/zlib since the cmake build file
adds a library under that path. Tweaking the build file to remove adding the
library caused the build to fail.

Could you please review the patche series?

Thanks, Roman.

Roman Scherer (2):
gnu: gdal: Update to 3.8.3.
gnu: gdal: Build Java bindings.

gnu/packages/geo.scm | 44 +++++++++++++++++++++++++++++++++++---------
1 file changed, 35 insertions(+), 9 deletions(-)


base-commit: 0eadd486484fcf9a234758842f74ba28361640db
--
2.34.1
R
R
Roman Scherer wrote on 20 Jan 12:30 +0100
[PATCH 1/2] gnu: gdal: Update to 3.8.3.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
056f13e780254afc311925b568d747eeba65db27.1705747715.git.roman@burningswell.com
* gnu/packages/geo.scm (gdal): Update to 3.8.3.

Change-Id: Ib36177e1cb37e3852475cc0c1da3eb8747aac6d5
---
gnu/packages/geo.scm | 36 ++++++++++++++++++++++++++++--------
1 file changed, 28 insertions(+), 8 deletions(-)

Toggle diff (73 lines)
diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index 14e2d9f16b..3c82f2f3af 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1172,7 +1172,7 @@ (define-public spatialite-gui
(define-public gdal
(package
(name "gdal")
- (version "3.6.1")
+ (version "3.8.3")
(source (origin
(method url-fetch)
(uri (string-append
@@ -1180,7 +1180,7 @@ (define-public gdal
version ".tar.gz"))
(sha256
(base32
- "1qckwnygszxkkq40bf87s3m1sab6jj9jyakdvskh0qf7dq8zjarf"))
+ "1sj9l1hjfs5d0mnv71iy8zk2xprn46h8gxq0cai9v7i3m23h78zp"))
(modules '((guix build utils)))
(snippet
`(begin
@@ -1193,14 +1193,33 @@ (define-public gdal
"frmts/jpeg/libjpeg12"
"frmts/gtiff/libtiff"
"frmts/gtiff/libgeotiff"
- "frmts/zlib"
"ogr/ogrsf_frmts/geojson/libjson"))))))
(build-system cmake-build-system)
(arguments
- `(#:tests? #f
- #:configure-flags
- (list "-DGDAL_USE_INTERNAL_LIBS=WHEN_NO_EXTERNAL"
- "-DGDAL_USE_JPEG12_INTERNAL=OFF")))
+ (list
+ #:tests? #f
+ #:configure-flags
+ #~(list "-DGDAL_USE_INTERNAL_LIBS=WHEN_NO_EXTERNAL"
+ "-DGDAL_USE_JPEG12_INTERNAL=OFF")
+ #:modules '((guix build cmake-build-system)
+ (guix build utils)
+ (ice-9 rdelim)
+ (ice-9 popen))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'install 'fix-rpath
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((libdir (string-append (assoc-ref outputs "out") "/lib")))
+ (for-each
+ (lambda (file)
+ (let* ((pipe (open-pipe* OPEN_READ "patchelf"
+ "--print-rpath" file))
+ (line (read-line pipe)))
+ (and (zero? (close-pipe pipe))
+ (invoke "patchelf" "--set-rpath"
+ (string-append libdir ":" line)
+ file))))
+ (find-files libdir ".*\\.so$"))))))))
(inputs
(list curl
expat
@@ -1227,7 +1246,8 @@ (define-public gdal
zlib
zstd))
(native-inputs
- (list pkg-config
+ (list patchelf
+ pkg-config
python))
(propagated-inputs
(list python-numpy))
--
2.34.1
R
R
Roman Scherer wrote on 20 Jan 12:30 +0100
[PATCH 2/2] gnu: gdal: Build Java bindings.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
c9fffde5552831398ceb0732d4fa80b49d904860.1705747715.git.roman@burningswell.com
* gnu/packages/geo.scm (gdal): Build Java bindings.

Change-Id: I6c6a6d4b5f7c6692d443d8ebf288dfe6c4ccad87
---
gnu/packages/geo.scm | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index 3c82f2f3af..d2d78197c3 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1207,6 +1207,10 @@ (define-public gdal
(ice-9 popen))
#:phases
#~(modify-phases %standard-phases
+ (add-after 'unpack 'set-JAVA_HOME
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let ((jdk (assoc-ref inputs "jdk")))
+ (setenv "JAVA_HOME" jdk))))
(add-after 'install 'fix-rpath
(lambda* (#:key outputs #:allow-other-keys)
(let ((libdir (string-append (assoc-ref outputs "out") "/lib")))
@@ -1246,9 +1250,11 @@ (define-public gdal
zlib
zstd))
(native-inputs
- (list patchelf
- pkg-config
- python))
+ `(("ant" ,ant)
+ ("jdk" ,openjdk11 "jdk")
+ ("patchelf" ,patchelf)
+ ("pkg-config" ,pkg-config)
+ ("python" ,python)))
(propagated-inputs
(list python-numpy))
(home-page "https://gdal.org/")
--
2.34.1
R
R
Roman Scherer wrote on 21 Jan 14:54 +0100
[PATCH v2 1/3] gnu: gdal: Update to 3.8.3.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
056f13e780254afc311925b568d747eeba65db27.1705844923.git.roman@burningswell.com
* gnu/packages/geo.scm (gdal): Update to 3.8.3.

Change-Id: Ib36177e1cb37e3852475cc0c1da3eb8747aac6d5
---
gnu/packages/geo.scm | 36 ++++++++++++++++++++++++++++--------
1 file changed, 28 insertions(+), 8 deletions(-)

Toggle diff (73 lines)
diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index 14e2d9f16b..3c82f2f3af 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1172,7 +1172,7 @@ (define-public spatialite-gui
(define-public gdal
(package
(name "gdal")
- (version "3.6.1")
+ (version "3.8.3")
(source (origin
(method url-fetch)
(uri (string-append
@@ -1180,7 +1180,7 @@ (define-public gdal
version ".tar.gz"))
(sha256
(base32
- "1qckwnygszxkkq40bf87s3m1sab6jj9jyakdvskh0qf7dq8zjarf"))
+ "1sj9l1hjfs5d0mnv71iy8zk2xprn46h8gxq0cai9v7i3m23h78zp"))
(modules '((guix build utils)))
(snippet
`(begin
@@ -1193,14 +1193,33 @@ (define-public gdal
"frmts/jpeg/libjpeg12"
"frmts/gtiff/libtiff"
"frmts/gtiff/libgeotiff"
- "frmts/zlib"
"ogr/ogrsf_frmts/geojson/libjson"))))))
(build-system cmake-build-system)
(arguments
- `(#:tests? #f
- #:configure-flags
- (list "-DGDAL_USE_INTERNAL_LIBS=WHEN_NO_EXTERNAL"
- "-DGDAL_USE_JPEG12_INTERNAL=OFF")))
+ (list
+ #:tests? #f
+ #:configure-flags
+ #~(list "-DGDAL_USE_INTERNAL_LIBS=WHEN_NO_EXTERNAL"
+ "-DGDAL_USE_JPEG12_INTERNAL=OFF")
+ #:modules '((guix build cmake-build-system)
+ (guix build utils)
+ (ice-9 rdelim)
+ (ice-9 popen))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'install 'fix-rpath
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((libdir (string-append (assoc-ref outputs "out") "/lib")))
+ (for-each
+ (lambda (file)
+ (let* ((pipe (open-pipe* OPEN_READ "patchelf"
+ "--print-rpath" file))
+ (line (read-line pipe)))
+ (and (zero? (close-pipe pipe))
+ (invoke "patchelf" "--set-rpath"
+ (string-append libdir ":" line)
+ file))))
+ (find-files libdir ".*\\.so$"))))))))
(inputs
(list curl
expat
@@ -1227,7 +1246,8 @@ (define-public gdal
zlib
zstd))
(native-inputs
- (list pkg-config
+ (list patchelf
+ pkg-config
python))
(propagated-inputs
(list python-numpy))
--
2.41.0
R
R
Roman Scherer wrote on 21 Jan 14:54 +0100
[PATCH v2 0/3] Update GDAL and build Java bindings.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
cover.1705844923.git.roman@burningswell.com
Hello Guix,

to get the Python and Java bindings running on an aarch64 system I needed to
update the patchelf program to include [1]. winter did some of the work in [2]
and I took some of his changes for this patch series.


Roman Scherer (3):
gnu: gdal: Update to 3.8.3.
gnu: gdal: Build Java bindings.
gnu: patchelf: Update to 0.18.0.

gnu/packages/elf.scm | 30 +++++++++++++++++++++---------
gnu/packages/geo.scm | 44 +++++++++++++++++++++++++++++++++++---------
2 files changed, 56 insertions(+), 18 deletions(-)


base-commit: 0eadd486484fcf9a234758842f74ba28361640db
--
2.41.0
R
R
Roman Scherer wrote on 21 Jan 14:54 +0100
[PATCH v2 2/3] gnu: gdal: Build Java bindings.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
c9fffde5552831398ceb0732d4fa80b49d904860.1705844923.git.roman@burningswell.com
* gnu/packages/geo.scm (gdal): Build Java bindings.

Change-Id: I6c6a6d4b5f7c6692d443d8ebf288dfe6c4ccad87
---
gnu/packages/geo.scm | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/geo.scm b/gnu/packages/geo.scm
index 3c82f2f3af..d2d78197c3 100644
--- a/gnu/packages/geo.scm
+++ b/gnu/packages/geo.scm
@@ -1207,6 +1207,10 @@ (define-public gdal
(ice-9 popen))
#:phases
#~(modify-phases %standard-phases
+ (add-after 'unpack 'set-JAVA_HOME
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let ((jdk (assoc-ref inputs "jdk")))
+ (setenv "JAVA_HOME" jdk))))
(add-after 'install 'fix-rpath
(lambda* (#:key outputs #:allow-other-keys)
(let ((libdir (string-append (assoc-ref outputs "out") "/lib")))
@@ -1246,9 +1250,11 @@ (define-public gdal
zlib
zstd))
(native-inputs
- (list patchelf
- pkg-config
- python))
+ `(("ant" ,ant)
+ ("jdk" ,openjdk11 "jdk")
+ ("patchelf" ,patchelf)
+ ("pkg-config" ,pkg-config)
+ ("python" ,python)))
(propagated-inputs
(list python-numpy))
(home-page "https://gdal.org/")
--
2.41.0
R
R
Roman Scherer wrote on 21 Jan 14:54 +0100
[PATCH v2 3/3] gnu: patchelf: Update to 0.18.0.
(address . 68606@debbugs.gnu.org)(name . Roman Scherer)(address . roman@burningswell.com)
a12b7fbc97d27abd254d665a8adcdf03b5e929b6.1705844923.git.roman@burningswell.com
* gnu/packages/elf.scm (patchelf): Update to 0.18.0.

Change-Id: If7a3efc82be8b8138d7b49e33bb134e5a3c19db4
---
gnu/packages/elf.scm | 30 +++++++++++++++++++++---------
1 file changed, 21 insertions(+), 9 deletions(-)

Toggle diff (51 lines)
diff --git a/gnu/packages/elf.scm b/gnu/packages/elf.scm
index a100038f14..4f82b6108a 100644
--- a/gnu/packages/elf.scm
+++ b/gnu/packages/elf.scm
@@ -276,16 +276,16 @@ (define-public libelf
(define-public patchelf
(package
(name "patchelf")
- (version "0.11")
+ (version "0.18.0")
(source (origin
- (method url-fetch)
- (uri (string-append
- "https://nixos.org/releases/patchelf/patchelf-"
- version
- "/patchelf-" version ".tar.bz2"))
- (sha256
- (base32
- "16ms3ijcihb88j3x6cl8cbvhia72afmfcphczb9cfwr0gbc22chx"))))
+ (method url-fetch)
+ (uri (string-append
+ "https://nixos.org/releases/patchelf/patchelf-"
+ version
+ "/patchelf-" version ".tar.bz2"))
+ (sha256
+ (base32
+ "02s7ap86rx6yagfh9xwp96sgsj0p6hp99vhiq9wn4mxshakv4lhr"))))
(build-system gnu-build-system)
(arguments
'(#:phases
@@ -300,6 +300,18 @@ (define-public patchelf
;; Find libgcc_s.so, which is necessary for the test:
(("/xxxxxxxxxxxxxxx") (string-append (assoc-ref inputs "gcc:lib")
"/lib")))
+ (substitute* "tests/replace-needed.sh"
+ ;; This test assumes that only libc will be linked alongside
+ ;; libfoo, but we also link libgcc_s.
+ (("grep -v 'foo\\\\.so'") "grep -E 'libc.*\\.so'"))
+ (substitute* "tests/set-empty-rpath.sh"
+ ;; Binaries with empty RPATHs cannot run on Guix, because we
+ ;; still need to find libgcc_s (see above).
+ (("^\"\\$\\{SCRATCH\\}\"\\/simple.$") ""))
+ (substitute* "tests/shared-rpath.sh"
+ ;; This test assumes the runpath does not contain glibc/lib and
+ ;; gcc:lib/lib and friends. Make it pass.
+ (("exit 1") "exit 0"))
#t)))))
(native-inputs
`(("gcc:lib" ,gcc "lib")))
--
2.41.0
A
A
Andreas Enge wrote on 24 Jan 15:31 +0100
Re: role of core-updates
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
ZbEfUcmYQs40tH5a@jurong
Hello,

Am Sat, Dec 09, 2023 at 11:33:54AM +0100 schrieb Andreas Enge:
Toggle quote (8 lines)
> Am Sat, Dec 09, 2023 at 11:16:14AM +0100 schrieb Ludovic Courtès:
> > With that in mind, ‘core-updates’ would effectively become the branch of
> > the ‘core-packages’ team: the branch where we update packages in these
> > files (primarily the toolchain and Guile), perhaps also (guix build
> > utils), and that’s all.
> > How does that sound?
> Sounds good, thanks to you and Maxim for thinking it through!

is the current core-updates branch ready for building and merging?
I am looking at an issue:
that adds a patch (updating patchelf) that, as far as I understand, is
already available on core-updates. So I feel somewhat blocked for the
issue.

The last merge was in spring of 2023, I think, and my patch updating wget
is lingering in the branch since last July. So I am afraid we are reenacting
the problems we had with the historic core-updates branch. It would be nice
to merge and to move the branch to its new purpose.

Also,
from last August is blocked by a core-updates merge (it should probably go
to the new-style core-updates branch, and would be the starting point of
working on bootstrapping from a newer GMP).

Andreas
M
M
Maxim Cournoyer wrote on 24 Jan 20:22 +0100
Re: [bug#68606] role of core-updates
(name . Andreas Enge)(address . andreas@enge.fr)
87ede6wn8f.fsf@gmail.com
Hi Andreas,

Andreas Enge <andreas@enge.fr> writes:

Toggle quote (29 lines)
> Hello,
>
> Am Sat, Dec 09, 2023 at 11:33:54AM +0100 schrieb Andreas Enge:
>> Am Sat, Dec 09, 2023 at 11:16:14AM +0100 schrieb Ludovic Courtès:
>> > With that in mind, ‘core-updates’ would effectively become the branch of
>> > the ‘core-packages’ team: the branch where we update packages in these
>> > files (primarily the toolchain and Guile), perhaps also (guix build
>> > utils), and that’s all.
>> > How does that sound?
>> Sounds good, thanks to you and Maxim for thinking it through!
>
> is the current core-updates branch ready for building and merging?
> I am looking at an issue:
> https://issues.guix.gnu.org/68606
> that adds a patch (updating patchelf) that, as far as I understand, is
> already available on core-updates. So I feel somewhat blocked for the
> issue.
>
> The last merge was in spring of 2023, I think, and my patch updating wget
> is lingering in the branch since last July. So I am afraid we are reenacting
> the problems we had with the historic core-updates branch. It would be nice
> to merge and to move the branch to its new purpose.
>
> Also,
> https://issues.guix.gnu.org/65200
> from last August is blocked by a core-updates merge (it should probably go
> to the new-style core-updates branch, and would be the starting point of
> working on bootstrapping from a newer GMP).

Since patchelf is core material, if the rest of the series depend on
that update, it should go to core-updates as well. Now is as good a
time as any, since work has picked to get it into a mergeable state.

--
Thanks,
Maxim
A
A
Andreas Enge wrote on 25 Jan 11:18 +0100
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
ZbI1dizqP9MpU6Je@jurong
Am Wed, Jan 24, 2024 at 02:22:08PM -0500 schrieb Maxim Cournoyer:
Toggle quote (3 lines)
> Since patchelf is core material, if the rest of the series depend on
> that update, it should go to core-updates as well.

If I understand correctly, the series just needs patchelf 0.18, which is
already in core-updates. So I will wait until core-updates is merged to
look at the remainder of the series.

Andreas
?