[PATCH 0/5] MariaDB updates

  • Done
  • quality assurance status badge
Details
One participant
  • Marius Bakke
Owner
unassigned
Submitted by
Marius Bakke
Severity
normal
M
M
Marius Bakke wrote on 20 Jul 2018 13:35
(address . guix-patches@gnu.org)
20180720113553.18138-1-mbakke@fastmail.com
These patches gives us more control of the MariaDB package. In
particular, we now invoke the test runner directly, so we can pass
custom arguments; there's a phase for disabling plugins in an attempt
to fix the armhf build; we now use system zlib, pcre, xz and snappy;
and the package is ~37 MiB smaller thanks to a new "static" output.

I've verified that 'qtbase' builds against this MariaDB.

Marius Bakke (5):
gnu: mariadb: Disable plugin that fails on armhf.
gnu: mariadb: Run the full test suite.
gnu: mariadb: Update to 10.1.34.
gnu: mariadb: Remove some bundled libraries.
gnu: mariadb: Move static libraries to separate output.

gnu/packages/databases.scm | 104 ++++++++++++++++++++++++++++++++++---
1 file changed, 97 insertions(+), 7 deletions(-)

--
2.18.0
M
M
Marius Bakke wrote on 20 Jul 2018 13:38
[PATCH 1/5] gnu: mariadb: Disable plugin that fails on armhf.
(address . 32221@debbugs.gnu.org)
20180720113837.29412-1-mbakke@fastmail.com
* gnu/packages/databases.scm (mariadb)[arguments]: Add 'disable-plugins' phase.
---
gnu/packages/databases.scm | 13 +++++++++++++
1 file changed, 13 insertions(+)

Toggle diff (26 lines)
diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index b9ae9ee15..c27381ac2 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -665,6 +665,19 @@ Language.")
"-DINSTALL_SHAREDIR=share/mysql")
#:phases
(modify-phases %standard-phases
+ (add-before 'configure 'disable-plugins
+ (lambda _
+ (let ((disable-plugin (lambda (name)
+ (call-with-output-file
+ (string-append "plugin/" name
+ "/CMakeLists.txt")
+ (lambda (port)
+ (format port "\n")))))
+ (disabled-plugins '(;; FIXME: On armhf-linux, this plugin
+ ;; triggers a GCC ICE. Disable for now.
+ "semisync")))
+ (for-each disable-plugin disabled-plugins)
+ #t)))
(add-before
'configure 'pre-configure
(lambda _
--
2.18.0
M
M
Marius Bakke wrote on 20 Jul 2018 13:38
[PATCH 2/5] gnu: mariadb: Run the full test suite.
(address . 32221@debbugs.gnu.org)
20180720113837.29412-2-mbakke@fastmail.com
* gnu/packages/databases.scm (mariadb)[arguments]: Override 'check' phase.
Add phase 'adjust-tests'. Disable one more plugin.
[properties]: New field.
---
gnu/packages/databases.scm | 35 ++++++++++++++++++++++++++++++++++-
1 file changed, 34 insertions(+), 1 deletion(-)

Toggle diff (69 lines)
diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index c27381ac2..9e4adc340 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -665,6 +665,29 @@ Language.")
"-DINSTALL_SHAREDIR=share/mysql")
#:phases
(modify-phases %standard-phases
+ (add-after 'unpack 'adjust-tests
+ (lambda _
+ (let ((disabled-tests
+ '(;; These fail because root@hostname == root@localhost in
+ ;; the build environment, causing a user count mismatch.
+ ;; See <https://jira.mariadb.org/browse/MDEV-7761>.
+ "main.join_cache"
+ "main.explain_non_select"
+ "roles.acl_statistics"))
+
+ ;; This file contains a list of known-flaky tests for this
+ ;; release. Append our own list.
+ (unstable-tests (open-file "mysql-test/unstable-tests" "a")))
+ (for-each (lambda (test)
+ (format unstable-tests "~a : ~a\n"
+ test "Disabled in Guix"))
+ disabled-tests)
+ (close-port unstable-tests)
+
+ (substitute* "mysql-test/mysql-test-run.pl"
+ (("/bin/ls") (which "ls"))
+ (("/bin/sh") (which "sh")))
+ #t)))
(add-before 'configure 'disable-plugins
(lambda _
(let ((disable-plugin (lambda (name)
@@ -675,7 +698,9 @@ Language.")
(format port "\n")))))
(disabled-plugins '(;; FIXME: On armhf-linux, this plugin
;; triggers a GCC ICE. Disable for now.
- "semisync")))
+ "semisync"
+ ;; XXX: Causes a test failure.
+ "disks")))
(for-each disable-plugin disabled-plugins)
#t)))
(add-before
@@ -683,6 +708,11 @@ Language.")
(lambda _
(setenv "CONFIG_SHELL" (which "sh"))
#t))
+ (replace 'check
+ (lambda _
+ (with-directory-excursion "mysql-test"
+ (invoke "./mtr" "--parallel" (number->string (parallel-job-count))
+ "--verbose" "--skip-test-list=unstable-tests"))))
(add-after
'install 'post-install
(lambda* (#:key outputs #:allow-other-keys)
@@ -708,6 +738,9 @@ Language.")
("openssl" ,openssl)
("pcre" ,pcre)
("zlib" ,zlib)))
+ ;; The test suite is very resource intensive and can take more than three
+ ;; hours on a x86_64 system. Give slow and busy machines some leeway.
+ (properties '((timeout . 43200))) ;12 hours
(home-page "https://mariadb.org/")
(synopsis "SQL database server")
(description
--
2.18.0
M
M
Marius Bakke wrote on 20 Jul 2018 13:38
[PATCH 3/5] gnu: mariadb: Update to 10.1.34.
(address . 32221@debbugs.gnu.org)
20180720113837.29412-3-mbakke@fastmail.com
* gnu/packages/databases.scm (mariadb): Update to 10.1.34.
---
gnu/packages/databases.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 9e4adc340..3b767664a 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -627,7 +627,7 @@ Language.")
(define-public mariadb
(package
(name "mariadb")
- (version "10.1.33")
+ (version "10.1.34")
(source (origin
(method url-fetch)
(uri (string-append "https://downloads.mariadb.org/f/"
@@ -635,7 +635,7 @@ Language.")
name "-" version ".tar.gz"))
(sha256
(base32
- "0bax748j4srsyhw5cs5jvwigndh0zwmf4r2cjvhja31ckx8jqccl"))))
+ "0j2mdpyvj41vkq2rwrzky88b7170hzz6gy2vb2bc1447s2gp3q67"))))
(build-system cmake-build-system)
(arguments
'(#:configure-flags
--
2.18.0
M
M
Marius Bakke wrote on 20 Jul 2018 13:38
[PATCH 4/5] gnu: mariadb: Remove some bundled libraries.
(address . 32221@debbugs.gnu.org)
20180720113837.29412-4-mbakke@fastmail.com
* gnu/packages/databases.scm (mariadb)[source](snippet, modules): New fields.
[arguments]: Add explicit #:configure-flags for system libraries. Add
'unbundle' phase. Remove 'pre-configure' phase.
[inputs]: Add SNAPPY and XZ.
---
gnu/packages/databases.scm | 43 ++++++++++++++++++++++++++++++++------
1 file changed, 37 insertions(+), 6 deletions(-)

Toggle diff (84 lines)
diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 3b767664a..de1db2f93 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -635,7 +635,16 @@ Language.")
name "-" version ".tar.gz"))
(sha256
(base32
- "0j2mdpyvj41vkq2rwrzky88b7170hzz6gy2vb2bc1447s2gp3q67"))))
+ "0j2mdpyvj41vkq2rwrzky88b7170hzz6gy2vb2bc1447s2gp3q67"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file-recursively "storage/tokudb/PerconaFT/third_party")
+ (for-each (lambda (file)
+ (unless (string-suffix? "CMakeLists.txt" file)
+ (delete-file file)))
+ (append (find-files "pcre") (find-files "zlib")))
+ #t))))
(build-system cmake-build-system)
(arguments
'(#:configure-flags
@@ -649,6 +658,11 @@ Language.")
;; For now, disable the features that that use libarchive (xtrabackup).
"-DWITH_LIBARCHIVE=OFF"
+ ;; Ensure the system libraries are used.
+ "-DWITH_JEMALLOC=yes"
+ "-DWITH_PCRE=system"
+ "-DWITH_ZLIB=system"
+
"-DDEFAULT_CHARSET=utf8"
"-DDEFAULT_COLLATION=utf8_general_ci"
"-DMYSQL_DATADIR=/var/lib/mysql"
@@ -665,6 +679,26 @@ Language.")
"-DINSTALL_SHAREDIR=share/mysql")
#:phases
(modify-phases %standard-phases
+ (add-after 'unpack 'unbundle
+ (lambda _
+ ;; The bundled PCRE in MariaDB has a patch that was upstreamed
+ ;; in version 8.34. Unfortunately the upstream patch behaves
+ ;; slightly differently and the build system fails to detect it.
+ ;; See <https://bugs.exim.org/show_bug.cgi?id=2173>.
+ ;; XXX: Consider patching PCRE instead.
+ (substitute* "cmake/pcre.cmake"
+ ((" OR NOT PCRE_STACK_SIZE_OK") ""))
+
+ (substitute* "storage/tokudb/PerconaFT/ft/CMakeLists.txt"
+ ;; Remove dependency on these CMake targets.
+ ((" build_lzma build_snappy") ""))
+
+ (substitute* "storage/tokudb/PerconaFT/CMakeLists.txt"
+ ;; This file checks that the bundled sources are present and
+ ;; declares build procedures for them. We don't need that.
+ (("^include\\(TokuThirdParty\\)") ""))
+
+ #t))
(add-after 'unpack 'adjust-tests
(lambda _
(let ((disabled-tests
@@ -703,11 +737,6 @@ Language.")
"disks")))
(for-each disable-plugin disabled-plugins)
#t)))
- (add-before
- 'configure 'pre-configure
- (lambda _
- (setenv "CONFIG_SHELL" (which "sh"))
- #t))
(replace 'check
(lambda _
(with-directory-excursion "mysql-test"
@@ -737,6 +766,8 @@ Language.")
("ncurses" ,ncurses)
("openssl" ,openssl)
("pcre" ,pcre)
+ ("snappy" ,snappy)
+ ("xz" ,xz)
("zlib" ,zlib)))
;; The test suite is very resource intensive and can take more than three
;; hours on a x86_64 system. Give slow and busy machines some leeway.
--
2.18.0
M
M
Marius Bakke wrote on 20 Jul 2018 13:38
[PATCH 5/5] gnu: mariadb: Move static libraries to separate output.
(address . 32221@debbugs.gnu.org)
20180720113837.29412-5-mbakke@fastmail.com
* gnu/packages/databases.scm (mariadb)[outputs]: New field.
[arguments]: Add 'move-static-libs' phase.
---
gnu/packages/databases.scm | 13 +++++++++++++
1 file changed, 13 insertions(+)

Toggle diff (33 lines)
diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index de1db2f93..cbc530bec 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -646,6 +646,7 @@ Language.")
(append (find-files "pcre") (find-files "zlib")))
#t))))
(build-system cmake-build-system)
+ (outputs '("out" "static"))
(arguments
'(#:configure-flags
'("-DBUILD_CONFIG=mysql_release"
@@ -742,6 +743,18 @@ Language.")
(with-directory-excursion "mysql-test"
(invoke "./mtr" "--parallel" (number->string (parallel-job-count))
"--verbose" "--skip-test-list=unstable-tests"))))
+ (add-after 'install 'move-static-libs
+ ;; Move ~37 MiB worth of static libraries to a separate output.
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((lib (string-append (assoc-ref outputs "out") "/lib"))
+ (slib (string-append (assoc-ref outputs "static") "/lib")))
+ (mkdir-p slib)
+ (with-directory-excursion lib
+ (for-each (lambda (ar)
+ (link ar (string-append slib "/" (basename ar)))
+ (delete-file ar))
+ (find-files "." "\\.a$"))
+ #t))))
(add-after
'install 'post-install
(lambda* (#:key outputs #:allow-other-keys)
--
2.18.0
M
M
Marius Bakke wrote on 28 Jul 2018 17:39
Re: [bug#32221] [PATCH 0/5] MariaDB updates
(address . 32221-done@debbugs.gnu.org)
87lg9v4bfp.fsf@fastmail.com
Marius Bakke <mbakke@fastmail.com> writes:

Toggle quote (15 lines)
> These patches gives us more control of the MariaDB package. In
> particular, we now invoke the test runner directly, so we can pass
> custom arguments; there's a phase for disabling plugins in an attempt
> to fix the armhf build; we now use system zlib, pcre, xz and snappy;
> and the package is ~37 MiB smaller thanks to a new "static" output.
>
> I've verified that 'qtbase' builds against this MariaDB.
>
> Marius Bakke (5):
> gnu: mariadb: Disable plugin that fails on armhf.
> gnu: mariadb: Run the full test suite.
> gnu: mariadb: Update to 10.1.34.
> gnu: mariadb: Remove some bundled libraries.
> gnu: mariadb: Move static libraries to separate output.

I've pushed this patchset with slight modifications:

* Also unbundled YaSSL
* Deleted static library instead separate output
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAltcjhoACgkQoqBt8qM6
VPobvQgAiSPy2slx2ffpJ3lMuGWq5nlP4D8gqXiSOw9MtbbWsJimJWcL9QJeD8XO
8GQCw51tUSirL+P1ComxUVgSWz1uTWjd18GPxK3z0oHiCfLNhqmALT/VxKqpYBdi
7ICP4Y2WSp2uWcRbmH/JA9bc1Hh84LmFazSjefD9P+aCckFQvX/o4X2SqYJOmr/p
lyNs4pF4AJsqUXxpteR/kUqUZJIQC6vIbDboWuoe417PF7tCAfajTekWu/FsIHlm
Evw6hplmApj79l5DVmVTDTeWQxPBViCJzCEKM6qweqFnD7C1ToL3zQVf00ecbE5P
77U48cIdyhpz8a9s88sEtH/W4IEtyg==
=egMZ
-----END PGP SIGNATURE-----

Closed
?