[PATCH 0/2]: ruby-build-system: Error or return #t from all phases.

  • Done
  • quality assurance status badge
Details
2 participants
  • Christopher Baines
  • Marius Bakke
Owner
unassigned
Submitted by
Christopher Baines
Severity
normal

Debbugs page

Christopher Baines wrote 7 years ago
(address . guix-patches@gnu.org)
87wotydqm0.fsf@cbaines.net
I'm trying to continue along with the Rails packaging (#30689), but I
noticed that currently if the tests fail for packages using the ruby
build system, then the package build doesn't fail.

These patches should get most of the packages using the ruby build
system to raise exceptions when there are errors, and return #t
otherwise.

I'm hopeful that this can be merged directly in to master, I build 180
packages in not that much time at all to test this change [1].

1: ./pre-inst-env guix package -s ruby- | recsel -P name | xargs ./pre-inst-env guix build


Christopher Baines (2):
ruby-build-system: Error or return #t from all phases.
gnu: ruby-options: Return #t from set-LIB phase.

gnu/packages/ruby.scm | 3 +-
guix/build/ruby-build-system.scm | 109 ++++++++++++++++---------------
2 files changed, 58 insertions(+), 54 deletions(-)
-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAltJ2QdfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE
9XdjNA//a4kkwywgONlHGfQo/Hf2ThFC+ouZeMFMT83qvuVtnVAaymm/u82g5Zbb
F88GVBmzvejNA1j49nSXTrcTB54KIGdGwvwKYy3c9k1mvqQAE2M79Q7A/8Plo+ur
gpTg20Qv+0KL7fwYECLGWz7gv2Z0NbSgc0HaYvjtsHMDQrKFHMQHZex1d6qattUw
kbnrd0tEjh/VLWJyr+VjnxkvFL1s5XfK9/PRBWa5CFb5alz2mZmdfYtcJXOnCmcT
4eli/oSq6tRcnrmPrzgClRRPRofUkKiJDao6qsOvjBUdegEhA+n/ExPJaP/xb6BR
sbWTKANAqwwhD9Att7FNvVe+lRIHw51a73/wx4Wxy4N65qPj8XHcNNFykb7yJvVe
40Bo6hcsZF1fPLEtVkWVVdMjbXbndB/98khnFMQjnth73DnjhwnH+4ttSH3IdX3f
ml8FWu3Jp2CXcL1Ufnl2fHhB55Mrd7UsmFJQtMnzrbNZ4LZZzkjRWkUBhtGYfxHt
q0e61Cb4YqAQOLk/sRRB3jKfQA0cw4eKiSgPSKbPR+nMtt/iHySDBF/AGP5qV9EE
XS8Vhbb+BFRAwHDH/TMjJI62GYcmk5XnJm2PWQtcXTuoP3OHHtmH3CkZIpNyAb1l
9/OKsNN+0GtY2FUFHMFHlvqID+fC4bTJ6qzDlMhio7GaOhlnz90=
=Slg7
-----END PGP SIGNATURE-----

Christopher Baines wrote 7 years ago
[PATCH 1/2] ruby-build-system: Error or return #t from all phases.
(address . 32153@debbugs.gnu.org)
20180714111022.12609-1-mail@cbaines.net
Previously, if the tests didn't pass, the check phase would evaluate to #f,
but the package would be built sucessfully. This changes all the phases to
raise exceptions if errors are encountered, and return #t otherwise.

This involves using invoke rather than system*, so that exceptions are raised
if the program exits with a status other than 0, and also returning #t at the
end of functions.

* gnu/build/ruby-build-system.scm (unpack): Use invoke rather than system*,
and return #t at the end.
(build, check): Use invoke rather than system*.
(install): Remove the use of "and", and rewrite the error handling to raise an
exception.
(wrap): Return #t.
---
guix/build/ruby-build-system.scm | 109 ++++++++++++++++---------------
1 file changed, 56 insertions(+), 53 deletions(-)

Toggle diff (155 lines)
diff --git a/guix/build/ruby-build-system.scm b/guix/build/ruby-build-system.scm
index abef6937b..0a2b223db 100644
--- a/guix/build/ruby-build-system.scm
+++ b/guix/build/ruby-build-system.scm
@@ -52,18 +52,19 @@ directory."
(define* (unpack #:key source #:allow-other-keys)
"Unpack the gem SOURCE and enter the resulting directory."
(if (gem-archive? source)
- (and (zero? (system* "gem" "unpack" source))
- ;; The unpacked gem directory is named the same as the archive,
- ;; sans the ".gem" extension. It is renamed to simply "gem" in an
- ;; effort to keep file names shorter to avoid UNIX-domain socket
- ;; file names and shebangs that exceed the system's fixed maximum
- ;; length when running test suites.
- (let ((dir (match:substring (string-match "^(.*)\\.gem$"
- (basename source))
- 1)))
- (rename-file dir "gem")
- (chdir "gem")
- #t))
+ (begin
+ (invoke "gem" "unpack" source)
+ ;; The unpacked gem directory is named the same as the archive,
+ ;; sans the ".gem" extension. It is renamed to simply "gem" in an
+ ;; effort to keep file names shorter to avoid UNIX-domain socket
+ ;; file names and shebangs that exceed the system's fixed maximum
+ ;; length when running test suites.
+ (let ((dir (match:substring (string-match "^(.*)\\.gem$"
+ (basename source))
+ 1)))
+ (rename-file dir "gem")
+ (chdir "gem"))
+ #t)
;; Use GNU unpack strategy for things that aren't gem archives.
(gnu:unpack #:source source)))
@@ -104,7 +105,8 @@ generate the files list."
(write-char (read-char pipe) out))))
#t)
(lambda ()
- (close-pipe pipe)))))))
+ (close-pipe pipe)))))
+ #t))
(define* (build #:key source #:allow-other-keys)
"Build a new gem using the gemspec from the SOURCE gem."
@@ -112,13 +114,13 @@ generate the files list."
;; Build a new gem from the current working directory. This also allows any
;; dynamic patching done in previous phases to be present in the installed
;; gem.
- (zero? (system* "gem" "build" (first-gemspec))))
+ (invoke "gem" "build" (first-gemspec)))
(define* (check #:key tests? test-target #:allow-other-keys)
"Run the gem's test suite rake task TEST-TARGET. Skip the tests if TESTS?
is #f."
(if tests?
- (zero? (system* "rake" test-target))
+ (invoke "rake" test-target)
#t))
(define* (install #:key inputs outputs (gem-flags '())
@@ -137,43 +139,43 @@ GEM-FLAGS are passed to the 'gem' invokation, if present."
0
(- (string-length gem-file-basename) 4))))
(setenv "GEM_VENDOR" vendor-dir)
- (and (let ((install-succeeded?
- (zero?
- (apply system* "gem" "install" gem-file
- "--local" "--ignore-dependencies" "--vendor"
- ;; Executables should go into /bin, not
- ;; /lib/ruby/gems.
- "--bindir" (string-append out "/bin")
- gem-flags))))
- (or install-succeeded?
- (begin
- (simple-format #t "installation failed\n")
- (let ((failed-output-dir (string-append (getcwd) "/out")))
- (mkdir failed-output-dir)
- (copy-recursively out failed-output-dir))
- #f)))
- (begin
- ;; Remove the cached gem file as this is unnecessary and contains
- ;; timestamped files rendering builds not reproducible.
- (let ((cached-gem (string-append vendor-dir "/cache/" gem-file)))
- (log-file-deletion cached-gem)
- (delete-file cached-gem))
- ;; For gems with native extensions, several Makefile-related files
- ;; are created that contain timestamps or other elements making
- ;; them not reproducible. They are unnecessary so we remove them.
- (if (file-exists? (string-append vendor-dir "/ext"))
- (begin
- (for-each (lambda (file)
- (log-file-deletion file)
- (delete-file file))
- (append
- (find-files (string-append vendor-dir "/doc")
- "page-Makefile.ri")
- (find-files (string-append vendor-dir "/extensions")
- "gem_make.out")
- (find-files (string-append vendor-dir "/ext")
- "Makefile")))))
- #t))))
+
+ (or (zero?
+ (apply system* "gem" "install" gem-file
+ "--local" "--ignore-dependencies" "--vendor"
+ ;; Executables should go into /bin, not
+ ;; /lib/ruby/gems.
+ "--bindir" (string-append out "/bin")
+ gem-flags))
+ (begin
+ (let ((failed-output-dir (string-append (getcwd) "/out")))
+ (mkdir failed-output-dir)
+ (copy-recursively out failed-output-dir))
+ (error "installation failed")))
+
+ ;; Remove the cached gem file as this is unnecessary and contains
+ ;; timestamped files rendering builds not reproducible.
+ (let ((cached-gem (string-append vendor-dir "/cache/" gem-file)))
+ (log-file-deletion cached-gem)
+ (delete-file cached-gem))
+
+ ;; For gems with native extensions, several Makefile-related files
+ ;; are created that contain timestamps or other elements making
+ ;; them not reproducible. They are unnecessary so we remove them.
+ (if (file-exists? (string-append vendor-dir "/ext"))
+ (begin
+ (for-each (lambda (file)
+ (log-file-deletion file)
+ (delete-file file))
+ (append
+ (find-files (string-append vendor-dir "/doc")
+ "page-Makefile.ri")
+ (find-files (string-append vendor-dir "/extensions")
+ "gem_make.out")
+ (find-files (string-append vendor-dir "/ext")
+ "Makefile")))))
+
+ #t))
(define* (wrap-ruby-program prog #:key (gem-clear-paths #t) #:rest vars)
"Make a wrapper for PROG. VARS should look like this:
@@ -301,7 +303,8 @@ extended with definitions for VARS."
(let ((files (list-of-files dir)))
(for-each (cut wrap-ruby-program <> var)
files)))
- bindirs)))
+ bindirs))
+ #t)
(define (log-file-deletion file)
(display (string-append "deleting '" file "' for reproducibility\n")))
--
2.17.1
Christopher Baines wrote 7 years ago
[PATCH 2/2] gnu: ruby-options: Return #t from set-LIB phase.
(address . 32153@debbugs.gnu.org)
20180714111022.12609-2-mail@cbaines.net
* gnu/packages/ruby.scm (ruby-options)[arguments]: Return #t from the set-LIB
phase.
---
gnu/packages/ruby.scm | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)

Toggle diff (16 lines)
diff --git a/gnu/packages/ruby.scm b/gnu/packages/ruby.scm
index 1602fd5d5..9a74f16c0 100644
--- a/gnu/packages/ruby.scm
+++ b/gnu/packages/ruby.scm
@@ -883,7 +883,8 @@ complexity.")
(lambda _
;; This is used in the Rakefile, and setting it avoids an issue
;; with running the tests.
- (setenv "LIB" "options"))))))
+ (setenv "LIB" "options")
+ #t)))))
(synopsis "Ruby library to parse options from *args cleanly")
(description
"The @code{options} library helps with parsing keyword options in Ruby
--
2.17.1
Marius Bakke wrote 7 years ago
Re: [bug#32153] [PATCH 1/2] ruby-build-system: Error or return #t from all phases.
87y3ed5s6t.fsf@fastmail.com
Christopher Baines <mail@cbaines.net> writes:

Toggle quote (15 lines)
> Previously, if the tests didn't pass, the check phase would evaluate to #f,
> but the package would be built sucessfully. This changes all the phases to
> raise exceptions if errors are encountered, and return #t otherwise.
>
> This involves using invoke rather than system*, so that exceptions are raised
> if the program exits with a status other than 0, and also returning #t at the
> end of functions.
>
> * gnu/build/ruby-build-system.scm (unpack): Use invoke rather than system*,
> and return #t at the end.
> (build, check): Use invoke rather than system*.
> (install): Remove the use of "and", and rewrite the error handling to raise an
> exception.
> (wrap): Return #t.

Thanks! The changes LGTM. I will suggest a micro-improvement not
related to this patch since it was there from before:

Toggle quote (18 lines)
> + ;; For gems with native extensions, several Makefile-related files
> + ;; are created that contain timestamps or other elements making
> + ;; them not reproducible. They are unnecessary so we remove them.
> + (if (file-exists? (string-append vendor-dir "/ext"))
> + (begin
> + (for-each (lambda (file)
> + (log-file-deletion file)
> + (delete-file file))
> + (append
> + (find-files (string-append vendor-dir "/doc")
> + "page-Makefile.ri")
> + (find-files (string-append vendor-dir "/extensions")
> + "gem_make.out")
> + (find-files (string-append vendor-dir "/ext")
> + "Makefile")))))
> +
> + #t))

I was confused whether the #t was the "else" clause for the "if"
expression until I realized it didn't have one.

Could you turn this into a (when (file-exists? ...) (for-each ...))
while at it? It also makes the (begin ...) redundant.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAltKgwoACgkQoqBt8qM6
VPokiQf/RtAYhOZJ9lOGLVoUzC8d9qh0NcGX440pJ7ao3+u3Dw8Kq06nUu0Q/Vqf
1RUCoFN2kQz8Ckq/v5Yzu2fTYqcEoRsIvHOuty3Iktywtv4cDzpXkn6VvbhVEykl
IrB8c9Xdp+jPnTN1diRuERb7aMLTp00KxCM0DKVywCPCnIe0cKorVHe6HTvcU9Ab
ALCTsxyuKZOy3RsiMmLucuu/92e3uwi8Gi1CrAsQZkNf5LWFVyfsJw+WImNG+0jZ
iCBixs2W8Cq4txug/MfbENE18o8MvTFG/QMzk2fglregncNPyz4Hj6oOr8pTVgCo
LXvENdfuWltxwL2GRdWnePKcPm8TEg==
=5PzX
-----END PGP SIGNATURE-----

Marius Bakke wrote 7 years ago
Re: [bug#32153] [PATCH 0/2]: ruby-build-system: Error or return #t from all phases.
87va9h5rv5.fsf@fastmail.com
Christopher Baines <mail@cbaines.net> writes:

Toggle quote (13 lines)
> I'm trying to continue along with the Rails packaging (#30689), but I
> noticed that currently if the tests fail for packages using the ruby
> build system, then the package build doesn't fail.
>
> These patches should get most of the packages using the ruby build
> system to raise exceptions when there are errors, and return #t
> otherwise.
>
> I'm hopeful that this can be merged directly in to master, I build 180
> packages in not that much time at all to test this change [1].
>
> 1: ./pre-inst-env guix package -s ruby- | recsel -P name | xargs ./pre-inst-env guix build

Thank you for fixing it! Since this only affects gems, not ruby itself
(which has ~900 dependencies), I think it can go on 'master' too[1].

[1] guix refresh -l $(guix package -s ^ruby- | recsel -P name)

By the way, Ruby 2.5 is out. Are you willing to try upgrading it for
'staging'? :-)
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAltKhK4ACgkQoqBt8qM6
VPrBXQf+P/N02WeLj4b8liWVvdHYNGOkjiD2rLF/rvbOALUHUM35jowDCkr8AeJs
SEupTbM2sYuaoUQ1R7yayyv0EkzMFQ0Tza+qODVzQoiPK8r3haD6fiOHA4vw89eD
bdyNle5RV0ZaYlDLB/EyQ1vuQaaTjTMhMtn7GMWn/BSqyPByL4oCe1nNbrXoiLVU
1hpuTMtTABseC6THI9JinNGWcVXkrkujYCvi//fr2TBlmD8kzvrNE66HZ19iTOwd
P0CleT4zsJuzXVmrDB8NYQMN0rvvRE+HSnX84Cg7pzHJjpn//q5B1/cqxQq79FKk
NpnpcCHXAaOLp1k80mK1PRzQ3UJY8Q==
=PQVI
-----END PGP SIGNATURE-----

Christopher Baines wrote 7 years ago
(name . Marius Bakke)(address . mbakke@fastmail.com)(address . 32153@debbugs.gnu.org)
87va9gewla.fsf@cbaines.net
Marius Bakke <mbakke@fastmail.com> writes:

Toggle quote (20 lines)
> Christopher Baines <mail@cbaines.net> writes:
>
>> I'm trying to continue along with the Rails packaging (#30689), but I
>> noticed that currently if the tests fail for packages using the ruby
>> build system, then the package build doesn't fail.
>>
>> These patches should get most of the packages using the ruby build
>> system to raise exceptions when there are errors, and return #t
>> otherwise.
>>
>> I'm hopeful that this can be merged directly in to master, I build 180
>> packages in not that much time at all to test this change [1].
>>
>> 1: ./pre-inst-env guix package -s ruby- | recsel -P name | xargs ./pre-inst-env guix build
>
> Thank you for fixing it! Since this only affects gems, not ruby itself
> (which has ~900 dependencies), I think it can go on 'master' too[1].
>
> [1] guix refresh -l $(guix package -s ^ruby- | recsel -P name)

Great :)

Toggle quote (3 lines)
> By the way, Ruby 2.5 is out. Are you willing to try upgrading it for
> 'staging'? :-)

Sure :) I'll take a look.
-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAltLBIFfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE
9Xdc2RAAs4S14gS9/yb8Uhis1CdFlKDe3dz1i9SY8hTlCpRbm6vTplpPT/Jvrs33
K290UJsjjpolscJOWrw19jGRiqOlr7Zaf8KTyvJNnMbJdBO9ZSjuydP89XtyA/V3
V5ywPp44gdjzMfBgdx3aqBc71asrlfH/qAfAcmhWtK2lbXm3Y3s8lZEnYdtu3Mu1
aroKDXBnOXrUtcb6B4h34g708hjvPnx7AG5KGEInfIOAR/QsWAc0XddA/XOgBSpS
f1mrigI76KL0gaOF1epaEtLBzTRsaqmun4AGj4ZuF9wGuoUx7jgaHjUAPijK15m3
Cqgeb+j5IjG4JByLxZScizv2v46X4ivqzPcC3SxY1ffIv7EgR3EkZdzB4OCHDYOO
OD2Eyt8ucO09FbyIhEfk5C2ADA4jC2jKvVREySqvDDePjPYFNmM//6vn7bZKSllM
2NQa6H/+Uu64muBbXL6mnQ6NDg1xlOLFBqrNIo+2CYM63nt6G2udtt0IYlurvqyo
TidQ4+kG9oNp3JBqOcMsQ3EqjmHZo14+fjp2huqQ3l79kToULX7wrZNCUVJtQaOI
8d8Tzzp8OpOJ7BBiS/nOLMHdBejIcL7kpOJbvTEXv920VLcFItzsKfgP4PC2d87v
onXJC/76EhUE0J3ZfFAb14H+17DSNoeR6sT6zO9cG+MrkrAnA1M=
=k6mN
-----END PGP SIGNATURE-----

Christopher Baines wrote 7 years ago
Re: [bug#32153] [PATCH 1/2] ruby-build-system: Error or return #t from all phases.
(name . Marius Bakke)(address . mbakke@fastmail.com)(address . 32153-done@debbugs.gnu.org)
87tvp0dwbc.fsf@cbaines.net
Marius Bakke <mbakke@fastmail.com> writes:

Toggle quote (44 lines)
> Christopher Baines <mail@cbaines.net> writes:
>
>> Previously, if the tests didn't pass, the check phase would evaluate to #f,
>> but the package would be built sucessfully. This changes all the phases to
>> raise exceptions if errors are encountered, and return #t otherwise.
>>
>> This involves using invoke rather than system*, so that exceptions are raised
>> if the program exits with a status other than 0, and also returning #t at the
>> end of functions.
>>
>> * gnu/build/ruby-build-system.scm (unpack): Use invoke rather than system*,
>> and return #t at the end.
>> (build, check): Use invoke rather than system*.
>> (install): Remove the use of "and", and rewrite the error handling to raise an
>> exception.
>> (wrap): Return #t.
>
> Thanks! The changes LGTM. I will suggest a micro-improvement not
> related to this patch since it was there from before:
>
>> + ;; For gems with native extensions, several Makefile-related files
>> + ;; are created that contain timestamps or other elements making
>> + ;; them not reproducible. They are unnecessary so we remove them.
>> + (if (file-exists? (string-append vendor-dir "/ext"))
>> + (begin
>> + (for-each (lambda (file)
>> + (log-file-deletion file)
>> + (delete-file file))
>> + (append
>> + (find-files (string-append vendor-dir "/doc")
>> + "page-Makefile.ri")
>> + (find-files (string-append vendor-dir "/extensions")
>> + "gem_make.out")
>> + (find-files (string-append vendor-dir "/ext")
>> + "Makefile")))))
>> +
>> + #t))
>
> I was confused whether the #t was the "else" clause for the "if"
> expression until I realized it didn't have one.
>
> Could you turn this into a (when (file-exists? ...) (for-each ...))
> while at it? It also makes the (begin ...) redundant.

I've made this change, and pushed the patches to master, thanks again
for taking a look :)

Chris
-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAltLvCdfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE
9XfrBw/+PgYrXmDY7eYy0ndfHTqEk1McFX0CbdzH2db/LymQ3sglMcOGHg6seQKE
zKs6Cd3ePKPV5pivPMTtiOAHlnBBgspbeC+tJShnKH/Wy8p82KhdO0G+7sx+Z63R
/+njiHe6iGd+bB3W9iKSmF82e3yCuZZW7Kmr4Bcb5YtB/TEL0Rkg5NuT0YaAGTnC
BIiybF/GovXe2OurGEXWmKEzKVFGhP90UHq+rs/0YgyyBr4g2yb0ObIxQPcUjXe0
7PDSRI0qXS2YBXQxtLkJnYiM3/L0UN4j7FySIiA8375DdLsvRth82lneRSSh5rJj
H5JQUuC71+KHs10bfwXhkiVfvqbqcqUATkLGrz38g2DRXpYm302C+blIiXUKHLE4
fNr36nrSH5i2SPKNAlYozPOvl75uGt7EjxK9fdYNsQSyNUf5R7KBAHBm6Wd+CoMD
ikUCNwCswAZ7adB720ZdpLgoc7KTvLT6+DjuEaw8ABTggYSn2WK7h4jcK1giSAFw
m5YCKuPDv0rlN15GPUyciJcqOQmZa/Dkh9EmnrpQHWDk9W5OcU0S44ECINMso/W7
hZvf2GPLuSkyliP3hdpp+7G4CeEwSg3iPHmqrzir/cStwi9ASsCzlwG1rDKUymoe
3G19VHkkwmovmS7Gfmk9vsbHRfA1lHwyrokvJ4CEyF3aIOkskeQ=
=V+Ss
-----END PGP SIGNATURE-----

Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 32153@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 32153
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help