core-update: Raw origin tarballs are not handled correctly.

OpenSubmitted by Mathieu Othacehe.
Details
2 participants
  • Maxim Cournoyer
  • Mathieu Othacehe
Owner
unassigned
Severity
important
M
M
Mathieu Othacehe wrote on 15 Aug 10:42 +0200
(address . bug-guix@gnu.org)(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87bl5z85ga.fsf@gnu.org
Hello,
When building "emacs-org-contrib" on core-updates-frozen, the followingmessage is printed:
Toggle snippet (12 lines)tar: Exiting with failure status due to previous errorserror: in phase 'unpack': uncaught exception:%exception #<&invoke-error program: "tar" arguments: ("xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar") exit-status: 2 term-signal: #f stop-signal: #f> phase `unpack' failed after 0.0 secondscommand "tar" "xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar" failed with status 2note: keeping build directory `/tmp/guix-build-emacs-org-contrib-20210809.drv-6'builder for `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed with exit code 1build of /gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv failedView build log at '/var/log/guix/drvs/52/jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv.bz2'.guix build: error: build of `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed
That's because"/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar"is empty. I believe this is a consequence of:cfcead2e515c0dae02127e5a76496463898be6b6, and in particular thissection:
Toggle snippet (12 lines) ;; If SOURCE is a directory (such as a checkout), return a ;; directory. Otherwise create a tarball. (cond ((file-is-directory? #+source) (copy-recursively directory #$output #:log (%make-void-port "w"))) ((not #+comp) (copy-file file #$output)) <---------- We should call repack instead here for raw tarballs? (else (repack directory #$output)))))))
Maxim, could you please have a look?
Thanks,
Mathieu
M
M
Maxim Cournoyer wrote on 27 Aug 06:19 +0200
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . bug-guix@gnu.org)
87lf4nsene.fsf@gmail.com
Hello Mathieu,
Mathieu Othacehe <othacehe@gnu.org> writes:
Toggle quote (36 lines)> Hello,>> When building "emacs-org-contrib" on core-updates-frozen, the following> message is printed:>> tar: Exiting with failure status due to previous errors> error: in phase 'unpack': uncaught exception:> %exception #<&invoke-error program: "tar" arguments: ("xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar") exit-status: 2 term-signal: #f stop-signal: #f> > phase `unpack' failed after 0.0 seconds> command "tar" "xvf" "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar" failed with status 2> note: keeping build directory `/tmp/guix-build-emacs-org-contrib-20210809.drv-6'> builder for `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed with exit code 1> build of /gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv failed> View build log at '/var/log/guix/drvs/52/jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv.bz2'.> guix build: error: build of `/gnu/store/52jcx6rwnf0wrkrpcwkyz766fzg6asjy-emacs-org-contrib-20210809.drv' failed>>> That's because> "/gnu/store/r3v4gkqqbxp2w739a0llvk9nrda9xn89-org-plus-contrib-20210809.tar"> is empty. I believe this is a consequence of:> cfcead2e515c0dae02127e5a76496463898be6b6, and in particular this> section:>> ;; If SOURCE is a directory (such as a checkout), return a> ;; directory. Otherwise create a tarball.> (cond> ((file-is-directory? #+source)> (copy-recursively directory #$output> #:log (%make-void-port "w")))> ((not #+comp)> (copy-file file #$output)) <---------- We should call repack instead here for raw tarballs?> (else> (repack directory #$output)))))))>> Maxim, could you please have a look?
Thanks for the report; it looks like a bug indeed. I had probablyoverlooked that case (where there is no compression but we are stilldealing with an archive format). The annoying thing is that touchingthis code will trigger a world rebuild; so unless we find other bigproblems and batch the fixes, it may not be worth it.
The site.py problem you reported may be one, if I find a solution.
Other things on my mind:
1. rustc bootstrap from 1.39 (the itch to make it faster comes backeverytime I have to build it ;-)).
2. fontconfig update that should allow per-profile fonts management via XDG_DATA_DIRS (I have the commit ready, but failed to test it due to having to build rust and getting distracted by 1. :-))
Not a very useful reply in itself, but I felt that communicating whatI've been busy with was better than a longer silence :-).
Thank you,
Maxim
M
M
Maxim Cournoyer wrote on 27 Aug 19:03 +0200
control message for bug #50066
(address . control@debbugs.gnu.org)
87k0k6stuw.fsf@gmail.com
severity 50066 importantquit
M
M
Maxim Cournoyer wrote on 31 Aug 05:50 +0200
Re: bug#50066: core-update: Raw origin tarballs are not handled correctly.
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 50066@debbugs.gnu.org)
874kb6jmrn.fsf@gmail.com
Hello Mathieu,
[...]
Toggle quote (13 lines)> Thanks for the report; it looks like a bug indeed. I had probably> overlooked that case (where there is no compression but we are still> dealing with an archive format). The annoying thing is that touching> this code will trigger a world rebuild; so unless we find other big> problems and batch the fixes, it may not be worth it.>> The site.py problem you reported may be one, if I find a solution.>> Other things on my mind:>> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back> everytime I have to build it ;-)).
I'm nearing completion on this front.
Toggle quote (4 lines)> 2. fontconfig update that should allow per-profile fonts management via> XDG_DATA_DIRS (I have the commit ready, but failed to test it due to> having to build rust and getting distracted by 1. :-))
I'll propose a few patches to include in the 'frozen' core-updatesbranch since we've found a world-rebuilding change worthy of addressinganyway (the patch below).
1 file changed, 4 insertions(+), 4 deletions(-)guix/packages.scm | 8 ++++----
modified guix/packages.scm@@ -836,10 +836,10 @@ specifies modules in scope when evaluating SNIPPET." ((file-is-directory? #+source) (copy-recursively directory #$output #:log (%make-void-port "w")))- ((not #+comp)- (copy-file file #$output))- (else- (repack directory #$output)))))))+ ((or #+comp (tarball? #+source))+ (repack directory #$output))+ (else ;single uncompressed file+ (copy-file file #$output))))))) (let ((name (if (or (checkout? original-file-name) (not (compressor original-file-name)))
Sadly the plain .tar file was/is not covered by tests (and not easilyso), but if his was my only omission, it should be correct (TM).Attentive review needed :-).
Thank you!
Maxim
M
M
Maxim Cournoyer wrote on 31 Aug 15:47 +0200
control message for bug #50066
(address . control@debbugs.gnu.org)
87v93liv4s.fsf@gmail.com
tags 50066 + patchquit
M
M
Mathieu Othacehe wrote on 1 Sep 09:09 +0200
Re: bug#50066: core-update: Raw origin tarballs are not handled correctly.
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 50066@debbugs.gnu.org)
87lf4gvkkd.fsf@gnu.org
Hello Maxim,
Toggle quote (5 lines)>> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back>> everytime I have to build it ;-)).>> I'm nearing completion on this front.
That will be a welcomed improvement!
Toggle quote (8 lines)>> 2. fontconfig update that should allow per-profile fonts management via>> XDG_DATA_DIRS (I have the commit ready, but failed to test it due to>> having to build rust and getting distracted by 1. :-))>> I'll propose a few patches to include in the 'frozen' core-updates> branch since we've found a world-rebuilding change worthy of addressing> anyway (the patch below).
Great! As we are going for a full rebuild, maybe we should also considersome of the Thiago fixes targeting core-updates-frozen such ashttps://issues.guix.gnu.org/50239.
Toggle quote (4 lines)> Sadly the plain .tar file was/is not covered by tests (and not easily> so), but if his was my only omission, it should be correct (TM).> Attentive review needed :-).
This seems reasonable and I'm currently rebuilding the world to testthis fix and the Python fix. I should maybe apply your Rust bootstrappatch if you have it around to gain some time.
Thanks for all those fixes!
Mathieu
M
M
Maxim Cournoyer wrote on 1 Sep 14:57 +0200
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 50066@debbugs.gnu.org)
87y28gcv2j.fsf@gmail.com
Hi Mathieu,
Mathieu Othacehe <othacehe@gnu.org> writes:
Toggle quote (21 lines)> Hello Maxim,>>>> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back>>> everytime I have to build it ;-)).>>>> I'm nearing completion on this front.>> That will be a welcomed improvement!>>>> 2. fontconfig update that should allow per-profile fonts management via>>> XDG_DATA_DIRS (I have the commit ready, but failed to test it due to>>> having to build rust and getting distracted by 1. :-))>>>> I'll propose a few patches to include in the 'frozen' core-updates>> branch since we've found a world-rebuilding change worthy of addressing>> anyway (the patch below).>> Great! As we are going for a full rebuild, maybe we should also consider> some of the Thiago fixes targeting core-updates-frozen such as> https://issues.guix.gnu.org/50239.
Yes, I think we should cherry-pick anything valuable before the bigrebuild of the frozen branch (perhaps we should simply sync withcore-updates?)
Toggle quote (8 lines)>> Sadly the plain .tar file was/is not covered by tests (and not easily>> so), but if his was my only omission, it should be correct (TM).>> Attentive review needed :-).>> This seems reasonable and I'm currently rebuilding the world to test> this fix and the Python fix. I should maybe apply your Rust bootstrap> patch if you have it around to gain some time.
It's currently building Rust 1.47 (it takes about 15 minutes per Rust).I'll build it all until 1.54, then recheck a few ones forreproducibility (at least the first and last ones). If all is good I'llsend the patches I have.
Toggle quote (2 lines)> Thanks for all those fixes!
Thank you for the testing/reports!
Maxim
M
M
Maxim Cournoyer wrote on 3 Sep 15:30 +0200
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 50066@debbugs.gnu.org)
877dfxdbwq.fsf@gmail.com
Hi Mathieu,
Mathieu Othacehe <othacehe@gnu.org> writes:
Toggle quote (29 lines)> Hello Maxim,>>>> 1. rustc bootstrap from 1.39 (the itch to make it faster comes back>>> everytime I have to build it ;-)).>>>> I'm nearing completion on this front.>> That will be a welcomed improvement!>>>> 2. fontconfig update that should allow per-profile fonts management via>>> XDG_DATA_DIRS (I have the commit ready, but failed to test it due to>>> having to build rust and getting distracted by 1. :-))>>>> I'll propose a few patches to include in the 'frozen' core-updates>> branch since we've found a world-rebuilding change worthy of addressing>> anyway (the patch below).>> Great! As we are going for a full rebuild, maybe we should also consider> some of the Thiago fixes targeting core-updates-frozen such as> https://issues.guix.gnu.org/50239.>>> Sadly the plain .tar file was/is not covered by tests (and not easily>> so), but if his was my only omission, it should be correct (TM).>> Attentive review needed :-).>> This seems reasonable and I'm currently rebuilding the world to test> this fix and the Python fix. I should maybe apply your Rust bootstrap> patch if you have it around to gain some time.
successfully built /gnu/store/vag2a53byhgrkpchviqz78lilall6412-emacs-org-contrib-20210519.drv/gnu/store/40hl0dli745i2ggs1aqhn2drh0fkmrbx-emacs-org-contrib-20210519
:-).
I'm trying to fix one last thing with my local patch set (mozjs notbuilding with rust@1.54), but if takes too much time, I'll send what Ihave and we can figure it out together.
Thanks,
Maxim
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 50066@debbugs.gnu.org