A number of corrupt packages on Hydra?

  • Done
  • quality assurance status badge
Details
5 participants
  • Ben Woodcroft
  • Christopher Allan Webber
  • Leo Famulari
  • Ludovic Courtès
  • Pjotr Prins
Owner
unassigned
Submitted by
Christopher Allan Webber
Severity
normal
C
C
Christopher Allan Webber wrote on 24 Mar 2016 00:48
(address . bug-guix@gnu.org)
87h9fwiwm5.fsf@dustycloud.org
For at least boost and hydra I'm getting errors like the following:

Downloading jaf2s3...-ruby-2.3.0 (41.9MiB installed)...
bzip2: Compressed file ends unexpectedly;
perhaps it is corrupted? *Possible* reason follows.
bzip2: Inappropriate ioctl for device
Input file = (stdin), output file = (stdout)
It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.
You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.
guix substitute: error: corrupt input while restoring '/gnu/store/jaf2s3paa3fswfpzi6amlqjxd3fwiw9x-ruby-2.3.0/lib/ruby/2.3.0/rdoc/generator/template/darkfish/fonts/SourceCodePro-Regular.ttf' from #{read pipe}#
killing process 8510
guix package: error: build failed: some substitutes for the outputs of derivation `/gnu/store/l7ffwqbkrvd4vk3h51p597psyl44gv67-ruby-2.3.0.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source
P
P
Pjotr Prins wrote on 24 Mar 2016 14:25
(name . Christopher Allan Webber)(address . cwebber@dustycloud.org)(address . 23103@debbugs.gnu.org)
20160324132559.GA16739@thebird.nl
I am also seeing such corruption on my substitutes server. Is there a way
we can validate files in a running cache so they can be rebuild? Would
be useful for the mirror.guixsd.org too.

I am glad they get picked up (even so) and that we have the --fallback
option :)

Pj.

On Wed, Mar 23, 2016 at 04:48:50PM -0700, Christopher Allan Webber wrote:
Toggle quote (23 lines)
> For at least boost and hydra I'm getting errors like the following:
>
> From http://hydra.gnunet.org/nar/jaf2s3paa3fswfpzi6amlqjxd3fwiw9x-ruby-2.3.0
> Downloading jaf2s3...-ruby-2.3.0 (41.9MiB installed)...
> http://hydra.gnunet.org/nar/jaf2s3paa3fswfpzi6amlqjxd3fwiw9x-ruby-2.3.0 2.0MiB/s 00:02 | 3.4MiB transferredd
> bzip2: Compressed file ends unexpectedly;
> perhaps it is corrupted? *Possible* reason follows.
> bzip2: Inappropriate ioctl for device
> Input file = (stdin), output file = (stdout)
>
> It is possible that the compressed file(s) have become corrupted.
> You can use the -tvv option to test integrity of such files.
>
> You can use the `bzip2recover' program to attempt to recover
> data from undamaged sections of corrupted files.
>
> guix substitute: error: corrupt input while restoring '/gnu/store/jaf2s3paa3fswfpzi6amlqjxd3fwiw9x-ruby-2.3.0/lib/ruby/2.3.0/rdoc/generator/template/darkfish/fonts/SourceCodePro-Regular.ttf' from #{read pipe}#
> killing process 8510
> guix package: error: build failed: some substitutes for the outputs of derivation `/gnu/store/l7ffwqbkrvd4vk3h51p597psyl44gv67-ruby-2.3.0.drv' failed (usually happens due to networking issues); try `--fallback' to build derivation from source
>
>
>

--
P
L
L
Ludovic Courtès wrote on 25 Mar 2016 14:47
(name . Pjotr Prins)(address . pjotr.public12@thebird.nl)
877fgqoejb.fsf@gnu.org
Pjotr Prins <pjotr.public12@thebird.nl> skribis:

Toggle quote (4 lines)
> bzip2: Compressed file ends unexpectedly;
> perhaps it is corrupted? *Possible* reason follows.

This one should be fixed now.

I’m considering about removing all the cached items that could
potentially be corrupt (those that were transferred before we forced the
use of chunked transfer encoding, in commit
ea01e13f185b6e02c795b3c162c9a551162e4c9d of guix-maintenance.git, March
22nd) but that would cause 7.3 GiB of transfers. Needs more thought.

I’ll followup on guix-sysadmin.

Ludo’.
C
Closed
L
L
Ludovic Courtès wrote on 27 Mar 2016 23:22
(address . 23103@debbugs.gnu.org)(address . cwebber@dustycloud.org)
87y493wr9g.fsf@gnu.org
Christopher Allan Webber <cwebber@dustycloud.org> skribis:

Toggle quote (4 lines)
> Allegedly, Ludo has fixed this, or at least
> https://lists.gnu.org/archive/html/guix-devel/2016-03/msg01039.html
> makes it sound like it :)

Right. :-) AIUI, in its current config nginx is not support to cache
truncated items, but Mark thinks it’s already done so in the past.

So if that pops up again, do let us know!

Ludo’.
L
L
Leo Famulari wrote on 29 Mar 2016 23:23
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 23103@debbugs.gnu.org)
20160329212319.GA844@jasmine
On Sun, Mar 27, 2016 at 11:22:03PM +0200, Ludovic Courtès wrote:
Toggle quote (11 lines)
> Christopher Allan Webber <cwebber@dustycloud.org> skribis:
>
> > Allegedly, Ludo has fixed this, or at least
> > https://lists.gnu.org/archive/html/guix-devel/2016-03/msg01039.html
> > makes it sound like it :)
>
> Right. :-) AIUI, in its current config nginx is not support to cache
> truncated items, but Mark thinks it’s already done so in the past.
>
> So if that pops up again, do let us know!

IRC user 'ziz15' had problems downloading these archives while
initializing a new system:

/gnu/store/kcc3cxnx9l2hbg7pjhxsa0r5ypq2j2f38-python-2.7.10
/gnu/store/klb2s6r0f6cgpns65gkjwd5m8ygw73wj-wicd-1.7.3.drv
P
P
Pjotr Prins wrote on 3 Apr 2016 10:20
(name . Leo Famulari)(address . leo@famulari.name)
20160403082000.GA24869@thebird.nl
bzip2: Compressed file ends unexpectedly;
perhaps it is corrupted? *Possible* reason follows.
bzip2: Inappropriate ioctl for device
Input file = (stdin), output file = (stdout)

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

guix substitute: error: corrupt input while restoring '/gnu/store/6kvy3ryb04nl49wwdy0dmhhfnfbwrmna-tcl-8.6.4/lib/tcl8.6/encoding/cp936.enc' from #{read pipe}#
killing process 55815
B
B
Ben Woodcroft wrote on 3 Apr 2016 12:55
(address . 23103@debbugs.gnu.org)
5700F6BF.7090104@uq.edu.au
On 03/04/16 18:20, Pjotr Prins wrote:
Toggle quote (5 lines)
> bzip2: Compressed file ends unexpectedly;
> perhaps it is corrupted? *Possible* reason follows.
> bzip2: Inappropriate ioctl for device
> Input file = (stdin), output file = (stdout)
I think this only is a problem on the mirror, substituting from hydra
itself works for me. You?

ben
L
L
Leo Famulari wrote on 3 Apr 2016 20:16
(name . Ben Woodcroft)(address . b.woodcroft@uq.edu.au)
20160403181611.GA9334@jasmine
On Sun, Apr 03, 2016 at 08:55:59PM +1000, Ben Woodcroft wrote:
Toggle quote (11 lines)
>
>
> On 03/04/16 18:20, Pjotr Prins wrote:
> > http://mirror.guixsd.org/nar/6kvy3ryb04nl49wwdy0dmhhfnfbwrmna-tcl-8.6.4 1017KiB/s 00:02 | 1.7MiB transferred
> >bzip2: Compressed file ends unexpectedly;
> > perhaps it is corrupted? *Possible* reason follows.
> >bzip2: Inappropriate ioctl for device
> > Input file = (stdin), output file = (stdout)
> I think this only is a problem on the mirror, substituting from hydra itself
> works for me. You?

Most of these recent corrupt archives problems are due to corruptions in
the mirror's cache — the mirror will only try (and possibly fail) to
pull an archive from hydra.gnu.org if hydra has built it successfully.

So, passing --substitute-urls=https://hydra.gnu.orgor --fallback are
both reasonable ways to work around the failure.

Please keep reporting these!
L
P
P
Pjotr Prins wrote on 4 Apr 2016 17:14
(name . Pjotr Prins)(address . pjotr.public12@thebird.nl)
20160404151433.GA31308@thebird.nl
Eeep

bzip2: Compressed file ends unexpectedly;
perhaps it is corrupted? *Possible* reason follows.
bzip2: Inappropriate ioctl for device
Input file = (stdin), output file = (stdout)

It is possible that the compressed file(s) have become corrupted.
You can use the -tvv option to test integrity of such files.

You can use the `bzip2recover' program to attempt to recover
data from undamaged sections of corrupted files.

guix substitute: error: corrupt input while restoring '/gnu/store/kcc3cxnx9l2hbg7pjhxsa0r5yq2j2f38-python-2.7.10/lib/python2.7/email/test/test_email_renamed.pyc' from #{read pipe}#
killing process 17595


On Sun, Apr 03, 2016 at 10:20:00AM +0200, Pjotr Prins wrote:
Toggle quote (19 lines)
> bzip2: Compressed file ends unexpectedly;
> perhaps it is corrupted? *Possible* reason follows.
> bzip2: Inappropriate ioctl for device
> Input file = (stdin), output file = (stdout)
>
> It is possible that the compressed file(s) have become corrupted.
> You can use the -tvv option to test integrity of such files.
>
> You can use the `bzip2recover' program to attempt to recover
> data from undamaged sections of corrupted files.
>
> guix substitute: error: corrupt input while restoring '/gnu/store/6kvy3ryb04nl49wwdy0dmhhfnfbwrmna-tcl-8.6.4/lib/tcl8.6/encoding/cp936.enc' from #{read pipe}#
> killing process 55815
>
>
>
>

--
L
?
Your comment

This issue is archived.

To comment on this conversation send an email to 23103@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 23103
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch