linux-libre-3.3.8-gnu disappeared

  • Done
  • quality assurance status badge
Details
5 participants
  • Andreas Enge
  • ng0
  • Jason Self
  • Ludovic Courtès
  • Alexandre Oliva
Owner
unassigned
Submitted by
Andreas Enge
Severity
normal

Debbugs page

Andreas Enge wrote 12 years ago
(address . bug-guix@gnu.org)
201307121302.05303.andreas@enge.fr
The build of linux-libre-headers-3.3.8 currently fails because the tarball
cannot be downloaded any more from
Instead, there is a new version in

Should we switch to this one? Or update to the newest version? In all
cases, I am afraid a complete rebuild will be triggered, so we might as
well advance in the version numbers.

Andreas
Ludovic Courtès wrote 12 years ago
(address . 14851@debbugs.gnu.org)
87zjtrfm39.fsf@gnu.org
Hello,

(Cc: Alexandre Oliva of Linux-Libre.)

Andreas Enge <andreas@enge.fr> skribis:

Toggle quote (6 lines)
> The build of linux-libre-headers-3.3.8 currently fails because the tarball
> cannot be downloaded any more from
> http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu/
> Instead, there is a new version in
> http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu1/

I just checked and the latter has a different SHA256
(1860as3dwh7f3im1sq3x06awmil9vppl6igg2gnva5w9mbkw2fc8).

Toggle quote (4 lines)
> Should we switch to this one? Or update to the newest version? In all
> cases, I am afraid a complete rebuild will be triggered, so we might as
> well advance in the version numbers.

Indeed.

Since we’re about to release a new version of Guix, I’d rather keep
using 3.3.8.

Alexandre: could you reinstate the original

It would be ideal if the tarballs were on ftp.gnu.org. I could do it if
you don’t want to bother, provided the FTP admins allow it. WDYT?

Thanks,
Ludo’.
Ludovic Courtès wrote 12 years ago
(name . Jason Self)(address . jason@bluehome.net)
87vc4ddz50.fsf@gnu.org
"Jason Self" <jason@bluehome.net> skribis:

Toggle quote (5 lines)
> My understanding is that the change from gnu to gnu1 is the result of
> changes that were needed to the deblobbing. Specifically to allow the
> loading of the firmware used by ath9k-htc, now that it's free [0]...
> i.e. 3.3.8-gnu and 3.3.8-gnu1 aren't the same source code.

OK, thanks for the info.

It’s hard for us to deal with disappearing software. In this case,
we’re currently using this tarball just to install the Linux-Libre
headers to build libc, and our ‘linux-libre’ package can use a different
upstream version than ‘linux-libre-headers’ anyway.

Thanks,
Ludo’.
Jason Self wrote 12 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)
1373776087.18732@bluehome.net
My understanding is that the change from gnu to gnu1 is the result of
changes that were needed to the deblobbing. Specifically to allow the
loading of the firmware used by ath9k-htc, now that it's free [0]...
i.e. 3.3.8-gnu and 3.3.8-gnu1 aren't the same source code.

[0]
Ludovic Courtès wrote 12 years ago
(address . 14851@debbugs.gnu.org)
87k3kps475.fsf@gnu.org
I’ve asked ftp-upload@ if they could allow me to upload linux-libre
tarballs to ftp.gnu.org.

In the meantime, I’ve uploaded this tarball to
ftp://alpha.gnu.org/gnu/guix/mirror and changed linux.scm to refer to it.

Thanks,
Ludo’.
Ludovic Courtès wrote 12 years ago
(name . Alexandre Oliva)(address . lxoliva@fsfla.org)
8761w6hhtq.fsf@gnu.org
Alexandre Oliva <lxoliva@fsfla.org> skribis:

Toggle quote (11 lines)
> On Jul 12, 2013, ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Since we’re about to release a new version of Guix, I’d rather keep
>> using 3.3.8.
>
>> Alexandre: could you reinstate the original
>> http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu/linux-libre-3.3.8-gnu.tar.xz?
>
> I suppose you don't want to prevent users of guix from using ath9k wifi
> cards, so I strongly suggest switching to 3.3.8-gnu1.

Of course not, but again, this one is used to get headers against which
to build glibc, so that’s not a problem.

Toggle quote (3 lines)
> Indeed, I think you'd be better off with some LTS version of GNU
> Linux-libre, rather than the dead 3.3 branch. But that's your call.

When we have a stand-alone, bootable distro, we’ll certainly want to
synchronize with you for the choice of the default kernel version.

Toggle quote (5 lines)
>> It would be ideal if the tarballs were on ftp.gnu.org. I could do it if
>> you don’t want to bother, provided the FTP admins allow it. WDYT?
>
> I'd be glad with such an arrangement.

Great.

Toggle quote (4 lines)
> Now, another possibility that I think would make more sense for guix is
> to have its sources consolidated in a single place, rather than
> scattered all over and at risk of having them pulled from under you.

Actually, our continuous integration server at http://hydra.gnu.orgdoes
that transparently: it caches all the source tarballs, along with build
byproducts.

So when a tarball vanishes from its upstream site, it’s usually not a
blocking problem. Yet, it’s preferable to have them elsewhere, because
they will eventually be garbage-collected from hydra.gnu.org.

[...]

Toggle quote (11 lines)
> When we get GNU Linux-libre at ftp.gnu.org, it could then be hard links,
> so that if we remove some tarball it won't go away from your “copy”, but
> until then, you might be better off holding your own copy rather than
> assuming our primary repository has infinite space. Unfortunately it
> doesn't, and I have to clean things up quite often. For sources, I at
> least keep enough bits around that the tarballs can be reconstructed in
> a bit-exact fashion, but for binaries, when they're gone, they're gone
> forever. However, considering we put out multiple GBs of builds per
> week, I don't think it's realistic to keep them all forever. Not in our
> own server, not at ftp.gnu.org.

What do you mean by “multiple GBs of builds per week”? Linux{,-Libre}
releases are not that frequent, are they?

The policy at ftp.gnu.org has always been to keep everything forever,
AFAIK.

If size turns out to be a problem, we could choose to keep only LTS
releases on ftp.gnu.org, for instance. That’s something to discuss
with the GNU sysadmins.

Thanks for your feedback!

Ludo’.
Alexandre Oliva wrote 12 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)
orzjtin6x0.fsf@livre.home
On Jul 12, 2013, ludo@gnu.org (Ludovic Courtès) wrote:

Toggle quote (6 lines)
> Since we’re about to release a new version of Guix, I’d rather keep
> using 3.3.8.

> Alexandre: could you reinstate the original
> http://linux-libre.fsfla.org/pub/linux-libre/releases/3.3.8-gnu/linux-libre-3.3.8-gnu.tar.xz?

I suppose you don't want to prevent users of guix from using ath9k wifi
cards, so I strongly suggest switching to 3.3.8-gnu1. Indeed, I think
you'd be better off with some LTS version of GNU Linux-libre, rather
than the dead 3.3 branch. But that's your call.

Toggle quote (3 lines)
> It would be ideal if the tarballs were on ftp.gnu.org. I could do it if
> you don’t want to bother, provided the FTP admins allow it. WDYT?

I'd be glad with such an arrangement. I keep on failing to figure out
how to fit the weekly publishing of multiple releases into a workflow
that includes collecting information and sending it to ftp.gnu.org
without keeping another local copy of stuff that was published before.
That's one of the hold-up factors for me. As for the tarballs, they're
all signed, so figuring out a way to upload just the bits created since
the last push, and pushing them to live, is what's missing.

Now, another possibility that I think would make more sense for guix is
to have its sources consolidated in a single place, rather than
scattered all over and at risk of having them pulled from under you. At
the very least, you ought to keep a copy of sources you use to build
binaries you publish, so that you can satisfy your obligation to offer
the corresponding source, be it a legal (copyleft) or moral (software in
gnu ought to be free) obligation.

When we get GNU Linux-libre at ftp.gnu.org, it could then be hard links,
so that if we remove some tarball it won't go away from your “copy”, but
until then, you might be better off holding your own copy rather than
assuming our primary repository has infinite space. Unfortunately it
doesn't, and I have to clean things up quite often. For sources, I at
least keep enough bits around that the tarballs can be reconstructed in
a bit-exact fashion, but for binaries, when they're gone, they're gone
forever. However, considering we put out multiple GBs of builds per
week, I don't think it's realistic to keep them all forever. Not in our
own server, not at ftp.gnu.org.

--
Alexandre Oliva, freedom fighter http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/ FSF Latin America board member
Free Software Evangelist Red Hat Brazil Compiler Engineer
Andreas Enge wrote 12 years ago
(name . Alexandre Oliva)(address . lxoliva@fsfla.org)
201307191851.47553.andreas@enge.fr
Am Freitag, 19. Juli 2013 schrieb Alexandre Oliva:
Toggle quote (4 lines)
> However, considering we put out multiple GBs of builds per
> week, I don't think it's realistic to keep them all forever. Not in our
> own server, not at ftp.gnu.org.

As far as I know, ftp.gnu.org would only be concerned with the source, that
is, the content of
and not with binary packages in upper level directories, and these source
tarballs are usually kept indefinitely.

One also need not add the patch and diff files, so that the amount of
storage would be quite manageable.

Andreas
Alexandre Oliva wrote 12 years ago
(name . Andreas Enge)(address . andreas@enge.fr)
orfvvacp8z.fsf@livre.home
On Jul 19, 2013, Andreas Enge <andreas@enge.fr> wrote:

Toggle quote (7 lines)
> Am Freitag, 19. Juli 2013 schrieb Alexandre Oliva:
>> However, considering we put out multiple GBs of builds per
>> week, I don't think it's realistic to keep them all forever. Not in our
>> own server, not at ftp.gnu.org.

> As far as I know, ftp.gnu.org would only be concerned with the source,

I don't know about that. I've seen binaries in ftp.gnu.org in the past.
This should indeed reduce significantly the space requirements to a bit
less than 1GB per week, considering 4 releases per week in 3 different
compression formats.

OTOH, if we're to be strict, we should publish only the deblob scripts,
for those are the ultimate sources produced by the GNU Linux-libre
project. The tarballs and diffs and deltas are just the result of
running those scripts on tarballs produced by third parties.

The scripts would be trivial to retain indefinitely; even more so
because they seldom change for stable releases.

Toggle quote (5 lines)
> and these source tarballs are usually kept indefinitely.

> One also need not add the patch and diff files, so that the amount of
> storage would be quite manageable.

The diff files are probably no big deal (10% increase, not an order of
magnitude like the build tarballs and debuginfo rpms). Excluding them
would probably make for more work in the upload script, though.

--
Alexandre Oliva, freedom fighter http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/ FSF Latin America board member
Free Software Evangelist Red Hat Brazil Compiler Engineer
Alexandre Oliva wrote 12 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)
orbo5ycosg.fsf@livre.home
On Jul 19, 2013, ludo@gnu.org (Ludovic Courtès) wrote:

Toggle quote (3 lines)
> What do you mean by “multiple GBs of builds per week”? Linux{,-Libre}
> releases are not that frequent, are they?

They are. ATM there are 4 active stable releases that each get one new
release per week. There are 3 other LTS stable branches that get
releases less often. And then, there are the weekly development -rcs
for the next release, that I seldom start tracking long before the
release is out. This is just for sources, and it adds up to almost 1GB
per week.

Adding binaries to the picture grows the entire size by a little bit
when it comes to the freesh and planet .debs, and to a larger extent
when it comes to gnewsense/yeeloong (that gets one build per source
release we put out, with a total deb+tar size about the same size of the
source release subdir), and the freed-ora builds (that involve 1-4 rpm
builds per active Fedora release per week, each one weighting some 2GB
total; for most of the time there are 2 or 3 active releases; sometimes
there are 4, depending on whether or not I've already cleaned up the -rc
series for the upcoming release that gets built for the rawhide tree).

Toggle quote (3 lines)
> If size turns out to be a problem, we could choose to keep only LTS
> releases on ftp.gnu.org, for instance.

Keeping only LTS wouldn't help guix, though, since you're not using an
LTS release.

--
Alexandre Oliva, freedom fighter http://FSFLA.org/~lxoliva/
You must be the change you wish to see in the world. -- Gandhi
Be Free! -- http://FSFLA.org/ FSF Latin America board member
Free Software Evangelist Red Hat Brazil Compiler Engineer
Ludovic Courtès wrote 12 years ago
(name . Alexandre Oliva)(address . lxoliva@fsfla.org)
8761w6uwat.fsf@gnu.org
Alexandre Oliva <lxoliva@fsfla.org> skribis:

Toggle quote (12 lines)
> On Jul 19, 2013, ludo@gnu.org (Ludovic Courtès) wrote:
>
>> What do you mean by “multiple GBs of builds per week”? Linux{,-Libre}
>> releases are not that frequent, are they?
>
> They are. ATM there are 4 active stable releases that each get one new
> release per week. There are 3 other LTS stable branches that get
> releases less often. And then, there are the weekly development -rcs
> for the next release, that I seldom start tracking long before the
> release is out. This is just for sources, and it adds up to almost 1GB
> per week.

Ah OK, I wasn’t aware of that.

Toggle quote (2 lines)
> Adding binaries to the picture grows the entire size by a little bit

Yes, but binaries is not what we’re interested in here. :-)

Toggle quote (6 lines)
>> If size turns out to be a problem, we could choose to keep only LTS
>> releases on ftp.gnu.org, for instance.
>
> Keeping only LTS wouldn't help guix, though, since you're not using an
> LTS release.

Heh but that’s not set in stone, and we don’t use Linux-Libre for
anything beyond building libc.

That’ll change in the near future though.

Thanks,
Ludo’.
ng0 wrote 8 years ago
#14851 - linux-libre-3.3.8-gnu disappeared
(address . 14851@debbugs.gnu.org)
20170403105323.qorgkh3kdfg37vqo@abyayala
Andreas, Ludovic, and others:

This bug report was last modified 3 years and 258 days ago.

It is my understanding that this bug could be closed. I see no more
current problems and it makes no sense to keep the bug open just in case
the problem appears again.

Would you agree?
Ludovic Courtès wrote 8 years ago
(address . 14851-done@debbugs.gnu.org)
87y3ub16ac.fsf@gnu.org
ng0 <contact.ng0@cryptolab.net> skribis:

Toggle quote (10 lines)
> Andreas, Ludovic, and others:
>
> This bug report was last modified 3 years and 258 days ago.
>
> It is my understanding that this bug could be closed. I see no more
> current problems and it makes no sense to keep the bug open just in case
> the problem appears again.
>
> Would you agree?

I do! Especially with the content-addressed mirror we’ve been hosting
with ‘guix publish’.

Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 14851@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 14851
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help