source tarballs potentially built for each derivation

  • Open
  • quality assurance status badge
Details
One participant
  • Vagrant Cascadian
Owner
unassigned
Submitted by
Vagrant Cascadian
Severity
normal
V
V
Vagrant Cascadian wrote on 1 Aug 2021 22:45
(address . bug-guix@gnu.org)(name . guix-devel)(address . guix-devel@gnu.org)
87o8ag6gi2.fsf@yucca
Turning this conversation into a bug, original thread around here:


On 2021-05-29, Vagrant Cascadian wrote:
Toggle quote (38 lines)
> On 2021-05-01, Leo Famulari wrote:
>> On Sat, May 01, 2021 at 06:45:32PM -0700, Vagrant Cascadian wrote:
>>> Pragmatically speaking, on slower platforms this is a huge resource
>>> overhead. So much so that ci.guix.gnu.org *usually* times out when
>>> generating the linux-libre aarch64 tarballs:
>>>
>>> https://ci.guix.gnu.org/search?query=system%3Aaarch64-linux+linux-libre-arm64-generic
>>
>> Thanks for letting me know. I didn't know this was happening.
>>
>> The immediate solution is for me to make sure the tarballs have built
>> before committing the updates. I already do this for x86_64 and I can
>> start doing it for aarch64 too.
>
> This has definitely helped sometimes, thanks! I even saw a substitute of
> linux-libre for aarch64 earlier today! :)
>
> Still, I'm noticing another problem with the way way the tarballs are
> generated on ci.guix.gnu.org ...
>
> When it generates a tarball, all the various packages independently try
> to recreate the source tarball; so you have at least fours jobs
> ("linux-libre", "linux-libre-arm64-generic", "linux-libre-headers",
> "linux-libre-bpf") all concurrently trying to build the very same
> very-slow-to-build tarball on ci.guix.gnu.org. Sometimes one of them
> might succeed, but the others may not, and even though one of them
> succeeded, none of the failing ones retry...
>
> Not knowing exactly how ci.guix.gnu.org works, would it make sense to
> create a tarball package instead of the ... computed origin(?) tarball,
> so it could be better represented in the package dependency graph, and
> the various linux-libre-* packages can wait till it is available rather
> than all trying to recreate the same thing?
>
> That still requires the tarball generation to not time out in the first
> place, but maybe it would help with the resource limitations a bit to
> only build the source tarball once per architecture?

This seems to still be an issue for ci.guix.gnu.org, but the
linux-libre* substitutes for aarch64 seem to be available on

$ guix weather linux-libre linux-libre-arm64-generic
computing 2 package derivations for aarch64-linux...
looking for 2 store items on https://ci.guix.gnu.org...
0.0% substitutes available (0 out of 2)
unknown substitute sizes
0.0 MiB on disk (uncompressed)
0.740 seconds per request (0.7 seconds in total)
1.4 requests per second

0.0% (0 out of 2) of the missing items are queued
1 queued builds
aarch64-linux: 1 (100.0%)
build rate: .00 builds per hour
x86_64-linux: 0.00 builds per hour
aarch64-linux: 0.00 builds per hour
i686-linux: 0.00 builds per hour
looking for 2 store items on https://bordeaux.guix.gnu.org...
100.0% substitutes available (2 out of 2)
83.9 MiB of nars (compressed)
202.2 MiB on disk (uncompressed)
(continuous integration information unavailable)


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYQcH9gAKCRDcUY/If5cW
qp1LAP9eMIp0x8ygd5rht//VE9o8QBYm4cXec2LBxwOevDVwKwEAn10lZqhwXTPH
1XPirQBwEbLSI5V0P2x8DWY+PwgGIAU=
=uCAx
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 1 Aug 2021 23:04
(address . bug-guix@gnu.org)(name . guix-devel)(address . guix-devel@gnu.org)
87im0o6fm3.fsf@yucca
On 2021-08-01, Vagrant Cascadian wrote:
Toggle quote (4 lines)
> Turning this conversation into a bug, original thread around here:
>
> https://lists.gnu.org/archive/html/guix-devel/2021-05/threads.html#00427

For reference:

bug#49810: source tarballs potentially built for each derivation


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYQcMdQAKCRDcUY/If5cW
qqOTAP0VHHcUNYSak2xx0iyNqub2Bj4Cu1AFN9M2rxHIQFokxwEAv64LslHkI6YT
fOPWh6ADXXU+0KahK6NPk4WL1fXU/A0=
=M4gC
-----END PGP SIGNATURE-----

?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 49810@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 49810
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch