Builds are *not* offloaded when using the "--keep-failed" or -K option.

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Maxim Cournoyer
Owner
unassigned
Submitted by
Maxim Cournoyer
Severity
normal
M
M
Maxim Cournoyer wrote on 2 Feb 2020 05:57
(name . bug-guix)(address . bug-guix@gnu.org)
87o8uhr3gx.fsf@gmail.com
Consider the following, where 10.42.0.199 is a machine described in
/etc/guix/machines.scm for offloading:

$ ./pre-inst-env guix build php
The following derivation will be built:
/gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv
process 8241 acquired build slot '/var/guix/offload/10.42.0.199:22/0'
load on machine '10.42.0.199' is 0.04 (normalized: 0.04)
building /gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv...
^C

$ ./pre-inst-env guix build php --keep-failed
The following derivation will be built:
/gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv
building /gnu/store/dj9qi9lj4xz0dk6agq9mndh8mhi8ifrg-php-7.4.1.drv...
starting phase `set-SOURCE-DATE-EPOCH'
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
starting phase `set-paths'
environment variable `PATH' set to
`/gnu/store/niqa6dlvw4r2bfbabn65kq1kgdfwcb4b-pkg-config-0.29.2/bin:/gnu/store/qz06d1dz19wkrmibcr1485wqqia9zzhi-bison-3.5.1/bin:/gnu/store/3fzw8qb3cmrdkr46y55njjrfkvxd53lr-intltool-0.51.0/bin:
[...]

In the second case, the build is started on the local machine instead of
being dispatched to the offload machine. Presumably, this is done so
that the files are available locally; but I'd prefer if it'd offload and
keep the files on the remote. An alternative would be to copy the remote build directory
locally after failure.
L
L
Ludovic Courtès wrote on 2 Feb 2020 18:43
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 39387@debbugs.gnu.org)
87y2tkzxyh.fsf@gnu.org
Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (6 lines)
> In the second case, the build is started on the local machine instead of
> being dispatched to the offload machine. Presumably, this is done so
> that the files are available locally; but I'd prefer if it'd offload and
> keep the files on the remote. An alternative would be to copy the remote build directory
> locally after failure.

Indeed, it’s a feature! :-) It was discussed here:


There we discussed the possibility of printing a warning, which wasn’t
implemented.

I’m closing in the meantime, but let me know what you think!

Thank you,
Ludo’.
L
L
Ludovic Courtès wrote on 2 Feb 2020 18:44
control message for bug #39387
(address . control@debbugs.gnu.org)
87wo94zxy5.fsf@gnu.org
tags 39387 notabug
close 39387
quit
M
M
Maxim Cournoyer wrote on 6 Feb 2020 05:55
Re: bug#39387: Builds are *not* offloaded when using the "--keep-failed" or -K option.
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 39387@debbugs.gnu.org)
87sgjomi14.fsf@gmail.com
Hello!

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (17 lines)
> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> In the second case, the build is started on the local machine instead of
>> being dispatched to the offload machine. Presumably, this is done so
>> that the files are available locally; but I'd prefer if it'd offload and
>> keep the files on the remote. An alternative would be to copy the remote build directory
>> locally after failure.
>
> Indeed, it’s a feature! :-) It was discussed here:
>
> https://issues.guix.gnu.org/issue/38312
>
> There we discussed the possibility of printing a warning, which wasn’t
> implemented.

I see!

Toggle quote (2 lines)
> I’m closing in the meantime, but let me know what you think!

OK! I think I now see the technical difficulties in implementing a
useful, offloaded interrupted build: the build artifacts, even if copied
locally, would require syncing all the build time dependencies in order
to allow sourcing the "environment-variables" script and be able to
debug. That's more the job of 'guix publish', not of 'guix offload'.

Maxim
M
M
Maxim Cournoyer wrote on 6 Feb 2020 05:56
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 39387@debbugs.gnu.org)
87o8ucmhzu.fsf@gmail.com
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (22 lines)
> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> In the second case, the build is started on the local machine instead of
>> being dispatched to the offload machine. Presumably, this is done so
>> that the files are available locally; but I'd prefer if it'd offload and
>> keep the files on the remote. An alternative would be to copy the remote build directory
>> locally after failure.
>
> Indeed, it’s a feature! :-) It was discussed here:
>
> https://issues.guix.gnu.org/issue/38312
>
> There we discussed the possibility of printing a warning, which wasn’t
> implemented.
>
> I’m closing in the meantime, but let me know what you think!
>
> Thank you,
> Ludo’.

Still, it'd be nice to have an option to keep failed build on the
*remote* machines. I don't think that's possible currently?

Maxim
L
L
Ludovic Courtès wrote on 6 Feb 2020 14:54
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 39387@debbugs.gnu.org)
87k14z95y9.fsf@gnu.org
Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (2 lines)
> Ludovic Courtès <ludo@gnu.org> writes:

[...]

Toggle quote (17 lines)
>> Indeed, it’s a feature! :-) It was discussed here:
>>
>> https://issues.guix.gnu.org/issue/38312
>>
>> There we discussed the possibility of printing a warning, which wasn’t
>> implemented.
>
> I see!
>
>> I’m closing in the meantime, but let me know what you think!
>
> OK! I think I now see the technical difficulties in implementing a
> useful, offloaded interrupted build: the build artifacts, even if copied
> locally, would require syncing all the build time dependencies in order
> to allow sourcing the "environment-variables" script and be able to
> debug. That's more the job of 'guix publish', not of 'guix offload'.

Right. Ideally the build directory of an offloaded could be copied back
to the host of the caller, but that’s not implemented and a bit tricky
maybe.

Before this change, my main use of ‘--no-build-hook’ would be together
with ‘-K’ for that reason.

Ludo’.
L
L
Ludovic Courtès wrote on 6 Feb 2020 14:55
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 39387@debbugs.gnu.org)
87ftfn95wb.fsf@gnu.org
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (3 lines)
> Still, it'd be nice to have an option to keep failed build on the
> *remote* machines. I don't think that's possible currently?

In some cases it’s not even desirable because you could fill up /tmp
without noticing.

But note that another option is to do:

GUIX_DAEMON_SOCKET=ssh://remote.example.org guix build foo -K

WDYT?

Ludo’.
M
M
Maxim Cournoyer wrote on 8 Feb 2020 18:04
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 39387@debbugs.gnu.org)
877e0xc8nj.fsf@gmail.com
Hello Ludovic!

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (16 lines)
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Still, it'd be nice to have an option to keep failed build on the
>> *remote* machines. I don't think that's possible currently?
>
> In some cases it’s not even desirable because you could fill up /tmp
> without noticing.
>
> But note that another option is to do:
>
> GUIX_DAEMON_SOCKET=ssh://remote.example.org guix build foo -K
>
> WDYT?
>
> Ludo’.

This fulfill my use case well and appears to be sufficiently documented
already. We're gold!

Thank you,

Maxim
?