do not attempt to build a package known to be broken

OpenSubmitted by Ricardo Wurmus.
Details
5 participants
  • Björn Höfling
  • Danny Milosavljevic
  • Ludovic Courtès
  • Ricardo Wurmus
  • znavko
Owner
unassigned
Severity
normal
R
R
Ricardo Wurmus wrote on 14 Dec 2018 05:41
(address . bug-guix@gnu.org)
87efakyb26.fsf@elephly.net
The default use case is for users to have substitutes enabled and to
fetch binaries from a build farm. The build farm tries to build all
packages, but not all builds succeed. The daemon caches build failures.

This information, however, is not available for users. The default
behaviour of Guix is to attempt building the package locally. This is
fine for packages that the build farm hasn’t attempted yet, but it is a
waste of time when the build has already been attempted without success.

I propose that “guix publish” should also convey information about
failed builds. The Guix client can then abort right away without
wasting time and power to attempt a doomed build. Users may override
this with “--fallback” or a new option.

--
Ricardo
B
B
Björn Höfling wrote on 15 Dec 2018 19:54
(name . Ricardo Wurmus)(address . rekado@elephly.net)(address . 33737@debbugs.gnu.org)
20181215195437.32313415@alma-ubu
On Fri, 14 Dec 2018 05:41:37 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:

Toggle quote (11 lines)
> The default use case is for users to have substitutes enabled and to
> fetch binaries from a build farm. The build farm tries to build all
> packages, but not all builds succeed. The daemon caches build
> failures.
>
> This information, however, is not available for users. The default
> behaviour of Guix is to attempt building the package locally. This is
> fine for packages that the build farm hasn’t attempted yet, but it is
> a waste of time when the build has already been attempted without
> success.

+1

Very good idea.

Björn
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlwVTe4ACgkQvyhstlk+X/0awQCcCWfhH+4An4y2ajHvKn1yGD46
kwMAnjwZwWwFEDdWtClwBVYrYlapHk5d
=7tPv
-----END PGP SIGNATURE-----


Z
Z
znavko wrote on 15 Dec 2018 20:39
LTnNKXp--3-1@tutanota.com
Package can have additional fields in guix repository, like
architecture, version, buid_status, source_type
build_status may contain such info.
source_type could show if the package source url is one of those public git services (thus, user could enable proxy first).


Dec 15, 2018, 6:54 PM by bjoern.hoefling@bjoernhoefling.de:

Toggle quote (21 lines)
> On Fri, 14 Dec 2018 05:41:37 +0100
> Ricardo Wurmus <> rekado@elephly.net <mailto:rekado@elephly.net>> > wrote:
>
>> The default use case is for users to have substitutes enabled and to
>> fetch binaries from a build farm. The build farm tries to build all
>> packages, but not all builds succeed. The daemon caches build
>> failures.
>>
>> This information, however, is not available for users. The default
>> behaviour of Guix is to attempt building the package locally. This is
>> fine for packages that the build farm hasn’t attempted yet, but it is
>> a waste of time when the build has already been attempted without
>> success.
>>
>
> +1
>
> Very good idea.
>
> Björn
>
Attachment: file
D
D
Danny Milosavljevic wrote on 16 Dec 2018 21:14
(name . Ricardo Wurmus)(address . rekado@elephly.net)(address . 33737@debbugs.gnu.org)
20181216211451.7e15fdc7@scratchpost.org
On Fri, 14 Dec 2018 05:41:37 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:

Toggle quote (14 lines)
> The default use case is for users to have substitutes enabled and to
> fetch binaries from a build farm. The build farm tries to build all
> packages, but not all builds succeed. The daemon caches build failures.
>
> This information, however, is not available for users. The default
> behaviour of Guix is to attempt building the package locally. This is
> fine for packages that the build farm hasn’t attempted yet, but it is a
> waste of time when the build has already been attempted without success.
>
> I propose that “guix publish” should also convey information about
> failed builds. The Guix client can then abort right away without
> wasting time and power to attempt a doomed build. Users may override
> this with “--fallback” or a new option.

As long as we distinguish transient build machine errors (disk full, can't
resolve hostname, too many names in directory, no inodes left etc) from actual
errors in the source code, +1.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlwWsjsACgkQ5xo1VCww
uqUFYAf/S5B2aa9pyMonY36WhPle4KORF+QS8z4HVlypP3CTusWpDMehomUQIhDA
O/Ph6RJysg/FID1NTbHnMwWKfUg8g8OmxCtG/iyAm+5hwtRWsmaurS6Wg8oGq5rw
rzL2fcbSfnQhT0KYKdHwdzrZowhC8QGQyBy6QqZMzvpGRRf+T5Rxpb29/PEJ59pC
2iHz/pLEh288K2Q6L0olUlIMncAssgORM/fcgSGx2vkTOKD2NmE15SCKfbxy0cNt
kAh4++QajEWhrC0HNI9roJfh7+aDClDDG9UEAv00yI7lyGn1pO5TMBmGXGWFAX6F
Llj9JxhVbcw/YwAfOEJDZKjpYMmH3g==
=cIY8
-----END PGP SIGNATURE-----


R
R
Ricardo Wurmus wrote on 16 Dec 2018 22:55
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)(address . 33737@debbugs.gnu.org)
87imztw2zx.fsf@elephly.net
Danny Milosavljevic <dannym@scratchpost.org> writes:

Toggle quote (9 lines)
>> I propose that “guix publish” should also convey information about
>> failed builds. The Guix client can then abort right away without
>> wasting time and power to attempt a doomed build. Users may override
>> this with “--fallback” or a new option.
>
> As long as we distinguish transient build machine errors (disk full, can't
> resolve hostname, too many names in directory, no inodes left etc) from actual
> errors in the source code, +1.

I was thinking of something very simple. The daemon is already capable
of caching build failures; I think it may be good to simply pass that
information on to clients.

I don’t know how we could distinguish different kinds of build failures
with the current daemon.

--
Ricardo
L
L
Ludovic Courtès wrote on 18 Dec 2018 11:50
(name . Ricardo Wurmus)(address . rekado@elephly.net)
87mup3hzxi.fsf@gnu.org
Hello!

Ricardo Wurmus <rekado@elephly.net> skribis:

Toggle quote (7 lines)
> Danny Milosavljevic <dannym@scratchpost.org> writes:
>
>>> I propose that “guix publish” should also convey information about
>>> failed builds. The Guix client can then abort right away without
>>> wasting time and power to attempt a doomed build. Users may override
>>> this with “--fallback” or a new option.

Good idea!

The difficulty is that the protocol between ‘guix substitute --query’
and the daemon is currently very much Boolean: either ‘guix substitute’
provides the binary or it doesn’t. We’d have to adjust it.

Toggle quote (8 lines)
>> As long as we distinguish transient build machine errors (disk full, can't
>> resolve hostname, too many names in directory, no inodes left etc) from actual
>> errors in the source code, +1.
>
> I was thinking of something very simple. The daemon is already capable
> of caching build failures; I think it may be good to simply pass that
> information on to clients.

The daemon tries to cache only “persistent” failures. In particular,
local failures likely due to lack of space are not cached.

However, the daemon currently doesn’t distinguish between ENOSPC and
actual failures for offloaded builds. We may be able to fix this by
having ‘guix offload’ check the available disk space upon failure.

Then there’s the problem of non-deterministic failures. For these it
would be nicer to let the user build stuff anyway. There’s no good
solution here.

Ludo’.
?