unpack phase in gnu-build-system is non-deterministic

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Christopher Baines
Owner
unassigned
Submitted by
Christopher Baines
Severity
normal

Debbugs page

Christopher Baines wrote 6 years ago
unpack phase in the gnu-build-system is sometimes non-deterministic
(address . bug-guix@gnu.org)
87mukh6vjq.fsf@cbaines.net
I believe that the direnv package has encountered an issue with the
gnu-build-system [1].


Due to the combination of the 'setup-go-environment phase from the
go-build-system, and the 'unpack phase of the gnu-build-system, there
are two directories to be considered by first-subdirectory when called
from the unpack phase.

It seems from direnv that this either consistently, with the package
working on one machine, or failing consistently on another.

To avoid issues like this in the future, I think it would be good to
have first-subdirectory raise an error if it's behaviour could be
non-deterministic.
-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAly+u8lfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE
9XevhRAAjrtgjyfuyX893fzE8MtSkxUsWvONwg+fixonyRD/SIU/rNqgGTahpYPx
kPiQ20MA8nZD2NtBmWC+JNKCQ6DVeIbfp9+uFyYqGZ6To6H9GNufC+BSyfr2I3+v
6TFLnNiGtDgJdEV/3kAndq3aB2bBme9jCpVVrHMd6K3y+dewN6+XLpWUbvTNgu5C
rB50TMaIyG8M0rBuf7oS+aPY48rV6NAaA3/f4gRl2kn2GflSfd+LI7T3sJeOYcA8
+u3jfmuHdpml0aWFWKU8t34Ixr2f3+Ynbax4WKLXN7G3cW/0mt8GSBfsVUV73yBE
koTsMry5UBsLw8ws4ZhkWuuCOFN3FAGID7+WCoHxpCUmkuEux3b3bQIZm22cKyQJ
1amXfh5jLbp2Dz2Dg0sXbnDEDR2mQ/OYlKIoOge75wO6fSQMvzZhbS7xIgzZufKG
gpfxLFRSfWOawvn3j+rf3XTuri98roKJSAQIB16kpBjSILfVEEn/WpR48RLGnExj
U2l7059LEwEo5O4F9A/pLvsfcQIDUHeWN4gAtidJO7w4nzYB8mcuunt/GCoIuYST
HUxBG2uk68ymj9sW6TGFRYeT9UFwqjKmfh9jJhE9Po83c655AU/BlKMGphNOA8v+
9jo2B0RFJOpg8EAOYx80kxtQvKuFNUuqDxs0Z2kH0lWJ2a1ftXE=
=F4zF
-----END PGP SIGNATURE-----

Ludovic Courtès wrote 6 years ago
(name . Christopher Baines)(address . mail@cbaines.net)(address . 35387@debbugs.gnu.org)
87ef5shjpj.fsf@gnu.org
Hi Chris,

Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (17 lines)
> I believe that the direnv package has encountered an issue with the
> gnu-build-system [1].
>
> 1: https://issues.guix.info/issue/35386
>
> Due to the combination of the 'setup-go-environment phase from the
> go-build-system, and the 'unpack phase of the gnu-build-system, there
> are two directories to be considered by first-subdirectory when called
> from the unpack phase.
>
> It seems from direnv that this either consistently, with the package
> working on one machine, or failing consistently on another.
>
> To avoid issues like this in the future, I think it would be good to
> have first-subdirectory raise an error if it's behaviour could be
> non-deterministic.

‘file-system-fold’ is just a wrapper around ‘readdir’ so the order in
which it sees directory entries is non-deterministic.

What about writing it like this:

(define (first-subdirectory directory)
"Return the file name of the first sub-directory of DIRECTORY."
(match (scandir directory
(lambda (file)
(and (not (member file '("." "..")))
(file-is-directory? (string-append directory "/"
file)))))
((first . _) first)))

The result will be deterministic since ‘scandir’ sorts entries.

Thanks,
Ludo’.
Christopher Baines wrote 6 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 35387@debbugs.gnu.org)
87sgu0q8lv.fsf@cbaines.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (37 lines)
> Hi Chris,
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> I believe that the direnv package has encountered an issue with the
>> gnu-build-system [1].
>>
>> 1: https://issues.guix.info/issue/35386
>>
>> Due to the combination of the 'setup-go-environment phase from the
>> go-build-system, and the 'unpack phase of the gnu-build-system, there
>> are two directories to be considered by first-subdirectory when called
>> from the unpack phase.
>>
>> It seems from direnv that this either consistently, with the package
>> working on one machine, or failing consistently on another.
>>
>> To avoid issues like this in the future, I think it would be good to
>> have first-subdirectory raise an error if it's behaviour could be
>> non-deterministic.
>
> ‘file-system-fold’ is just a wrapper around ‘readdir’ so the order in
> which it sees directory entries is non-deterministic.
>
> What about writing it like this:
>
> (define (first-subdirectory directory)
> "Return the file name of the first sub-directory of DIRECTORY."
> (match (scandir directory
> (lambda (file)
> (and (not (member file '("." "..")))
> (file-is-directory? (string-append directory "/"
> file)))))
> ((first . _) first)))
>
> The result will be deterministic since ‘scandir’ sorts entries.

That sounds great :)
-----BEGIN PGP SIGNATURE-----

iQKTBAEBCgB9FiEEPonu50WOcg2XVOCyXiijOwuE9XcFAlzH8uxfFIAAAAAALgAo
aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldDNF
ODlFRUU3NDU4RTcyMEQ5NzU0RTBCMjVFMjhBMzNCMEI4NEY1NzcACgkQXiijOwuE
9XfqXQ//aiqO7MFYvGFJ2ejPVQOp8kXlRrf9mQWMfhzAWZNmPilE2BmDks1CFRtG
VSS8EpKKAfpRB/7U7iS5UcApxMsTDTKj+VQA0GsAidQFZOskxKbkM5HY5+NP37kX
j6csre5ydym3vOWq457g0Oj0xUFRzpzM6JtkNZB3wmGnInU9/H7MQ+u+nH8j92nP
MMEA4P5rwHRB7MjZgSwGwh8GZrigu8m/2wwLUq68wl/DzfuQCVL0UuBlPEn0uSpk
FNUFm3BWHYRM0InjhPZiDQ7qvXu4XfymDmrKE3/1q5/ZBu5MUgJnOn8rVGUQU5QD
wxERxKcf4rVFPqXxVVeip4qqKXv0v950wKVc5p3x4uMVjAclQkAbncNfDBqoLOzB
txvA/FPuVNCJGYxGIHe7TQ2LMREJrrBHV0SGde0nTXXNWFOhMCHJfj7NvyBtVDDP
6i6MQW/hD/mxQ2ooQA0nV8rgj7/lu4lorgTLw+xxM4pV7PG8MB/y+2dKuho1B8FX
a5wBhY1raUEqc93qhdkJd6vwGr3ajyqejhsU3drSUVTRw5kX2JscLtmD/XQV+dh/
Gagu7YjLdgVFXfE1wjS8en+WbOog4KZ2a08IX+uXQeG0SmHIi/M94ECIUTU5iqYl
wMHtADYDbFCnhVo4plZsx8VOA8l6tgqrP6HXvn4QDmoMoRgEGVQ=
=RtLT
-----END PGP SIGNATURE-----

Ludovic Courtès wrote 6 years ago
control message for bug #35387
(address . control@debbugs.gnu.org)
874l4rkhfa.fsf@gnu.org
retitle 35387 unpack phase in gnu-build-system is non-deterministic
quit
Ludovic Courtès wrote 6 years ago
Re: bug#35387: unpack phase in the gnu-build-system is sometimes non-deterministic
(name . Christopher Baines)(address . mail@cbaines.net)(address . 35387-done@debbugs.gnu.org)
87tvcrhmgg.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (13 lines)
> What about writing it like this:
>
> (define (first-subdirectory directory)
> "Return the file name of the first sub-directory of DIRECTORY."
> (match (scandir directory
> (lambda (file)
> (and (not (member file '("." "..")))
> (file-is-directory? (string-append directory "/"
> file)))))
> ((first . _) first)))
>
> The result will be deterministic since ‘scandir’ sorts entries.

Pushed as cfd4e4d06e3cda0f3eed8d6b9277ce53e55404b8 on ‘core-updates’.

Thanks,
Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 35387@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 35387
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help