[PATCH] guix build: Support '--remote-log-file=PACKAGE'.

  • Open
  • quality assurance status badge
Details
3 participants
  • Oleg Pykhalov
  • Ludovic Courtès
  • Tobias Geerinckx-Rice
Owner
unassigned
Submitted by
Oleg Pykhalov
Severity
normal
O
O
Oleg Pykhalov wrote on 28 Feb 2018 15:19
(address . guix-patches@gnu.org)(name . Oleg Pykhalov)(address . go.wigust@gmail.com)
20180228141959.19789-1-go.wigust@gmail.com
‘--remote-log-file’ allows to get a URL for a build log file on a substitute
server regardless is it built locally. ‘--log-file’ returns always local
build log file.

* guix/scripts/build.scm (show-build-log): Split function.
(show-remote-build-log): New function.
(guix-build): Add this.
* doc/guix.texi (Invoking guix build): Document this.
---
doc/guix.texi | 18 +++++++++---------
guix/scripts/build.scm | 31 ++++++++++++++++++++++---------
2 files changed, 31 insertions(+), 18 deletions(-)

Toggle diff (115 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 24db16761..782e532ce 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5812,9 +5812,8 @@ more on GC roots.
@item --log-file
@cindex build logs, access
-Return the build log file names or URLs for the given
-@var{package-or-derivation}, or raise an error if build logs are
-missing.
+Return the build log file names @var{package-or-derivation}, or raise an
+error if build logs are missing.
This works regardless of how packages or derivations are specified. For
instance, the following invocations are equivalent:
@@ -5826,15 +5825,16 @@ guix build --log-file guile
guix build --log-file -e '(@@ (gnu packages guile) guile-2.0)'
@end example
-If a log is unavailable locally, and unless @code{--no-substitutes} is
-passed, the command looks for a corresponding log on one of the
-substitute servers (as specified with @code{--substitute-urls}.)
+@item --remote-log-file
+@cindex build logs, access
+
+Same as @code{--log-file} but on one of the substitute servers (as
+specified with @code{--substitute-urls}.
-So for instance, imagine you want to see the build log of GDB on MIPS,
-but you are actually on an @code{x86_64} machine:
+For example, you want to see the build log of GDB on MIPS:
@example
-$ guix build --log-file gdb -s mips64el-linux
+$ guix build --remote-log-file gdb -s mips64el-linux
https://hydra.gnu.org/log/@dots{}-gdb-7.10
@end example
diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm
index 57f2d82c5..c45271e50 100644
--- a/guix/scripts/build.scm
+++ b/guix/scripts/build.scm
@@ -601,6 +601,9 @@ must be one of 'package', 'all', or 'transitive'~%")
(option '("log-file") #f #f
(lambda (opt name arg result)
(alist-cons 'log-file? #t result)))
+ (option '("remote-log-file") #f #f
+ (lambda (opt name arg result)
+ (alist-cons 'remote-log-file? #t result)))
(append %transformation-options
%standard-build-options)))
@@ -691,15 +694,20 @@ package '~a' has no source~%")
(map (cut transform store <>)
(options->things-to-build opts)))))
-(define (show-build-log store file urls)
- "Show the build log for FILE, falling back to remote logs from URLS if
-needed."
- (let ((log (or (log-file store file)
- (log-url store file #:base-urls urls))))
+(define (show-build-log store file)
+ "Show the build log for FILE."
+ (let ((log (log-file store file)))
(if log
(format #t "~a~%" log)
(leave (G_ "no build log for '~a'~%") file))))
+(define (show-remote-build-log store file urls)
+ "Show the remote build log for FILE from URLS."
+ (let ((log (log-url store file #:base-urls urls)))
+ (if log
+ (format #t "~a~%" log)
+ (leave (G_ "no remote build log for '~a'~%") file))))
+
;;;
;;; Entry point.
@@ -713,6 +721,9 @@ needed."
(define quiet?
(assoc-ref opts 'quiet?))
+ (define (derivation-file-names drv items)
+ (delete-duplicates (append (map derivation-file-name drv) items)))
+
(with-error-handling
;; Ask for absolute file names so that .drv file names passed from the
;; user to 'read-derivation' are absolute when it returns.
@@ -744,6 +755,7 @@ needed."
opts)))
(unless (or (assoc-ref opts 'log-file?)
+ (assoc-ref opts 'remote-log-file?)
(assoc-ref opts 'derivations-only?))
(show-what-to-build store drv
#:use-substitutes?
@@ -752,10 +764,11 @@ needed."
#:mode mode))
(cond ((assoc-ref opts 'log-file?)
- (for-each (cut show-build-log store <> urls)
- (delete-duplicates
- (append (map derivation-file-name drv)
- items))))
+ (for-each (cut show-build-log store <>)
+ (derivation-file-names drv items)))
+ ((assoc-ref opts 'remote-log-file?)
+ (for-each (cut show-remote-build-log store <> urls)
+ (derivation-file-names drv items)))
((assoc-ref opts 'derivations-only?)
(format #t "~{~a~%~}" (map derivation-file-name drv))
(for-each (cut register-root store <> <>)
--
2.16.1
L
L
Ludovic Courtès wrote on 28 Feb 2018 23:13
(name . Oleg Pykhalov)(address . go.wigust@gmail.com)(address . 30647@debbugs.gnu.org)
87po4ou5ie.fsf@gnu.org
Hello Oleg,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

Toggle quote (4 lines)
> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
> server regardless is it built locally. ‘--log-file’ returns always local
> build log file.

What did you think of having ‘--log-file’ transparently fall back to
searching for log files on substitute servers?

I find it handy, but also wondered if it might surprise users that such
a trivially-looking option connects to external servers. I thought
about having it print something when it does so. Would address your
concerns?

Thanks,
Ludo’.
O
O
Oleg Pykhalov wrote on 1 Mar 2018 05:19
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 30647@debbugs.gnu.org)
87606ge8c5.fsf@gmail.com
Hello Ludovic,

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (9 lines)
> Oleg Pykhalov <go.wigust@gmail.com> skribis:
>
>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>> server regardless is it built locally. ‘--log-file’ returns always local
>> build log file.
>
> What did you think of having ‘--log-file’ transparently fall back to
> searching for log files on substitute servers?

Sorry, I don't understand the question. Does the “fall back” mean the
behavior before a patch?

Toggle quote (5 lines)
> I find it handy, but also wondered if it might surprise users that such
> a trivially-looking option connects to external servers. I thought
> about having it print something when it does so. Would address your
> concerns?

Do you mean always connect to the external server and print a URL for a
log file in addition to path of local log file? I don't think mixing
those in one output is good, because for example you cannot do like:
Toggle snippet (4 lines)
diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)


As a better approach in addition to ‘--no-substitutes’, maybe we could
implement ‘--only-substitutes’ (as I remember Nix has it)? Such flag
will return a remote log file and will avoid building packages locally.

Oleg.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEc+OyAXw1EaDPCmAPckbhHGm3lWkFAlqXf0oACgkQckbhHGm3
lWkSsA/9HW9SxDKpaOTVNhsCIaAhE7h8WqJR7kLfKAlPPoy2afR9bVgGpoUNqt+f
yW2tqBxUmaQLZg8vzv8/RedHKY9x2yAa1svIPTjmEeagj883dy8I1eEXqMn293m2
omWJpvBLiFFV5WwRautx52W6SsCHB+5n1HuQnYiAwXCk8Z8SfXeYy2XgeqMue8DU
ZmKqlZ36lrEGWLobKFzXUeYMnm3KcBMUIVKFc0Rf3HmmrkWdzNd3U8sU2wTOoKJJ
aKUjtTxywhbG+CLaHb0OF6R/b6a0Advtii/s7O+egs+lNNJlDJbXfMKi8Dm+MeKt
o6+Gyi7Z4vGoNVPNZhqQduDQo/QbkxPUct3fQanMc+Ciq8Ts5+cR3lKQ2m7VIC5A
yn10ipMgS50dhM8nbq6PZRZn1zudg2wvxot5y5u1aePWKBpvTondgKzIAj7gz1ku
mkAuyLI52qWM6PpF9NGkrW3kTEFZzyj3190/9Lo6wQSch//PVq2dbDTJL5TCbVz3
J8vlasekHwWeVaNYhc/kE/a1SKktg5C6woyITA3oD2ZuriwK6MQAc4Xpp0Rrsoer
yvARKOGsiwh/nYbqsv6+eKZbvK5uXsm3hwn8U6Da21aFQygeuq3s+/sQ1kSmTukO
7mhDCpauUxzT8784ALl6pU+Ixi6Tow0kqNPEo19ptw+/UFW61XM=
=M7dp
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 1 Mar 2018 14:22
(name . Oleg Pykhalov)(address . go.wigust@gmail.com)(address . 30647@debbugs.gnu.org)
874lm06ic9.fsf@gnu.org
Hello,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

Toggle quote (14 lines)
> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Oleg Pykhalov <go.wigust@gmail.com> skribis:
>>
>>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>>> server regardless is it built locally. ‘--log-file’ returns always local
>>> build log file.
>>
>> What did you think of having ‘--log-file’ transparently fall back to
>> searching for log files on substitute servers?
>
> Sorry, I don't understand the question. Does the “fall back” mean the
> behavior before a patch?

To put it differently: what do you dislike about the current behavior?

Toggle quote (8 lines)
>> I find it handy, but also wondered if it might surprise users that such
>> a trivially-looking option connects to external servers. I thought
>> about having it print something when it does so. Would address your
>> concerns?
>
> Do you mean always connect to the external server and print a URL for a
> log file in addition to path of local log file?

No no: keep the current behavior, but print something when we’re looking
for a remote log file (currently it silently checks whether the remote
log file is available.)

Toggle quote (5 lines)
> I don't think mixing those in one output is good, because for example
> you cannot do like:
>
> diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)

I see. I guess I’ve never wanted that, or rather, when I do, I
explicitly wget the remote log file. :-)

So I guess I’m unconvinced about the need for a separate
‘--remote-log-file’ option.

What do people think? Ricardo?

Toggle quote (4 lines)
> As a better approach in addition to ‘--no-substitutes’, maybe we could
> implement ‘--only-substitutes’ (as I remember Nix has it)? Such flag
> will return a remote log file and will avoid building packages locally.

That could be an option, but that’s much more work (not limited to log
file handling.)

Thanks,
Ludo’.
T
T
Tobias Geerinckx-Rice wrote on 1 Mar 2018 16:16
(address . ludo@gnu.org)(address . 30647@debbugs.gnu.org)
940aba4e5d289b70aa499525c0093704@tobias.gr
Hullo,

On 2018-02-28 23:13, ludo@gnu.org wrote:
Toggle quote (6 lines)
> What did you think of having ‘--log-file’ transparently fall back to
> searching for log files on substitute servers?
>
> I find it handy, but also wondered if it might surprise users that such
> a trivially-looking option connects to external servers.

It would pleasantly surprise me :-)

(...well, not really ‘surprise’ — just more Guixy network-transparent
goodness.)

TBH, I don't see the difference between this & all other actions that
already connect to external servers.

Is this different? Will it ignore ’--no-substitutes’, or ping servers
outside of user-approved substitute-urls? Does it need to?

Toggle quote (2 lines)
> I thought about having it print something when it does so.

Like a progress bar? Seems more eye-catching than yet another warning.
Or something more permanent in/next to the downloaded log?

Kind regards,

T G-R

Sent from a Web browser. Excuse or enjoy my brevity.
O
O
Oleg Pykhalov wrote on 1 Mar 2018 16:40
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 30647@debbugs.gnu.org)
87606f3it9.fsf@gmail.com
ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (9 lines)
>>>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>>>> server regardless is it built locally. ‘--log-file’ returns always local
>>>> build log file.
>>>
>>> What did you think of having ‘--log-file’ transparently fall back to
>>> searching for log files on substitute servers?
>
> To put it differently: what do you dislike about the current behavior?

Suppose package build failed locally. I want to receive a log from a
remote server. I could do it manually by:

1. Removing local failed log.
2. ‘wget’, but I need to know a URL.
3. Hydra web interface, which is slow (especially multiple packages).

Toggle quote (4 lines)
> No no: keep the current behavior, but print something when we’re looking
> for a remote log file (currently it silently checks whether the remote
> log file is available.)

Still not clear to me. If ‘guix --log-file’ checks for a remote log
file, then it gets a valid URL to a remote build log file for free,
doesn't it?

Toggle quote (8 lines)
>> I don't think mixing those in one output is good, because for example
>> you cannot do like:
>>
>> diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)
>
> I see. I guess I’ve never wanted that, or rather, when I do, I
> explicitly wget the remote log file. :-)

Could I ask What's your workflow for ‘wget’?

Toggle quote (5 lines)
> So I guess I’m unconvinced about the need for a separate
> ‘--remote-log-file’ option.
>
> What do people think? Ricardo?

Maybe CC him? Or is it a bad etiquette for a mailing list, because he
is subscribed?

Toggle quote (7 lines)
>> As a better approach in addition to ‘--no-substitutes’, maybe we could
>> implement ‘--only-substitutes’ (as I remember Nix has it)? Such flag
>> will return a remote log file and will avoid building packages locally.
>
> That could be an option, but that’s much more work (not limited to log
> file handling.)

Yes, but benefits (especially avoid building packages locally) are
worth.

If you don't agree with the patch, I'll not complain and will try to
work on ‘--only-substitutes’. :-)

Oleg.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEc+OyAXw1EaDPCmAPckbhHGm3lWkFAlqYHwIACgkQckbhHGm3
lWnx8w//Zve5wIw99HUQccFzPYQT016DdInm6OVQ3uDQLNT9c+3EjG43j7E21df0
qJJkMj+iUWGyc9P3Wny8yF4jU2Pemw+sz3WSGH5czhD4/a+5XE4FfnkT/aQhTV4N
zbQzbsYwUxxPC4MJVLP3xpoN48DAGGRA4s11cYtELKnfSpsqByENfPkJ2JDYU0vj
FmYHu97LTSrdX49NMwTof2PtJeshtR7sLUMXD2SkKixj+LfRVv+jafbS29gk71vr
LqIydTuUuHArAdzdb9/CWtmzY1stEd0h0HAtuySGL7BRitOu+i/rz8uj96vlxYrm
/gOPcGIbYHfhZUNt61yP9H8SGGgLiCeHV2CgfpzItp7Cea9IO8v+/O2x3Rpw3bXr
a/yILKVZO4p/BLAjVGxNMKBH1P5Sy+Tl1lzLgwvjkL0JYXzKILYsZaP71nO8WHJ2
2KVjG6Eth6aBXtHkL6rvOK3qZFC9hYYk46SYoKb9Z++AaioNK9JaARYLzZLBFcmv
uAWQ+vgjpAiKnPLXtOjcVt1iLw7uaE9X9e397FFWid94wlidCzgqu7OQnp6R10Fr
gXrvd6QgREewUU7P13eH1s4IKDgs8WDP+mbnzV8Rm+nBHqqWi3PNXzCMTxtHzORI
cWFhCVKZ1YM0/2oapNWH5bIcpEvy202gBmNPEGyHmPRv+oQ2uWQ=
=aijF
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 1 Mar 2018 22:23
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)(address . 30647@debbugs.gnu.org)
87d10n4hil.fsf@gnu.org
Hey Tobias,

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

Toggle quote (9 lines)
> On 2018-02-28 23:13, ludo@gnu.org wrote:
>> What did you think of having ‘--log-file’ transparently fall back to
>> searching for log files on substitute servers?
>>
>> I find it handy, but also wondered if it might surprise users that such
>> a trivially-looking option connects to external servers.
>
> It would pleasantly surprise me :-)

Good. :-)

Toggle quote (6 lines)
> TBH, I don't see the difference between this & all other actions that
> already connect to external servers.
>
> Is this different? Will it ignore ’--no-substitutes’, or ping servers
> outside of user-approved substitute-urls? Does it need to?

It’s not really different, indeed. (The only thing is that the list of
substitute URLs that is used by ‘--log-file’ can differ from that of
guix-daemon; but that’s a bug, really.)

Toggle quote (5 lines)
>> I thought about having it print something when it does so.
>
> Like a progress bar? Seems more eye-catching than yet another
> warning. Or something more permanent in/next to the downloaded log?

‘--log-file’ just probes remote servers without actually downloading the
log, so a progress bar wouldn’t be useful.

I was thinking of a message like:

info: looking for build log at https://…/log/…-foo

Ludo’.
L
L
Ludovic Courtès wrote on 1 Mar 2018 22:30
(name . Oleg Pykhalov)(address . go.wigust@gmail.com)(address . 30647@debbugs.gnu.org)
878tbb4h7f.fsf@gnu.org
Oleg Pykhalov <go.wigust@gmail.com> skribis:

Toggle quote (17 lines)
> ludo@gnu.org (Ludovic Courtès) writes:
>
>>>>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>>>>> server regardless is it built locally. ‘--log-file’ returns always local
>>>>> build log file.
>>>>
>>>> What did you think of having ‘--log-file’ transparently fall back to
>>>> searching for log files on substitute servers?
>>
>> To put it differently: what do you dislike about the current behavior?
>
> Suppose package build failed locally. I want to receive a log from a
> remote server. I could do it manually by:
>
> 1. Removing local failed log.
> 2. ‘wget’, but I need to know a URL.

The URL scheme is documented and easy to use (info "(guix) Invoking guix
publish"). That’s why I don’t find wget to be much of a problem.

Toggle quote (8 lines)
>> No no: keep the current behavior, but print something when we’re looking
>> for a remote log file (currently it silently checks whether the remote
>> log file is available.)
>
> Still not clear to me. If ‘guix --log-file’ checks for a remote log
> file, then it gets a valid URL to a remote build log file for free,
> doesn't it?

Correct. See ‘log-url’ in (guix scripts build).

Toggle quote (10 lines)
>>> I don't think mixing those in one output is good, because for example
>>> you cannot do like:
>>>
>>> diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)
>>
>> I see. I guess I’ve never wanted that, or rather, when I do, I
>> explicitly wget the remote log file. :-)
>
> Could I ask What's your workflow for ‘wget’?

Something like:

$ guix build foo
/gnu/store/xyz-foo
Toggle quote (13 lines)
>>> As a better approach in addition to ‘--no-substitutes’, maybe we could
>>> implement ‘--only-substitutes’ (as I remember Nix has it)? Such flag
>>> will return a remote log file and will avoid building packages locally.
>>
>> That could be an option, but that’s much more work (not limited to log
>> file handling.)
>
> Yes, but benefits (especially avoid building packages locally) are
> worth.
>
> If you don't agree with the patch, I'll not complain and will try to
> work on ‘--only-substitutes’. :-)

Heheh. :-)

IIRC, --only-substitutes in Nix is used together with --upgrade, no?
Some considered it questionable from a security standpoint when we last

Thanks,
Ludo’.
?