[PATCH] guix build: Support '--remote-log-file=PACKAGE'.

  • Open
  • quality assurance status badge
Details
3 participants
  • Oleg Pykhalov
  • Ludovic Courtès
  • Tobias Geerinckx-Rice
Owner
unassigned
Submitted by
Oleg Pykhalov
Severity
normal
O
O
Oleg Pykhalov wrote on 28 Feb 2018 15:19
(address . guix-patches@gnu.org)(name . Oleg Pykhalov)(address . go.wigust@gmail.com)
20180228141959.19789-1-go.wigust@gmail.com
‘--remote-log-file’ allows to get a URL for a build log file on a substitute
server regardless is it built locally. ‘--log-file’ returns always local
build log file.

* guix/scripts/build.scm (show-build-log): Split function.
(show-remote-build-log): New function.
(guix-build): Add this.
* doc/guix.texi (Invoking guix build): Document this.
---
doc/guix.texi | 18 +++++++++---------
guix/scripts/build.scm | 31 ++++++++++++++++++++++---------
2 files changed, 31 insertions(+), 18 deletions(-)

Toggle diff (115 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 24db16761..782e532ce 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -5812,9 +5812,8 @@ more on GC roots.
@item --log-file
@cindex build logs, access
-Return the build log file names or URLs for the given
-@var{package-or-derivation}, or raise an error if build logs are
-missing.
+Return the build log file names @var{package-or-derivation}, or raise an
+error if build logs are missing.
This works regardless of how packages or derivations are specified. For
instance, the following invocations are equivalent:
@@ -5826,15 +5825,16 @@ guix build --log-file guile
guix build --log-file -e '(@@ (gnu packages guile) guile-2.0)'
@end example
-If a log is unavailable locally, and unless @code{--no-substitutes} is
-passed, the command looks for a corresponding log on one of the
-substitute servers (as specified with @code{--substitute-urls}.)
+@item --remote-log-file
+@cindex build logs, access
+
+Same as @code{--log-file} but on one of the substitute servers (as
+specified with @code{--substitute-urls}.
-So for instance, imagine you want to see the build log of GDB on MIPS,
-but you are actually on an @code{x86_64} machine:
+For example, you want to see the build log of GDB on MIPS:
@example
-$ guix build --log-file gdb -s mips64el-linux
+$ guix build --remote-log-file gdb -s mips64el-linux
https://hydra.gnu.org/log/@dots{}-gdb-7.10
@end example
diff --git a/guix/scripts/build.scm b/guix/scripts/build.scm
index 57f2d82c5..c45271e50 100644
--- a/guix/scripts/build.scm
+++ b/guix/scripts/build.scm
@@ -601,6 +601,9 @@ must be one of 'package', 'all', or 'transitive'~%")
(option '("log-file") #f #f
(lambda (opt name arg result)
(alist-cons 'log-file? #t result)))
+ (option '("remote-log-file") #f #f
+ (lambda (opt name arg result)
+ (alist-cons 'remote-log-file? #t result)))
(append %transformation-options
%standard-build-options)))
@@ -691,15 +694,20 @@ package '~a' has no source~%")
(map (cut transform store <>)
(options->things-to-build opts)))))
-(define (show-build-log store file urls)
- "Show the build log for FILE, falling back to remote logs from URLS if
-needed."
- (let ((log (or (log-file store file)
- (log-url store file #:base-urls urls))))
+(define (show-build-log store file)
+ "Show the build log for FILE."
+ (let ((log (log-file store file)))
(if log
(format #t "~a~%" log)
(leave (G_ "no build log for '~a'~%") file))))
+(define (show-remote-build-log store file urls)
+ "Show the remote build log for FILE from URLS."
+ (let ((log (log-url store file #:base-urls urls)))
+ (if log
+ (format #t "~a~%" log)
+ (leave (G_ "no remote build log for '~a'~%") file))))
+
;;;
;;; Entry point.
@@ -713,6 +721,9 @@ needed."
(define quiet?
(assoc-ref opts 'quiet?))
+ (define (derivation-file-names drv items)
+ (delete-duplicates (append (map derivation-file-name drv) items)))
+
(with-error-handling
;; Ask for absolute file names so that .drv file names passed from the
;; user to 'read-derivation' are absolute when it returns.
@@ -744,6 +755,7 @@ needed."
opts)))
(unless (or (assoc-ref opts 'log-file?)
+ (assoc-ref opts 'remote-log-file?)
(assoc-ref opts 'derivations-only?))
(show-what-to-build store drv
#:use-substitutes?
@@ -752,10 +764,11 @@ needed."
#:mode mode))
(cond ((assoc-ref opts 'log-file?)
- (for-each (cut show-build-log store <> urls)
- (delete-duplicates
- (append (map derivation-file-name drv)
- items))))
+ (for-each (cut show-build-log store <>)
+ (derivation-file-names drv items)))
+ ((assoc-ref opts 'remote-log-file?)
+ (for-each (cut show-remote-build-log store <> urls)
+ (derivation-file-names drv items)))
((assoc-ref opts 'derivations-only?)
(format #t "~{~a~%~}" (map derivation-file-name drv))
(for-each (cut register-root store <> <>)
--
2.16.1
L
L
Ludovic Courtès wrote on 28 Feb 2018 23:13
(name . Oleg Pykhalov)(address . go.wigust@gmail.com)(address . 30647@debbugs.gnu.org)
87po4ou5ie.fsf@gnu.org
Hello Oleg,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

Toggle quote (4 lines)
> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
> server regardless is it built locally. ‘--log-file’ returns always local
> build log file.

What did you think of having ‘--log-file’ transparently fall back to
searching for log files on substitute servers?

I find it handy, but also wondered if it might surprise users that such
a trivially-looking option connects to external servers. I thought
about having it print something when it does so. Would address your
concerns?

Thanks,
Ludo’.
O
O
Oleg Pykhalov wrote on 1 Mar 2018 05:19
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 30647@debbugs.gnu.org)
87606ge8c5.fsf@gmail.com
Hello Ludovic,

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (9 lines)
> Oleg Pykhalov <go.wigust@gmail.com> skribis:
>
>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>> server regardless is it built locally. ‘--log-file’ returns always local
>> build log file.
>
> What did you think of having ‘--log-file’ transparently fall back to
> searching for log files on substitute servers?

Sorry, I don't understand the question. Does the “fall back” mean the
behavior before a patch?

Toggle quote (5 lines)
> I find it handy, but also wondered if it might surprise users that such
> a trivially-looking option connects to external servers. I thought
> about having it print something when it does so. Would address your
> concerns?

Do you mean always connect to the external server and print a URL for a
log file in addition to path of local log file? I don't think mixing
those in one output is good, because for example you cannot do like:
Toggle snippet (4 lines)
diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)


As a better approach in addition to ‘--no-substitutes’, maybe we could
implement ‘--only-substitutes’ (as I remember Nix has it)? Such flag
will return a remote log file and will avoid building packages locally.

Oleg.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEc+OyAXw1EaDPCmAPckbhHGm3lWkFAlqXf0oACgkQckbhHGm3
lWkSsA/9HW9SxDKpaOTVNhsCIaAhE7h8WqJR7kLfKAlPPoy2afR9bVgGpoUNqt+f
yW2tqBxUmaQLZg8vzv8/RedHKY9x2yAa1svIPTjmEeagj883dy8I1eEXqMn293m2
omWJpvBLiFFV5WwRautx52W6SsCHB+5n1HuQnYiAwXCk8Z8SfXeYy2XgeqMue8DU
ZmKqlZ36lrEGWLobKFzXUeYMnm3KcBMUIVKFc0Rf3HmmrkWdzNd3U8sU2wTOoKJJ
aKUjtTxywhbG+CLaHb0OF6R/b6a0Advtii/s7O+egs+lNNJlDJbXfMKi8Dm+MeKt
o6+Gyi7Z4vGoNVPNZhqQduDQo/QbkxPUct3fQanMc+Ciq8Ts5+cR3lKQ2m7VIC5A
yn10ipMgS50dhM8nbq6PZRZn1zudg2wvxot5y5u1aePWKBpvTondgKzIAj7gz1ku
mkAuyLI52qWM6PpF9NGkrW3kTEFZzyj3190/9Lo6wQSch//PVq2dbDTJL5TCbVz3
J8vlasekHwWeVaNYhc/kE/a1SKktg5C6woyITA3oD2ZuriwK6MQAc4Xpp0Rrsoer
yvARKOGsiwh/nYbqsv6+eKZbvK5uXsm3hwn8U6Da21aFQygeuq3s+/sQ1kSmTukO
7mhDCpauUxzT8784ALl6pU+Ixi6Tow0kqNPEo19ptw+/UFW61XM=
=M7dp
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 1 Mar 2018 14:22
(name . Oleg Pykhalov)(address . go.wigust@gmail.com)(address . 30647@debbugs.gnu.org)
874lm06ic9.fsf@gnu.org
Hello,

Oleg Pykhalov <go.wigust@gmail.com> skribis:

Toggle quote (14 lines)
> ludo@gnu.org (Ludovic Courtès) writes:
>
>> Oleg Pykhalov <go.wigust@gmail.com> skribis:
>>
>>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>>> server regardless is it built locally. ‘--log-file’ returns always local
>>> build log file.
>>
>> What did you think of having ‘--log-file’ transparently fall back to
>> searching for log files on substitute servers?
>
> Sorry, I don't understand the question. Does the “fall back” mean the
> behavior before a patch?

To put it differently: what do you dislike about the current behavior?

Toggle quote (8 lines)
>> I find it handy, but also wondered if it might surprise users that such
>> a trivially-looking option connects to external servers. I thought
>> about having it print something when it does so. Would address your
>> concerns?
>
> Do you mean always connect to the external server and print a URL for a
> log file in addition to path of local log file?

No no: keep the current behavior, but print something when we’re looking
for a remote log file (currently it silently checks whether the remote
log file is available.)

Toggle quote (5 lines)
> I don't think mixing those in one output is good, because for example
> you cannot do like:
>
> diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)

I see. I guess I’ve never wanted that, or rather, when I do, I
explicitly wget the remote log file. :-)

So I guess I’m unconvinced about the need for a separate
‘--remote-log-file’ option.

What do people think? Ricardo?

Toggle quote (4 lines)
> As a better approach in addition to ‘--no-substitutes’, maybe we could
> implement ‘--only-substitutes’ (as I remember Nix has it)? Such flag
> will return a remote log file and will avoid building packages locally.

That could be an option, but that’s much more work (not limited to log
file handling.)

Thanks,
Ludo’.
T
T
Tobias Geerinckx-Rice wrote on 1 Mar 2018 16:16
(address . ludo@gnu.org)(address . 30647@debbugs.gnu.org)
940aba4e5d289b70aa499525c0093704@tobias.gr
Hullo,

On 2018-02-28 23:13, ludo@gnu.org wrote:
Toggle quote (6 lines)
> What did you think of having ‘--log-file’ transparently fall back to
> searching for log files on substitute servers?
>
> I find it handy, but also wondered if it might surprise users that such
> a trivially-looking option connects to external servers.

It would pleasantly surprise me :-)

(...well, not really ‘surprise’ — just more Guixy network-transparent
goodness.)

TBH, I don't see the difference between this & all other actions that
already connect to external servers.

Is this different? Will it ignore ’--no-substitutes’, or ping servers
outside of user-approved substitute-urls? Does it need to?

Toggle quote (2 lines)
> I thought about having it print something when it does so.

Like a progress bar? Seems more eye-catching than yet another warning.
Or something more permanent in/next to the downloaded log?

Kind regards,

T G-R

Sent from a Web browser. Excuse or enjoy my brevity.
O
O
Oleg Pykhalov wrote on 1 Mar 2018 16:40
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 30647@debbugs.gnu.org)
87606f3it9.fsf@gmail.com
ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (9 lines)
>>>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>>>> server regardless is it built locally. ‘--log-file’ returns always local
>>>> build log file.
>>>
>>> What did you think of having ‘--log-file’ transparently fall back to
>>> searching for log files on substitute servers?
>
> To put it differently: what do you dislike about the current behavior?

Suppose package build failed locally. I want to receive a log from a
remote server. I could do it manually by:

1. Removing local failed log.
2. ‘wget’, but I need to know a URL.
3. Hydra web interface, which is slow (especially multiple packages).

Toggle quote (4 lines)
> No no: keep the current behavior, but print something when we’re looking
> for a remote log file (currently it silently checks whether the remote
> log file is available.)

Still not clear to me. If ‘guix --log-file’ checks for a remote log
file, then it gets a valid URL to a remote build log file for free,
doesn't it?

Toggle quote (8 lines)
>> I don't think mixing those in one output is good, because for example
>> you cannot do like:
>>
>> diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)
>
> I see. I guess I’ve never wanted that, or rather, when I do, I
> explicitly wget the remote log file. :-)

Could I ask What's your workflow for ‘wget’?

Toggle quote (5 lines)
> So I guess I’m unconvinced about the need for a separate
> ‘--remote-log-file’ option.
>
> What do people think? Ricardo?

Maybe CC him? Or is it a bad etiquette for a mailing list, because he
is subscribed?

Toggle quote (7 lines)
>> As a better approach in addition to ‘--no-substitutes’, maybe we could
>> implement ‘--only-substitutes’ (as I remember Nix has it)? Such flag
>> will return a remote log file and will avoid building packages locally.
>
> That could be an option, but that’s much more work (not limited to log
> file handling.)

Yes, but benefits (especially avoid building packages locally) are
worth.

If you don't agree with the patch, I'll not complain and will try to
work on ‘--only-substitutes’. :-)

Oleg.
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEc+OyAXw1EaDPCmAPckbhHGm3lWkFAlqYHwIACgkQckbhHGm3
lWnx8w//Zve5wIw99HUQccFzPYQT016DdInm6OVQ3uDQLNT9c+3EjG43j7E21df0
qJJkMj+iUWGyc9P3Wny8yF4jU2Pemw+sz3WSGH5czhD4/a+5XE4FfnkT/aQhTV4N
zbQzbsYwUxxPC4MJVLP3xpoN48DAGGRA4s11cYtELKnfSpsqByENfPkJ2JDYU0vj
FmYHu97LTSrdX49NMwTof2PtJeshtR7sLUMXD2SkKixj+LfRVv+jafbS29gk71vr
LqIydTuUuHArAdzdb9/CWtmzY1stEd0h0HAtuySGL7BRitOu+i/rz8uj96vlxYrm
/gOPcGIbYHfhZUNt61yP9H8SGGgLiCeHV2CgfpzItp7Cea9IO8v+/O2x3Rpw3bXr
a/yILKVZO4p/BLAjVGxNMKBH1P5Sy+Tl1lzLgwvjkL0JYXzKILYsZaP71nO8WHJ2
2KVjG6Eth6aBXtHkL6rvOK3qZFC9hYYk46SYoKb9Z++AaioNK9JaARYLzZLBFcmv
uAWQ+vgjpAiKnPLXtOjcVt1iLw7uaE9X9e397FFWid94wlidCzgqu7OQnp6R10Fr
gXrvd6QgREewUU7P13eH1s4IKDgs8WDP+mbnzV8Rm+nBHqqWi3PNXzCMTxtHzORI
cWFhCVKZ1YM0/2oapNWH5bIcpEvy202gBmNPEGyHmPRv+oQ2uWQ=
=aijF
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 1 Mar 2018 22:23
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)(address . 30647@debbugs.gnu.org)
87d10n4hil.fsf@gnu.org
Hey Tobias,

Tobias Geerinckx-Rice <me@tobias.gr> skribis:

Toggle quote (9 lines)
> On 2018-02-28 23:13, ludo@gnu.org wrote:
>> What did you think of having ‘--log-file’ transparently fall back to
>> searching for log files on substitute servers?
>>
>> I find it handy, but also wondered if it might surprise users that such
>> a trivially-looking option connects to external servers.
>
> It would pleasantly surprise me :-)

Good. :-)

Toggle quote (6 lines)
> TBH, I don't see the difference between this & all other actions that
> already connect to external servers.
>
> Is this different? Will it ignore ’--no-substitutes’, or ping servers
> outside of user-approved substitute-urls? Does it need to?

It’s not really different, indeed. (The only thing is that the list of
substitute URLs that is used by ‘--log-file’ can differ from that of
guix-daemon; but that’s a bug, really.)

Toggle quote (5 lines)
>> I thought about having it print something when it does so.
>
> Like a progress bar? Seems more eye-catching than yet another
> warning. Or something more permanent in/next to the downloaded log?

‘--log-file’ just probes remote servers without actually downloading the
log, so a progress bar wouldn’t be useful.

I was thinking of a message like:

info: looking for build log at https://…/log/…-foo

Ludo’.
L
L
Ludovic Courtès wrote on 1 Mar 2018 22:30
(name . Oleg Pykhalov)(address . go.wigust@gmail.com)(address . 30647@debbugs.gnu.org)
878tbb4h7f.fsf@gnu.org
Oleg Pykhalov <go.wigust@gmail.com> skribis:

Toggle quote (17 lines)
> ludo@gnu.org (Ludovic Courtès) writes:
>
>>>>> ‘--remote-log-file’ allows to get a URL for a build log file on a substitute
>>>>> server regardless is it built locally. ‘--log-file’ returns always local
>>>>> build log file.
>>>>
>>>> What did you think of having ‘--log-file’ transparently fall back to
>>>> searching for log files on substitute servers?
>>
>> To put it differently: what do you dislike about the current behavior?
>
> Suppose package build failed locally. I want to receive a log from a
> remote server. I could do it manually by:
>
> 1. Removing local failed log.
> 2. ‘wget’, but I need to know a URL.

The URL scheme is documented and easy to use (info "(guix) Invoking guix
publish"). That’s why I don’t find wget to be much of a problem.

Toggle quote (8 lines)
>> No no: keep the current behavior, but print something when we’re looking
>> for a remote log file (currently it silently checks whether the remote
>> log file is available.)
>
> Still not clear to me. If ‘guix --log-file’ checks for a remote log
> file, then it gets a valid URL to a remote build log file for free,
> doesn't it?

Correct. See ‘log-url’ in (guix scripts build).

Toggle quote (10 lines)
>>> I don't think mixing those in one output is good, because for example
>>> you cannot do like:
>>>
>>> diff -u <(guix build --log-file hello) <(guix build --remote-log-file hello)
>>
>> I see. I guess I’ve never wanted that, or rather, when I do, I
>> explicitly wget the remote log file. :-)
>
> Could I ask What's your workflow for ‘wget’?

Something like:

$ guix build foo
/gnu/store/xyz-foo
Toggle quote (13 lines)
>>> As a better approach in addition to ‘--no-substitutes’, maybe we could
>>> implement ‘--only-substitutes’ (as I remember Nix has it)? Such flag
>>> will return a remote log file and will avoid building packages locally.
>>
>> That could be an option, but that’s much more work (not limited to log
>> file handling.)
>
> Yes, but benefits (especially avoid building packages locally) are
> worth.
>
> If you don't agree with the patch, I'll not complain and will try to
> work on ‘--only-substitutes’. :-)

Heheh. :-)

IIRC, --only-substitutes in Nix is used together with --upgrade, no?
Some considered it questionable from a security standpoint when we last

Thanks,
Ludo’.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 30647@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 30647
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch