[tentative PATCH] Failure to guix pull on aarch64 since recent make-linux-libre*

Done

Details

3 participants

Ludovic Courtès
Maxim Cournoyer
Pierre Langlois

Owner: unassigned

Submitted by: Pierre Langlois

Severity: important

Pierre Langlois wrote on 9 Dec 2022 00:31

Recipients:(address . bug-guix@gnu.org)

Message-ID:87k031xxss.fsf@gmx.com

Hi Guix!

I've been getting errors while running `guix pull' on an aarch64 system,

during the final guix-package-cache step:

Toggle snippet (43 lines)(repl-version 0 1 1)
Generating package cache for '/gnu/store/m8in1imi93snq711d7568dj9hlrx4diz-profile'...

Backtrace:
In ice-9/boot-9.scm:
  1747:15 19 (with-exception-handler #<procedure af1570 at ice-9/bo?> ?)
  1752:10 18 (with-exception-handler _ _ #:unwind? _ # _)
In guix/repl.scm:
    99:21 17 (_)
In unknown file:
          16 (_ #<procedure 82fd00 at guix/repl.scm:100:25 ()> #<pr?> ?)
          15 (primitive-load "/gnu/store/3x6g541ixbmdjav4ky6dp1ryj4l?")
In ice-9/boot-9.scm:
  1752:10 14 (with-exception-handler _ _ #:unwind? _ # _)
In gnu/packages.scm:
   438:11 13 (generate-package-cache _)
In srfi/srfi-1.scm:
   460:18 12 (fold #<procedure expand-cache expr> _ _)
In gnu/packages.scm:
    390:9 11 (expand-cache . _)
In guix/packages.scm:
  1317:17 10 (supported-package? #<package linux-libre@4.14.300 gnu?> ?)
In guix/memoization.scm:
    101:0  9 (_ #<hash-table 31605e0 13974/28099> #<package linux-l?> ?)
In guix/packages.scm:
  1295:37  8 (_)
  1555:16  7 (package->bag _ _ _ #:graft? _)
  1660:43  6 (thunk)
In gnu/packages/linux.scm:
   986:37  5 (arguments #<package linux-libre@4.14.300 gnu/packages/?>)
In guix/gexp.scm:
   460:52  4 (%local-file #f #<promise #<procedure 4df2660 at gnu/p?> ?)
In unknown file:
           3 (basename #f #<undefined>)
In ice-9/boot-9.scm:
  1685:16  2 (raise-exception _ #:continuable? _)
  1780:13  1 (_ #<&compound-exception components: (#<&assertion-fail?>)
In unknown file:
           0 (backtrace #<undefined>)

(exception wrong-type-arg (value "scm_to_utf8_stringn") (value "Wrong type argument in position ~A (expecting ~A): ~S") (value (1 "string" #f)) (value (#f)))

I was able to decipher the backtrace to *maybe* put together a fix, but
I'm unsure why the problem started. My best guess is that it started
with commit dfc6957a5af7d179d4618eb19d4f555c519bc6f2, even though I
can't find where the issue actually is, it looks fine to me!

What seems to happen is that the `kernel-config' function now receive an
`arch' argument for an architecture that isn't actually supported by
that kernel, as is the case for linux-libre@4.14.300.  And, correctly,
the function should not expect to ever get such arch value to begin
with, so we get a `(local-file #f)'.

Toggle snippet (9 lines)

(define* (kernel-config arch #:key variant)

"Return a file-like object of the Linux-Libre build configuration file for

ARCH and optionally VARIANT, or #f if there is no such configuration."

(let* ((name (string-append (if variant (string-append variant "-") "")

(if (string=? "i386" arch) "i686" arch) ".conf"))

(file (string-append "linux-libre/" name)))

(local-file (search-auxiliary-file file))))

I think it's fair for that function expect the arch to be valid (why

would you ask the config for an unsupported arch?).

I think it should be possible to fix this by checking the arch is

supported at the call site:

From 77829140f14928e30cbe4e53c625be3ba2f5895f Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois@gmx.com>
Date: Thu, 8 Dec 2022 23:41:40 +0000
Subject: [PATCH] gnu: make-linux-libre*: Do not get config for unsupported
 systems.

* gnu/packages/linux.scm (make-linux-libre*)[phases] <configure>: Check
arch is in supported-systems before calling configuration-file.
---
 gnu/packages/linux.scm | 1 +
 1 file changed, 1 insertion(+)

Toggle diff (14 lines)diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index 5ae6366593..87fc9fe94c 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -983,6 +983,7 @@ (define* (make-linux-libre* version gnu-revision source supported-systems
                                             (or (%current-target-system)
                                                 (%current-system))))))
                                 (and configuration-file arch
+                                     (member arch supported-systems)
                                      (configuration-file
                                       arch
                                       #:variant (version-major+minor version))))
-- 
2.38.1

But I'm not quite sure why this is happening, some quirk from moving

things over gexps?

I'm currently trying this fix to make sure it does solve the problem

(guix pull takes so long without substitutes :-) ). Will report back in

5-10 minutes.

Thanks!

Pierre

Pierre Langlois wrote on 9 Dec 2022 00:59

Recipients:(address . 59913@debbugs.gnu.org)

Message-ID:877cz1xx59.fsf@gmx.com

Pierre Langlois <pierre.langlois@gmx.com> writes:

(snip)

Toggle quote (10 lines)> What seems to happen is that the `kernel-config' function now receive an
> `arch' argument for an architecture that isn't actually supported by
> that kernel, as is the case for linux-libre@4.14.300.  And, correctly,
> the function should not expect to ever get such arch value to begin
> with, so we get a `(local-file #f)'.
>
> (define* (kernel-config arch #:key variant)
>   "Return a file-like object of the Linux-Libre build configuration file for
> ARCH and optionally VARIANT, or #f if there is no such configuration."

I pasted this bit of code but didn't read it properly, it specificaly

says that it should return #f if the config doesn't exist. I guess

that's now a bug? We could either fix the docstring and apply the fix I

proposed, or returning #f probably works.

Toggle quote (36 lines)>   (let* ((name (string-append (if variant (string-append variant "-") "")
>                               (if (string=? "i386" arch) "i686" arch) ".conf"))
>          (file (string-append "linux-libre/" name)))
>     (local-file (search-auxiliary-file file))))
>
> I think it's fair for that function expect the arch to be valid (why
> would you ask the config for an unsupported arch?).
>
> I think it should be possible to fix this by checking the arch is
> supported at the call site:
>
> [[End of PGP Signed Part]]
> From 77829140f14928e30cbe4e53c625be3ba2f5895f Mon Sep 17 00:00:00 2001
> From: Pierre Langlois <pierre.langlois@gmx.com>
> Date: Thu, 8 Dec 2022 23:41:40 +0000
> Subject: [PATCH] gnu: make-linux-libre*: Do not get config for unsupported
>  systems.
>
> * gnu/packages/linux.scm (make-linux-libre*)[phases] <configure>: Check
> arch is in supported-systems before calling configuration-file.
> ---
>  gnu/packages/linux.scm | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
> index 5ae6366593..87fc9fe94c 100644
> --- a/gnu/packages/linux.scm
> +++ b/gnu/packages/linux.scm
> @@ -983,6 +983,7 @@ (define* (make-linux-libre* version gnu-revision source supported-systems
>                                              (or (%current-target-system)
>                                                  (%current-system))))))
>                                  (and configuration-file arch
> +                                     (member arch supported-systems)
>                                       (configuration-file
>                                        arch
>                                        #:variant (version-major+minor version))))

Pierre Langlois wrote on 9 Dec 2022 01:36

Recipients:(address . 59913@debbugs.gnu.org)

Message-ID:87tu25wgx5.fsf@gmx.com

Pierre Langlois <pierre.langlois@gmx.com> writes:

(snip)

Toggle quote (4 lines)

> I'm currently trying this fix to make sure it does solve the problem

> (guix pull takes so long without substitutes :-) ). Will report back in

> 5-10 minutes.

Yeey the fixed worked!

Ludovic Courtès wrote on 9 Dec 2022 17:56

control message for bug #59913

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87edt87bxi.fsf@gnu.org

severity 59913 important

quit

Pierre Langlois wrote on 9 Dec 2022 20:18

Re: branch master updated: Revert "gnu: make-linux-libre*: Remove input labels."

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 59913@debbugs.gnu.org)

Message-ID:87k030wfhu.fsf@gmx.com

Hi Maxim,

guix-commits@gnu.org writes:

Toggle quote (19 lines)> This is an automated email from the git hooks/post-receive script.
>
> apteryx pushed a commit to branch master
> in repository guix.
>
> The following commit(s) were added to refs/heads/master by this push:
>      new 0ffa501f2b Revert "gnu: make-linux-libre*: Remove input labels."
> 0ffa501f2b is described below
>
> commit 0ffa501f2b3e83ae56e9c2bd31418439090e869a
> Author: Maxim Cournoyer <maxim.cournoyer@gmail.com>
> AuthorDate: Fri Dec 9 08:16:07 2022 -0500
>
>     Revert "gnu: make-linux-libre*: Remove input labels."
>     
>     This reverts commit dfc6957a5af7d179d4618eb19d4f555c519bc6f2.  git-bisect
>     found it broke guix pull on non-x86_64 systems like powerpc64le, for reasons
>     to investigate.

In case you've not seen, I noticed this as well and I /think/ worked out

what went wrong on https://issues.guix.gnu.org/59913.

Hopefully the report is helpful and you didn't spend too much time on it

if you've not seen it!

Thanks,

Pierre

Maxim Cournoyer wrote on 9 Dec 2022 21:06

Recipients:(name . Pierre Langlois)(address . pierre.langlois@gmx.com)(address . 59913@debbugs.gnu.org)

Message-ID:87ilik5oks.fsf@gmail.com

Hi Pierre,

Pierre Langlois <pierre.langlois@gmx.com> writes:

Toggle quote (29 lines)> Hi Maxim,
>
> guix-commits@gnu.org writes:
>
>> This is an automated email from the git hooks/post-receive script.
>>
>> apteryx pushed a commit to branch master
>> in repository guix.
>>
>> The following commit(s) were added to refs/heads/master by this push:
>>      new 0ffa501f2b Revert "gnu: make-linux-libre*: Remove input labels."
>> 0ffa501f2b is described below
>>
>> commit 0ffa501f2b3e83ae56e9c2bd31418439090e869a
>> Author: Maxim Cournoyer <maxim.cournoyer@gmail.com>
>> AuthorDate: Fri Dec 9 08:16:07 2022 -0500
>>
>>     Revert "gnu: make-linux-libre*: Remove input labels."
>>     
>>     This reverts commit dfc6957a5af7d179d4618eb19d4f555c519bc6f2.  git-bisect
>>     found it broke guix pull on non-x86_64 systems like powerpc64le, for reasons
>>     to investigate.
>
> In case you've not seen, I noticed this as well and I /think/ worked out
> what went wrong on https://issues.guix.gnu.org/59913.
>
> Hopefully the report is helpful and you didn't spend too much time on it
> if you've not seen it!

Thanks for finding a solution!  I was quite puzzled by the failure and
didn't have the time to investigate.  We could rebase it on master with
a patch reverting the revert, and post it here to see what QA thinks
about it.

-- 
Thanks,
Maxim

Pierre Langlois wrote on 9 Dec 2022 22:33

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)

Message-ID:87sfhouskl.fsf@gmx.com

Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:

Toggle quote (38 lines)> Hi Pierre,
>
> Pierre Langlois <pierre.langlois@gmx.com> writes:
>
>> Hi Maxim,
>>
>> guix-commits@gnu.org writes:
>>
>>> This is an automated email from the git hooks/post-receive script.
>>>
>>> apteryx pushed a commit to branch master
>>> in repository guix.
>>>
>>> The following commit(s) were added to refs/heads/master by this push:
>>>      new 0ffa501f2b Revert "gnu: make-linux-libre*: Remove input labels."
>>> 0ffa501f2b is described below
>>>
>>> commit 0ffa501f2b3e83ae56e9c2bd31418439090e869a
>>> Author: Maxim Cournoyer <maxim.cournoyer@gmail.com>
>>> AuthorDate: Fri Dec 9 08:16:07 2022 -0500
>>>
>>>     Revert "gnu: make-linux-libre*: Remove input labels."
>>>     
>>>     This reverts commit dfc6957a5af7d179d4618eb19d4f555c519bc6f2.  git-bisect
>>>     found it broke guix pull on non-x86_64 systems like powerpc64le, for reasons
>>>     to investigate.
>>
>> In case you've not seen, I noticed this as well and I /think/ worked out
>> what went wrong on https://issues.guix.gnu.org/59913.
>>
>> Hopefully the report is helpful and you didn't spend too much time on it
>> if you've not seen it!
>
> Thanks for finding a solution!  I was quite puzzled by the failure and
> didn't have the time to investigate.  We could rebase it on master with
> a patch reverting the revert, and post it here to see what QA thinks
> about it.

I'm not sure I follow, I'd suggest to revert the revert and then apply a

fix in the same commit, that way it can easily be reverted again if it's

problematic, that's probably what you meant already?

I think the following fix should do it, just to make sure the

`kernel-config' function returns #f on unsupported systems, as per its

docstring:

Toggle diff (19 lines)diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
index 5ae6366593..2d0d1aa29f 100644
--- a/gnu/packages/linux.scm
+++ b/gnu/packages/linux.scm
@@ -781,8 +781,10 @@ (define* (kernel-config arch #:key variant)
 ARCH and optionally VARIANT, or #f if there is no such configuration."
   (let* ((name (string-append (if variant (string-append variant "-") "")
                               (if (string=? "i386" arch) "i686" arch) ".conf"))
-         (file (string-append "linux-libre/" name)))
-    (local-file (search-auxiliary-file file))))
+         (file (string-append "linux-libre/" name))
+         (config (search-auxiliary-file file)))
+    (and config
+         (local-file config))))
 
 (define %default-extra-linux-options
   `(;; Make the kernel config available at /proc/config.gz

WDYT?

Ludovic Courtès wrote on 13 Dec 2022 10:52

Re: bug#59913: [tentative PATCH] Failure to guix pull on aarch64 since recent make-linux-libre*

Recipients:(name . Pierre Langlois)(address . pierre.langlois@gmx.com)

Message-ID:87fsdjy6ja.fsf_-_@gnu.org

Hi,

Pierre Langlois <pierre.langlois@gmx.com> skribis:

Toggle quote (4 lines)

> I'm not sure I follow, I'd suggest to revert the revert and then apply a

> fix in the same commit, that way it can easily be reverted again if it's

> problematic, that's probably what you meant already?

Sounds good to me. The commit log can be similar to the original one

(rather than “Revert: "Revert: "whatever"”), with a couple of lines

like:

This restores commit XYZ, with an additional fix for …

Fixes https://issues.guix.gnu.org/59913.

Toggle quote (19 lines)> I think the following fix should do it, just to make sure the
> `kernel-config' function returns #f on unsupported systems, as per its
> docstring:
>
> diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
> index 5ae6366593..2d0d1aa29f 100644
> --- a/gnu/packages/linux.scm
> +++ b/gnu/packages/linux.scm
> @@ -781,8 +781,10 @@ (define* (kernel-config arch #:key variant)
>  ARCH and optionally VARIANT, or #f if there is no such configuration."
>    (let* ((name (string-append (if variant (string-append variant "-") "")
>                                (if (string=? "i386" arch) "i686" arch) ".conf"))
> -         (file (string-append "linux-libre/" name)))
> -    (local-file (search-auxiliary-file file))))
> +         (file (string-append "linux-libre/" name))
> +         (config (search-auxiliary-file file)))
> +    (and config
> +         (local-file config))))

LGTM!

Thanks,

Ludo’.

Maxim Cournoyer wrote on 13 Jan 2023 16:39

Recipients:(name . Pierre Langlois)(address . pierre.langlois@gmx.com)(address . 59913@debbugs.gnu.org)

Message-ID:87pmbifnpa.fsf@gmail.com

Hi Pierre,

Pierre Langlois <pierre.langlois@gmx.com> writes:

Toggle quote (98 lines)> Hi Guix!
>
> I've been getting errors while running `guix pull' on an aarch64 system,
> during the final guix-package-cache step:
>
> (repl-version 0 1 1)
> Generating package cache for '/gnu/store/m8in1imi93snq711d7568dj9hlrx4diz-profile'...
>
> Backtrace:
> In ice-9/boot-9.scm:
>   1747:15 19 (with-exception-handler #<procedure af1570 at ice-9/bo?> ?)
>   1752:10 18 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/repl.scm:
>     99:21 17 (_)
> In unknown file:
>           16 (_ #<procedure 82fd00 at guix/repl.scm:100:25 ()> #<pr?> ?)
>           15 (primitive-load "/gnu/store/3x6g541ixbmdjav4ky6dp1ryj4l?")
> In ice-9/boot-9.scm:
>   1752:10 14 (with-exception-handler _ _ #:unwind? _ # _)
> In gnu/packages.scm:
>    438:11 13 (generate-package-cache _)
> In srfi/srfi-1.scm:
>    460:18 12 (fold #<procedure expand-cache expr> _ _)
> In gnu/packages.scm:
>     390:9 11 (expand-cache . _)
> In guix/packages.scm:
>   1317:17 10 (supported-package? #<package linux-libre@4.14.300 gnu?> ?)
> In guix/memoization.scm:
>     101:0  9 (_ #<hash-table 31605e0 13974/28099> #<package linux-l?> ?)
> In guix/packages.scm:
>   1295:37  8 (_)
>   1555:16  7 (package->bag _ _ _ #:graft? _)
>   1660:43  6 (thunk)
> In gnu/packages/linux.scm:
>    986:37  5 (arguments #<package linux-libre@4.14.300 gnu/packages/?>)
> In guix/gexp.scm:
>    460:52  4 (%local-file #f #<promise #<procedure 4df2660 at gnu/p?> ?)
> In unknown file:
>            3 (basename #f #<undefined>)
> In ice-9/boot-9.scm:
>   1685:16  2 (raise-exception _ #:continuable? _)
>   1780:13  1 (_ #<&compound-exception components: (#<&assertion-fail?>)
> In unknown file:
>            0 (backtrace #<undefined>)
>
> (exception wrong-type-arg (value "scm_to_utf8_stringn") (value "Wrong type argument in position ~A (expecting ~A): ~S") (value (1 "string" #f)) (value (#f)))
>
>
> I was able to decipher the backtrace to *maybe* put together a fix, but
> I'm unsure why the problem started. My best guess is that it started
> with commit dfc6957a5af7d179d4618eb19d4f555c519bc6f2, even though I
> can't find where the issue actually is, it looks fine to me!
>
> What seems to happen is that the `kernel-config' function now receive an
> `arch' argument for an architecture that isn't actually supported by
> that kernel, as is the case for linux-libre@4.14.300.  And, correctly,
> the function should not expect to ever get such arch value to begin
> with, so we get a `(local-file #f)'.
>
> (define* (kernel-config arch #:key variant)
>   "Return a file-like object of the Linux-Libre build configuration file for
> ARCH and optionally VARIANT, or #f if there is no such configuration."
>   (let* ((name (string-append (if variant (string-append variant "-") "")
>                               (if (string=? "i386" arch) "i686" arch) ".conf"))
>          (file (string-append "linux-libre/" name)))
>     (local-file (search-auxiliary-file file))))
>
> I think it's fair for that function expect the arch to be valid (why
> would you ask the config for an unsupported arch?).
>
> I think it should be possible to fix this by checking the arch is
> supported at the call site:
>
>>From 77829140f14928e30cbe4e53c625be3ba2f5895f Mon Sep 17 00:00:00 2001
> From: Pierre Langlois <pierre.langlois@gmx.com>
> Date: Thu, 8 Dec 2022 23:41:40 +0000
> Subject: [PATCH] gnu: make-linux-libre*: Do not get config for unsupported
>  systems.
>
> * gnu/packages/linux.scm (make-linux-libre*)[phases] <configure>: Check
> arch is in supported-systems before calling configuration-file.
> ---
>  gnu/packages/linux.scm | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm
> index 5ae6366593..87fc9fe94c 100644
> --- a/gnu/packages/linux.scm
> +++ b/gnu/packages/linux.scm
> @@ -983,6 +983,7 @@ (define* (make-linux-libre* version gnu-revision source supported-systems
>                                              (or (%current-target-system)
>                                                  (%current-system))))))
>                                  (and configuration-file arch
> +                                     (member arch supported-systems)
>                                       (configuration-file
>                                        arch
>                                        #:variant (version-major+minor version))))

Thanks for investigating the issue. I'm hitting the same kind of

problem (it seems) trying to migrate make-u-boot-package to gexps, so

I'm revisiting this.

It seems to me that the produced backtrace is overly cryptic; perhaps

configuration-file should throw an exception with a more detailed error

"unsupported architecture -- not generating configuration file" or

similar?

Or should the generate-package-cache machinery catch these #f itself?

That's the same place where it fails in my case with this patch applied:

https://issues.guix.gnu.org/60224#30,and with the reproducer:

Toggle snippet (3 lines)

(package-transitive-supported-systems u-boot-malta (%current-system))

I'm still digging into the issue to get a better understanding.

Thanks,

Maxim

Maxim Cournoyer wrote on 13 Jan 2023 22:04

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:878ri6f8nd.fsf@gmail.com

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (16 lines)> Hi,
>
> Pierre Langlois <pierre.langlois@gmx.com> skribis:
>
>> I'm not sure I follow, I'd suggest to revert the revert and then apply a
>> fix in the same commit, that way it can easily be reverted again if it's
>> problematic, that's probably what you meant already?
>
> Sounds good to me.  The commit log can be similar to the original one
> (rather than “Revert: "Revert: "whatever"”), with a couple of lines
> like:
>
>   This restores commit XYZ, with an additional fix for …
>
>   Fixes <https://issues.guix.gnu.org/59913>.

Commit restored as 4913ac74915c4229aeb3ca26a5f9920c759fb6a3, with

Pierre's fix (thanks!)

Closing.

Thanks,

Maxim

Closed

Your comment

This issue is archived.

To comment on this conversation send an email to 59913@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 59913

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date