Combining file-append with gexps results in incomprehensible errors

  • Done
  • quality assurance status badge
Details
4 participants
  • Brian Cully
  • Jakub K?dzio?ka
  • Maxim Cournoyer
  • Maxime Devos
Owner
unassigned
Submitted by
Jakub K?dzio?ka
Severity
normal
J
J
Jakub K?dzio?ka wrote on 2 Jul 2020 14:00
(address . bug-guix@gnu.org)
20200702120029.vglwgexyasdzhgll@gravity
Consider this minimal operating-system definition:

(use-modules (gnu))
(use-package-modules gcc)

(operating-system
(host-name "test")
(timezone "UTC")
(bootloader (bootloader-configuration
(bootloader grub-efi-bootloader)
(target "/boot/efi")))
(file-systems (cons*
(file-system
(device (file-system-label "root"))
(mount-point "/")
(type "ext4"))
%base-file-systems))
(packages '())
(services (cons*
(service special-files-service-type
`(("/lib64" ,(directory-union "rustup-libs"
(list
(file-append glibc "/lib")
(file-append #~#$gcc:lib "/lib"))))))
%base-services)))

I would expect this way of specifying a specific output of a package to
work, but it results in the following error:

ice-9/boot-9.scm:1515:18: object is not an exception of the right type #<&gexp-input-error input: #<gexp #<gexp-input #<package gcc@7.5.0 gnu/packages/gcc.scm:520 7f06c996c960>:lib> 7f06c6b06990>> #<record-type &package-input-error>

This also happens when I omit the directory-union:

(service special-files-service-type
`(("/lib64" ,(file-append #~#$gcc:lib "/lib"))))

What *does* work, is this variant with string-append:

(service special-files-service-type
`(("/lib64" ,#~(string-append #$gcc:lib "/lib"))))

However, using it in the directory-union breaks again:

(service special-files-service-type
`(("/lib64" ,(directory-union "rustup-libs"
(list
(file-append glibc "/lib")
#~(string-append #$gcc:lib "/lib"))))))

ERROR: In procedure opendir:
Wrong type (expecting string): (string-append "/gnu/store/mdxmdhrlkgdik6ay9pzmmy8mjcbibpwb-gcc-7.5.0-lib" "/lib")
builder for `/gnu/store/p5hf7hqxn35fgsb75s5i7326vyzb8jkr-rustup-libs.drv' failed with exit code 1

I have figured out that the following does work:

(service special-files-service-type
`(("/lib64" ,#~(directory-union "rustup-libs"
(list
(string-append #$glibc "/lib")
(string-append #$gcc:lib "/lib"))))))

However, I would expect the other variants to work as well. The
documentation and error messages are lacking in this regard.

Regards,
Jakub K?dzio?ka
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEE5Xa/ss9usT31cTO54xWnWEYTFWQFAl79zF0ACgkQ4xWnWEYT
FWRBAxAA0rX9+b79yvNhYWwxykCeKv2rd1YxjyEvkQdfNJ1ZUbaVORbc5AhOWrum
QBe/74r5HM6Hgt6KHEDRtI3FIR7hAQ+6ufz6w5RkVCiO/pLq3iin6kg/9neDwxVC
F6Mo4WXu8bQRg7QUIGTxEwpyrwq2qYYGtNn9FQ5eYxQnx8uziDWOid1N5GPQIWyq
jMWBHtMyDbw7F3AYFYh913NV3qPYMym4uKWMHazpBAwf3gF1+o4xnUu1G8MDfSWT
k/TvNUNWYNuI7JTvrcmb/i/A0bkCSFgKUMvlqeoyPpwmM1SSjz59/tYZv6xzWrXB
Zgp7+f6ksREpIbcs5EMkg1XG9IYXLG/B7HIYHWoV/mAvXA6f3t21q+FoPKfOrwrz
bDSkuB1+ZhI8JLFvpC/YQ1a5qKz8pGGsPvcxZ2cqU83eFRIZdWDKXfsObewnjTaF
BpqNdIV3WXH97PF7z65dFWc9fG+95IoeLfcZw9YqXt49+45XbzQA1+gkMrAdRY4o
9sdjwCMI/lxppHeUxJn/bHlQx/a6GX9EMHl9VlPYgIUICtPSIQyHw4KB4g3f8fee
sUhMOuAYA+6i32P5erRraaaqK/7KUcJm0r1e/0xOJAdtaOeXzia99+6Mj+f6Tcc1
Vru+BWD6SVSwjaY6OgPFjjI8jIcFBdf2frxiwE0+wBArth6hM54=
=Ppg3
-----END PGP SIGNATURE-----


M
M
Maxim Cournoyer wrote on 13 Apr 2021 03:58
(name . Jakub K?dzio?ka)(address . kuba@kadziolka.net)(address . 42164@debbugs.gnu.org)
874kgbc5de.fsf@gmail.com
Hi Jakub!

Jakub K?dzio?ka <kuba@kadziolka.net> writes:

Toggle quote (31 lines)
> Consider this minimal operating-system definition:
>
> (use-modules (gnu))
> (use-package-modules gcc)
>
> (operating-system
> (host-name "test")
> (timezone "UTC")
> (bootloader (bootloader-configuration
> (bootloader grub-efi-bootloader)
> (target "/boot/efi")))
> (file-systems (cons*
> (file-system
> (device (file-system-label "root"))
> (mount-point "/")
> (type "ext4"))
> %base-file-systems))
> (packages '())
> (services (cons*
> (service special-files-service-type
> `(("/lib64" ,(directory-union "rustup-libs"
> (list
> (file-append glibc "/lib")
> (file-append #~#$gcc:lib "/lib"))))))
> %base-services)))
>
> I would expect this way of specifying a specific output of a package to
> work, but it results in the following error:
>
> ice-9/boot-9.scm:1515:18: object is not an exception of the right type #<&gexp-input-error input: #<gexp #<gexp-input #<package gcc@7.5.0 gnu/packages/gcc.scm:520 7f06c996c960>:lib> 7f06c6b06990>> #<record-type &package-input-error>

[...]

Toggle quote (11 lines)
> I have figured out that the following does work:
>
> (service special-files-service-type
> `(("/lib64" ,#~(directory-union "rustup-libs"
> (list
> (string-append #$glibc "/lib")
> (string-append #$gcc:lib "/lib"))))))
>
> However, I would expect the other variants to work as well. The
> documentation and error messages are lacking in this regard.

This seems to come up every time I have to specify a non-default package
output; I've investigated the issue a bit and it seems to be caused by
the lack of a gexp-compiler for the gexp object resulting from
#$package:output, which is of type gexp-input as represented by, for
example:

#<gexp-input #<package opendht@2.2.0rc4 /home/maxim/src/guix/gnu/packages/networking.scm:3426 7f2dbcae1aa0>:tools>

From the REPL

(lower-object (gexp (ungexp opendht "tools")))
$17 = #<procedure 7f2db56ed100 at guix/gexp.scm:246:2 (state)>
scheme@(gnu services jami)> ,run-in-store $17
While executing meta-command:
ERROR:
1. &gexp-input-error: #<gexp #<gexp-input #<package opendht@2.2.0rc4
/home/maxim/src/guix/gnu/packages/networking.scm:3426
7f2dbcae1aa0>:tools> 7f2db7427690>
So if we were to define/register a gexp-compiler for this kind of
gexp-input, we could make it work, but I haven't yet investigated how
easy/difficult that would be.

Seems worthwhile though, this issue is quite annoying when attempting to
work with specific outputs!

Thanks for the detailed report.

Maxim
B
B
Brian Cully wrote on 23 Apr 2022 18:03
Combining file-append with gexps results in incomprehensible errors
(address . 42164@debbugs.gnu.org)
87k0bfkca3.fsf@ditto.jhoto.spork.org
I was having a similar issue, so I dug into this issue a bit,
and it turns out that you *can* select an output from file-append, but
the syntax is a bit wonky:

---[snip]---
(let ((fa (file-append pkg "/path")))
$~$#fa:output)
---[snip]---

I’ve tried to remove the let:

---[snip]---
$~$#(file-append pkg "/path"):output
---[snip]---

But this confuses the reader, an it things ‘:output’ is a
variable. Not too surprising, but it does make the Guix configuration
files more awkward, since now you have to ‘define’ the file-append
operations at the top level so you can refer to them with the syntax
above.

I would prefer to have something like any of the following (in
order of preference):

---[snip]---
(file-append pkg:output "/path")
(file-append (list pkg "output") "/path")
(file-append $~$#pkg:output "/path")
---[snip]---

The reason being that it’s awkward to select the output after
the file-append due to syntax, and it’s less intuitive than trying to
select the output directly from the package inside the file-append (and
it seems I’m not alone in this). I’ve been through the code, so to some
degree I realize why things are being done the way they are, but that
doesn’t change my current preferences.

I prefer the list over the gexp/ungexp because we already use
lists to select outputs in other parts of the config declarations, and
the gexp/ungexp stuff is harder to understand from the point-of-view of
a user, rather than package author. It’s also not present in the
majority of configs, I’d wager.

Before I found the “correct” way to do things, I did cobble
together some code that allowed the (list pkg "output") form to work,
and would be happy to submit it upstream. It’s not perfect, since it
ignores the ‘output’ argument in the gexp expander in favor of the
stored output in the file-append, but I don’t think that’s an actual
issue in practice.

-bjc
M
M
Maxime Devos wrote on 23 Apr 2022 19:06
c40e113e9455f467f2e8b2bf11d8c3ab087e0fd4.camel@telenet.be
Brian Cully schreef op za 23-04-2022 om 12:03 [-0400]:
Toggle quote (15 lines)
> I was having a similar issue, so I dug into this issue a bit,
> and it turns out that you *can* select an output from file-append, but
> the syntax is a bit wonky:
>
> ---[snip]---
> (let ((fa (file-append pkg "/path")))
>   $~$#fa:output)
> ---[snip]---
>
> I’ve tried to remove the let:
>
> ---[snip]---
> $~$#(file-append pkg "/path"):output
> ---[snip]---

Do you mean #~#$fa:output here?

Toggle quote (13 lines)
>
> But this confuses the reader, an it things ‘:output’ is a
> variable. Not too surprising, but it does make the Guix configuration
> files more awkward, since now you have to ‘define’ the file-append
> operations at the top level so you can refer to them with the syntax
> above.
>
> I would prefer to have something like any of the following (in
> order of preference):
>
> ---[snip]---
> (file-append pkg:output "/path")

This one is only possible if file-append becomes syntax instead of a
procedure.

Toggle quote (2 lines)
> (file-append (list pkg "output") "/path")

Possible, but seems semantically wrong to me -- what's a list doing in
a file-manipulating procedure?

Toggle quote (2 lines)
> (file-append $~$#pkg:output "/path")

I assume this would be #~#$pkg:output?

My bikeshed proposal:

(file-append (output glibc "static") "/lib/...")

Here, 'output' is a procedure constructing some kind of object.
This would also be useful outside 'file-append', as it allows
programmatically constructing equivalents #~#$pkg:output for multiple
'output' (there was some package definition some while back where this
would have been useful).

This 'output' could perhaps also be used in 'inputs' / 'native-inputs'
fields. YMMV.

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYmQyGhccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7ijSAP41ydOYBlz6vXJiKxcFUTy4DQCl
qYZg5zCfRoQwAr4mhgD/U2jNkxQsin69eY38p0aauSPaP5aaQnzuX59Opkqp7Qs=
=HXtc
-----END PGP SIGNATURE-----


M
M
Maxime Devos wrote on 23 Apr 2022 20:38
(name . Brian Cully)(address . bjc@spork.org)(address . 42164@debbugs.gnu.org)
2bda48c996ac6b0bae970d8e5559675d6984ac51.camel@telenet.be
Brian Cully schreef op za 23-04-2022 om 13:29 [-0400]:
Toggle quote (5 lines)
> I like this, and would prefer it over the ‘list’ proposal. I’m
> not sure how to implement it given my limited knowledge of Guix
> internals, though. Special-casing a list inside of file-append was
> fairly straightforward and solved the immediate issue.

Proposed implementation:

(define (output thing output)
(gexp-input package output))

Seems like this thing already exists, it's just not well-known and not
documented in the manual.

Greetings,
Maxime
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYmRHlBccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7mURAQCf5anTWJ2mMopZRSg9neOe9B/7
O7uNougodALjQL3PHwD/VwSul5BtVSAbFtijCRGScc3VXEnQqkQwmNBMT4y76wQ=
=dKxa
-----END PGP SIGNATURE-----


M
M
Maxime Devos wrote on 23 Apr 2022 23:07
(name . Brian Cully)(address . bjc@spork.org)(address . 42164@debbugs.gnu.org)
5be60a6138fde8bfe521e7e9c7c18f28a9c50742.camel@telenet.be
Brian Cully schreef op za 23-04-2022 om 16:40 [-0400]:
Toggle quote (2 lines)
> Where is this in the code?

Nowhere. I tried implementing 'output', and noticed I just ended up
with something equivalent to a simple call to 'gexp-input' (hence,
‘this thing already exists’). 'gexp-input' can be found in (guix
gexp).

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYmRqfBccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7nDlAQCATS8e50o67kiyOwdNYBwSgm0y
PYxHuFmULe7rZ/YUfQEA5EC+0TC29tGJh+F9WNM65OfPPNjOMDOx4QnzgkFNIAM=
=oUPK
-----END PGP SIGNATURE-----


B
B
Brian Cully wrote on 23 Apr 2022 19:29
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 42164@debbugs.gnu.org)
87bkwrk7zk.fsf@ditto.jhoto.spork.org
Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (2 lines)
> Do you mean #~#$fa:output here?

Yes, I did. Sorry, my fingers have minds of their own, and the
gexp syntax, in particular, really runs counter to their memory.

Toggle quote (5 lines)
>> (file-append pkg:output "/path")
>
> This one is only possible if file-append becomes syntax instead of a
> procedure.

I did not expect this to be particularly possible, but I find it
aesthetically and intuitively pleasing enough to propose it anyway.

Toggle quote (5 lines)
>> (file-append (list pkg "output") "/path")
>
> Possible, but seems semantically wrong to me -- what's a list doing in
> a file-manipulating procedure?

It need not be a list, but I do think it makes sense inside the
file-append, rather than a modification to file-append’s results, as is
currently the case. I picked list because it was being used elsewhere to
explicitly select a derivation’s output.

Toggle quote (13 lines)
> My bikeshed proposal:
>
> (file-append (output glibc "static") "/lib/...")
>
> Here, 'output' is a procedure constructing some kind of object.
> This would also be useful outside 'file-append', as it allows
> programmatically constructing equivalents #~#$pkg:output for multiple
> 'output' (there was some package definition some while back where this
> would have been useful).
>
> This 'output' could perhaps also be used in 'inputs' / 'native-inputs'
> fields. YMMV.

I like this, and would prefer it over the ‘list’ proposal. I’m
not sure how to implement it given my limited knowledge of Guix
internals, though. Special-casing a list inside of file-append was
fairly straightforward and solved the immediate issue.

I’d be happy to take a look at implementing this, but I’m a bit
clueless where to begin, as it’s much more general than my previous
attempt. If you (or others) have the time, I’d appreciate any pointers
that’d make it easier to begin.

For instance: I would expect the ‘output’ form to be lowerable,
which would be done in the ‘compile’ branch of a gexp-compiler, but then
it would have to be expanded later in the ‘expand’ branch, which
presents an issue: the third argument to the expander procedure is
already an output. Would we do away with that argument? Somehow fill it
in based on the ‘output’ form (either expressed or implicit)? Let the
‘output’ form, if present, override it?

-bjc
B
B
Brian Cully wrote on 23 Apr 2022 22:40
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 42164@debbugs.gnu.org)
877d7f7cn4.fsf@ditto.jhoto.spork.org
Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (8 lines)
> Proposed implementation:
>
> (define (output thing output)
> (gexp-input package output))
>
> Seems like this thing already exists, it's just not well-known and not
> documented in the manual.

Where is this in the code? I couldn’t find it myself (grepping
for these keywords turns up a lot of false positives). On the off chance
it might work, I did try ‘(file-append (output pkg "lib") "/path")’ and
was told that ‘output’ is an unbound variable.

-bjc
B
B
Brian Cully wrote on 25 Apr 2022 01:27
(name . Maxime Devos)(address . maximedevos@telenet.be)(address . 42164@debbugs.gnu.org)
87bkwqqckg.fsf@ditto.jhoto.spork.org
Maxime Devos <maximedevos@telenet.be> writes:

Toggle quote (8 lines)
> Nowhere. I tried implementing 'output', and noticed I just
> ended up
> with something equivalent to a simple call to 'gexp-input'
> (hence,
> ‘this thing already exists’). 'gexp-input' can be found in
> (guix
> gexp).

I finally had a chance to mess around with this, and it
does, indeed, do the job. Since it wasn’t clear to me how
it worked, I’d like to document it here.

To accomplish the goal at the begging of this thread,
rather than using:
Toggle snippet (3 lines)
(file-append #~#$gcc:lib “/lib”)

The “correct” way is this:
Toggle snippet (3 lines)
(gexp-input (file-append gcc "/lib") "lib")

The name seems a little weird, since we’re selecting an
output, but it makes sense from the other end: we’re
creating an input for the higher-level expression (in this
case, the special-file-services-type).

Anyway, this works for me, so this ticket can probably be
closed.

-bjc
M
M
Maxime Devos wrote on 25 Apr 2022 08:20
(name . Brian Cully)(address . bjc@spork.org)(address . 42164-done@debbugs.gnu.org)
c386156bf169235477ba9bd20c67ae2126a5b711.camel@telenet.be
Brian Cully schreef op zo 24-04-2022 om 19:27 [-0400]:
Toggle quote (9 lines)
> --8<---------------cut here---------------start------------->8---
> (file-append #~#$gcc:lib “/lib”)
> --8<---------------cut here---------------end--------------->8---
>
> The “correct” way is this:
> --8<---------------cut here---------------start------------->8---
> (gexp-input (file-append gcc "/lib") "lib")
> --8<---------------cut here---------------end--------------->8---

FWIW, I was actually thinking of

(file-append (gexp-input gcc "lib") "/lib")

(First select the output with 'gexp-input', then append a suffix.)

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYmY9yRccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7mSCAPwKWvIc4DAN93THkZgbxPIfxJuq
dN30/7NT/X4VQQCBwgEAwk6vULzBXY+n+Mg0GV0hy2AhQoqrjT/EMK2GPvexpwc=
=676A
-----END PGP SIGNATURE-----


Closed
?