'local-file' fails to resolve relative file names with Guile 3.0.8

Aleksandr Vityazev wrote on 14 Feb 22:50 +0100

local-file after Guile update

Recipients:(address . bug-guix@gnu.org)

Message-ID:87zgmtqfsf.fsf@posteo.org

Hi,

I have the following dotfiles directory structure where I store my Guix home

config:

|-etc

|-subdir

| |

| |-home

I often use the 'local-file' procedure in modules in the home directory. For

example in /subdir/home/shells.scm:

#+begin_src scheme :scheme guile :season guile

(simple-service 'add-zsh-files

home-files-service-type

`(("config/zsh/zshrc"

,(local-file "../../etc/zsh/zshrc"))))

#+end_src

And it works, as expected.

The following example also worked before the

076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as

'guile-3.0-latest') commit.

in /subdir/home/wm.scm:

#+begin_src scheme :scheme guile :season guile

(service

home-rofi-service-type

(home-rofi-configuration

(config-rasi

`(,#~(format

#f "@theme \"~a\""

#$(local-file "../../etc/rofi/nord.rasi"))))))

#+end_src

After 076e825dc5d585943ce820a279fffe4af09757fb

using (local-file "../../etc/rofi/nord.rasi") and not

(local-file "etc/rofi/nord.rasi") causes an error:

guix home: error: canonicalize-path: No such file or

directory: "../../etc/rofi/nord.rasi".

The example that causes the error uses the original Guix home implementation,

which uses gexps. When I figure out how to make a minimal reproducer that

does not depend on it, I will update this report.

Best regards,

Aleksandr Vityazev

Liliana Marie Prikler wrote on 15 Feb 09:21 +0100

Recipients:

Message-ID:3bde6d0ba002c1400176b1a2bce83f035afd01ad.camel@ist.tugraz.at

Hi Aleksandr

Am Montag, dem 14.02.2022 um 21:50 +0000 schrieb Aleksandr Vityazev:

Toggle quote (22 lines)> Hi,
> 
> I have the following dotfiles directory structure where I store my
> Guix home config:
> 
> > -etc
> > 
> > -subdir
> >   |
> >   |-home
> 
> 
> I often use the 'local-file' procedure in modules in the home
> directory. For example in /subdir/home/shells.scm:
> 
> #+begin_src scheme :scheme guile :season guile
> (simple-service 'add-zsh-files
>                 home-files-service-type
>                 `(("config/zsh/zshrc"
>                    ,(local-file "../../etc/zsh/zshrc"))))
> #+end_src
> And it works, as expected.

Be careful when invoking Hyrum's Law.  The meaning of this local-file
expression changes with the current working directory, which might be
different across invocations.  To avoid such issues, you can (almost?)
always use (current-file-name), which you might want to canonicalize
for the operations you're about to do.
Taking (dirname) thrice then brings you to $GUIX_HOME_CONFIG_ROOT,
which contains both etc and subdir.

Toggle quote (27 lines)> The following example also worked before the
> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version
> 3.0.8 as
> 'guile-3.0-latest') commit.
> 
> in /subdir/home/wm.scm: 
> #+begin_src scheme :scheme guile :season guile
> (service
>  home-rofi-service-type
>  (home-rofi-configuration
>   (config-rasi
>    `(,#~(format
>          #f "@theme \"~a\""
>          #$(local-file "../../etc/rofi/nord.rasi"))))))
> #+end_src
> 
> After 076e825dc5d585943ce820a279fffe4af09757fb
> using (local-file "../../etc/rofi/nord.rasi") and not
> (local-file "etc/rofi/nord.rasi") causes an error:
> 
> guix home: error: canonicalize-path: No such file or
> directory: "../../etc/rofi/nord.rasi".
> 
> The example that causes the error uses the original Guix home
> implementation, which uses gexps.  When I figure out how to make a
> minimal reproducer that does not depend on it, I will update this
> report.

I don't think this is an issue particularly in Guix Home, but for a MWE
you could try only writing a bashrc, which ought to work with upstream
as well.  Use `guix home build` rather than `guix home reconfigure` so
as to not activate it.

Cheers

Ludovic Courtès wrote on 15 Feb 16:00 +0100

Re: bug#54003: local-file after Guile update

Recipients:(name . Aleksandr Vityazev)(address . avityazev@posteo.org)(address . 54003@debbugs.gnu.org)

Message-ID:87pmnoyy2s.fsf@gnu.org

Hi,

Aleksandr Vityazev <avityazev@posteo.org> skribis:

Toggle quote (23 lines)> The following example also worked before the
> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
> 'guile-3.0-latest') commit.
>
> in /subdir/home/wm.scm: 
>
> #+begin_src scheme :scheme guile :season guile
> (service
>  home-rofi-service-type
>  (home-rofi-configuration
>   (config-rasi
>    `(,#~(format
>          #f "@theme \"~a\""
>          #$(local-file "../../etc/rofi/nord.rasi"))))))
> #+end_src
>
> After 076e825dc5d585943ce820a279fffe4af09757fb
> using (local-file "../../etc/rofi/nord.rasi") and not
> (local-file "etc/rofi/nord.rasi") causes an error:
>
> guix home: error: canonicalize-path: No such file or
> directory: "../../etc/rofi/nord.rasi".

I’ve witnessed a similar problem that I worked around in

c334b7c52fe77b68a90b23fbac5c9de7337e607b.

Needs some investigation…

Ludo’.

Ludovic Courtès wrote on 15 Feb 16:00 +0100

control message for bug #54003

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87o838yy2l.fsf@gnu.org

severity 54003 important

quit

Ludovic Courtès wrote on 15 Feb 16:00 +0100

control message for bug #53214

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87mtisyy21.fsf@gnu.org

block 53214 by 54003

quit

Maxim Cournoyer wrote on 15 Feb 16:57 +0100

Re: bug#54003: local-file after Guile update

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87iltgxgv4.fsf@gmail.com

Hello,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (30 lines)> Hi,
>
> Aleksandr Vityazev <avityazev@posteo.org> skribis:
>
>> The following example also worked before the
>> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
>> 'guile-3.0-latest') commit.
>>
>> in /subdir/home/wm.scm: 
>>
>> #+begin_src scheme :scheme guile :season guile
>> (service
>>  home-rofi-service-type
>>  (home-rofi-configuration
>>   (config-rasi
>>    `(,#~(format
>>          #f "@theme \"~a\""
>>          #$(local-file "../../etc/rofi/nord.rasi"))))))
>> #+end_src
>>
>> After 076e825dc5d585943ce820a279fffe4af09757fb
>> using (local-file "../../etc/rofi/nord.rasi") and not
>> (local-file "etc/rofi/nord.rasi") causes an error:
>>
>> guix home: error: canonicalize-path: No such file or
>> directory: "../../etc/rofi/nord.rasi".
>
> I’ve witnessed a similar problem that I worked around in
> c334b7c52fe77b68a90b23fbac5c9de7337e607b.

Another example is the berlin configuration; attempting to reconfigure

it currently yields:

Toggle snippet (6 lines)

$ sudo guix system reconfigure -L modules berlin.scm

guix system: warning: the following accounts appear more than once: zabbix

guix system: warning: the following groups appear more than once: zabbix

guix system: error: canonicalize-path: No such file or directory: "../../sync-disarchive-db.scm"

I'm positive this is new behavior.

Thanks,

Maxim

Ludovic Courtès wrote on 15 Feb 20:42 +0100

Recipients:(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)

Message-ID:871r03zzky.fsf@gnu.org

Hi!

For now, here’s a workaround: instead of writing

#~(… #$(local-file …) …)

you can write:

(gexp (… (ungexp (local-file …)) …))

or:

(let ((file (local-file …)))

#~(… #$file …))

The problem is that ‘read’ hash extensions (like those for #~ and #$)

would so far return sexps with associated source properties, but psyntax

in 3.0.8 ignores those source properties.

I believe this is due to Guile commit

54bbe0b2846c5b1aa366c91d679ba724869c8cda, specifically this bit:

(define source-annotation
       (lambda (x)
-        (if (syntax? x)
-            (syntax-sourcev x)
-            (datum-sourcev x))))
+        (and (syntax? x)
+             (syntax-sourcev x))))

I’m looking for a reasonable workaround we could apply.

Ludo’.

Maxime Devos wrote on 15 Feb 21:56 +0100

Recipients:

Message-ID:695f30cb2a9857b3f616333b9a9fafb347f60061.camel@telenet.be

Ludovic Courtès schreef op di 15-02-2022 om 20:42 [+0100]:

Toggle quote (5 lines)

> The problem is that ‘read’ hash extensions (like those for #~ and #$)

> would so far return sexps with associated source properties, but

> psyntax

> in 3.0.8 ignores those source properties.

Read hash extensions can return syntax objects. In a Guile 3.0.8 REPL:

(define-module (hat))

(define (hat s-exp)

`#(hat ,s-exp))

(read-hash-extend #\^

(lambda (_ port)

;; Use 'read-syntax' instead of a combination of 'read' and

;; 'datum->syntax' for source properties.

#`(hat '#,(read-syntax port))))

#^(foo bar)

$4 = #(hat (foo bar))

Doing something like this in the hash extension for G-exps might work.

Greetings,

Maxime.

-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYgwTaRccbWF4aW1lZGV2

b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7mqMAQC/jn/8Pl4z9evHlkrklOq2FoT9

f5/NdRFkD95zvclh7wEA3tKjMty1uwRhV3jJUmROtDO/I/rasyEY0EtWcgw1hAg=

=2BiF

-----END PGP SIGNATURE-----

Ludovic Courtès wrote on 15 Feb 23:49 +0100

Recipients:(name . Maxime Devos)(address . maximedevos@telenet.be)

Message-ID:87v8xfycde.fsf@gnu.org

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

Toggle quote (17 lines)> Read hash extensions can return syntax objects.  In a Guile 3.0.8 REPL:
>
> (define-module (hat))
> (define (hat s-exp)
>   `#(hat ,s-exp))
>
> (read-hash-extend #\^
>   (lambda (_ port)
>     ;; Use 'read-syntax' instead of a combination of 'read' and
>     ;; 'datum->syntax' for source properties.
>     #`(hat '#,(read-syntax port))))
>
> #^(foo bar)
> $4 = #(hat (foo bar))
>
> Doing something like this in the hash extension for G-exps might work.

Yes, that’s one possible workaround, but then ‘read’ would find itself

returning syntax objects as well.

We have to see to what extent this is a problem in Guix proper; I think

it will at least break ‘guix style’ and related code.

Ludo’.

Ludovic Courtès wrote on 16 Feb 11:40 +0100

Recipients:(name . Maxime Devos)(address . maximedevos@telenet.be)

Message-ID:87mtirxfg6.fsf@gnu.org

Hi!

This *ahem* pretty hack works around the problem. I have yet to see if

it works with earlier Guile versions. Thoughts?

I’d have preferred to avoid monkey-patching but I couldn’t think of a

way to do that.

Ludo’.

From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Wed, 16 Feb 2022 11:27:43 +0100
Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.

Read hash extensions preserve source location info as source properties
on their result.  However, in Guile 3.0.8, that location would be
dismissed, leading 'local-file' to fail to resolve file names relative
to the source directory.

Fixes https://issues.guix.gnu.org/54003.
Reported by Aleksandr Vityazev <avityazev@posteo.org>.

* guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
[read-syntax*]: New variables.
[read-ungexp]: Adjust to expect either sexps or syntax objects.
[read-gexp]: Call 'read-procedure'.
* tests/gexp.scm ("local-file, relative file name, within gexp")
("local-file, relative file name, within gexp, compiled"): New tests.
---
 guix/gexp.scm  | 50 +++++++++++++++++++++++++++++++++++++++++++++-----
 tests/gexp.scm | 27 +++++++++++++++++++++++++++
 2 files changed, 72 insertions(+), 5 deletions(-)

Toggle diff (128 lines)diff --git a/guix/gexp.scm b/guix/gexp.scm
index d23683e2a6..e229c1fc8f 100644
--- a/guix/gexp.scm
+++ b/guix/gexp.scm
@@ -2176,6 +2176,29 @@ (define log-port
 ;;;
 
 (eval-when (expand load eval)
+  (define-once read-syntax-redefined?
+    ;; Have we already redefined 'read-syntax'?  This needs to be done on
+    ;; 3.0.8 only to work around <https://issues.guix.gnu.org/54003>.
+    (or (not (module-variable the-scm-module 'read-syntax))
+        (not (guile-version>? "3.0.7"))))
+
+  (define read-procedure
+    ;; The current read procedure being called: either 'read' or
+    ;; 'read-syntax'.
+    (make-parameter read))
+
+  (define read-syntax*
+    ;; Replacement for 'read-syntax'.
+    (let ((read-syntax (and=> (module-variable the-scm-module 'read-syntax)
+                              variable-ref)))
+      (lambda (port . rest)
+        (parameterize ((read-procedure read-syntax))
+          (apply read-syntax port rest)))))
+
+  (unless read-syntax-redefined?
+    (set! (@ (guile) read-syntax) read-syntax*)
+    (set! read-syntax-redefined? #t))
+
   (define* (read-ungexp chr port #:optional native?)
     "Read an 'ungexp' or 'ungexp-splicing' form from PORT.  When NATIVE? is
 true, use 'ungexp-native' and 'ungexp-native-splicing' instead."
@@ -2191,22 +2214,39 @@ (define unquote-symbol
              'ungexp-native
              'ungexp))))
 
-    (match (read port)
-      ((? symbol? symbol)
-       (let ((str (symbol->string symbol)))
+    (define symbolic?
+      ;; Depending on whether (read-procedure) is 'read' or 'read-syntax', we
+      ;; might get either sexps or syntax objects.  Adjust accordingly.
+      (if (eq? (read-procedure) read)
+          symbol?
+          (compose symbol? syntax->datum)))
+
+    (define symbolic->string
+      (if (eq? (read-procedure) read)
+          symbol->string
+          (compose symbol->string syntax->datum)))
+
+    (define wrapped-symbol
+      (if (eq? (read-procedure) read)
+          (lambda (_ symbol) symbol)
+          datum->syntax))
+
+    (match ((read-procedure) port)
+      ((? symbolic? symbol)
+       (let ((str (symbolic->string symbol)))
          (match (string-index-right str #\:)
            (#f
             `(,unquote-symbol ,symbol))
            (colon
             (let ((name   (string->symbol (substring str 0 colon)))
                   (output (substring str (+ colon 1))))
-              `(,unquote-symbol ,name ,output))))))
+              `(,unquote-symbol ,(wrapped-symbol symbol name) ,output))))))
       (x
        `(,unquote-symbol ,x))))
 
   (define (read-gexp chr port)
     "Read a 'gexp' form from PORT."
-    `(gexp ,(read port)))
+    `(gexp ,((read-procedure) port)))
 
   ;; Extend the reader
   (read-hash-extend #\~ read-gexp)
diff --git a/tests/gexp.scm b/tests/gexp.scm
index bcda516623..33c0e4bf8c 100644
--- a/tests/gexp.scm
+++ b/tests/gexp.scm
@@ -28,6 +28,7 @@ (define-module (test-gexp)
   #:use-module (guix tests)
   #:use-module ((guix build utils) #:select (with-directory-excursion))
   #:use-module ((guix utils) #:select (call-with-temporary-directory))
+  #:use-module ((guix ui) #:select (load*))
   #:use-module (gnu packages)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bootstrap)
@@ -222,6 +223,32 @@ (define defmod 'define-module) ;fool Geiser
       (let ((file (local-file (string-copy "../base32.scm"))))
         (local-file-absolute-file-name file)))))
 
+(test-assert "local-file, relative file name, within gexp"
+  (let* ((file     (search-path %load-path "guix/base32.scm"))
+         (interned (add-to-store %store "base32.scm" #f "sha256" file)))
+    (equal? `(the file is ,interned)
+            (gexp->sexp*
+             #~(the file is #$(local-file "../guix/base32.scm"))))))
+
+(test-assert "local-file, relative file name, within gexp, compiled"
+  ;; In Guile 3.0.8, everything read by the #~ and #$ read hash extensions
+  ;; would lack source location info, which in turn would lead
+  ;; (current-source-directory), called by 'local-file', to return #f, thereby
+  ;; breaking 'local-file' resolution.  See
+  ;; <https://issues.guix.gnu.org/54003>.
+  (let ((file (tmpnam)))
+    (call-with-output-file file
+      (lambda (port)
+        (display (string-append "#~(this file is #$(local-file \""
+                                (basename file) "\" \"t.scm\"))")
+                 port)))
+
+    (let* ((interned (add-to-store %store "t.scm" #f "sha256" file))
+           (module   (make-fresh-user-module)))
+      (module-use! module (resolve-interface '(guix gexp)))
+      (equal? `(this file is ,interned)
+              (gexp->sexp* (load* file module))))))
+
 (test-assertm "local-file, #:select?"
   (mlet* %store-monad ((select? -> (lambda (file stat)
                                      (member (basename file)

base-commit: 791069737c8c51582cc021438dae32eb0fb7b8e0
-- 
2.34.0

Ludovic Courtès wrote on 16 Feb 15:39 +0100

control message for bug #54003

Recipients:(address . control@debbugs.gnu.org)

Message-ID:871r02yixy.fsf@gnu.org

retitle 54003 'local-file' fails to resolve relative file names with Guile 3.0.8

quit

Ludovic Courtès wrote on 16 Feb 17:00 +0100

Re: bug#54003: local-file after Guile update

Recipients:(name . Maxime Devos)(address . maximedevos@telenet.be)

Message-ID:87sfsivm1t.fsf@gnu.org

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (20 lines)>>From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
> Date: Wed, 16 Feb 2022 11:27:43 +0100
> Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.
>
> Read hash extensions preserve source location info as source properties
> on their result.  However, in Guile 3.0.8, that location would be
> dismissed, leading 'local-file' to fail to resolve file names relative
> to the source directory.
>
> Fixes <https://issues.guix.gnu.org/54003>.
> Reported by Aleksandr Vityazev <avityazev@posteo.org>.
>
> * guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
> [read-syntax*]: New variables.
> [read-ungexp]: Adjust to expect either sexps or syntax objects.
> [read-gexp]: Call 'read-procedure'.
> * tests/gexp.scm ("local-file, relative file name, within gexp")
> ("local-file, relative file name, within gexp, compiled"): New tests.

Pushed as ca155a20aea25003b03ef5e0420c77e416d5f425 after verifying that

it works with 3.0.7.

I’m leaving the bug open so we can see what to do on the Guile side.

Ludo’.

Ludovic Courtès wrote on 7 Mar 14:18 +0100

Re: bug#54003: 'local-file' fails to resolve relative file names with Guile 3.0.8

Recipients:(name . Maxime Devos)(address . maximedevos@telenet.be)

Message-ID:87wnh5294t.fsf_-_@gnu.org

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (27 lines)> Ludovic Courtès <ludo@gnu.org> skribis:
>
>>>>From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
>> Date: Wed, 16 Feb 2022 11:27:43 +0100
>> Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.
>>
>> Read hash extensions preserve source location info as source properties
>> on their result.  However, in Guile 3.0.8, that location would be
>> dismissed, leading 'local-file' to fail to resolve file names relative
>> to the source directory.
>>
>> Fixes <https://issues.guix.gnu.org/54003>.
>> Reported by Aleksandr Vityazev <avityazev@posteo.org>.
>>
>> * guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
>> [read-syntax*]: New variables.
>> [read-ungexp]: Adjust to expect either sexps or syntax objects.
>> [read-gexp]: Call 'read-procedure'.
>> * tests/gexp.scm ("local-file, relative file name, within gexp")
>> ("local-file, relative file name, within gexp, compiled"): New tests.
>
> Pushed as ca155a20aea25003b03ef5e0420c77e416d5f425 after verifying that
> it works with 3.0.7.
>
> I’m leaving the bug open so we can see what to do on the Guile side.

Fixed in Guile:

https://git.savannah.gnu.org/cgit/guile.git/commit/?id=347321ece9fc85ddf74af3c798230b7b187fbce9

Closing!

Ludo’.

Closed

Your comment

This issue is archived.

To comment on this conversation send email to 54003@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date