'local-file' fails to resolve relative file names with Guile 3.0.8

DoneSubmitted by Aleksandr Vityazev.
Details
5 participants
  • Aleksandr Vityazev
  • Liliana Marie Prikler
  • Ludovic Courtès
  • Maxim Cournoyer
  • Maxime Devos
Owner
unassigned
Severity
important
A
A
Aleksandr Vityazev wrote on 14 Feb 22:50 +0100
local-file after Guile update
(address . bug-guix@gnu.org)
87zgmtqfsf.fsf@posteo.org
Hi,

I have the following dotfiles directory structure where I store my Guix home
config:

|-etc
|
|-subdir
| |
| |-home


I often use the 'local-file' procedure in modules in the home directory. For
example in /subdir/home/shells.scm:

#+begin_src scheme :scheme guile :season guile
(simple-service 'add-zsh-files
home-files-service-type
`(("config/zsh/zshrc"
,(local-file "../../etc/zsh/zshrc"))))
#+end_src
And it works, as expected.

The following example also worked before the
076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
'guile-3.0-latest') commit.

in /subdir/home/wm.scm:
#+begin_src scheme :scheme guile :season guile
(service
home-rofi-service-type
(home-rofi-configuration
(config-rasi
`(,#~(format
#f "@theme \"~a\""
#$(local-file "../../etc/rofi/nord.rasi"))))))
#+end_src

After 076e825dc5d585943ce820a279fffe4af09757fb
using (local-file "../../etc/rofi/nord.rasi") and not
(local-file "etc/rofi/nord.rasi") causes an error:

guix home: error: canonicalize-path: No such file or
directory: "../../etc/rofi/nord.rasi".

The example that causes the error uses the original Guix home implementation,
which uses gexps. When I figure out how to make a minimal reproducer that
does not depend on it, I will update this report.

--
Best regards,
Aleksandr Vityazev
L
L
Liliana Marie Prikler wrote on 15 Feb 09:21 +0100
3bde6d0ba002c1400176b1a2bce83f035afd01ad.camel@ist.tugraz.at
Hi Aleksandr

Am Montag, dem 14.02.2022 um 21:50 +0000 schrieb Aleksandr Vityazev:
Toggle quote (22 lines)
> Hi,
>
> I have the following dotfiles directory structure where I store my
> Guix home config:
>
> > -etc
> >
> > -subdir
> >   |
> >   |-home
>
>
> I often use the 'local-file' procedure in modules in the home
> directory. For example in /subdir/home/shells.scm:
>
> #+begin_src scheme :scheme guile :season guile
> (simple-service 'add-zsh-files
>                 home-files-service-type
>                 `(("config/zsh/zshrc"
>                    ,(local-file "../../etc/zsh/zshrc"))))
> #+end_src
> And it works, as expected.
Be careful when invoking Hyrum's Law. The meaning of this local-file
expression changes with the current working directory, which might be
different across invocations. To avoid such issues, you can (almost?)
always use (current-file-name), which you might want to canonicalize
for the operations you're about to do.
Taking (dirname) thrice then brings you to $GUIX_HOME_CONFIG_ROOT,
which contains both etc and subdir.

Toggle quote (27 lines)
> The following example also worked before the
> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version
> 3.0.8 as
> 'guile-3.0-latest') commit.
>
> in /subdir/home/wm.scm:
> #+begin_src scheme :scheme guile :season guile
> (service
>  home-rofi-service-type
>  (home-rofi-configuration
>   (config-rasi
>    `(,#~(format
>          #f "@theme \"~a\""
>          #$(local-file "../../etc/rofi/nord.rasi"))))))
> #+end_src
>
> After 076e825dc5d585943ce820a279fffe4af09757fb
> using (local-file "../../etc/rofi/nord.rasi") and not
> (local-file "etc/rofi/nord.rasi") causes an error:
>
> guix home: error: canonicalize-path: No such file or
> directory: "../../etc/rofi/nord.rasi".
>
> The example that causes the error uses the original Guix home
> implementation, which uses gexps.  When I figure out how to make a
> minimal reproducer that does not depend on it, I will update this
> report.
I don't think this is an issue particularly in Guix Home, but for a MWE
you could try only writing a bashrc, which ought to work with upstream
as well. Use `guix home build` rather than `guix home reconfigure` so
as to not activate it.

Cheers
L
L
Ludovic Courtès wrote on 15 Feb 16:00 +0100
Re: bug#54003: local-file after Guile update
(name . Aleksandr Vityazev)(address . avityazev@posteo.org)(address . 54003@debbugs.gnu.org)
87pmnoyy2s.fsf@gnu.org
Hi,

Aleksandr Vityazev <avityazev@posteo.org> skribis:

Toggle quote (23 lines)
> The following example also worked before the
> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
> 'guile-3.0-latest') commit.
>
> in /subdir/home/wm.scm:
>
> #+begin_src scheme :scheme guile :season guile
> (service
> home-rofi-service-type
> (home-rofi-configuration
> (config-rasi
> `(,#~(format
> #f "@theme \"~a\""
> #$(local-file "../../etc/rofi/nord.rasi"))))))
> #+end_src
>
> After 076e825dc5d585943ce820a279fffe4af09757fb
> using (local-file "../../etc/rofi/nord.rasi") and not
> (local-file "etc/rofi/nord.rasi") causes an error:
>
> guix home: error: canonicalize-path: No such file or
> directory: "../../etc/rofi/nord.rasi".

I’ve witnessed a similar problem that I worked around in
c334b7c52fe77b68a90b23fbac5c9de7337e607b.

Needs some investigation…

Ludo’.
L
L
Ludovic Courtès wrote on 15 Feb 16:00 +0100
control message for bug #54003
(address . control@debbugs.gnu.org)
87o838yy2l.fsf@gnu.org
severity 54003 important
quit
L
L
Ludovic Courtès wrote on 15 Feb 16:00 +0100
control message for bug #53214
(address . control@debbugs.gnu.org)
87mtisyy21.fsf@gnu.org
block 53214 by 54003
quit
M
M
Maxim Cournoyer wrote on 15 Feb 16:57 +0100
Re: bug#54003: local-file after Guile update
(name . Ludovic Courtès)(address . ludo@gnu.org)
87iltgxgv4.fsf@gmail.com
Hello,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (30 lines)
> Hi,
>
> Aleksandr Vityazev <avityazev@posteo.org> skribis:
>
>> The following example also worked before the
>> 076e825dc5d585943ce820a279fffe4af09757fb (gnu: guile: Add version 3.0.8 as
>> 'guile-3.0-latest') commit.
>>
>> in /subdir/home/wm.scm:
>>
>> #+begin_src scheme :scheme guile :season guile
>> (service
>> home-rofi-service-type
>> (home-rofi-configuration
>> (config-rasi
>> `(,#~(format
>> #f "@theme \"~a\""
>> #$(local-file "../../etc/rofi/nord.rasi"))))))
>> #+end_src
>>
>> After 076e825dc5d585943ce820a279fffe4af09757fb
>> using (local-file "../../etc/rofi/nord.rasi") and not
>> (local-file "etc/rofi/nord.rasi") causes an error:
>>
>> guix home: error: canonicalize-path: No such file or
>> directory: "../../etc/rofi/nord.rasi".
>
> I’ve witnessed a similar problem that I worked around in
> c334b7c52fe77b68a90b23fbac5c9de7337e607b.

Another example is the berlin configuration; attempting to reconfigure
it currently yields:

Toggle snippet (6 lines)
$ sudo guix system reconfigure -L modules berlin.scm
guix system: warning: the following accounts appear more than once: zabbix
guix system: warning: the following groups appear more than once: zabbix
guix system: error: canonicalize-path: No such file or directory: "../../sync-disarchive-db.scm"

I'm positive this is new behavior.

Thanks,

Maxim
L
L
Ludovic Courtès wrote on 15 Feb 20:42 +0100
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
871r03zzky.fsf@gnu.org
Hi!

For now, here’s a workaround: instead of writing

#~(… #$(local-file …) …)

you can write:

(gexp (… (ungexp (local-file …)) …))

or:

(let ((file (local-file …)))
#~(… #$file …))

The problem is that ‘read’ hash extensions (like those for #~ and #$)
would so far return sexps with associated source properties, but psyntax
in 3.0.8 ignores those source properties.

I believe this is due to Guile commit
54bbe0b2846c5b1aa366c91d679ba724869c8cda, specifically this bit:
(define source-annotation
(lambda (x)
- (if (syntax? x)
- (syntax-sourcev x)
- (datum-sourcev x))))
+ (and (syntax? x)
+ (syntax-sourcev x))))
I’m looking for a reasonable workaround we could apply.

Ludo’.
M
M
Maxime Devos wrote on 15 Feb 21:56 +0100
695f30cb2a9857b3f616333b9a9fafb347f60061.camel@telenet.be
Ludovic Courtès schreef op di 15-02-2022 om 20:42 [+0100]:
Toggle quote (5 lines)
> The problem is that ‘read’ hash extensions (like those for #~ and #$)
> would so far return sexps with associated source properties, but
> psyntax
> in 3.0.8 ignores those source properties.

Read hash extensions can return syntax objects. In a Guile 3.0.8 REPL:

(define-module (hat))
(define (hat s-exp)
`#(hat ,s-exp))

(read-hash-extend #\^
(lambda (_ port)
;; Use 'read-syntax' instead of a combination of 'read' and
;; 'datum->syntax' for source properties.
#`(hat '#,(read-syntax port))))

#^(foo bar)
$4 = #(hat (foo bar))

Doing something like this in the hash extension for G-exps might work.

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYgwTaRccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7mqMAQC/jn/8Pl4z9evHlkrklOq2FoT9
f5/NdRFkD95zvclh7wEA3tKjMty1uwRhV3jJUmROtDO/I/rasyEY0EtWcgw1hAg=
=2BiF
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 15 Feb 23:49 +0100
(name . Maxime Devos)(address . maximedevos@telenet.be)
87v8xfycde.fsf@gnu.org
Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

Toggle quote (17 lines)
> Read hash extensions can return syntax objects. In a Guile 3.0.8 REPL:
>
> (define-module (hat))
> (define (hat s-exp)
> `#(hat ,s-exp))
>
> (read-hash-extend #\^
> (lambda (_ port)
> ;; Use 'read-syntax' instead of a combination of 'read' and
> ;; 'datum->syntax' for source properties.
> #`(hat '#,(read-syntax port))))
>
> #^(foo bar)
> $4 = #(hat (foo bar))
>
> Doing something like this in the hash extension for G-exps might work.

Yes, that’s one possible workaround, but then ‘read’ would find itself
returning syntax objects as well.

We have to see to what extent this is a problem in Guix proper; I think
it will at least break ‘guix style’ and related code.

Ludo’.
L
L
Ludovic Courtès wrote on 16 Feb 11:40 +0100
(name . Maxime Devos)(address . maximedevos@telenet.be)
87mtirxfg6.fsf@gnu.org
Hi!

This *ahem* pretty hack works around the problem. I have yet to see if
it works with earlier Guile versions. Thoughts?

I’d have preferred to avoid monkey-patching but I couldn’t think of a
way to do that.

Ludo’.
From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
Date: Wed, 16 Feb 2022 11:27:43 +0100
Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.

Read hash extensions preserve source location info as source properties
on their result. However, in Guile 3.0.8, that location would be
dismissed, leading 'local-file' to fail to resolve file names relative
to the source directory.

Reported by Aleksandr Vityazev <avityazev@posteo.org>.

* guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
[read-syntax*]: New variables.
[read-ungexp]: Adjust to expect either sexps or syntax objects.
[read-gexp]: Call 'read-procedure'.
* tests/gexp.scm ("local-file, relative file name, within gexp")
("local-file, relative file name, within gexp, compiled"): New tests.
---
guix/gexp.scm | 50 +++++++++++++++++++++++++++++++++++++++++++++-----
tests/gexp.scm | 27 +++++++++++++++++++++++++++
2 files changed, 72 insertions(+), 5 deletions(-)

Toggle diff (128 lines)
diff --git a/guix/gexp.scm b/guix/gexp.scm
index d23683e2a6..e229c1fc8f 100644
--- a/guix/gexp.scm
+++ b/guix/gexp.scm
@@ -2176,6 +2176,29 @@ (define log-port
 ;;;
 
 (eval-when (expand load eval)
+  (define-once read-syntax-redefined?
+    ;; Have we already redefined 'read-syntax'?  This needs to be done on
+    ;; 3.0.8 only to work around <https://issues.guix.gnu.org/54003>.
+    (or (not (module-variable the-scm-module 'read-syntax))
+        (not (guile-version>? "3.0.7"))))
+
+  (define read-procedure
+    ;; The current read procedure being called: either 'read' or
+    ;; 'read-syntax'.
+    (make-parameter read))
+
+  (define read-syntax*
+    ;; Replacement for 'read-syntax'.
+    (let ((read-syntax (and=> (module-variable the-scm-module 'read-syntax)
+                              variable-ref)))
+      (lambda (port . rest)
+        (parameterize ((read-procedure read-syntax))
+          (apply read-syntax port rest)))))
+
+  (unless read-syntax-redefined?
+    (set! (@ (guile) read-syntax) read-syntax*)
+    (set! read-syntax-redefined? #t))
+
   (define* (read-ungexp chr port #:optional native?)
     "Read an 'ungexp' or 'ungexp-splicing' form from PORT.  When NATIVE? is
 true, use 'ungexp-native' and 'ungexp-native-splicing' instead."
@@ -2191,22 +2214,39 @@ (define unquote-symbol
              'ungexp-native
              'ungexp))))
 
-    (match (read port)
-      ((? symbol? symbol)
-       (let ((str (symbol->string symbol)))
+    (define symbolic?
+      ;; Depending on whether (read-procedure) is 'read' or 'read-syntax', we
+      ;; might get either sexps or syntax objects.  Adjust accordingly.
+      (if (eq? (read-procedure) read)
+          symbol?
+          (compose symbol? syntax->datum)))
+
+    (define symbolic->string
+      (if (eq? (read-procedure) read)
+          symbol->string
+          (compose symbol->string syntax->datum)))
+
+    (define wrapped-symbol
+      (if (eq? (read-procedure) read)
+          (lambda (_ symbol) symbol)
+          datum->syntax))
+
+    (match ((read-procedure) port)
+      ((? symbolic? symbol)
+       (let ((str (symbolic->string symbol)))
          (match (string-index-right str #\:)
            (#f
             `(,unquote-symbol ,symbol))
            (colon
             (let ((name   (string->symbol (substring str 0 colon)))
                   (output (substring str (+ colon 1))))
-              `(,unquote-symbol ,name ,output))))))
+              `(,unquote-symbol ,(wrapped-symbol symbol name) ,output))))))
       (x
        `(,unquote-symbol ,x))))
 
   (define (read-gexp chr port)
     "Read a 'gexp' form from PORT."
-    `(gexp ,(read port)))
+    `(gexp ,((read-procedure) port)))
 
   ;; Extend the reader
   (read-hash-extend #\~ read-gexp)
diff --git a/tests/gexp.scm b/tests/gexp.scm
index bcda516623..33c0e4bf8c 100644
--- a/tests/gexp.scm
+++ b/tests/gexp.scm
@@ -28,6 +28,7 @@ (define-module (test-gexp)
   #:use-module (guix tests)
   #:use-module ((guix build utils) #:select (with-directory-excursion))
   #:use-module ((guix utils) #:select (call-with-temporary-directory))
+  #:use-module ((guix ui) #:select (load*))
   #:use-module (gnu packages)
   #:use-module (gnu packages base)
   #:use-module (gnu packages bootstrap)
@@ -222,6 +223,32 @@ (define defmod 'define-module) ;fool Geiser
       (let ((file (local-file (string-copy "../base32.scm"))))
         (local-file-absolute-file-name file)))))
 
+(test-assert "local-file, relative file name, within gexp"
+  (let* ((file     (search-path %load-path "guix/base32.scm"))
+         (interned (add-to-store %store "base32.scm" #f "sha256" file)))
+    (equal? `(the file is ,interned)
+            (gexp->sexp*
+             #~(the file is #$(local-file "../guix/base32.scm"))))))
+
+(test-assert "local-file, relative file name, within gexp, compiled"
+  ;; In Guile 3.0.8, everything read by the #~ and #$ read hash extensions
+  ;; would lack source location info, which in turn would lead
+  ;; (current-source-directory), called by 'local-file', to return #f, thereby
+  ;; breaking 'local-file' resolution.  See
+  ;; <https://issues.guix.gnu.org/54003>.
+  (let ((file (tmpnam)))
+    (call-with-output-file file
+      (lambda (port)
+        (display (string-append "#~(this file is #$(local-file \""
+                                (basename file) "\" \"t.scm\"))")
+                 port)))
+
+    (let* ((interned (add-to-store %store "t.scm" #f "sha256" file))
+           (module   (make-fresh-user-module)))
+      (module-use! module (resolve-interface '(guix gexp)))
+      (equal? `(this file is ,interned)
+              (gexp->sexp* (load* file module))))))
+
 (test-assertm "local-file, #:select?"
   (mlet* %store-monad ((select? -> (lambda (file stat)
                                      (member (basename file)

base-commit: 791069737c8c51582cc021438dae32eb0fb7b8e0
-- 
2.34.0
L
L
Ludovic Courtès wrote on 16 Feb 15:39 +0100
control message for bug #54003
(address . control@debbugs.gnu.org)
871r02yixy.fsf@gnu.org
retitle 54003 'local-file' fails to resolve relative file names with Guile 3.0.8
quit
L
L
Ludovic Courtès wrote on 16 Feb 17:00 +0100
Re: bug#54003: local-file after Guile update
(name . Maxime Devos)(address . maximedevos@telenet.be)
87sfsivm1t.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (20 lines)
>>From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
> Date: Wed, 16 Feb 2022 11:27:43 +0100
> Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.
>
> Read hash extensions preserve source location info as source properties
> on their result. However, in Guile 3.0.8, that location would be
> dismissed, leading 'local-file' to fail to resolve file names relative
> to the source directory.
>
> Fixes <https://issues.guix.gnu.org/54003>.
> Reported by Aleksandr Vityazev <avityazev@posteo.org>.
>
> * guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
> [read-syntax*]: New variables.
> [read-ungexp]: Adjust to expect either sexps or syntax objects.
> [read-gexp]: Call 'read-procedure'.
> * tests/gexp.scm ("local-file, relative file name, within gexp")
> ("local-file, relative file name, within gexp, compiled"): New tests.

Pushed as ca155a20aea25003b03ef5e0420c77e416d5f425 after verifying that
it works with 3.0.7.

I’m leaving the bug open so we can see what to do on the Guile side.

Ludo’.
L
L
Ludovic Courtès wrote on 7 Mar 14:18 +0100
Re: bug#54003: 'local-file' fails to resolve relative file names with Guile 3.0.8
(name . Maxime Devos)(address . maximedevos@telenet.be)
87wnh5294t.fsf_-_@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (27 lines)
> Ludovic Courtès <ludo@gnu.org> skribis:
>
>>>>From 7fd25c6f15f74fb6e45fc3f0db21a110267f262c Mon Sep 17 00:00:00 2001
>> From: =?UTF-8?q?Ludovic=20Court=C3=A8s?= <ludo@gnu.org>
>> Date: Wed, 16 Feb 2022 11:27:43 +0100
>> Subject: [PATCH] gexp: Preserve source location for #~ and #$ read extensions.
>>
>> Read hash extensions preserve source location info as source properties
>> on their result. However, in Guile 3.0.8, that location would be
>> dismissed, leading 'local-file' to fail to resolve file names relative
>> to the source directory.
>>
>> Fixes <https://issues.guix.gnu.org/54003>.
>> Reported by Aleksandr Vityazev <avityazev@posteo.org>.
>>
>> * guix/gexp.scm <eval-when> [read-syntax-redefined?, read-procedure]
>> [read-syntax*]: New variables.
>> [read-ungexp]: Adjust to expect either sexps or syntax objects.
>> [read-gexp]: Call 'read-procedure'.
>> * tests/gexp.scm ("local-file, relative file name, within gexp")
>> ("local-file, relative file name, within gexp, compiled"): New tests.
>
> Pushed as ca155a20aea25003b03ef5e0420c77e416d5f425 after verifying that
> it works with 3.0.7.
>
> I’m leaving the bug open so we can see what to do on the Guile side.

Closed
?
Your comment

This issue is archived.

To comment on this conversation send email to 54003@debbugs.gnu.org