[wip-zig-bootstrap] zig cache is not updated for dependencies of dependencies

  • Open
  • quality assurance status badge
Details
2 participants
  • Hilton Chain
  • Noé Lopez
Owner
unassigned
Submitted by
Noé Lopez
Severity
normal
N
N
Noé Lopez wrote on 2 Dec 2024 22:15
(address . bug-guix@gnu.org)
87ed2p7pmn.fsf@xn--no-cja.eu
Hi,

I got my hands on packaging liskvork, a gomoku server. The build fails
because dependencies of dependencies are not added to the zig cache, and
so can’t be found correctly by the zig build system.

I attached a patch that reproduces this, a workaround when reproducing
the steps in a local checkout is to call “zig fetch” on the missing
dependencies:

$ zig fetch /gnu/store/j0x1vl6w2vgr3fz60l4jgyx5gx53af61-zig-httpz-0-0.7d2ddae/src/zig-httpz-0-0.7d2ddae --save=httpz
warning: overwriting existing dependency named 'httpz'
$ zig build --summary all
/home/noe/.cache/zig/p/1220476906a8f57d6cbaaaeb05d44a41311a5bb6ca74bb86bc3aa1467506c241b29b/build.zig.zon:7:20: error: invalid URI: UnexpectedCharacter
$ zig fetch /gnu/store/59w4s1g1y4vy4gw3mwn9fjsfcqc5454i-zig-metrics-0-0.fcf9e94/src/zig-metrics-0-0.fcf9e94 --save=metrics
$ zig fetch /gnu/store/izxqkxplp7alrh0n3c8j8cv277nqa04a-zig-websocket-0-0.cf89cb8/src/zig-websocket-0-0.cf89cb8 --save=websocket
$ zig build --summary all
Build Summary: 5/5 steps succeeded
install success
+- install liskvork success
+- zig build-exe liskvork Debug native-native success 3s MaxRSS:276M
+- options cached
+- options cached

This has the issue of adding the packages to build.zig.zon, so a better
solution is required.

I’m attaching the patch with the liskvork package and dependencies. The
error can be reproduced with “./pre-inst-env guix build liskvork”.

Have a nice day,
Noé
H
H
Hilton Chain wrote on 3 Dec 2024 01:43
(name . Noé Lopez)(address . noe@xn--no-cja.eu)
875xo11tr6.wl-hako@ultrarare.space
Hi Noé,

On Tue, 03 Dec 2024 05:15:44 +0800,
Noé Lopez wrote:
Toggle quote (34 lines)
>
> [1 <text/plain; utf-8 (quoted-printable)>]
> Hi,
>
> I got my hands on packaging liskvork, a gomoku server. The build fails
> because dependencies of dependencies are not added to the zig cache, and so
> can’t be found correctly by the zig build system.
>
> I attached a patch that reproduces this, a workaround when reproducing the
> steps in a local checkout is to call “zig fetch” on the missing dependencies:
>
> $ zig fetch /gnu/store/j0x1vl6w2vgr3fz60l4jgyx5gx53af61-zig-httpz-0-0.7d2ddae/src/zig-httpz-0-0.7d2ddae --save=httpz
> warning: overwriting existing dependency named 'httpz'
> $ zig build --summary all
> /home/noe/.cache/zig/p/1220476906a8f57d6cbaaaeb05d44a41311a5bb6ca74bb86bc3aa1467506c241b29b/build.zig.zon:7:20: error: invalid URI: UnexpectedCharacter
> $ zig fetch /gnu/store/59w4s1g1y4vy4gw3mwn9fjsfcqc5454i-zig-metrics-0-0.fcf9e94/src/zig-metrics-0-0.fcf9e94 --save=metrics
> $ zig fetch /gnu/store/izxqkxplp7alrh0n3c8j8cv277nqa04a-zig-websocket-0-0.cf89cb8/src/zig-websocket-0-0.cf89cb8 --save=websocket
> $ zig build --summary all
> Build Summary: 5/5 steps succeeded
> install success
> +- install liskvork success
> +- zig build-exe liskvork Debug native-native success 3s MaxRSS:276M
> +- options cached
> +- options cached
>
> This has the issue of adding the packages to build.zig.zon, so a better
> solution is required.
>
> I’m attaching the patch with the liskvork package and dependencies. The error
> can be reproduced with “./pre-inst-env guix build liskvork”.
>
> Have a nice day,
> Noé

Good example, that's the current limitation of #:zig-inputs, and I'm planning to
remove it.

Please apply the attached patch onto your change.

Thanks
Toggle diff (90 lines)
diff --git a/gnu/packages/zig-xyz.scm b/gnu/packages/zig-xyz.scm
index 027377fa1b..b98a160e27 100644
--- a/gnu/packages/zig-xyz.scm
+++ b/gnu/packages/zig-xyz.scm
@@ -186,11 +186,20 @@ (define-public zig-logz
(file-name (git-file-name name version))
(sha256
(base32
- "01xihyvyx3rpv0kvjh6mg1b99d6agq683q4iyn39nwqb6ma0i0sz"))))
+ "01xihyvyx3rpv0kvjh6mg1b99d6agq683q4iyn39nwqb6ma0i0sz"))
+ (modules '((guix build utils)))
+ (snippet
+ #~(for-each
+ (lambda (dep)
+ (substitute* "build.zig"
+ (((string-append "(b\\.dependency.\")" (car dep)) _ prefix)
+ (string-append prefix (cdr dep))))
+ (substitute* "build.zig.zon"
+ (((string-append "\\." (car dep)))
+ (format #f ".@\"~a\"" (cdr dep)))))
+ '(("metrics" . "zig-metrics"))))))
(build-system zig-build-system)
- (arguments
- (list
- #:zig-inputs `(("metrics" ,zig-metrics))))
+ (propagated-inputs (list zig-metrics))
(home-page "https://github.com/karlseguin/log.zig")
(synopsis "Structured Logging for Zig")
(description "logz is an opinionated structured logger that outputs to stdout,
@@ -265,12 +274,21 @@ (define-public zig-httpz
(file-name (git-file-name name version))
(sha256
(base32
- "02hixvyx1r04lg0nzvhkyrqwcwm8m8rs8hm01n2nzw6jv935frh8"))))
+ "02hixvyx1r04lg0nzvhkyrqwcwm8m8rs8hm01n2nzw6jv935frh8"))
+ (modules '((guix build utils)))
+ (snippet
+ #~(for-each
+ (lambda (dep)
+ (substitute* "build.zig"
+ (((string-append "(b\\.dependency.\")" (car dep)) _ prefix)
+ (string-append prefix (cdr dep))))
+ (substitute* "build.zig.zon"
+ (((string-append "\\." (car dep)))
+ (format #f ".@\"~a\"" (cdr dep)))))
+ '(("metrics" . "zig-metrics")
+ ("websocket" . "zig-websocket"))))))
(build-system zig-build-system)
- (arguments
- (list #:zig-inputs
- `(("metrics" ,zig-metrics)
- ("websocket" ,zig-websocket))))
+ (propagated-inputs (list zig-metrics zig-websocket))
(home-page "https://github.com/karlseguin/http.zig")
(synopsis "HTTP/1.1 server for Zig")
(description "")
@@ -288,16 +306,26 @@ (define-public liskvork
(file-name (git-file-name name version))
(sha256
(base32
- "1x7cif9wpaq7mk1pqmixq3flymrradb6zpx5qnmiihw699zr2xhw"))))
+ "1x7cif9wpaq7mk1pqmixq3flymrradb6zpx5qnmiihw699zr2xhw"))
+ (modules '((guix build utils)))
+ (snippet
+ #~(for-each
+ (lambda (dep)
+ (substitute* "build.zig"
+ (((string-append "(b\\.dependency.\")" (car dep)) _ prefix)
+ (string-append prefix (cdr dep))))
+ (substitute* "build.zig.zon"
+ (((string-append "\\." (car dep)))
+ (format #f ".@\"~a\"" (cdr dep)))))
+ '(("ini" . "zig-ini")
+ ("logz" . "zig-logz")
+ ("zul" . "zig-zul")
+ ("httpz" . "zig-httpz"))))))
(build-system zig-build-system)
(arguments
(list #:install-source? #f
- #:zig-release-type "safe"
- #:zig-inputs
- `(("ini" ,zig-ini)
- ("logz" ,zig-logz)
- ("zul" ,zig-zul)
- ("httpz" ,zig-httpz))))
+ #:zig-release-type "safe"))
+ (inputs (list zig-httpz zig-ini zig-logz zig-zul))
(home-page "https://liskvork.org")
(synopsis "Modern multi-platform gomoku game server.")
(description "liskvork is a modern Gomoku game server that is purpose-built for
N
N
Noé Lopez wrote on 3 Dec 2024 10:56
(name . Hilton Chain)(address . hako@ultrarare.space)
87zfldxf7m.fsf@xn--no-cja.eu
Hi,

Its working well with your changes, do you plan on keeping the snippet
syntax for every package? It seems like a lot of boilerplate.

Personally I like the #:zig-inputs syntax better, why not keep it and do
the snippet in zig-build-system ?

Thanks,
Noé

Hilton Chain <hako@ultrarare.space> writes:

Toggle quote (134 lines)
> Hi Noé,
>
> On Tue, 03 Dec 2024 05:15:44 +0800,
> Noé Lopez wrote:
>>
>> [1 <text/plain; utf-8 (quoted-printable)>]
>> Hi,
>>
>> I got my hands on packaging liskvork, a gomoku server. The build fails
>> because dependencies of dependencies are not added to the zig cache, and so
>> can’t be found correctly by the zig build system.
>>
>> I attached a patch that reproduces this, a workaround when reproducing the
>> steps in a local checkout is to call “zig fetch” on the missing dependencies:
>>
>> $ zig fetch /gnu/store/j0x1vl6w2vgr3fz60l4jgyx5gx53af61-zig-httpz-0-0.7d2ddae/src/zig-httpz-0-0.7d2ddae --save=httpz
>> warning: overwriting existing dependency named 'httpz'
>> $ zig build --summary all
>> /home/noe/.cache/zig/p/1220476906a8f57d6cbaaaeb05d44a41311a5bb6ca74bb86bc3aa1467506c241b29b/build.zig.zon:7:20: error: invalid URI: UnexpectedCharacter
>> $ zig fetch /gnu/store/59w4s1g1y4vy4gw3mwn9fjsfcqc5454i-zig-metrics-0-0.fcf9e94/src/zig-metrics-0-0.fcf9e94 --save=metrics
>> $ zig fetch /gnu/store/izxqkxplp7alrh0n3c8j8cv277nqa04a-zig-websocket-0-0.cf89cb8/src/zig-websocket-0-0.cf89cb8 --save=websocket
>> $ zig build --summary all
>> Build Summary: 5/5 steps succeeded
>> install success
>> +- install liskvork success
>> +- zig build-exe liskvork Debug native-native success 3s MaxRSS:276M
>> +- options cached
>> +- options cached
>>
>> This has the issue of adding the packages to build.zig.zon, so a better
>> solution is required.
>>
>> I’m attaching the patch with the liskvork package and dependencies. The error
>> can be reproduced with “./pre-inst-env guix build liskvork”.
>>
>> Have a nice day,
>> Noé
>
> Good example, that's the current limitation of #:zig-inputs, and I'm planning to
> remove it.
>
> Please apply the attached patch onto your change.
>
> Thanks
> diff --git a/gnu/packages/zig-xyz.scm b/gnu/packages/zig-xyz.scm
> index 027377fa1b..b98a160e27 100644
> --- a/gnu/packages/zig-xyz.scm
> +++ b/gnu/packages/zig-xyz.scm
> @@ -186,11 +186,20 @@ (define-public zig-logz
> (file-name (git-file-name name version))
> (sha256
> (base32
> - "01xihyvyx3rpv0kvjh6mg1b99d6agq683q4iyn39nwqb6ma0i0sz"))))
> + "01xihyvyx3rpv0kvjh6mg1b99d6agq683q4iyn39nwqb6ma0i0sz"))
> + (modules '((guix build utils)))
> + (snippet
> + #~(for-each
> + (lambda (dep)
> + (substitute* "build.zig"
> + (((string-append "(b\\.dependency.\")" (car dep)) _ prefix)
> + (string-append prefix (cdr dep))))
> + (substitute* "build.zig.zon"
> + (((string-append "\\." (car dep)))
> + (format #f ".@\"~a\"" (cdr dep)))))
> + '(("metrics" . "zig-metrics"))))))
> (build-system zig-build-system)
> - (arguments
> - (list
> - #:zig-inputs `(("metrics" ,zig-metrics))))
> + (propagated-inputs (list zig-metrics))
> (home-page "https://github.com/karlseguin/log.zig")
> (synopsis "Structured Logging for Zig")
> (description "logz is an opinionated structured logger that outputs to stdout,
> @@ -265,12 +274,21 @@ (define-public zig-httpz
> (file-name (git-file-name name version))
> (sha256
> (base32
> - "02hixvyx1r04lg0nzvhkyrqwcwm8m8rs8hm01n2nzw6jv935frh8"))))
> + "02hixvyx1r04lg0nzvhkyrqwcwm8m8rs8hm01n2nzw6jv935frh8"))
> + (modules '((guix build utils)))
> + (snippet
> + #~(for-each
> + (lambda (dep)
> + (substitute* "build.zig"
> + (((string-append "(b\\.dependency.\")" (car dep)) _ prefix)
> + (string-append prefix (cdr dep))))
> + (substitute* "build.zig.zon"
> + (((string-append "\\." (car dep)))
> + (format #f ".@\"~a\"" (cdr dep)))))
> + '(("metrics" . "zig-metrics")
> + ("websocket" . "zig-websocket"))))))
> (build-system zig-build-system)
> - (arguments
> - (list #:zig-inputs
> - `(("metrics" ,zig-metrics)
> - ("websocket" ,zig-websocket))))
> + (propagated-inputs (list zig-metrics zig-websocket))
> (home-page "https://github.com/karlseguin/http.zig")
> (synopsis "HTTP/1.1 server for Zig")
> (description "")
> @@ -288,16 +306,26 @@ (define-public liskvork
> (file-name (git-file-name name version))
> (sha256
> (base32
> - "1x7cif9wpaq7mk1pqmixq3flymrradb6zpx5qnmiihw699zr2xhw"))))
> + "1x7cif9wpaq7mk1pqmixq3flymrradb6zpx5qnmiihw699zr2xhw"))
> + (modules '((guix build utils)))
> + (snippet
> + #~(for-each
> + (lambda (dep)
> + (substitute* "build.zig"
> + (((string-append "(b\\.dependency.\")" (car dep)) _ prefix)
> + (string-append prefix (cdr dep))))
> + (substitute* "build.zig.zon"
> + (((string-append "\\." (car dep)))
> + (format #f ".@\"~a\"" (cdr dep)))))
> + '(("ini" . "zig-ini")
> + ("logz" . "zig-logz")
> + ("zul" . "zig-zul")
> + ("httpz" . "zig-httpz"))))))
> (build-system zig-build-system)
> (arguments
> (list #:install-source? #f
> - #:zig-release-type "safe"
> - #:zig-inputs
> - `(("ini" ,zig-ini)
> - ("logz" ,zig-logz)
> - ("zul" ,zig-zul)
> - ("httpz" ,zig-httpz))))
> + #:zig-release-type "safe"))
> + (inputs (list zig-httpz zig-ini zig-logz zig-zul))
> (home-page "https://liskvork.org")
> (synopsis "Modern multi-platform gomoku game server.")
> (description "liskvork is a modern Gomoku game server that is purpose-built for
H
H
Hilton Chain wrote on 3 Dec 2024 18:07
(name . Noé Lopez)(address . noe@xn--no-cja.eu)
87bjxs3dba.wl-hako@ultrarare.space
On Tue, 03 Dec 2024 17:56:13 +0800,
Noé Lopez wrote:
Toggle quote (4 lines)
>
> Its working well with your changes, do you plan on keeping the snippet
> syntax for every package? It seems like a lot of boilerplate.

I can make it a procedure later, if only using this pattern is enough.

Toggle quote (3 lines)
> Personally I like the #:zig-inputs syntax better, why not keep it and do
> the snippet in zig-build-system ?

Metadata for Guix UI, and not repeating the work of inputs propagation.
H
H
Hilton Chain wrote on 4 Dec 2024 19:03
(name . Noé Lopez)(address . noe@xn--no-cja.eu)
87jzcf1g2u.wl-hako@ultrarare.space
Added your packages (except liskvork, please tell me how to use it) to
wip-zig-bootstrap. There're various modifications, please see the current
definitions.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 74656@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 74656
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch