[PATCH 0/2] Add librsvg-bootstrap

  • Done
  • quality assurance status badge
Details
5 participants
  • Efraim Flashner
  • Kaelyn
  • Liliana Marie Prikler
  • Ludovic Courtès
  • Ricardo Wurmus
Owner
unassigned
Submitted by
Efraim Flashner
Severity
normal
E
E
Efraim Flashner wrote on 14 Nov 2021 15:07
(address . guix-patches@gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)
cover.1636898737.git.efraim@flashner.co.il
librsvg is an input for emacs, gtk+@2 and gtk+@3. With the rust inputs
this leads to (unknown) rust libraries causing the rebuild of over 3000
packages on core-updates-frozen. Rather than hunt them down I tracked
down the packages which would have many rebuilds and added a copy of
librsvg for them to use.

Efraim Flashner (2):
Add librsvg-bootstrap.
gnu: Use librsvg-bootstrap.

gnu/packages/emacs.scm | 2 +-
gnu/packages/gnome.scm | 23 +++++++++++++++++++++++
gnu/packages/gtk.scm | 4 ++--
3 files changed, 26 insertions(+), 3 deletions(-)


base-commit: 75b5ad6aa3b55b2cbd7f333411cbc9e21ab1e186
--
2.33.1
E
E
Efraim Flashner wrote on 14 Nov 2021 15:14
[PATCH 1/2] gnu: Add librsvg-bootstrap.
(address . 51845@debbugs.gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)
4a1df5a86f181bcc2086b1b407dbeaba4d01ba88.1636898737.git.efraim@flashner.co.il
* gnu/packages/gnome.scm (librsvg-bootstrap): New variable.
---
gnu/packages/gnome.scm | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)

Toggle diff (36 lines)
diff --git a/gnu/packages/gnome.scm b/gnu/packages/gnome.scm
index 924d1326cc..a0436a4edb 100644
--- a/gnu/packages/gnome.scm
+++ b/gnu/packages/gnome.scm
@@ -3580,6 +3580,29 @@ (define-public librsvg
(home-page "https://wiki.gnome.org/LibRsvg")
(license license:lgpl2.1+)))
+;; This copy of librsvg uses the bundled rust libraries. It is useful for
+;; packages which have too many dependencies to be rebuilt as frequently
+;; as the rust inputs are updated.
+(define-public librsvg-bootstrap
+ (package
+ (inherit librsvg)
+ (name "librsvg")
+ (version "2.50.7")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append "mirror://gnome/sources/librsvg/"
+ (version-major+minor version) "/"
+ "librsvg-" version ".tar.xz"))
+ (sha256
+ (base32
+ "1g3f8byg5w08fx1bka12mmpl59v6a4q2p827w6m2la6mijq63yzz"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments librsvg)
+ ((#:vendor-dir _ "vendor") "vendor")
+ ((#:cargo-inputs _) '())
+ ((#:cargo-development-inputs _) '())))
+ (properties '((hidden? . #t)))))
+
(define-public libidl
(package
(name "libidl")
--
2.33.1
E
E
Efraim Flashner wrote on 14 Nov 2021 15:14
[PATCH 2/2] gnu: Use librsvg-bootstrap.
(address . 51845@debbugs.gnu.org)(name . Efraim Flashner)(address . efraim@flashner.co.il)
e44665cc7de39ca1228876b741feca4f5b1b7434.1636898737.git.efraim@flashner.co.il
* gnu/packages/emacs.scm (inputs): Use librsvg-bootstrap.
* gnu/pacakges/gtk.scm (gtk+-2, gtk+)[propagated-inputs]: Same.
---
gnu/packages/emacs.scm | 2 +-
gnu/packages/gtk.scm | 4 ++--
2 files changed, 3 insertions(+), 3 deletions(-)

Toggle diff (37 lines)
diff --git a/gnu/packages/emacs.scm b/gnu/packages/emacs.scm
index 734f3dfaa3..3a5215b31a 100644
--- a/gnu/packages/emacs.scm
+++ b/gnu/packages/emacs.scm
@@ -269,7 +269,7 @@ (define* (emacs-byte-compile-directory dir)
;; supported well on every architecture yet.
,@(if (string-prefix? "x86_64" (or (%current-target-system)
(%current-system)))
- `(("librsvg" ,librsvg))
+ `(("librsvg" ,librsvg-bootstrap))
'())
("libxpm" ,libxpm)
("libxml2" ,libxml2)
diff --git a/gnu/packages/gtk.scm b/gnu/packages/gtk.scm
index 199ca13981..4dfeba4b7f 100644
--- a/gnu/packages/gtk.scm
+++ b/gnu/packages/gtk.scm
@@ -867,7 +867,7 @@ (define-public gtk+-2
;; Rust is not supported well on every architecture yet.
("gdk-pixbuf" ,(if (string-prefix? "x86_64" (or (%current-target-system)
(%current-system)))
- librsvg
+ librsvg-bootstrap
gdk-pixbuf))
("glib" ,glib)
("pango" ,pango)))
@@ -969,7 +969,7 @@ (define-public gtk+
;; SVG support is optional and requires librsvg, which pulls in rust.
;; Rust is not supported well on every architecture yet.
("gdk-pixbuf" ,(if (target-x86-64?)
- librsvg
+ librsvg-bootstrap
gdk-pixbuf))
("glib" ,glib)
("libcloudproviders" ,libcloudproviders-minimal)
--
2.33.1
L
L
Liliana Marie Prikler wrote on 14 Nov 2021 18:27
Re: [PATCH 0/2] Add librsvg-bootstrap
68012880ef968bf2d5ab3d7e967b06bafb9ea10f.camel@gmail.com
Hi,

Am Sonntag, den 14.11.2021, 16:07 +0200 schrieb Efraim Flashner:
Toggle quote (5 lines)
> librsvg is an input for emacs, gtk+@2 and gtk+@3. With the rust
> inputs this leads to (unknown) rust libraries causing the rebuild of
> over 3000 packages on core-updates-frozen. Rather than hunt them down
> I tracked down the packages which would have many rebuilds and added
> a copy of librsvg for them to use.
In my opinion, one of the selling points of Guix is that of
bootstrappability. I don't think adding big blobs to Emacs of all
things is a great way of delivering on that promise. I think we ought
to rather "invest" in alternatives to Rust and Rust-locked libraries or
make Rust packaging itself sane (if it can at all).

I think librsvg is optional already and people who want to save on
compilation time can decide to replace it with e.g. GNU hello using the
--input option. In the similar case of mozjs, a replacement with
duktape is discussed on guix-devel, at least for polkit.

As a temporary resolution to the rebuild issue, we could pin the
dependencies of librsvg to some specific versions and only bump them
when something awful happens. I'm not sure whether librsvg exposes any
of the Rust nastiness to its dependencies, ideally hoping that it would
not.

WDYT?
E
E
Efraim Flashner wrote on 14 Nov 2021 19:07
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)(address . 51845@debbugs.gnu.org)
YZFQRR5PDcnazL04@3900XT
On Sun, Nov 14, 2021 at 06:27:02PM +0100, Liliana Marie Prikler wrote:
Toggle quote (19 lines)
> Hi,
>
> Am Sonntag, den 14.11.2021, 16:07 +0200 schrieb Efraim Flashner:
> > librsvg is an input for emacs, gtk+@2 and gtk+@3. With the rust
> > inputs this leads to (unknown) rust libraries causing the rebuild of
> > over 3000 packages on core-updates-frozen. Rather than hunt them down
> > I tracked down the packages which would have many rebuilds and added
> > a copy of librsvg for them to use.
> In my opinion, one of the selling points of Guix is that of
> bootstrappability. I don't think adding big blobs to Emacs of all
> things is a great way of delivering on that promise. I think we ought
> to rather "invest" in alternatives to Rust and Rust-locked libraries or
> make Rust packaging itself sane (if it can at all).
>
> I think librsvg is optional already and people who want to save on
> compilation time can decide to replace it with e.g. GNU hello using the
> --input option. In the similar case of mozjs, a replacement with
> duktape is discussed on guix-devel, at least for polkit.

It seems I was wrong about emacs; both emacs-minimal and emacs-no-x are
built without librsvg.

Toggle quote (6 lines)
> As a temporary resolution to the rebuild issue, we could pin the
> dependencies of librsvg to some specific versions and only bump them
> when something awful happens. I'm not sure whether librsvg exposes any
> of the Rust nastiness to its dependencies, ideally hoping that it would
> not.

I don't believe librsvg exposes any rust-y stuff.

Toggle quote (2 lines)
> WDYT?

(ins)efraim@3900XT /tmp/librsvg-2.50.7$ ls vendor/ | wc -l
226

There are 226 crates that upstream bundles with their source. I suppose
we could pare it down to about 200 by careful pruning but it's part of
librsvg and not going away.

(ins)efraim@3900XT ~/workspace/guix-core-updates$ git grep \,librsvg | wc -l
103

I'm suggesting that for gtk+@2 and gtk+@3 we use the bundled crates and
for the other 101 packages we continue to use our current version, where
we replace all of the bundled crates with our own copies, which get
updated more often than librsvg does.

With our current rust tooling I don't think it'd be that easy to find
the ~226 crates that librsvg depends on, and it wouldn't be great to
lock them due to librsvg being an input for gtk2/3.

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmGRUEIACgkQQarn3Mo9
g1E9cg//ba+VZ+T1B8kaxKqJ6mmgGwo/32aXUwh48K1Z5IJx3zUAC7mhWs9yE/gI
n+D9ea+JbbExwbO8+LYUsmp4LgCYoVDk3Cp91+kf7n1K3Jkxd/JV712tngyisX0o
u4F1CZ8KfSosRhoS05ryQugvpm1KBuNPV0f6mJ2GmdePXFD9IA+bjABBCya5/hNP
zOkO5EqlhYuVwbm53Xj88nYxnLGM7jxoXQM+08TwVjIgEwR3dFOruD9mtVF6kb9j
K7Cdmc+nmtH2rsv+YnUzL3+Irqa9RzubTwBeUJCFDAYU8DylBGD6lsNE27diQFmJ
f5aSYpqLMiXK1nIUJn4IRGpnAE0soBy3tabq+m0IZiM9pxXOCPCNlvyKEiRgKnbs
rFmPUT2h06AFuxz1Y3yDCI0jggHr1WxMNjPHVOqBbtiK3YbgJgAMXS19FIkoGj/g
ys+vbFjeBK1lA2vSA7HrdmtGgTeAJyzZ8yA8JbKmo6NcX5nOIvKCMn9QZDA+HKwE
+cydZ/4Kyztv+dcvnBQST8Ffi1Ft3dUdeOEdbF0XrN7vkWWuDjkQIYvLGOelvsNU
emZr4iN0S8yEFSC6Map/2BvM/GcI5/FdSoSgBaNImamw3cmYXqTrVtwTO2PWyawp
H8xru2poBiLHWjMuOc+OM4TVg3Ue4RtAQWeUhy9eQOhn4qE7trk=
=XF6T
-----END PGP SIGNATURE-----


L
L
Liliana Marie Prikler wrote on 14 Nov 2021 20:05
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 51845@debbugs.gnu.org)
49868b918eb291fd2761c48c7617fecd6bb98764.camel@gmail.com
Hi,

Am Sonntag, den 14.11.2021, 20:07 +0200 schrieb Efraim Flashner:
Toggle quote (7 lines)
> > As a temporary resolution to the rebuild issue, we could pin the
> > dependencies of librsvg to some specific versions and only bump
> > them something awful happens. I'm not sure whether librsvg
> > exposes any of the Rust nastiness to its dependencies, ideally
> > hoping that it would not.
>
> I don't believe librsvg exposes any rust-y stuff.
That sounds like a good start for once.

Toggle quote (8 lines)
> > WDYT?
>
> (ins)efraim@3900XT /tmp/librsvg-2.50.7$ ls vendor/ | wc -l
> 226
>
> There are 226 crates that upstream bundles with their source. I
> suppose we could pare it down to about 200 by careful pruning but
> it's part of librsvg and not going away.
Well, I'd suggest snippeting them away, but that's a different topic.

Toggle quote (3 lines)
> (ins)efraim@3900XT ~/workspace/guix-core-updates$ git grep \,librsvg
> | wc -l
> 103
I'd hazard a guess that most if not all of these 103 packages are
themselves gtk-adjacent, so what really is the issue we're solving
here? What is the point of maintaining an extra version for 101 of
them when a potentially vulnerable GTK sits right next to them?

Toggle quote (8 lines)
> I'm suggesting that for gtk+@2 and gtk+@3 we use the bundled crates
> and for the other 101 packages we continue to use our current
> version, where we replace all of the bundled crates with our own
> copies, which get updated more often than librsvg does.
>
> With our current rust tooling I don't think it'd be that easy to find
> the ~226 crates that librsvg depends on, and it wouldn't be great to
> lock them due to librsvg being an input for gtk2/3.
Said input exists due to gdk-pixbuf+svg, with the +svg part being
largely optional – the most common failure mode of it not being
included are broken button textures, which we could fix by pre-
rendering images with a suitable tool, such as inkscape. We could
easily do a minimal gtk[+]? without it.

As for the lock, why can't we? gtk+ is already core-updates material,
so it stands to reason that anything causing it to rebuild is too.
Rather than push down blobs to the users because we can't deal with
Rust, we should fix Rust or make it go away from the build.

WDYT?
L
L
Ludovic Courtès wrote on 6 Dec 2021 13:17
Re: bug#51845: [PATCH 0/2] Add librsvg-bootstrap
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 51845@debbugs.gnu.org)
87zgpend04.fsf@gnu.org
Hi Efraim,

I had completely overlooked these patches, oops!

Efraim Flashner <efraim@flashner.co.il> skribis:

Toggle quote (6 lines)
> librsvg is an input for emacs, gtk+@2 and gtk+@3. With the rust inputs
> this leads to (unknown) rust libraries causing the rebuild of over 3000
> packages on core-updates-frozen. Rather than hunt them down I tracked
> down the packages which would have many rebuilds and added a copy of
> librsvg for them to use.

[...]

Toggle quote (9 lines)
> I'm suggesting that for gtk+@2 and gtk+@3 we use the bundled crates and
> for the other 101 packages we continue to use our current version, where
> we replace all of the bundled crates with our own copies, which get
> updated more often than librsvg does.
>
> With our current rust tooling I don't think it'd be that easy to find
> the ~226 crates that librsvg depends on, and it wouldn't be great to
> lock them due to librsvg being an input for gtk2/3.

Yes, that’s a problem, though Liliana is right that bundling isn’t great
either.

I’m annoyed by this whole librsvg situation. On non-x86_64, we now
depend on librsvg 2.40, the old C version, and guess what, it just
works. That has me tempted to stick with 2.40 all along because these
Rust problems don’t seem to have a pleasant, or even an easy solution.

Now, using the proposed ‘librsvg-bootstrap’ in GTK+ looks like a lesser
evil.

Thoughts?

Ludo’.
E
E
Efraim Flashner wrote on 6 Dec 2021 14:06
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 51845@debbugs.gnu.org)
Ya4KyruleTGdYU7o@3900XT
On Mon, Dec 06, 2021 at 01:17:47PM +0100, Ludovic Courtès wrote:
Toggle quote (36 lines)
> Hi Efraim,
>
> I had completely overlooked these patches, oops!
>
> Efraim Flashner <efraim@flashner.co.il> skribis:
>
> > librsvg is an input for emacs, gtk+@2 and gtk+@3. With the rust inputs
> > this leads to (unknown) rust libraries causing the rebuild of over 3000
> > packages on core-updates-frozen. Rather than hunt them down I tracked
> > down the packages which would have many rebuilds and added a copy of
> > librsvg for them to use.
>
> [...]
>
> > I'm suggesting that for gtk+@2 and gtk+@3 we use the bundled crates and
> > for the other 101 packages we continue to use our current version, where
> > we replace all of the bundled crates with our own copies, which get
> > updated more often than librsvg does.
> >
> > With our current rust tooling I don't think it'd be that easy to find
> > the ~226 crates that librsvg depends on, and it wouldn't be great to
> > lock them due to librsvg being an input for gtk2/3.
>
> Yes, that’s a problem, though Liliana is right that bundling isn’t great
> either.
>
> I’m annoyed by this whole librsvg situation. On non-x86_64, we now
> depend on librsvg 2.40, the old C version, and guess what, it just
> works. That has me tempted to stick with 2.40 all along because these
> Rust problems don’t seem to have a pleasant, or even an easy solution.
>
> Now, using the proposed ‘librsvg-bootstrap’ in GTK+ looks like a lesser
> evil.
>
> Thoughts?

Unbundling the rust crates is the right option, but not the easy option.
With the assumption that rust-libc-0.2 is in the graph for librsvg, we
add another copy named rust-libc-0.2.101 (the current version) and a
comment that it only gets adjusted on core-updates or that it causes
XXXX package rebuilds.

On a small tangent, the work I do sometimes to try to actually have a
dependency graph with the crates would only make these easier to find,
not actually address the issue here.

I'm not sure if it'd be better to mostly copy the packages with a new
name and keep the cargo-inputs or to actually adjust the
cargo-inputs->inputs and cargo-development-inputs->native-inputs so we
get the dependency graph from rust-libc-0.2.101 to librsvg. I'd like to
make the change but if we don't get the others changed then we
effectively really have two sets of rust crates.

If we have both cargo-inputs and inputs then the cargo-build-system
doesn't have issues with using either type with later packages, so that
might be the best option for now.

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmGuCsgACgkQQarn3Mo9
g1GelA//SkbqRAKmwoL+6GnM4ZlsmG6xOqqk1rHt1qHk/cOHKi+cbx4Xfcb1nj7I
zSotDjmlf+wPZI3wG1ljw91Y+RSvvnlMg6Y0pqFSO/ERDaapaei2EvNVqggZmbC5
BxsqMzDqd1UsCrPJNj71ckcnreRmF2+/TBIO6uswdGyavid6TP9F8uy/VJcjXeJN
vWGBXiP1oAjLzli7AO1ya7sUuZdsPdGUIUBwJbqV5/9Na0WvdR27has2UcdNKoRV
vc+KcKnwX2c32zhnfcwQomT/8JcvND/Ip0iXggzqDrJtFmpBEhxJ0r5froSZme9+
gPp/1KOQ8dwHvD1DHlCj4MgCneXkkGJNz5ZTieDgaj6VO1emGS/XBl8cX/i4owrj
+aosRoVE4KhVsRnlVEjsBcrh6YxZDK4U6m/i7/rU0JmISeG6+tZghvQjHQdq5J/m
tnEnwBmENGvAlQXxTPimSzfyQKScbs47nmDpkw9ZxWC4rQC3rj4nEgk/l/CJuX0E
e6+XuYFJFJv4ZlfwvMSmNx08z1uijoCibOp2GwGF91Of9Z3CFZ9VBf1yqGIU22bA
jP8IpWXhkArQXgFuNsMPf1MQCqWlb7zxU88tNww1AMtSig/I8oUkj0Yo93T0y/RD
mglhoJDPamVxUz7clXUoEGxg6G/GbFXwnJd4pQmvTOcBcPupwW0=
=Nn2l
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 6 Dec 2021 17:37
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 51845@debbugs.gnu.org)
87wnkhlmfb.fsf@gnu.org
Hi Efraim,

Efraim Flashner <efraim@flashner.co.il> skribis:

Toggle quote (15 lines)
> On a small tangent, the work I do sometimes to try to actually have a
> dependency graph with the crates would only make these easier to find,
> not actually address the issue here.
>
> I'm not sure if it'd be better to mostly copy the packages with a new
> name and keep the cargo-inputs or to actually adjust the
> cargo-inputs->inputs and cargo-development-inputs->native-inputs so we
> get the dependency graph from rust-libc-0.2.101 to librsvg. I'd like to
> make the change but if we don't get the others changed then we
> effectively really have two sets of rust crates.
>
> If we have both cargo-inputs and inputs then the cargo-build-system
> doesn't have issues with using either type with later packages, so that
> might be the best option for now.

Thinking out loud… would it work to change:

(arguments '(#:cargo-inputs X #:cargo-development-inputs Y))

to:

(native-inputs (map package-source Y))
(inputs (map package-source X))

?

Or am I just saying nonsense?

Thanks,
Ludo’.
E
E
Efraim Flashner wrote on 6 Dec 2021 18:02
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 51845@debbugs.gnu.org)
405F2D9A-C0E6-42EC-9F81-F52F305B723C@flashner.co.il
On December 6, 2021 4:37:12 PM UTC, "Ludovic Courtès" <ludo@gnu.org> wrote:
Toggle quote (35 lines)
>Hi Efraim,
>
>Efraim Flashner <efraim@flashner.co.il> skribis:
>
>> On a small tangent, the work I do sometimes to try to actually have a
>> dependency graph with the crates would only make these easier to find,
>> not actually address the issue here.
>>
>> I'm not sure if it'd be better to mostly copy the packages with a new
>> name and keep the cargo-inputs or to actually adjust the
>> cargo-inputs->inputs and cargo-development-inputs->native-inputs so we
>> get the dependency graph from rust-libc-0.2.101 to librsvg. I'd like to
>> make the change but if we don't get the others changed then we
>> effectively really have two sets of rust crates.
>>
>> If we have both cargo-inputs and inputs then the cargo-build-system
>> doesn't have issues with using either type with later packages, so that
>> might be the best option for now.
>
>Thinking out loud… would it work to change:
>
> (arguments '(#:cargo-inputs X #:cargo-development-inputs Y))
>
>to:
>
> (native-inputs (map package-source Y))
> (inputs (map package-source X))
>
>?
>
>Or am I just saying nonsense?
>
>Thanks,
>Ludo’.

Then we lose the transitive package sources, which is how we ended up where we are today.

I can go and change the cargo-build-system to use the skip-build flag in more phases to skip them when we aren't going to be building them anyway. No need to generate cargo checksums if we're not building I think.
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.
L
L
Ludovic Courtès wrote on 6 Dec 2021 23:17
Using ‘native-inputs’ and ‘inputs’ for Cargo packages?
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 51845@debbugs.gnu.org)
878rwxl6no.fsf_-_@gnu.org
Efraim Flashner <efraim@flashner.co.il> skribis:

Toggle quote (2 lines)
> On December 6, 2021 4:37:12 PM UTC, "Ludovic Courtès" <ludo@gnu.org> wrote:

[...]

Toggle quote (11 lines)
>>Thinking out loud… would it work to change:
>>
>> (arguments '(#:cargo-inputs X #:cargo-development-inputs Y))
>>
>>to:
>>
>> (native-inputs (map package-source Y))
>> (inputs (map package-source X))
>>
>>?

[...]

Toggle quote (2 lines)
> Then we lose the transitive package sources, which is how we ended up where we are today.

True.

With the minimal changes to (guix build-system cargo) below, one can use
either the current style or pass “development inputs” as ‘native-inputs’
and other dependencies as ‘inputs’. Source transitivity is preserved
but you can write packages the normal way.

I modified some of the dependencies of librsvg to use
native-inputs/inputs and you can see when applying this part of the
patch that the librsvg derivation is unchanged. Good thing is that
‘guix graph’ and ‘guix refresh -l’ work for these packages.

Is this a direction we want to take?

If so, we can have ‘guix style’ automate transformations.

Thanks,
Ludo’.
Attachment: file
E
E
Efraim Flashner wrote on 7 Dec 2021 11:11
Re: Using ‘native-inputs ’ and ‘inputs’ for Cargo packages?
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 51845@debbugs.gnu.org)
Ya8zONvzYXJcgL53@3900XT
Attachment: file
Attachment: rust-changes-c-u-f
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmGvMzYACgkQQarn3Mo9
g1H/Zg/7BW7UW0K9xwiQrVcU4/FQLQmwh0HwjaFeL30pSjIqj42cgIYC0QaLj9Cf
K3J8cML9o3rR6kuvF6PgFLwC/TEaN3iB4qy2XIn5GC/hzS0jdo1OQpdauaA2P1Ky
/ZMvcFSbhAoExW4Wt3S1wk9m2V5x69Hin5QKY8e3J+dYUrWrxc17vf3/ZZ5UPy5M
5Rk0BikU64QS/vU544kFXHpvT3vR1JjZmi0Twn1oDfzSqK5q+Vp7VDxZF+fByEkU
nWrI1x/44cDIutPonYTTGIaEdnxfQftsty43kcrezCooYkx+C3olAhv5Vg69X3eY
5Ad3BG2YisHPZkRMOfbIC/RKIaBxBZMrRQ8xu0m7fRvusrnXGdHCm8vo1EZmaedH
eH5+/jYtK3JWcrQkTb8uWsn5mcOGgKowkw26XPsSsUnFGnP/VDdv+u99Jv6iKp0O
bUes14o+2pr2pZ7j7mzSx6W/zkOLikNHv7eKr5/NwJK6OXnEyKjJz7gB+uTP+zEY
wl06CTAi4yr4Z/uIibT+kp4EmCUHZ5FgbiLOqDB565RKoNHQMnUhbQeYKk5a1sRg
VetRwuwFv+ARSUtq+O8JEmBilRl1ZnLsiDmzlzUf5rDUA5orUr8jlv2RR99Ol49x
4tuQKMsd0vLXF1v2w2RIf7Pyj261ZxOXWooTuTjZpASXN/hNGtU=
=Tdlm
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 7 Dec 2021 20:48
Re: Using ‘native-inputs’ and ‘inputs’ for Cargo packages?
(name . Efraim Flashner)(address . efraim@flashner.co.il)(address . 51845@debbugs.gnu.org)
87a6hci4cv.fsf@gnu.org
Howdy!

Efraim Flashner <efraim@flashner.co.il> skribis:

Toggle quote (2 lines)
> On Mon, Dec 06, 2021 at 11:17:47PM +0100, Ludovic Courtès wrote:

[...]

Toggle quote (19 lines)
>> With the minimal changes to (guix build-system cargo) below, one can use
>> either the current style or pass “development inputs” as ‘native-inputs’
>> and other dependencies as ‘inputs’. Source transitivity is preserved
>> but you can write packages the normal way.
>>
>> I modified some of the dependencies of librsvg to use
>> native-inputs/inputs and you can see when applying this part of the
>> patch that the librsvg derivation is unchanged. Good thing is that
>> ‘guix graph’ and ‘guix refresh -l’ work for these packages.
>>
>> Is this a direction we want to take?
>
> I like the way it works out, and has Guix do the magic to give us the
> crates in the graph. On the other hand I tried changing the cargo-inputs
> from librsvg to regular inputs, and after 2.5 minutes of trying to run
> `guix show librsvg` I still wasn't seeing the dependencies and my RAM
> usage was still increasing. Also gnu/packages/gnome.scm didn't fail to
> compile, so there was no notice of the loop.

Notice that the dependency cycle, as discussed on IRC, breaks things
like ‘guix show’ (?) and ‘guix graph’, but doesn’t break actual package
builds because it’s a <package> cycle that vanishes once packages are
lowered to bags and derivations.

Regardless, it would be nice not to have cycles in the first place.

[...]

Toggle quote (6 lines)
> So to summarize, between your diff to treat inputs built using
> cargo-build-system as cargo-inputs and my changes to save previous
> crates for the next input we reach a place where we can start to change
> the crates over to use inputs and native-inputs instead of cargo-inputs
> and cargo-development-inputs without needing to flip everything at once.

What are the “changes to save previous crates for the next input”?

Toggle quote (3 lines)
> So I'd go with it's good, but I'm not sure it directly works to fix the
> problem we're having with librsvg.

No no, it’s completely unrelated to the librsvg issue, which is why I
changed subject lines. :-) I think it’d be nice to have anyway.

Toggle quote (9 lines)
>> +(define (cargo-input? input)
>> + (match input
>> + ((label (? package? p))
>> + (eq? cargo-build-system (package-build-system p)))
>> + (_ #f)))
>> +
>
> I would've sorted based on the name starting with 'rust-'.

OK.

[...]

Toggle quote (23 lines)
>> (define (package-cargo-inputs p)
>> - (apply
>> - (lambda* (#:key (cargo-inputs '()) #:allow-other-keys)
>> - cargo-inputs)
>> - (package-arguments p)))
>> + (match (member #:cargo-inputs (package-arguments p))
>> + (#f (filter cargo-input? (package-inputs p)))
>> + ((_ inputs . _) inputs)))
>>
>> (define (package-cargo-development-inputs p)
>> - (apply
>> - (lambda* (#:key (cargo-development-inputs '()) #:allow-other-keys)
>> - cargo-development-inputs)
>> - (package-arguments p)))
>> + (match (member #:cargo-development-inputs (package-arguments p))
>> + (#f (filter cargo-input? (package-native-inputs p)))
>> + ((_ inputs . _) inputs)))
>
> I see we don't get rid of #:cargo-inputs or #:cargo-development-inputs.
> So even if applying the style change to all the crates causes circular
> dependency problems we can fall back to the current method. I ran into
> problems once I hit all the rust-bindgen crates.

Right. Support for #:cargo-development-inputs and #:cargo-inputs is
here so we could have a smooth “upgrade” without breaking compatibility.

Anyway, I think the priority is to get ‘core-updates-frozen’, which
probably involves either pinning librsvg dependencies or using the
bundled libraries as you showed at the beginning of this thread.
We can resume work on prettified Rust packages after that; it’ll be
useful to be able to use ‘guix refresh -l’.

Ludo’.
L
L
Ludovic Courtès wrote on 8 Dec 2021 15:24
Re: [CORE-UPDATES] librsvg and rust
(address . guix-devel@gnu.org)
87tufjgooo.fsf@gnu.org
Hello!

For the record, this is a followup to Efraim’s proposal in

Efraim Flashner <efraim@flashner.co.il> skribis:

Toggle quote (6 lines)
> Option 1:
> Track down the ~220 crates which form the dependency graph (of crates)
> for librsvg and pin them until the next core-updates cycle. Continue
> like with other packages and add newer versions (like cmake or meson) as
> packages need them.¹

The advantage of this approach is that we could do it incrementally: we
could merge ‘core-updates-frozen’ today and just add pinned variants of
these 200+ crates as needed as time passes. The downside is that it’s a
lot of crates to take care of, and we might still accidentally overlook
seemingly innocuous crate upgrades that end up causing major rebuilds.

Toggle quote (8 lines)
> Option 2:
> Use the bundled crates and treat it as just part of the librsvg source
> code.²
>
> Option 2b:
> Use the bundled crates for now to finish with core-updates-frozen and
> revisit this immediately on core-updates (not frozen).

This option will involved a rebuild on x86_64, but the advantage is that
we’ll be safe going forward: we won’t accidentally cause world rebuilds
just because an obscure crate somewhere has been upgraded.

[...]

Toggle quote (4 lines)
> I'm currently leaning option 2b, it'll get us past this hurdle for
> core-updates-frozen and let us make changes to the crates as we work to
> integrate them more fully into Guix.

Same here; it’s not ideal, but it seems like the most reasonable
short-term option.

If there are no objections, I’d suggest that you go ahead with this
plan.

Thanks for keeping the ball rolling!

Ludo’.
R
R
Ricardo Wurmus wrote on 8 Dec 2021 15:36
(name . Ludovic Courtès)(address . ludo@gnu.org)
87r1antb7w.fsf@elephly.net
Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (43 lines)
> Hello!
>
> For the record, this is a followup to Efraim’s proposal in
> <https://issues.guix.gnu.org/51845>.
>
> Efraim Flashner <efraim@flashner.co.il> skribis:
>
>> Option 1:
>> Track down the ~220 crates which form the dependency graph (of crates)
>> for librsvg and pin them until the next core-updates cycle. Continue
>> like with other packages and add newer versions (like cmake or meson) as
>> packages need them.¹
>
> The advantage of this approach is that we could do it incrementally: we
> could merge ‘core-updates-frozen’ today and just add pinned variants of
> these 200+ crates as needed as time passes. The downside is that it’s a
> lot of crates to take care of, and we might still accidentally overlook
> seemingly innocuous crate upgrades that end up causing major rebuilds.
>
>> Option 2:
>> Use the bundled crates and treat it as just part of the librsvg source
>> code.²
>>
>> Option 2b:
>> Use the bundled crates for now to finish with core-updates-frozen and
>> revisit this immediately on core-updates (not frozen).
>
> This option will involved a rebuild on x86_64, but the advantage is that
> we’ll be safe going forward: we won’t accidentally cause world rebuilds
> just because an obscure crate somewhere has been upgraded.
>
> [...]
>
>> I'm currently leaning option 2b, it'll get us past this hurdle for
>> core-updates-frozen and let us make changes to the crates as we work to
>> integrate them more fully into Guix.
>
> Same here; it’s not ideal, but it seems like the most reasonable
> short-term option.
>
> If there are no objections, I’d suggest that you go ahead with this
> plan.

I agree that 2b is the most sensible option in our current situation.

--
Ricardo
K
K
Kaelyn wrote on 8 Dec 2021 21:01
(address . guix-devel@gnu.org)(address . 51845@debbugs.gnu.org)
Mh_qMDO-pyLFJbyx5QhBFi8kUbyq65rLVgA8htVH57yYQzCuWyLA0vMJnv0Ru3jrmpKo17lk1W7HEqvb-E_8EwudF1o9xYMqrkxkrZhJR-E=@protonmail.com
On Wednesday, December 8th, 2021 at 6:36 AM, Ricardo Wurmus <rekado@elephly.net> wrote:

Toggle quote (66 lines)
> Ludovic Courtès ludo@gnu.org writes:
>
> > Hello!
> >
> > For the record, this is a followup to Efraim’s proposal in
> >
> > https://issues.guix.gnu.org/51845.
> >
> > Efraim Flashner efraim@flashner.co.il skribis:
> >
> > > Option 1:
> > >
> > > Track down the ~220 crates which form the dependency graph (of crates)
> > >
> > > for librsvg and pin them until the next core-updates cycle. Continue
> > >
> > > like with other packages and add newer versions (like cmake or meson) as
> > >
> > > packages need them.¹
> >
> > The advantage of this approach is that we could do it incrementally: we
> >
> > could merge ‘core-updates-frozen’ today and just add pinned variants of
> >
> > these 200+ crates as needed as time passes. The downside is that it’s a
> >
> > lot of crates to take care of, and we might still accidentally overlook
> >
> > seemingly innocuous crate upgrades that end up causing major rebuilds.
> >
> > > Option 2:
> > >
> > > Use the bundled crates and treat it as just part of the librsvg source
> > >
> > > code.²
> > >
> > > Option 2b:
> > >
> > > Use the bundled crates for now to finish with core-updates-frozen and
> > >
> > > revisit this immediately on core-updates (not frozen).
> >
> > This option will involved a rebuild on x86_64, but the advantage is that
> >
> > we’ll be safe going forward: we won’t accidentally cause world rebuilds
> >
> > just because an obscure crate somewhere has been upgraded.
> >
> > [...]
> >
> > > I'm currently leaning option 2b, it'll get us past this hurdle for
> > >
> > > core-updates-frozen and let us make changes to the crates as we work to
> > >
> > > integrate them more fully into Guix.
> >
> > Same here; it’s not ideal, but it seems like the most reasonable
> >
> > short-term option.
> >
> > If there are no objections, I’d suggest that you go ahead with this
> >
> > plan.
>
> I agree that 2b is the most sensible option in our current situation.

As a developer and new-ish Guix user (and not someone familiar with rust), I am also in favor of 2b. Dealing with 200+ dependencies takes time, and core-updates-frozen has been on the cusp of merging for some time now.

I'd like to see c-u-f merged back into master sooner, as master lacks support for newer hardware while also getting regular package updates that are only periodically merged to core-updates-frozen. Even before the c-u-f sprint last month where I switched all of my systems to c-u-f, I had one system that was first a frankensteined master before finally managing to switch it to c-u-f, to pick up a newer mesa that wasn't unusably buggy on a Radeon RX 6700 XT.

Cheers,
Kaelyn

P.S. Regarding switching my systems, the only issue I've run into that hasn't been fixed is that tigervnc only recently added support for building against xorg-server 21.1.1, and the current tigervnc release (1.12.0) was released before that support landed. (I have a standalone package definition for building a recent git commit.)
Toggle quote (4 lines)
>
> ------------------------------------------------------------------------
>
> Ricardo
E
E
Efraim Flashner wrote on 12 Dec 2021 14:20
Re: [bug#51845] [CORE-UPDATES] librsvg and rust
(name . Ricardo Wurmus)(address . rekado@elephly.net)
YbX3GXfkWWFejouP@3900XT
On Wed, Dec 08, 2021 at 02:36:11PM +0000, Ricardo Wurmus wrote:
Toggle quote (48 lines)
>
> Ludovic Courtès <ludo@gnu.org> writes:
>
> > Hello!
> >
> > For the record, this is a followup to Efraim’s proposal in
> > <https://issues.guix.gnu.org/51845>.
> >
> > Efraim Flashner <efraim@flashner.co.il> skribis:
> >
> >> Option 1:
> >> Track down the ~220 crates which form the dependency graph (of crates)
> >> for librsvg and pin them until the next core-updates cycle. Continue
> >> like with other packages and add newer versions (like cmake or meson) as
> >> packages need them.¹
> >
> > The advantage of this approach is that we could do it incrementally: we
> > could merge ‘core-updates-frozen’ today and just add pinned variants of
> > these 200+ crates as needed as time passes. The downside is that it’s a
> > lot of crates to take care of, and we might still accidentally overlook
> > seemingly innocuous crate upgrades that end up causing major rebuilds.
> >
> >> Option 2:
> >> Use the bundled crates and treat it as just part of the librsvg source
> >> code.²
> >>
> >> Option 2b:
> >> Use the bundled crates for now to finish with core-updates-frozen and
> >> revisit this immediately on core-updates (not frozen).
> >
> > This option will involved a rebuild on x86_64, but the advantage is that
> > we’ll be safe going forward: we won’t accidentally cause world rebuilds
> > just because an obscure crate somewhere has been upgraded.
> >
> > [...]
> >
> >> I'm currently leaning option 2b, it'll get us past this hurdle for
> >> core-updates-frozen and let us make changes to the crates as we work to
> >> integrate them more fully into Guix.
> >
> > Same here; it’s not ideal, but it seems like the most reasonable
> > short-term option.
> >
> > If there are no objections, I’d suggest that you go ahead with this
> > plan.
>
> I agree that 2b is the most sensible option in our current situation.

Patches pushed to core-updates-frozen. I added a TODO to fix the
situation.

Thanks for the input everyone.

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmG19xkACgkQQarn3Mo9
g1F0chAAqda7uUzc2xvk9q9xPOzlmkzfF6ENpH2bhJu2rkppBmgCVsVPoBSyTxQv
LEJ8WCpMjo9h8V50+VYT/XeA1MdpnGGZwZjIV8rX4of8otupp1qBGiK3MlLORSam
MgfF5NPkhI7Z8m8644JllD8Mb0V9i4gIjIa424ZnFNo5WoRw52RgKbQkzr9uuZPK
t68uS5w9CVrEQnhVL78/x5KQAK0oCRE2bVTv9ETNtpD06iqkKvvsu/Jq4VaN/PWP
q2Rqe17wlQFk6TXCR0fAjZqZMDkLKkoycTOrzXGJ16eoCG8DcIb0GlF67fQRpO1+
JzAZwcMkQ18peuG/MUDs0zmEHDfTtbASDILBnMQmwDcPZZyysWVOltXxyrVrvZh9
c1TtdeYT2XiYtV1oK6Rm93jiigVbZG4lHzPak5C1/f0v3Y+GTMx8vQWfUfzNSQzV
yuSrfEpbG9Qf2YnxunNDAVNQQ3P4H8jsDsq0VwNqzlEILXrSaumtTNMjFkzWDkr1
seC0PZjdlUDKD0RMXfbuM15Gc/xweonxM/+BHQjJTsrVl7uqnRRw9xxfwv0meZDJ
7fmx09YQVYnfL2TI6XKLYhlu8NpbBMOuNgtYCDBm96/CPq9ycbl5Ly7BpQP3/cek
lYnoGTwmXxSNiq7ts7da4MJaBHV4nc9li+RMHItR0NEwq6g/03E=
=dLbl
-----END PGP SIGNATURE-----


Closed
?