Why is only rust-1.60 exported when 1.65 is defined?

Done

Details

7 participants

Greg Hogan
Jonas Møller
Matthew James Kraai
Maxim Cournoyer
(
ROCKTAKEY
Simon Tournier

Owner: unassigned

Submitted by: Jonas Møller

Severity: wishlist

Jonas Møller wrote on 8 Mar 2023 17:09

Recipients:(address . bug-guix@gnu.org)

Message-ID:0f93d56a-f744-b7c6-2be6-2dcf7fe08741@moesys.no

Hi Guix! The rust.scm file says

Toggle quote (8 lines)

> ;;; Note: Only the latest versions of Rust are supported and tested. The

> ;;; intermediate rusts are built for bootstrapping purposes and should not

> ;;; be relied upon. This is to ease maintenance and reduce the time

> ;;; required to build the full Rust bootstrap chain.

> ;;;

> ;;; Here we take the latest included Rust, make it public, and re-enable tests

> ;;; and extra components such as rustfmt.

And then proceeds to define-public rust as rust-1.60, and I was wondering if there's any particular reason why a year-old version is used rather than the 1.65 version. This seems like a mistake, given that the comment claims that the "latest included Rust" should be made public.

This is especially troublesome for Rust on Guix because of both how fast its ecosystem moves onto new language/tooling features, and because using rustup (the solution for this on other slow-moving distros) relies on pre-built executables that don't work out-of-the-box on Guix.

— Mvh Jonas Møller

Attachment: file

Matthew James Kraai wrote on 12 Mar 2023 15:14

Recipients:(address . 62064@debbugs.gnu.org)

Message-ID:ZA3eSbhOmLqXT27r@carlo

Hi,

I believe that rust-1.60 is exported when 1.65 is defined because

changing the exported version would require rebuilding too many

dependencies.

unmatched-paren mentioned adding a rust-latest package on IRC to avoid

this issue.

Matt

Greg Hogan wrote on 13 Mar 2023 14:36

Re: bug#62064: Why is only rust-1.60 exported when 1.65 is defined?

Recipients:(name . Jonas Møller)(address . jonas@moesys.no)(address . 62064@debbugs.gnu.org)

Message-ID:CA+3U0ZkA-JMgyTNn374TTLBPhBfYw76cnO1f1smScmtU17OpFA@mail.gmail.com

On Sun, Mar 12, 2023 at 8:12?AM Jonas Møller via Bug reports for GNU

Guix <bug-guix@gnu.org> wrote:

Toggle quote (18 lines)>
> Hi Guix! The rust.scm file says
>
> ;;; Note: Only the latest versions of Rust are supported and tested.  The
> ;;; intermediate rusts are built for bootstrapping purposes and should not
> ;;; be relied upon.  This is to ease maintenance and reduce the time
> ;;; required to build the full Rust bootstrap chain.
> ;;;
> ;;; Here we take the latest included Rust, make it public, and re-enable tests
> ;;; and extra components such as rustfmt.
>
> And then proceeds to define-public rust as rust-1.60, and I was wondering if there's any particular reason why a year-old version is used rather than the 1.65 version. This seems like a mistake, given that the comment claims that the "latest included Rust" should be made public.
>
> This is especially troublesome for Rust on Guix because of both how fast its ecosystem moves onto new language/tooling features, and because using rustup (the solution for this on other slow-moving distros) relies on pre-built executables that don't work out-of-the-box on Guix.
>
>
> — Mvh Jonas Møller

Hi Jonas,

Is this question in regards to the current or future status of Rust in

Guix? Historically, updating Rust has been considered a core-updates

level change due to the number of dependent packages [0]. Under the

new feature branch workflow there was a recent announcement [1] by the

Rust team that 1.67 is in testing [2].

Can Guix's build farm keep up with Rust's six week release cycle? The

compute looks to be available but I don't recall a discussion for

consideration of storage or a blueprint for pruning old packages.

[0] https://guix.gnu.org/manual/en/html_node/Submitting-Patches.html

[1] https://lists.gnu.org/archive/html/guix-devel/2023-03/msg00002.html

[2] https://ci.guix.gnu.org/jobset/rust-team

Greg

Simon Tournier wrote on 13 Mar 2023 15:32

Recipients:

Message-ID:873568lo20.fsf@gmail.com

Hi,

On mer., 08 mars 2023 at 16:09, Jonas Møller via Bug reports for GNU Guix <bug-guix@gnu.org> wrote:

Toggle quote (14 lines)>> ;;; Note: Only the latest versions of Rust are supported and tested. The
>> ;;; intermediate rusts are built for bootstrapping purposes and should not
>> ;;; be relied upon. This is to ease maintenance and reduce the time
>> ;;; required to build the full Rust bootstrap chain.
>> ;;;
>> ;;; Here we take the latest included Rust, make it public, and re-enable tests
>> ;;; and extra components such as rustfmt.
>
> And then proceeds to define-public rust as rust-1.60, and I was
> wondering if there's any particular reason why a year-old version is
> used rather than the 1.65 version. This seems like a mistake, given
> that the comment claims that the "latest included Rust" should be made
> public.

Well, I know few about Rust and I guess it is because using ’rust-1.65’

as default Rust is a world-rebuild.

Toggle snippet (5 lines)

$ guix refresh rust -l | cut -f1 -d':'

Building the following 4560 packages would ensure 6637 dependent packages are rebuilt

Toggle quote (6 lines)

> This is especially troublesome for Rust on Guix because of both how

> fast its ecosystem moves onto new language/tooling features, and

> because using rustup (the solution for this on other slow-moving

> distros) relies on pre-built executables that don't work

> out-of-the-box on Guix.

Well, the issue when exporting ’rust-1.65’ is that it would possible

incompatible with the Rust packages provided by Guix and compiled with

’rust’ (1.60).

Maybe, one could imagine a package transformation

’package-with-explicit-rust’ as it is done for Python and OCaml.

Cheers,

simon

Jonas Møller wrote on 13 Mar 2023 21:59

Recipients:

Message-ID:66be3bb1-5adb-4e6b-f050-6159da88124e@moesys.no

Toggle quote (4 lines)

> Well, the issue when exporting ’rust-1.65’ is that it would possible

> incompatible with the Rust packages provided by Guix and compiled with

> ’rust’ (1.60).

Rust has very strong stability guarantees between 1.x releases, and has a very extensive set of tooling/infrastructure to make sure no breaking changes make their way into a release (see crater [1]). Any Rust-based package big enough to be packaged by Guix will have had its test-suite confirmed to work with the new release of cargo/rustc long before any new release is made (and in many cases before pull-requests are merged.)

Nevertheless, rebuilding all Rust packages on every rustc release isn't strictly necessary. (Improvements in optimizations do happen, but they are mosty minor release-to-release.)

But I agree with your ’package-with-explicit-rust’ idea, it is very unfortunate that "Rust, the development environment used by Guix users building Rust software" and "Rust, the build environment we use to build all Guix Rust packages" got conflated in Guix.

[1]: https://github.com/rust-lang/crater

— Mvh Jonas Møller

Attachment: file

(

( wrote on 13 Mar 2023 22:11

Recipients:

Message-ID:CR5KC0A0DKM7.2RF34MV6GTLZI@guix-framework

Hi,

On Wed Mar 8, 2023 at 4:09 PM GMT, Jonas Møller via Bug reports for GNU Guix wrote:

Toggle quote (2 lines)

> And then proceeds to define-public rust as rust-1.60, and I was wondering if there's any particular reason why a year-old version is used rather than the 1.65 version. This seems like a mistake, given that the comment claims that the "latest included Rust" should be made public.

There's another important reason:

rust != rust-1.60

The packages are not one and the same, despite the former being derived from the

latter. The RUST-1.X packages are not full Rust packages; they are cut down

intentionally (no tests, no non-essential tools) to reduce bootstrapping time,

which means we need to essentially revert those changes in the public RUST package

to get something that we can actually expose to users.

That's why the RUST variable isn't defined like this:

(define-public rust rust-1.60)

It's defined like this:

(define-public rust

(package

(inherit rust-1.60)

;; --8<-- a VERY extensive set of overrides to readd removed stuff --8<--

…))

So to add a RUST-LATEST, we'd need to have another one of these monstrosities

(seriously, it's evil https://git.savannah.gnu.org/cgit/guix.git/tree/gnu/packages/rust.scm#n663)

but because it's mostly source code patches, we couldn't just make some kind of

MAKE-PUBLIC-RUST procedure, as a SUBSTITUTE* that works for 1.60 may not work

for 1.65...

-- (

Simon Tournier wrote on 14 Mar 2023 09:32

Recipients:

Message-ID:864jqn91hw.fsf@gmail.com

Hi Jonas,

On Mon, 13 Mar 2023 at 20:59, Jonas Møller via Bug reports for GNU Guix <bug-guix@gnu.org> wrote:

Toggle quote (12 lines)>> Well, the issue when exporting ’rust-1.65’ is that it would possible
>> incompatible with the Rust packages provided by Guix and compiled with
>> ’rust’ (1.60).
>
> Rust has very strong stability guarantees between 1.x releases, and
> has a very extensive set of tooling/infrastructure to make sure no
> breaking changes make their way into a release (see crater [1]). Any
> Rust-based package big enough to be packaged by Guix will have had its
> test-suite confirmed to work with the new release of cargo/rustc long
> before any new release is made (and in many cases before pull-requests
> are merged.)

To test your claim, I would suggest to write a manifest using package

transformations [1] and then rebuild all Rust packages from Guix.

1: https://guix.gnu.org/manual/devel/en/guix.html#Package-Transformation-Options

2: https://guix.gnu.org/manual/devel/en/guix.html#Defining-Package-Variants

Toggle quote (4 lines)

> Nevertheless, rebuilding all Rust packages on every rustc release

> isn't strictly necessary. (Improvements in optimizations do happen,

> but they are mosty minor release-to-release.)

I do not understand how this could be implemented with Guix. Could you

be more specific?

Well, all in all, I do not think this report is a bug but a wish list.

Instead, I would suggest to discuss the actionable tasks for updating

Rust [3] in the mailing list guix-devel, WDYT?

3: https://guix.gnu.org/manual/devel/en/guix.html#Defining-Package-Variants

Cheers,

simon

Jonas Møller wrote on 14 Mar 2023 09:59

Recipients:

Message-ID:587407cb-5045-2c86-cf9b-6d331c8bc1b3@moesys.no

I think it's fine for Guix to use a specific version of Rust internally, and to be conservative about updating that.

But Guix becomes unsuitable for anyone developing software in Rust, if this technical detail in the package manager means that they can't use newer versions of the toolchain.

I think it would be best if Guix separated these concerns, by disconnecting the "Rust we use to build packages internally" with the "Rust that gets installed with `guix install rust`"