Mutable guix shell environments

sample useage:

$ cd project

$ guix package --manifest=manifest.scm --profile=.guix-profile

$ guix shell # --profile=.guix-profile is implicit

do some stuff. realize that you want to bring in another package.

$ guix install --profile=.guix-profile additional-package

Then additional package is instantly available. This is especially useful to lisp programmers because, currently, bringing in an additional library involves restarting shell, lisp process, reloading source files, regenerating process state.

Am Sonntag, dem 13.03.2022 um 18:21 +0000 schrieb Charles:

This one LGTM and it even avoids some problems we're having with

caches. I'll keep it open for bikeshedding suggestions w.r.t. the name

of the directory to detect, though.

This one not so much. You already have GUIX_ENVIRONMENT set, which

should be enough for lookup purposes. If it's about manipulating PATH

and other environment variables, I think it'd better to do this

manually – you could even spawn a lightweight shell on your own by

simply doing the following:

$ $SHELL

$ GUIX_PROFILE=/path/to/profile

$ source $GUIX_PROFILE/etc/profile

If you want mutable environments, that's the easiest and imho best way

of achieving such a thing. Guix is functional by design and we should

not compromise on that.

As far as using $GUIX_PROFILE together with `guix shell' is concerned,

if anything is even holding it back, it's that we don't respect

GUIX_PROFILE in the Scheme code that adds those variables. Which to be

fair might be for the best, as GUIX_PROFILE is not guaranteed to

correspond to this invocation of `guix shell' when people are careless.

That's not to say that support for this couldn't be added, but at the

very least we'd have to be more careful about it.

Cheers

Thanks for taking a look Liliana. I'm glad you like guix shell detecting a local profile.

This is basically what I was going for. I just find this useful to be built in. When I think "guix shell", I think: "make an environment where I have everything I need". If I want to adjust what I need, why do I need to restart everything from scratch. If I did not know about guix shell and did all my development using ~/.guix-profile, this functionallity would already be available.

not compromise on that.

I totally agree, and I do not think this does compromise it. With my patch, guix shells using guix.scm or manifest, would be unchanged. Currently, guix shell --profile, has strickly less functionallity than using the profile otherwise, this could be confusing (as it is to me). Profiles are not really mutable since they use generations; the subject line is a bit of a misnomer.

Yeah, before my patch, guix shell --profile, would not be set to the profile I'm using. This actually makes sense, because, we are not really using it. Guix shell just happens to be using all the same packages that were available in that profile. Confusingly, If the profile, is updated, the shell is now out of sync with it. My patch should make using guix shell --profile more intuitive.

------- Original Message -------

On Sunday, March 13th, 2022 at 2:51 PM, Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:

Hi Charles,

Am Sonntag, dem 13.03.2022 um 23:38 +0000 schrieb Charles:

Yes, but it'd also be broken af. You are focusing on a very particular

use case and ignoring all others. What if instead of just "a package"

I wanted to add or remove a package that defines a search path, e.g.

Emacs? In the adding case, EMACSLOADPATH would not get defined and in

the removing case it would still linger. What you describe as desired

functionality is not at all well-defined. Using `guix shell --profile'

on the other hand has the same semantics as the other `guix shell'

commands, and is thus imo less confusing once you start to think about

it.

Cheers

As with ~/.guix-profile, you will have to . "$GUIX_PROFILE/etc/profile" just like the hint says. For removing, they linger, that is exactly what the default profile does too.

This comes down to us disagreeing which is more important & useful: guix shell semantics vs profile semantics. guix shell --profile could go either way.

Another way to look at it is that I this sequence of 3 commands very useful (so might others). They clearly have something to do with guix profiles, and they spawn a new shell. Thus I thought guix shell --profile was the perfect place for them. You claim they do not belong there. Is there a better place you would suggest?

Charles via Guix-patches via schreef op zo 13-03-2022 om 18:21 [+0000]:

If this is added, then this needs to be documented in the manual,

preferably with some examples on how to use it.

Greetings,

Maxime.

Charles via Guix-patches via schreef op zo 13-03-2022 om 18:21 [+0000]:

Alternative suggestion, which IMHO fits the non-persistent naturre of

"guix shell" better:

$ cd project

$ guix shell --allow-temporary-modifications # fails on read-only file systems

$ echo $GUIX_ENVIRONMENT

# a temporary (mutable) profile was created

# install things in the $GUIX_ENVIRONMENT profile

$ guix install foo bar ...

As noted by lilyp, this won't work in all situations due to search path

issues. These could be resolved by always setting a few search paths

(PATH, EMACSLOADPATH, GUILE_LOAD_PATH, INFOPATH, ...) (even in not currently

present in the profile!), at cost of having to make them ‘magical’ in some

sense, which ludo does not seem to like IIUC. (This could also be done

for ~/.guix-profile)

(I don't like it much either, but it seems a very convenient solution that

does not seem to have any concrete downsides and IMHO it seems much better

than telling the user to re-source the profile.)

Greetings,

Maxime.

Thanks for taking a look Maxime.

I actually like Your alternate suggestion better (though the name --allow-temporary-modifications is a bit long). I'm not sure how to implement it though.

Hi,

Maxime Devos <maximedevos@telenet.be> skribis:

Or ‘--transient’.

I like this approach.

Charles’ proposal reminds me of ‘module’, a venerable “environment

management” tool widely used in high-performance computing (HPC):

http://modules.sourceforge.net/

That command allows for an incremental style like Charles proposes:

module load gcc # adds GCC to $PATH

module load libgc # adds libgc to $C_INCLUDE_PATH

module unload libgc # removes libgc from $C_INCLUDE_PATH

For this to work though, ‘module’ is actually a shell function: that

lets it adjust environment variables here and now.

Anyway, I digress…

I thought about this other approach:

1. ‘guix shell’ always builds a new profile, as it already does.

2. When ‘GUIX_ENVIRONMENT’ is defined, it computes search paths as a

combination of the new profile and that pointed to by

$GUIX_ENVIRONMENT.

3. It spawns a new shell, like it already does.

Thus, instead of doing upfront:

guix shell a b c

One could do:

$ guix shell a

[env]$ guix shell b

[env]$ guix shell c

It is more accurate but less flexible than what you propose though,

because only child processes of the nested shells would see the

“changes”. So maybe not a good idea, after all.

Hmm, need more thought…

Thanks,

Ludo’.

Hi,

On Mon, 14 Mar 2022 at 23:19, Ludovic Courtès <ludo@gnu.org> wrote:

While I understand the need of Charles's proposal, especially in

development or exploration, I am convinced that this "incremental

style" is the root of many unreproducible computational environments.

The "incremental style" is a quick and dirty approach for creating a

computational environment and because the discipline is wrong then it

lead more than often to hard-to-reproduce computational environment,

therefore Guix should not try to mimick, IMHO. I agree with Liliana's

words: «Guix is functional by design and we should not compromise on

that.». To me, it is as programmers being strongly used to imperative

style complaining about the paradigm shift of the functional style --

or vice-versa... I digress. :-)

To mitigate because we have to work and sadly need to have things done...

..somehow, it would ease if when requiring the package 'b', the

environment is "aware" of the search paths of the environment required

by the package 'a'. For instance,

Here, 'coreutils' is not "aware" of the parent 'man-db' and thus

MANPATH does not contains the expected 114 files.

Using an option, say '--transient', a new profile is created each time

but what this new profile contains would depends of the parent

profile. Last, the manipulation of a "temporary" profile is done via

"guix shell" thus this "transient" profile should be manipulated via

"guix shell" and not "guix package" (or aliases as "guix install").

However, what could appear still annoying is the stack of environment:

guix shell a

guix shell b

guix shell c

then remove the package 'a' from the environment requires to exit 2

environments and then re-create them. Need more thoughts. :-)

Cheers,

simon

Hello simon; thank you for your input.

If profiles are used, it can be exported to a manifest.

I want to reiterate that I am not proposing anything less FP than profiles.

Nested shells do not fit my use case because a long running process started in shell "a" does not become aware of packages installed on shell "b".

I chose "guix install" for my proposal because it already has the functionality of adding a new generation. I do not think it would be good to change the semantics of nested shells.

I failed to mention earlier that I like the name "--transient"