Duplicate entries in various environment variables

Open

Details

5 participants

Chris Marusich
Leo Prikler
Leo Famulari
Ludovic Courtès
Maxim Cournoyer

Owner: unassigned

Submitted by: Chris Marusich

Severity: important

Chris Marusich wrote on 26 Mar 2016 04:59

Recipients:(address . bug-guix@gnu.org)

Message-ID:874mbt28k4.fsf@gmail.com

Hi,

I've noticed that my environment variables contain duplicate (sometimes

more) entries. This occurs regardless of whether the user logs in

directly via a tty or via a desktop environment like GNOME.

This behavior should be reproducible. I created a new user to test

this. When I logged into GNOME on GuixSD (built via "guix system

reconfigure" using git commit 43a58615bfc37a053ddaca1fcf11af5c14c0b4b6)

as the new user, I opened up the "Terminal" program (which launched bash

as an interactive, non-login shell). I then typed

Toggle snippet (3 lines)

env > /tmp/myenv

Then, I analyzed the results. First, I defined a little helper function

to do the analysis. For each environment variable, it counts the number

of times each colon-delimited entry occurs in that environment variable

and prints them out:

Toggle snippet (10 lines)

count_env_var_occurrences ()

{

for environment_variable in $(cat $1 | cut -d = -f 1 | sort);

echo $environment_variable;

cat $1 | grep ^${environment_variable}= | cut -d = -f 2 | tr : '\n' | sort | uniq -c;

done

}

Then I ran it, and got the following results:

Toggle snippet (166 lines)$ count_env_var_occurrences /tmp/myenv 
_
/run/current-system/profile/bin/env
DBUS_FATAL_WARNINGS
0
DBUS_SESSION_BUS_ADDRESS
abstract
unix
DISPLAY

0.0
GIO_EXTRA_MODULES
/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/lib/gio/modules
/gnu/store/ygn4vavcgr79p86jrlmvdycjs81ki4dl-dconf-0.24.0/lib/gio/modules
/run/current-system/profile/lib/gio/modules
GIT_SSL_CAINFO
/etc/ssl/certs/ca-certificates.crt
GI_TYPELIB_PATH
/gnu/store/1sp33d4j5gx2xd64mncc30q0x01hjbf4-cogl-1.22.0/lib/girepository-1.0
/gnu/store/38ir3s81qxhbpa8v8rwf8cbjdcmsr2q0-atk-2.18.0/lib/girepository-1.0
/gnu/store/3k8lhk39979kbi3m26fa191wq5jgxf7f-polkit-0.113/lib/girepository-1.0
/gnu/store/5l1fab8vi6zpiy8m61aiv2qbr8gfsjha-json-glib-1.0.4/lib/girepository-1.0
/gnu/store/67a1962vx46b5shf3myxi23jvziwaclj-gdm-3.18.2/lib/girepository-1.0
/gnu/store/73wzvqzffyx867r1rbck5w65by6lbapy-gnome-bluetooth-3.18.1/lib/girepository-1.0
/gnu/store/apf78hww8abnhsg0pr6d00gc5w1g7y5h-libgweather-3.18.1/lib/girepository-1.0
/gnu/store/b2qdk8kxcb6ls2hdbzgv51smljk2bqh3-clutter-1.24.2/lib/girepository-1.0
/gnu/store/bxb7i5qiml1vlxbsks4fxnv4vv5pzac3-gnome-desktop-3.18.1/lib/girepository-1.0
/gnu/store/cr289j8247p0mv39j26l4gxbdf6b7g11-network-manager-1.0.10/lib/girepository-1.0
/gnu/store/dzdqr0yh7csf3j4ash6akzxjdbqgfln2-network-manager-applet-1.0.10/lib/girepository-1.0
/gnu/store/fqp33841n42qn6qx3ma1w9xyq329abyh-gst-plugins-base-1.6.3/lib/girepository-1.0
/gnu/store/g7ii95bmy2aq4z5x17wvsqlz4f8lmn5a-gcr-3.18.0/lib/girepository-1.0
/gnu/store/h4p0b9qp1v1pxnmlgdpcy53xxddjzajp-gobject-introspection-1.46.0/lib/girepository-1.0
/gnu/store/hkl1hwx7cqgb23115rnacin4lywy39fq-libxklavier-5.3/lib/girepository-1.0
/gnu/store/l2qcfja7vspdki8birvm80xs82acm2va-gdk-pixbuf-2.32.3/lib/girepository-1.0
/gnu/store/m67pffz37qvkr63a7d5i92z10cwznsy9-accountsservice-0.6.40/lib/girepository-1.0
/gnu/store/mfdgb0i95bf4j89zc4mnnbd6hp3v08yx-at-spi2-core-2.18.1/lib/girepository-1.0
/gnu/store/p79p86324sb5yc80m21nk7q8cjsixb3s-libgee-0.18.0/lib/girepository-1.0
/gnu/store/pc9w7sz63d0mims6hbxnzc47g3jy55id-pango-1.38.1/lib/girepository-1.0
/gnu/store/pil657vzrprd2bfdhg7q8vhd3f6jwzpb-libsoup-2.52.1/lib/girepository-1.0
/gnu/store/rdmyj3ylsfbw57kpdgi9iz15s7xfzxzr-libsecret-0.18.3/lib/girepository-1.0
/gnu/store/rjhhm5ixkmgqpxfiik6k1rncwrvdz0z1-upower-0.99.3/lib/girepository-1.0
/gnu/store/rsywwl626s8fmmwf154sn2x9si62b4k1-evolution-data-server-3.18.2/lib/girepository-1.0
/gnu/store/s1nig50yjs5g6fnbl29dyi0d6cp1vza9-telepathy-glib-0.24.1/lib/girepository-1.0
/gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/lib/girepository-1.0
/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/girepository-1.0
/gnu/store/vmh2w8bsh5w3arrrkcvb468j98nvbjqf-gstreamer-1.6.3/lib/girepository-1.0
/gnu/store/ymx1l8l2gm1jmljh5y24p4f5gk8i2j3c-caribou-0.4.19/lib/girepository-1.0
/gnu/store/z2dh08qp3k1829dcsz7wcrinccmj6743-ibus-1.5.11/lib/girepository-1.0
/gnu/store/zlq812qd3cykkizdip0l7qw7m2pffj1h-telepathy-logger-0.8.2/lib/girepository-1.0
/gnu/store/zvi3k12q95i992gnvhddqs0iildyz4lp-harfbuzz-1.0.6/lib/girepository-1.0
GJS_DEBUG_OUTPUT
stderr
GJS_DEBUG_TOPICS
JS ERROR;JS LOG
GNOME_DESKTOP_SESSION_ID
this-is-deprecated
GST_PLUGIN_PATH
/home/alice/.guix-profile/lib/gstreamer-1.0
GTK_PATH
/gnu/store/7hdyj0dkzp3dbl878g042g300yycasrx-libcanberra-0.30/lib/gtk-3.0
/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/gtk-3.0
/gnu/store/ymx1l8l2gm1jmljh5y24p4f5gk8i2j3c-caribou-0.4.19/lib/gtk-3.0
/gnu/store/z2dh08qp3k1829dcsz7wcrinccmj6743-ibus-1.5.11/lib/gtk-3.0
GUILE_LOAD_COMPILED_PATH
/run/current-system/profile/lib/guile/2.0/ccache
/run/current-system/profile/share/guile/site/2.0
GUILE_LOAD_PATH
/run/current-system/profile/share/guile/site/2.0
GUIX_GTK2_PATH
/run/current-system/profile/lib/gtk-2.0
GUIX_GTK3_PATH
/run/current-system/profile/lib/gtk-3.0
HOME
/home/alice
INFOPATH
/home/alice/.guix-profile/share/info
/run/current-system/profile/share/info
LANG
en_US.UTF-8
LINUX_MODULE_DIRECTORY
/run/booted-system/kernel/lib/modules
LOGNAME
alice
MAIL
/var/mail/alice
MANPATH
/home/alice/.guix-profile/share/man
/run/current-system/profile/share/man
PATH
/gnu/store/a9sqlwcl02z89kh6v5n6yzxvxw5qbs14-glib-2.46.1-bin/bin
/home/alice/.guix-profile/bin
/run/current-system/profile/bin
/run/current-system/profile/sbin
/run/setuid-programs
PS1
\u@\h \w\$ 
PWD
/home/alice
SESSION_MANAGER
local/carbuncle
/tmp/.ICE-unix/383
@/tmp/.ICE-unix/383,unix/carbuncle
SHELL
/gnu/store/7cdd8s466qyjh64m0byq0rz9gk1jid40-bash-4.3.42/bin/bash
SHLVL
2
SSH_AUTH_SOCK
/run/user/30011/keyring/ssh
SSL_CERT_DIR
/etc/ssl/certs
SSL_CERT_FILE
/etc/ssl/certs/ca-certificates.crt
TERM
xterm-256color
TZ
America/Los_Angeles
TZDIR
/gnu/store/n80pk0smpzdlshkcgccgga8xy5p2vwz3-tzdata-2015g/share/zoneinfo
USER
alice
VTE_VERSION
4203
WINDOWID
23068678
XAUTHORITY
/home/alice/.Xauthority
XDG_CONFIG_DIRS
/home/alice/.guix-profile/etc/xdg
/run/current-system/profile/etc/xdg
XDG_CURRENT_DESKTOP
GNOME
XDG_DATA_DIRS
/gnu/store/27c82f3szfn2d97ms56hkpsb78q9myhd-gnome-settings-daemon-3.18.2/share
/gnu/store/67a1962vx46b5shf3myxi23jvziwaclj-gdm-3.18.2/share
/gnu/store/73wzvqzffyx867r1rbck5w65by6lbapy-gnome-bluetooth-3.18.1/share
/gnu/store/apf78hww8abnhsg0pr6d00gc5w1g7y5h-libgweather-3.18.1/share
/gnu/store/bahndy6sxfxw6g49djc10mpa7292na40-gnome-session-3.18.1.2/share
/gnu/store/dzdqr0yh7csf3j4ash6akzxjdbqgfln2-network-manager-applet-1.0.10/share
/gnu/store/g7ii95bmy2aq4z5x17wvsqlz4f8lmn5a-gcr-3.18.0/share
/gnu/store/kc8j8zcwjzr04nlkm16gy44a7vfr26i8-mutter-3.18.1/share
/gnu/store/km25kxpv999ihj058xnvniwycl1zhajl-gnome-control-center-3.18.2/share
/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/share
/gnu/store/rsywwl626s8fmmwf154sn2x9si62b4k1-evolution-data-server-3.18.2/share
/gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/share
/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/share
/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/share
/gnu/store/xnkkvlwvwbv2zzvbca3sclf41m6jxzza-gnome-shell-3.18.3/share
/gnu/store/ymx1l8l2gm1jmljh5y24p4f5gk8i2j3c-caribou-0.4.19/share
/gnu/store/z2dh08qp3k1829dcsz7wcrinccmj6743-ibus-1.5.11/share
/gnu/store/zlq812qd3cykkizdip0l7qw7m2pffj1h-telepathy-logger-0.8.2/share
/home/alice/.guix-profile/share
/run/current-system/profile/share
XDG_MENU_PREFIX
gnome-
XDG_RUNTIME_DIR
/run/user/30011
XDG_SEAT
seat0
XDG_SESSION_ID
c1
XDG_VTNR
7
X_XFCE4_LIB_DIRS
/run/current-system/profile/lib/xfce4

As you can see, there are some environment variables with up to 9
duplicate entries.  Is this expected?  Is it a problem?  Why is it
happening?  Assuming that it is not expected and that it is a problem,
how can we prevent it from happening?

I've tried opening multiple subshells (e.g., by running "bash" on the
command line) to see if the number of duplicate entries increases with
each subshell, but that is not the case.  Opening up a new subshell does
not seem to trigger (in the default GuixSD configuration with GNOME, at
least) whatever it is that it causing this duplication.

-- 
Chris

Ludovic Courtès wrote on 26 Mar 2016 19:53

Recipients:(name . Chris Marusich)(address . cmmarusich@gmail.com)(address . 23118@debbugs.gnu.org)

Message-ID:87io09ystm.fsf@gnu.org

Chris Marusich <cmmarusich@gmail.com> skribis:

Toggle quote (6 lines)

> This behavior should be reproducible. I created a new user to test

> this. When I logged into GNOME on GuixSD (built via "guix system

> reconfigure" using git commit 43a58615bfc37a053ddaca1fcf11af5c14c0b4b6)

> as the new user, I opened up the "Terminal" program (which launched bash

> as an interactive, non-login shell). I then typed

[...]

I think the main problem is that gnome-terminal is wrapped:

Toggle snippet (8 lines)

$ cat /gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/gnome-terminal

#!/gnu/store/b1yqjimbdh5bf9jnizd4h7yf110744j2-bash-4.3.42/bin/bash

export XDG_DATA_DIRS="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/share:/gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/share:/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/share:/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/share${XDG_DATA_DIRS:+:}$XDG_DATA_DIRS"

export GTK_PATH="/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/gtk-3.0${GTK_PATH:+:}$GTK_PATH"

export GIO_EXTRA_MODULES="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/lib/gio/modules:/gnu/store/ygn4vavcgr79p86jrlmvdycjs81ki4dl-dconf-0.24.0/lib/gio/modules${GIO_EXTRA_MODULES:+:}$GIO_EXTRA_MODULES"

exec -a "$0" "/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/.gnome-terminal-real" "$@"

This is done automatically by ‘glib-or-gtk-build-system’, but obviously

it’s not so great here.

I suppose that in practice, it would work to simply switch

gnome-terminal to ‘gnu-build-system’.

Could you try that?

There may still be other places that add extra entries, though.

Ludo’.

Ludovic Courtès wrote on 28 Mar 2016 18:53

control message for bug #23118

Recipients:(address . control@debbugs.gnu.org)

Message-ID:877fgmr1cf.fsf@gnu.org

severity 23118 important

Leo Famulari wrote on 28 Mar 2016 19:53

Re: bug#23118: Duplicate entries in various environment variables

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:20160328175301.GA9856@jasmine

On Sat, Mar 26, 2016 at 07:53:09PM +0100, Ludovic Courtès wrote:

Toggle quote (29 lines)> Chris Marusich <cmmarusich@gmail.com> skribis:
> 
> > This behavior should be reproducible.  I created a new user to test
> > this.  When I logged into GNOME on GuixSD (built via "guix system
> > reconfigure" using git commit 43a58615bfc37a053ddaca1fcf11af5c14c0b4b6)
> > as the new user, I opened up the "Terminal" program (which launched bash
> > as an interactive, non-login shell).  I then typed
> 
> [...]
> 
> I think the main problem is that gnome-terminal is wrapped:
> 
> --8<---------------cut here---------------start------------->8---
> $ cat /gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/gnome-terminal 
> #!/gnu/store/b1yqjimbdh5bf9jnizd4h7yf110744j2-bash-4.3.42/bin/bash
> export XDG_DATA_DIRS="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/share:/gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/share:/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/share:/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/share${XDG_DATA_DIRS:+:}$XDG_DATA_DIRS"
> export GTK_PATH="/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/gtk-3.0${GTK_PATH:+:}$GTK_PATH"
> export GIO_EXTRA_MODULES="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/lib/gio/modules:/gnu/store/ygn4vavcgr79p86jrlmvdycjs81ki4dl-dconf-0.24.0/lib/gio/modules${GIO_EXTRA_MODULES:+:}$GIO_EXTRA_MODULES"
> exec -a "$0" "/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/.gnome-terminal-real" "$@"
> --8<---------------cut here---------------end--------------->8---
> 
> This is done automatically by ‘glib-or-gtk-build-system’, but obviously
> it’s not so great here.
> 
> I suppose that in practice, it would work to simply switch
> gnome-terminal to ‘gnu-build-system’.
> 
> Could you try that?

I tried it. I had to add glib:bin as a native-input.

Trying to run gnome-terminal produces this error (I wrapped the lines):

$ gnome-terminal

Error constructing proxy for

org.gnome.Terminal:/org/gnome/Terminal/Factory0: Error calling

StartServiceByName for org.gnome.Terminal:

GDBus.Error:org.freedesktop.DBus.Error.Spawn.ChildSignaled: Process

org.gnome.Terminal received signal 5

Toggle quote (7 lines)> 
> There may still be other places that add extra entries, though.
> 
> Ludo’.
> 
> 
>

Leo Prikler wrote on 3 Dec 2020 11:58

Re: Duplicate entries in various environment variables

Recipients:(name . Chris Marusich)(address . cmmarusich@gmail.com)(address . 23118@debbugs.gnu.org)

Message-ID:6ca9bc2b5bb96b0ea6740c3412d6aa4379d553e9.camel@student.tugraz.at

Hello, Chris

Am Freitag, den 25.03.2016, 20:59 -0700 schrieb Chris Marusich:

Toggle quote (14 lines)> Hi,
> 
> I've noticed that my environment variables contain duplicate
> (sometimes
> more) entries.  This occurs regardless of whether the user logs in
> directly via a tty or via a desktop environment like GNOME.
> 
> [...]
> 
> As you can see, there are some environment variables with up to 9
> duplicate entries.  Is this expected?  Is it a problem?  Why is it
> happening?  Assuming that it is not expected and that it is a
> problem,
> how can we prevent it from happening?

It has been pointed out, that this is somewhat expected when wrapping
the same environment variable multiple times.  For instance, you as a
GNOME user might already have gtk+ in your GTK path if you run GNOME,
but GNOME applications can not rely on that and thus need to add their
own.  As the number of software components, that use it increases, so
does the number of mentions.  
In the special case of gnome-terminal, this is user-visible by printing
out env, but other applications get launched in a similar manner all
the time without you noticing.  Perhaps one could patch GNOME terminal
to clear those variables before spawning the shell, but that's going to
be a bit fiddly.  Alternatively, one could enforce GTK_PATH by using
"=" instead of prefix.

More generally, this can become an issue when environment variables
reach a certain size (and has led to bug reports in Guix before, which
have since been fixed).  POSIX mandates a syntax, that would allow
removing already present components first (see [1]), but I can hardly
imagine what monstrosities we would need to cook up to do this
reliably.  Not to mention, that some otherwise POSIX-compliant shells
might not implement that syntax (correctly).

Regards,
Leo

[1] 
https://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_02

Maxim Cournoyer wrote on 8 Oct 2022 03:51

Re: bug#23118: Duplicate entries in various environment variables

Recipients:(address . liliana.prikler@student.tugraz)

Message-ID:87fsfz3wwx.fsf@gmail.com

Hi,

[...]

Toggle quote (8 lines)

> More generally, this can become an issue when environment variables

> reach a certain size (and has led to bug reports in Guix before, which

> have since been fixed). POSIX mandates a syntax, that would allow

> removing already present components first (see [1]), but I can hardly

> imagine what monstrosities we would need to cook up to do this

> reliably. Not to mention, that some otherwise POSIX-compliant shells

> might not implement that syntax (correctly).

Why are we talking about POSIX here?  Our wrap-program procedure makes
use of Bash; we could make use of its features to filter duplicates,
right?  Unless I'm missing something.

-- 
Thanks,
Maxim

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 23118@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date