Duplicate entries in various environment variables

  • Open
  • quality assurance status badge
Details
5 participants
  • Chris Marusich
  • Leo Prikler
  • Leo Famulari
  • Ludovic Courtès
  • Maxim Cournoyer
Owner
unassigned
Submitted by
Chris Marusich
Severity
important
C
C
Chris Marusich wrote on 26 Mar 2016 04:59
(address . bug-guix@gnu.org)
874mbt28k4.fsf@gmail.com
Hi,

I've noticed that my environment variables contain duplicate (sometimes
more) entries. This occurs regardless of whether the user logs in
directly via a tty or via a desktop environment like GNOME.

This behavior should be reproducible. I created a new user to test
this. When I logged into GNOME on GuixSD (built via "guix system
reconfigure" using git commit 43a58615bfc37a053ddaca1fcf11af5c14c0b4b6)
as the new user, I opened up the "Terminal" program (which launched bash
as an interactive, non-login shell). I then typed

Toggle snippet (3 lines)
env > /tmp/myenv

Then, I analyzed the results. First, I defined a little helper function
to do the analysis. For each environment variable, it counts the number
of times each colon-delimited entry occurs in that environment variable
and prints them out:

Toggle snippet (10 lines)
count_env_var_occurrences ()
{
for environment_variable in $(cat $1 | cut -d = -f 1 | sort);
do
echo $environment_variable;
cat $1 | grep ^${environment_variable}= | cut -d = -f 2 | tr : '\n' | sort | uniq -c;
done
}

Then I ran it, and got the following results:

Toggle snippet (166 lines)
$ count_env_var_occurrences /tmp/myenv
_
1 /run/current-system/profile/bin/env
DBUS_FATAL_WARNINGS
1 0
DBUS_SESSION_BUS_ADDRESS
1 abstract
1 unix
DISPLAY
1
1 0.0
GIO_EXTRA_MODULES
9 /gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/lib/gio/modules
1 /gnu/store/ygn4vavcgr79p86jrlmvdycjs81ki4dl-dconf-0.24.0/lib/gio/modules
1 /run/current-system/profile/lib/gio/modules
GIT_SSL_CAINFO
1 /etc/ssl/certs/ca-certificates.crt
GI_TYPELIB_PATH
1 /gnu/store/1sp33d4j5gx2xd64mncc30q0x01hjbf4-cogl-1.22.0/lib/girepository-1.0
1 /gnu/store/38ir3s81qxhbpa8v8rwf8cbjdcmsr2q0-atk-2.18.0/lib/girepository-1.0
1 /gnu/store/3k8lhk39979kbi3m26fa191wq5jgxf7f-polkit-0.113/lib/girepository-1.0
1 /gnu/store/5l1fab8vi6zpiy8m61aiv2qbr8gfsjha-json-glib-1.0.4/lib/girepository-1.0
1 /gnu/store/67a1962vx46b5shf3myxi23jvziwaclj-gdm-3.18.2/lib/girepository-1.0
1 /gnu/store/73wzvqzffyx867r1rbck5w65by6lbapy-gnome-bluetooth-3.18.1/lib/girepository-1.0
1 /gnu/store/apf78hww8abnhsg0pr6d00gc5w1g7y5h-libgweather-3.18.1/lib/girepository-1.0
1 /gnu/store/b2qdk8kxcb6ls2hdbzgv51smljk2bqh3-clutter-1.24.2/lib/girepository-1.0
1 /gnu/store/bxb7i5qiml1vlxbsks4fxnv4vv5pzac3-gnome-desktop-3.18.1/lib/girepository-1.0
1 /gnu/store/cr289j8247p0mv39j26l4gxbdf6b7g11-network-manager-1.0.10/lib/girepository-1.0
1 /gnu/store/dzdqr0yh7csf3j4ash6akzxjdbqgfln2-network-manager-applet-1.0.10/lib/girepository-1.0
1 /gnu/store/fqp33841n42qn6qx3ma1w9xyq329abyh-gst-plugins-base-1.6.3/lib/girepository-1.0
1 /gnu/store/g7ii95bmy2aq4z5x17wvsqlz4f8lmn5a-gcr-3.18.0/lib/girepository-1.0
1 /gnu/store/h4p0b9qp1v1pxnmlgdpcy53xxddjzajp-gobject-introspection-1.46.0/lib/girepository-1.0
1 /gnu/store/hkl1hwx7cqgb23115rnacin4lywy39fq-libxklavier-5.3/lib/girepository-1.0
1 /gnu/store/l2qcfja7vspdki8birvm80xs82acm2va-gdk-pixbuf-2.32.3/lib/girepository-1.0
1 /gnu/store/m67pffz37qvkr63a7d5i92z10cwznsy9-accountsservice-0.6.40/lib/girepository-1.0
1 /gnu/store/mfdgb0i95bf4j89zc4mnnbd6hp3v08yx-at-spi2-core-2.18.1/lib/girepository-1.0
1 /gnu/store/p79p86324sb5yc80m21nk7q8cjsixb3s-libgee-0.18.0/lib/girepository-1.0
1 /gnu/store/pc9w7sz63d0mims6hbxnzc47g3jy55id-pango-1.38.1/lib/girepository-1.0
1 /gnu/store/pil657vzrprd2bfdhg7q8vhd3f6jwzpb-libsoup-2.52.1/lib/girepository-1.0
1 /gnu/store/rdmyj3ylsfbw57kpdgi9iz15s7xfzxzr-libsecret-0.18.3/lib/girepository-1.0
1 /gnu/store/rjhhm5ixkmgqpxfiik6k1rncwrvdz0z1-upower-0.99.3/lib/girepository-1.0
1 /gnu/store/rsywwl626s8fmmwf154sn2x9si62b4k1-evolution-data-server-3.18.2/lib/girepository-1.0
1 /gnu/store/s1nig50yjs5g6fnbl29dyi0d6cp1vza9-telepathy-glib-0.24.1/lib/girepository-1.0
1 /gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/lib/girepository-1.0
1 /gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/girepository-1.0
1 /gnu/store/vmh2w8bsh5w3arrrkcvb468j98nvbjqf-gstreamer-1.6.3/lib/girepository-1.0
1 /gnu/store/ymx1l8l2gm1jmljh5y24p4f5gk8i2j3c-caribou-0.4.19/lib/girepository-1.0
1 /gnu/store/z2dh08qp3k1829dcsz7wcrinccmj6743-ibus-1.5.11/lib/girepository-1.0
1 /gnu/store/zlq812qd3cykkizdip0l7qw7m2pffj1h-telepathy-logger-0.8.2/lib/girepository-1.0
1 /gnu/store/zvi3k12q95i992gnvhddqs0iildyz4lp-harfbuzz-1.0.6/lib/girepository-1.0
GJS_DEBUG_OUTPUT
1 stderr
GJS_DEBUG_TOPICS
1 JS ERROR;JS LOG
GNOME_DESKTOP_SESSION_ID
1 this-is-deprecated
GST_PLUGIN_PATH
1 /home/alice/.guix-profile/lib/gstreamer-1.0
GTK_PATH
3 /gnu/store/7hdyj0dkzp3dbl878g042g300yycasrx-libcanberra-0.30/lib/gtk-3.0
9 /gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/gtk-3.0
3 /gnu/store/ymx1l8l2gm1jmljh5y24p4f5gk8i2j3c-caribou-0.4.19/lib/gtk-3.0
3 /gnu/store/z2dh08qp3k1829dcsz7wcrinccmj6743-ibus-1.5.11/lib/gtk-3.0
GUILE_LOAD_COMPILED_PATH
1 /run/current-system/profile/lib/guile/2.0/ccache
1 /run/current-system/profile/share/guile/site/2.0
GUILE_LOAD_PATH
1 /run/current-system/profile/share/guile/site/2.0
GUIX_GTK2_PATH
1 /run/current-system/profile/lib/gtk-2.0
GUIX_GTK3_PATH
1 /run/current-system/profile/lib/gtk-3.0
HOME
1 /home/alice
INFOPATH
1 /home/alice/.guix-profile/share/info
2 /run/current-system/profile/share/info
LANG
1 en_US.UTF-8
LINUX_MODULE_DIRECTORY
1 /run/booted-system/kernel/lib/modules
LOGNAME
1 alice
MAIL
1 /var/mail/alice
MANPATH
1 /home/alice/.guix-profile/share/man
2 /run/current-system/profile/share/man
PATH
1 /gnu/store/a9sqlwcl02z89kh6v5n6yzxvxw5qbs14-glib-2.46.1-bin/bin
1 /home/alice/.guix-profile/bin
1 /run/current-system/profile/bin
1 /run/current-system/profile/sbin
1 /run/setuid-programs
PS1
1 \u@\h \w\$
PWD
1 /home/alice
SESSION_MANAGER
1 local/carbuncle
1 /tmp/.ICE-unix/383
1 @/tmp/.ICE-unix/383,unix/carbuncle
SHELL
1 /gnu/store/7cdd8s466qyjh64m0byq0rz9gk1jid40-bash-4.3.42/bin/bash
SHLVL
1 2
SSH_AUTH_SOCK
1 /run/user/30011/keyring/ssh
SSL_CERT_DIR
1 /etc/ssl/certs
SSL_CERT_FILE
1 /etc/ssl/certs/ca-certificates.crt
TERM
1 xterm-256color
TZ
1 America/Los_Angeles
TZDIR
1 /gnu/store/n80pk0smpzdlshkcgccgga8xy5p2vwz3-tzdata-2015g/share/zoneinfo
USER
1 alice
VTE_VERSION
1 4203
WINDOWID
1 23068678
XAUTHORITY
1 /home/alice/.Xauthority
XDG_CONFIG_DIRS
1 /home/alice/.guix-profile/etc/xdg
1 /run/current-system/profile/etc/xdg
XDG_CURRENT_DESKTOP
1 GNOME
XDG_DATA_DIRS
3 /gnu/store/27c82f3szfn2d97ms56hkpsb78q9myhd-gnome-settings-daemon-3.18.2/share
3 /gnu/store/67a1962vx46b5shf3myxi23jvziwaclj-gdm-3.18.2/share
3 /gnu/store/73wzvqzffyx867r1rbck5w65by6lbapy-gnome-bluetooth-3.18.1/share
3 /gnu/store/apf78hww8abnhsg0pr6d00gc5w1g7y5h-libgweather-3.18.1/share
5 /gnu/store/bahndy6sxfxw6g49djc10mpa7292na40-gnome-session-3.18.1.2/share
3 /gnu/store/dzdqr0yh7csf3j4ash6akzxjdbqgfln2-network-manager-applet-1.0.10/share
3 /gnu/store/g7ii95bmy2aq4z5x17wvsqlz4f8lmn5a-gcr-3.18.0/share
3 /gnu/store/kc8j8zcwjzr04nlkm16gy44a7vfr26i8-mutter-3.18.1/share
3 /gnu/store/km25kxpv999ihj058xnvniwycl1zhajl-gnome-control-center-3.18.2/share
9 /gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/share
3 /gnu/store/rsywwl626s8fmmwf154sn2x9si62b4k1-evolution-data-server-3.18.2/share
9 /gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/share
9 /gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/share
1 /gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/share
3 /gnu/store/xnkkvlwvwbv2zzvbca3sclf41m6jxzza-gnome-shell-3.18.3/share
3 /gnu/store/ymx1l8l2gm1jmljh5y24p4f5gk8i2j3c-caribou-0.4.19/share
3 /gnu/store/z2dh08qp3k1829dcsz7wcrinccmj6743-ibus-1.5.11/share
3 /gnu/store/zlq812qd3cykkizdip0l7qw7m2pffj1h-telepathy-logger-0.8.2/share
1 /home/alice/.guix-profile/share
2 /run/current-system/profile/share
XDG_MENU_PREFIX
1 gnome-
XDG_RUNTIME_DIR
1 /run/user/30011
XDG_SEAT
1 seat0
XDG_SESSION_ID
1 c1
XDG_VTNR
1 7
X_XFCE4_LIB_DIRS
1 /run/current-system/profile/lib/xfce4

As you can see, there are some environment variables with up to 9
duplicate entries. Is this expected? Is it a problem? Why is it
happening? Assuming that it is not expected and that it is a problem,
how can we prevent it from happening?

I've tried opening multiple subshells (e.g., by running "bash" on the
command line) to see if the number of duplicate entries increases with
each subshell, but that is not the case. Opening up a new subshell does
not seem to trigger (in the default GuixSD configuration with GNOME, at
least) whatever it is that it causing this duplication.

--
Chris
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iQIcBAEBCAAGBQJW9gkuAAoJEN1AmhXYIkadJCsP/R/uxvRiYgaIxlPKdBgosIZM
a9N9thDFhii/xbiWPJdFoxT/on+jNLB2phfseFGjkA8ezftJfTppj993JPfMwcve
0OZaQbL+ZLIvmLprQg62j8LJ3VHgSrmf1kp7nU1wb9N8wWbPlSfrSCSCjaVL/n63
vNWa+3LlNpHMIWbaOJbjUOaet/FXvqHDKd4klXymacBLt4v9LUU2eayZZhcCkoq0
csnGKiVvsnEOcIzL7G3tmeh+w/2f3hLlMu2kvL3//B9YB7Nn4IIqAKHAeWNtOg3H
jdR/gWfGMrP9FKJtTwnuk5Bo/GRgyBc3uc83WoNb3LkZQLEUikVIKSWiF3jkNonP
wrolqd8FhV9uTNXebI/Y+2ssabW4RU6tUW+uQwmDRNRQxwQAuryJHjI7OeN0JYWS
Asavd6edxRwH9qhmE9vn7owuIGN3t1tSoKbUgOuvCM56ZQ6tmSR9CNaSBJ0s73Q7
5SVs9AL4BpGSAL1jcxvbsZ2q2qPvq7MRSRy3FRUBJ2NJr7G81bmoxk5Vn08k+nKS
qGdEtnGWnqOlbT/4Uw70tqrkbE9jwmq6NKsN2rciCc2rO/wP2TSvJTjsNpp/XqlB
9XTj9WVJ+9l7k42O0E/tPSjV0Y5/qdxspOut4CXt+4yPE/jzCgfdVzz7J0S/spcp
s+TqkK5THz215gydpwpI
=494g
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 26 Mar 2016 19:53
(name . Chris Marusich)(address . cmmarusich@gmail.com)(address . 23118@debbugs.gnu.org)
87io09ystm.fsf@gnu.org
Chris Marusich <cmmarusich@gmail.com> skribis:

Toggle quote (6 lines)
> This behavior should be reproducible. I created a new user to test
> this. When I logged into GNOME on GuixSD (built via "guix system
> reconfigure" using git commit 43a58615bfc37a053ddaca1fcf11af5c14c0b4b6)
> as the new user, I opened up the "Terminal" program (which launched bash
> as an interactive, non-login shell). I then typed

[...]

I think the main problem is that gnome-terminal is wrapped:

Toggle snippet (8 lines)
$ cat /gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/gnome-terminal
#!/gnu/store/b1yqjimbdh5bf9jnizd4h7yf110744j2-bash-4.3.42/bin/bash
export XDG_DATA_DIRS="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/share:/gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/share:/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/share:/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/share${XDG_DATA_DIRS:+:}$XDG_DATA_DIRS"
export GTK_PATH="/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/gtk-3.0${GTK_PATH:+:}$GTK_PATH"
export GIO_EXTRA_MODULES="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/lib/gio/modules:/gnu/store/ygn4vavcgr79p86jrlmvdycjs81ki4dl-dconf-0.24.0/lib/gio/modules${GIO_EXTRA_MODULES:+:}$GIO_EXTRA_MODULES"
exec -a "$0" "/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/.gnome-terminal-real" "$@"

This is done automatically by ‘glib-or-gtk-build-system’, but obviously
it’s not so great here.

I suppose that in practice, it would work to simply switch
gnome-terminal to ‘gnu-build-system’.

Could you try that?

There may still be other places that add extra entries, though.

Ludo’.
L
L
Ludovic Courtès wrote on 28 Mar 2016 18:53
control message for bug #23118
(address . control@debbugs.gnu.org)
877fgmr1cf.fsf@gnu.org
severity 23118 important
L
L
Leo Famulari wrote on 28 Mar 2016 19:53
Re: bug#23118: Duplicate entries in various environment variables
(name . Ludovic Courtès)(address . ludo@gnu.org)
20160328175301.GA9856@jasmine
On Sat, Mar 26, 2016 at 07:53:09PM +0100, Ludovic Courtès wrote:
Toggle quote (29 lines)
> Chris Marusich <cmmarusich@gmail.com> skribis:
>
> > This behavior should be reproducible. I created a new user to test
> > this. When I logged into GNOME on GuixSD (built via "guix system
> > reconfigure" using git commit 43a58615bfc37a053ddaca1fcf11af5c14c0b4b6)
> > as the new user, I opened up the "Terminal" program (which launched bash
> > as an interactive, non-login shell). I then typed
>
> [...]
>
> I think the main problem is that gnome-terminal is wrapped:
>
> --8<---------------cut here---------------start------------->8---
> $ cat /gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/gnome-terminal
> #!/gnu/store/b1yqjimbdh5bf9jnizd4h7yf110744j2-bash-4.3.42/bin/bash
> export XDG_DATA_DIRS="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/share:/gnu/store/sm1i06dhgpkhvgj2bhcn6nfi275slgjz-gsettings-desktop-schemas-3.18.0/share:/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/share:/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/share${XDG_DATA_DIRS:+:}$XDG_DATA_DIRS"
> export GTK_PATH="/gnu/store/sqd3q1xq5fsbga00bwhghi9shi7xdaac-gtk+-3.18.2/lib/gtk-3.0${GTK_PATH:+:}$GTK_PATH"
> export GIO_EXTRA_MODULES="/gnu/store/lbkdykh3aa1469li01rwyya02q50r3b9-glib-2.46.1/lib/gio/modules:/gnu/store/ygn4vavcgr79p86jrlmvdycjs81ki4dl-dconf-0.24.0/lib/gio/modules${GIO_EXTRA_MODULES:+:}$GIO_EXTRA_MODULES"
> exec -a "$0" "/gnu/store/x2hz7q806ry8lmwdya0jlff01w7djbn2-gnome-terminal-3.18.2/bin/.gnome-terminal-real" "$@"
> --8<---------------cut here---------------end--------------->8---
>
> This is done automatically by ‘glib-or-gtk-build-system’, but obviously
> it’s not so great here.
>
> I suppose that in practice, it would work to simply switch
> gnome-terminal to ‘gnu-build-system’.
>
> Could you try that?

I tried it. I had to add glib:bin as a native-input.

Trying to run gnome-terminal produces this error (I wrapped the lines):

$ gnome-terminal
Error constructing proxy for
org.gnome.Terminal:/org/gnome/Terminal/Factory0: Error calling
StartServiceByName for org.gnome.Terminal:
GDBus.Error:org.freedesktop.DBus.Error.Spawn.ChildSignaled: Process
org.gnome.Terminal received signal 5

Toggle quote (7 lines)
>
> There may still be other places that add extra entries, though.
>
> Ludo’.
>
>
>
L
L
Leo Prikler wrote on 3 Dec 2020 11:58
Re: Duplicate entries in various environment variables
(name . Chris Marusich)(address . cmmarusich@gmail.com)(address . 23118@debbugs.gnu.org)
6ca9bc2b5bb96b0ea6740c3412d6aa4379d553e9.camel@student.tugraz.at
Hello, Chris

Am Freitag, den 25.03.2016, 20:59 -0700 schrieb Chris Marusich:
Toggle quote (14 lines)
> Hi,
>
> I've noticed that my environment variables contain duplicate
> (sometimes
> more) entries. This occurs regardless of whether the user logs in
> directly via a tty or via a desktop environment like GNOME.
>
> [...]
>
> As you can see, there are some environment variables with up to 9
> duplicate entries. Is this expected? Is it a problem? Why is it
> happening? Assuming that it is not expected and that it is a
> problem,
> how can we prevent it from happening?
It has been pointed out, that this is somewhat expected when wrapping
the same environment variable multiple times. For instance, you as a
GNOME user might already have gtk+ in your GTK path if you run GNOME,
but GNOME applications can not rely on that and thus need to add their
own. As the number of software components, that use it increases, so
does the number of mentions.
In the special case of gnome-terminal, this is user-visible by printing
out env, but other applications get launched in a similar manner all
the time without you noticing. Perhaps one could patch GNOME terminal
to clear those variables before spawning the shell, but that's going to
be a bit fiddly. Alternatively, one could enforce GTK_PATH by using
"=" instead of prefix.

More generally, this can become an issue when environment variables
reach a certain size (and has led to bug reports in Guix before, which
have since been fixed). POSIX mandates a syntax, that would allow
removing already present components first (see [1]), but I can hardly
imagine what monstrosities we would need to cook up to do this
reliably. Not to mention, that some otherwise POSIX-compliant shells
might not implement that syntax (correctly).

Regards,
Leo

[1]
M
M
Maxim Cournoyer wrote on 8 Oct 2022 03:51
Re: bug#23118: Duplicate entries in various environment variables
(address . liliana.prikler@student.tugraz)
87fsfz3wwx.fsf@gmail.com
Hi,

[...]

Toggle quote (8 lines)
> More generally, this can become an issue when environment variables
> reach a certain size (and has led to bug reports in Guix before, which
> have since been fixed). POSIX mandates a syntax, that would allow
> removing already present components first (see [1]), but I can hardly
> imagine what monstrosities we would need to cook up to do this
> reliably. Not to mention, that some otherwise POSIX-compliant shells
> might not implement that syntax (correctly).

Why are we talking about POSIX here? Our wrap-program procedure makes
use of Bash; we could make use of its features to filter duplicates,
right? Unless I'm missing something.

--
Thanks,
Maxim
?