wishlist: addessing statefulness of .cache(s)

  • Open
  • quality assurance status badge
Details
6 participants
  • Danny Milosavljevic
  • Giovanni Biscuolo
  • Julien Lepiller
  • Ludovic Courtès
  • Tobias Geerinckx-Rice
  • swedebugia
Owner
unassigned
Submitted by
Giovanni Biscuolo
Severity
wishlist
G
G
Giovanni Biscuolo wrote on 11 May 2019 09:32
(address . bug-guix@gnu.org)
878svdh2ec.fsf@roquette.mug.biscuolo.net
Hi Guix,

AFAIU issues like the two I point below are becoming a common pattern
and are *critical*

1. gnome session not starting due to state in $HOME/.cache
Message-ID: <87ef68ibfy.fsf@elephly.net>

Ricardo Wurmus:
Toggle snippet (6 lines)
What should we do about this? For gdm I think it would make sense to
add an activation service extension that clears the gdm user’s home
directory. And more generally, maybe we should offer a generic cache
cleaner service.

2. X broken display transitioning from llvm6 to llvm7 in the mesa package
Message-ID: <20190511022009.nnu6szga6desvfwd@cf0>

ison:
Toggle snippet (10 lines)
Note that deleting both shader caches was required, and also if the caches get
rebuilt on a new generation and then I try to boot into an older previously
working generation then that generation will display graphics artifacts until
the caches are deleted again.

So switching between mesa compiled with llvm 6 and 7 on AMD RX 580 either
backward or forward requires manually deleting the shader caches.


AFAIU unfortunately we have application/library state all over .cache(s)
that sometimes crashes software *and* trying to fix this upstream it's
_not_ an option [1]

often users have to delete something in some .cache by guessing, "just"
to solve some strange software crash (this is common to all distros)

maybe an activation service extension proposed by Ricardo (see above)
is the right solution: I'll try to make a summary of prevoius
discussions on this topic on guix-devel to help address this (class of)
issue(s)... sorry I cannot help coding it

WDYT?

Thanks! Gio'



[1] I say this observing this class of issues since I started using free
software: am I wrong?


--
Giovanni Biscuolo

Xelera IT Infrastructures
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEERcxjuFJYydVfNLI5030Op87MORIFAlzWepwACgkQ030Op87M
ORJC9BAAzAUXxugtcoC+qRSWcJnEMR29uuxvKdYk4w+/XGkvj+VQrrxUMkYa1mTf
Jt5U5EXm1P6Vu7KwVnajffC5Ml3Tjf+0zZadxjmK5bcZmAU2VF0W6NqyVG9MqJFR
CJIusxuvwdxjBPlal3feFR8G1DrksF3O/iLkLj0vtCC+a2Or69qgSS5ZcMkDpMcq
f8CfMLD7fNU2ewIz7Y1JhZiR492F2JIvgA7DHxoUzj8IcCE6BVDF0EaW/RC63kXg
/9T+nyLChyZ1GXvfmNRr1ykPcIb9hFnEIleglnPXESkOr1JY47o+RqSapVRvW+5a
YghTMXHlC3+G7XKpUzo0yvhG352z4W2LiAamoiAP1oPnG13oT6dC3JqtOQ8Gz3Hq
ersy7L40NG0fZ9P9gh9eG/HxUcdJ2J/RH2elVnB3lHCZhLrs72pSD8/A5+8IvNlU
F0NxglZ4SEUF+DeQHcj0A/OdD4zLFFPOFq8TFEke9G7Bke0p95g339JBGdrrDCfg
KhnOPFToarptOSCnOP5u64TCwgBS7E5K6fypp5sfPGusw6Y6840mM9SkXEwmEMTK
I4mFqzEBQztwYpcdQ2EgpWzKNSXj8GnnS3HdM6QiNiQqSsRpWLxxEiJgGmeMiCh6
4JBJHAG961O4NmUnUE2YLv42tETNHiOqygHNXETBghGzRYLQPmU=
=CUo9
-----END PGP SIGNATURE-----

J
J
Julien Lepiller wrote on 11 May 2019 09:43
052F5639-AD7F-4AF5-8225-CFC5FBA8E3F3@lepiller.eu
Le 11 mai 2019 09:32:43 GMT+02:00, Giovanni Biscuolo <g@xelera.eu> a écrit :
Toggle quote (62 lines)
>Hi Guix,
>
>AFAIU issues like the two I point below are becoming a common pattern
>and are *critical*
>
>1. gnome session not starting due to state in $HOME/.cache
> http://lists.gnu.org/archive/html/guix-devel/2019-04/msg00177.html
> Message-ID: <87ef68ibfy.fsf@elephly.net>
>
>Ricardo Wurmus:
>--8<---------------cut here---------------start------------->8---
>What should we do about this? For gdm I think it would make sense to
>add an activation service extension that clears the gdm user’s home
>directory. And more generally, maybe we should offer a generic cache
>cleaner service.
>--8<---------------cut here---------------end--------------->8---
>
>2. X broken display transitioning from llvm6 to llvm7 in the mesa
>package
> http://lists.gnu.org/archive/html/guix-devel/2019-05/msg00223.html
> Message-ID: <20190511022009.nnu6szga6desvfwd@cf0>
> see also https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35575
>
>ison:
>--8<---------------cut here---------------start------------->8---
>Note that deleting both shader caches was required, and also if the
>caches get
>rebuilt on a new generation and then I try to boot into an older
>previously
>working generation then that generation will display graphics artifacts
>until
>the caches are deleted again.
>
>So switching between mesa compiled with llvm 6 and 7 on AMD RX 580
>either
>backward or forward requires manually deleting the shader caches.
>--8<---------------cut here---------------end--------------->8---
>
>
>AFAIU unfortunately we have application/library state all over
>.cache(s)
>that sometimes crashes software *and* trying to fix this upstream it's
>_not_ an option [1]
>
>often users have to delete something in some .cache by guessing, "just"
>to solve some strange software crash (this is common to all distros)
>
>maybe an activation service extension proposed by Ricardo (see above)
>is the right solution: I'll try to make a summary of prevoius
>discussions on this topic on guix-devel to help address this (class of)
>issue(s)... sorry I cannot help coding it
>
>WDYT?
>
>Thanks! Gio'
>
>
>
>[1] I say this observing this class of issues since I started using
>free
>software: am I wrong?

I wonder if we could mount a tmpfs on .cache? Would that be enough?
G
G
Giovanni Biscuolo wrote on 11 May 2019 09:50
Re: wishlist: addessing statefulness of .cache(s)
(address . 35683@debbugs.gnu.org)
8736llh1ks.fsf@roquette.mug.biscuolo.net
Giovanni Biscuolo <g@xelera.eu> writes:

Toggle quote (3 lines)
> AFAIU issues like the two I point below are becoming a common pattern
> and are *critical*

linked bug report:

- bug#35669: Mesa is failing an assertion

[...]

--
Giovanni Biscuolo

Xelera IT Infrastructures
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEERcxjuFJYydVfNLI5030Op87MORIFAlzWfsMACgkQ030Op87M
ORLJYBAAoU69aLa9+sprjXwA8jwFn1QJ05Xjdj0SAfcKw74ylj+NBiORkuL8/hrC
fmrC9nMQdXtYMpep8I1rvJpki/XkSq4GQEZZZVvmZS9LoG0OrxPRt7e1d5+UqkV8
Th6VNvV+nJGwLqVsGPSozNwDB4wb3tDI+4yGFBXdCT2+iJvfFOh0aPZb7GKyyJdL
nimDKgyvLiT/mmV0CTFvygDE/nsXplc37Xkvk59tdnH9al612EM5dehE6oVTZ8bl
I7x2bTiVZ8d/98T0CuR/f1M8F/O89ElE9ShjP0SYQRD9ShwZOCA9baxg8AtmDWkA
GFrcX0fhtCWpMTh68iFO5ZzxLpTfZPkjGSksysvcll/ShgAsnsinc6vlqeqp7mKm
KimHBAmEErRjj24adrH62Euz+VFLYeMZkVerBtlqyxGUANf0mG3u2wVMRp0H/g74
Hm7NrPMJ9n1Qa72HelrMdiBSlc+MzBrc0jADBjhftTNmQU9iet2dF1qHc96+rdUP
3fMkbetdz5Nhn02KYawu8nUcAiNelTAwV82csn4bNwuvYF7uOXczOJcir4lhYWGA
IktLT1eeS3A9rSOIq/9Idy7oB9T8dbeWuaK4kOoVzbJ1dJZQlIWTeT8INt4iYTAG
Y2wjT5bohr/di+VC7WLCljliFPno7VdRN2U4LmGt2WLoMncJxRo=
=lQQa
-----END PGP SIGNATURE-----

T
T
Tobias Geerinckx-Rice wrote on 11 May 2019 13:45
Re: bug#35683: wishlist: addessing statefulness of .cache(s)
(address . bug-guix@gnu.org)(address . 35683@debbugs.gnu.org)
87zhntxlj4.fsf@nckx
ehlo Giovanni,

Giovanni Biscuolo wrote:
Toggle quote (6 lines)
> AFAIU unfortunately we have application/library state all over
> .cache(s)
> that sometimes crashes software *and* trying to fix this
> upstream it's
> _not_ an option [1]

Oh. That's… disappointing to say the least, since these are both
upstream bugs that Guix can't fix :-(

What exactly did you ask? What was their response?

Toggle quote (5 lines)
> often users have to delete something in some .cache by guessing,
> "just"
> to solve some strange software crash (this is common to all
> distros)

I have never had to do this, ever.

Toggle quote (7 lines)
> maybe an activation service extension proposed by Ricardo (see
> above)
> is the right solution: I'll try to make a summary of prevoius
> discussions on this topic on guix-devel to help address this
> (class of)
> issue(s)... sorry I cannot help coding it

We can randomly delete whole caches in the user's stead but it's
never the ‘right’ solution. Only the application can manage its
caches properly. I still hope this is possible in both cases
here.

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQT12iAyS4c9C3o4dnINsP+IT1VteQUCXNa1vwAKCRANsP+IT1Vt
eY9CAP9j7HD4E0GYs+y2XfY2uv1czexfcR59RMGz0poLNSKFCAEArzyw8t7wNbEr
iBpoLXIAW9d94G3VdwbFsTZpcPu1lwE=
=N4JP
-----END PGP SIGNATURE-----

T
T
Tobias Geerinckx-Rice wrote on 11 May 2019 13:51
(name . Julien Lepiller)(address . julien@lepiller.eu)
87y33dxl8s.fsf@nckx
Julien,

Julien Lepiller wrote:
Toggle quote (3 lines)
> I wonder if we could mount a tmpfs on .cache? Would that be
> enough?

Seems a shame to waste RAM like that, when we could (ab)use FUSE:

;-)

T G-R
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQT12iAyS4c9C3o4dnINsP+IT1VteQUCXNa3MwAKCRANsP+IT1Vt
eRCGAQDihdDMllPuGVY4nz9gs+tl/pk0k1lwBLGC1dXVNGlxKQD+ND9ANhyPKpiQ
W4sAtc+yuIKo4sWVaIMphrVi2mydKAo=
=mEqf
-----END PGP SIGNATURE-----

G
G
Giovanni Biscuolo wrote on 11 May 2019 14:59
87tve1f8oq.fsf@roquette.mug.biscuolo.net
Hello Tobias,

Tobias Geerinckx-Rice <me@tobias.gr> writes:

Toggle quote (12 lines)
> Giovanni Biscuolo wrote:
>> AFAIU unfortunately we have application/library state all over
>> .cache(s)
>> that sometimes crashes software *and* trying to fix this
>> upstream it's
>> _not_ an option [1]
>
> Oh. That's… disappointing to say the least, since these are both
> upstream bugs that Guix can't fix :-(
>
> What exactly did you ask? What was their response?

I did not ask upstream, sorry for the misunderstanding: I'm just sharing
my *personal* experience, and I confess I never tried to report this
class of bugs upstream, I just fixed the issue with "rm
<somewhere>/.cache/<something>"

AFAIO (as far as I'm observing) this is a common pattern, for some
current Guix bug reports too (see previously provided links)

to be clear: I'm not stating we should not report upstream and help them
:-)

Toggle quote (7 lines)
>> often users have to delete something in some .cache by guessing,
>> "just"
>> to solve some strange software crash (this is common to all
>> distros)
>
> I have never had to do this, ever.

lucky? :-) or Very Stateless™ userland configuration?

I'm not saying I had to do this sort of cache cleaning every week, but I
had to do that Too Often™ to be accepteble to me, on multiple
installations in 10 years

[...]

Toggle quote (3 lines)
> We can randomly delete whole caches in the user's stead but it's
> never the ‘right’ solution.

I agree, but please consider that we have to manage some upstream
defects [1], sometimes :-S

Toggle quote (3 lines)
> Only the application can manage its caches properly. I still hope
> this is possible in both cases here.

OK, but meanwhile?
IMHO it's not acceptable to have critical Guix services (e.g. GDM)
blocked by similar issues

...and sometimes (often?) statefullness of .cache is not considered
upstream, so I suspect this class of upstream bugs are _not_ going to
end soon


Thank you for your contribution! Gio'



[1] having data in .cache that crashes applications and services is bad
design IMHO, let alone having configuration in .cache

--
Giovanni Biscuolo

Xelera IT Infrastructures
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEERcxjuFJYydVfNLI5030Op87MORIFAlzWx0UACgkQ030Op87M
ORLSfg//XKNN1zHZ9jCaGUP+BC5n8yilLHIWYAXD4iYSWsbz68VXT9uPOtSyzz5I
+xjs7Js5dThAAQu2ivbL4LtN/4twcChx1Ri2JOLBogkKzMTBn37gCr5AIKVODXB0
SojObxe1TKIqg7XUdXgfz3qTSkgk53op2B2ZOWLCAbxRH51hXWdzwqVrrZRphcGa
hfppOjCkae1+pgt4TZt2U1inzFzG/M6/ybXEIBrQYIeykWFtKvP4Dw5lEt1FYnvC
YTAR91B/Di4W2KJoFEdbNOTftvS24G+lo1oPvOt22s/fO7P7ScFjmGzLj834DVc/
2dO25sJ/8DoHmV7UF8PvqJ8p+QJiYpqw6zGefq/17kTwc0Xr5RTg5yqSLSzZQoGE
uEG+A4pa+DLcBf64qDRj8fYm/zQH5tUqpWhzeSjxw4Fi63yzabB9pqGN/3HQe1Ib
PQqDIJhH9fLpNurin0xu0LA5JN5wy9tbgOQJ03XLFG9Efe2dD/zMZBoxdqSgD3sA
iYa6bD5TYXkkg+/Q2T/wj5w984fBZ8jWmGE2Jx3C+6U2yUIY9z8U6yAy6SNRylNq
LxfGFxTCfp+jzO4YCDWMLn4ViDrh2XfyymsSiEx7GCcu3y2Iki/q/fEdone/vZCG
NGlr32MxEHqinlEXyEK3NwJQ771to7oMqm7yoInTyuZ3KsQLJzA=
=kxT/
-----END PGP SIGNATURE-----

D
D
Danny Milosavljevic wrote on 12 May 2019 11:32
(name . Giovanni Biscuolo)(address . g@xelera.eu)(address . 35683@debbugs.gnu.org)
20190512113242.583f9229@scratchpost.org
Toggle quote (6 lines)
> --8<---------------cut here---------------start------------->8---
> What should we do about this? For gdm I think it would make sense to
> add an activation service extension that clears the gdm user’s home
> directory. And more generally, maybe we should offer a generic cache
> cleaner service.

I don't like that workaround much. I mean for the time being I guess it's
OK, but let's file bug reports upstream so they are aware of the problem.

Better would be if the cache directory contained a "cache-protocol-version"
file or something and make the client program heed it and make it clear the
cache if it's the wrong version, without any Guix special case (the problem
is not not Guix-specific anyway).

It's not exactly difficult. Most of the time the bug reports just don't
get filed--and cache invalidation is always an afterthought when
implementing a cache (sadly).

If they say no we can still keep the workaround.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlzX6DoACgkQ5xo1VCww
uqXB6ggAjLdkLT4kE6S9kmGZxGwycdzccd2j9OVoZU8JfqA/0iPHDEQeGKzR+Mt2
B9H8Yif5FAX8NAMdVXAlrVarE13K6FPMgzhxhTivPEilWlgfEoMHUbDlpcQCDAzJ
ZSuENiOX7oDOhCvj78dooSv5HqwdorssLlrHivWX1Cko+USKiTCDa323lxtBMF5m
MQ1EJQUBlqUd5/93jZBY5Iu59wQF87af0lgltBxZg0J+LkQKGgqth5dzYoqqyWrH
wm6HUwT5/utMZEunB8kUkF57YJB341V60aftRL8pL40pgYGFQpFlxhsaFD9Mcelb
s0tObzJwmjccFFuFIa+ipncN46bIOQ==
=nBAn
-----END PGP SIGNATURE-----


S
S
swedebugia wrote on 12 May 2019 23:29
(address . 35683@debbugs.gnu.org)
7367fb7b-7ee9-6a33-4a99-aee6be025276@riseup.net
On 2019-05-11 14:59, Giovanni Biscuolo wrote:
Toggle quote (3 lines)
> Hello Tobias,
>
> Tobias Geerinckx-Rice <me@tobias.gr> writes:
...


Toggle quote (7 lines)
>>> often users have to delete something in some .cache by guessing,
>>> "just"
>>> to solve some strange software crash (this is common to all
>>> distros)
>>
>> I have never had to do this, ever.

I have no memories having to do this either during the last 15 years.

...

Toggle quote (3 lines)
> [1] having data in .cache that crashes applications and services is bad
> design IMHO, let alone having configuration in .cache

+1


--
Cheers
Swedebugia
L
L
Ludovic Courtès wrote on 14 May 2019 09:47
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
87tvdx32ac.fsf@gnu.org
Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

Toggle quote (5 lines)
> Better would be if the cache directory contained a "cache-protocol-version"
> file or something and make the client program heed it and make it clear the
> cache if it's the wrong version, without any Guix special case (the problem
> is not not Guix-specific anyway).

I’d be surprised if applications and libraries using ~/.cache such as
Mesa didn’t have some versioning mechanism allowing them to detect stale
cache files. Overall, Guix isn’t special in that respect.

The problem we experienced with Mesa shader caches are probably not
Guix-specific. I think we should try to gather more info about the
stale .cache files that trigger a Mesa crash so that we can make a
useful bug report to the Mesa developers.

(I’m saying this as someone who did not experience the crash, heheh. :-))

Thoughts?

Thanks,
Ludo’.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 35683@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 35683
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch