wishlist: addessing statefulness of .cache(s)

Open

Details

6 participants

Danny Milosavljevic
Giovanni Biscuolo
Julien Lepiller
Ludovic Courtès
Tobias Geerinckx-Rice
swedebugia

Owner: unassigned

Submitted by: Giovanni Biscuolo

Severity: wishlist

Giovanni Biscuolo wrote on 11 May 2019 09:32

Recipients:(address . bug-guix@gnu.org)

Message-ID:878svdh2ec.fsf@roquette.mug.biscuolo.net

Hi Guix,

AFAIU issues like the two I point below are becoming a common pattern

and are *critical*

1. gnome session not starting due to state in $HOME/.cache

http://lists.gnu.org/archive/html/guix-devel/2019-04/msg00177.html

Message-ID: <87ef68ibfy.fsf@elephly.net>

Ricardo Wurmus:

Toggle snippet (6 lines)

What should we do about this? For gdm I think it would make sense to

add an activation service extension that clears the gdm user’s home

directory. And more generally, maybe we should offer a generic cache

cleaner service.

2. X broken display transitioning from llvm6 to llvm7 in the mesa package
   http://lists.gnu.org/archive/html/guix-devel/2019-05/msg00223.html
   Message-ID: <20190511022009.nnu6szga6desvfwd@cf0>
   see also https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35575

ison:

Toggle snippet (10 lines)

Note that deleting both shader caches was required, and also if the caches get

rebuilt on a new generation and then I try to boot into an older previously

working generation then that generation will display graphics artifacts until

the caches are deleted again.

So switching between mesa compiled with llvm 6 and 7 on AMD RX 580 either

backward or forward requires manually deleting the shader caches.

AFAIU unfortunately we have application/library state all over .cache(s)

that sometimes crashes software *and* trying to fix this upstream it's

_not_ an option [1]

often users have to delete something in some .cache by guessing, "just"

to solve some strange software crash (this is common to all distros)

maybe an activation service extension proposed by Ricardo (see above)

is the right solution: I'll try to make a summary of prevoius

discussions on this topic on guix-devel to help address this (class of)

issue(s)... sorry I cannot help coding it

WDYT?

Thanks! Gio'

[1] I say this observing this class of issues since I started using free

software: am I wrong?

Giovanni Biscuolo

Xelera IT Infrastructures

Julien Lepiller wrote on 11 May 2019 09:43

Recipients:

Message-ID:052F5639-AD7F-4AF5-8225-CFC5FBA8E3F3@lepiller.eu

Le 11 mai 2019 09:32:43 GMT+02:00, Giovanni Biscuolo <g@xelera.eu> a écrit :

Toggle quote (62 lines)>Hi Guix,
>
>AFAIU issues like the two I point below are becoming a common pattern
>and are *critical*
>
>1. gnome session not starting due to state in $HOME/.cache
>   http://lists.gnu.org/archive/html/guix-devel/2019-04/msg00177.html
>   Message-ID: <87ef68ibfy.fsf@elephly.net>
>
>Ricardo Wurmus:
>--8<---------------cut here---------------start------------->8---
>What should we do about this?  For gdm I think it would make sense to
>add an activation service extension that clears the gdm user’s home
>directory.  And more generally, maybe we should offer a generic cache
>cleaner service.
>--8<---------------cut here---------------end--------------->8---
>
>2. X broken display transitioning from llvm6 to llvm7 in the mesa
>package
>   http://lists.gnu.org/archive/html/guix-devel/2019-05/msg00223.html
>   Message-ID: <20190511022009.nnu6szga6desvfwd@cf0>
>   see also https://debbugs.gnu.org/cgi/bugreport.cgi?bug=35575
>
>ison:
>--8<---------------cut here---------------start------------->8---
>Note that deleting both shader caches was required, and also if the
>caches get
>rebuilt on a new generation and then I try to boot into an older
>previously
>working generation then that generation will display graphics artifacts
>until
>the caches are deleted again.
>
>So switching between mesa compiled with llvm 6 and 7 on AMD RX 580
>either
>backward or forward requires manually deleting the shader caches.
>--8<---------------cut here---------------end--------------->8---
>
>
>AFAIU unfortunately we have application/library state all over
>.cache(s)
>that sometimes crashes software *and* trying to fix this upstream it's
>_not_ an option [1]
>
>often users have to delete something in some .cache by guessing, "just"
>to solve some strange software crash (this is common to all distros)
>
>maybe an activation service extension proposed by Ricardo (see above)
>is the right solution: I'll try to make a summary of prevoius
>discussions on this topic on guix-devel to help address this (class of)
>issue(s)... sorry I cannot help coding it
>
>WDYT?
>
>Thanks! Gio'
>
>
>
>[1] I say this observing this class of issues since I started using
>free
>software: am I wrong?

I wonder if we could mount a tmpfs on .cache? Would that be enough?

Giovanni Biscuolo wrote on 11 May 2019 09:50

Re: wishlist: addessing statefulness of .cache(s)

Recipients:(address . 35683@debbugs.gnu.org)

Message-ID:8736llh1ks.fsf@roquette.mug.biscuolo.net

Giovanni Biscuolo <g@xelera.eu> writes:

Toggle quote (3 lines)

> AFAIU issues like the two I point below are becoming a common pattern

> and are *critical*

linked bug report:

- bug#35669: Mesa is failing an assertion

[...]

Giovanni Biscuolo

Xelera IT Infrastructures

Tobias Geerinckx-Rice wrote on 11 May 2019 13:45

Re: bug#35683: wishlist: addessing statefulness of .cache(s)

Recipients:(address . bug-guix@gnu.org)(address . 35683@debbugs.gnu.org)

Message-ID:87zhntxlj4.fsf@nckx

ehlo Giovanni,

Giovanni Biscuolo wrote:

Toggle quote (6 lines)

> AFAIU unfortunately we have application/library state all over

> .cache(s)

> that sometimes crashes software *and* trying to fix this

> upstream it's

> _not_ an option [1]

Oh. That's… disappointing to say the least, since these are both

upstream bugs that Guix can't fix :-(

What exactly did you ask? What was their response?

Toggle quote (5 lines)

> often users have to delete something in some .cache by guessing,

> "just"

> to solve some strange software crash (this is common to all

> distros)

I have never had to do this, ever.

Toggle quote (7 lines)

> maybe an activation service extension proposed by Ricardo (see

> above)

> is the right solution: I'll try to make a summary of prevoius

> discussions on this topic on guix-devel to help address this

> (class of)

> issue(s)... sorry I cannot help coding it

We can randomly delete whole caches in the user's stead but it's

never the ‘right’ solution. Only the application can manage its

caches properly. I still hope this is possible in both cases

here.

Kind regards,

T G-R

-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQT12iAyS4c9C3o4dnINsP+IT1VteQUCXNa1vwAKCRANsP+IT1Vt

eY9CAP9j7HD4E0GYs+y2XfY2uv1czexfcR59RMGz0poLNSKFCAEArzyw8t7wNbEr

iBpoLXIAW9d94G3VdwbFsTZpcPu1lwE=

=N4JP

-----END PGP SIGNATURE-----

Tobias Geerinckx-Rice wrote on 11 May 2019 13:51

Recipients:(name . Julien Lepiller)(address . julien@lepiller.eu)

Message-ID:87y33dxl8s.fsf@nckx

Julien,

Julien Lepiller wrote:

Toggle quote (3 lines)

> I wonder if we could mount a tmpfs on .cache? Would that be

> enough?

Seems a shame to waste RAM like that, when we could (ab)use FUSE:

https://github.com/xrgtn/nullfs.

;-)

T G-R

-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQT12iAyS4c9C3o4dnINsP+IT1VteQUCXNa3MwAKCRANsP+IT1Vt

eRCGAQDihdDMllPuGVY4nz9gs+tl/pk0k1lwBLGC1dXVNGlxKQD+ND9ANhyPKpiQ

W4sAtc+yuIKo4sWVaIMphrVi2mydKAo=

=mEqf

-----END PGP SIGNATURE-----

Giovanni Biscuolo wrote on 11 May 2019 14:59

Recipients:

Message-ID:87tve1f8oq.fsf@roquette.mug.biscuolo.net

Hello Tobias,

Tobias Geerinckx-Rice <me@tobias.gr> writes:

Toggle quote (12 lines)> Giovanni Biscuolo wrote:
>> AFAIU unfortunately we have application/library state all over 
>> .cache(s)
>> that sometimes crashes software *and* trying to fix this 
>> upstream it's
>> _not_ an option [1]
>
> Oh.  That's… disappointing to say the least, since these are both 
> upstream bugs that Guix can't fix :-(
>
> What exactly did you ask?  What was their response?

I did not ask upstream, sorry for the misunderstanding: I'm just sharing

my *personal* experience, and I confess I never tried to report this

class of bugs upstream, I just fixed the issue with "rm

<somewhere>/.cache/<something>"

AFAIO (as far as I'm observing) this is a common pattern, for some

current Guix bug reports too (see previously provided links)

to be clear: I'm not stating we should not report upstream and help them

:-)

Toggle quote (7 lines)

>> often users have to delete something in some .cache by guessing,

>> "just"

>> to solve some strange software crash (this is common to all

>> distros)

> I have never had to do this, ever.

lucky? :-) or Very Stateless™ userland configuration?

I'm not saying I had to do this sort of cache cleaning every week, but I

had to do that Too Often™ to be accepteble to me, on multiple

installations in 10 years

[...]

Toggle quote (3 lines)

> We can randomly delete whole caches in the user's stead but it's

> never the ‘right’ solution.

I agree, but please consider that we have to manage some upstream

defects [1], sometimes :-S

Toggle quote (3 lines)

> Only the application can manage its caches properly. I still hope

> this is possible in both cases here.

OK, but meanwhile?

IMHO it's not acceptable to have critical Guix services (e.g. GDM)

blocked by similar issues

...and sometimes (often?) statefullness of .cache is not considered

upstream, so I suspect this class of upstream bugs are _not_ going to

end soon

Thank you for your contribution! Gio'

[1] having data in .cache that crashes applications and services is bad

design IMHO, let alone having configuration in .cache

Giovanni Biscuolo

Xelera IT Infrastructures

Danny Milosavljevic wrote on 12 May 2019 11:32

Recipients:(name . Giovanni Biscuolo)(address . g@xelera.eu)(address . 35683@debbugs.gnu.org)

Message-ID:20190512113242.583f9229@scratchpost.org

Toggle quote (6 lines)

> --8<---------------cut here---------------start------------->8---

> What should we do about this? For gdm I think it would make sense to

> add an activation service extension that clears the gdm user’s home

> directory. And more generally, maybe we should offer a generic cache

> cleaner service.

I don't like that workaround much. I mean for the time being I guess it's

OK, but let's file bug reports upstream so they are aware of the problem.

Better would be if the cache directory contained a "cache-protocol-version"

file or something and make the client program heed it and make it clear the

cache if it's the wrong version, without any Guix special case (the problem

is not not Guix-specific anyway).

It's not exactly difficult. Most of the time the bug reports just don't

get filed--and cache invalidation is always an afterthought when

implementing a cache (sadly).

If they say no we can still keep the workaround.

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlzX6DoACgkQ5xo1VCww

uqXB6ggAjLdkLT4kE6S9kmGZxGwycdzccd2j9OVoZU8JfqA/0iPHDEQeGKzR+Mt2

B9H8Yif5FAX8NAMdVXAlrVarE13K6FPMgzhxhTivPEilWlgfEoMHUbDlpcQCDAzJ

ZSuENiOX7oDOhCvj78dooSv5HqwdorssLlrHivWX1Cko+USKiTCDa323lxtBMF5m

MQ1EJQUBlqUd5/93jZBY5Iu59wQF87af0lgltBxZg0J+LkQKGgqth5dzYoqqyWrH

wm6HUwT5/utMZEunB8kUkF57YJB341V60aftRL8pL40pgYGFQpFlxhsaFD9Mcelb

s0tObzJwmjccFFuFIa+ipncN46bIOQ==

=nBAn

-----END PGP SIGNATURE-----

swedebugia wrote on 12 May 2019 23:29

Recipients:(address . 35683@debbugs.gnu.org)

Message-ID:7367fb7b-7ee9-6a33-4a99-aee6be025276@riseup.net

On 2019-05-11 14:59, Giovanni Biscuolo wrote:

Toggle quote (3 lines)

> Hello Tobias,

> Tobias Geerinckx-Rice <me@tobias.gr> writes:

...

Toggle quote (7 lines)

>>> often users have to delete something in some .cache by guessing,

>>> "just"

>>> to solve some strange software crash (this is common to all

>>> distros)

>> I have never had to do this, ever.

I have no memories having to do this either during the last 15 years.

...

Toggle quote (3 lines)

> [1] having data in .cache that crashes applications and services is bad

> design IMHO, let alone having configuration in .cache

Cheers

Swedebugia

Ludovic Courtès wrote on 14 May 2019 09:47

Recipients:(name . Danny Milosavljevic)(address . dannym@scratchpost.org)

Message-ID:87tvdx32ac.fsf@gnu.org

Hi Danny,

Danny Milosavljevic <dannym@scratchpost.org> skribis:

Toggle quote (5 lines)

> Better would be if the cache directory contained a "cache-protocol-version"

> file or something and make the client program heed it and make it clear the

> cache if it's the wrong version, without any Guix special case (the problem

> is not not Guix-specific anyway).

I’d be surprised if applications and libraries using ~/.cache such as

Mesa didn’t have some versioning mechanism allowing them to detect stale

cache files. Overall, Guix isn’t special in that respect.

The problem we experienced with Mesa shader caches are probably not

Guix-specific. I think we should try to gather more info about the

stale .cache files that trigger a Mesa crash so that we can make a

useful bug report to the Mesa developers.

(I’m saying this as someone who did not experience the crash, heheh. :-))

Thoughts?

Thanks,

Ludo’.

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 35683@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 35683

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date