[optimization] Grafting is too slow

> Hi Maxim,

>

>> Judging from the above, it seems this issue has been resolved.

> grafting is still a performance issue imo. Compare for example:

>

> $ time guix environment --ad-hoc --search-paths r-learnr

> guix environment --ad-hoc --search-paths r-learnr 5,90s user 0,09s system 210% cpu 2,844 total

> $ time guix environment --ad-hoc --search-paths r-learnr --no-grafts

> guix environment --ad-hoc --search-paths r-learnr --no-grafts 2,03s user 0,08s system 164% cpu 1,277 total

> Lars-Dominik Braun <ldb@leibniz-psychology.org> writes:

>

>> Hi Maxim,

>>

>>> Judging from the above, it seems this issue has been resolved.

>> grafting is still a performance issue imo. Compare for example:

>>

>> $ time guix environment --ad-hoc --search-paths r-learnr

>> guix environment --ad-hoc --search-paths r-learnr 5,90s user 0,09s system 210% cpu 2,844 total

>> $ time guix environment --ad-hoc --search-paths r-learnr --no-grafts

>> guix environment --ad-hoc --search-paths r-learnr --no-grafts 2,03s user 0,08s system 164% cpu 1,277 total

>

> I'm opening a new issue to track optimizing the grafting code, since

> it's independent of environments (grafts are applied anytime a

> derivation is built, AFAICT). Grafting is inherently IO-bound,

> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

>

>> Lars-Dominik Braun <ldb@leibniz-psychology.org> writes:

>>

>>> Hi Maxim,

>>>

>>>> Judging from the above, it seems this issue has been resolved.

>>> grafting is still a performance issue imo. Compare for example:

>>>

>>> $ time guix environment --ad-hoc --search-paths r-learnr

>>> guix environment --ad-hoc --search-paths r-learnr 5,90s user 0,09s system 210% cpu 2,844 total

>>> $ time guix environment --ad-hoc --search-paths r-learnr --no-grafts

>>> guix environment --ad-hoc --search-paths r-learnr --no-grafts 2,03s user 0,08s system 164% cpu 1,277 total

>>

>> I'm opening a new issue to track optimizing the grafting code, since

>> it's independent of environments (grafts are applied anytime a

>> derivation is built, AFAICT). Grafting is inherently IO-bound,

>

> What is slow above is not grafting itself: it’s determining what to

> graft that takes CPU time.

> I had reopened the initial bug at https://issues.guix.gnu.org/41702;

> should we close this one?

>>>> $ time guix environment --ad-hoc --search-paths r-learnr

>>>> guix environment --ad-hoc --search-paths r-learnr 5,90s user 0,09s system 210% cpu 2,844 total

>>>> $ time guix environment --ad-hoc --search-paths r-learnr --no-grafts

>>>> guix environment --ad-hoc --search-paths r-learnr --no-grafts 2,03s user 0,08s system 164% cpu 1,277 total

>>>

>>> I'm opening a new issue to track optimizing the grafting code, since

>>> it's independent of environments (grafts are applied anytime a

>>> derivation is built, AFAICT). Grafting is inherently IO-bound,

>>

>> What is slow above is not grafting itself: it’s determining what to

>> graft that takes CPU time.

>

> On my system, grafting seems IO rather than CPU bound, I'm guessing

> because of the need to scan all the files for strings to replace in the

> graft process.

>> I had reopened the initial bug at https://issues.guix.gnu.org/41702;

>> should we close this one?

>

> Many optimizations were made in the above issue that were not related to

> the grafting process, so to me a fresh entry such as this one is clearer

> to follow. That said, feel free to proceed as you see fit, being the

> issue "owner" :-).

> Yes, you’re right of course. But I think in the example above Lars was

> reporting the run time of the ‘guix’ command when the graft itself is

> already built. Just the extra code to compute the graft derivation (not

> actually building them) leads to x2 wall-clock time or so.