[PATCH 0/5] Optimize profile hooks

  • Done
  • quality assurance status badge
Details
3 participants
  • ???
  • Leo Prikler
  • Ludovic Courtès
Owner
unassigned
Submitted by
???
Severity
normal
Merged with
?
?
??? wrote on 1 Jan 2018 11:33
(address . guix-patches@gnu.org)(name . ???)(address . iyzsong@member.fsf.org)
20180101103336.8613-1-iyzsong@member.fsf.org
Hello, these patches make each profile hook run upon its specified interested
inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
install a package without info files won't trigger it. Thus reduce the chance
and time to rerun them when your profile changed.


One drawback is 'guix package --dry-run' no longer report the derivations of
profile hooks, and the derivation of profile it reports is not the real one.
Addition files will be built when the profiles hooks are run.


Sou Bunnbu (???) (5):
gexp: Add 'eval-gexp'.
profiles: info-dir-file: Don't consider unwanted manifest entries.
guix package: Disable profile hooks on dry runs.
profiles: Filter out unwanted manifest entries for profile hooks.
profiles: Sort manifest inputs for profile hooks.

guix/gexp.scm | 16 ++++
guix/profiles.scm | 188 ++++++++++++++++++++++++++++++++---------------
guix/scripts/package.scm | 2 +-
3 files changed, 146 insertions(+), 60 deletions(-)

--
2.13.3
L
L
Ludovic Courtès wrote on 11 Jan 2018 23:45
(name . ???)(address . iyzsong@member.fsf.org)(address . 29928@debbugs.gnu.org)
876088m2vd.fsf@gnu.org
Hello!

??? <iyzsong@member.fsf.org> skribis:

Toggle quote (5 lines)
> Hello, these patches make each profile hook run upon its specified interested
> inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
> install a package without info files won't trigger it. Thus reduce the chance
> and time to rerun them when your profile changed.

That’s an area in need of improvements, thanks for looking into it!

Toggle quote (4 lines)
> One drawback is 'guix package --dry-run' no longer report the derivations of
> profile hooks, and the derivation of profile it reports is not the real one.
> Addition files will be built when the profiles hooks are run.

FWIW I’m not entirely convinced by the approach. As discussed earlier,
I’d like to experiment with a notion of “build rounds”: the first round
would build a profile without any hooks, the second round would,
depending on what the profile contains, rebuild it with certain hooks.

I don’t have anything to show yet, though…

Ludo’.
?
?
??? wrote on 19 Jan 2018 15:42
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 29928@debbugs.gnu.org)
87vafxvrj7.fsf@member.fsf.org
ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (17 lines)
> Hello!
>
> ??? <iyzsong@member.fsf.org> skribis:
>
>> Hello, these patches make each profile hook run upon its specified interested
>> inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
>> install a package without info files won't trigger it. Thus reduce the chance
>> and time to rerun them when your profile changed.
>
> That’s an area in need of improvements, thanks for looking into it!
>
>> One drawback is 'guix package --dry-run' no longer report the derivations of
>> profile hooks, and the derivation of profile it reports is not the real one.
>> Addition files will be built when the profiles hooks are run.
>
> FWIW I’m not entirely convinced by the approach.

Well.. these patches modify package hooks to:

1. build all manifest inputs first.

2. filter manifest inputs to get interested ones.

3. run hook with its interested inputs.

I think reducing the inputs of hook from the whole manifest to its
interested ones is the only way to avoid unneeded reruns.


Toggle quote (5 lines)
> As discussed earlier, I’d like to experiment with a notion of “build
> rounds”: the first round would build a profile without any hooks, the
> second round would, depending on what the profile contains, rebuild it
> with certain hooks.

I think "build rounds" would improve the UI/UX, and by changing inputs
from manifest inputs to a built profile, it would simply the current
implementations of profile hooks, but it won't avoid unneeded reruns
compare to the filtered interested inputs way.

Is my understanding correct?
L
L
Ludovic Courtès wrote on 19 Jan 2018 17:04
(name . ???)(address . iyzsong@member.fsf.org)(address . 29928@debbugs.gnu.org)
87o9lp3kdl.fsf@gnu.org
Hi,

iyzsong@member.fsf.org (???) skribis:

Toggle quote (2 lines)
> ludo@gnu.org (Ludovic Courtès) writes:

[...]

Toggle quote (17 lines)
>>> One drawback is 'guix package --dry-run' no longer report the derivations of
>>> profile hooks, and the derivation of profile it reports is not the real one.
>>> Addition files will be built when the profiles hooks are run.
>>
>> FWIW I’m not entirely convinced by the approach.
>
> Well.. these patches modify package hooks to:
>
> 1. build all manifest inputs first.
>
> 2. filter manifest inputs to get interested ones.
>
> 3. run hook with its interested inputs.
>
> I think reducing the inputs of hook from the whole manifest to its
> interested ones is the only way to avoid unneeded reruns.

Agreed.

Toggle quote (12 lines)
>> As discussed earlier, I’d like to experiment with a notion of “build
>> rounds”: the first round would build a profile without any hooks, the
>> second round would, depending on what the profile contains, rebuild it
>> with certain hooks.
>
> I think "build rounds" would improve the UI/UX, and by changing inputs
> from manifest inputs to a built profile, it would simply the current
> implementations of profile hooks, but it won't avoid unneeded reruns
> compare to the filtered interested inputs way.
>
> Is my understanding correct?

A “build round” is something that computes derivations based on the
output of previously-built derivations. So it’s just another way to
structure steps 1/2/3 above; it should not reduce expressivity.

Build rounds would allow us to ensure that ‘build-derivations’ is not
called right in the middle of Guix, and is instead always under the
control of the “top level”.

Ludo’.
?
?
??? wrote on 20 Jan 2018 13:52
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 29928@debbugs.gnu.org)
87wp0cvgj9.fsf@member.fsf.org
ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (48 lines)
> Hi,
>
> iyzsong@member.fsf.org (???) skribis:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>>>> One drawback is 'guix package --dry-run' no longer report the derivations of
>>>> profile hooks, and the derivation of profile it reports is not the real one.
>>>> Addition files will be built when the profiles hooks are run.
>>>
>>> FWIW I’m not entirely convinced by the approach.
>>
>> Well.. these patches modify package hooks to:
>>
>> 1. build all manifest inputs first.
>>
>> 2. filter manifest inputs to get interested ones.
>>
>> 3. run hook with its interested inputs.
>>
>> I think reducing the inputs of hook from the whole manifest to its
>> interested ones is the only way to avoid unneeded reruns.
>
> Agreed.
>
>>> As discussed earlier, I’d like to experiment with a notion of “build
>>> rounds”: the first round would build a profile without any hooks, the
>>> second round would, depending on what the profile contains, rebuild it
>>> with certain hooks.
>>
>> I think "build rounds" would improve the UI/UX, and by changing inputs
>> from manifest inputs to a built profile, it would simply the current
>> implementations of profile hooks, but it won't avoid unneeded reruns
>> compare to the filtered interested inputs way.
>>
>> Is my understanding correct?
>
> A “build round” is something that computes derivations based on the
> output of previously-built derivations. So it’s just another way to
> structure steps 1/2/3 above; it should not reduce expressivity.
>
> Build rounds would allow us to ensure that ‘build-derivations’ is not
> called right in the middle of Guix, and is instead always under the
> control of the “top level”.
>

I see, looking forward it, thanks!
L
L
Ludovic Courtès wrote on 7 Mar 2018 15:03
control message for bug #29927
(address . control@debbugs.gnu.org)
87o9k0rnhv.fsf@gnu.org
merge 29927 29928
L
L
Leo Prikler wrote on 11 May 2021 15:34
Re: [PATCH 0/5] Optimize profile hooks
(name . ???)(address . iyzsong@member.fsf.org)
65a2d7596fb69e0d139b4e2b7a63b354a8f6bf48.camel@student.tugraz.at
merge 29928 29926
merge 29928 29927
thanks

This series (29928 29926 29927 29925 29929 29930) has by now slept on
the mailing list for more than three years.
Should we still try to merge it? If so, could you send an updated
version, that is not spread across six threads?

Thanks,
Leo
?