[PATCH 0/5] Optimize profile hooks

  • Done
  • quality assurance status badge
Details
3 participants
  • 宋文武
  • Leo Prikler
  • Ludovic Courtès
Owner
unassigned
Submitted by
宋文武
Severity
normal
Merged with

Debbugs page

宋文武 wrote 7 years ago
(address . guix-patches@gnu.org)(name . 宋文武)(address . iyzsong@member.fsf.org)
20180101103336.8613-1-iyzsong@member.fsf.org
Hello, these patches make each profile hook run upon its specified interested
inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
install a package without info files won't trigger it. Thus reduce the chance
and time to rerun them when your profile changed.


One drawback is 'guix package --dry-run' no longer report the derivations of
profile hooks, and the derivation of profile it reports is not the real one.
Addition files will be built when the profiles hooks are run.


Sou Bunnbu (宋文武) (5):
gexp: Add 'eval-gexp'.
profiles: info-dir-file: Don't consider unwanted manifest entries.
guix package: Disable profile hooks on dry runs.
profiles: Filter out unwanted manifest entries for profile hooks.
profiles: Sort manifest inputs for profile hooks.

guix/gexp.scm | 16 ++++
guix/profiles.scm | 188 ++++++++++++++++++++++++++++++++---------------
guix/scripts/package.scm | 2 +-
3 files changed, 146 insertions(+), 60 deletions(-)

--
2.13.3
Ludovic Courtès wrote 7 years ago
(name . 宋文武)(address . iyzsong@member.fsf.org)(address . 29928@debbugs.gnu.org)
876088m2vd.fsf@gnu.org
Hello!

宋文武 <iyzsong@member.fsf.org> skribis:

Toggle quote (5 lines)
> Hello, these patches make each profile hook run upon its specified interested
> inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
> install a package without info files won't trigger it. Thus reduce the chance
> and time to rerun them when your profile changed.

That’s an area in need of improvements, thanks for looking into it!

Toggle quote (4 lines)
> One drawback is 'guix package --dry-run' no longer report the derivations of
> profile hooks, and the derivation of profile it reports is not the real one.
> Addition files will be built when the profiles hooks are run.

FWIW I’m not entirely convinced by the approach. As discussed earlier,
I’d like to experiment with a notion of “build rounds”: the first round
would build a profile without any hooks, the second round would,
depending on what the profile contains, rebuild it with certain hooks.

I don’t have anything to show yet, though…

Ludo’.
宋文武 wrote 7 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 29928@debbugs.gnu.org)
87vafxvrj7.fsf@member.fsf.org
ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (17 lines)
> Hello!
>
> 宋文武 <iyzsong@member.fsf.org> skribis:
>
>> Hello, these patches make each profile hook run upon its specified interested
>> inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
>> install a package without info files won't trigger it. Thus reduce the chance
>> and time to rerun them when your profile changed.
>
> That’s an area in need of improvements, thanks for looking into it!
>
>> One drawback is 'guix package --dry-run' no longer report the derivations of
>> profile hooks, and the derivation of profile it reports is not the real one.
>> Addition files will be built when the profiles hooks are run.
>
> FWIW I’m not entirely convinced by the approach.

Well.. these patches modify package hooks to:

1. build all manifest inputs first.

2. filter manifest inputs to get interested ones.

3. run hook with its interested inputs.

I think reducing the inputs of hook from the whole manifest to its
interested ones is the only way to avoid unneeded reruns.


Toggle quote (5 lines)
> As discussed earlier, I’d like to experiment with a notion of “build
> rounds”: the first round would build a profile without any hooks, the
> second round would, depending on what the profile contains, rebuild it
> with certain hooks.

I think "build rounds" would improve the UI/UX, and by changing inputs
from manifest inputs to a built profile, it would simply the current
implementations of profile hooks, but it won't avoid unneeded reruns
compare to the filtered interested inputs way.

Is my understanding correct?
Ludovic Courtès wrote 7 years ago
(name . 宋文武)(address . iyzsong@member.fsf.org)(address . 29928@debbugs.gnu.org)
87o9lp3kdl.fsf@gnu.org
Hi,

iyzsong@member.fsf.org (宋文武) skribis:

Toggle quote (2 lines)
> ludo@gnu.org (Ludovic Courtès) writes:

[...]

Toggle quote (17 lines)
>>> One drawback is 'guix package --dry-run' no longer report the derivations of
>>> profile hooks, and the derivation of profile it reports is not the real one.
>>> Addition files will be built when the profiles hooks are run.
>>
>> FWIW I’m not entirely convinced by the approach.
>
> Well.. these patches modify package hooks to:
>
> 1. build all manifest inputs first.
>
> 2. filter manifest inputs to get interested ones.
>
> 3. run hook with its interested inputs.
>
> I think reducing the inputs of hook from the whole manifest to its
> interested ones is the only way to avoid unneeded reruns.

Agreed.

Toggle quote (12 lines)
>> As discussed earlier, I’d like to experiment with a notion of “build
>> rounds”: the first round would build a profile without any hooks, the
>> second round would, depending on what the profile contains, rebuild it
>> with certain hooks.
>
> I think "build rounds" would improve the UI/UX, and by changing inputs
> from manifest inputs to a built profile, it would simply the current
> implementations of profile hooks, but it won't avoid unneeded reruns
> compare to the filtered interested inputs way.
>
> Is my understanding correct?

A “build round” is something that computes derivations based on the
output of previously-built derivations. So it’s just another way to
structure steps 1/2/3 above; it should not reduce expressivity.

Build rounds would allow us to ensure that ‘build-derivations’ is not
called right in the middle of Guix, and is instead always under the
control of the “top level”.

Ludo’.
宋文武 wrote 7 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 29928@debbugs.gnu.org)
87wp0cvgj9.fsf@member.fsf.org
ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (48 lines)
> Hi,
>
> iyzsong@member.fsf.org (宋文武) skribis:
>
>> ludo@gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>>>> One drawback is 'guix package --dry-run' no longer report the derivations of
>>>> profile hooks, and the derivation of profile it reports is not the real one.
>>>> Addition files will be built when the profiles hooks are run.
>>>
>>> FWIW I’m not entirely convinced by the approach.
>>
>> Well.. these patches modify package hooks to:
>>
>> 1. build all manifest inputs first.
>>
>> 2. filter manifest inputs to get interested ones.
>>
>> 3. run hook with its interested inputs.
>>
>> I think reducing the inputs of hook from the whole manifest to its
>> interested ones is the only way to avoid unneeded reruns.
>
> Agreed.
>
>>> As discussed earlier, I’d like to experiment with a notion of “build
>>> rounds”: the first round would build a profile without any hooks, the
>>> second round would, depending on what the profile contains, rebuild it
>>> with certain hooks.
>>
>> I think "build rounds" would improve the UI/UX, and by changing inputs
>> from manifest inputs to a built profile, it would simply the current
>> implementations of profile hooks, but it won't avoid unneeded reruns
>> compare to the filtered interested inputs way.
>>
>> Is my understanding correct?
>
> A “build round” is something that computes derivations based on the
> output of previously-built derivations. So it’s just another way to
> structure steps 1/2/3 above; it should not reduce expressivity.
>
> Build rounds would allow us to ensure that ‘build-derivations’ is not
> called right in the middle of Guix, and is instead always under the
> control of the “top level”.
>

I see, looking forward it, thanks!
Ludovic Courtès wrote 7 years ago
control message for bug #29927
(address . control@debbugs.gnu.org)
87o9k0rnhv.fsf@gnu.org
merge 29927 29928
Leo Prikler wrote 4 years ago
Re: [PATCH 0/5] Optimize profile hooks
(address . control@debbugs.gnu.org)(address . 29928@debbugs.gnu.org)(name . 宋文武)(address . iyzsong@member.fsf.org)
65a2d7596fb69e0d139b4e2b7a63b354a8f6bf48.camel@student.tugraz.at
merge 29928 29926
merge 29928 29927
thanks

This series (29928 29926 29927 29925 29929 29930) has by now slept on
the mailing list for more than three years.
Should we still try to merge it? If so, could you send an updated
version, that is not spread across six threads?

Thanks,
Leo
?
Your comment

This issue is archived.

To comment on this conversation send an email to 29928@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 29928
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help