can't compose 'with-patch' with 'with-source'

  • Open
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Maxim Cournoyer
  • Simon Tournier
Owner
unassigned
Submitted by
Maxim Cournoyer
Severity
normal
M
M
Maxim Cournoyer wrote on 21 Feb 2023 19:08
(name . bug-guix)(address . bug-guix@gnu.org)
87mt56gac6.fsf@gmail.com
Hi,

Given 'with-source' discards any patch from the original source, I thought
I could at least add them back via 'with-patch', but it appears this
does not work:

Toggle snippet (13 lines)
scheme@(gnu packages jami)> (options->transformation
`((with-source . "libjami@20230220.0=/home/maxim/src/jami/jami-20230220.0.tar.gz")
(with-patch . ,(string-append
"libjami=" (search-patch
"jami-disable-integration-tests.patch")))))
$6 = #<procedure 7f2cd01a97e0 at guix/transformations.scm:1010:2 (obj)>
scheme@(gnu packages jami)> ($6 libjami)
$7 = #<package libjami@20230220.0 guix/transformations.scm:1002 7f2ccc8386e0>
scheme@(gnu packages jami)> (package-source $7)
$8 = #<<downloaded-file> uri: "/home/maxim/src/jami/jami-20230220.0.tar.gz" recursive?: #t>
scheme@(gnu packages jami)>

The downloaded-file resulting package source has lost the patch, and no
error got produced, leaving the user to discover this limitation by
themselves.

--
Thanks,
Maxim
S
S
Simon Tournier wrote on 21 Feb 2023 20:05
86sfeyvnyb.fsf@gmail.com
Hi Maxim,

On Tue, 21 Feb 2023 at 13:08, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

Toggle quote (4 lines)
> Given 'with-source' discards any patch from the original source, I thought
> I could at least add them back via 'with-patch', but it appears this
> does not work:

I remember some headaches with this thread [1]. :-)


Toggle quote (19 lines)
>
> --8<---------------cut here---------------start------------->8---
> scheme@(gnu packages jami)> (options->transformation
> `((with-source . "libjami@20230220.0=/home/maxim/src/jami/jami-20230220.0.tar.gz")
> (with-patch . ,(string-append
> "libjami=" (search-patch
> "jami-disable-integration-tests.patch")))))
> $6 = #<procedure 7f2cd01a97e0 at guix/transformations.scm:1010:2 (obj)>
> scheme@(gnu packages jami)> ($6 libjami)
> $7 = #<package libjami@20230220.0 guix/transformations.scm:1002 7f2ccc8386e0>
> scheme@(gnu packages jami)> (package-source $7)
> $8 = #<<downloaded-file> uri: "/home/maxim/src/jami/jami-20230220.0.tar.gz" recursive?: #t>
> scheme@(gnu packages jami)>
> --8<---------------cut here---------------end--------------->8---
>
> The downloaded-file resulting package source has lost the patch, and no
> error got produced, leaving the user to discover this limitation by
> themselves.

Well, it is probably unrelated because I guess the transformation makes
sense here but indeed you can have bad surprise if the transformation
does not make sense and then silently ignored.

Well, I have never finished my attempt to raise more information about
the transformation because the code about the transformation is hard to
follow, from my point of view. Anyway!

Thanks for having open this ticket. :-) Maybe with-source and
with-patch are the first incremental change. ;-)

Cheers,
simon
L
L
Ludovic Courtès wrote on 23 Feb 2023 15:08
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 61684@debbugs.gnu.org)
871qmg79u7.fsf@gnu.org
Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (20 lines)
> Given 'with-source' discards any patch from the original source, I thought
> I could at least add them back via 'with-patch', but it appears this
> does not work:
>
> scheme@(gnu packages jami)> (options->transformation
> `((with-source . "libjami@20230220.0=/home/maxim/src/jami/jami-20230220.0.tar.gz")
> (with-patch . ,(string-append
> "libjami=" (search-patch
> "jami-disable-integration-tests.patch")))))
> $6 = #<procedure 7f2cd01a97e0 at guix/transformations.scm:1010:2 (obj)>
> scheme@(gnu packages jami)> ($6 libjami)
> $7 = #<package libjami@20230220.0 guix/transformations.scm:1002 7f2ccc8386e0>
> scheme@(gnu packages jami)> (package-source $7)
> $8 = #<<downloaded-file> uri: "/home/maxim/src/jami/jami-20230220.0.tar.gz" recursive?: #t>
> scheme@(gnu packages jami)>
>
> The downloaded-file resulting package source has lost the patch, and no
> error got produced, leaving the user to discover this limitation by
> themselves.

The order of options matters; in this case, you need to do it the other
way around:

Toggle snippet (9 lines)
scheme@(guile-user)> (options->transformation '((with-patch . "jami=/tmp/t.patch")
(with-source . "jami=http://example.org/foo.tar.gz")))
$18 = #<procedure 7f7e6b1fd0c0 at guix/transformations.scm:1010:2 (obj)>
scheme@(guile-user)> ($18 jami)
$19 = #<package jami@20230206.0 guix/transformations.scm:1002 7f7e6b1602c0>
scheme@(guile-user)> (package-source $19)
$20 = #<<computed-file> name: "jami-20230206.0-source" gexp: #<gexp guix/transformations.scm:688:19 7f7e6b6c8d50> guile: #f options: (#:local-build? #t)>

The <computed-file> bit comes from the ‘with-patch’ transformation, to
apply the patch to the <downloaded-file>.

If you do it the other way around, the effect of ‘with-patch’ is
canceled by that of ‘with-source’, which does not preserve patches. So
strictly speaking, both options had an effect, but they were
contradictory.

Note that ordering is “specified”, notably with the test added in
0f024554e63a49e20c2a7a67e928073c266bf5c5 (this is crucial for our HPC
users, who routinely combine a whole bunch of options; you have no idea
how far they go once you give them the tool :-)). I don’t think the
manual explicitly states it though.

HTH,
Ludo’.
M
M
Maxim Cournoyer wrote on 23 Feb 2023 23:27
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 61684@debbugs.gnu.org)
87356whvaa.fsf@gmail.com
Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (43 lines)
> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Given 'with-source' discards any patch from the original source, I thought
>> I could at least add them back via 'with-patch', but it appears this
>> does not work:
>>
>> scheme@(gnu packages jami)> (options->transformation
>> `((with-source . "libjami@20230220.0=/home/maxim/src/jami/jami-20230220.0.tar.gz")
>> (with-patch . ,(string-append
>> "libjami=" (search-patch
>> "jami-disable-integration-tests.patch")))))
>> $6 = #<procedure 7f2cd01a97e0 at guix/transformations.scm:1010:2 (obj)>
>> scheme@(gnu packages jami)> ($6 libjami)
>> $7 = #<package libjami@20230220.0 guix/transformations.scm:1002 7f2ccc8386e0>
>> scheme@(gnu packages jami)> (package-source $7)
>> $8 = #<<downloaded-file> uri: "/home/maxim/src/jami/jami-20230220.0.tar.gz" recursive?: #t>
>> scheme@(gnu packages jami)>
>>
>> The downloaded-file resulting package source has lost the patch, and no
>> error got produced, leaving the user to discover this limitation by
>> themselves.
>
> The order of options matters; in this case, you need to do it the other
> way around:
>
> scheme@(guile-user)> (options->transformation '((with-patch . "jami=/tmp/t.patch")
> (with-source . "jami=http://example.org/foo.tar.gz")))
> $18 = #<procedure 7f7e6b1fd0c0 at guix/transformations.scm:1010:2 (obj)>
> scheme@(guile-user)> ($18 jami)
> $19 = #<package jami@20230206.0 guix/transformations.scm:1002 7f7e6b1602c0>
> scheme@(guile-user)> (package-source $19)
> $20 = #<<computed-file> name: "jami-20230206.0-source" gexp: #<gexp guix/transformations.scm:688:19 7f7e6b6c8d50> guile: #f options: (#:local-build? #t)>
>
> The <computed-file> bit comes from the ‘with-patch’ transformation, to
> apply the patch to the <downloaded-file>.
>
> If you do it the other way around, the effect of ‘with-patch’ is
> canceled by that of ‘with-source’, which does not preserve patches. So
> strictly speaking, both options had an effect, but they were
> contradictory.

Hm. That seems sub-optimal; it seems to me that ideally, the
transformations would be additive, so that users would not need to care
about the ordering. Or perhaps, alternatively, we could enforce such
ordering at the implementation level (sorting the transformations in the
order that is required).

Toggle quote (6 lines)
> Note that ordering is “specified”, notably with the test added in
> 0f024554e63a49e20c2a7a67e928073c266bf5c5 (this is crucial for our HPC
> users, who routinely combine a whole bunch of options; you have no idea
> how far they go once you give them the tool :-)). I don’t think the
> manual explicitly states it though.

It doesn't seem to be, indeed. But I wish we wouldn't need to (see my
above suggestions/ideas).

--
Thanks,
Maxim
S
S
Simon Tournier wrote on 24 Feb 2023 13:02
(address . 61684@debbugs.gnu.org)
875ybrmfuv.fsf@gmail.com
Hi,

On jeu., 23 févr. 2023 at 17:27, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

Toggle quote (6 lines)
> Hm. That seems sub-optimal; it seems to me that ideally, the
> transformations would be additive, so that users would not need to care
> about the ordering. Or perhaps, alternatively, we could enforce such
> ordering at the implementation level (sorting the transformations in the
> order that is required).

From my point of view (and what I tried stopping in the middle :-)) is
to report if the transformation makes sense or not. For instance,

with-patch
with-source

makes sense contrary to

with-source
with-patch

and it would already be an improvement to report that the latter
transformation does not make sense instead of silently does nothing or
raises some weird errors.

Well, I am not convinced that enforce the ordering is a good thing
because as Ludo said, some HPC user exploits this control of ordering to
generate complex transformations.

To me, the fix is:

1. document the ordering bits
2. check if the ordering “makes sense“ and raises if not.


Cheers,
simon
M
M
Maxim Cournoyer wrote on 24 Feb 2023 14:21
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87h6vbgpxf.fsf@gmail.com
Hello,

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (29 lines)
> Hi,
>
> On jeu., 23 févr. 2023 at 17:27, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>> Hm. That seems sub-optimal; it seems to me that ideally, the
>> transformations would be additive, so that users would not need to care
>> about the ordering. Or perhaps, alternatively, we could enforce such
>> ordering at the implementation level (sorting the transformations in the
>> order that is required).
>
> From my point of view (and what I tried stopping in the middle :-)) is
> to report if the transformation makes sense or not. For instance,
>
> with-patch
> with-source
>
> makes sense contrary to
>
> with-source
> with-patch
>
> and it would already be an improvement to report that the latter
> transformation does not make sense instead of silently does nothing or
> raises some weird errors.
>
> Well, I am not convinced that enforce the ordering is a good thing
> because as Ludo said, some HPC user exploits this control of ordering to
> generate complex transformations.

Could we gather more information about that use case? It needs to be
clear if we are to constrain the design (solution) by it.

Toggle quote (5 lines)
> To me, the fix is:
>
> 1. document the ordering bits
> 2. check if the ordering “makes sense“ and raises if not.

If the use case above (the one where transformation order matters and
are useful) is expounded and appears reasonable, then yes, that seems
like a good solution. Such a use case could appear in a complex example
next to the documentation that explains the ordering "rules".

--
Thanks,
Maxim
S
S
Simon Tournier wrote on 24 Feb 2023 14:43
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87y1onkwm0.fsf@gmail.com
Hi Maxim,

On ven., 24 févr. 2023 at 08:21, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

Toggle quote (7 lines)
>> Well, I am not convinced that enforce the ordering is a good thing
>> because as Ludo said, some HPC user exploits this control of ordering to
>> generate complex transformations.
>
> Could we gather more information about that use case? It needs to be
> clear if we are to constrain the design (solution) by it.

Well, I do not have the details. Just to mention two presentations
[1,2] exposing how transformations help for them.


There they intensively uses transformations. For instance, p.4 of [2]
it reads,

guix environment --pure --with-input=pastix-5=pastix-5-mkl \
--with-input=mumps-scotch-openmpi=mumps-mkl-scotch-openmpi \
--with-input=openblas=mkl --with-git-url=gcvb=$HOME/src/gcvb \
--with-commit=gcvb=40d88ba241db4c71ac3e1fe8024fba4d906f45b1 \
--preserve=^SLURM --ad-hoc bash coreutils inetutils findutils \
grep sed bc openssh python python-psutil gcvb scab slurm@19 openmpi

For this specific example, the order may or not matter. The point is
that HPC folks are intensively using transformations and, since the
order currently matters, enforcing one specific order could break their
workflow, and even could make impossible what is currently possible.

Quoting Ludo,

(this is crucial for our HPC
users, who routinely combine a whole bunch of options; you have no idea
how far they go once you give them the tool :-))


and I agree with « you have no idea how far they go once you give them
the tool :-)) ».

For what my opinion is worth. :-)

Cheers,
simon
M
M
Maxim Cournoyer wrote on 24 Feb 2023 14:52
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87cz5zgogo.fsf@gmail.com
Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (32 lines)
> Hi Maxim,
>
> On ven., 24 févr. 2023 at 08:21, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>>> Well, I am not convinced that enforce the ordering is a good thing
>>> because as Ludo said, some HPC user exploits this control of ordering to
>>> generate complex transformations.
>>
>> Could we gather more information about that use case? It needs to be
>> clear if we are to constrain the design (solution) by it.
>
> Well, I do not have the details. Just to mention two presentations
> [1,2] exposing how transformations help for them.
>
> 1: <https://10years.guix.gnu.org/static/slides/07-swartvagher.pdf>
> 2: <https://hpc.guix.info/static/doc/atelier-reproductibilit%C3%A9-2021/marek-fel%C5%A1%C3%B6ci-org-guix.pdf>
>
> There they intensively uses transformations. For instance, p.4 of [2]
> it reads,
>
> guix environment --pure --with-input=pastix-5=pastix-5-mkl \
> --with-input=mumps-scotch-openmpi=mumps-mkl-scotch-openmpi \
> --with-input=openblas=mkl --with-git-url=gcvb=$HOME/src/gcvb \
> --with-commit=gcvb=40d88ba241db4c71ac3e1fe8024fba4d906f45b1 \
> --preserve=^SLURM --ad-hoc bash coreutils inetutils findutils \
> grep sed bc openssh python python-psutil gcvb scab slurm@19 openmpi
>
> For this specific example, the order may or not matter. The point is
> that HPC folks are intensively using transformations and, since the
> order currently matters, enforcing one specific order could break their
> workflow, and even could make impossible what is currently possible.

I'd argue that in the above case, the order doesn't or shouldn't matter.
The user clearly intended for them to be additive.

Toggle quote (11 lines)
> Quoting Ludo,
>
> (this is crucial for our HPC
> users, who routinely combine a whole bunch of options; you have no idea
> how far they go once you give them the tool :-))
>
> <https://issues.guix.gnu.org/msgid/871qmg79u7.fsf@gnu.org>
>
> and I agree with « you have no idea how far they go once you give them
> the tool :-)) ».

It seems a bit hypothetical at this point, so I wouldn't want to cripple
the design by it. Sure, users will find by experimentation how to
accomplish something, perhaps twisting it in odd ways to, but in my
opinion if the API was consistent it'd be much easier for everyone to
accomplish what they are after. *If* a valid use case of the current
quirky behavior is discovered, we could consider adding a
'--order-sensitive' option or similar to restore that behavior, I just
don't think it should be the default.

Toggle quote (2 lines)
> For what my opinion is worth. :-)

Thanks for sharing it!

--
Thanks,
Maxim
M
M
Maxim Cournoyer wrote on 24 Feb 2023 14:59
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 61684@debbugs.gnu.org)
877cw7go5k.fsf@gmail.com
Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (38 lines)
> Hi Maxim,
>
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>> Given 'with-source' discards any patch from the original source, I thought
>> I could at least add them back via 'with-patch', but it appears this
>> does not work:
>>
>> scheme@(gnu packages jami)> (options->transformation
>> `((with-source . "libjami@20230220.0=/home/maxim/src/jami/jami-20230220.0.tar.gz")
>> (with-patch . ,(string-append
>> "libjami=" (search-patch
>> "jami-disable-integration-tests.patch")))))
>> $6 = #<procedure 7f2cd01a97e0 at guix/transformations.scm:1010:2 (obj)>
>> scheme@(gnu packages jami)> ($6 libjami)
>> $7 = #<package libjami@20230220.0 guix/transformations.scm:1002 7f2ccc8386e0>
>> scheme@(gnu packages jami)> (package-source $7)
>> $8 = #<<downloaded-file> uri: "/home/maxim/src/jami/jami-20230220.0.tar.gz" recursive?: #t>
>> scheme@(gnu packages jami)>
>>
>> The downloaded-file resulting package source has lost the patch, and no
>> error got produced, leaving the user to discover this limitation by
>> themselves.
>
> The order of options matters; in this case, you need to do it the other
> way around:
>
> scheme@(guile-user)> (options->transformation '((with-patch . "jami=/tmp/t.patch")
> (with-source . "jami=http://example.org/foo.tar.gz")))
> $18 = #<procedure 7f7e6b1fd0c0 at guix/transformations.scm:1010:2 (obj)>
> scheme@(guile-user)> ($18 jami)
> $19 = #<package jami@20230206.0 guix/transformations.scm:1002 7f7e6b1602c0>
> scheme@(guile-user)> (package-source $19)
> $20 = #<<computed-file> name: "jami-20230206.0-source" gexp: #<gexp guix/transformations.scm:688:19 7f7e6b6c8d50> guile: #f options: (#:local-build? #t)>
>
> The <computed-file> bit comes from the ‘with-patch’ transformation, to
> apply the patch to the <downloaded-file>.

I tried to do that, like so:

Toggle snippet (21 lines)
(define (with-latest-sources name)
(options->transformation
;; XXX: The ordering is important; with-patch must appear before
;; with-source, otherwise it is silently discarded.
`(,@(if (string=? name "libjami")
`((with-patch . ,(string-append
name "="
(search-patch
"jami-disable-integration-tests.patch")))
(with-patch . ,(string-append
name "="
(search-patch
"jami-libjami-headers-search.patch"))))
'())
(with-source . ,(format #f "~a@~a=~a" name
%release-version %release-file-name)))))

(define libjami/latest
((with-latest-sources "libjami") libjami))

Unfortunately the source derivation fails because it attempts to apply a
patch (a single one?) to a tarball:

Toggle snippet (22 lines)
(begin
(use-modules
(guix build utils))
(setenv "PATH" "/gnu/store/mp0syh29rjknflaiv0hkpdlb2mjk0rlx-patch-2.7.6/bin")
(copy-recursively "/gnu/store/ig8awlxbzrasp9p4f9vq8fqcidrma5qj-jami-20230224.0.tar.gz"
((@
(guile)
getenv)
"out"))
(chdir
((@
(guile)
getenv)
"out"))
(for-each
(lambda
(patch)
(invoke "patch" "-p1" "--batch" "-i" patch))
(quote
("/gnu/store/iq7hd3f9kr2kvz7lvnygq51yxp85gxbn-jami-libjami-headers-search.patch"))))

My expectation of the source rewriting options is that it should end up
with something like what I'd write manually, e.g. an origin with an
appropriate fetch method and its patches field populated, etc., so that
it'd work the same.

--
Thanks,
Maxim
S
S
Simon Tournier wrote on 25 Feb 2023 13:29
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
86zg92apyb.fsf@gmail.com
Hi Maxim,

On Fri, 24 Feb 2023 at 08:52, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:

Toggle quote (2 lines)
> if the API was consistent it'd be much easier for everyone

Indeed. However, when it is currently not, the implicit rule is to not
break backward compatibility. That’s the whole point. :-)

We need to be very cautious when we change the API; even when it is for
fixing an inconsistency.

Cheers,
simon
M
M
Maxim Cournoyer wrote on 25 Feb 2023 20:43
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87a611fs4v.fsf@gmail.com
Hi,

Simon Tournier <zimon.toutoune@gmail.com> writes:

Toggle quote (12 lines)
> Hi Maxim,
>
> On Fri, 24 Feb 2023 at 08:52, Maxim Cournoyer <maxim.cournoyer@gmail.com> wrote:
>
>> if the API was consistent it'd be much easier for everyone
>
> Indeed. However, when it is currently not, the implicit rule is to not
> break backward compatibility. That’s the whole point. :-)
>
> We need to be very cautious when we change the API; even when it is for
> fixing an inconsistency.

Agreed; if the current behavior really provides well outlined benefits,
we could always introduce a "--order-sensitive" option for its users.

--
Thanks,
Maxim
L
L
Ludovic Courtès wrote on 27 Feb 2023 15:09
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
878rgjkxnw.fsf@gnu.org
Hi!

Simon Tournier <zimon.toutoune@gmail.com> skribis:

Toggle quote (3 lines)
>>From my point of view (and what I tried stopping in the middle :-)) is
> to report if the transformation makes sense or not. For instance,

You stated that multiple times and there’s general consensus that
reporting the issue would be great.

However, as I explained before, there’s no clear way to do that for two
reasons:

1. Transformations apply to bags, not packages, so we cannot tell
whether a transformation has an effect until after the transformed
package has been lowered. Even then, it’s tricky.

2. In this case, this has to do with the semantics of transformations
themselves: by definition, ‘with-source’ dismisses patches.

[...]

Toggle quote (4 lines)
> Well, I am not convinced that enforce the ordering is a good thing
> because as Ludo said, some HPC user exploits this control of ordering to
> generate complex transformations.

My point is that there are folks who have been using package
transformation options for years; any claim has to be evaluated in that
light, and any change would have to be considered very carefully.

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 27 Feb 2023 15:12
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
87zg8zjiya.fsf@gnu.org
Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (14 lines)
>> Quoting Ludo,
>>
>> (this is crucial for our HPC
>> users, who routinely combine a whole bunch of options; you have no idea
>> how far they go once you give them the tool :-))
>>
>> <https://issues.guix.gnu.org/msgid/871qmg79u7.fsf@gnu.org>
>>
>> and I agree with « you have no idea how far they go once you give them
>> the tool :-)) ».
>
> It seems a bit hypothetical at this point, so I wouldn't want to cripple
> the design by it.

It baffles me that you would consider other people’s experience as
“hypothetical”. It’s not, really.

The choice of words after the comma is also unfortunate in several ways.

Ludo’.
M
M
Maxim Cournoyer wrote on 27 Feb 2023 17:29
(name . Ludovic Courtès)(address . ludo@gnu.org)
87fsarkr61.fsf@gmail.com
Hi Ludovic,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (19 lines)
> Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:
>
>>> Quoting Ludo,
>>>
>>> (this is crucial for our HPC
>>> users, who routinely combine a whole bunch of options; you have no idea
>>> how far they go once you give them the tool :-))
>>>
>>> <https://issues.guix.gnu.org/msgid/871qmg79u7.fsf@gnu.org>
>>>
>>> and I agree with « you have no idea how far they go once you give them
>>> the tool :-)) ».
>>
>> It seems a bit hypothetical at this point, so I wouldn't want to cripple
>> the design by it.
>
> It baffles me that you would consider other people’s experience as
> “hypothetical”. It’s not, really.

I didn't mean to dismiss other people experiences. The reason I'd like
to orient this discussion toward use cases, is that these are easier to
work with (more concrete/well defined) than user experiences. I'm sure
people use it in many ways I can't think of, and what I'd like us to do
is be able to put a finger on what these ways are, and how the current
API could be improved in a way that satisfies them all (if possible).

Toggle quote (2 lines)
> The choice of words after the comma is also unfortunate in several ways.

Noted. For the record I used in for its "flawed or imperfect"
definition.

--
Thanks,
Maxim
S
S
Simon Tournier wrote on 28 Feb 2023 11:00
(name . Ludovic Courtès)(address . ludo@gnu.org)
87y1oijejb.fsf@gmail.com
Hi Ludo,

On lun., 27 févr. 2023 at 15:09, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (10 lines)
> However, as I explained before, there’s no clear way to do that for two
> reasons:
>
> 1. Transformations apply to bags, not packages, so we cannot tell
> whether a transformation has an effect until after the transformed
> package has been lowered. Even then, it’s tricky.
>
> 2. In this case, this has to do with the semantics of transformations
> themselves: by definition, ‘with-source’ dismisses patches.

I probably miss many details and that’s why I do not understand
correctly your words. Or maybe we are not using the same meaning behind
“report if the transformation makes sense or not”.


1. From my point of view, the transformations are functions that you
compose. The composition rule is not commutative maybe neither
associative. Writing down how each function (transformation)
composes with the others allows to specify the composition rules.

2. All the code in (guix transformations) acts at the package level, so
I am still missing why it would not be possible to detect some
issues there.


For instance,

Toggle snippet (16 lines)
(define applicable
;; List of applicable transformations as symbol/procedure pairs in the
;; order in which they appear on the command line.
(filter-map (match-lambda
((key . value)
(match (transformation-procedure key)
(#f
#f)
(transform
;; XXX: We used to pass TRANSFORM a list of several
;; arguments, but we now pass only one, assuming that
;; transform composes well.
(list key value (transform (list value)))))))
(reverse opts)))

and I miss why,

1. it would not be possible to check if the transforms compose well;
somehow verify the assumption.

For instance, ’package-with-upstream-version’ raises many warnings
depending on various cases, and I miss why,

2. it would not be possible to have similar warnings for other
transformations.


Cheers,
simon
L
L
Ludovic Courtès wrote on 1 Mar 2023 16:49
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)
87pm9sbhf6.fsf@gnu.org
Hi,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

Toggle quote (9 lines)
> 1. From my point of view, the transformations are functions that you
> compose. The composition rule is not commutative maybe neither
> associative. Writing down how each function (transformation)
> composes with the others allows to specify the composition rules.
>
> 2. All the code in (guix transformations) acts at the package level, so
> I am still missing why it would not be possible to detect some
> issues there.

Please check out ‘package-input-rewriting’ and its behavior
with #:deep? #t. Design discussion: https://issues.guix.gnu.org/43578.

Toggle quote (3 lines)
> 1. it would not be possible to check if the transforms compose well;
> somehow verify the assumption.

Try it. :-) I tried to explain it multiple times, I really did, but I
guess there’s no substitute to first-hand experience.

Toggle quote (6 lines)
> For instance, ’package-with-upstream-version’ raises many warnings
> depending on various cases, and I miss why,
>
> 2. it would not be possible to have similar warnings for other
> transformations.

‘package-with-upstream-version’ raises warnings that depend exclusively
on local knowledge.

Here we’re talking about warnings related to the composition to two
different options which, like you wrote, are independent functions.
It’s not similar.

HTH!

Ludo’.
L
L
Ludovic Courtès wrote on 3 Mar 2023 10:43
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 61684@debbugs.gnu.org)
87h6v2187r.fsf@gnu.org
Hi Maxim,

Maxim Cournoyer <maxim.cournoyer@gmail.com> skribis:

Toggle quote (24 lines)
> Unfortunately the source derivation fails because it attempts to apply a
> patch (a single one?) to a tarball:
>
> (begin
> (use-modules
> (guix build utils))
> (setenv "PATH" "/gnu/store/mp0syh29rjknflaiv0hkpdlb2mjk0rlx-patch-2.7.6/bin")
> (copy-recursively "/gnu/store/ig8awlxbzrasp9p4f9vq8fqcidrma5qj-jami-20230224.0.tar.gz"
> ((@
> (guile)
> getenv)
> "out"))
> (chdir
> ((@
> (guile)
> getenv)
> "out"))
> (for-each
> (lambda
> (patch)
> (invoke "patch" "-p1" "--batch" "-i" patch))
> (quote
> ("/gnu/store/iq7hd3f9kr2kvz7lvnygq51yxp85gxbn-jami-libjami-headers-search.patch"))))

That’s a bug we should fix (and indeed, ‘patched-source’ has an XXX
comment about it…).

Ludo’.
?