[PATCH 0/2] Unify 'computed-origin-method' (linux, icecat)

DoneSubmitted by zimoun.
Details
4 participants
  • Liliana Marie Prikler
  • Ludovic Courtès
  • Mark H Weaver
  • zimoun
Owner
unassigned
Severity
normal
Z
Z
zimoun wrote on 16 Sep 13:45 +0200
(address . guix-patches@gnu.org)(name . zimoun)(address . zimon.toutoune@gmail.com)
20210916114505.2686370-1-zimon.toutoune@gmail.com
Hi,
This patch follows the discussion from [0]. In short, it simplifies the codegenerating the file 'sources.json' used by Software Heritage to ingest all thetarballs. It would partially fix the issue raised here [1].
The first patch moves the duplicate of 'computed-origin-method' and addscomments. Please proof-read. :-)
The second patch update the package guix. Since I cannot know the commit hashand checksum beforehand, it is set to xxxx and should be updated by thecommiter. This update is necessary to get the correct Guix-as-library used bythe website generator of 'sources.json', IIUC.
All the best,simon
0: http://issues.guix.gnu.org/50515#51: https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00050.html
zimoun (2): guix: packages: Document 'computed-origin-method'. gnu: guix: Update to xxxx.
gnu/packages/gnuzilla.scm | 14 ++------------ gnu/packages/linux.scm | 14 ++------------ gnu/packages/package-management.scm | 6 +++--- guix/packages.scm | 23 ++++++++++++++++++++++- 4 files changed, 29 insertions(+), 28 deletions(-)

base-commit: 33bc3fb2a5f30a6e21f1b8d6d43867d921bd951c-- 2.32.0
Z
Z
zimoun wrote on 16 Sep 13:47 +0200
[PATCH 2/2] gnu: guix: Update to xxxx.
(address . 50620@debbugs.gnu.org)(name . zimoun)(address . zimon.toutoune@gmail.com)
20210916114734.2686426-2-zimon.toutoune@gmail.com
* gnu/packages/package-management.scm (guix): Update to xxxx.--- gnu/packages/package-management.scm | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
Toggle diff (26 lines)diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scmindex 2611951a52..458e121ef2 100644--- a/gnu/packages/package-management.scm+++ b/gnu/packages/package-management.scm@@ -137,8 +137,8 @@ ;; Note: the 'update-guix-package.scm' script expects this definition to ;; start precisely like this. (let ((version "1.3.0")- (commit "6243ad3812f8c689599a19f0e8b9719ba14461f2")- (revision 5))+ (commit "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")+ (revision 6)) (package (name "guix") @@ -154,7 +154,7 @@ (commit commit))) (sha256 (base32- "0i3sgk2w2yjy9ip47vk0h17afk16yl5ih3p3q75083kgjzyjdm3d"))+ "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx")) (file-name (string-append "guix-" version "-checkout")))) (build-system gnu-build-system) (arguments-- 2.32.0
Z
Z
zimoun wrote on 16 Sep 13:47 +0200
[PATCH 1/2] guix: packages: Document 'computed-origin-method'.
(address . 50620@debbugs.gnu.org)(name . zimoun)(address . zimon.toutoune@gmail.com)
20210916114734.2686426-1-zimon.toutoune@gmail.com
The 'computed-origin-method' had been introduced as a workaround limitationsin the 'snippet' mechanism. The procedure is duplicated which makes hard toautomatically detects packages using it.
* guix/packages.scm (computed-origin-method): Move procedure from...* gnu/packages/gnuzilla.scm: ...here and...* gnu/packages/gnuzilla.scm: ...there.--- gnu/packages/gnuzilla.scm | 14 ++------------ gnu/packages/linux.scm | 14 ++------------ guix/packages.scm | 23 ++++++++++++++++++++++- 3 files changed, 26 insertions(+), 25 deletions(-)
Toggle diff (92 lines)diff --git a/gnu/packages/gnuzilla.scm b/gnu/packages/gnuzilla.scmindex 431b487fd0..9f6e1f24e1 100644--- a/gnu/packages/gnuzilla.scm+++ b/gnu/packages/gnuzilla.scm@@ -682,18 +682,8 @@ in C/C++.") ("1j6l66v1xw27z8w78mpsnmqgv8m277mf4r0hgqcrb4zx7xc2vqyy" "527e5e090608" "zh-CN") ("1frwx35klpyz3sdwrkz7945ivb2dwaawhhyfnz4092h9hn7rc4ky" "6cd366ad2947" "zh-TW"))) -(define* (computed-origin-method gexp-promise hash-algo hash- #:optional (name "source")- #:key (system (%current-system))- (guile (default-guile)))- "Return a derivation that executes the G-expression that results-from forcing GEXP-PROMISE."- (mlet %store-monad ((guile (package->derivation guile system)))- (gexp->derivation (or name "computed-origin")- (force gexp-promise)- #:graft? #f ;nothing to graft- #:system system- #:guile-for-build guile)))+;; XXXX: Workaround 'snippet' limitations.+(define computed-origin-method (@@ (guix packages) computed-origin-method)) (define %icecat-version "78.14.0-guix0-preview1") (define %icecat-build-id "20210907000000") ;must be of the form YYYYMMDDhhmmssdiff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scmindex 285eb132f4..eb792be9a3 100644--- a/gnu/packages/linux.scm+++ b/gnu/packages/linux.scm@@ -216,18 +216,8 @@ defconfig. Return the appropriate make target if applicable, otherwise return (file-name (string-append "linux-libre-deblob-check-" version "-" gnu-revision)) (sha256 deblob-check-hash)))) -(define* (computed-origin-method gexp-promise hash-algo hash- #:optional (name "source")- #:key (system (%current-system))- (guile (default-guile)))- "Return a derivation that executes the G-expression that results-from forcing GEXP-PROMISE."- (mlet %store-monad ((guile (package->derivation guile system)))- (gexp->derivation (or name "computed-origin")- (force gexp-promise)- #:graft? #f ;nothing to graft- #:system system- #:guile-for-build guile)))+;; XXXX: Workaround 'snippet' limitations+(define computed-origin-method (@@ (guix packages) computed-origin-method)) (define (make-linux-libre-source version upstream-sourcediff --git a/guix/packages.scm b/guix/packages.scmindex ad7937b4fb..8c3a0b0b7b 100644--- a/guix/packages.scm+++ b/guix/packages.scm@@ -1,6 +1,6 @@ ;;; GNU Guix --- Functional package management for GNU ;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021 Ludovic Courtès <ludo@gnu.org>-;;; Copyright © 2014, 2015, 2017, 2018 Mark H Weaver <mhw@netris.org>+;;; Copyright © 2014, 2015, 2017, 2018, 2019 Mark H Weaver <mhw@netris.org> ;;; Copyright © 2015 Eric Bavier <bavier@member.fsf.org> ;;; Copyright © 2016 Alex Kost <alezost@gmail.com> ;;; Copyright © 2017, 2019, 2020 Efraim Flashner <efraim@flashner.co.il>@@ -344,6 +344,27 @@ name of its URI." ;; git, svn, cvs, etc. reference #f)))) +;; Work around limitations in the 'snippet' mechanism. It is not possible for+;; a 'snippet' to produce a tarball with a different base name than the+;; original downloaded source. Moreover, cherry picking dozens of upsteam+;; patches and applying them suddenly is often impractical; especially when a+;; comprehensive code reformatting is done upstream. Mainly designed for+;; Linux and IceCat packages.+;; XXXX: do not make part of public API (export) such radical capability+;; before a detailed review process.+(define* (computed-origin-method gexp-promise hash-algo hash+ #:optional (name "source")+ #:key (system (%current-system))+ (guile (default-guile)))+ "Return a derivation that executes the G-expression that results+from forcing GEXP-PROMISE."+ (mlet %store-monad ((guile (package->derivation guile system)))+ (gexp->derivation (or name "computed-origin")+ (force gexp-promise)+ #:graft? #f ;nothing to graft+ #:system system+ #:guile-for-build guile)))+ (define %supported-systems ;; This is the list of system types that are supported. By default, we-- 2.32.0
L
L
Liliana Marie Prikler wrote on 16 Sep 17:53 +0200
9b6ee27ff10e1042a5d61d0f93d957cf760e9ecb.camel@gmail.com
Hi,
Am Donnerstag, den 16.09.2021, 13:47 +0200 schrieb zimoun:
Toggle quote (121 lines)> The 'computed-origin-method' had been introduced as a workaround> limitations in the 'snippet' mechanism. The procedure is duplicated> which makes hard to automatically detects packages using it.> > * guix/packages.scm (computed-origin-method): Move procedure from...> * gnu/packages/gnuzilla.scm: ...here and...> * gnu/packages/gnuzilla.scm: ...there.> ---> gnu/packages/gnuzilla.scm | 14 ++------------> gnu/packages/linux.scm | 14 ++------------> guix/packages.scm | 23 ++++++++++++++++++++++-> 3 files changed, 26 insertions(+), 25 deletions(-)> > diff --git a/gnu/packages/gnuzilla.scm b/gnu/packages/gnuzilla.scm> index 431b487fd0..9f6e1f24e1 100644> --- a/gnu/packages/gnuzilla.scm> +++ b/gnu/packages/gnuzilla.scm> @@ -682,18 +682,8 @@ in C/C++.")> ("1j6l66v1xw27z8w78mpsnmqgv8m277mf4r0hgqcrb4zx7xc2vqyy"> "527e5e090608" "zh-CN")> ("1frwx35klpyz3sdwrkz7945ivb2dwaawhhyfnz4092h9hn7rc4ky"> "6cd366ad2947" "zh-TW")))> > -(define* (computed-origin-method gexp-promise hash-algo hash> - #:optional (name "source")> - #:key (system (%current-system))> - (guile (default-guile)))> - "Return a derivation that executes the G-expression that results> -from forcing GEXP-PROMISE."> - (mlet %store-monad ((guile (package->derivation guile system)))> - (gexp->derivation (or name "computed-origin")> - (force gexp-promise)> - #:graft? #f ;nothing to graft> - #:system system> - #:guile-for-build guile)))> +;; XXXX: Workaround 'snippet' limitations.> +(define computed-origin-method (@@ (guix packages) computed-origin-> method))> > (define %icecat-version "78.14.0-guix0-preview1")> (define %icecat-build-id "20210907000000") ;must be of the form> YYYYMMDDhhmmss> diff --git a/gnu/packages/linux.scm b/gnu/packages/linux.scm> index 285eb132f4..eb792be9a3 100644> --- a/gnu/packages/linux.scm> +++ b/gnu/packages/linux.scm> @@ -216,18 +216,8 @@ defconfig. Return the appropriate make target> if applicable, otherwise return> (file-name (string-append "linux-libre-deblob-check-"> version "-" gnu-revision))> (sha256 deblob-check-hash))))> > -(define* (computed-origin-method gexp-promise hash-algo hash> - #:optional (name "source")> - #:key (system (%current-system))> - (guile (default-guile)))> - "Return a derivation that executes the G-expression that results> -from forcing GEXP-PROMISE."> - (mlet %store-monad ((guile (package->derivation guile system)))> - (gexp->derivation (or name "computed-origin")> - (force gexp-promise)> - #:graft? #f ;nothing to graft> - #:system system> - #:guile-for-build guile)))> +;; XXXX: Workaround 'snippet' limitations> +(define computed-origin-method (@@ (guix packages) computed-origin-> method))> > (define (make-linux-libre-source version> upstream-source> diff --git a/guix/packages.scm b/guix/packages.scm> index ad7937b4fb..8c3a0b0b7b 100644> --- a/guix/packages.scm> +++ b/guix/packages.scm> @@ -1,6 +1,6 @@> ;;; GNU Guix --- Functional package management for GNU> ;;; Copyright © 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019,> 2020, 2021 Ludovic Courtès <ludo@gnu.org>> -;;; Copyright © 2014, 2015, 2017, 2018 Mark H Weaver <mhw@netris.org> >> +;;; Copyright © 2014, 2015, 2017, 2018, 2019 Mark H Weaver <> mhw@netris.org>> ;;; Copyright © 2015 Eric Bavier <bavier@member.fsf.org>> ;;; Copyright © 2016 Alex Kost <alezost@gmail.com>> ;;; Copyright © 2017, 2019, 2020 Efraim Flashner <> efraim@flashner.co.il>> @@ -344,6 +344,27 @@ name of its URI."> ;; git, svn, cvs, etc. reference> #f))))> > +;; Work around limitations in the 'snippet' mechanism. It is not> possible for> +;; a 'snippet' to produce a tarball with a different base name than> the> +;; original downloaded source. Moreover, cherry picking dozens of> upsteam> +;; patches and applying them suddenly is often impractical;> especially when a> +;; comprehensive code reformatting is done upstream. Mainly> designed for> +;; Linux and IceCat packages.> +;; XXXX: do not make part of public API (export) such radical> capability> +;; before a detailed review process.> +(define* (computed-origin-method gexp-promise hash-algo hash> + #:optional (name "source")> + #:key (system (%current-system))> + (guile (default-guile)))> + "Return a derivation that executes the G-expression that results> +from forcing GEXP-PROMISE."> + (mlet %store-monad ((guile (package->derivation guile system)))> + (gexp->derivation (or name "computed-origin")> + (force gexp-promise)> + #:graft? #f ;nothing to graft> + #:system system> + #:guile-for-build guile)))> +> > (define %supported-systems> ;; This is the list of system types that are supported. By> default, we
I think that rather than putting this into (guix packages) itself, wemight want to put it into its own file like (guix computed-origins) andchoose a method name that is actually a verb, similar to git-fetch orsvn-fetch. Perhaps simply call it compute-origin?
If done this way, there'd be the benefit that modules with packagesusing this thing would have to explicitly request the presence of thesymbol through their use-modules clauses. WDYT?
M
M
Mark H Weaver wrote on 17 Sep 01:38 +0200
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
87v930ay5y.fsf@netris.org
Hi Liliana,
Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:
Toggle quote (5 lines)> I think that rather than putting this into (guix packages) itself, we> might want to put it into its own file like (guix computed-origins) and> choose a method name that is actually a verb, similar to git-fetch or> svn-fetch. Perhaps simply call it compute-origin?
These suggestions sound fine to me, although I don't have a strongopinion either way. I'm happy to leave these details to others todecide.
Toggle quote (4 lines)> If done this way, there'd be the benefit that modules with packages> using this thing would have to explicitly request the presence of the> symbol through their use-modules clauses.
Actually, for better or worse, Guile's '@@' form does not require thenamed module to be imported using 'use-modules', so I don't think thisbenefit strictly exists as stated above. However, I agree that it'sgood practice to list all imported modules in the '#:use-module' clausesat the top of the file wherever possible [*], and that there may be somebenefit in declaring the use of 'computed-origins' at the top of eachfile.
Thanks, Mark
[*] It's not always possible in the presence of cyclic module dependencies.
-- Disinformation flourishes because many people care deeply about injusticebut very few check the facts. Ask me about https://stallmansupport.org.
Z
Z
zimoun wrote on 17 Sep 10:41 +0200
(name . Mark H Weaver)(address . mhw@netris.org)
CAJ3okZ2LVCLTF7TmkmSk3TrJnjYy9YODve19k8MqCf7-pEGn1Q@mail.gmail.com
Hi Liliana and Mark,
Thanks for the comments.
On Fri, 17 Sept 2021 at 01:40, Mark H Weaver <mhw@netris.org> wrote:
Toggle quote (11 lines)> Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:
> > I think that rather than putting this into (guix packages) itself, we> > might want to put it into its own file like (guix computed-origins) and> > choose a method name that is actually a verb, similar to git-fetch or> > svn-fetch. Perhaps simply call it compute-origin?>> These suggestions sound fine to me, although I don't have a strong> opinion either way. I'm happy to leave these details to others to> decide.
I do not have a strong opinion either. If no one comes with a bettername, I will pick the suggested one. :-)
There are only two hard things in Computer Science: cache invalidation and naming things. -- Internet

Toggle quote (12 lines)> > If done this way, there'd be the benefit that modules with packages> > using this thing would have to explicitly request the presence of the> > symbol through their use-modules clauses.>> Actually, for better or worse, Guile's '@@' form does not require the> named module to be imported using 'use-modules', so I don't think this> benefit strictly exists as stated above. However, I agree that it's> good practice to list all imported modules in the '#:use-module' clauses> at the top of the file wherever possible [*], and that there may be some> benefit in declaring the use of 'computed-origins' at the top of each> file.
I am not deeply familiar with Guile module.
I chose to put this in (guix packages) instead of its own modulebecause the module would contain only one function and nothingexported. The aim for now, as discussed, is to not make this 'method'part of the public API.
Then if the function is not exported by the module, the '#:use-module'does not have an effect, right?
In this case, does it make sense to put this in its own module?
Initially, I put the '@@' right after the '#:use-module's but then Ichanged my mind; I do not remember why. Anyway, yeah it is better atthe top.

Cheers,simon
M
M
Mark H Weaver wrote on 28 Sep 11:36 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
87pmstghx0.fsf@netris.org
Hi Simon,
zimoun <zimon.toutoune@gmail.com> writes:
Toggle quote (2 lines)> On Fri, 17 Sept 2021 at 01:40, Mark H Weaver <mhw@netris.org> wrote:>> Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:
[...]
Toggle quote (22 lines)>> > If done this way, there'd be the benefit that modules with packages>> > using this thing would have to explicitly request the presence of the>> > symbol through their use-modules clauses.>>>> Actually, for better or worse, Guile's '@@' form does not require the>> named module to be imported using 'use-modules', so I don't think this>> benefit strictly exists as stated above. However, I agree that it's>> good practice to list all imported modules in the '#:use-module' clauses>> at the top of the file wherever possible [*], and that there may be some>> benefit in declaring the use of 'computed-origins' at the top of each>> file.>> I am not deeply familiar with Guile module.>> I chose to put this in (guix packages) instead of its own module> because the module would contain only one function and nothing> exported. The aim for now, as discussed, is to not make this 'method'> part of the public API.>> Then if the function is not exported by the module, the '#:use-module'> does not have an effect, right?
It's true that it would have no effect on the set of available bindings,and that's an excellent point. Perhaps Liliana could clarify what shehad in mind, or better yet, propose a patch.
Please don't let me be a blocker on this thread. I contributed a fewthoughts, but I don't have time right now to shepherd this issue, sorry.
Regards, Mark
-- Disinformation flourishes because many people care deeply about injusticebut very few check the facts. Ask me about https://stallmansupport.org.
L
L
Liliana Marie Prikler wrote on 28 Sep 18:01 +0200
(address . 50620@debbugs.gnu.org)
e2a5b36cb941ad715c7a33a2f400af760d19f1d4.camel@gmail.com
Hi everyone,
Am Dienstag, den 28.09.2021, 05:36 -0400 schrieb Mark H Weaver:
Toggle quote (27 lines)> Hi Simon,> > zimoun <zimon.toutoune@gmail.com> writes:> > On Fri, 17 Sept 2021 at 01:40, Mark H Weaver <mhw@netris.org>> > wrote:> > > Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:> [...]> > > > If done this way, there'd be the benefit that modules with> > > > packages> > > > using this thing would have to explicitly request the presence> > > > of the> > > > symbol through their use-modules clauses.> > > > > > Actually, for better or worse, Guile's '@@' form does not require> > > the named module to be imported using 'use-modules', so I don't> > > think this benefit strictly exists as stated above. However, I> > > agree that it's good practice to list all imported modules in> > > the '#:use-module' clauses at the top of the file wherever> > > possible [*], and that there may be somebenefit in declaring the> > > use of 'computed-origins' at the top of each file.> > > > I am not deeply familiar with Guile module.> > > > I chose to put this in (guix packages) instead of its own module> > because the module would contain only one function and nothing> > exported. The aim for now, as discussed, is to not make this> > 'method' part of the public API.
If so, one could argue that (gnu packages) is a better location to hideit, but my main issue is that we still need to hide it! This willcause other channels to refer to it using @@ or roll their ownimplementations.
Toggle quote (6 lines)> > Then if the function is not exported by the module, the '#:use-> > module' does not have an effect, right?> > It's true that it would have no effect on the set of available> bindings, and that's an excellent point. Perhaps Liliana could> clarify what she had in mind, or better yet, propose a patch.
I would argue that something like computed-origin-method *does* deserveto be an exported binding, but ought not to be grouped together into(guix packages). The latter only provides record types, not methods(which are typically implemented elsewhere), and I'd like to keep itthat way.
I've attached a patch to illustrate my point, but please don't apply itas is. I have not put in the necessary git blame research to find outwho would need to be copyrighted here.
Regards,Liliana
From ea138fdb145224a04a2bad27e214df7e283ccee7 Mon Sep 17 00:00:00 2001From: Liliana Marie Prikler <liliana.prikler@gmail.com>Date: Tue, 28 Sep 2021 17:54:23 +0200Subject: [PATCH] guix: Add computed-origins.MIME-Version: 1.0Content-Type: text/plain; charset=UTF-8Content-Transfer-Encoding: 8bit
This adds the ‘computed-origin-method’ used by linux-libre or icecat underthe new name ‘compute-origin’ as public Guix API.
* guix/computed-origins: New file.--- Makefile.am | 1 + guix/computed-origins.scm | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 38 insertions(+) create mode 100644 guix/computed-origins.scm
Toggle diff (57 lines)diff --git a/Makefile.am b/Makefile.amindex b66789fa0b..e8f0c63e2b 100644--- a/Makefile.am+++ b/Makefile.am@@ -98,6 +98,7 @@ MODULES = \ guix/bzr-download.scm \ guix/git-download.scm \ guix/hg-download.scm \+ guix/computed-origins.scm \ guix/swh.scm \ guix/monads.scm \ guix/monad-repl.scm \diff --git a/guix/computed-origins.scm b/guix/computed-origins.scmnew file mode 100644index 0000000000..f7c83df0bf--- /dev/null+++ b/guix/computed-origins.scm@@ -0,0 +1,37 @@+;;; GNU Guix --- Functional package management for GNU+;;; Copyright © 1312 Max Müller <max@müller.gmbh>+;;;+;;; This file is part of GNU Guix.+;;;+;;; GNU Guix is free software; you can redistribute it and/or modify it+;;; under the terms of the GNU General Public License as published by+;;; the Free Software Foundation; either version 3 of the License, or (at+;;; your option) any later version.+;;;+;;; GNU Guix is distributed in the hope that it will be useful, but+;;; WITHOUT ANY WARRANTY; without even the implied warranty of+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the+;;; GNU General Public License for more details.+;;;+;;; You should have received a copy of the GNU General Public License+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.++(define-module (guix computed-origins)+ #:use-module (guix monads)+ #:use-module (guix store)+ #:use-module (guix packages)+ #:use-module (guix gexp)+ #:export (compute-origin))++(define* (compute-origin gexp-promise hash-algo hash+ #:optional (name "source")+ #:key (system (%current-system))+ (guile (default-guile)))+ "Return a derivation that executes the G-expression that results+from forcing GEXP-PROMISE."+ (mlet %store-monad ((guile (package->derivation guile system)))+ (gexp->derivation (or name "computed-origin")+ (force gexp-promise)+ #:graft? #f ;nothing to graft+ #:system system+ #:guile-for-build guile)))-- 2.33.0
Z
Z
zimoun wrote on 28 Sep 18:37 +0200
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
CAJ3okZ0HqfWqjirF-=U87yD1wtxUwOwOSYe3xtQ-BSHWtNy+=A@mail.gmail.com
Hi,
On Tue, 28 Sept 2021 at 18:01, Liliana Marie Prikler<liliana.prikler@gmail.com> wrote:
Toggle quote (9 lines)> > zimoun <zimon.toutoune@gmail.com> writes:
> > > I chose to put this in (guix packages) instead of its own module> > > because the module would contain only one function and nothing> > > exported. The aim for now, as discussed, is to not make this> > > 'method' part of the public API.
> If so, one could argue that (gnu packages) is a better location to hide
Ok. I do not find it better than (guix packages) where 'origin' isdefined but anyway.I will send a v2 considering this and the rename you proposed.
Toggle quote (4 lines)> it, but my main issue is that we still need to hide it! This will> cause other channels to refer to it using @@ or roll their own> implementations.
This patch is not about discussing if this method should be public ornot. It is private. Please discuss that elsewhere.
Mark commented in [0]:
Toggle snippet (5 lines)The reason 'computed-origin-method' is not exported is because itnever went through the review process that such a radical new capabilityin Guix should go through before becoming part of it's public API.
and this patch is about improving the situation (by removing the codeduplication). That's all. The aim of this improvement is related tosaving these IceCat and Linux Libre packages by Software Heritage [1].
0: http://issues.guix.gnu.org/50515#31: http://issues.guix.gnu.org/50515
Toggle quote (4 lines)> I've attached a patch to illustrate my point, but please don't apply it> as is. I have not put in the necessary git blame research to find out> who would need to be copyrighted here.
As I said, the point of my patch is not to discuss if this'compute-origin' should be part or not to the public API. It issimply a cleanup to ease the patch#50515 [1]. Therefore, I do not seewhat is the point to create its own module.
All the best,simon
L
L
Liliana Marie Prikler wrote on 28 Sep 19:24 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
1803ff0456849f456c6994d47cbe50d1a8ff6a09.camel@gmail.com
Hi,
Am Dienstag, den 28.09.2021, 18:37 +0200 schrieb zimoun:
Toggle quote (16 lines)> Hi,> > On Tue, 28 Sept 2021 at 18:01, Liliana Marie Prikler> <liliana.prikler@gmail.com> wrote:> > > zimoun <zimon.toutoune@gmail.com> writes:> > > > I chose to put this in (guix packages) instead of its own> > > > module> > > > because the module would contain only one function and nothing> > > > exported. The aim for now, as discussed, is to not make this> > > > 'method' part of the public API.> > If so, one could argue that (gnu packages) is a better location to> > hide> > Ok. I do not find it better than (guix packages) where 'origin' is> defined but anyway.> I will send a v2 considering this and the rename you proposed.
By that logic all of (guix git-download), (guix svn-download), etc.could be inlined there as well. Obviously that's a bad idea, but *why*is it a bad idea? I'd argue it's because we have a clear separation ofthe record descriptor for an origin and the ways it can be computed(the former in (guix packages), the latter elsewhere) and that it'sgood to keep those concerns separate.
I also personally find the name "computed-origin" to be somewhat weirdnaming choice. I could just as well write the entire source code forsome given package in the snippet part of an origin, perhaps applyingsome weird tricks in the category of Kolmogorov code golf – would thatorigin not be computed?
Toggle quote (20 lines)> > it, but my main issue is that we still need to hide it! This will> > cause other channels to refer to it using @@ or roll their own> > implementations.> > This patch is not about discussing if this method should be public or> not. It is private. Please discuss that elsewhere.> > Mark commented in [0]:> > --8<---------------cut here---------------start------------->8---> The reason 'computed-origin-method' is not exported is because it> never went through the review process that such a radical new> capability in Guix should go through before becoming part of it's> public API.> --8<---------------cut here---------------end--------------->8---> > and this patch is about improving the situation (by removing the code> duplication). That's all. The aim of this improvement is related to> saving these IceCat and Linux Libre packages by Software Heritage> [1].
I don't think delaying this review is a good idea, though. When you'reremoving code duplication, you ought to do it in a way that allduplicated code can indeed be removed, at least in my opinion. As-isthis patch just invites practises otherwise discouraged by Guix.
Cheers
Z
Z
zimoun wrote on 29 Sep 10:32 +0200
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
CAJ3okZ1uoKr1m21tWpZP0biyE5vhqtjogHgqwBVoZNWmm+MY1Q@mail.gmail.com
Hi,
On Tue, 28 Sept 2021 at 19:24, Liliana Marie Prikler<liliana.prikler@gmail.com> wrote:
Toggle quote (17 lines)> > Ok. I do not find it better than (guix packages) where 'origin' is> > defined but anyway.> > I will send a v2 considering this and the rename you proposed.>> By that logic all of (guix git-download), (guix svn-download), etc.> could be inlined there as well. Obviously that's a bad idea, but *why*> is it a bad idea? I'd argue it's because we have a clear separation of> the record descriptor for an origin and the ways it can be computed> (the former in (guix packages), the latter elsewhere) and that it's> good to keep those concerns separate.>> I also personally find the name "computed-origin" to be somewhat weird> naming choice. I could just as well write the entire source code for> some given package in the snippet part of an origin, perhaps applying> some weird tricks in the category of Kolmogorov code golf – would that> origin not be computed?
Are we bikeshedding here? ;-)
Again, the aim of this patch it not to fix the'computed-origin-method'. The aim of this patch is to improve thereadibility of the patch#50515 [1] which allows linux-libre and icecatto be ingested by SWH from 'guix.gnu.org/sources.json'.
Maybe there is an original issue with 'computed-origin-method', asMark explained [0]. But that's another story than the SWH andsources.json one!
Toggle snippet (15 lines)At the time that I added 'computed-origin-method', I was under timepressure to push security updates for IceCat, and my previous method ofcherry picking dozens of upsteam patches and applying them to the mostrecent IceCat release suddenly became impractical due to comprehensivecode reformatting done upstream.
I've always viewed 'computed-origin-method' as a temporary hack to workaround limitations in the 'snippet' mechanism. Most importantly, last Ichecked, it was not possible for a 'snippet' to produce a tarball with adifferent base name than the original downloaded source. I consider ita *requirement* for the 'icecat' source tarball and it's unpackeddirectory to be named "icecat-…" and not "firefox-…", and similarlyfor'linux-libre'.
Toggle quote (26 lines)> > > it, but my main issue is that we still need to hide it! This will> > > cause other channels to refer to it using @@ or roll their own> > > implementations.> >> > This patch is not about discussing if this method should be public or> > not. It is private. Please discuss that elsewhere.> >> > Mark commented in [0]:> >> > --8<---------------cut here---------------start------------->8---> > The reason 'computed-origin-method' is not exported is because it> > never went through the review process that such a radical new> > capability in Guix should go through before becoming part of it's> > public API.> > --8<---------------cut here---------------end--------------->8---> >> > and this patch is about improving the situation (by removing the code> > duplication). That's all. The aim of this improvement is related to> > saving these IceCat and Linux Libre packages by Software Heritage> > [1].>> I don't think delaying this review is a good idea, though. When you're> removing code duplication, you ought to do it in a way that all> duplicated code can indeed be removed, at least in my opinion. As-is> this patch just invites practises otherwise discouraged by Guix.
If you go this road, please push patch#50515 [1] as-is. It perfectlyworks and fixes the issue with 'guix.gnu.org/sources.json' and SWH.This current patch#50620 is a way to improve the readibility ofpatch#50515 but then reading all this discussion I miss whypatch#50620 is thus a blocker for patch#50515. Because I feel we areentering into the famous "Bigger problem" from xkcd. ;-)
Patch#50515 is the first part of a concrete answer tohttps://lists.gnu.org/archive/html/guix-devel/2021-09/msg00106.htmland https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00290.html.It is discussed to use SWH for such situations but today ourlinux-libre is not ingested by SWH. Therefore, let first ingestit--which does patch#50515.
All the best,simon
PS: I disagree with your last statement. Because I am in favour forincremental improvements and not "The Right Thing or nothing". That'sout of scope of the patch at hand. ;-)
L
L
Liliana Marie Prikler wrote on 29 Sep 12:10 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
56dcce10a751153d89f515028cd18c9125f6b84f.camel@gmail.com
Hi,
Am Mittwoch, den 29.09.2021, 10:32 +0200 schrieb zimoun:
Toggle quote (33 lines)> Hi,> > On Tue, 28 Sept 2021 at 19:24, Liliana Marie Prikler> <liliana.prikler@gmail.com> wrote:> > > > Ok. I do not find it better than (guix packages) where 'origin'> > > is> > > defined but anyway.> > > I will send a v2 considering this and the rename you proposed.> > > > By that logic all of (guix git-download), (guix svn-download), etc.> > could be inlined there as well. Obviously that's a bad idea, but> > *why* is it a bad idea? I'd argue it's because we have a clear> > separation of the record descriptor for an origin and the ways it> > can be computed (the former in (guix packages), the latter> > elsewhere) and that it's good to keep those concerns separate.> > > > I also personally find the name "computed-origin" to be somewhat> > weird naming choice. I could just as well write the entire source> > code for some given package in the snippet part of an origin,> > perhaps applying some weird tricks in the category of Kolmogorov> > code golf – would that origin not be computed?> > Are we bikeshedding here? ;-)> > Again, the aim of this patch it not to fix the> 'computed-origin-method'. The aim of this patch is to improve the> readibility of the patch#50515 [1] which allows linux-libre and> icecat to be ingested by SWH from 'guix.gnu.org/sources.json'.> > Maybe there is an original issue with 'computed-origin-method', as> Mark explained [0]. But that's another story than the SWH and> sources.json one!
Perhaps we're bikeshedding, but you started to shed the bike when youchose to move this procedure to (guix packages) rather thanimplementing one of Mark's suggestions in [0]. I do think we shouldallow for bikeshedding comments from both sides if that is the case.
Toggle quote (39 lines)> [...]> > > > > it, but my main issue is that we still need to hide it! This> > > > will> > > > cause other channels to refer to it using @@ or roll their own> > > > implementations.> > > > > > This patch is not about discussing if this method should be> > > public or> > > not. It is private. Please discuss that elsewhere.> > > > > > Mark commented in [0]:> > > > > > --8<---------------cut here---------------start------------->8---> > > The reason 'computed-origin-method' is not exported is because it> > > never went through the review process that such a radical new> > > capability in Guix should go through before becoming part of it's> > > public API.> > > --8<---------------cut here---------------end--------------->8---> > > > > > and this patch is about improving the situation (by removing the> > > code> > > duplication). That's all. The aim of this improvement is> > > related to> > > saving these IceCat and Linux Libre packages by Software Heritage> > > [1].> > > > I don't think delaying this review is a good idea, though. When> > you're removing code duplication, you ought to do it in a way that> > all duplicated code can indeed be removed, at least in my> > opinion. As-is this patch just invites practises otherwise> > discouraged by Guix.> > If you go this road, please push patch#50515 [1] as-is. It perfectly> works and fixes the issue with 'guix.gnu.org/sources.json' and SWH.> This current patch#50620 is a way to improve the readibility of> patch#50515 but then reading all this discussion I miss why> patch#50620 is thus a blocker for patch#50515. Because I feel we are> entering into the famous "Bigger problem" from xkcd. ;-)
I don't think #50515 is "perfect as-is", though. Mark's (1) suggestionwas to put computed-origin-method into its own module, which is thesame as my long-term position. Mark suggested a short-term fix (2) ofstill comparing by eq?, which I believe you dismissed for the wrongreasons. Yes, you would still need to check the promise, but youwouldn't invert said check, i.e. you would still first look for themethod and then assert that the uri makes sense. I think it is saferto err on the side of conservatism here rather than claim that thiscode will work for future computed-origin-esque methods.
Instead of doing either (1) or (2), you opted for your own solution(3), which is to put this method into (guix packages). In my personalopinion, this is a half-baked solution, because it puts computed-origin-method into the (guix ...) without addressing any of the morefundamental issues raised by Mark. If you really, really can't putthis into its own module, then I'd at least suggest (3a), which is toput it into (gnu packages) instead. That way, the definition is atleast closer to where it's used and it's clearer that it's a hack topackage certain things, not a core part of Guix. Perhaps you can evenmake use of it without needing to rebuild the guix package in [2/2],but don't quote me on that.
Toggle quote (15 lines)> Patch#50515 is the first part of a concrete answer to> <https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00106.html>> and <> https://lists.gnu.org/archive/html/guix-devel/2021-09/msg00290.html>;> .> It is discussed to use SWH for such situations but today our> linux-libre is not ingested by SWH. Therefore, let first ingest> it--which does patch#50515.> > All the best,> simon> > PS: I disagree with your last statement. Because I am in favour for> incremental improvements and not "The Right Thing or> nothing". That's out of scope of the patch at hand. ;-)
Perhaps you are right in that we can't wait for a lengthy discussion ofwhether computed-origin-method can be public (guix) API or not. Eitherway, it does not look as though this thread attracts enough attentionto that issue, which is why we can ignore Mark's option (1) for now.
For the right amount of incremental change, I'd suggest the following:Try to see whether you can do with computed-origin-method in (gnupackages) and without rebuilding the guix package, and if so, submit av2 to #50515, which implements that. Otherwise, submit a v2 to #50515that implements Mark's option (2).
If you are also interested in the more long-term discussion of allowingcomputed origins into the (guix) tree, I suggest sending a v2 patch tothis thread, which creates a new module, adds documentation, and so on,and so forth, and also link to it on guix-devel. For the time being,this v2 should also refrain from touching anything that uses thecurrent computed-origin-method, as that can be addressed in rathersimple follow-up commits, particularly if we also implement a #50515-v2 before. Then we can fully use this to bikeshed about making it a verband what not.
WDYT?
L
L
Ludovic Courtès wrote on 29 Sep 15:16 +0200
Re: bug#50620: [PATCH 0/2] Unify 'computed-origin-method' (linux, icecat)
(name . Mark H Weaver)(address . mhw@netris.org)
87a6jv8qu7.fsf_-_@gnu.org
Hi there!
I’d rather go with zimoun’s original patch, which is focused and doesnothing more than what was originally intended, which is to factorizethe procedure. I’ll go ahead and apply it shortly if there are noobjections.

As Mark wrote, ‘computed-origin-method’ remains a hack, so we candiscuss about the best way to improve on it, but that’s a separatediscussion. What you propose, Liliana, is one possible way to improveon the situation, but only on the surface: the hack remains, it justgets its own module.
A better solution IMO would be to improve the ‘snippet’ mechanism in thefirst place. ‘computed-origin-method’ improves on it in two ways: (1)lazy evaluation of the gexp, and (2) allows the use of a different basename.
I would think #2 is addressed by the ‘file-name’ field (isn’t it?).
As for #1, it can be addressed by making the ‘snippet’ field delayed orthunked. It’s a one line change; the only thing we need is to measure,or attempt to measure, the impact it has on module load time.
Thoughts?
Ludo’.
Z
Z
zimoun wrote on 29 Sep 15:17 +0200
Re: [PATCH 1/2] guix: packages: Document 'computed-origin-method'.
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
CAJ3okZ2Po4ZHMr0Aa=5ProJYbm_USXvrxEG_U21H0DzhA5QXOQ@mail.gmail.com
Hi,
On Wed, 29 Sept 2021 at 12:10, Liliana Marie Prikler<liliana.prikler@gmail.com> wrote:
Toggle quote (5 lines)> Perhaps we're bikeshedding, but you started to shed the bike when you> chose to move this procedure to (guix packages) rather than> implementing one of Mark's suggestions in [0]. I do think we should> allow for bikeshedding comments from both sides if that is the case.
I do not have time to bikeshed. :-) (Even if I like to practise it. ;-))
(Note that Mark agreed on my proposal as a variant of one of theirsuggestions [0].)
0: http://issues.guix.gnu.org/50515#5

Toggle quote (10 lines)> I don't think #50515 is "perfect as-is", though. Mark's (1) suggestion> was to put computed-origin-method into its own module, which is the> same as my long-term position. Mark suggested a short-term fix (2) of> still comparing by eq?, which I believe you dismissed for the wrong> reasons. Yes, you would still need to check the promise, but you> wouldn't invert said check, i.e. you would still first look for the> method and then assert that the uri makes sense. I think it is safer> to err on the side of conservatism here rather than claim that this> code will work for future computed-origin-esque methods.
Maybe. Well, I commented there [1], reworded here:
The option (1) with its own module means make computed-origin-methodpublic which implies a lengthy discussion, therefore it is not anoption for me. We agree an option (1), I guess. ;-) From my point ofview, the long-term solution is to improve snippet and remove thiscomputed-origin-method; not make it public.
Perhaps I am wrong about option (2) -- my claim is thatcomputed-origin-method is *always* used with a promise so it is forsure an half-baked guess but enough; and it avoids to hard code themodules from where the packages come from. Therefore, option (2) doesnot improve, IMHO.
For sure, I am right about this [1]:
Toggle snippet (5 lines)However, I would not like that the sources.json situation stays blockedby the computed-origin-method situation when sources.json is produced bythe website independently of Guix, somehow. :-)
because it is exactly what it is: blocked by thecomputed-origin-method situation.
1: http://issues.guix.gnu.org/50515#4

Toggle quote (12 lines)> Instead of doing either (1) or (2), you opted for your own solution> (3), which is to put this method into (guix packages). In my personal> opinion, this is a half-baked solution, because it puts computed-> origin-method into the (guix ...) without addressing any of the more> fundamental issues raised by Mark. If you really, really can't put> this into its own module, then I'd at least suggest (3a), which is to> put it into (gnu packages) instead. That way, the definition is at> least closer to where it's used and it's clearer that it's a hack to> package certain things, not a core part of Guix. Perhaps you can even> make use of it without needing to rebuild the guix package in [2/2],> but don't quote me on that.
All the solutions are half-baked! :-) Also, generating thissources.json using the website is half-backed at first. ;-)
Options (1) and (2) are more half-baked than my initial submission (3)(guix packages) or (3a) (gnu packages), IMHO.
I still think that (guix packages) is better than (gnu packages).Maintainers, what do you think?
About update guix package [2/2], it has to be done, IIUC. The filesources.json contains what the package guix provides, not what thecurrent Guix has. The website -- built using the Guile tool haunt --uses Guix as a Guile library. Maybe I miss something.
Toggle quote (4 lines)> For the right amount of incremental change, I'd suggest the following:> Try to see whether you can do with computed-origin-method in (gnu> packages) and without rebuilding the guix package, and if so, submit a
This is what I suggested earlier ;-) [2]: send a v2 moving to '(gnupackages)' instead and rename to 'compute-origin'. Although Idisagree on (gnu packages). :-)
I need explanations if rebuild the guix package is not necessary.
2: http://issues.guix.gnu.org/50620#8
Toggle quote (10 lines)> If you are also interested in the more long-term discussion of allowing> computed origins into the (guix) tree, I suggest sending a v2 patch to> this thread, which creates a new module, adds documentation, and so on,> and so forth, and also link to it on guix-devel. For the time being,> this v2 should also refrain from touching anything that uses the> current computed-origin-method, as that can be addressed in rather> simple follow-up commits, particularly if we also implement a #50515-v2> before. Then we can fully use this to bikeshed about making it a verb> and what not.
For long-term, the road seems to improve the 'snippet' mechanism, notmake computed-origin-method public, IMHO.
All the best,simon
L
L
Liliana Marie Prikler wrote on 29 Sep 16:36 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
756ae01852047a7adc2522c025c8cd7283dc7e55.camel@gmail.com
Hi,
Am Mittwoch, den 29.09.2021, 15:17 +0200 schrieb zimoun:
Toggle quote (18 lines)> Hi,> > On Wed, 29 Sept 2021 at 12:10, Liliana Marie Prikler> <liliana.prikler@gmail.com> wrote:> > > Perhaps we're bikeshedding, but you started to shed the bike when> > you chose to move this procedure to (guix packages) rather than> > implementing one of Mark's suggestions in [0]. I do think we> > should allow for bikeshedding comments from both sides if that is> > the case.> > I do not have time to bikeshed. :-) (Even if I like to practise it.> ;-))> > (Note that Mark agreed on my proposal as a variant of one of their> suggestions [0].)> > 0: <http://issues.guix.gnu.org/50515#5>
While Mark did agree to that, I still think that (guix packages) is thewrong location for this procedure. Multiple reviewers can holddifferent opinions on that matter.
Toggle quote (18 lines)> > I don't think #50515 is "perfect as-is", though. Mark's (1)> > suggestion was to put computed-origin-method into its own module,> > which is the same as my long-term position. Mark suggested a> > short-term fix (2) of still comparing by eq?, which I believe you> > dismissed for the wrong reasons. Yes, you would still need to> > check the promise, but you wouldn't invert said check, i.e. you> > would still first look for the method and then assert that the uri> > makes sense. I think it is safer to err on the side of> > conservatism here rather than claim that this> > code will work for future computed-origin-esque methods.> > Maybe. Well, I commented there [1], reworded here:> > The option (1) with its own module means make computed-origin-method> public which implies a lengthy discussion, therefore it is not an> option for me. We agree an option (1), I guess. ;-) From my point> of view, the long-term solution is to improve snippet and remove this> computed-origin-method; not make it public.
I personally think there are certain cases where it would make sense tocompute origins, but they are not widely applied because computed-origin-method is hidden and clunky. For instance, there are severalpackages, that pull in extra origins as inputs, which could however beargued to be part of the source closure.
Again, there is room to bikeshed far and wide here and all we can agreeto currently is that we need some solution long-term.
Toggle quote (5 lines)> Perhaps I am wrong about option (2) -- my claim is that> computed-origin-method is *always* used with a promise so it is for> sure an half-baked guess but enough; and it avoids to hard code the> modules from where the packages come from. Therefore, option (2)> does not improve, IMHO.
The probability of having a promise when using computed-origin-methodis 100%. What is the probability of having computed-origin-method whenyou see a promise? The answer is: we don't know. We can see from theexisting two copies of computed-origin-method, that they use promises,but you could imagine an origin method that takes a promise only aspart of its input and then transforms it in some way under the hood. You could also imagine a different computed-origin-method that isactually thunked or uses a raw gexp.
Toggle quote (12 lines)> For sure, I am right about this [1]:> > --8<---------------cut here---------------start------------->8---> However, I would not like that the sources.json situation stays> blocked by the computed-origin-method situation when sources.json is> produced by the website independently of Guix, somehow. :-)> --8<---------------cut here---------------end--------------->8---> > because it is exactly what it is: blocked by the> computed-origin-method situation.> > 1: <http://issues.guix.gnu.org/50515#4>
Sure, I agree that we need to divorce these discussions if this one isgoing to take longer – which from the looks of it, it is.
Toggle quote (17 lines)> > Instead of doing either (1) or (2), you opted for your own solution> > (3), which is to put this method into (guix packages). In my> > personal opinion, this is a half-baked solution, because it puts> > computed-origin-method into the (guix ...) without addressing any> > of the more fundamental issues raised by Mark. If you really,> > really can't put this into its own module, then I'd at least> > suggest (3a), which is to put it into (gnu packages) instead. That> > way, the definition is at least closer to where it's used and it's> > clearer that it's a hack to package certain things, not a core part> > of Guix. Perhaps you can even make use of it without needing to> > rebuild the guix package in [2/2], but don't quote me on that.> > All the solutions are half-baked! :-) Also, generating this> sources.json using the website is half-backed at first. ;-)> > Options (1) and (2) are more half-baked than my initial submission> (3) (guix packages) or (3a) (gnu packages), IMHO.
I don't think option (1) is half-baked, but it does entail makingcomputed-origin-method somewhat public API, which would need morecareful review than initially assumed. (2) is half-baked indeed, butbecause it is minimal effort. Instead of touching the modules in whichthe definitions are made, it just references them.
Toggle quote (4 lines)> About update guix package [2/2], it has to be done, IIUC. The file> sources.json contains what the package guix provides, not what the> current Guix has. The website -- built using the Guile tool haunt --> uses Guix as a Guile library. Maybe I miss something.
What I was trying to say was that you wouldn't need to rebuild the guixpackage before applying the 50515 changes, which this one seems torequire. Again, I'm not 100% sure that's the case, but IIUC (gnupackages) is its own realm in this regard.
Toggle quote (12 lines)> > For the right amount of incremental change, I'd suggest the> > following: Try to see whether you can do with computed-origin-> > method in (gnu packages) and without rebuilding the guix package,> > and if so, submit a> > This is what I suggested earlier ;-) [2]: send a v2 moving to '(gnu> packages)' instead and rename to 'compute-origin'. Although I> disagree on (gnu packages). :-)> > I need explanations if rebuild the guix package is not necessary.> > 2: <http://issues.guix.gnu.org/50620#8>
I don't think a rename is necessary if we want a "quick and dirty"version, this suggestion was rather made in the intention that it wouldbe public API. However, if you like the naming choice, feel free toapply it.
Toggle quote (13 lines)> > If you are also interested in the more long-term discussion of> > allowing computed origins into the (guix) tree, I suggest sending a> > v2 patch to this thread, which creates a new module, adds> > documentation, and so on, and so forth, and also link to it on> > guix-devel. For the time being, this v2 should also refrain from> > touching anything that uses the current computed-origin-method, as> > that can be addressed in rather simple follow-up commits,> > particularly if we also implement a #50515-v2> > before. Then we can fully use this to bikeshed about making it a> > verb and what not.> > For long-term, the road seems to improve the 'snippet' mechanism, not> make computed-origin-method public, IMHO.
I do have some opinions on that, but I'll type them out in response toLudo, so as to not repeat myself too often.
Cheers
L
L
Liliana Marie Prikler wrote on 29 Sep 17:34 +0200
Re: bug#50620: [PATCH 0/2] Unify 'computed-origin-method' (linux, icecat)
ccdfce2abdd5c4559de9c83fb1d1c2127e52ab3d.camel@gmail.com
Hi there,
Am Mittwoch, den 29.09.2021, 15:16 +0200 schrieb Ludovic Courtès:
Toggle quote (6 lines)> Hi there!> > I’d rather go with zimoun’s original patch, which is focused and does> nothing more than what was originally intended, which is to factorize> the procedure. I’ll go ahead and apply it shortly if there are no> objections.
I have trouble understanding this paragraph. What exactly is "thispatch" and what do you mean by "factorizing"? If it means movingcomputed-origin-method elsewhere, then yes, for a short-time solutiononly moving it is a wise choice in my opinion, but zimoun and I stilldisagree on the target. zimoun says (guix packages) for reasonsunknown to me, whereas I say (gnu packages), because it's closer towhere it's used and doesn't imply that this is going to be a part ofthe (guix) download schemes anytime soon.
Toggle quote (5 lines)> As Mark wrote, ‘computed-origin-method’ remains a hack, so we can> discuss about the best way to improve on it, but that’s a separate> discussion. What you propose, Liliana, is one possible way to> improve on the situation, but only on the surface: the hack remains,> it just gets its own module.
I don't necessarily perceive computed-origin-method as a hack, though,at least not in concept. The current implementation may be a hackindeed, but I don't think the very concept of expressing an input as afunction is. We are functional programmers after all :)
Toggle quote (13 lines)> A better solution IMO would be to improve the ‘snippet’ mechanism in> the first place. ‘computed-origin-method’ improves on it in two> ways: (1) lazy evaluation of the gexp, and (2) allows the use of a> different base name.> > I would think #2 is addressed by the ‘file-name’ field (isn’t it?).> > As for #1, it can be addressed by making the ‘snippet’ field delayed> or thunked. It’s a one line change; the only thing we need is to> measure, or attempt to measure, the impact it has on module load> time.> > Thoughts?
This would work for packages, whose source are some base source withpatches or snippets applied, as is indeed the case for linux andicecat. However, there are also other potential uses for computedorigins. Grepping for the string "'unpack 'unpack", which indicatesthat more sources are unpacked at build-time returns more than fiftyinstances in a number of locations. This is potentially dangerous, asfor one, we will probably want to phase out the "origins as inputs"idiom once we have short-form inputs and also it could be argued, that`guix build -S` returns something wrong in those packages.
I think that some version of `computed-origin-method' will eventuallyneed to become public API as such packages may not always be bestdescribed as "a base package with a snippet". If we had recursiveorigins – i.e. origins, that can take origins as inputs – we might beable to do some of that, but I don't think it would necessarily workfor linux-libre or icecat, as with those you don't want the taintedversions to be kept around. Perhaps this could be worked around by notinterning the intermediate origins, but only using their file-namesinside the temporary directory in which the snippet is applied?
Another thing is that the final act of the linux-libre promise is notthe packing of the tarball, but the deblob-check. Guix currently lacksa way of modeling such checks in their origin, but I'd argue it wouldneed one if we wanted to do computed origins via snippets. This is notrequired by icecat and so one "simplification" could be that computed-origin-method would not require the user to create a tarball, butinstead simply provide a name for the tarball and a directory to createit from (via a promise again).
A combination of the above might make computed origins obsolete forgood, but the question remains whether that is a better design. Whatdo y'all think?
Z
Z
zimoun wrote on 29 Sep 19:48 +0200
Re: [PATCH 1/2] guix: packages: Document 'computed-origin-method'.
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
CAJ3okZ3MitVtxDuPvmiJWi8b5sWac0aO0kYX+4QXDcs=KFA8+g@mail.gmail.com
Hi,
On Wed, 29 Sept 2021 at 16:36, Liliana Marie Prikler<liliana.prikler@gmail.com> wrote:
Toggle quote (10 lines)> > Perhaps I am wrong about option (2) -- my claim is that> > computed-origin-method is *always* used with a promise so it is for> > sure an half-baked guess but enough; and it avoids to hard code the> > modules from where the packages come from. Therefore, option (2)> > does not improve, IMHO.>> The probability of having a promise when using computed-origin-method> is 100%. What is the probability of having computed-origin-method when> you see a promise? The answer is: we don't know. We can see from the
You mean, what is the probability of having a computed-origin-methodwhen the origin-uri is a promise? We do not know, but pragmatically,for now 100%. :-)
Option (2) is:
___ (or (eq? method (@@ (gnu packages gnuzilla) computed-origin-method)) _______ (eq? method (@@ (gnu packages linux) computed-origin-method)))
then I ask you similarly: what is the probability of having packagesusing computed-origin-method in these 2 modules only? We do not know,but pragmatically, for now 100%. :-)
The hypothetical probabilities to evaluate are:
- what would be the probability that a new package having a promiseas origin-uri is not indeed a package with a computed-origin-method?vs - what would be the probability that a new package usingcomputed-origin-method is not part of either (gnu packages gnuzilla)or (gnu packages linux)?
Anyway! Well, I am not convinced that it is worth to tackle thesehypothetical issues. :-)
That's why the option (3):
(eq? method (@@ (guix packages) computed-origin-method))
which means refactorize*. It is somehow the two worlds: check i.e.,safer, no modules hard-coded and keep private the time to have TheRight Plan for this computed-origin-method.
*refactorize: I think (guix packages) is better because it defines'<origin>' and other tooling friends. Because'computed-origin-method' is somehow a temporary tool about origin,i.e., a mechanism to define packages, it makes sense to me to put itthere. However, (gnu packages) is about tools to deal with packages,not to define them; although one could argue that 'search-patch' isthere is used to define package. For what my rationale behind thechoice of (guix packages) is worth. And indeed, I have onlyhalf-mentioned this rationale.

As I said, generating this sources.json file by the website is clunky.Somehow, it is a quick hack to have something up waiting The RightWay; the long-term generations should be done through the DataService, as it had been already discussed but not yet implemented.Help welcome. :-)

Toggle quote (10 lines)> > About update guix package [2/2], it has to be done, IIUC. The file> > sources.json contains what the package guix provides, not what the> > current Guix has. The website -- built using the Guile tool haunt --> > uses Guix as a Guile library. Maybe I miss something.>> What I was trying to say was that you wouldn't need to rebuild the guix> package before applying the 50515 changes, which this one seems to> require. Again, I'm not 100% sure that's the case, but IIUC (gnu> packages) is its own realm in this regard.
Hum, maybe there is a misunderstanding here. On one hand 50620applies to the guix.git repo and on the other hand 50515 applies toguix-artwork.git repo.
To have the sources of linux-libre and icecat reported in sources.jsonand thus to get a chance to have them archived by SWH, we need:
a- if computed-origin-method is factorized then update the guixpackage (Guix as a library) else do nothing; patch applied to guix.git b- tweak how sources.json is built; patch applied to guix-artwork.git
Well, the aim of 50620 is to deal with a) whereas the aim of 50515 isto deal with b). Note that 50515 requires a v2 ifcomputed-origin-method is factorized.
Maybe I miss something. From my understanding, all the modules arepart of Guix as a library. Therefore, it does not depends on where werefactorize.


To be honest, I thought that this tiny improvement of the SWH coveragewould have been much more easier and that that trivial task would nothave taken more than 15 days with lengthy discussions. :-)
Toggle quote (3 lines)> I do have some opinions on that, but I'll type them out in response to> Ludo, so as to not repeat myself too often.
Thanks. I will comment overthere or maybe raise the discussion to guix-devel.
All the best,simon
L
L
Liliana Marie Prikler wrote on 29 Sep 21:10 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)
a6341f4e1af750f563df700ad97f8336214a1e25.camel@gmail.com
Hi,
Am Mittwoch, den 29.09.2021, 19:48 +0200 schrieb zimoun:
Toggle quote (43 lines)> Hi,> > On Wed, 29 Sept 2021 at 16:36, Liliana Marie Prikler> <liliana.prikler@gmail.com> wrote:> > > > Perhaps I am wrong about option (2) -- my claim is that> > > computed-origin-method is *always* used with a promise so it is> > > for sure an half-baked guess but enough; and it avoids to hard> > > code the modules from where the packages come from. Therefore,> > > option (2) does not improve, IMHO.> > > > The probability of having a promise when using computed-origin-> > method is 100%. What is the probability of having computed-origin-> > method when you see a promise? The answer is: we don't know. We> > can see from the> > You mean, what is the probability of having a computed-origin-method> when the origin-uri is a promise? We do not know, but pragmatically,> for now 100%. :-)> > Option (2) is:> > ___ (or (eq? method (@@ (gnu packages gnuzilla) computed-origin-> method))> _______ (eq? method (@@ (gnu packages linux) computed-origin-> method)))> > then I ask you similarly: what is the probability of having packages> using computed-origin-method in these 2 modules only? We do not> know,> but pragmatically, for now 100%. :-)> > The hypothetical probabilities to evaluate are:> > - what would be the probability that a new package having a promise> as origin-uri is not indeed a package with a computed-origin-method?> vs> - what would be the probability that a new package using> computed-origin-method is not part of either (gnu packages gnuzilla)> or (gnu packages linux)?> > Anyway! Well, I am not convinced that it is worth to tackle these> hypothetical issues. :-)
I meant that only in reference to a comparison of your original patchin #50515 vs. option (2). Option (2) is conservative, it only detectscomputed origin which it knows how to handle. Your original patch ismore liberal in that it detects anything that has a promise as uri as acomputed origin, which might however not have the same semantics asthose two copies of the same code. Obviously, both (3) and (3a) don'thave this problem, but they're also conservative in that e.g. I couldroll my own channel with the exact same computed-origin-methodcopypasta'd once more and it wouldn't be detected, though that'sprobably off-topic.[1]
Toggle quote (7 lines)> That's why the option (3):> > (eq? method (@@ (guix packages) computed-origin-method))> > which means refactorize*. It is somehow the two worlds: check i.e.,> safer, no modules hard-coded and keep private the time to have The> Right Plan for this computed-origin-method.
I was a little confused when I read factorize from Ludo or refactorizefrom you. I know this technique under the name "refactoring".
Toggle quote (9 lines)> *refactorize: I think (guix packages) is better because it defines> '<origin>' and other tooling friends. Because> 'computed-origin-method' is somehow a temporary tool about origin,> i.e., a mechanism to define packages, it makes sense to me to put it> there. However, (gnu packages) is about tools to deal with packages,> not to define them; although one could argue that 'search-patch' is> there is used to define package. For what my rationale behind the> choice of (guix packages) is worth. And indeed, I have only> half-mentioned this rationale.
To that I would counter, that (guix packages) only defines package andorigin records and how to compile them to bags and derivations. Allthe methods through which those fields are filled are defined outside,be it the occasional local-file used in "Build it with Guix"-stylerecipes or the closer to our methods svn-fetch and git-fetch.
Were it not for the fact that it uses procedures defined in (guixpackages), you might have a better time reasoning that this should be ahidden part of (guix gexp), but since it does, it would introduce acycle if you did. While (gnu packages) does offer "tools to deal withpackages" as you put it, I'd argue the way it does is in establishing anamespace for them (fold-packages etc.) and that this namespace (the"GNU namespace") is the best location for computed-origin-method.
The reason we use computed-origin-method at all, is because as GNU Guixwe take a hard stance on the FSDG and do our damndest not to lead usersto nonfree software. This includes producing clean --source tarballs. Were it not for that, we could easily deblob at build time, and this isan important observation to make, because it means that projects thatdon't need this level of control can "safely" defer it the way we docurrently for "unpack more after unpack".[2] In other words, Istrongly believe that users of compute-origin-method do the hard workof computing origins to follow GNU standards and will thereby have noissue referencing the GNU namespace to get to it.
Toggle quote (6 lines)> As I said, generating this sources.json file by the website is> clunky.> Somehow, it is a quick hack to have something up waiting The Right> Way; the long-term generations should be done through the Data> Service, as it had been already discussed but not yet implemented.> Help welcome. :-)
I have no opinion on that as I don't work on the Data Service.
Toggle quote (16 lines)> > > About update guix package [2/2], it has to be done, IIUC. The> > > file> > > sources.json contains what the package guix provides, not what> > > the> > > current Guix has. The website -- built using the Guile tool> > > haunt --> > > uses Guix as a Guile library. Maybe I miss something.> > > > What I was trying to say was that you wouldn't need to rebuild the> > guix package before applying the 50515 changes, which this one> > seems to require. Again, I'm not 100% sure that's the case, but> > IIUC (gnu packages) is its own realm in this regard.> > Hum, maybe there is a misunderstanding here. On one hand 50620> applies to the guix.git repo and on the other hand 50515 applies to> guix-artwork.git repo.
Oh, I somehow missed that completely. Oops.
Toggle quote (17 lines)> To have the sources of linux-libre and icecat reported in> sources.json and thus to get a chance to have them archived by SWH,> we need:> > a- if computed-origin-method is factorized then update the guix> package (Guix as a library) else do nothing; patch applied to> guix.git> b- tweak how sources.json is built; patch applied to guix-> artwork.git> > Well, the aim of 50620 is to deal with a) whereas the aim of 50515 is> to deal with b). Note that 50515 requires a v2 if> computed-origin-method is factorized.> > Maybe I miss something. From my understanding, all the modules are> part of Guix as a library. Therefore, it does not depends on where> we refactorize.
No, no, I was wrongly under the impression that this sources.json wouldbe generated by Guix itself what with all the other SWH interaction wealready have. Again, a mistake on my part.
Toggle quote (3 lines)> To be honest, I thought that this tiny improvement of the SWH> coverage would have been much more easier and that that trivial task> would not have taken more than 15 days with lengthy discussions. :-)
To be honest, part of the lengthy discussion was me being confusedabout your intent – in multiple ways. If you wanted a "quick and dirtyfix" you could have went with checking those two modules explicitly onthe guix-artwork side and it would have had a fairly small impact. Reading this patch first and the discussion second, I had assumed yourintent was rather to formalize a method that had hitherto only beenused informally and the move to the guix namespace amplifies that imo.
All the best,Liliana
[1] The only solution is of course to compare via equal? instead ofeq?, which if implemented for functions would be Turing-hard :)[2] Of course, this assumes that other projects don't want to actuallyunpack dependencies when using --source, but I think it's likelier theywill just use recursive submodule checkout.
Z
Z
zimoun wrote on 29 Sep 22:15 +0200
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
861r57dtq1.fsf@gmail.com
Hi Liliana,
On Wed, 29 Sep 2021 at 21:10, Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:
Toggle quote (5 lines)> I could> roll my own channel with the exact same computed-origin-method> copypasta'd once more and it wouldn't be detected, though that's> probably off-topic.[1]
If it is in your own channel, then it will not be part of the filehttps://guix.gnu.org/sources.json.
From my understanding, you are arguing about corner cases that does nothappen now. And if it happens in the near future, we will fix it,depending on what will really happen in this very future. ;-)

Toggle quote (3 lines)> I was a little confused when I read factorize from Ludo or refactorize> from you. I know this technique under the name "refactoring".
Indeed. Maybe a false-friend from French. :-)

Toggle quote (2 lines)>> *refactorize: I think (guix packages) is better because it defines
[...]
Toggle quote (4 lines)>> half-mentioned this rationale.>> To that I would counter, that (guix packages) only defines package and
[...]
Toggle quote (2 lines)> issue referencing the GNU namespace to get to it.
I hear your argument. Well, I will not discuss it. Raise as an answerto Ludo, maybe.

Toggle quote (12 lines)>> To be honest, I thought that this tiny improvement of the SWH>> coverage would have been much more easier and that that trivial task>> would not have taken more than 15 days with lengthy discussions. :-)>> To be honest, part of the lengthy discussion was me being confused> about your intent – in multiple ways. If you wanted a "quick and dirty> fix" you could have went with checking those two modules explicitly on> the guix-artwork side and it would have had a fairly small impact.> Reading this patch first and the discussion second, I had assumed your> intent was rather to formalize a method that had hitherto only been> used informally and the move to the guix namespace amplifies that imo.
The cover letter [1] says: «This patch follows the discussion from [0].»where [0] points to the Mark’s approval as an answer to a patch whichapplies to website/apps/packages/builder.scm.
Then the cover letter [1] says: «In short, it simplifies the codegenerating the file 'sources.json' used by Software Heritage to ingestall the tarballs.»
1: http://issues.guix.gnu.org/50620#0
I am sorry if this cover letter was not enough explicit about my intent.From my point of view, the aim of this cover letter was to invite toread first the discussion and second read the patch. My bad if this aimhad been missed. I apologize for the confusion.
Being optimistic, this discussion leads to some concerns about this’computed-origin-method’ and ideas for improving. IMHO, it is worth toopen another issue providing the wish of multi-origin packages andreference to this. WDYT?
All the best,simon
M
M
Mark H Weaver wrote on 29 Sep 22:42 +0200
Re: bug#50620: [PATCH 0/2] Unify 'computed-origin-method' (linux, icecat)
(name . Ludovic Courtès)(address . ludo@gnu.org)
87h7e3glkn.fsf@netris.org
Hi Ludovic,
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (7 lines)> A better solution IMO would be to improve the ‘snippet’ mechanism in the> first place. ‘computed-origin-method’ improves on it in two ways: (1)> lazy evaluation of the gexp, and (2) allows the use of a different base> name.>> I would think #2 is addressed by the ‘file-name’ field (isn’t it?).
Using the 'file-name' field would not satisfy my requirements. It hastwo problems:
(1) It would cause the unmodified upstream Firefox source tarball to be named "icecat-…" in the store.
(2) Although the resulting tarball would be named "icecat-…", the toplevel directory name within that tarball would still be named "firefox-…".
I consider each of these flaws to be unacceptable.
What do you think?
Thanks, Mark
-- Disinformation flourishes because many people care deeply about injusticebut very few check the facts. Ask me about https://stallmansupport.org.
L
L
Ludovic Courtès wrote on 29 Sep 23:34 +0200
(name . Mark H Weaver)(address . mhw@netris.org)
875yuj6p7r.fsf_-_@gnu.org
Hi Mark,
Mark H Weaver <mhw@netris.org> skribis:
Toggle quote (21 lines)> Ludovic Courtès <ludo@gnu.org> writes:>>> A better solution IMO would be to improve the ‘snippet’ mechanism in the>> first place. ‘computed-origin-method’ improves on it in two ways: (1)>> lazy evaluation of the gexp, and (2) allows the use of a different base>> name.>>>> I would think #2 is addressed by the ‘file-name’ field (isn’t it?).>> Using the 'file-name' field would not satisfy my requirements. It has> two problems:>> (1) It would cause the unmodified upstream Firefox source tarball to be> named "icecat-…" in the store.>> (2) Although the resulting tarball would be named "icecat-…", the> toplevel directory name within that tarball would still be named> "firefox-…".>> I consider each of these flaws to be unacceptable.
Oh I see. I agree it’d be nice to have a way to fix those.
Perhaps by allowing for custom pack-and-repack procedures?
Thanks,Ludo’.
M
M
Mark H Weaver wrote on 29 Sep 23:40 +0200
Re: [PATCH 1/2] guix: packages: Document 'computed-origin-method'.
(address . 50620@debbugs.gnu.org)
87ee97givh.fsf@netris.org
Hi Simon,
zimoun <zimon.toutoune@gmail.com> writes:
Toggle quote (17 lines)> On Wed, 29 Sept 2021 at 16:36, Liliana Marie Prikler> <liliana.prikler@gmail.com> wrote:>>> > Perhaps I am wrong about option (2) -- my claim is that>> > computed-origin-method is *always* used with a promise so it is for>> > sure an half-baked guess but enough; and it avoids to hard code the>> > modules from where the packages come from. Therefore, option (2)>> > does not improve, IMHO.>>>> The probability of having a promise when using computed-origin-method>> is 100%. What is the probability of having computed-origin-method when>> you see a promise? The answer is: we don't know. We can see from the>> You mean, what is the probability of having a computed-origin-method> when the origin-uri is a promise? We do not know, but pragmatically,> for now 100%. :-)
To my mind, that's not good enough. I consider it unsafe, and poorprogramming practice, to force a promise without first knowing what thatpromise represents and what are the implications of forcing it.
In projects as large as Guix, if it becomes accepted practice tointroduce lots of assumptions scattered around the code that are"for now 100%" true, the result is eventually a very brittle projectwhere it's difficult to make changes without random stuff breaking.
Toggle quote (9 lines)> Option (2) is:>> ___ (or (eq? method (@@ (gnu packages gnuzilla) computed-origin-method))> _______ (eq? method (@@ (gnu packages linux) computed-origin-method)))>> then I ask you similarly: what is the probability of having packages> using computed-origin-method in these 2 modules only? We do not know,> but pragmatically, for now 100%. :-)
The potential failure mode here is far less bad. In this case, ifsomeone else makes another clone of 'computed-origin-method' in anothermodule and forgets to update this code, the worst case is that somesource code fails to be added to SWH. Incidentally, I guess that's thesame outcome that would happen if someone adds a brand new'origin-method' and forgets to update this code.
Incidentally, I have a suggestion for how to avoid that failure modeproperly, once and for all: issue a warning if we're unable to identifythe 'method' of the origin at hand, calling attention to the fact thatthere's an unhandled case in this code. This is precisely analogous toStandard ML's *very* useful feature of issuing warnings at compile timein case of an non-exhaustive 'match' form.
What do you think?
In any case, thanks very much for your efforts to push this issue towardresolution.
Regards, Mark
-- Disinformation flourishes because many people care deeply about injusticebut very few check the facts. Ask me about https://stallmansupport.org.
L
L
Ludovic Courtès wrote on 29 Sep 23:47 +0200
Re: bug#50620: [PATCH 0/2] Unify 'computed-origin-method' (linux, icecat)
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
87pmsr5a0z.fsf_-_@gnu.org
Hi Liliana,
Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:
Toggle quote (12 lines)> Am Mittwoch, den 29.09.2021, 15:16 +0200 schrieb Ludovic Courtès:>> Hi there!>> >> I’d rather go with zimoun’s original patch, which is focused and does>> nothing more than what was originally intended, which is to factorize>> the procedure. I’ll go ahead and apply it shortly if there are no>> objections.> I have trouble understanding this paragraph. What exactly is "this> patch" and what do you mean by "factorizing"? If it means moving> computed-origin-method elsewhere, then yes, for a short-time solution> only moving it is a wise choice in my opinion,
OK, I agree too.
Toggle quote (5 lines)> but zimoun and I still disagree on the target. zimoun says (guix> packages) for reasons unknown to me, whereas I say (gnu packages),> because it's closer to where it's used and doesn't imply that this is> going to be a part of the (guix) download schemes anytime soon.
(gnu packages) is higher-level: it’s part of the distro and includes CLIhelpers such as ‘specification->package’. So I think (guix …) issomewhat more appropriate.
(That said, what matters more to me is how we’re going to replace itwith a proper solution.)
[...]
Toggle quote (18 lines)>> A better solution IMO would be to improve the ‘snippet’ mechanism in>> the first place. ‘computed-origin-method’ improves on it in two>> ways: (1) lazy evaluation of the gexp, and (2) allows the use of a>> different base name.>> >> I would think #2 is addressed by the ‘file-name’ field (isn’t it?).>> >> As for #1, it can be addressed by making the ‘snippet’ field delayed>> or thunked. It’s a one line change; the only thing we need is to>> measure, or attempt to measure, the impact it has on module load>> time.>> >> Thoughts?> This would work for packages, whose source are some base source with> patches or snippets applied, as is indeed the case for linux and> icecat. However, there are also other potential uses for computed> origins.
It’s hard for me to talk about potential uses in the abstract. :-)
There might be cases where an origin simply isn’t the right tool and onewould prefer ‘computed-file’ or something else. It really depends onthe context.
[...]
Toggle quote (10 lines)> I think that some version of `computed-origin-method' will eventually> need to become public API as such packages may not always be best> described as "a base package with a snippet". If we had recursive> origins – i.e. origins, that can take origins as inputs – we might be> able to do some of that, but I don't think it would necessarily work> for linux-libre or icecat, as with those you don't want the tainted> versions to be kept around. Perhaps this could be worked around by not> interning the intermediate origins, but only using their file-names> inside the temporary directory in which the snippet is applied?
“Recursive origins” are a bit of a stretch as a concept IMO; what youdescribe is a case where I’d probably use ‘computed-file’ instead.
Toggle quote (9 lines)> Another thing is that the final act of the linux-libre promise is not> the packing of the tarball, but the deblob-check. Guix currently lacks> a way of modeling such checks in their origin, but I'd argue it would> need one if we wanted to do computed origins via snippets. This is not> required by icecat and so one "simplification" could be that computed-> origin-method would not require the user to create a tarball, but> instead simply provide a name for the tarball and a directory to create> it from (via a promise again).
Ah, I had overlooked that ‘deblob-check’ bit. It could be that allowingfor custom pack-and-repack procedures would be enough to address it.
Toggle quote (4 lines)> A combination of the above might make computed origins obsolete for> good, but the question remains whether that is a better design. What> do y'all think?
The design goal is to have clearly identified types: <package>,<origin>, <operating-system>. For each of these, we want someflexibility: build system, origin method, etc. However, beyond somelevel of stretching, it may be clearer to just use the catch-all‘computed-file’ or to devise a new type. After all, that’s how <origin>came to be (we could have used <package> instead with a suitable buildsystem).
There’s a tension between “purely declarative” and “flexible”, and it’sabout striking a balance, subjectively.
Hope that makes sense!
Ludo’.
L
L
Liliana Marie Prikler wrote on 30 Sep 00:13 +0200
Re: [PATCH 1/2] guix: packages: Document 'computed-origin-method'.
(name . zimoun)(address . zimon.toutoune@gmail.com)
d1d2f23fb9a727fe7e79097a09ea1cd2b14675b0.camel@gmail.com
Hi zimoun,
Am Mittwoch, den 29.09.2021, 22:15 +0200 schrieb zimoun:
Toggle quote (15 lines)> Hi Liliana,> > On Wed, 29 Sep 2021 at 21:10, Liliana Marie Prikler <> liliana.prikler@gmail.com> wrote:> > > I could roll my own channel with the exact same computed-origin-> > method copypasta'd once more and it wouldn't be detected, though> > that's probably off-topic.[1]> > If it is in your own channel, then it will not be part of the file> https://guix.gnu.org/sources.json.> > From my understanding, you are arguing about corner cases that does> not happen now. And if it happens in the near future, we will fix> it, depending on what will really happen in this very future. ;-)
The patch mentions "automatic detection of computed-origin-method"which I would assume has implication beyond this sources.json. Butyeah, I can see that it's off-topic to this discussion, hence why Iwrote that it's probably off-topic to this dicussion.
Toggle quote (16 lines)> > > *refactorize: I think (guix packages) is better because it> > > defines> > [...]> > > > half-mentioned this rationale.> > > > To that I would counter, that (guix packages) only defines package> > and> > [...]> > > issue referencing the GNU namespace to get to it.> > I hear your argument. Well, I will not discuss it. Raise as an> answer to Ludo, maybe.
I did already mention that in my reply to Ludo, so we'll see.
Toggle quote (30 lines)> > > To be honest, I thought that this tiny improvement of the SWH> > > coverage would have been much more easier and that that trivial> > > task> > > would not have taken more than 15 days with lengthy discussions.> > > :-)> > > > To be honest, part of the lengthy discussion was me being confused> > about your intent – in multiple ways. If you wanted a "quick and> > dirty fix" you could have went with checking those two modules> > explicitly on the guix-artwork side and it would have had a fairly> > small impact.> > Reading this patch first and the discussion second, I had assumed> > your intent was rather to formalize a method that had hitherto only> > been used informally and the move to the guix namespace amplifies> > that imo.> > The cover letter [1] says: «This patch follows the discussion from> [0].» where [0] points to the Mark’s approval as an answer to a patch> which applies to website/apps/packages/builder.scm.> > Then the cover letter [1] says: «In short, it simplifies the code> generating the file 'sources.json' used by Software Heritage to> ingest all the tarballs.»> > 1: <http://issues.guix.gnu.org/50620#0>> > I am sorry if this cover letter was not enough explicit about my> intent. From my point of view, the aim of this cover letter was to> invite to read first the discussion and second read the patch. My> bad if this aim had been missed. I apologize for the confusion.
Again, I read the patch itself first and the context second, butspeaking about "simplifying the code generating sources.json", the realchange were we to compare (2) and (3) or (3a) to each other would be a3 line diff (two deletions, one insertion). So I do think it is fairto also talk about implications beyond those three lines.
Also, even with this context in mind, the patch at first appeared to meas a way of sneaking (1) past the radar, rather than the three-linediff that one would see when looking at it from 50515 with (2) applied.
Toggle quote (4 lines)> Being optimistic, this discussion leads to some concerns about this> ’computed-origin-method’ and ideas for improving. IMHO, it is worth> to open another issue providing the wish of multi-origin packages and> reference to this. WDYT?
Since it's but an idea sketch in my head at the moment, I think thebest we could muster discussing this outside of this thread would be onguix-devel. Which is fine and all, but since we're looking in thisthread for something comparatively small in scale I'd say let's look atthe small issue first and the big issue once we've fixed the small one.
Let's shortly recap what options we have.
A: Push a v2 of 50515 guix-artwork, which references (gnu packageslinux) and (gnu packages gnuzilla) using @@. Then decide on whichmodule we want to have computed-origin-method to be in and update theguix package. Finally, update the sources.json generator to use thesingular reference.
B: Push the lazy v2 as above, but instead hold up the cleaning up partuntil we find a solution for the computed-origin-method in this threador guix-devel.
C: Discuss the (gnu packages) vs. (guix packages) thing some more,merge this patch (with perhaps a move), update the guix package andthen do a v2 of 50515.
C2: Have Ludo flip a coin and decide.
D: Have computed-origin-method block the sources.json generator untilit is completely resolved.
We obviously want to avoid D here and are somewhat aiming for C at themoment instead. However, we are kinda stuck here as even though wedon't want this situation to continue indefinitely, we can't seem toreach a consensus quickly.
WDYT? Does it make sense to do the "redundant test" [1], knowing thatit'll be soon simplified? Can we hold off more computed-origin-methodclones until we find a way of making do without it or actually decidethat it's public API?
All the best,Liliana
[1] http://issues.guix.gnu.org/50515#4
Z
Z
zimoun wrote on 30 Sep 00:45 +0200
(address . 50620@debbugs.gnu.org)
86v92jc861.fsf@gmail.com
Hi Mark,
On Wed, 29 Sep 2021 at 17:40, Mark H Weaver <mhw@netris.org> wrote:
Toggle quote (11 lines)> zimoun <zimon.toutoune@gmail.com> writes:
> To my mind, that's not good enough. I consider it unsafe, and poor> programming practice, to force a promise without first knowing what that> promise represents and what are the implications of forcing it.>> In projects as large as Guix, if it becomes accepted practice to> introduce lots of assumptions scattered around the code that are> "for now 100%" true, the result is eventually a very brittle project> where it's difficult to make changes without random stuff breaking.
I agree…
Toggle quote (16 lines)>> Option (2) is:>>>> ___ (or (eq? method (@@ (gnu packages gnuzilla) computed-origin-method))>> _______ (eq? method (@@ (gnu packages linux) computed-origin-method)))>>>> then I ask you similarly: what is the probability of having packages>> using computed-origin-method in these 2 modules only? We do not know,>> but pragmatically, for now 100%. :-)>> The potential failure mode here is far less bad. In this case, if> someone else makes another clone of 'computed-origin-method' in another> module and forgets to update this code, the worst case is that some> source code fails to be added to SWH. Incidentally, I guess that's the> same outcome that would happen if someone adds a brand new> 'origin-method' and forgets to update this code.
…and I also agree. That’s why, right after this quotation, I wrote:
That's why the option (3):
(eq? method (@@ (guix packages) computed-origin-method))
which means refactorize*. It is somehow the two worlds: check i.e., safer, no modules hard-coded and keep private the time to have The Right Plan for this computed-origin-method.
which is *exactly* what you are asking, IIUC.
To be honest, I do not understand why we are discussing at length thistrivial path:
for guix.git:
1. move the duplicate computed-origin-method to a single place 2. keep it private 3. add comments about that
for guix-artwork.git:
4. guard the promise using a check against: (@@ (module somewhere) computed-origin-method)
for guix.git
5. update the package guix
done! :-)
It changes nothing on the Guix side.
Do we not agree on this trivial path? Re-reading from the startingpoint 50515 and then 50620, the trivial road appears to me clear. Iapologize if it was not or to not make it explicit earlier.
From my understanding, we all agree, somehow; because it fixes thecurrent situation and let the time to cook The Right Plan for thiscomputed-origin-method. Where we can discuss is #2 but as it is alreadymentioned, it is out of scope for sources.json, IMHO.
If I knew all what would happen, then I would send a v2 for 50515 usingwhat you described as option (2). :-) My aim with this 50620 was just tosimplify at very low cost the code (belonging to the repoguix-artwork.git) which generates https://guix.gnu.org/sources.json.
The v2 (adding a guard) for 50515 is simply waiting the output of this50620; because this guard depends if computed-origin-method is definedat an unique location or at two different locations.
Toggle quote (7 lines)> Incidentally, I have a suggestion for how to avoid that failure mode> properly, once and for all: issue a warning if we're unable to identify> the 'method' of the origin at hand, calling attention to the fact that> there's an unhandled case in this code. This is precisely analogous to> Standard ML's *very* useful feature of issuing warnings at compile time> in case of an non-exhaustive 'match' form.
The SWH reader which consumes this sources.json file does not care abouta warning. And AFAIK no one is reviewing by hand this sources.jsonfile. Specifically, the only purpose of this very file is to beconsumed by the SWH infrastructure. It is automatically generated whenthe Guix website rebuilds; the content of this file depends on theversion of the package guix.
That’s said, there is room of improvement to track what is the archivalcoverage by SWH. Today, we do not have a clear picture of the packagesarchived by SWH. By archived, it means packages for which Guix is ableto fallback and lookup using the SWH archive. Well, now one foot, nowthe other. :-)
(On a side note, I agree that this ML feature is very useful. From myunderstanding, it requires a kind of type system that Guile does nothave. Sadly.)
Toggle quote (3 lines)> In any case, thanks very much for your efforts to push this issue toward> resolution.
Thanks. At one point, I felt demotivated then reconsidering the energywe all are putting it, we have to resolve. :-)
All the best,simon
Z
Z
zimoun wrote on 30 Sep 01:31 +0200
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
86o88bc62v.fsf@gmail.com
Hi,
On Thu, 30 Sep 2021 at 00:13, Liliana Marie Prikler <liliana.prikler@gmail.com> wrote:
Toggle quote (4 lines)> C: Discuss the (gnu packages) vs. (guix packages) thing some more,> merge this patch (with perhaps a move), update the guix package and> then do a v2 of 50515.
This is the option I am for. Even, the patch is ready and waiting since«Fri, 10 Sep 2021 18:01:22 +0200». ;-)
The patch reads:
Toggle snippet (13 lines)+ (if (eq? method (@@ (guix packages) computed-origin-method))+ ;; Packages in gnu/packages/gnuzilla.scm and gnu/packages/linux.scm+ ;; represent their 'uri' as 'promise'.+ (match uri+ ((? promise? promise)
[...]
+ (_ `((type . #nil))))))+ ;;Regular packages represent 'uri' as string.+ `((type . ,(cond ((or (eq? url-fetch method)
and I find better (guix packages) but I do not have a strong opinion; Iaccepted previously in this thread to send a v2 with (gnu packages) orwhatever other location.

Toggle quote (3 lines)> WDYT? Does it make sense to do the "redundant test" [1], knowing that> it'll be soon simplified?
I do not mind about option (2) which reads:
Toggle snippet (12 lines)+ (if (or (eq? method (@@ (gnu packages linux) computed-origin-method))+ (eq? method (@@ (gnu packages gnuzilla) computed-origin-method)))+ (match uri+ ((? promise? promise)
[...]
+ (_ `((type . #nil))))))+ ;;Regular packages represent 'uri' as string.+ `((type . ,(cond ((or (eq? url-fetch method)
Whatever.
However, since it is me who takes care about how this sources.json isgenerated, I find easier to have one location and forget about thiscase. The only thing I am asking here with this patch 50620 is tolocate computed-origin-method to one unique place. If people stronglydisagree, then let do this option (2) and move on.
Last, I am confused why all this is so complicated when it is trivialand for something outside Guix proper. I do not understand what we arediscussing when my request is trivial, IMHO.
This discussion has eaten all my energy allowed for Guix.See you next week.
All the best,simon
L
L
Liliana Marie Prikler wrote on 30 Sep 01:44 +0200
Re: bug#50620: [PATCH 0/2] Unify 'computed-origin-method' (linux, icecat)
(name . Ludovic Courtès)(address . ludo@gnu.org)
09c009ec61b8f9c1746f98916f492b8953b16dcc.camel@gmail.com
Hi Ludo,
Am Mittwoch, den 29.09.2021, 23:47 +0200 schrieb Ludovic Courtès:
Toggle quote (14 lines)> [...]> > > but zimoun and I still disagree on the target. zimoun says (guix> > packages) for reasons unknown to me, whereas I say (gnu packages),> > because it's closer to where it's used and doesn't imply that this> > is> > going to be a part of the (guix) download schemes anytime soon.> > (gnu packages) is higher-level: it’s part of the distro and includes> CLI helpers such as ‘specification->package’. So I think (guix …) is> somewhat more appropriate.> > (That said, what matters more to me is how we’re going to replace it> with a proper solution.)
(gnu packages) being high-level is part of the reason I want it there. Stuff that's hidden quite deep inside (guix something) will be slowerto change and replace with the proper solution. When you pull on alever, the outside moves faster :)
Toggle quote (40 lines)> > > A better solution IMO would be to improve the ‘snippet’ mechanism> > > in the first place. ‘computed-origin-method’ improves on it in> > > two ways: (1) lazy evaluation of the gexp, and (2) allows the use> > > of a different base name.> > > > > > I would think #2 is addressed by the ‘file-name’ field (isn’t> > > it?).> > > > > > As for #1, it can be addressed by making the ‘snippet’ field> > > delayed or thunked. It’s a one line change; the only thing we> > > need is to measure, or attempt to measure, the impact it has on> > > module load time.> > > > > > Thoughts?> > This would work for packages, whose source are some base source> > with patches or snippets applied, as is indeed the case for linux> > and icecat. However, there are also other potential uses for> > computed origins.> > It’s hard for me to talk about potential uses in the abstract. :-)> > There might be cases where an origin simply isn’t the right tool and> one would prefer ‘computed-file’ or something else. It really> depends on the context.> > [...]> > > I think that some version of `computed-origin-method' will> > eventually need to become public API as such packages may not> > always be best described as "a base package with a snippet". If we> > had recursive origins – i.e. origins, that can take origins as> > inputs – we might be able to do some of that, but I don't think it> > would necessarily work for linux-libre or icecat, as with those you> > don't want the tainted versions to be kept around. Perhaps this> > could be worked around by not interning the intermediate origins,> > but only using their file-names inside the temporary directory in> > which the snippet is applied?> > “Recursive origins” are a bit of a stretch as a concept IMO; what you> describe is a case where I’d probably use ‘computed-file’ instead.
In other words, we could/should use computed-file for linux-libre andicecat? If we reasonably can, would it make sense to use that in lieuof computed-origin-method to actually advertise the existence ofcomputed-file to Guix users/packagers?
Toggle quote (13 lines)> > Another thing is that the final act of the linux-libre promise is> > not the packing of the tarball, but the deblob-check. Guix> > currently lacks a way of modeling such checks in their origin, but> > I'd argue it would need one if we wanted to do computed origins via> > snippets. This is not required by icecat and so one> > "simplification" could be that computed-origin-method would not> > require the user to create a tarball, but instead simply provide a> > name for the tarball and a directory to create it from (via a> > promise again).> > Ah, I had overlooked that ‘deblob-check’ bit. It could be that> allowing for custom pack-and-repack procedures would be enough to> address it.
I think asking users to supply their own implementation of a 200 linelong function to be a bit much to only do part of the job. On theother hand, the promise for linux-libre takes 400 lines and for icecatmore than 600, but I think there are some things we ought to factorout. Particularly, looking up tools like tar or gzip and even theactual packing are always the same.
What we can't currently control is the top directory name and theoutput name. Both of that could be customized by supplying a "repack-name" field, which is used as basis for the directory name and thetarball name.Another thing we can't easily control are extraneous inputs to thepatches, although the patch-inputs field *does* exist.
Toggle quote (14 lines)> > A combination of the above might make computed origins obsolete for> > good, but the question remains whether that is a better> > design. What do y'all think?> > The design goal is to have clearly identified types: <package>,> <origin>, <operating-system>. For each of these, we want some> flexibility: build system, origin method, etc. However, beyond some> level of stretching, it may be clearer to just use the catch-all> ‘computed-file’ or to devise a new type. After all, that’s how> <origin> came to be (we could have used <package> instead with a> suitable build system).> > There’s a tension between “purely declarative” and “flexible”, and> it’s about striking a balance, subjectively.
To be fair, I did think that "computed-tarball" might be a goodabstraction in some sense, but on another hand origins are computedtarballs with a record interface.
On a somewhat related note, origins have this weird situation going onwhere some things like git or svn checkouts need to be defined throughthem, whereas others may pass unhindered. I feel that this contributesto the equation of source = origin, that might have caused "computed-origin-method" to exist in the first place.
What do you think?
Liliana
L
L
Liliana Marie Prikler wrote on 30 Sep 09:11 +0200
Re: [PATCH 1/2] guix: packages: Document 'computed-origin-method'.
(address . 50620@debbugs.gnu.org)
40b6ad5258fd8a2789ed62f857a98ac474c208fb.camel@gmail.com
Hi zimoun,
Am Donnerstag, den 30.09.2021, 00:45 +0200 schrieb zimoun:
Toggle quote (21 lines)> To be honest, I do not understand why we are discussing at length> this trivial path:> > for guix.git:> > 1. move the duplicate computed-origin-method to a single place> 2. keep it private> 3. add comments about that> > for guix-artwork.git:> > 4. guard the promise using a check against:> (@@ (module somewhere) computed-origin-method)> > for guix.git> > 5. update the package guix> > done! :-)> > It changes nothing on the Guix side.
I've started discussing this path because we are currently stuck at 1.for some time. Given that this drags on for so long and you arelooking for a "quick solution" for guix-artwork.git, I called intoquestion whether it is really necessary to modify guix.git first. Thisdoes not change nothing for the Guix side either, it changes thelocation of a hack most of us wish to rather avoid than to give moreattention to in the code base :)
Sorry for forcing you through this, I had not intended to spark thislengthy debates.
L
L
Ludovic Courtès wrote on 30 Sep 10:28 +0200
Re: bug#50620: [PATCH 0/2] Unify 'computed-origin-method' (linux, icecat)
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
87r1d64gd3.fsf_-_@gnu.org
Hi!
Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:
Toggle quote (7 lines)> I think asking users to supply their own implementation of a 200 line> long function to be a bit much to only do part of the job. On the> other hand, the promise for linux-libre takes 400 lines and for icecat> more than 600, but I think there are some things we ought to factor> out. Particularly, looking up tools like tar or gzip and even the> actual packing are always the same.
True, there’s a lot going on there, though that’s partly because it’sgeneric.
Toggle quote (7 lines)> What we can't currently control is the top directory name and the> output name. Both of that could be customized by supplying a "repack-> name" field, which is used as basis for the directory name and the> tarball name.> Another thing we can't easily control are extraneous inputs to the> patches, although the patch-inputs field *does* exist.
It’s possible to use a gexp as the snippet, where you can refer toadditional things in there (though in practice this is currentlyimpractical due to snippets not being thunks/promises.)
Toggle quote (24 lines)>> > A combination of the above might make computed origins obsolete for>> > good, but the question remains whether that is a better>> > design. What do y'all think?>> >> The design goal is to have clearly identified types: <package>,>> <origin>, <operating-system>. For each of these, we want some>> flexibility: build system, origin method, etc. However, beyond some>> level of stretching, it may be clearer to just use the catch-all>> ‘computed-file’ or to devise a new type. After all, that’s how>> <origin> came to be (we could have used <package> instead with a>> suitable build system).>> >> There’s a tension between “purely declarative” and “flexible”, and>> it’s about striking a balance, subjectively.> To be fair, I did think that "computed-tarball" might be a good> abstraction in some sense, but on another hand origins are computed> tarballs with a record interface.>> On a somewhat related note, origins have this weird situation going on> where some things like git or svn checkouts need to be defined through> them, whereas others may pass unhindered. I feel that this contributes> to the equation of source = origin, that might have caused "computed-> origin-method" to exist in the first place.
I’m not sure what you mean by “others may pass unhindered”? You meanother VCS checkouts?
Toggle quote (2 lines)> What do you think?
I think the situation of IceCat and Linux-libre is unusual: 2 packagesout of 18K. That probably explains why we have a hard time figuring outhow to generalize the issues that ‘computed-origin-method’ addresses.
What you propose (IIUC) sounds interesting: we’d provide a<computed-tarball> data type, which would make the source URL manifest(something that’s useful for https://issues.guix.gnu.org/50515, forinstance), but the lowering step would be entirely custom, similar towhat it already looks like:
(define-record-type* <computed-tarball> computed-tarball make-computed-tarball computed-tarball? this-computed-tarball (url computed-tarball-url) ;or could be an <origin> (builder computer-tarball-builder (thunked)) ;gexp (location computed-tarball-location (innate) (default (current-source-location))))
Is this what you had in mind?
Thanks,Ludo’.
L
L
Liliana Marie Prikler wrote on 30 Sep 16:17 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
735304d110f5e99c66904fc7ced3465bf1815baf.camel@gmail.com
Hi,
Am Donnerstag, den 30.09.2021, 10:28 +0200 schrieb Ludovic Courtès:
Toggle quote (11 lines)> [...]> > What we can't currently control is the top directory name and the> > output name. Both of that could be customized by supplying a> > "repack-name" field, which is used as basis for the directory name> > and the tarball name.> > Another thing we can't easily control are extraneous inputs to the> > patches, although the patch-inputs field *does* exist.> > It’s possible to use a gexp as the snippet, where you can refer to> additional things in there (though in practice this is currently> impractical due to snippets not being thunks/promises.)
Which is a practical issue because it'd mean that the tarball getsbuilt as soon as the source is interpreted?
Toggle quote (29 lines)> > > > A combination of the above might make computed origins obsolete> > > > for> > > > good, but the question remains whether that is a better> > > > design. What do y'all think?> > > > > > The design goal is to have clearly identified types: <package>,> > > <origin>, <operating-system>. For each of these, we want some> > > flexibility: build system, origin method, etc. However, beyond> > > some> > > level of stretching, it may be clearer to just use the catch-all> > > ‘computed-file’ or to devise a new type. After all, that’s how> > > <origin> came to be (we could have used <package> instead with a> > > suitable build system).> > > > > > There’s a tension between “purely declarative” and “flexible”,> > > and> > > it’s about striking a balance, subjectively.> > To be fair, I did think that "computed-tarball" might be a good> > abstraction in some sense, but on another hand origins are computed> > tarballs with a record interface.> > > > On a somewhat related note, origins have this weird situation going> > on where some things like git or svn checkouts need to be defined> > through them, whereas others may pass unhindered. I feel that this> > contributes to the equation of source = origin, that might have> > caused "computed-origin-method" to exist in the first place.> > I’m not sure what you mean by “others may pass unhindered”? You mean> other VCS checkouts?
I mean that we don't need to wrap local-file inside an origin forexample whereas we do need to wrap e.g. svn-fetch instead of having ansvn-checkout constructor at the top. It's not really that noticablenormally, but weird once you start thinking a little too hard about it.
Toggle quote (24 lines)> > What do you think?> > I think the situation of IceCat and Linux-libre is unusual: 2> packages out of 18K. That probably explains why we have a hard time> figuring out how to generalize the issues that ‘computed-origin-> method’ addresses.> > What you propose (IIUC) sounds interesting: we’d provide a> <computed-tarball> data type, which would make the source URL> manifest (something that’s useful for <> https://issues.guix.gnu.org/50515>;,> for instance), but the lowering step would be entirely custom,> similar to what it already looks like:> > (define-record-type* <computed-tarball> computed-tarball make-> computed-tarball> computed-tarball?> this-computed-tarball> (url computed-tarball-url) ;or could be an <origin>> (builder computer-tarball-builder (thunked)) ;gexp> (location computed-tarball-location (innate) (default (current-> source-location))))> > Is this what you had in mind?
Slightly similar, but I don't think I'd want a singular source url. Instead
(define-record-type* <computed-tarball> computed-tarball make-computed-tarball computed-tarball? this-computed-tarball (sources computed-tarball-sources) ; list of origins, local ; files or other things (builder computer-tarball-builder (thunked)) ; gexp (name computed-tarball-name) ; perhaps? (location computed-tarball-location (innate) (default (current-source-location))))
At the start of BUILDER, SOURCES are already unpacked to the currentworking directory under their stripped file names. After builderreturns, we either package the contents of the current workingdirectory up into a tarball (variant A) or we have builder return alist of files to pack up (variant B) which we then post-process maybe. WDYT?
L
L
Ludovic Courtès wrote on 30 Sep 22:09 +0200
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
87wnmx3jx3.fsf@gnu.org
Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:
Toggle quote (15 lines)> Am Donnerstag, den 30.09.2021, 10:28 +0200 schrieb Ludovic Courtès:>> [...]>> > What we can't currently control is the top directory name and the>> > output name. Both of that could be customized by supplying a>> > "repack-name" field, which is used as basis for the directory name>> > and the tarball name.>> > Another thing we can't easily control are extraneous inputs to the>> > patches, although the patch-inputs field *does* exist.>> >> It’s possible to use a gexp as the snippet, where you can refer to>> additional things in there (though in practice this is currently>> impractical due to snippets not being thunks/promises.)> Which is a practical issue because it'd mean that the tarball gets> built as soon as the source is interpreted?
It’s impractical because typical usage introduces top-level circularreferences (e.g., if you write #$gzip).
Toggle quote (5 lines)> I mean that we don't need to wrap local-file inside an origin for> example whereas we do need to wrap e.g. svn-fetch instead of having an> svn-checkout constructor at the top. It's not really that noticable> normally, but weird once you start thinking a little too hard about it.
Hmm yeah, I must not be thinking hard enough. :-)
Toggle quote (22 lines)> Slightly similar, but I don't think I'd want a singular source url. > Instead>> (define-record-type* <computed-tarball> computed-tarball > make-computed-tarball> computed-tarball?> this-computed-tarball> (sources computed-tarball-sources) ; list of origins, local > ; files or other things> (builder computer-tarball-builder (thunked)) ; gexp> (name computed-tarball-name) ; perhaps?> (location computed-tarball-location (innate) > (default (current-source-location))))>> At the start of BUILDER, SOURCES are already unpacked to the current> working directory under their stripped file names. After builder> returns, we either package the contents of the current working> directory up into a tarball (variant A) or we have builder return a> list of files to pack up (variant B) which we then post-process maybe.> > WDYT?
Overall LGTM! IWBN to see if there are other potential users in thetree (I can’t think of any), but for IceCat and Linux-libre, it couldalready improve the situation.
Thanks,Ludo’.
L
L
Liliana Marie Prikler wrote on 30 Sep 23:49 +0200
(name . Ludovic Courtès)(address . ludo@gnu.org)
0541a55a86b604ec45980341ea64ace7b9312db3.camel@gmail.com
Hi,
Am Donnerstag, den 30.09.2021, 22:09 +0200 schrieb Ludovic Courtès:
Toggle quote (8 lines)> Liliana Marie Prikler <liliana.prikler@gmail.com> skribis:> > > [...]> > Which is a practical issue because it'd mean that the tarball gets> > built as soon as the source is interpreted?> > It’s impractical because typical usage introduces top-level circular> references (e.g., if you write #$gzip).
Ah, right.
Toggle quote (7 lines)> > I mean that we don't need to wrap local-file inside an origin for> > example whereas we do need to wrap e.g. svn-fetch instead of having> > an svn-checkout constructor at the top. It's not really that> > noticable normally, but weird once you start thinking a little too> > hard about it.> > Hmm yeah, I must not be thinking hard enough. :-)
Perhaps it's just me who finds it weird tho. ¯\_(ツ)_/¯
Toggle quote (26 lines)> > Slightly similar, but I don't think I'd want a singular source> > url. Instead> > > > (define-record-type* <computed-tarball> computed-tarball > > make-computed-tarball> > computed-tarball?> > this-computed-tarball> > (sources computed-tarball-sources) ; list of origins, local > > ; files or other things> > (builder computer-tarball-builder (thunked)) ; gexp> > (name computed-tarball-name) ; perhaps?> > (location computed-tarball-location (innate) > > (default (current-source-location))))> > > > At the start of BUILDER, SOURCES are already unpacked to the> > current working directory under their stripped file names. After> > builder returns, we either package the contents of the current> > working directory up into a tarball (variant A) or we have builder> > return a list of files to pack up (variant B) which we then post-> > process maybe.> > > > WDYT?> > Overall LGTM! IWBN to see if there are other potential users in the> tree (I can’t think of any), but for IceCat and Linux-libre, it could> already improve the situation.
Concrete examples which currently use "unpack more after unpack":
- chez-scheme with nanopass and stex- xen's mini-os- lbzip2's gnulib (and probably gnulib in other locations)- similarly libgd in gedit and gnome-recipes (same origin for both)
The builder for those would always be a simple (series of) directoryrename(s) as well :)
This list might not be complete, at least I haven't checked whether itis. Also, packages which have (package-source some-other-package) asinput somewhere don't count here, as the missing sources can triviallybe found and inserted imo.
Regards,Liliana
L
L
Ludovic Courtès wrote on 1 Oct 00:17 +0200
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 50620-done@debbugs.gnu.org)
877dex1zfm.fsf_-_@gnu.org
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (8 lines)> The 'computed-origin-method' had been introduced as a workaround limitations> in the 'snippet' mechanism. The procedure is duplicated which makes hard to> automatically detects packages using it.>> * guix/packages.scm (computed-origin-method): Move procedure from...> * gnu/packages/gnuzilla.scm: ...here and...> * gnu/packages/gnuzilla.scm: ...there.
I tweaked the commit log and applied, thanks!
Hopefully we’ll grow a “proper” solution thanks to the discussions about<computed-tarball> & co. that we’ve had.
Thanks everyone,Ludo’.
Closed
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 50620@debbugs.gnu.org