[PATCH] Fixes init of #42162: gforge.inria.fr down Dec. 2020

  • Open
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Ludovic Courtès
  • zimoun
Owner
unassigned
Submitted by
zimoun
Severity
normal
Z
Z
zimoun wrote on 16 Sep 2020 10:14
Download
(address . guix-patches@gnu.org)
20200916081411.5801-1-zimon.toutoune@gmail.com
Download
Dear,

The first message in [1] explains the concrete issue. To avoid to pollute the
already long thread discussing long term Tarball Archiving (which will not be
ready before the down), here is sent patches fixing the concrete issue.

Aside, note that the tarballs should be now ingested by Software Heritage via:


but the issue is to reach them; well see [2].


From [1], the non-archived packages (yet) are:

Toggle snippet (9 lines)
scheme@(guile-user)> ,pp (lset-difference eq? $7 $8)
$11 = (
#<package r-spams@2.6-2017-03-22 gnu/packages/statistics.scm:3931 7f632401a640>
#<package mpfi@1.5.4 gnu/packages/multiprecision.scm:158 7f632ee3adc0>
#<package gf2x@1.2 gnu/packages/algebra.scm:103 7f6323ea1280>
#<package gmp-ecm@7.0.4 gnu/packages/algebra.scm:658 7f6323eb4960>
#<package cmh@1.0 gnu/packages/algebra.scm:322 7f6323eb4dc0>)

However, some have migrated to gitlab:

- r-spams
- gf2x
- cmh

Note that repackage 'r-spams' from the Git checkout needs some love and is not
straightforward. (Not tried yet the 2 others).


The aim of switching the 2 packages from 'url-fetch' to 'svn-fetch' is
twofolds, test case for improving:

- "guix lint -c archival" (support 'svn' and 'hg')
- the fallback to SWH (for non-git VCS source)

WDYT?



All the best,
simon

PS:
I am not sure how to deal with <control@debbugs.gnu.org> to "clone" (split)
the bug #42162. That's why this one. :-)


zimoun (2):
gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.
gnu: gmp-ecm: Replace 'url-fetch' by 'svn-fetch'.

gnu/packages/algebra.scm | 28 +++++++++++++++++++++-------
gnu/packages/multiprecision.scm | 13 ++++++++-----
2 files changed, 29 insertions(+), 12 deletions(-)


base-commit: f5163902d7c3cfed5a97033f38e92d7158b19fa8
--
2.28.0
Z
Z
zimoun wrote on 16 Sep 2020 10:16
Download
[PATCH 2/2] gnu: gmp-ecm: Replace 'url-fetch' by 'svn-fetch'.
(address . 43442@debbugs.gnu.org)(name . zimoun)(address . zimon.toutoune@gmail.com)
20200916081642.6716-2-zimon.toutoune@gmail.com
Download

* gnu/packages/algebra.scm (gmp-ecm)[source]: Replace 'url-fetch' by
'svn-fetch'.
[natice-inputs]: Add requirements.
[arguments]: Adjust 'check phase to substitute correct path.
---
gnu/packages/algebra.scm | 28 +++++++++++++++++++++-------
1 file changed, 21 insertions(+), 7 deletions(-)

Toggle diff (65 lines)
diff --git a/gnu/packages/algebra.scm b/gnu/packages/algebra.scm
index 318d653618..1cf293f17a 100644
--- a/gnu/packages/algebra.scm
+++ b/gnu/packages/algebra.scm
@@ -13,6 +13,7 @@
;;; Copyright © 2020 Jakub Kądziołka <kuba@kadziolka.net>
;;; Copyright © 2020 Vincent Legoll <vincent.legoll@gmail.com>
;;; Copyright © 2020 Vinicius Monego <monego@posteo.net>
+;;; Copyright © 2020 Simon Tournier <zimon.toutoune@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -66,6 +67,7 @@
#:use-module (guix download)
#:use-module (guix git-download)
#:use-module (guix hg-download)
+ #:use-module (guix svn-download)
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix packages)
#:use-module (guix utils))
@@ -670,15 +672,20 @@ geometry and singularity theory.")
(define-public gmp-ecm
(package
(name "gmp-ecm")
- (version "7.0.4")
+ (version "7.0.4-3088")
(source (origin
- (method url-fetch)
- ;; Use the ‘Latest version’ link for a stable URI across releases.
- (uri (string-append "https://gforge.inria.fr/frs/download.php/"
- "latestfile/160/ecm-" version ".tar.gz"))
+ (method svn-fetch)
+ (uri
+ (svn-reference
+ (url "https://scm.gforge.inria.fr/anonscm/svn/ecm/trunk")
+ (revision 3088)))
(sha256 (base32
- "0hxs24c2m3mh0nq1zz63z3sb7dhy1rilg2s1igwwcb26x3pb7xqc"))))
+ "0g1jqgp9baqz4x9mzksch69i4kp3207d3wn56g9vnsmanrrb4qvb"))))
(build-system gnu-build-system)
+ (native-inputs
+ `(("autoconf" ,autoconf)
+ ("automake" ,automake)
+ ("libtool" ,libtool)))
(inputs
`(("gmp" ,gmp)))
(arguments
@@ -686,7 +693,14 @@ geometry and singularity theory.")
;; Disable specific assembly routines, which depend
;; on the subarchitecture of the build machine,
;; and use gmp instead.
- "--disable-asm-redc")))
+ "--disable-asm-redc")
+ #:phases (modify-phases %standard-phases
+ (add-before 'check 'replace-/bin/rm
+ (lambda _
+ (substitute*
+ (list "test.ecm")
+ (("/bin/rm") (which "rm")))
+ #t)))))
(synopsis "Integer factorization library using the elliptic curve method")
(description
"GMP-ECM factors integers using the elliptic curve method (ECM) as well
--
2.28.0
Z
Z
zimoun wrote on 16 Sep 2020 10:16
Download
[PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.
(address . 43442@debbugs.gnu.org)(name . zimoun)(address . zimon.toutoune@gmail.com)
20200916081642.6716-1-zimon.toutoune@gmail.com
Download

* gnu/packages/multiprecision.scm (mpfi)[source]: Replace 'url-fetch' by
'svn-fetch'.
---
gnu/packages/multiprecision.scm | 13 ++++++++-----
1 file changed, 8 insertions(+), 5 deletions(-)

Toggle diff (43 lines)
diff --git a/gnu/packages/multiprecision.scm b/gnu/packages/multiprecision.scm
index b3a5ec5894..f1b152d9aa 100644
--- a/gnu/packages/multiprecision.scm
+++ b/gnu/packages/multiprecision.scm
@@ -7,6 +7,7 @@
;;; Copyright © 2018, 2019 Tobias Geerinckx-Rice <me@tobias.gr>
;;; Copyright © 2018 Eric Bavier <bavier@member.fsf.org>
;;; Copyright © 2018, 2019 Efraim Flashner <efraim@flashner.co.il>
+;;; Copyright © 2020 Simon Tournier <zimon.toutoune@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -32,6 +33,7 @@
#:use-module (gnu packages texinfo)
#:use-module (guix packages)
#:use-module (guix download)
+ #:use-module (guix svn-download)
#:use-module (guix utils)
#:use-module (guix build-system gnu))
@@ -157,14 +159,15 @@ precision and correctly rounds the results.")
(define-public mpfi
(package
(name "mpfi")
- (version "1.5.4")
+ (version "1.5.4-688")
(source
(origin
- (method url-fetch)
- (uri (string-append "https://gforge.inria.fr/frs/download.php"
- "/latestfile/181/mpfi-" version ".tgz"))
+ (method svn-fetch)
+ (uri (svn-reference
+ (url "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
+ (revision 688)))
(sha256
- (base32 "0mismr1ll3wp788dq2n22s5irm0dziy75byyfdwz22kjbmckhf9v"))))
+ (base32 "13pqkdvi8maj0mm76zh8p7jcc4c4569zh4kq336shh9cxlrh8qqx"))))
(build-system gnu-build-system)
(arguments
`(#:tests? #f ;tests are broken in this release
--
2.28.0
L
L
Ludovic Courtès wrote on 17 Sep 2020 10:14
Download
Re: [PATCH] Fixes init of #42162: gforge.inria.fr down Dec. 2020
(name . zimoun)(address . zimon.toutoune@gmail.com)
87imcczuol.fsf@inria.fr
Download
Hello!

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (6 lines)
> The aim of switching the 2 packages from 'url-fetch' to 'svn-fetch' is
> twofolds, test case for improving:
>
> - "guix lint -c archival" (support 'svn' and 'hg')
> - the fallback to SWH (for non-git VCS source)

Woow, it’s ambitious given that SWH/SVN integration is currently lacking
on our side. :-)

It’s a good idea to switch to gitlab.inria.fr for those that have
migrated, though.

Thanks,
Ludo’.
L
L
Ludovic Courtès wrote on 21 Sep 2020 23:19
Download
Re: [bug#43442] [PATCH 1/2] gnu: mpfi: Replace 'url-fetch' by 'svn-fetch'.
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 43442@debbugs.gnu.org)
87o8lyj090.fsf@gnu.org
Download
Hi!

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (5 lines)
>
> * gnu/packages/multiprecision.scm (mpfi)[source]: Replace 'url-fetch' by
> 'svn-fetch'.

[...]

Toggle quote (5 lines)
> + (method svn-fetch)
> + (uri (svn-reference
> + (url "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
> + (revision 688)))

Does this have any chance of working with SWH? Does their HTTP API
support looking up svn revision for a given origin URL?

Also:

scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
$6 = #f

So the effect today would be the opposite of what we’re attempting: when
GForge goes down, we’d lose it completely.

Thoughts?

Ludo’.
Z
Z
zimoun wrote on 21 Sep 2020 23:51
Download
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 43442@debbugs.gnu.org)
86zh5iol1n.fsf@gmail.com
Download
Hi,

On Mon, 21 Sep 2020 at 23:19, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (8 lines)
>> + (method svn-fetch)
>> + (uri (svn-reference
>> + (url "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
>> + (revision 688)))
>
> Does this have any chance of working with SWH? Does their HTTP API
> support looking up svn revision for a given origin URL?

Well, I have not yet did my homework on the topic, neither asked on
#swh-devel.

Probably not!

In any case, we should find a way because there are a lot of Subversion
sources.


Toggle quote (5 lines)
> Also:
>
> scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
> $6 = #f

Since “guix lint -c archival” I should have forgotten to submit the
request via their Web interface. Now, it is scheduled:



Toggle quote (3 lines)
> So the effect today would be the opposite of what we’re attempting: when
> GForge goes down, we’d lose it completely.

I agree.


Well, the shutdown of the INRIA’s GForge is a concrete deadline to add
the support of Subversion (and Mercurial) sources to:

- fallback to SWH
- guix lint -c archival

I am working on it. :-)


All the best,
simon
L
L
Ludovic Courtès wrote on 23 Sep 2020 18:21
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 43442@debbugs.gnu.org)
87h7rocvkt.fsf@gnu.org
Download
Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (18 lines)
> On Mon, 21 Sep 2020 at 23:19, Ludovic Courtès <ludo@gnu.org> wrote:
>
>>> + (method svn-fetch)
>>> + (uri (svn-reference
>>> + (url "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
>>> + (revision 688)))
>>
>> Does this have any chance of working with SWH? Does their HTTP API
>> support looking up svn revision for a given origin URL?
>
> Well, I have not yet did my homework on the topic, neither asked on
> #swh-devel.
>
> Probably not!
>
> In any case, we should find a way because there are a lot of Subversion
> sources.

Yes, we should implement it. But until that is done, I’m not sure it’s
worth replacing tarballs with svn references. WDYT?

Toggle quote (10 lines)
>> Also:
>>
>> scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
>> $6 = #f
>
> Since “guix lint -c archival” I should have forgotten to submit the
> request via their Web interface. Now, it is scheduled:
>
> <https://archive.softwareheritage.org/save/#requests>

‘lookup-origin’ still returns #f though.

Toggle quote (14 lines)
>> So the effect today would be the opposite of what we’re attempting: when
>> GForge goes down, we’d lose it completely.
>
> I agree.
>
>
> Well, the shutdown of the INRIA’s GForge is a concrete deadline to add
> the support of Subversion (and Mercurial) sources to:
>
> - fallback to SWH
> - guix lint -c archival
>
> I am working on it. :-)

Awesome.

(Note that it seems possible that the shutdown of gforge.inria.fr will
be delayed a bit or at least be made less brutally that initially
announced.)

Thanks,
Ludo’.
Z
Z
zimoun wrote on 23 Sep 2020 19:07
Download
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 43442@debbugs.gnu.org)
CAJ3okZ175Tjr9ZjfatMDaogNT1guMZBfF+U1eGssUj+2_xtvww@mail.gmail.com
Download
On Wed, 23 Sep 2020 at 18:22, Ludovic Courtès <ludo@gnu.org> wrote:
Toggle quote (23 lines)
> zimoun <zimon.toutoune@gmail.com> skribis:
>
> > On Mon, 21 Sep 2020 at 23:19, Ludovic Courtès <ludo@gnu.org> wrote:
> >
> >>> + (method svn-fetch)
> >>> + (uri (svn-reference
> >>> + (url "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
> >>> + (revision 688)))
> >>
> >> Does this have any chance of working with SWH? Does their HTTP API
> >> support looking up svn revision for a given origin URL?
> >
> > Well, I have not yet did my homework on the topic, neither asked on
> > #swh-devel.
> >
> > Probably not!
> >
> > In any case, we should find a way because there are a lot of Subversion
> > sources.
>
> Yes, we should implement it. But until that is done, I’m not sure it’s
> worth replacing tarballs with svn references. WDYT?

I do not have a strong opinion.

Toggle quote (12 lines)
> >> Also:
> >>
> >> scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/trunk/mpfi")
> >> $6 = #f
> >
> > Since “guix lint -c archival” I should have forgotten to submit the
> > request via their Web interface. Now, it is scheduled:
> >
> > <https://archive.softwareheritage.org/save/#requests>
>
> ‘lookup-origin’ still returns #f though.

Weird because SWH says it is in. Requests at 21/09/2020, 23:47:09 then
click. :-)

- scheduled: 21/09/2020, 23:47:21
- start: 21/09/2020, 23:47:24
- end: 21/09/2020, 23:53:26
- log: [2020-09-21 21:53:26,098: INFO/ForkPoolWorker-1] Task
swh.loader.svn.tasks.DumpMountAndLoadSvnRepository[b13942f6-b415-4831-b25f-e68bd4515cf2]
succeeded in 361.4525023885071s: {'status': 'eventful'}

Well, the issue is:



as you can see here:



Toggle quote (4 lines)
> (Note that it seems possible that the shutdown of gforge.inria.fr will
> be delayed a bit or at least be made less brutally that initially
> announced.)

Good to know. :-)

All the best,
simon
L
L
Ludovic Courtès wrote on 25 Sep 2020 10:56
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 43442@debbugs.gnu.org)
877dsi44lu.fsf@gnu.org
Download
Hi!

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (4 lines)
> as you can see here:
>
> <https://archive.softwareheritage.org/browse/origin/directory/?origin_url=https://scm.gforge.inria.fr/anonscm/svn/mpfi>

Oh indeed:

Toggle snippet (9 lines)
scheme@(guile-user)> ,use(guix swh)
scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi")
$14 = #<<origin> visits-url: "https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visits/" type: #f url: "https://scm.gforge.inria.fr/anonscm/svn/mpfi">
scheme@(guile-user)> (origin-visits $14)
$15 = (#<<visit> date: #<date nanosecond: 902765 second: 25 minute: 53 hour: 21 day: 21 month: 9 year: 2020 zone-offset: 0> origin: "https://scm.gforge.inria.fr/anonscm/svn/mpfi" url: "https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visit/1/" snapshot-url: "https://archive.softwareheritage.org/api/1/snapshot/e7fdd4dc6230f710dbc55c1b308804fa1b5f51f0/" status: full number: 1>)
scheme@(guile-user)> (visit-snapshot (car $15))
$16 = #<<snapshot> branches: (#<<branch> name: "HEAD" target-type: revision target-url: "https://archive.softwareheritage.org/api/1/revision/f7b445a6bdc38bf075f29265120ca49824f698ea/">)>

Note that the original svn revision number is discarded, AFAICS.

Ludo’.
Z
Z
zimoun wrote on 1 Oct 2020 22:26
Download
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 43442@debbugs.gnu.org)
CAJ3okZ1i4E0iKM1nYCtQ2N+hD2tE7=hcOAUihuay-x0bZQmLeQ@mail.gmail.com
Download
Hi Ludo,

On Fri, 25 Sep 2020 at 10:56, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (2 lines)
> Note that the original svn revision number is discarded, AFAICS.

No, it is not... but to find it one needs a bit of guidance by SWH folks. :-)

1. With the URL, fetch:

2. From #1 one gets 'snapshot' and fetch:

3. From #3, one gets the 'target' and fetch:

And #3 lists all the revisions in SWH.

Well, this should be enough for the saving request. WDYT?


However, it is not clear how to fetch cooking from the vault. To be
continued...

Cheers,
simon
Z
Z
zimoun wrote on 1 Oct 2020 23:01
Download
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 43442@debbugs.gnu.org)
CAJ3okZ1+SZbkqRkAe1Q6uAHoHrAugp99aouRKYFkc=98z1h_-Q@mail.gmail.com
Download
On Thu, 1 Oct 2020 at 22:26, zimoun <zimon.toutoune@gmail.com> wrote:

Toggle quote (17 lines)
> 1. With the URL, fetch:
> https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visit/latest/
>
> 2. From #1 one gets 'snapshot' and fetch:
> https://archive.softwareheritage.org/api/1/snapshot/e7fdd4dc6230f710dbc55c1b308804fa1b5f51f0/
>
> 3. From #3, one gets the 'target' and fetch:
> https://archive.softwareheritage.org/api/1/revision/f7b445a6bdc38bf075f29265120ca49824f698ea/log/
>
> And #3 lists all the revisions in SWH.
>
> Well, this should be enough for the saving request. WDYT?
>

> However, it is not clear how to fetch cooking from the vault. To be
> continued...

From #3, using 'directory' (4193e93b4aea0abeaff715b1a1274c4d7eb4a27d),
it seems possible to cook from the vault and get the tarball (I did
with the web interface). And the integrity field (guix hash -r) is
the same as the one I put in the patch submission.
L
L
Ludovic Courtès wrote on 3 Oct 2020 10:59
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 43442@debbugs.gnu.org)
87y2knhei3.fsf@gnu.org
Download
Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (17 lines)
> On Fri, 25 Sep 2020 at 10:56, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> Note that the original svn revision number is discarded, AFAICS.
>
> No, it is not... but to find it one needs a bit of guidance by SWH folks. :-)
>
> 1. With the URL, fetch:
> https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visit/latest/
>
> 2. From #1 one gets 'snapshot' and fetch:
> https://archive.softwareheritage.org/api/1/snapshot/e7fdd4dc6230f710dbc55c1b308804fa1b5f51f0/
>
> 3. From #3, one gets the 'target' and fetch:
> https://archive.softwareheritage.org/api/1/revision/f7b445a6bdc38bf075f29265120ca49824f698ea/log/
>
> And #3 lists all the revisions in SWH.

Ah yes, under “extra_headers” there’s the SVN revision number. (guix
swh) doesn’t expose “extra_headers” yet, but once it does, we can walk
snapshots until we find the SVN revision we’re looking for.

Toggle snippet (8 lines)
scheme@(guile-user)> (lookup-origin "https://scm.gforge.inria.fr/anonscm/svn/mpfi/")
$3 = #<<origin> visits-url: "https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visits/" type: #f url: "https://scm.gforge.inria.fr/anonscm/svn/mpfi">
scheme@(guile-user)> (origin-visits $3)
$4 = (#<<visit> date: #<date nanosecond: 902765 second: 25 minute: 53 hour: 21 day: 21 month: 9 year: 2020 zone-offset: 0> origin: "https://scm.gforge.inria.fr/anonscm/svn/mpfi" url: "https://archive.softwareheritage.org/api/1/origin/https://scm.gforge.inria.fr/anonscm/svn/mpfi/visit/1/" snapshot-url: "https://archive.softwareheritage.org/api/1/snapshot/e7fdd4dc6230f710dbc55c1b308804fa1b5f51f0/" status: full number: 1>)
scheme@(guile-user)> (visit-snapshot (car $4))
$5 = #<<snapshot> branches: (#<<branch> name: "HEAD" target-type: revision target-url: "https://archive.softwareheritage.org/api/1/revision/f7b445a6bdc38bf075f29265120ca49824f698ea/">)>

So the next step is to augment (guix swh) with a
‘lookup-subversion-revision’ procedure.

Thanks for investigating!

Ludo’.
?