‘guix lint -c archival python-scikit-learn’ crashes

  • Open
  • quality assurance status badge
Details
3 participants
  • Antoine Lambert
  • Ludovic Courtès
  • Simon Tournier
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 12 Feb 11:11 +0100
‘guix lint -c archival python-sciki t-learn’ crashes
(address . bug-guix@gnu.org)
87v86uf0uw.fsf@inria.fr
Hi!

Starting from a couple of weeks ago, probably around the time SWH
deployed an updated version of their infrastructure, we get crashes like
this:

Toggle snippet (34 lines)
$ guix describe
Generation 290 Feb 11 2024 23:01:35 (current)
guix c4372f7
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: c4372f7ebfac17236299346fe60c6ca125336b1f
$ guix lint -c archival python-scikit-learn
Backtrace:ython-scikit-learn@1.3.2 [archival]...
12 (primitive-load "/home/ludo/.config/guix/current/bin/gu…")
In guix/ui.scm:
2324:7 11 (run-guix . _)
2287:10 10 (run-guix-command _ . _)
In ice-9/boot-9.scm:
1752:10 9 (with-exception-handler _ _ #:unwind? _ # _)
In srfi/srfi-1.scm:
634:9 8 (for-each #<procedure 7f45354e9400 at guix/scripts/lin…> …)
In guix/scripts/lint.scm:
65:4 7 (run-checkers _ _ #:store _)
In srfi/srfi-1.scm:
634:9 6 (for-each #<procedure 7f4535405d50 at guix/scripts/lin…> …)
In guix/scripts/lint.scm:
74:21 5 (_ _)
In guix/lint.scm:
1662:4 4 (check-archival _)
877:2 3 (call-with-networking-fail-safe _ _ _)
In ice-9/boot-9.scm:
1752:10 2 (with-exception-handler _ _ #:unwind? _ # _)
1685:16 1 (raise-exception _ #:continuable? _)
1683:16 0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1683:16: In procedure raise-exception:
Throw to key `match-error' with args `("match" "no matching pattern" directory)'.

This can be reproduced by merely evaluating:

(lookup-origin-revision "https://github.com/scikit-learn/scikit-learn""1.3.2")

The crux of the problem is that ‘snapshot-branches’ used to return
either a “revision” or a “release”, but now it can also return a
“directory”:

Toggle snippet (19 lines)
$ wget -qO- https://archive.softwareheritage.org/api/1/snapshot/c405490b80fb35cc58b34260d9ec6add8248857f |jq
{
"id": "c405490b80fb35cc58b34260d9ec6add8248857f",
"branches": {
"1.3.2": {
"target": "e4a4be18fae8d9c6528abff3bc9088feb19a76c7",
"target_type": "directory",
"target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
},
"HEAD": {
"target": "1.3.2",
"target_type": "alias",
"target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
}
},
"next_branch": null
}

I’ll check with comrades at SWH whether this is intended.

(Note that this problem may vanish once

Ludo’.
L
L
Ludovic Courtès wrote on 12 Feb 17:04 +0100
No releases/revisions in /snapshot endpoints?
(address . swh-devel@inria.fr)(address . 69070@debbugs.gnu.org)
87il2td5yc.fsf@gnu.org
Dear SWH,

In https://issues.guix.gnu.org/69070 I stumbled upon what might be new
behavior from the /snapshot endpoint:

Toggle quote (22 lines)
> The crux of the problem is that ‘snapshot-branches’ used to return
> either a “revision” or a “release”, but now it can also return a
> “directory”:
>
> $ wget -qO- https://archive.softwareheritage.org/api/1/snapshot/c405490b80fb35cc58b34260d9ec6add8248857f |jq
> {
> "id": "c405490b80fb35cc58b34260d9ec6add8248857f",
> "branches": {
> "1.3.2": {
> "target": "e4a4be18fae8d9c6528abff3bc9088feb19a76c7",
> "target_type": "directory",
> "target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
> },
> "HEAD": {
> "target": "1.3.2",
> "target_type": "alias",
> "target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
> }
> },
> "next_branch": null
> }

Current documentation mentions the ‘target_types’ query parameter, which
I could use to restrict results to “revision” and “release”:


However, that’d be the empty list of the case above.

Other snapshots seem to be missing releases and revisions as well, such
as this one:


Did anything change here? Am I holding it wrong? :-)

Thanks in advance,
Ludo’.
A
A
Antoine Lambert wrote on 12 Feb 17:37 +0100
f11611f7-9f37-42b1-8e58-726d8223ffb0@inria.fr
Hello Ludovic,

The SWH data model also allows for snapshot branches to target simple directories.

The new loaders we implemented to ingest a tarball content (visit type: tarball-directory)
or a directory targeted by a CVS tag or commit (visit types: git-checkout, hg-checkout)
produce such snapshot with a single non alias branch targeting a directory.

We do not produce revisions or releases with these loaders as their purpose is to
archive a single directory (with extid mapping) without any development history.

Antoine Lambert

On 12/02/2024 17:04, Ludovic Courtès wrote:
Toggle quote (43 lines)
> Dear SWH,
>
> In <https://issues.guix.gnu.org/69070> I stumbled upon what might be new
> behavior from the /snapshot endpoint:
>
>> The crux of the problem is that ‘snapshot-branches’ used to return
>> either a “revision” or a “release”, but now it can also return a
>> “directory”:
>>
>> $ wget -qO- https://archive.softwareheritage.org/api/1/snapshot/c405490b80fb35cc58b34260d9ec6add8248857f |jq
>> {
>> "id": "c405490b80fb35cc58b34260d9ec6add8248857f",
>> "branches": {
>> "1.3.2": {
>> "target": "e4a4be18fae8d9c6528abff3bc9088feb19a76c7",
>> "target_type": "directory",
>> "target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
>> },
>> "HEAD": {
>> "target": "1.3.2",
>> "target_type": "alias",
>> "target_url": "https://archive.softwareheritage.org/api/1/directory/e4a4be18fae8d9c6528abff3bc9088feb19a76c7/"
>> }
>> },
>> "next_branch": null
>> }
> Current documentation mentions the ‘target_types’ query parameter, which
> I could use to restrict results to “revision” and “release”:
>
> https://archive.softwareheritage.org/api/1/snapshot/doc/
>
> However, that’d be the empty list of the case above.
>
> Other snapshots seem to be missing releases and revisions as well, such
> as this one:
>
> https://archive.softwareheritage.org/api/1/snapshot/55bda9214f962550baad25026fb3ac3ad12e3eb9/
>
> Did anything change here? Am I holding it wrong? :-)
>
> Thanks in advance,
> Ludo’.

--
Antoine Lambert
Software engineer for Software Heritage
Inria, Paris
S
S
Simon Tournier wrote on 10 Sep 15:58 +0200
Re: bug#69070: ‘guix lint -c archival python-scikit-learn’ crashes
(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)(address . 69070@debbugs.gnu.org)
87h6anehl0.fsf@gmail.com
Hi Ludo,

On Mon, 12 Feb 2024 at 11:11, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:

Toggle quote (3 lines)
> (Note that this problem may vanish once
> <https://issues.guix.gnu.org/68741> has been merged.)

This had been merged. Now, it does not crash. Is the problem hidden or
vanished?

Cheers,
simon
L
L
Ludovic Courtès wrote on 12 Sep 10:31 +0200
(name . Simon Tournier)(address . zimon.toutoune@gmail.com)(address . 69070@debbugs.gnu.org)
87wmjhjmru.fsf@inria.fr
Hi Simon,

Simon Tournier <zimon.toutoune@gmail.com> skribis:

Toggle quote (8 lines)
> On Mon, 12 Feb 2024 at 11:11, Ludovic Courtès <ludovic.courtes@inria.fr> wrote:
>
>> (Note that this problem may vanish once
>> <https://issues.guix.gnu.org/68741> has been merged.)
>
> This had been merged. Now, it does not crash. Is the problem hidden or
> vanished?

I think the initial problem (that ‘snapshot-branches’ returns a
directory) is still there, although I cannot reproduce it right now
because SWH returns 500 currently:

Toggle snippet (14 lines)
scheme@(guile-user)> ,use(guix swh)
scheme@(guile-user)> (lookup-origin-revision "https://github.com/scikit-learn/scikit-learn" "1.3.2")
ice-9/boot-9.scm:1676:22: In procedure raise-exception:
Throw to key `swh-error' with args `("https://archive.softwareheritage.org/api/1/origin/https://github.com/scikit-learn/scikit-learn/visits/?per_page=30" #<procedure http-get* (uri . rest)> #<<response> version: (1 . 1) code: 500 reason-phrase: "Internal Server Error" headers: ((date . #<date nanosecond: 0 second: 48 minute: 28 hour: 8 day: 12 month: 9 year: 2024 zone-offset: 0>) (content-type application/json) (content-length . 130) (vary accept origin cookie) (allow HEAD OPTIONS GET OPTIONS) (x-ratelimit-limit . "120") (x-ratelimit-remaining . "118") (x-ratelimit-reset . "1726133328") (x-frame-options . "DENY") (x-content-type-options . "nosniff") (referrer-policy . "same-origin") (cross-origin-opener-policy . "same-origin") (x-varnish . "665128879") (age . 0) (via "1.1 varnish (Varnish/6.5)") (strict-transport-security . "max-age=15768000;") (connection close)) port: #<closed: string 7f0c8c5aea10>>)'.

Entering a new prompt. Type `,bt' for a backtrace or `,q' to continue.
scheme@(guile-user) [1]> ,bt
In guix/swh.scm:
579:11 2 (lookup-origin-revision _ _)
296:16 1 (call "https://archive.softwareheritage.org/api/1/origin/https://github.com/scikit-learn/scikit-learn/visits/?per_page=30" _ _ #:false-if-404? _)
In ice-9/boot-9.scm:
1676:22 0 (raise-exception _ #:continuable? _)

That said, that code is much less useful now that
https://issues.guix.gnu.org/68741 has been merged, so perhaps we could
close the issue?

Thanks,
Ludo’.
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 69070@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 69070
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch