Software Heritage (swh): Fix API change, causing repeatedly submitting archives

  • Done
  • quality assurance status badge
Details
3 participants
  • Björn Höfling
  • Jonathan Brielmaier
  • Ludovic Courtès
Owner
unassigned
Submitted by
Björn Höfling
Severity
normal
B
B
Björn Höfling wrote on 15 Dec 2019 22:34
(address . guix-patches@gnu.org)
20191215223433.5ce5be20@alma-ubu
Hi Guix,

when linting, I recognized that the archival linter was repeatedly
sending packages to the SWH, despite that the package was already
archived.

With some investigations, I noticed they changed the API. Here's a
patch that fixes it.

I have not communicated with SWH about that, especially I have not asked
SWH why they broke the API without updating the version number.

Björn
From a5ada85d168b45d517d8faf0d79ad5beccc26911 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Bj=C3=B6rn=20H=C3=B6fling?=
<bjoern.hoefling@bjoernhoefling.de>
Date: Sun, 15 Dec 2019 22:00:56 +0100
Subject: [PATCH] swh: Fix API call for getting origin.

When using the archival linter, git origins already in the archive where
not recognized due to an API change and where repeatedly asked for archival.
This is fixed here.

* guix/swh.scm (lookup-origin): Fix API URI for getting origin.
(<origin>): Fix comment with API URI example.
---
guix/swh.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (24 lines)
diff --git a/guix/swh.scm b/guix/swh.scm
index 7acad05928..372e4c84d1 100644
--- a/guix/swh.scm
+++ b/guix/swh.scm
@@ -244,7 +244,7 @@ FALSE-IF-404? is true, return #f upon 404 responses."
docstring
(call (swh-url components ...) json->value)))))
-;; <https://archive.softwareheritage.org/api/1/origin/git/url/https://github.com/guix-mirror/guix/>
+;; <https://archive.softwareheritage.org/api/1/origin//https://github.com/guix-mirror/guix/get>
(define-json-mapping <origin> make-origin origin?
json->origin
(id origin-id)
@@ -365,7 +365,7 @@ FALSE-IF-404? is true, return #f upon 404 responses."
(define-query (lookup-origin url)
"Return an origin for URL."
- (path "/api/1/origin/git/url" url)
+ (path "/api/1/origin" url "get")
json->origin)
(define-query (lookup-content hash type)
--
2.23.0
-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQQiGUP0np8nb5SZM4K/KGy2WT5f/QUCXfam6QAKCRC/KGy2WT5f
/fenAJ9bGb6G8EjV7iP6S4BMKtD2zH5jfQCbBYi4fjTU290htYpCOSewZiFv5Jk=
=JIMw
-----END PGP SIGNATURE-----


L
L
Ludovic Courtès wrote on 17 Dec 2019 17:41
(name . Björn Höfling)(address . bjoern.hoefling@bjoernhoefling.de)(address . 38630@debbugs.gnu.org)
87h81yrjkv.fsf@gnu.org
Hello!

Björn Höfling <bjoern.hoefling@bjoernhoefling.de> skribis:

Toggle quote (7 lines)
> when linting, I recognized that the archival linter was repeatedly
> sending packages to the SWH, despite that the package was already
> archived.
>
> With some investigations, I noticed they changed the API. Here's a
> patch that fixes it.

Good catch!

Toggle quote (3 lines)
> I have not communicated with SWH about that, especially I have not asked
> SWH why they broke the API without updating the version number.

Yeah, that’s weird, I’ve pinged them on IRC.

It seems that the removal was intentional as part of
https://forge.softwareheritage.org/D2167, but it wasn’t clear whether
the API breakage should have happened. This led them to open a new
task: https://forge.softwareheritage.org/T2158. We should keep an eye
on it.

Toggle quote (13 lines)
> From a5ada85d168b45d517d8faf0d79ad5beccc26911 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Bj=C3=B6rn=20H=C3=B6fling?=
> <bjoern.hoefling@bjoernhoefling.de>
> Date: Sun, 15 Dec 2019 22:00:56 +0100
> Subject: [PATCH] swh: Fix API call for getting origin.
>
> When using the archival linter, git origins already in the archive where
> not recognized due to an API change and where repeatedly asked for archival.
> This is fixed here.
>
> * guix/swh.scm (lookup-origin): Fix API URI for getting origin.
> (<origin>): Fix comment with API URI example.

In the meantime, your patch looks like the right move, so you can push it.

Thank you!

Ludo’.
B
B
Björn Höfling wrote on 17 Dec 2019 23:32
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 38630-done@debbugs.gnu.org)
20191217233240.09d4166a@alma-ubu
On Tue, 17 Dec 2019 17:41:04 +0100
Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (12 lines)
> > I have not communicated with SWH about that, especially I have not
> > asked SWH why they broke the API without updating the version
> > number.
>
> Yeah, that’s weird, I’ve pinged them on IRC.
>
> It seems that the removal was intentional as part of
> <https://forge.softwareheritage.org/D2167>, but it wasn’t clear
> whether the API breakage should have happened. This led them to open
> a new task: <https://forge.softwareheritage.org/T2158>. We should
> keep an eye on it.

I will be frequently linting and see what's happening :-)

[..]

Toggle quote (3 lines)
> In the meantime, your patch looks like the right move, so you can
> push it.

Thanks for your review and further information, pushed as

356a79becc4061d158c68718ad169abac1ab672f

Closing this ticket.

Björn
-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQQiGUP0np8nb5SZM4K/KGy2WT5f/QUCXflXiAAKCRC/KGy2WT5f
/e0ZAKC1v1xhBRgrZXBJBZ3Wcebj7HH99QCeJjpqgcd0IohBPDvsQ1ewmoKBArg=
=/YrK
-----END PGP SIGNATURE-----


Closed
J
J
Jonathan Brielmaier wrote on 18 Dec 2019 10:22
Re: bug#38630: Software Heritage (swh): Fix API change, causing repeatedly submitting archives
322949b2-93fe-d741-3208-d5c0c30c7511@web.de
On 17.12.19 23:32, Björn Höfling wrote:
Toggle quote (6 lines)
> Thanks for your review and further information, pushed as
>
> 356a79becc4061d158c68718ad169abac1ab672f
>
> Closing this ticket.

I guess the "origin//" in the comment should be "origin/" :)
B
B
Björn Höfling wrote on 18 Dec 2019 22:44
(name . Jonathan Brielmaier)(address . jonathan.brielmaier@web.de)(address . 38630-done@debbugs.gnu.org)
20191218224459.72594029@alma-ubu
On Wed, 18 Dec 2019 10:22:27 +0100
Jonathan Brielmaier <jonathan.brielmaier@web.de> wrote:

Toggle quote (9 lines)
> On 17.12.19 23:32, Björn Höfling wrote:
> > Thanks for your review and further information, pushed as
> >
> > 356a79becc4061d158c68718ad169abac1ab672f
> >
> > Closing this ticket.
>
> I guess the "origin//" in the comment should be "origin/" :)

Hi Jonathan,

thanks for that hint, fixed in:

6afea7489b76c8db58d4f389fdbedc7c2b8992bd

Björn
-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQQiGUP0np8nb5SZM4K/KGy2WT5f/QUCXfqd2wAKCRC/KGy2WT5f
/c6pAJ4uP8AHtQNWkVm6478QeyaoLzqASwCgnsmUHclTDs5UC/mdceNqs+FFmYM=
=KtXj
-----END PGP SIGNATURE-----


Closed
J
J
Jonathan Brielmaier wrote on 20 Dec 2019 01:48
(name . Björn Höfling)(address . bjoern.hoefling@bjoernhoefling.de)(address . 38630-done@debbugs.gnu.org)
18a2a404-6394-4578-8783-114239497d1d@web.de
On 18.12.19 22:44, Björn Höfling wrote:
Toggle quote (18 lines)
> On Wed, 18 Dec 2019 10:22:27 +0100
> Jonathan Brielmaier <jonathan.brielmaier@web.de> wrote:
>
>> On 17.12.19 23:32, Björn Höfling wrote:
>>> Thanks for your review and further information, pushed as
>>>
>>> 356a79becc4061d158c68718ad169abac1ab672f
>>>
>>> Closing this ticket.
>>
>> I guess the "origin//" in the comment should be "origin/" :)
>
> Hi Jonathan,
>
> thanks for that hint, fixed in:
>
> 6afea7489b76c8db58d4f389fdbedc7c2b8992bd

Now we lost the h in https, but it's only a comment so not really
important :)
Closed
B
B
Björn Höfling wrote on 26 Dec 2019 00:27
Re: guix: swh: Fix example URI in comment for <origin>.
(name . Mark H Weaver)(address . mhw@netris.org)
20191226002723.5a6e448b@alma-ubu
Hi Mark and Jonathan,

On Wed, 18 Dec 2019 21:42:38 -0500
Mark H Weaver <mhw@netris.org> wrote:

Toggle quote (7 lines)
> Hi Björn,
>
> Your recent commit 6afea7489b76c8db58d4f389fdbedc7c2b8992bd is
> erroneous. It claims to remove a slash from an example URI, but it
> actually removed both a slash and the 'h' from 'https'. See the diff
> below.

What a shame. Thanks for looking into the details.

I hope that:

e507d30c482cf018b44e70931a6153e5d3ea93f1

is the last in that series :-)

Björn
-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQQiGUP0np8nb5SZM4K/KGy2WT5f/QUCXgPwWwAKCRC/KGy2WT5f
/X+yAJwP6tZU/AMg8pLlld8peNrrBmmkcACgqo6gBPhwOa8hawit4TMzVx/H/0w=
=mguR
-----END PGP SIGNATURE-----


Closed
?