[PATCH 0/3] Update public-inbox to 1.8.0

  • Done
  • quality assurance status badge
Details
3 participants
  • Thiago Jung Bauermann
  • Kyle Meyer
  • Ludovic Courtès
Owner
unassigned
Submitted by
Thiago Jung Bauermann
Severity
normal
T
T
Thiago Jung Bauermann wrote on 15 May 2022 23:54
(address . guix-patches@gnu.org)(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
20220515215420.503806-1-bauermann@kolabnow.com
Hello,

These patches fix a few issues with the current public-inbox package, add a
new package which is needed by version 1.8.0, and finally updates
public-inbox.

A few notes regarding the new package perl-socket-msghdr:

- It's based on the package generated by the CPAN importer. As such, the
origin URL is on CPAN. There's an upstream repo on GitHub¹, but it
appears inactive. The contents there are identical to the CPAN tarball,
modulo the files META.json, META.yml and MANIFEST (the former two exist
only on CPAN, and the latter is changed to include them) so I left the
origin unchanged.

- There's a file called ‘ppport.h’ with 7k lines that was automatically
generated by some tool called “Devel::PPPort”. Ideally we should delete
it and regenerate it, but I haven't done that.

And regarding public-inbox 1.8.0:

- The diff from current version 1.6.1 has about 57k lines (including
context). I didn't do even a cursory review of the changes.

- The new tool ‘lei’ depends on curl to work. I tried substituting its path
directly in the source code, but for some reason that didn't work.
Therefore I settled for adding curl to the PATH with ‘program-wrapper’.

- The tests are disabled because more changes are needed to enable them.
I have them working in a local branch, but there's one patch to the
public-inbox testsuite which I still need to cleanup and post upstream.
If anyone is curious about it, I pushed the branch ‘public-inbox-updates’
to my GitLab repo².

Thiago Jung Bauermann (3):
gnu: public-inbox: Some packaging fixes
gnu: Add perl-socket-msghdr
gnu: public-inbox: Update to version 1.8.0

gnu/packages/mail.scm | 38 +++++++++++++++++++++-----------------
gnu/packages/perl.scm | 24 ++++++++++++++++++++++++
2 files changed, 45 insertions(+), 17 deletions(-)

base-commit: 527952945c59f69665e40b6b416141554084e057

--

T
T
Thiago Jung Bauermann wrote on 16 May 2022 00:02
[PATCH 1/3] gnu: public-inbox: Some packaging fixes
(address . 55436@debbugs.gnu.org)(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
20220515220230.504057-1-bauermann@kolabnow.com
The file ‘script/public-inbox-compact’ doesn't call ‘xapian-compact’ directly
anymore, so the substitution in the ‘qualify-paths’ phase doesn't do anything.

Now there's a reference to ‘xapian-compact’ in ‘lib/PublicInbox/Xapcmd.pm’,
so substitute the path there.

And move xapian from native-inputs to inputs and use its input path in the
substitution, since ‘xapian-compact’ is called during normal execution of
public-inbox.

Also add ‘bash-minimal’ to the package's inputs, since it uses ‘wrap-program’.

Finally, take the opportunity to change build phases to not return #t
unconditionally, since that isn't necessary anymore.

* gnu/packages/mail.scm (public-inbox)[arguments]<#:phases>: Don't return #t
from each phase.
{qualify-paths}: Substitute ‘xapian-compact’ path in the correct file. Use
‘search-input-file’ rather than ‘which’ to find it.
[native-inputs]: Move ‘xapian’ from here ...
[inputs]: ... to here. Add ‘bash-minimal’.
---
gnu/packages/mail.scm | 26 +++++++++++++-------------
1 file changed, 13 insertions(+), 13 deletions(-)

Toggle diff (69 lines)
diff --git a/gnu/packages/mail.scm b/gnu/packages/mail.scm
index c4ccc53e84ef..0728d65d1d10 100644
--- a/gnu/packages/mail.scm
+++ b/gnu/packages/mail.scm
@@ -47,6 +47,7 @@
;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
;;; Copyright © 2022 Andrew Tropin <andrew@trop.in>
;;; Copyright © 2022 Justin Veilleux <terramorpha@cock.li>
+;;; Copyright © 2022 Thiago Jung Bauermann <bauermann@kolabnow.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -4101,12 +4102,12 @@ (define-public public-inbox
'(#:phases
(modify-phases %standard-phases
(add-before 'configure 'qualify-paths
- (lambda _
+ (lambda* (#:key inputs #:allow-other-keys)
;; Use absolute paths for 'xapian-compact'.
- (let ((xapian-compact (which "xapian-compact")))
- (substitute* "script/public-inbox-compact"
- (("xapian-compact") xapian-compact)))
- #t))
+ (substitute* "lib/PublicInbox/Xapcmd.pm"
+ (("'xapian-compact'")
+ (format #f "'~a'" (search-input-file inputs
+ "/bin/xapian-compact"))))))
(add-before 'check 'pre-check
(lambda _
(substitute* "t/spawn.t"
@@ -4117,8 +4118,7 @@ (define-public public-inbox
;; XXX: This test fails due to zombie process is not reaped by
;; the builder.
(substitute* "t/httpd-unix.t"
- (("^SKIP: \\{") "SKIP: { skip('Guix');"))
- #t))
+ (("^SKIP: \\{") "SKIP: { skip('Guix');"))))
(add-after 'install 'wrap-programs
(lambda* (#:key inputs outputs #:allow-other-keys)
(let ((out (assoc-ref outputs "out")))
@@ -4133,14 +4133,13 @@ (define-public public-inbox
;; perl module.
`("PATH" ":" prefix
(,(dirname (search-input-file inputs "/bin/git"))))))
- (find-files (string-append out "/bin"))))
- #t)))))
+ (find-files (string-append out "/bin")))))))))
(native-inputs
- (list xapian
- ;; For testing.
+ (list ;; For testing.
lsof openssl))
(inputs
- (list git
+ (list bash-minimal
+ git
perl-dbd-sqlite
perl-dbi
perl-email-address-xs
@@ -4162,7 +4161,8 @@ (define-public public-inbox
;; ("highlight" ,highlight)
;; For testing.
perl-ipc-run
- perl-xml-feed))
+ perl-xml-feed
+ xapian))
(home-page "https://public-inbox.org/README.html")
(synopsis "Archive mailing lists in Git repositories")
(description
T
T
Thiago Jung Bauermann wrote on 16 May 2022 00:02
[PATCH 2/3] gnu: Add perl-socket-msghdr
(address . 55436@debbugs.gnu.org)(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
20220515220230.504057-2-bauermann@kolabnow.com
* gnu/packages/perl.scm (perl-socket-msghdr): New variable.
---
gnu/packages/perl.scm | 24 ++++++++++++++++++++++++
1 file changed, 24 insertions(+)

Toggle diff (32 lines)
diff --git a/gnu/packages/perl.scm b/gnu/packages/perl.scm
index ba65cc6e6d8e..5fc46452440f 100644
--- a/gnu/packages/perl.scm
+++ b/gnu/packages/perl.scm
@@ -11849,3 +11849,27 @@ (define-public perl-data-sexpression
(description "Data::SExpression parses Lisp S-Expressions into Perl data
structures.")
(license license:perl-license)))
+
+(define-public perl-socket-msghdr
+ (package
+ (name "perl-socket-msghdr")
+ (version "0.05")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append
+ "mirror://cpan/authors/id/F/FE/FELIPE/Socket-MsgHdr-"
+ version
+ ".tar.gz"))
+ (sha256
+ (base32 "0g3qa7xn0aqn417jfvnc0i3ksyqa7bnvws0wihldir6ywcaiql4n"))))
+ (build-system perl-build-system)
+ (home-page "https://metacpan.org/release/Socket-MsgHdr")
+ (synopsis "Perform advanced operations via sendmsg and recvmsg")
+ (description "Socket::MsgHdr provides advanced socket messaging operations
+via sendmsg and recvmsg.
+
+It also allows manipulating ancillary data or so-called control
+information (cmsghdr). This ancillary data may be used for file descriptor
+passing, IPv6 operations, and a host of implemenation-specific extensions.")
+ (license license:perl-license)))
T
T
Thiago Jung Bauermann wrote on 16 May 2022 00:02
[PATCH 3/3] gnu: public-inbox: Update to version 1.8.0
(address . 55436@debbugs.gnu.org)(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
20220515220230.504057-3-bauermann@kolabnow.com
Some fixes are needed in both the testsuite and the package itself before
tests can pass so disable them for now.

Also, the lei command needs curl to work, so add it to the PATH in
‘wrap-program’.

* gnu/packages/mail.scm (public-inbox)[version]: Change to 1.8.0.
[source]: Update hash.
[arguments]<#:tests?>: Set to #f.
[arguments]<#:phases>{wrap-programs}: Add curl to PATH.
[inputs]: Add curl and perl-socket-msghdr.
---
gnu/packages/mail.scm | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)

Toggle diff (53 lines)
diff --git a/gnu/packages/mail.scm b/gnu/packages/mail.scm
index 0728d65d1d10..781c3bd028ff 100644
--- a/gnu/packages/mail.scm
+++ b/gnu/packages/mail.scm
@@ -4087,7 +4087,7 @@ (define-public l2md
(define-public public-inbox
(package
(name "public-inbox")
- (version "1.6.1")
+ (version "1.8.0")
(source
(origin (method git-fetch)
(uri (git-reference
@@ -4095,11 +4095,12 @@ (define-public public-inbox
(commit (string-append "v" version))))
(sha256
(base32
- "0mlwnp5knr7rk9kv8grlh342wsq2193m22zs83cjn9p7x9r2x5f9"))
+ "0xni1l54v1z3p0zb52807maay0yqabp8jgf5iras5zmhgjyk3swz"))
(file-name (git-file-name name version))))
(build-system perl-build-system)
(arguments
- '(#:phases
+ '(#:tests? #f
+ #:phases
(modify-phases %standard-phases
(add-before 'configure 'qualify-paths
(lambda* (#:key inputs #:allow-other-keys)
@@ -4132,13 +4133,15 @@ (define-public public-inbox
;; 'git' is invoked in various files of the PublicInbox
;; perl module.
`("PATH" ":" prefix
- (,(dirname (search-input-file inputs "/bin/git"))))))
+ (,(dirname (search-input-file inputs "/bin/git"))
+ ,(dirname (search-input-file inputs "/bin/curl"))))))
(find-files (string-append out "/bin")))))))))
(native-inputs
(list ;; For testing.
lsof openssl))
(inputs
(list bash-minimal
+ curl
git
perl-dbd-sqlite
perl-dbi
@@ -4151,6 +4154,7 @@ (define-public public-inbox
perl-plack-middleware-reverseproxy
perl-plack
perl-search-xapian
+ perl-socket-msghdr
perl-timedate
perl-uri-escape
perl-inline-c
K
K
Kyle Meyer wrote on 16 May 2022 04:10
Re: [bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)(address . 55436@debbugs.gnu.org)
87zgji2png.fsf@kyleam.com
Thiago Jung Bauermann via Guix-patches via writes:

Toggle quote (6 lines)
> Hello,
>
> These patches fix a few issues with the current public-inbox package, add a
> new package which is needed by version 1.8.0, and finally updates
> public-inbox.

Thanks for working on this.

Toggle quote (7 lines)
> [...]
> - The tests are disabled because more changes are needed to enable them.
> I have them working in a local branch, but there's one patch to the
> public-inbox testsuite which I still need to cleanup and post upstream.
> If anyone is curious about it, I pushed the branch ‘public-inbox-updates’
> to my GitLab repo².

Fwiw I applied your patches and tested it a bit with my local inboxes.
I didn't spot any issues.

Toggle snippet (25 lines)
[guix env]% realpath $(which lei)
/gnu/store/564p55ajr8wlbml1rafs4pggjzpvmqaf-public-inbox-1.8.0/bin/lei

[guix env]% lei daemon-pid
199095

[guix env]% lei q -f ldjson -O guix-patches nq:lei d:30.days.ago.. | jq '.s' | sort | uniq
"[bug#54997] [PATCH 04/12] Add (guix least-authority)."
"[bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0"
"[bug#55436] [PATCH 3/3] gnu: public-inbox: Update to version 1.8.0"

[guix env]% lei daemon-kill
[guix env]% lei daemon-pid
199200

[guix env]% lei q -O https://lore.kernel.org/git -o mboxcl2:/tmp/t.mbox nq:pack d:20.days.ago..
# /gnu/store/i11i4xrx3gibdm012bqqhgrjs69qyia3-curl-7.79.1/bin/curl -Sf -s -d '' https://lore.kernel.org/git/?x=m&q=nq%3Apack+dt%3A20220426015402..
# https://lore.kernel.org/git/ 57/57
# 57 written to /tmp/t.mbox (57 matches)

[guix env]% public-inbox-httpd &
[1] 199585
# browsed around http://127.0.0.1:8080/guix-patches/ a bit
K
K
Kyle Meyer wrote on 16 May 2022 04:13
Re: [bug#55436] [PATCH 3/3] gnu: public-inbox: Update to version 1.8.0
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)(address . 55436@debbugs.gnu.org)
87wnem2pid.fsf@kyleam.com
Thiago Jung Bauermann via Guix-patches via writes:

Toggle quote (11 lines)
> [...]
> [inputs]: Add curl and perl-socket-msghdr.
> ---
> [...]
> (inputs
> (list bash-minimal
> + curl
> git
> perl-dbd-sqlite
> perl-dbi

I think it'd be good to add libgit2 as well.
T
T
Thiago Jung Bauermann wrote on 16 May 2022 04:31
Re: [bug#55436] [PATCH 0/3] Update public-inbox to 1.8.0
(name . Kyle Meyer)(address . kyle@kyleam.com)(address . 55436@debbugs.gnu.org)
87wnemurxl.fsf@kolabnow.com
Kyle Meyer <kyle@kyleam.com> writes:

Toggle quote (10 lines)
> Thiago Jung Bauermann via Guix-patches via writes:
>
>> Hello,
>>
>> These patches fix a few issues with the current public-inbox package, add a
>> new package which is needed by version 1.8.0, and finally updates
>> public-inbox.
>
> Thanks for working on this.

No problem. Sorry to take so long. There were a surprising number of
little issues along the way…

Toggle quote (10 lines)
>> [...]
>> - The tests are disabled because more changes are needed to enable them.
>> I have them working in a local branch, but there's one patch to the
>> public-inbox testsuite which I still need to cleanup and post upstream.
>> If anyone is curious about it, I pushed the branch ‘public-inbox-updates’
>> to my GitLab repo².
>
> Fwiw I applied your patches and tested it a bit with my local inboxes.
> I didn't spot any issues.

<snip>

Awesome! Thank you for testing. You certainly use a lot more features of
the package than I do.

--
Thanks
Thiago
T
T
Thiago Jung Bauermann wrote on 16 May 2022 04:42
Re: [bug#55436] [PATCH 3/3] gnu: public-inbox: Update to version 1.8.0
(name . Kyle Meyer)(address . kyle@kyleam.com)(address . 55436@debbugs.gnu.org)
87sfpaurho.fsf@kolabnow.com
Hello Kyle,

Thank you for the feedback.

Kyle Meyer <kyle@kyleam.com> writes:

Toggle quote (15 lines)
> Thiago Jung Bauermann via Guix-patches via writes:
>
>> [...]
>> [inputs]: Add curl and perl-socket-msghdr.
>> ---
>> [...]
>> (inputs
>> (list bash-minimal
>> + curl
>> git
>> perl-dbd-sqlite
>> perl-dbi
>
> I think it'd be good to add libgit2 as well.

Ah, I wasn't aware of that. I'll wait a bit to see if there are more
comments and spin a v2 with the change.

--
Thanks
Thiago
L
L
Ludovic Courtès wrote on 24 May 2022 16:47
Re: bug#55436: [PATCH 0/3] Update public-inbox to 1.8.0
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
87v8tvj8cn.fsf_-_@gnu.org
Hi,

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (3 lines)
> Ah, I wasn't aware of that. I'll wait a bit to see if there are more
> comments and spin a v2 with the change.

I haven’t tested it but at first sight it LGTM.

Ludo’.
T
T
Thiago Jung Bauermann wrote on 25 May 2022 17:21
(name . Ludovic Courtès)(address . ludo@gnu.org)
87ilptfwd2.fsf@kolabnow.com
Hello,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (7 lines)
> Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:
>
>> Ah, I wasn't aware of that. I'll wait a bit to see if there are more
>> comments and spin a v2 with the change.
>
> I haven’t tested it but at first sight it LGTM.

Thank you for your review!

I tried adding both libgit2 and pkg-config (which public-inbox uses to
find the library) to the package inputs, but unfortunately the resulting
package doesn't retain a reference to libgit2, so I don't think it
worked:

Toggle snippet (7 lines)
popigai ?env?: ./pre-inst-env guix gc --references \
$(./pre-inst-env guix build --no-grafts public-inbox) \
| grep git
/gnu/store/afr6sh2vap4vdvj09s1q0rmcc3gl51rc-git-2.36.0
popigai ?env?:

There's more work to be done to make Perl's Inline::C mechanism work in
Guix's environment. To be honest, I won't be able to work on that.

Would it be possible to commit these patches as they are now, and then
if someone can make public-inbox use libgit2 then that change can be
submitted as a separate patch?

--
Thanks
Thiago
L
L
Ludovic Courtès wrote on 1 Jun 2022 22:45
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
871qw8t8o9.fsf_-_@gnu.org
Hi Thiago,

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (2 lines)
> Would it be possible to commit these patches as they are now,

Definitely, done!

Toggle quote (3 lines)
> and then if someone can make public-inbox use libgit2 then that change
> can be submitted as a separate patch?

Sure, we can keep that as future work. :-)

Thanks,
Ludo’.
Closed
T
T
Thiago Jung Bauermann wrote on 2 Jun 2022 01:03
(name . Ludovic Courtès)(address . ludo@gnu.org)
87wne0vvdm.fsf@kolabnow.com
Hello Ludo,

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (6 lines)
> Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:
>
>> Would it be possible to commit these patches as they are now,
>
> Definitely, done!

Awesome, thank you!

--
Thanks
Thiago
Closed
?