RFC packaging for lcsync

  • Done
  • quality assurance status badge
Details
2 participants
  • Maxime Devos
  • Vagrant Cascadian
Owner
unassigned
Submitted by
Vagrant Cascadian
Severity
normal
V
V
Vagrant Cascadian wrote on 27 May 2022 21:10
(address . guix-patches@gnu.org)
87v8tq6bby.fsf@yucca
Have a work-in-progress patch series for lcsync:


Librecast file and data syncing tool.
Compare data with merkle trees, sync via multicast.

blockers seem to be:

* issues around test suites

* not knowing the appropriate place or way to run "setcap
cap_net_raw=eip bin/lcsync" ... setuid root would not be appropriate
as that would allow overwriting nearly any file on the filesystem.

Will send patches shortly, one for the librecast library, and one for
lcsync itself.

live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYpEiIQAKCRDcUY/If5cW
qmeyAP96227cOjJmxuMZQW3//+yl3et+Pf/3ZhyOHtxyhcTZzAEAy2BYGt9yFhJe
DCfDUTwgNv9W5+mcWlWql6tiLuwfKwA=
=CyU8
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 27 May 2022 21:27
(address . 55682@debbugs.gnu.org)
87pmjy6akb.fsf@yucca
From 9b34c9939ac92f73d4bcbe5ae5850fbba11d19f0 Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian <vagrant@debian.org>
Date: Thu, 26 May 2022 14:27:41 -0700
Subject: [PATCH 1/2] gnu: Add librecast.

FIXME tests fail to even execute with:

ld: cannot find crt1.o: No such file or directory
ld: cannot find crti.o: No such file or directory

* gnu/packages/networking.scm (librecast): New variable.
---
gnu/packages/networking.scm | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)

Toggle diff (52 lines)
diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index 99145b2b47..ae780d5a05 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -345,6 +345,43 @@ GLib-based library, libnice, as well as GStreamer elements to use it.")
license:lgpl2.1+
license:mpl1.1)))))
+(define-public librecast
+ (package
+ (name "librecast")
+ (version "0.4.5")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/librestack/librecast")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "07zrl7qhdwz4x3b80crfqaxdphsa2ij68wj8iszlg8x0lkffy0bv"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:parallel-tests? #f
+ #:make-flags
+ (let ((target ,(%current-target-system)))
+ (list ,(string-append "CC="
+ (cc-for-target))
+ (string-append "PREFIX="
+ (assoc-ref %outputs "out"))))
+ #:phases
+ (modify-phases %standard-phases
+ (delete 'configure) ;no configure script
+ (add-before 'build 'add-library-paths
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let* ((libsodium (assoc-ref inputs "libsodium")))
+ (substitute* "./src/Makefile"
+ (("-lsodium") (string-append "-L" libsodium "/lib -lsodium")))))))))
+ (inputs (list libsodium))
+ (synopsis "librecast IPv6 multicast library")
+ (description "Librecast is a C library which supports IPv6 multicast
+networking.")
+ (home-page "https://librecast.net/librecast.html")
+ (license (list license:gpl2 license:gpl3))))
+
(define-public rtmpdump
;; There are no tags in the repository, and the project is unlikely to
;; make new releases. Take a recent commit for multiple security fixes

base-commit: 6e9d99f97f15347f44df0518faa5e3b8b9d5184e
--
2.30.2
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYpEmBAAKCRDcUY/If5cW
qgq3AQC7SFHTI0ivbjCMd1TrNgQkq0R9FKDySrKzCWDLMx7NJQD5AbQkENzsz9pP
c3nMlLo2EI0lP6ZQ1OC0EKTPs9l2gQM=
=uu6D
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 27 May 2022 21:27
(address . 55682@debbugs.gnu.org)
87mtf26aje.fsf@yucca
From c23ef5bf9248921af15e337d328b2fffb7edc996 Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian <vagrant@debian.org>
Date: Thu, 26 May 2022 14:37:32 -0700
Subject: [PATCH 2/2] gnu: Add lcsync.

FIXME tests run, but all fail. test failures do not trigger build failures.
FIXME figure out how to properly:

setcap cap_net_raw=eip bin/lcsync

* gnu/packages/networking.scm (lcsync): New variable.
---
gnu/packages/networking.scm | 42 +++++++++++++++++++++++++++++++++++++
1 file changed, 42 insertions(+)

Toggle diff (55 lines)
diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index ae780d5a05..605173faa9 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -207,6 +207,48 @@ on byte-critical systems. It supports HTTP, HTTPS, FTP and FTPS
protocols.")
(license license:gpl2+)))
+(define-public lcsync
+ (package
+ (name "lcsync")
+ (version "0.0.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/librestack/lcsync")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0s038b4xg9nlzhrganzjyfvc6n6cgd6kilnpik4axp62j2n5q11q"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:parallel-tests? #f
+ #:make-flags
+ (let ((target ,(%current-target-system)))
+ (list ,(string-append "CC="
+ (cc-for-target))
+ ;; avoid running setcap in the install process
+ "SETCAP_PROGRAM=true"
+ (string-append "prefix="
+ (assoc-ref %outputs "out"))))
+ #:phases
+ (modify-phases %standard-phases
+ (delete 'configure) ;no configure script
+ (add-before 'build 'add-library-paths
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let* ((librecast (assoc-ref inputs "librecast")))
+ (substitute* (list "./src/Makefile" "./test/Makefile")
+ (("-llibrecast") (string-append "-L" librecast
+ "/lib -llibrecast")))))))))
+ (inputs (list librecast libsodium))
+ (home-page "https://librecast.net/lcsync.html")
+ (synopsis "librecast file and data syncing tool")
+ (description
+ "lcsync is a tool to sync files over IPv6 multicast or the
+local filesystem. It splits the file into blocks, hashes them, and compares
+them in order to efficiently transfer a minimal amount of data.")
+ (license (list license:gpl2 license:gpl3))))
+
;; This package does not have a release yet.
;; But this is required to provide a feature in PipeWire.
(define-public libcamera
--
2.30.2
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYpEmJgAKCRDcUY/If5cW
qpDaAP0cgpulj3v6k3qkTbiO2Hj87Q1VYm4ol+gEBAuCIJipxAD/UKr7g/N8IJA8
PYpdBo/MKp+d63y1QVok1L2ixDaTvAI=
=JqWr
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 31 Jul 2022 01:39
(address . 55682@debbugs.gnu.org)
87wnbuyxu5.fsf@contorta
On 2022-05-27, Vagrant Cascadian wrote:
Toggle quote (11 lines)
> Have a work-in-progress patch series for lcsync:
>
> https://librecast.net/lcsync.html
>
> Librecast file and data syncing tool.
> Compare data with merkle trees, sync via multicast.
>
> blockers seem to be:
>
> * issues around test suites

Have mostly fixed these with some new patches.

There are some tests that require networking, and upstream is working on
a way to skip those when networking is unavailable.


Toggle quote (4 lines)
> * not knowing the appropriate place or way to run "setcap
> cap_net_raw=eip bin/lcsync" ... setuid root would not be appropriate
> as that would allow overwriting nearly any file on the filesystem.

This is still outstanding, feature request reported as:

#55683: Support binaries that need "setcap" similar to "setuid-programs"


Toggle quote (3 lines)
> Will send patches shortly, one for the librecast library, and one for
> lcsync itself.

Will submit an updated patch series to new usptream versions and to run
the correct test suites, and also added the lcrq library.


Happy multicasting!


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYuXBMwAKCRDcUY/If5cW
qgZIAQD3puue2r/XCywsD02BFpvvXPPBrXssy44mXXNRd334VQEA9Lg1EjaJPUAa
VmFLnrN1eAUidDsk6aMl4L9VYg+5IgI=
=Pgph
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 31 Jul 2022 01:41
[PATCH V2 1/3] gnu: Add lcrq.
(address . 55682@debbugs.gnu.org)
87tu6yyxr2.fsf@contorta
From 4632adcfda6149945b7b02c2a3160ba38cf1cd03 Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian <vagrant@debian.org>
Date: Fri, 29 Jul 2022 15:01:49 -0700
Subject: [PATCH 1/3] gnu: Add lcrq.

* gnu/packages/networking.scm (lcrq): New variable.
---
gnu/packages/networking.scm | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)

Toggle diff (49 lines)
diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index a48c4997ef..6c1adf43bc 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -239,6 +239,42 @@ (define-public axel
protocols.")
(license license:gpl2+)))
+(define-public lcrq
+ (package
+ (name "lcrq")
+ (version "0.0.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://codeberg.org/librecast/lcrq")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0jf7x3zcdbz5b99qz7liw4i90hn9s457zr82n0r8g9qsi81a1d8c"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:parallel-tests? #f
+ #:make-flags (let ((target ,(%current-target-system)))
+ (list ,(string-append "CC="
+ (cc-for-target))
+ (string-append "PREFIX="
+ (assoc-ref %outputs "out"))))
+ #:phases (modify-phases %standard-phases
+ (replace 'check
+ (lambda _
+ (invoke "make" "test"))))))
+ (home-page "https://librecast.net/lcrq.html")
+ (synopsis "librecast RaptorQ library")
+ (description
+ "C library implementation of RaptorQ Forward Error Correction for
+Librecast. RFC6330 (IETF) describes the RaptorQ proposed standard, which LCRQ
+more-or-less follows. The primary focus has been on building a fast, simple
+and dependency-free FEC implementation for use with Librecast, and not on
+strict standards compliance. The code does, however, fairly closely follow
+the RFC.")
+ (license (list license:gpl2 license:gpl3))))
+
;; This package does not have a release yet.
;; But this is required to provide a feature in PipeWire.
(define-public libcamera
--
2.35.1
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYuXBoQAKCRDcUY/If5cW
qkK3AQDeJ/B0p4GU1iB2uWtEMF5QjKhfqANi73bqKqcIRQ4vCAEA30T4vi/zpcOA
rh3IW/2Ipd+jzwgMug4RrzDwPnzExwY=
=8SaR
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 31 Jul 2022 01:42
[PATCH V2 2/3] gnu: Add librecast.
(address . 55682@debbugs.gnu.org)
87r122yxp9.fsf@contorta
From 9260aa91b6bc30b9abf0b6ead815d3a76f780a9b Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian <vagrant@debian.org>
Date: Thu, 26 May 2022 14:27:41 -0700
Subject: [PATCH 2/3] gnu: Add librecast.

* gnu/packages/networking.scm (librecast): New variable.
---
gnu/packages/networking.scm | 35 +++++++++++++++++++++++++++++++++++
1 file changed, 35 insertions(+)

Toggle diff (48 lines)
diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index 6c1adf43bc..41aa346036 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -419,6 +419,41 @@ (define-public libnice
license:lgpl2.1+
license:mpl1.1)))))
+(define-public librecast
+ (package
+ (name "librecast")
+ (version "0.5.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://codeberg.org/librecast/librecast")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1zzdxawzsj0lxyxm8c2wdqx3b633f8ybvlg6szs4v0y42xg4a829"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:parallel-tests? #f
+ #:make-flags (let ((target ,(%current-target-system)))
+ (list ,(string-append "CC="
+ (cc-for-target))
+ (string-append "PREFIX="
+ (assoc-ref %outputs "out"))))
+ #:phases (modify-phases %standard-phases
+ (add-before 'check 'drop-test
+ (lambda _
+ (delete-file "./test/0000-0040.c")))
+ (replace 'check
+ (lambda _
+ (invoke "make" "test"))))))
+ (inputs (list libsodium lcrq))
+ (synopsis "librecast IPv6 multicast library")
+ (description "Librecast is a C library which supports IPv6 multicast
+networking.")
+ (home-page "https://librecast.net/librecast.html")
+ (license (list license:gpl2 license:gpl3))))
+
(define-public rtmpdump
;; There are no tags in the repository, and the project is unlikely to
;; make new releases. Take a recent commit for multiple security fixes
--
2.35.1
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYuXB4wAKCRDcUY/If5cW
qjpeAQDnCJ/9nmO9L4n/WhBTa9PTaXP+Xw5XsfZQAzNJBd9z1AD9E4DMdEb0xD0m
fNrauybTStbuNlMu3LZlUOciE3GAKwo=
=/E4D
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 31 Jul 2022 01:43
[PATCH V2 3/3] gnu: Add lcsync.
(address . 55682@debbugs.gnu.org)
87o7x6yxmv.fsf@contorta
From 14cfc0c8ac18be80eb441df3d063c4dd1175f36b Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian <vagrant@debian.org>
Date: Thu, 26 May 2022 14:37:32 -0700
Subject: [PATCH 3/3] gnu: Add lcsync.

FIXME figure out how to properly:

setcap cap_net_raw=eip bin/lcsync

* gnu/packages/networking.scm (lcsync): New variable.
---
gnu/packages/networking.scm | 43 +++++++++++++++++++++++++++++++++++++
1 file changed, 43 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index 41aa346036..0d2ecfcad1 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -275,6 +275,49 @@ (define-public lcrq
the RFC.")
(license (list license:gpl2 license:gpl3))))
+(define-public lcsync
+ (package
+ (name "lcsync")
+ (version "0.0.1")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://codeberg.org/librecast/lcsync")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0s038b4xg9nlzhrganzjyfvc6n6cgd6kilnpik4axp62j2n5q11q"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:parallel-tests? #f
+ #:make-flags (let ((target ,(%current-target-system)))
+ (list ,(string-append "CC="
+ (cc-for-target))
+ ;; avoid running setcap in the install process
+ "SETCAP_PROGRAM=true"
+ (string-append "prefix="
+ (assoc-ref %outputs "out"))))
+ #:phases (modify-phases %standard-phases
+ (delete 'configure) ;no configure script
+ (replace 'check
+ (lambda _
+ (invoke "make" "test" "CC=gcc")))
+ (add-before 'build 'add-library-paths
+ (lambda* (#:key inputs #:allow-other-keys)
+ (let* ((librecast (assoc-ref inputs "librecast")))
+ (substitute* (list "./src/Makefile" "./test/Makefile")
+ (("-llibrecast")
+ (string-append "-L" librecast "/lib -llibrecast")))))))))
+ (inputs (list librecast libsodium))
+ (home-page "https://librecast.net/lcsync.html")
+ (synopsis "librecast file and data syncing tool")
+ (description
+ "lcsync is a tool to sync files over IPv6 multicast or the
+local filesystem. It splits the file into blocks, hashes them, and compares
+them in order to efficiently transfer a minimal amount of data.")
+ (license (list license:gpl2 license:gpl3))))
+
;; This package does not have a release yet.
;; But this is required to provide a feature in PipeWire.
(define-public libcamera
--
2.35.1
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYuXCOQAKCRDcUY/If5cW
qvvvAP43SbbqXAuDN5BC+2cdeic/wWGOky3uhlW7+UkRhb2f3wD+NgJHuJNGiexQ
Ydl4aCYXH2kECyeVa+uXssW9E1TlTgA=
=+O1t
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 11 Aug 2022 04:32
Re: RFC packaging for lcsync
(address . 55682-done@debbugs.gnu.org)
87mtcb7bnh.fsf@contorta
On 2022-07-30, Vagrant Cascadian wrote:
Toggle quote (17 lines)
> On 2022-05-27, Vagrant Cascadian wrote:
>> Have a work-in-progress patch series for lcsync:
>>
>> https://librecast.net/lcsync.html
>>
>> Librecast file and data syncing tool.
>> Compare data with merkle trees, sync via multicast.
>>
>> blockers seem to be:
>>
>> * issues around test suites
>
> Have mostly fixed these with some new patches.
>
> There are some tests that require networking, and upstream is working on
> a way to skip those when networking is unavailable.

I removed the failing network tests for now.


Toggle quote (8 lines)
>> * not knowing the appropriate place or way to run "setcap
>> cap_net_raw=eip bin/lcsync" ... setuid root would not be appropriate
>> as that would allow overwriting nearly any file on the filesystem.
>
> This is still outstanding, feature request reported as:
>
> #55683: Support binaries that need "setcap" similar to "setuid-programs"

Since this can still be useful as root, I pushed without resolving this
issue. I think the approach outlined in the above bug report should work
in theory...


Toggle quote (6 lines)
>> Will send patches shortly, one for the librecast library, and one for
>> lcsync itself.
>
> Will submit an updated patch series to new usptream versions and to run
> the correct test suites, and also added the lcrq library.

Push to guix master as:

6bdcd2f3f08d054f8a605fc33935c4057528cdcc gnu: Add lcsync.
78b81bf95166562921284a5257580eb2ff01e4f3 gnu: Add librecast.
24ec1b5a2046111d34bc5a965a6bef992f7c8d78 gnu: Add lcrq.


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYvRqQwAKCRDcUY/If5cW
qp1yAPoCWJbbOuzAP5D2B6onrunB9C9Lhul7NcA4rWWv3u/kkAD/VCPLPrO8D50x
Cm4Mvi5NrMVpFl7nZHAwWzEph+TUIws=
=rXsd
-----END PGP SIGNATURE-----

Closed
M
M
Maxime Devos wrote on 11 Aug 2022 11:48
Re: [bug#55682] [PATCH V2 1/3] gnu: Add lcrq.
902b4751-ee88-fe46-8d68-b1a61605dce5@telenet.be
On 31-07-2022 01:41, Vagrant Cascadian wrote:
Toggle quote (3 lines)
> + (replace 'check
> + (lambda _
> + (invoke "make" "test"))))))
I expect "./pre-inst-env guix lint lcrq" to have a warning about
respecting #:tests?. This is important for --without-tests and
cross-compilation. Likewise for the other patches.
Toggle quote (5 lines)
> + #:make-flags (let ((target ,(%current-target-system)))
> + (list ,(string-append "CC="
> + (cc-for-target))
> + (string-append "PREFIX="
> + (assoc-ref %outputs "out"))))
'target' is unused and can be removed.
Greetings,
Maxime.
Attachment: OpenPGP_0x49E3EE22191725EE.asc
Attachment: OpenPGP_signature
V
V
Vagrant Cascadian wrote on 12 Aug 2022 20:15
87a6898h19.fsf@contorta
On 2022-08-11, Maxime Devos wrote:
Toggle quote (9 lines)
> On 31-07-2022 01:41, Vagrant Cascadian wrote:
>> + (replace 'check
>> + (lambda _
>> + (invoke "make" "test"))))))
>
> I expect "./pre-inst-env guix lint lcrq" to have a warning about
> respecting #:tests?. This is important for --without-tests and
> cross-compilation. Likewise for the other patches.

Oops. Looks like Efraim caught this after I pushed, at least.

Toggle quote (7 lines)
>> + #:make-flags (let ((target ,(%current-target-system)))
>> + (list ,(string-append "CC="
>> + (cc-for-target))
>> + (string-append "PREFIX="
>> + (assoc-ref %outputs "out"))))
> 'target' is unused and can be removed.

Nice catch, will look at it!


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYvaYwwAKCRDcUY/If5cW
qhYHAP4jB36BaymYdb0y1zvuyuddQePOq2eVuaqT+aEcj3ScRAEAwKm8oNhxv73F
fMEAbwrnVw06DuhJkoCJylZt0+cEBAg=
=OaGK
-----END PGP SIGNATURE-----

?
Your comment

This issue is archived.

To comment on this conversation send an email to 55682@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 55682
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch