[PATCH 0/6] Add curl-full

  • Done
  • quality assurance status badge
Details
2 participants
  • Mathieu Othacehe
  • Tomas Volf
Owner
unassigned
Submitted by
Tomas Volf
Severity
normal
T
T
Tomas Volf wrote on 8 Jan 20:19 +0100
(address . guix-patches@gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
cover.1704741394.git.~@wolfsden.cz
Current curl package does not support websockets, does not support HTTP3 and
does not support Public Suffix List. The last one could even be considered a
security issue.

This commit adds new curl-full package, that enables everything that can be
enabled.

This opportunity was also used to update ngtcp2 and do some small cleaning.

Tomas Volf (6):
gnu: web: Delete trailing whitespace.
gnu: Add nghttp3.
gnu: ngtcp2: Update to 1.1.0.
gnu: ngtcp2: Build libngtcp2_crypto_gnutls.
gnu: Add curl-full.
gnu: curl-full: Update homepage.

gnu/local.mk | 1 +
gnu/packages/curl.scm | 71 ++++++++++++++++-
gnu/packages/networking.scm | 13 ++-
.../patches/curl-8.5-use-ssl-cert-env.patch | 79 +++++++++++++++++++
gnu/packages/web.scm | 38 ++++++++-
5 files changed, 197 insertions(+), 5 deletions(-)
create mode 100644 gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch


base-commit: b212e6934643e085f168a5364cb593f61aa616ba
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 3/6] gnu: ngtcp2: Update to 1.1.0.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
506698982472307161dddbb3c3c587bbc3cad57d.1704741394.git.~@wolfsden.cz
* gnu/packages/networking.scm (ngtcp2): Update to 1.1.0.

Change-Id: I7f6b08952ba508207c8dedbd68c7c48dd17dd0ff
---
gnu/packages/networking.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index 9844430656..c738716792 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -4385,7 +4385,7 @@ (define-public nbd
(define-public ngtcp2
(package
(name "ngtcp2")
- (version "1.0.1")
+ (version "1.1.0")
(source
(origin
(method url-fetch)
@@ -4393,7 +4393,7 @@ (define-public ngtcp2
"releases/download/v" version "/"
"ngtcp2-" version ".tar.xz"))
(sha256
- (base32 "0l84hnj9n4bfxjizgmqsqbz71jx7m00a7l1z43fg5ls3apx9ij11"))))
+ (base32 "1pppl6s25hz91w6321g1q7dqvfy4vccz9mmc5r8sfdvdc95fngl0"))))
(build-system gnu-build-system)
(native-inputs (list cunit))
(home-page "https://nghttp2.org/ngtcp2/")
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 1/6] gnu: web: Delete trailing whitespace.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
f144a2bd03efd34ccf22c2156cd729d3a55fa326.1704741394.git.~@wolfsden.cz
* gnu/packages/web.scm (krona-tools): Delete trailing whitespace.

Change-Id: Ide601b011d3e2c3c736bf041571f9ab00b5b5b1d
---
gnu/packages/web.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (15 lines)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 13ae53aa8c..e0f8bf4547 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -1481,7 +1481,7 @@ (define-public krona-tools
"ImportTaxonomy"
"ImportText"
"ImportXML"))
- (for-each
+ (for-each
(lambda (directory)
(copy-recursively directory
(string-append perl "/../" directory)))
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 6/6] gnu: curl-full: Update homepage.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
462759b14e3365bcb388d8792ab854776fe3bf16.1704741394.git.~@wolfsden.cz
Resolves following lint warning:

permanent redirect from https://curl.haxx.se/to https://curl.se/

* gnu/packages/curl.scm (curl-full)[homepage]: Update.

Change-Id: I88c899c0a79b80e1ce86add8227c43c12d8b00ce
---
gnu/packages/curl.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (15 lines)
diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
index 0f0336cb6a..f233eeca7a 100644
--- a/gnu/packages/curl.scm
+++ b/gnu/packages/curl.scm
@@ -173,7 +173,7 @@ (define-public curl
tunneling, and so on.")
(license (license:non-copyleft "file://COPYING"
"See COPYING in the distribution."))
- (home-page "https://curl.haxx.se/")))
+ (home-page "https://curl.se/")))
(define-public curl-full
(package/inherit curl
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 5/6] gnu: Add curl-full.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
d078e3ef92911da86eccf817d05cd0dd9a9d86e0.1704741394.git.~@wolfsden.cz
New variable providing curl compiled with all available features is added.

* gnu/packages/curl.scm (curl-full): New variable.
* gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.

Change-Id: Ic3c9b2b9bd0a81d6840dcdccd21d52a9d0b311e2
---
gnu/local.mk | 1 +
gnu/packages/curl.scm | 69 ++++++++++++++++
.../patches/curl-8.5-use-ssl-cert-env.patch | 79 +++++++++++++++++++
3 files changed, 149 insertions(+)
create mode 100644 gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch

Toggle diff (201 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index f804f4ef5b..73eab7cf93 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1060,6 +1060,7 @@ dist_patch_DATA = \
%D%/packages/patches/clucene-contribs-lib.patch \
%D%/packages/patches/cube-nocheck.patch \
%D%/packages/patches/curl-use-ssl-cert-env.patch \
+ %D%/packages/patches/curl-8.5-use-ssl-cert-env.patch \
%D%/packages/patches/curlftpfs-fix-error-closing-file.patch \
%D%/packages/patches/curlftpfs-fix-file-names.patch \
%D%/packages/patches/curlftpfs-fix-memory-leak.patch \
diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
index b33f4d36d4..0f0336cb6a 100644
--- a/gnu/packages/curl.scm
+++ b/gnu/packages/curl.scm
@@ -16,6 +16,7 @@
;;; Copyright © 2021 Felix Gruber <felgru@posteo.net>
;;; Copyright © 2023 Sharlatan Hellseher <sharlatanus@gmail.com>
;;; Copyright © 2023 John Kehayias <john.kehayias@protonmail.com>
+;;; Copyright © 2024 Tomas Volf <~@wolfsden.cz>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -46,14 +47,19 @@ (define-module (gnu packages curl)
#:use-module (guix build-system meson)
#:use-module ((guix search-paths) #:select ($SSL_CERT_DIR $SSL_CERT_FILE))
#:use-module (gnu packages)
+ #:use-module (gnu packages certs)
#:use-module (gnu packages check)
#:use-module (gnu packages compression)
#:use-module (gnu packages golang)
+ #:use-module (gnu packages gsasl)
+ #:use-module (gnu packages groff)
#:use-module (gnu packages guile)
#:use-module (gnu packages kerberos)
#:use-module (gnu packages logging)
#:use-module (gnu packages libevent)
#:use-module (gnu packages libidn)
+ #:use-module (gnu packages openldap)
+ #:use-module (gnu packages networking)
#:use-module (gnu packages perl)
#:use-module (gnu packages pkg-config)
#:use-module (gnu packages python)
@@ -169,6 +175,69 @@ (define-public curl
"See COPYING in the distribution."))
(home-page "https://curl.haxx.se/")))
+(define-public curl-full
+ (package/inherit curl
+ (name "curl-full")
+ (version "8.5.0")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append "https://curl.se/download/curl-"
+ version ".tar.gz"))
+ (sha256
+ (base32
+ "1pas8z1vijp08za5xgh2vvsajwi1p2208vlhl0vs94xp4pzigz05"))
+ (patches (search-patches "curl-8.5-use-ssl-cert-env.patch"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments curl)
+ ((#:configure-flags flags)
+ #~(cons* "--enable-manual"
+ ;; Not yet available: https://github.com/curl/curl/pull/11922
+ ;; "--enable-ech"
+ "--enable-websockets"
+ "--with-brotli"
+ "--with-ldap"
+ "--with-ldaps"
+ "--with-librtmp"
+ "--with-libssh2"
+ "--with-nghttp3"
+ "--with-ngtcp2"
+ "--with-zstd"
+ #$flags))
+ ((#:phases phases)
+ #~(modify-phases #$phases
+ (add-after 'unpack 'skip-tests
+ (lambda _
+ (let ((port (open-file "tests/data/DISABLED" "a")))
+ ;; I do not understand why curl package ignores this test
+ ;; just on some architectures. I had it fail even on amd64,
+ ;; it is just too flaky.
+ (display "1474\n" port)
+ ;; Required files are missing, should be fixed in >8.5.0:
+ ;; https://github.com/curl/curl/issues/12462
+ (display "1477\n" port)
+ (close port))))))))
+ (propagated-inputs
+ (list nss-certs))
+ (native-inputs
+ (modify-inputs (package-native-inputs curl)
+ (prepend groff)))
+ (inputs
+ (modify-inputs (package-inputs curl)
+ (prepend brotli
+ gsasl
+ libpsl
+ libssh2
+ nghttp3
+ ngtcp2
+ openldap
+ rtmpdump
+ `(,zstd "lib"))))
+ (description (string-append (package-description curl)
+ "\n\n"
+ "Compared to the regular curl package, this
+one enables everything that can be enabled, bringing in additional features
+and security."))))
+
(define-public curl-ssh
(package/inherit curl
(arguments
diff --git a/gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch b/gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch
new file mode 100644
index 0000000000..70f703a999
--- /dev/null
+++ b/gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch
@@ -0,0 +1,79 @@
+From 0a942e9f2f5255793e99408ca3da0940d52dfdad Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba@kadziolka.net>
+Date: Mon, 8 Jan 2024 15:07:38 +0100
+Subject: [PATCH] libcurl: Respect SSL_CERT_DIR and SSL_CERT_FILE.
+
+Make libcurl respect the SSL_CERT_{DIR,FILE} variables by default. The variables
+are fetched during initialization to preserve thread-safety (curl_global_init(3)
+must be called when no other threads exist).
+
+This fixes network functionality in rust:cargo, and probably removes the need
+for other future workarounds.
+---
+ lib/easy.c | 9 +++++++++
+ lib/url.c | 15 +++++++++++++++
+ 2 files changed, 24 insertions(+)
+
+diff --git a/lib/easy.c b/lib/easy.c
+index 322d1a41b..2a235a19c 100644
+--- a/lib/easy.c
++++ b/lib/easy.c
+@@ -137,6 +137,9 @@ curl_wcsdup_callback Curl_cwcsdup = Curl_wcsdup;
+ static char *leakpointer;
+ #endif
+
++char * Curl_ssl_cert_dir = NULL;
++char * Curl_ssl_cert_file = NULL;
++
+ /**
+ * curl_global_init() globally initializes curl given a bitwise set of the
+ * different features of what to initialize.
+@@ -163,6 +166,9 @@ static CURLcode global_init(long flags, bool memoryfuncs)
+ goto fail;
+ }
+
++ Curl_ssl_cert_dir = curl_getenv("SSL_CERT_DIR");
++ Curl_ssl_cert_file = curl_getenv("SSL_CERT_FILE");
++
+ if(!Curl_ssl_init()) {
+ DEBUGF(fprintf(stderr, "Error: Curl_ssl_init failed\n"));
+ goto fail;
+@@ -287,6 +293,9 @@ void curl_global_cleanup(void)
+ Curl_ssl_cleanup();
+ Curl_resolver_global_cleanup();
+
++ free(Curl_ssl_cert_dir);
++ free(Curl_ssl_cert_file);
++
+ #ifdef _WIN32
+ Curl_win32_cleanup(easy_init_flags);
+ #endif
+diff --git a/lib/url.c b/lib/url.c
+index b81785fe2..d607e74bb 100644
+--- a/lib/url.c
++++ b/lib/url.c
+@@ -454,6 +454,21 @@ CURLcode Curl_init_userdefined(struct Curl_easy *data)
+ if(result)
+ return result;
+ #endif
++ extern char * Curl_ssl_cert_dir;
++ extern char * Curl_ssl_cert_file;
++ if(Curl_ssl_cert_dir) {
++ if(result = Curl_setstropt(&set->str[STRING_SSL_CAPATH], Curl_ssl_cert_dir))
++ return result;
++ if(result = Curl_setstropt(&set->str[STRING_SSL_CAPATH_PROXY], Curl_ssl_cert_dir))
++ return result;
++ }
++
++ if(Curl_ssl_cert_file) {
++ if(result = Curl_setstropt(&set->str[STRING_SSL_CAFILE], Curl_ssl_cert_file))
++ return result;
++ if(result = Curl_setstropt(&set->str[STRING_SSL_CAFILE_PROXY], Curl_ssl_cert_file))
++ return result;
++ }
+ }
+
+ #ifndef CURL_DISABLE_FTP
+--
+2.41.0
+
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 2/6] gnu: Add nghttp3.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
9dda8133cace802ec620c95dd0ae741fc0bebe30.1704741394.git.~@wolfsden.cz
* gnu/packages/web.scm (nghttp3): New variable.

Change-Id: I4ed318506f6d3bdd0c47a367b1dcc1b4454c09b2
---
gnu/packages/web.scm | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index e0f8bf4547..3c7ff49fb9 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -65,6 +65,7 @@
;;; Copyright © 2023 Christopher Howard <christopher@librehacker.com>
;;; Copyright © 2023 Felix Lechner <felix.lechner@lease-up.com>
;;; Copyright © 2023 Evgeny Pisemsky <evgeny@pisemsky.com>
+;;; Copyright © 2024 Tomas Volf <~@wolfsden.cz>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -8137,6 +8138,41 @@ (define-public nghttp2-for-node
(("print \\(ver >= '3\\.8'\\)")
"print (tuple(map(int, ver.split('.'))) >= (3,8))")))))))))))
+(define-public nghttp3
+ (package
+ (name "nghttp3")
+ (version "1.1.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/ngtcp2/nghttp3/"
+ "releases/download/v" version "/"
+ "nghttp3-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1fzvadnwb03jlm180313gg5m4fg09qdcc67fwcfrv9zs22anaa55"))))
+ (build-system gnu-build-system)
+ (native-inputs
+ (list pkg-config
+ ;; Required by tests.
+ cunit))
+ (arguments
+ (list
+ #:configure-flags
+ #~(list "--enable-lib-only")))
+ (home-page "https://nghttp2.org/nghttp3/")
+ (synopsis "HTTP/3 protocol library")
+ (description
+ "nghttp3 is an implementation of RFC 9114 HTTP/3 mapping over QUIC and
+RFC 9204 QPACK in C. It does not depend on any particular QUIC transport
+implementation.
+
+It implements extensions specified in RFC 9218 and RFC 9220. It supports
+SETTINGS_H3_DATAGRAM from RFC 9297.
+
+It does not support server push.")
+ (license license:expat)))
+
(define-public hpcguix-web
(package
(name "hpcguix-web")
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 4/6] gnu: ngtcp2: Build libngtcp2_crypto_gnutls.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
84ab70db28e7aa5c9a90f14f489971a19601eaac.1704741394.git.~@wolfsden.cz
* gnu/packages/networking.scm (ngtcp2)[arguments]<#:configure-flags>: Add
--with-gnutls.
[native-inputs]: Add pkg-config.
[inputs]: Add gnutls.

Change-Id: Id3a9210992d203a57a9b791f3c3752f2ebd3e634
---
gnu/packages/networking.scm | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)

Toggle diff (29 lines)
diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index c738716792..40155480c9 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -61,6 +61,7 @@
;;; Copyright © 2023 Yovan Naumovski <yovan@gorski.stream>
;;; Copyright © 2023 Zheng Junjie <873216071@qq.com>
;;; Copyright © 2023 Artyom V. Poptsov <poptsov.artyom@gmail.com>
+;;; Copyright © 2024 Tomas Volf <~@wolfsden.cz>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -4395,7 +4396,13 @@ (define-public ngtcp2
(sha256
(base32 "1pppl6s25hz91w6321g1q7dqvfy4vccz9mmc5r8sfdvdc95fngl0"))))
(build-system gnu-build-system)
- (native-inputs (list cunit))
+ (arguments
+ (list
+ #:configure-flags
+ ;; openssl package does not support QUIC interface, so just gnutls
+ #~(list "--with-gnutls")))
+ (native-inputs (list cunit pkg-config))
+ (inputs (list gnutls))
(home-page "https://nghttp2.org/ngtcp2/")
(synopsis "QUIC protocol implementation")
(description
--
2.41.0
T
T
Tomas Volf wrote on 17 Jan 15:00 +0100
control message for bug #68332
(address . control@debbugs.gnu.org)
87cyu0qctm.fsf@wolfsden.cz
close 68332
quit

Was told on IRC that it would be ideal to drop the curl-full and just
enable the options for regular curl. Therefore I am closing this for
now and will send it for next core-updates cycle.
M
M
Mathieu Othacehe wrote on 17 Jan 18:22 +0100
Re: [bug#68332] [PATCH 0/6] Add curl-full
(name . Tomas Volf)(address . ~@wolfsden.cz)(address . 68332@debbugs.gnu.org)
875xzrkh78.fsf@gnu.org
Hello Tomas,

Toggle quote (5 lines)
> gnu: web: Delete trailing whitespace.
> gnu: Add nghttp3.
> gnu: ngtcp2: Update to 1.1.0.
> gnu: ngtcp2: Build libngtcp2_crypto_gnutls.

I took the liberty of applying those 4 even if you closed this issue, as
discussed on #guix.

Thanks,

Mathieu
?