[PATCH 0/6] Add curl-full

  • Done
  • quality assurance status badge
Details
2 participants
  • Mathieu Othacehe
  • Tomas Volf
Owner
unassigned
Submitted by
Tomas Volf
Severity
normal
T
T
Tomas Volf wrote on 8 Jan 20:19 +0100
(address . guix-patches@gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
cover.1704741394.git.~@wolfsden.cz
Current curl package does not support websockets, does not support HTTP3 and
does not support Public Suffix List. The last one could even be considered a
security issue.

This commit adds new curl-full package, that enables everything that can be
enabled.

This opportunity was also used to update ngtcp2 and do some small cleaning.

Tomas Volf (6):
gnu: web: Delete trailing whitespace.
gnu: Add nghttp3.
gnu: ngtcp2: Update to 1.1.0.
gnu: ngtcp2: Build libngtcp2_crypto_gnutls.
gnu: Add curl-full.
gnu: curl-full: Update homepage.

gnu/local.mk | 1 +
gnu/packages/curl.scm | 71 ++++++++++++++++-
gnu/packages/networking.scm | 13 ++-
.../patches/curl-8.5-use-ssl-cert-env.patch | 79 +++++++++++++++++++
gnu/packages/web.scm | 38 ++++++++-
5 files changed, 197 insertions(+), 5 deletions(-)
create mode 100644 gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch


base-commit: b212e6934643e085f168a5364cb593f61aa616ba
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 3/6] gnu: ngtcp2: Update to 1.1.0.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
506698982472307161dddbb3c3c587bbc3cad57d.1704741394.git.~@wolfsden.cz
* gnu/packages/networking.scm (ngtcp2): Update to 1.1.0.

Change-Id: I7f6b08952ba508207c8dedbd68c7c48dd17dd0ff
---
gnu/packages/networking.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index 9844430656..c738716792 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -4385,7 +4385,7 @@ (define-public nbd
(define-public ngtcp2
(package
(name "ngtcp2")
- (version "1.0.1")
+ (version "1.1.0")
(source
(origin
(method url-fetch)
@@ -4393,7 +4393,7 @@ (define-public ngtcp2
"releases/download/v" version "/"
"ngtcp2-" version ".tar.xz"))
(sha256
- (base32 "0l84hnj9n4bfxjizgmqsqbz71jx7m00a7l1z43fg5ls3apx9ij11"))))
+ (base32 "1pppl6s25hz91w6321g1q7dqvfy4vccz9mmc5r8sfdvdc95fngl0"))))
(build-system gnu-build-system)
(native-inputs (list cunit))
(home-page "https://nghttp2.org/ngtcp2/")
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 1/6] gnu: web: Delete trailing whitespace.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
f144a2bd03efd34ccf22c2156cd729d3a55fa326.1704741394.git.~@wolfsden.cz
* gnu/packages/web.scm (krona-tools): Delete trailing whitespace.

Change-Id: Ide601b011d3e2c3c736bf041571f9ab00b5b5b1d
---
gnu/packages/web.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (15 lines)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 13ae53aa8c..e0f8bf4547 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -1481,7 +1481,7 @@ (define-public krona-tools
"ImportTaxonomy"
"ImportText"
"ImportXML"))
- (for-each
+ (for-each
(lambda (directory)
(copy-recursively directory
(string-append perl "/../" directory)))
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 6/6] gnu: curl-full: Update homepage.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
462759b14e3365bcb388d8792ab854776fe3bf16.1704741394.git.~@wolfsden.cz
Resolves following lint warning:

permanent redirect from https://curl.haxx.se/to https://curl.se/

* gnu/packages/curl.scm (curl-full)[homepage]: Update.

Change-Id: I88c899c0a79b80e1ce86add8227c43c12d8b00ce
---
gnu/packages/curl.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Toggle diff (15 lines)
diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
index 0f0336cb6a..f233eeca7a 100644
--- a/gnu/packages/curl.scm
+++ b/gnu/packages/curl.scm
@@ -173,7 +173,7 @@ (define-public curl
tunneling, and so on.")
(license (license:non-copyleft "file://COPYING"
"See COPYING in the distribution."))
- (home-page "https://curl.haxx.se/")))
+ (home-page "https://curl.se/")))
(define-public curl-full
(package/inherit curl
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 5/6] gnu: Add curl-full.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
d078e3ef92911da86eccf817d05cd0dd9a9d86e0.1704741394.git.~@wolfsden.cz
New variable providing curl compiled with all available features is added.

* gnu/packages/curl.scm (curl-full): New variable.
* gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch: New file.
* gnu/local.mk (dist_patch_DATA): Add it.

Change-Id: Ic3c9b2b9bd0a81d6840dcdccd21d52a9d0b311e2
---
gnu/local.mk | 1 +
gnu/packages/curl.scm | 69 ++++++++++++++++
.../patches/curl-8.5-use-ssl-cert-env.patch | 79 +++++++++++++++++++
3 files changed, 149 insertions(+)
create mode 100644 gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch

Toggle diff (201 lines)
diff --git a/gnu/local.mk b/gnu/local.mk
index f804f4ef5b..73eab7cf93 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -1060,6 +1060,7 @@ dist_patch_DATA = \
%D%/packages/patches/clucene-contribs-lib.patch \
%D%/packages/patches/cube-nocheck.patch \
%D%/packages/patches/curl-use-ssl-cert-env.patch \
+ %D%/packages/patches/curl-8.5-use-ssl-cert-env.patch \
%D%/packages/patches/curlftpfs-fix-error-closing-file.patch \
%D%/packages/patches/curlftpfs-fix-file-names.patch \
%D%/packages/patches/curlftpfs-fix-memory-leak.patch \
diff --git a/gnu/packages/curl.scm b/gnu/packages/curl.scm
index b33f4d36d4..0f0336cb6a 100644
--- a/gnu/packages/curl.scm
+++ b/gnu/packages/curl.scm
@@ -16,6 +16,7 @@
;;; Copyright © 2021 Felix Gruber <felgru@posteo.net>
;;; Copyright © 2023 Sharlatan Hellseher <sharlatanus@gmail.com>
;;; Copyright © 2023 John Kehayias <john.kehayias@protonmail.com>
+;;; Copyright © 2024 Tomas Volf <~@wolfsden.cz>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -46,14 +47,19 @@ (define-module (gnu packages curl)
#:use-module (guix build-system meson)
#:use-module ((guix search-paths) #:select ($SSL_CERT_DIR $SSL_CERT_FILE))
#:use-module (gnu packages)
+ #:use-module (gnu packages certs)
#:use-module (gnu packages check)
#:use-module (gnu packages compression)
#:use-module (gnu packages golang)
+ #:use-module (gnu packages gsasl)
+ #:use-module (gnu packages groff)
#:use-module (gnu packages guile)
#:use-module (gnu packages kerberos)
#:use-module (gnu packages logging)
#:use-module (gnu packages libevent)
#:use-module (gnu packages libidn)
+ #:use-module (gnu packages openldap)
+ #:use-module (gnu packages networking)
#:use-module (gnu packages perl)
#:use-module (gnu packages pkg-config)
#:use-module (gnu packages python)
@@ -169,6 +175,69 @@ (define-public curl
"See COPYING in the distribution."))
(home-page "https://curl.haxx.se/")))
+(define-public curl-full
+ (package/inherit curl
+ (name "curl-full")
+ (version "8.5.0")
+ (source (origin
+ (method url-fetch)
+ (uri (string-append "https://curl.se/download/curl-"
+ version ".tar.gz"))
+ (sha256
+ (base32
+ "1pas8z1vijp08za5xgh2vvsajwi1p2208vlhl0vs94xp4pzigz05"))
+ (patches (search-patches "curl-8.5-use-ssl-cert-env.patch"))))
+ (arguments
+ (substitute-keyword-arguments (package-arguments curl)
+ ((#:configure-flags flags)
+ #~(cons* "--enable-manual"
+ ;; Not yet available: https://github.com/curl/curl/pull/11922
+ ;; "--enable-ech"
+ "--enable-websockets"
+ "--with-brotli"
+ "--with-ldap"
+ "--with-ldaps"
+ "--with-librtmp"
+ "--with-libssh2"
+ "--with-nghttp3"
+ "--with-ngtcp2"
+ "--with-zstd"
+ #$flags))
+ ((#:phases phases)
+ #~(modify-phases #$phases
+ (add-after 'unpack 'skip-tests
+ (lambda _
+ (let ((port (open-file "tests/data/DISABLED" "a")))
+ ;; I do not understand why curl package ignores this test
+ ;; just on some architectures. I had it fail even on amd64,
+ ;; it is just too flaky.
+ (display "1474\n" port)
+ ;; Required files are missing, should be fixed in >8.5.0:
+ ;; https://github.com/curl/curl/issues/12462
+ (display "1477\n" port)
+ (close port))))))))
+ (propagated-inputs
+ (list nss-certs))
+ (native-inputs
+ (modify-inputs (package-native-inputs curl)
+ (prepend groff)))
+ (inputs
+ (modify-inputs (package-inputs curl)
+ (prepend brotli
+ gsasl
+ libpsl
+ libssh2
+ nghttp3
+ ngtcp2
+ openldap
+ rtmpdump
+ `(,zstd "lib"))))
+ (description (string-append (package-description curl)
+ "\n\n"
+ "Compared to the regular curl package, this
+one enables everything that can be enabled, bringing in additional features
+and security."))))
+
(define-public curl-ssh
(package/inherit curl
(arguments
diff --git a/gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch b/gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch
new file mode 100644
index 0000000000..70f703a999
--- /dev/null
+++ b/gnu/packages/patches/curl-8.5-use-ssl-cert-env.patch
@@ -0,0 +1,79 @@
+From 0a942e9f2f5255793e99408ca3da0940d52dfdad Mon Sep 17 00:00:00 2001
+From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba@kadziolka.net>
+Date: Mon, 8 Jan 2024 15:07:38 +0100
+Subject: [PATCH] libcurl: Respect SSL_CERT_DIR and SSL_CERT_FILE.
+
+Make libcurl respect the SSL_CERT_{DIR,FILE} variables by default. The variables
+are fetched during initialization to preserve thread-safety (curl_global_init(3)
+must be called when no other threads exist).
+
+This fixes network functionality in rust:cargo, and probably removes the need
+for other future workarounds.
+---
+ lib/easy.c | 9 +++++++++
+ lib/url.c | 15 +++++++++++++++
+ 2 files changed, 24 insertions(+)
+
+diff --git a/lib/easy.c b/lib/easy.c
+index 322d1a41b..2a235a19c 100644
+--- a/lib/easy.c
++++ b/lib/easy.c
+@@ -137,6 +137,9 @@ curl_wcsdup_callback Curl_cwcsdup = Curl_wcsdup;
+ static char *leakpointer;
+ #endif
+
++char * Curl_ssl_cert_dir = NULL;
++char * Curl_ssl_cert_file = NULL;
++
+ /**
+ * curl_global_init() globally initializes curl given a bitwise set of the
+ * different features of what to initialize.
+@@ -163,6 +166,9 @@ static CURLcode global_init(long flags, bool memoryfuncs)
+ goto fail;
+ }
+
++ Curl_ssl_cert_dir = curl_getenv("SSL_CERT_DIR");
++ Curl_ssl_cert_file = curl_getenv("SSL_CERT_FILE");
++
+ if(!Curl_ssl_init()) {
+ DEBUGF(fprintf(stderr, "Error: Curl_ssl_init failed\n"));
+ goto fail;
+@@ -287,6 +293,9 @@ void curl_global_cleanup(void)
+ Curl_ssl_cleanup();
+ Curl_resolver_global_cleanup();
+
++ free(Curl_ssl_cert_dir);
++ free(Curl_ssl_cert_file);
++
+ #ifdef _WIN32
+ Curl_win32_cleanup(easy_init_flags);
+ #endif
+diff --git a/lib/url.c b/lib/url.c
+index b81785fe2..d607e74bb 100644
+--- a/lib/url.c
++++ b/lib/url.c
+@@ -454,6 +454,21 @@ CURLcode Curl_init_userdefined(struct Curl_easy *data)
+ if(result)
+ return result;
+ #endif
++ extern char * Curl_ssl_cert_dir;
++ extern char * Curl_ssl_cert_file;
++ if(Curl_ssl_cert_dir) {
++ if(result = Curl_setstropt(&set->str[STRING_SSL_CAPATH], Curl_ssl_cert_dir))
++ return result;
++ if(result = Curl_setstropt(&set->str[STRING_SSL_CAPATH_PROXY], Curl_ssl_cert_dir))
++ return result;
++ }
++
++ if(Curl_ssl_cert_file) {
++ if(result = Curl_setstropt(&set->str[STRING_SSL_CAFILE], Curl_ssl_cert_file))
++ return result;
++ if(result = Curl_setstropt(&set->str[STRING_SSL_CAFILE_PROXY], Curl_ssl_cert_file))
++ return result;
++ }
+ }
+
+ #ifndef CURL_DISABLE_FTP
+--
+2.41.0
+
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 2/6] gnu: Add nghttp3.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
9dda8133cace802ec620c95dd0ae741fc0bebe30.1704741394.git.~@wolfsden.cz
* gnu/packages/web.scm (nghttp3): New variable.

Change-Id: I4ed318506f6d3bdd0c47a367b1dcc1b4454c09b2
---
gnu/packages/web.scm | 36 ++++++++++++++++++++++++++++++++++++
1 file changed, 36 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index e0f8bf4547..3c7ff49fb9 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -65,6 +65,7 @@
;;; Copyright © 2023 Christopher Howard <christopher@librehacker.com>
;;; Copyright © 2023 Felix Lechner <felix.lechner@lease-up.com>
;;; Copyright © 2023 Evgeny Pisemsky <evgeny@pisemsky.com>
+;;; Copyright © 2024 Tomas Volf <~@wolfsden.cz>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -8137,6 +8138,41 @@ (define-public nghttp2-for-node
(("print \\(ver >= '3\\.8'\\)")
"print (tuple(map(int, ver.split('.'))) >= (3,8))")))))))))))
+(define-public nghttp3
+ (package
+ (name "nghttp3")
+ (version "1.1.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/ngtcp2/nghttp3/"
+ "releases/download/v" version "/"
+ "nghttp3-" version ".tar.gz"))
+ (sha256
+ (base32
+ "1fzvadnwb03jlm180313gg5m4fg09qdcc67fwcfrv9zs22anaa55"))))
+ (build-system gnu-build-system)
+ (native-inputs
+ (list pkg-config
+ ;; Required by tests.
+ cunit))
+ (arguments
+ (list
+ #:configure-flags
+ #~(list "--enable-lib-only")))
+ (home-page "https://nghttp2.org/nghttp3/")
+ (synopsis "HTTP/3 protocol library")
+ (description
+ "nghttp3 is an implementation of RFC 9114 HTTP/3 mapping over QUIC and
+RFC 9204 QPACK in C. It does not depend on any particular QUIC transport
+implementation.
+
+It implements extensions specified in RFC 9218 and RFC 9220. It supports
+SETTINGS_H3_DATAGRAM from RFC 9297.
+
+It does not support server push.")
+ (license license:expat)))
+
(define-public hpcguix-web
(package
(name "hpcguix-web")
--
2.41.0
T
T
Tomas Volf wrote on 8 Jan 20:38 +0100
[PATCH 4/6] gnu: ngtcp2: Build libngtcp2_crypto_gnutls.
(address . 68332@debbugs.gnu.org)(name . Tomas Volf)(address . ~@wolfsden.cz)
84ab70db28e7aa5c9a90f14f489971a19601eaac.1704741394.git.~@wolfsden.cz
* gnu/packages/networking.scm (ngtcp2)[arguments]<#:configure-flags>: Add
--with-gnutls.
[native-inputs]: Add pkg-config.
[inputs]: Add gnutls.

Change-Id: Id3a9210992d203a57a9b791f3c3752f2ebd3e634
---
gnu/packages/networking.scm | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)

Toggle diff (29 lines)
diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index c738716792..40155480c9 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -61,6 +61,7 @@
;;; Copyright © 2023 Yovan Naumovski <yovan@gorski.stream>
;;; Copyright © 2023 Zheng Junjie <873216071@qq.com>
;;; Copyright © 2023 Artyom V. Poptsov <poptsov.artyom@gmail.com>
+;;; Copyright © 2024 Tomas Volf <~@wolfsden.cz>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -4395,7 +4396,13 @@ (define-public ngtcp2
(sha256
(base32 "1pppl6s25hz91w6321g1q7dqvfy4vccz9mmc5r8sfdvdc95fngl0"))))
(build-system gnu-build-system)
- (native-inputs (list cunit))
+ (arguments
+ (list
+ #:configure-flags
+ ;; openssl package does not support QUIC interface, so just gnutls
+ #~(list "--with-gnutls")))
+ (native-inputs (list cunit pkg-config))
+ (inputs (list gnutls))
(home-page "https://nghttp2.org/ngtcp2/")
(synopsis "QUIC protocol implementation")
(description
--
2.41.0
T
T
Tomas Volf wrote on 17 Jan 15:00 +0100
control message for bug #68332
(address . control@debbugs.gnu.org)
87cyu0qctm.fsf@wolfsden.cz
close 68332
quit

Was told on IRC that it would be ideal to drop the curl-full and just
enable the options for regular curl. Therefore I am closing this for
now and will send it for next core-updates cycle.
M
M
Mathieu Othacehe wrote on 17 Jan 18:22 +0100
Re: [bug#68332] [PATCH 0/6] Add curl-full
(name . Tomas Volf)(address . ~@wolfsden.cz)(address . 68332@debbugs.gnu.org)
875xzrkh78.fsf@gnu.org
Hello Tomas,

Toggle quote (5 lines)
> gnu: web: Delete trailing whitespace.
> gnu: Add nghttp3.
> gnu: ngtcp2: Update to 1.1.0.
> gnu: ngtcp2: Build libngtcp2_crypto_gnutls.

I took the liberty of applying those 4 even if you closed this issue, as
discussed on #guix.

Thanks,

Mathieu
?
Your comment

This issue is archived.

To comment on this conversation send an email to 68332@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 68332
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch