[PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.

DoneSubmitted by Philip McGrath.
Details
7 participants
  • Jelle Licht
  • Leo Famulari
  • Liliana Marie Prikler
  • Philip McGrath
  • Pierre Langlois
  • Ryan Sundberg
  • Timothy Sample
Owner
unassigned
Severity
normal
P
P
Philip McGrath wrote on 14 Nov 2021 13:41
(address . guix-patches@gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114124151.36743-1-philip@philipmcgrath.com
This patch series adds support to `node-build-system` for building native
add-ons using `node-gyp`. To confirm that the changes to the build system
work, the patch series also adds two packages (plus a few dependencies)
representing two major ways of writing Node.js native add-ons:

1. `node-openzwave-shared` uses `node-nan`: “Native Abstractions for Node.js”.

2. `node-sqlite3` uses the Node-API via `node-addon-api`.

The patches are adapted from part of a series by Pierre Langlois
at https://issues.guix.gnu.org/49946: you can find discussion of an earlier
version of this spun-off series from https://issues.guix.gnu.org/49946#71
They are also available in a Git repository

In the earlier discussion, Pierre and I both thought it was quite confusing
that the Scheme variable `node` refers to the bootstrap Node.js, while
`node-lts` refers to the newer Node.js that a package specification of
`"node"` would resolve to. My analysis is
at https://issues.guix.gnu.org/49946#75 (tl;dr it seems like an unintentional
mistake), and Pierre suggested a solution
in https://issues.guix.gnu.org/49946#81. I haven't taken that on in this patch
series, but I do think its worth considering.

Philip McGrath (11):
gnu: node: Avoid duplicating build phases.
gnu: node: Update to 10.24.1 for bootstrapping.
guix: node-build-system: Support compiling add-ons with node-gyp.
gnu: Add node-inherits.
gnu: Add node-safe-buffer.
gnu: Add node-string-decoder.
gnu: Add node-readable-stream.
gnu: Add node-nan.
gnu: Add node-openzwave-shared.
gnu: Add node-addon-api.
gnu: Add node-sqlite3.

gnu/packages/node-xyz.scm | 335 +++++++++++++++++++++++++++++++
gnu/packages/node.scm | 143 +++++--------
gnu/packages/zwave.scm | 55 +++++
guix/build-system/node.scm | 11 +-
guix/build/node-build-system.scm | 23 +++
5 files changed, 473 insertions(+), 94 deletions(-)

--
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 13:58
[PATCH 01/11] gnu: node: Avoid duplicating build phases.
(address . 51838@debbugs.gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114125830.45427-1-philip@philipmcgrath.com
* gnu/packages/node.scm (node)[arguments]: Split 'patch-files phase
into 'patch-hardcoded-program-references and
'delete-problematic-tests. Adapt 'patch-hardcoded-program-references
and 'configure to work unmodified on node-lts, but don't try to
share 'delete-problematic-tests, because those details seem to change
too much between node versions.
(node, node-lts)[inputs]: Use bash-minimal rather than bash.
(node-lts)[arguments]: Inherit 'patch-hardcoded-program-references,
and 'configure phases from the bootstrap node.
Remove the 'patch-files phase, keeping its remaining
non-inherited work in a new 'replace-llhttp-sources phase.
---
gnu/packages/node.scm | 105 +++++++++++++-----------------------------
1 file changed, 32 insertions(+), 73 deletions(-)

Toggle diff (170 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index d0ffe8a398..db1658f03d 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -9,6 +9,7 @@
 ;;; Copyright © 2020, 2021 Pierre Langlois <pierre.langlois@gmx.com>
 ;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -105,14 +106,22 @@ (define-public node
        #:test-target "test-ci-js"
        #:phases
        (modify-phases %standard-phases
-         (add-before 'configure 'patch-files
+         (add-before 'configure 'patch-hardcoded-program-references
            (lambda* (#:key inputs #:allow-other-keys)
+
              ;; Fix hardcoded /bin/sh references.
-             (substitute* '("lib/child_process.js"
-                            "lib/internal/v8_prof_polyfill.js"
-                            "test/parallel/test-child-process-spawnsync-shell.js"
-                            "test/parallel/test-stdio-closed.js"
-                            "test/sequential/test-child-process-emfile.js")
+             (substitute*
+                 (let ((common
+                        '("lib/child_process.js"
+                          "lib/internal/v8_prof_polyfill.js"
+                          "test/parallel/test-child-process-spawnsync-shell.js"
+                          "test/parallel/test-stdio-closed.js"
+                          "test/sequential/test-child-process-emfile.js"))
+                       ;; not in bootstap node:
+                       (sigxfsz "test/parallel/test-fs-write-sigxfsz.js"))
+                   (if (file-exists? sigxfsz)
+                       (cons sigxfsz common)
+                       common))
                (("'/bin/sh'")
                 (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
 
@@ -122,7 +131,10 @@ (define-public node
                             "test/parallel/test-child-process-exec-env.js")
                (("'/usr/bin/env'")
                 (string-append "'" (assoc-ref inputs "coreutils")
-                               "/bin/env'")))
+                               "/bin/env'")))))
+         (add-after 'patch-hardcoded-program-references
+             'delete-problematic-tests
+           (lambda* (#:key inputs #:allow-other-keys)
 
              ;; FIXME: These tests fail in the build container, but they don't
              ;; seem to be indicative of real problems in practice.
@@ -217,9 +229,13 @@ (define-public node
                (setenv "CXX" ,(cxx-for-target))
                (setenv "PKG_CONFIG" ,(pkg-config-for-target))
                (apply invoke
-                      (string-append (assoc-ref (or native-inputs inputs)
-                                                "python")
-                                     "/bin/python")
+                      (let ((python
+                             (string-append (assoc-ref (or native-inputs inputs)
+                                                       "python")
+                                            "/bin/python")))
+                        (if (file-exists? python)
+                            python
+                            (string-append python "3")))
                       "configure" flags))))
          (add-after 'patch-shebangs 'patch-npm-shebang
            (lambda* (#:key outputs #:allow-other-keys)
@@ -256,7 +272,7 @@ (define-public node
             (variable "NODE_PATH")
             (files '("lib/node_modules")))))
     (inputs
-     `(("bash" ,bash)
+     `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
        ("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
@@ -711,66 +727,8 @@ (define-public node-lts
                                    libuv "/lib:"
                                    zlib "/lib"
                                    "'],"))))))
-           (replace 'configure
-             ;; Node's configure script is actually a python script, so we can't
-             ;; run it with bash.
-             (lambda* (#:key outputs (configure-flags '()) native-inputs inputs
-                       #:allow-other-keys)
-               (let* ((prefix (assoc-ref outputs "out"))
-                      (xflags ,(if (%current-target-system)
-                                   `'("--cross-compiling"
-                                     ,(string-append
-                                       "--dest-cpu="
-                                       (match (%current-target-system)
-                                         ((? (cut string-prefix? "arm" <>))
-                                          "arm")
-                                         ((? (cut string-prefix? "aarch64" <>))
-                                          "arm64")
-                                         ((? (cut string-prefix? "i686" <>))
-                                          "ia32")
-                                         ((? (cut string-prefix? "x86_64" <>))
-                                          "x64")
-                                         ((? (cut string-prefix? "powerpc64" <>))
-                                          "ppc64")
-                                         (_ "unsupported"))))
-                                   ''()))
-                      (flags (cons
-                               (string-append "--prefix=" prefix)
-                               (append xflags configure-flags))))
-                 (format #t "build directory: ~s~%" (getcwd))
-                 (format #t "configure flags: ~s~%" flags)
-                 ;; Node's configure script expects the CC environment variable to
-                 ;; be set.
-                 (setenv "CC_host" "gcc")
-                 (setenv "CXX_host" "g++")
-                 (setenv "CC" ,(cc-for-target))
-                 (setenv "CXX" ,(cxx-for-target))
-                 (setenv "PKG_CONFIG" ,(pkg-config-for-target))
-                 (apply invoke
-                        (string-append (assoc-ref (or native-inputs inputs)
-                                                  "python")
-                                       "/bin/python3")
-                        "configure" flags))))
-           (replace 'patch-files
+           (replace 'delete-problematic-tests
              (lambda* (#:key inputs #:allow-other-keys)
-               ;; Fix hardcoded /bin/sh references.
-               (substitute* '("lib/child_process.js"
-                              "lib/internal/v8_prof_polyfill.js"
-                              "test/parallel/test-child-process-spawnsync-shell.js"
-                              "test/parallel/test-fs-write-sigxfsz.js"
-                              "test/parallel/test-stdio-closed.js"
-                              "test/sequential/test-child-process-emfile.js")
-                 (("'/bin/sh'")
-                  (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
-
-               ;; Fix hardcoded /usr/bin/env references.
-               (substitute* '("test/parallel/test-child-process-default-options.js"
-                              "test/parallel/test-child-process-env.js"
-                              "test/parallel/test-child-process-exec-env.js")
-                 (("'/usr/bin/env'")
-                  (string-append "'" (assoc-ref inputs "coreutils")
-                                 "/bin/env'")))
-
                ;; FIXME: These tests fail in the build container, but they don't
                ;; seem to be indicative of real problems in practice.
                (for-each delete-file
@@ -806,8 +764,9 @@ (define-public node-lts
                ;; TODO: Regenerate certs instead.
                (for-each delete-file
                          '("test/parallel/test-tls-passphrase.js"
-                           "test/parallel/test-tls-server-verify.js"))
-
+                           "test/parallel/test-tls-server-verify.js"))))
+           (add-after 'delete-problematic-tests 'replace-llhttp-sources
+             (lambda* (#:key inputs #:allow-other-keys)
                ;; Replace pre-generated llhttp sources
                (let ((llhttp (assoc-ref inputs "llhttp")))
                  (copy-file (string-append llhttp "/src/llhttp.c")
@@ -834,7 +793,7 @@ (define-public node-lts
        ("python" ,python)
        ("util-linux" ,util-linux)))
     (inputs
-     `(("bash" ,bash)
+     `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
        ("c-ares" ,c-ares)
        ("icu4c" ,icu4c-67)
-- 
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 14:04
[PATCH 02/11] gnu: node: Update to 10.24.1 for bootstrapping.
(address . 51838@debbugs.gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114130409.49241-1-philip@philipmcgrath.com
* gnu/packages/node.scm (node): Update to 10.24.1.
---
gnu/packages/node.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (23 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index db1658f03d..89a9bc7533 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -57,14 +57,14 @@ (define-module (gnu packages node)
 (define-public node
   (package
     (name "node")
-    (version "10.24.0")
+    (version "10.24.1")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://nodejs.org/dist/v" version
                                   "/node-v" version ".tar.xz"))
               (sha256
                (base32
-                "1k1srdis23782hnd1ymgczs78x9gqhv77v0am7yb54gqcspp70hm"))
+                "032801kg24j04xmf09m0vxzlcz86sv21s24lv9l4cfv08k1c4byp"))
               (modules '((guix build utils)))
               (snippet
                `(begin
-- 
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 14:04
[PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp.
(address . 51838@debbugs.gnu.org)
20211114130409.49241-2-philip@philipmcgrath.com
* gnu/packages/node.scm (node)[arguments]: Replace 'patch-npm-shebang
and 'patch-node-shebang with a new 'patch-nested-shebangs that also
handles node-gyp and other shebangs under "/lib/node_modules".
[inputs]: Add Python for node-gyp as "python-for-target".
(node-lts)[inputs]: Likewise.
(libnode)[arguments]: Adjust to delete 'patch-nested-shebangs rather
than 'patch-npm-shebang and 'patch-node-shebang.
* guix/build-system/node.scm (lower): Add optional #:python argument
and corresponding implicit input. Add the version of libuv used
as an input to the #:node package as a new implicit input.
* guix/build/node-build-system.scm (set-node-gyp-paths): New
function. Sets the "npm_config_nodedir" and "npm_config_python"
environment variables. Adds the "node-gyp-bin" directory to "PATH".
(configure-gyp): New function. Run `node-gyp configure` if we see
a `binding.gyp` file.
(%standard-phases): Add 'set-node-gyp-paths after 'set-paths.
Add 'configure-gyp after 'configure.

Co-authored-by: Pierre Langlois <pierre.langlois@gmx.com>
---
gnu/packages/node.scm | 34 ++++++++++++++++----------------
guix/build-system/node.scm | 11 +++++++++--
guix/build/node-build-system.scm | 23 +++++++++++++++++++++
3 files changed, 49 insertions(+), 19 deletions(-)

Toggle diff (173 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 89a9bc7533..ad5179426a 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -237,21 +237,20 @@ (define-public node
                             python
                             (string-append python "3")))
                       "configure" flags))))
-         (add-after 'patch-shebangs 'patch-npm-shebang
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((bindir (string-append (assoc-ref outputs "out")
-                                           "/bin"))
-                    (npm    (string-append bindir "/npm"))
-                    (target (readlink npm)))
-               (with-directory-excursion bindir
-                 (patch-shebang target (list bindir))))))
-         (add-after 'install 'patch-node-shebang
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((bindir (string-append (assoc-ref outputs "out")
-                                           "/bin"))
-                    (npx    (readlink (string-append bindir "/npx"))))
-               (with-directory-excursion bindir
-                 (patch-shebang npx (list bindir)))))))))
+         (add-after 'patch-shebangs 'patch-nested-shebangs
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let* ((prefix (assoc-ref outputs "out"))
+                    (path (map (lambda (dir)
+                                 (string-append dir "/bin"))
+                               (list prefix
+                                     (assoc-ref inputs "python-for-target")))))
+               (for-each
+                (lambda (file)
+                  (patch-shebang file path))
+                (find-files (string-append prefix "/lib/node_modules")
+                            (lambda (file stat)
+                              (executable-file? file))
+                            #:stat lstat))))))))
     (native-inputs
      `(;; Runtime dependencies for binaries used as a bootstrap.
        ("c-ares" ,c-ares)
@@ -274,6 +273,7 @@ (define-public node
     (inputs
      `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
+       ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
        ("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
        ("icu4c" ,icu4c)
@@ -795,6 +795,7 @@ (define-public node-lts
     (inputs
      `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
+       ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
        ("c-ares" ,c-ares)
        ("icu4c" ,icu4c-67)
        ("libuv" ,libuv-for-node)
@@ -813,5 +814,4 @@ (define-public libnode
         `(cons* "--shared" "--without-npm" ,flags))
        ((#:phases phases '%standard-phases)
         `(modify-phases ,phases
-           (delete 'patch-npm-shebang)
-           (delete 'patch-node-shebang)))))))
+           (delete 'patch-nested-shebangs)))))))
diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 98f63f87ef..fee4142a99 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -1,6 +1,8 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
 ;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -24,6 +26,7 @@ (define-module (guix build-system node)
   #:use-module (guix search-paths)
   #:use-module (guix build-system)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system python)
   #:use-module (ice-9 match)
   #:export (%node-build-system-modules
             node-build
@@ -44,11 +47,12 @@ (define (default-node)
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
                 (node (default-node))
+                (python (default-python)) ;; for node-gyp
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
   (define private-keywords
-    '(#:source #:target #:node #:inputs #:native-inputs))
+    '(#:source #:target #:node #:python #:inputs #:native-inputs))
 
   (and (not target)                    ;XXX: no cross-compilation
        (bag
@@ -58,10 +62,13 @@ (define private-keywords
                               `(("source" ,source))
                               '())
                         ,@inputs
-
                         ;; Keep the standard inputs of 'gnu-build-system'.
                         ,@(standard-packages)))
          (build-inputs `(("node" ,node)
+                         ("python" ,python)
+                        ;; We don't always need libuv, but the libuv and
+                        ;; node versions need to match:
+                        ("libuv" ,@(assoc-ref (package-inputs node) "libuv"))
                          ,@native-inputs))
          (outputs outputs)
          (build node-build)
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 70a367618e..6aeb0149dd 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -2,6 +2,8 @@
 ;;; Copyright © 2015 David Thompson <davet@gnu.org>
 ;;; Copyright © 2016, 2020 Jelle Licht <jlicht@fsfe.org>
 ;;; Copyright © 2019, 2021 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -46,6 +48,19 @@ (define (set-home . _)
               (format #t "set HOME to ~s~%" (getenv "HOME")))))))
   #t)
 
+(define* (set-node-gyp-paths #:key inputs #:allow-other-keys)
+  "Initialize environment variables needed for building native addons."
+  (setenv "npm_config_nodedir" (assoc-ref inputs "node"))
+  (setenv "npm_config_python" (assoc-ref inputs "python"))
+  (setenv "PATH"
+          (string-append (getenv "PATH")
+                         ":"
+                         ;; Put this at the end to make it easier to override,
+                         ;; just in case that should ever be necessary:
+                         (assoc-ref inputs "node")
+                         "/lib/node_modules/npm/bin/node-gyp-bin"))
+  #t)
+
 (define (module-name module)
   (let* ((package.json (string-append module "/package.json"))
          (package-meta (call-with-input-file package.json read-json)))
@@ -101,6 +116,12 @@ (define* (configure #:key outputs inputs #:allow-other-keys)
     (invoke npm "--offline" "--ignore-scripts" "install")
     #t))
 
+(define* (configure-gyp #:key inputs #:allow-other-keys)
+  "Run 'node-gyp configure' if we see a 'binding.gyp' file."
+  (if (file-exists? "binding.gyp")
+      (invoke (which "node-gyp") "configure")
+      #t))
+
 (define* (build #:key inputs #:allow-other-keys)
   (let ((package-meta (call-with-input-file "package.json" read-json)))
     (if (and=> (assoc-ref package-meta "scripts")
@@ -144,9 +165,11 @@ (define* (install #:key outputs inputs #:allow-other-keys)
 
 (define %standard-phases
   (modify-phases gnu:%standard-phases
+    (add-after 'set-paths 'set-node-gyp-paths set-node-gyp-paths)
     (add-after 'unpack 'set-home set-home)
     (add-before 'configure 'patch-dependencies patch-dependencies)
     (replace 'configure configure)
+    (add-after 'configure 'configure-gyp configure-gyp)
     (replace 'build build)
     (replace 'check check)
     (add-before 'install 'repack repack)
-- 
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 14:04
[PATCH 04/11] gnu: Add node-inherits.
(address . 51838@debbugs.gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114130409.49241-3-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-inherits): New variable.
---
gnu/packages/node-xyz.scm | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)

Toggle diff (52 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 98c611f227..da6affeb1f 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2020 Giacomo Leidi <goodoldpaul@autistici.org>
 ;;; Copyright © 2021 Noisytoot <noisytoot@disroot.org>
 ;;; Copyright © 2021 Charles <charles.b.jackson@protonmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -358,6 +359,37 @@ (define-public node-once
 if desired.")
     (license license:isc)))
 
+(define-public node-inherits
+  (package
+    (name "node-inherits")
+    (version "2.0.4")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/isaacs/inherits")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0cpsr5yqwkxpbbbbl0rwk4mcby6zbx841k2zb4c3gb1579i5wq9p"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f
+       #:phases
+       (modify-phases %standard-phases
+         ;; Ignore our lack of tap (for tests) as with node-once.
+         (delete 'configure))))
+    (home-page
+     "https://github.com/isaacs/inherits")
+    (synopsis
+     "Browser-friendly inheritance Node.js")
+    (description
+     "Browser-friendly inheritance fully compatible with standard Node.js
+@code{inherits()}.")
+    (license license:isc)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 14:04
[PATCH 05/11] gnu: Add node-safe-buffer.
(address . 51838@debbugs.gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114130409.49241-4-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-safe-buffer): New variable.
---
gnu/packages/node-xyz.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (43 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index da6affeb1f..44000e0568 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -390,6 +390,36 @@ (define-public node-inherits
 @code{inherits()}.")
     (license license:isc)))
 
+(define-public node-safe-buffer
+  (package
+    (name "node-safe-buffer")
+    (version "5.2.1")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/feross/safe-buffer")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0r26m0nl41h90ihnl2xf0cqs6z9z7jb87dl5j8yqb7887r9jlbpi"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f
+       #:phases
+       (modify-phases %standard-phases
+         ;; Skip checking devDependencies, since we don't have tape.
+         (delete 'configure))))
+    (home-page
+     "https://github.com/feross/safe-buffer")
+    (synopsis "Safer Node.js Buffer API")
+    (description "A safe drop-in replacement the Node.js @code{Buffer} API
+that works in all versions of Node.js, using the built-in implementation when
+available.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 14:04
[PATCH 06/11] gnu: Add node-string-decoder.
(address . 51838@debbugs.gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114130409.49241-5-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-string-decoder): New variable.
---
gnu/packages/node-xyz.scm | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)

Toggle diff (47 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 44000e0568..a5ca5631c7 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -420,6 +420,40 @@ (define-public node-safe-buffer
 available.")
     (license license:expat)))
 
+(define-public node-string-decoder
+  (package
+    (name "node-string-decoder")
+    (version "1.3.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/string_decoder")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0xxvyya9fl9rlkqwmxzqzbz4rdr3jgw4vf37hff7cgscxkhg266k"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f
+       #:phases
+       (modify-phases %standard-phases
+         ;; Ignore missing tap dependency (for tests).
+         (delete 'configure))))
+    (inputs
+     `(("node-safe-buffer" ,node-safe-buffer)
+       ("node-inherits" ,node-inherits)))
+    (home-page
+     "https://github.com/nodejs/string_decoder")
+    (synopsis
+     "Node.js core @code{string_decoder} for userland")
+    (description
+     "This package is a mirror of the @code{string_decoder} implementation in
+Node-core.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 14:04
[PATCH 07/11] gnu: Add node-readable-stream.
(address . 51838@debbugs.gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114130409.49241-6-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-readable-stream): New variable.
---
gnu/packages/node-xyz.scm | 38 ++++++++++++++++++++++++++++++++++++++
1 file changed, 38 insertions(+)

Toggle diff (51 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index a5ca5631c7..d0b4d1fa98 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -454,6 +454,44 @@ (define-public node-string-decoder
 Node-core.")
     (license license:expat)))
 
+(define-public node-readable-stream
+  (package
+    (name "node-readable-stream")
+    (version "3.6.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/readable-stream")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0ybl4cdgsm9c5jq3xq8s01201jk8w0yakh63hlclsfbcdfqhd9ri"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f
+       #:phases
+       (modify-phases %standard-phases
+         ;; Ignore numerous missing devDependencies
+         (delete 'configure))))
+    (inputs
+     `(("node-util-deprecate" ,node-util-deprecate)
+       ("node-string-decoder" ,node-string-decoder)
+       ("node-inherits" ,node-inherits)))
+    (home-page
+     "https://github.com/nodejs/readable-stream")
+    (synopsis
+     "Node.js core streams for userland")
+    (description
+     "This package is a mirror of the streams implementations in Node.js.
+
+If you want to guarantee a stable streams base, regardless of what version of
+Node you (or the users of your libraries) are using, use
+@code{readable-stream} only and avoid the @code{stream} module in Node-core.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 14:04
[PATCH 08/11] gnu: Add node-nan.
(address . 51838@debbugs.gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114130409.49241-7-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-nan): New variable.
---
gnu/packages/node-xyz.scm | 41 +++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)

Toggle diff (51 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index d0b4d1fa98..29ad14f810 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -549,3 +549,44 @@ (define-public node-irc
     (description "@code{node-irc} is an IRC client library for Node.js.
 It has functions for joining, parting, talking, and many other IRC commands.")
     (license license:gpl3+)))
+
+(define-public node-nan
+  (package
+    (name "node-nan")
+    (version "2.15.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/nan")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "18xslh9va5ld872scrp5y4251ax9s3c6qh0lnl1200lpzbsxy7yd"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       ;; We can't run the npm-driven tests,
+       ;; but a number of C++ tests do run anyway:
+       #f
+       #:phases
+       (modify-phases %standard-phases
+         ;; Ignore missing devDependencies (e.g. "tap", for tests):
+         (delete 'configure))))
+    (inputs
+     `(("readable-stream" ,node-readable-stream)))
+    (home-page "https://github.com/nodejs/nan")
+    (synopsis "Native Abstractions for Node.js")
+    (description "A header file filled with macro and utility goodness for
+making add-on development for Node.js easier across versions 0.8, 0.10, 0.12,
+1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 and 16.
+
+Thanks to the crazy changes in V8 (and some in Node core), keeping native
+addons compiling happily across versions, particularly 0.10 to 0.12 to 4.0, is
+a minor nightmare.  The goal of this project is to store all logic necessary
+to develop native Node.js addons without having to inspect
+@code{NODE_MODULE_VERSION} and get yourself into a macro-tangle.
+
+This project also contains some helper utilities that make addon development a
+bit more pleasant.")
+    (license license:expat)))
-- 
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 14:04
[PATCH 09/11] gnu: Add node-openzwave-shared.
(address . 51838@debbugs.gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114130409.49241-8-philip@philipmcgrath.com
* gnu/packages/zwave.scm (node-openzwave-shared): New variable.
---
gnu/packages/zwave.scm | 55 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)

Toggle diff (80 lines)
diff --git a/gnu/packages/zwave.scm b/gnu/packages/zwave.scm
index 2019ec32df..39028b4f34 100644
--- a/gnu/packages/zwave.scm
+++ b/gnu/packages/zwave.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2019 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -21,8 +22,10 @@ (define-module (gnu packages zwave)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system node)
   #:use-module (gnu packages)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages node-xyz)
   #:use-module (gnu packages libusb)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages perl)
@@ -88,3 +91,55 @@ (define-public open-zwave
 and respond to devices on a Z-Wave network, without requiring in-depth
 knowledge of the Z-Wave protocol.")
     (license license:lgpl3+)))
+
+(define-public node-openzwave-shared
+  (package
+    (name "node-openzwave-shared")
+    (version "1.7.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/OpenZWave/node-openzwave-shared")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "1bqqy12dzqj05j9wsh50dmn84dddmhz0gjzvd3y20z4hpy1v8rsc"))))
+    (inputs
+     `(("open-zwave" ,open-zwave)
+       ("node-nan" ,node-nan)))
+    (native-inputs
+     `(("which" ,which)
+       ("pkg-config" ,pkg-config)))
+    (build-system node-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (replace 'build
+           (lambda* (#:key inputs #:allow-other-keys)
+             (invoke (which "node-gyp") "build"))))))
+    (home-page "https://github.com/OpenZWave/node-openzwave-shared")
+    (synopsis "Node.js bindings for OpenZWave")
+    (description
+     "With the @code{node-openzwave-shared} package, you can easily control
+and manage your Z-Wave devices (lights, dimmers, blinds, you name it) from
+within Node.js applications.  This library also supports secure
+devices (e.g. door locks) that require encryption.  All widely used Node.js
+versions are supported with the help of @code{node-nan}.
+
+This library is currently able to:
+@itemize @bullet
+@item
+scan a Z-Wave network and report on connected devices;
+@item
+write values to Z-Wave nodes;
+@item
+monitor the network for changes;
+@item
+heal nodes and/or the network; and
+@item
+perform management tasks: add or remove nodes, replace failed nodes,
+manage their group associations, etc.
+@end itemize")
+    (license license:isc)))
-- 
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 14:04
[PATCH 10/11] gnu: Add node-addon-api.
(address . 51838@debbugs.gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114130409.49241-9-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-addon-api): New variable.
---
gnu/packages/node-xyz.scm | 46 +++++++++++++++++++++++++++++++++++++++
1 file changed, 46 insertions(+)

Toggle diff (56 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 29ad14f810..8aa93122df 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -590,3 +590,49 @@ (define-public node-nan
 This project also contains some helper utilities that make addon development a
 bit more pleasant.")
     (license license:expat)))
+
+(define-public node-addon-api
+  (package
+    (name "node-addon-api")
+    (version "4.2.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/node-addon-api")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "1bhvfi2m9nxfz418s619914vmidcnrzbjv6l9nid476c3zlpazch"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         ;; Ignore numerous missing devDependencies:
+         (delete 'configure)
+         (add-after 'unpack 'skip-js-tests
+           ;; We can't run the js-based tests,
+           ;; but we can still do the C++ parts
+           (lambda args
+             (substitute* "package.json"
+               (("\"test\": \"node test\"")
+                "\"test\": \"echo stopping after pretest on Guix\"")))))))
+    (home-page "https://github.com/nodejs/node-addon-api")
+    (synopsis "Node.js API (Node-API) header-only C++ wrappers")
+    (description "This module contains header-only C++ wrapper classes which
+simplify the use of the C based Node-API provided by Node.js when using C++.
+It provides a C++ object model and exception handling semantics with low
+overhead.
+
+Node-API is an ABI stable C interface provided by Node.js for building native
+addons.  It is intended to insulate native addons from changes in the
+underlying JavaScript engine and allow modules compiled for one version to run
+on later versions of Node.js without recompilation.  The @code{node-addon-api}
+module, which is not part of Node.js, preserves the benefits of the Node-API
+as it consists only of inline code that depends only on the stable API
+provided by Node-API.
+
+It is important to remember that @emph{other} Node.js interfaces such as
+@code{libuv} (included in a project via @code{#include <uv.h>}) are not
+ABI-stable across Node.js major versions.")
+    (license license:expat)))
-- 
2.32.0
P
P
Philip McGrath wrote on 14 Nov 2021 14:04
[PATCH 11/11] gnu: Add node-sqlite3.
(address . 51838@debbugs.gnu.org)(name . Philip McGrath)(address . philip@philipmcgrath.com)
20211114130409.49241-10-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-sqlite3): New variable.
---
gnu/packages/node-xyz.scm | 114 ++++++++++++++++++++++++++++++++++++++
1 file changed, 114 insertions(+)

Toggle diff (131 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 8aa93122df..d30b6f6e04 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -22,6 +22,8 @@
 
 (define-module (gnu packages node-xyz)
   #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (gnu packages sqlite)
+  #:use-module (guix gexp)
   #:use-module (guix packages)
   #:use-module (guix git-download)
   #:use-module (guix build-system node))
@@ -636,3 +638,115 @@ (define-public node-addon-api
 @code{libuv} (included in a project via @code{#include <uv.h>}) are not
 ABI-stable across Node.js major versions.")
     (license license:expat)))
+
+(define-public node-sqlite3
+  (package
+    (name "node-sqlite3")
+    (version "5.0.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/mapbox/node-sqlite3")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "0sbbzzli282nxyfha10zx0k5m8hdp0sf3ipl59khjb7wm449j86h"))
+       (snippet
+        (with-imported-modules '((guix build utils))
+          #~(begin
+              (use-modules (guix build utils))
+              ;; unbundle sqlite
+              '(for-each delete-file-recursively
+                         (find-files "deps" gzip-file?)))))))
+    (inputs
+     `(("node-addon-api" ,node-addon-api)
+       ("sqlite" ,sqlite)))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f ; FIXME: tests depend on node-mocha
+       #:modules
+       ((guix build node-build-system)
+        (guix build json)
+        (srfi srfi-1)
+        (ice-9 match)
+        (guix build utils))
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'npm-config-sqlite
+           (lambda* (#:key inputs #:allow-other-keys)
+             (setenv "npm_config_sqlite" (assoc-ref inputs "sqlite"))))
+         (add-after 'unpack 'un-cloud-ify
+           ;; Normally, this is "built" using @mapbox/node-pre-gyp,
+           ;; which publishes or downloads pre-built binaries
+           ;; or falls back to building from source.
+           ;; Here, we patch out all of that and just build directly.
+           ;; It would be better to patch a version of @mapbox/node-pre-gyp
+           ;; that always builds from source, but there are a number
+           ;; of dependencies that need to be packaged or removed.
+           (lambda args
+             (with-atomic-file-replacement "package.json"
+               (lambda (in out)
+                 (let ((js (read-json in)))
+                   (match (assoc-ref js "binary")
+                     (('@ . alist)
+                      (setenv "GYP_DEFINES"
+                              (string-append
+                               "module_name="
+                               (assoc-ref alist "module_name")
+                               " "
+                               "module_path="
+                               (assoc-ref alist "module_path")))))
+                   (write-json
+                    (fold (match-lambda*
+                            (((key update) js)
+                             (assoc-set! js
+                                         key
+                                         (match (assoc-ref js key)
+                                           (('@ . alist)
+                                            (cons '@ (update alist)))
+                                           (other
+                                            (update other))))))
+                          js
+                          `(("dependencies"
+                             ,(lambda (deps)
+                                (assoc-remove!
+                                 (assoc-remove! deps "@mapbox/node-pre-gyp")
+                                 "node-pre-gyp")))
+                            ("devDependencies"
+                             ,(lambda (deps)
+                                (assoc-remove!
+                                 (assoc-remove! deps "aws-sdk")
+                                 "@mapbox/cloudfriend")))
+                            ("scripts"
+                             ,(lambda (scripts)
+                                ;; install script would use node-pre-gyp
+                                (assoc-remove! scripts "install")))))
+                    out))))))
+         (add-after 'un-cloud-ify 'remove-missing-dev-deps
+           ;; Remove some missing testing/linting dependencies
+           ;; so we don't have to skip the configure phase entirely.
+           (lambda args
+             (with-atomic-file-replacement "package.json"
+               (lambda (in out)
+                 (write-json
+                  (let ((js (read-json in)))
+                    (assoc-set!
+                     (assoc-remove!
+                      (assoc-remove! (assoc-remove! js "peerDependencies")
+                                     "peerDependenciesMeta")
+                      "optionalDependencies")
+                     "devDependencies"
+                     (match (assoc-ref js "devDependencies")
+                       (('@ . deps)
+                        (cons '@ (assoc-remove! (assoc-remove! deps "eslint")
+                                                "mocha"))))))
+                  out))))))))
+    (home-page "https://github.com/mapbox/node-sqlite3")
+    (synopsis "Asynchronous, non-blocking SQLite3 bindings for Node.js")
+    (description
+     "The Node.js add-on @code{node-sqlite3} provides a set of a asynchronous,
+non-blocking bindings for SQLite3, written in modern C++ and tested for memory
+leaks.")
+     (license license:bsd-3)))
-- 
2.32.0
L
L
Liliana Marie Prikler wrote on 14 Nov 2021 21:44
Re: [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp.
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
48018e12484d19466d9c6f253a8d7ebeae93e947.camel@gmail.com
Hi Philip

Am Sonntag, den 14.11.2021, 08:04 -0500 schrieb Philip McGrath:
Toggle quote (19 lines)
> * gnu/packages/node.scm (node)[arguments]: Replace 'patch-npm-shebang
> and 'patch-node-shebang with a new 'patch-nested-shebangs that also
> handles node-gyp and other shebangs under "/lib/node_modules".
> [inputs]: Add Python for node-gyp as "python-for-target".
> (node-lts)[inputs]: Likewise.
> (libnode)[arguments]: Adjust to delete 'patch-nested-shebangs rather
> than 'patch-npm-shebang and 'patch-node-shebang.
> * guix/build-system/node.scm (lower): Add optional #:python argument
> and corresponding implicit input. Add the version of libuv used
> as an input to the #:node package as a new implicit input.
> * guix/build/node-build-system.scm (set-node-gyp-paths): New
> function. Sets the "npm_config_nodedir" and "npm_config_python"
> environment variables. Adds the "node-gyp-bin" directory to "PATH".
> (configure-gyp): New function. Run `node-gyp configure` if we see
> a `binding.gyp` file.
> (%standard-phases): Add 'set-node-gyp-paths after 'set-paths.
> Add 'configure-gyp after 'configure.
>
> Co-authored-by: Pierre Langlois <pierre.langlois@gmx.com>
Looking at this patch, it does *a lot* at once and could probably be
separated into more than one. Particularly, I'd suggest providing
capabilities in node-build-system first, then switching over to the new
thing in node.

Toggle quote (53 lines)
> [...]
> --- a/guix/build-system/node.scm
> +++ b/guix/build-system/node.scm
> @@ -1,6 +1,8 @@
> ;;; GNU Guix --- Functional package management for GNU
> ;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
> ;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
> +;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
> +;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -24,6 +26,7 @@ (define-module (guix build-system node)
> #:use-module (guix search-paths)
> #:use-module (guix build-system)
> #:use-module (guix build-system gnu)
> + #:use-module (guix build-system python)
> #:use-module (ice-9 match)
> #:export (%node-build-system-modules
> node-build
> @@ -44,11 +47,12 @@ (define (default-node)
> (define* (lower name
> #:key source inputs native-inputs outputs system
> target
> (node (default-node))
> + (python (default-python)) ;; for node-gyp
> #:allow-other-keys
> #:rest arguments)
> "Return a bag for NAME."
> (define private-keywords
> - '(#:source #:target #:node #:inputs #:native-inputs))
> + '(#:source #:target #:node #:python #:inputs #:native-inputs))
>
> (and (not target) ;XXX: no cross-compilation
> (bag
> @@ -58,10 +62,13 @@ (define private-keywords
> `(("source" ,source))
> '())
> ,@inputs
> -
> ;; Keep the standard inputs of 'gnu-build-
> system'.
> ,@(standard-packages)))
> (build-inputs `(("node" ,node)
> + ("python" ,python)
> + ;; We don't always need libuv, but the libuv
> and
> + ;; node versions need to match:
> + ("libuv" ,@(assoc-ref (package-inputs node)
> "libuv"))
> ,@native-inputs))
> (outputs outputs)
> (build node-build)
Will this python input always or often enough be needed? What's the
build system ratio on node like, gyp vs. anything else, particularly
with packages close to the node core?

Toggle quote (34 lines)
> diff --git a/guix/build/node-build-system.scm b/guix/build/node-
> build-system.scm
> index 70a367618e..6aeb0149dd 100644
> --- a/guix/build/node-build-system.scm
> +++ b/guix/build/node-build-system.scm
> @@ -2,6 +2,8 @@
> ;;; Copyright © 2015 David Thompson <davet@gnu.org>
> ;;; Copyright © 2016, 2020 Jelle Licht <jlicht@fsfe.org>
> ;;; Copyright © 2019, 2021 Timothy Sample <samplet@ngyro.com>
> +;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
> +;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -46,6 +48,19 @@ (define (set-home . _)
> (format #t "set HOME to ~s~%" (getenv "HOME")))))))
> #t)
>
> +(define* (set-node-gyp-paths #:key inputs #:allow-other-keys)
> + "Initialize environment variables needed for building native
> addons."
> + (setenv "npm_config_nodedir" (assoc-ref inputs "node"))
> + (setenv "npm_config_python" (assoc-ref inputs "python"))
> + (setenv "PATH"
> + (string-append (getenv "PATH")
> + ":"
> + ;; Put this at the end to make it easier to
> override,
> + ;; just in case that should ever be
> necessary:
> + (assoc-ref inputs "node")
> + "/lib/node_modules/npm/bin/node-gyp-bin"))
> + #t)
> +
Is this a necessary step to build node modules? If not can we skip
this step when packages don't need gyp?

Toggle quote (15 lines)
> (define (module-name module)
> (let* ((package.json (string-append module "/package.json"))
> (package-meta (call-with-input-file package.json read-
> json)))
> @@ -101,6 +116,12 @@ (define* (configure #:key outputs inputs
> #:allow-other-keys)
> (invoke npm "--offline" "--ignore-scripts" "install")
> #t))
>
> +(define* (configure-gyp #:key inputs #:allow-other-keys)
> + "Run 'node-gyp configure' if we see a 'binding.gyp' file."
> + (if (file-exists? "binding.gyp")
> + (invoke (which "node-gyp") "configure")
> + #t))
> +
You might want to make this part of configure itself, though I'm not
sure what the consensus is there when mixing different build system
styles. (invoke (which ...) ) is also a pretty rare pattern, used in
only four locations so far.

Also, while better than the previous thing in that it actually checks
whether we have something gyp-esque at hand, I'd still prefer users
being able to not run this portion through some flag. See e.g. #:use-
setuptools? in python or #:glib-or-gtk? in meson.

Cheers
P
P
Philip McGrath wrote on 20 Nov 2021 05:26
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
5a04aa92-e80d-e11b-235c-b7f5e3a92d00@philipmcgrath.com
Hi!

On 11/14/21 15:44, Liliana Marie Prikler wrote:
Toggle quote (5 lines)
> Looking at this patch, it does *a lot* at once and could probably be
> separated into more than one. Particularly, I'd suggest providing
> capabilities in node-build-system first, then switching over to the new
> thing in node.

Thanks for these comments. Some of the things to which you drew my
attention seem to have been workarounds for problems that had since been
solved at a deeper level. Then, in particular, this comment:


>> +(define* (configure-gyp #:key inputs #:allow-other-keys)
>> + "Run 'node-gyp configure' if we see a 'binding.gyp' file."
>> + (if (file-exists? "binding.gyp")
>> + (invoke (which "node-gyp") "configure")
>> + #t))
>> +
> You might want to make this part of configure itself, though I'm not
> sure what the consensus is there when mixing different build system
> styles. (invoke (which ...) ) is also a pretty rare pattern, used in
> only four locations so far.

prompted me to look more closely at why so much manual work was needed
in the first place.

It turns out that the `npm install` in the `'configure` phase should
have handled most of it automatically, but the Guix packages were
deleting the configure phase to avoid checking for devDependencies that
aren't in Guix (or that we just don't want, e.g. some dependencies of
node-sqlite3).

In v2 of this series (which will follow this email), I've removed all of
the `node-gyp`-specific build-side code and tried a more general
solution, adding an `#:absent-packages` argument to instruct the
`'patch-dependencies` phase to remove the specified packages from the
"package.json" file. This means that the Guix package can still run
`'configure`/`npm install`, checking properly for the packages that we
*do* have/want and doing all of the other automatic work `npm install`
does. I also like that listing the missing packages in the Guix package
definition provides a sort of documentation of what is missing: for
example, it is clear which packages could have their tests enabled with
the addition of a `node-tap` package, without having to inspect all of
the individual "package.json" files.

I've updated all of the existing Node.js packages that deleted their
`'configure` phase to use `#:absent-dependencies` instead.

Toggle quote (22 lines)
>> @@ -58,10 +62,13 @@ (define private-keywords
>> `(("source" ,source))
>> '())
>> ,@inputs
>> -
>> ;; Keep the standard inputs of 'gnu-build-
>> system'.
>> ,@(standard-packages)))
>> (build-inputs `(("node" ,node)
>> + ("python" ,python)
>> + ;; We don't always need libuv, but the libuv
>> and
>> + ;; node versions need to match:
>> + ("libuv" ,@(assoc-ref (package-inputs node)
>> "libuv"))
>> ,@native-inputs))
>> (outputs outputs)
>> (build node-build)
> Will this python input always or often enough be needed? What's the
> build system ratio on node like, gyp vs. anything else, particularly
> with packages close to the node core?

GYP is a Python program, and it (or at least node's fork of it) expects
to have a python executable available to invoke with sub-programs. Since
npm depends on node-gyp and GYP, it is pretty close to the core.

In v2, I've just had packages that use node-gyp add Python to their
inputs. IIRC, that used to not be enough, but it seems underlying
problems were fixed in the mean time.

An alternative approach would be to configure it using the npmrc file,
as I do for `nodedir` in v2. I'm not sure that makes much difference
either way, but in v2 I've tried to minimize the amount of
`node-gyp`-specific handling.

-Philip
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 01/26] gnu: node: Avoid duplicating build phases.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-1-philip@philipmcgrath.com
* gnu/packages/node.scm (node)[arguments]: Split 'patch-files phase
into 'patch-hardcoded-program-references and
'delete-problematic-tests. Adapt 'patch-hardcoded-program-references
and 'configure to work unmodified on node-lts, but don't try to
share 'delete-problematic-tests, because those details seem to change
too much between node versions.
(node, node-lts)[inputs]: Use bash-minimal rather than bash.
(node-lts)[arguments]: Inherit 'patch-hardcoded-program-references,
and 'configure phases from the bootstrap node.
Remove the 'patch-files phase, keeping its remaining
non-inherited work in a new 'replace-llhttp-sources phase.
---
gnu/packages/node.scm | 105 +++++++++++++-----------------------------
1 file changed, 32 insertions(+), 73 deletions(-)

Toggle diff (170 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index dccf871d2c..7ebbbc3060 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -9,6 +9,7 @@
 ;;; Copyright © 2020, 2021 Pierre Langlois <pierre.langlois@gmx.com>
 ;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -105,14 +106,22 @@ (define-public node
        #:test-target "test-ci-js"
        #:phases
        (modify-phases %standard-phases
-         (add-before 'configure 'patch-files
+         (add-before 'configure 'patch-hardcoded-program-references
            (lambda* (#:key inputs #:allow-other-keys)
+
              ;; Fix hardcoded /bin/sh references.
-             (substitute* '("lib/child_process.js"
-                            "lib/internal/v8_prof_polyfill.js"
-                            "test/parallel/test-child-process-spawnsync-shell.js"
-                            "test/parallel/test-stdio-closed.js"
-                            "test/sequential/test-child-process-emfile.js")
+             (substitute*
+                 (let ((common
+                        '("lib/child_process.js"
+                          "lib/internal/v8_prof_polyfill.js"
+                          "test/parallel/test-child-process-spawnsync-shell.js"
+                          "test/parallel/test-stdio-closed.js"
+                          "test/sequential/test-child-process-emfile.js"))
+                       ;; not in bootstap node:
+                       (sigxfsz "test/parallel/test-fs-write-sigxfsz.js"))
+                   (if (file-exists? sigxfsz)
+                       (cons sigxfsz common)
+                       common))
                (("'/bin/sh'")
                 (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
 
@@ -122,7 +131,10 @@ (define-public node
                             "test/parallel/test-child-process-exec-env.js")
                (("'/usr/bin/env'")
                 (string-append "'" (assoc-ref inputs "coreutils")
-                               "/bin/env'")))
+                               "/bin/env'")))))
+         (add-after 'patch-hardcoded-program-references
+             'delete-problematic-tests
+           (lambda* (#:key inputs #:allow-other-keys)
 
              ;; FIXME: These tests fail in the build container, but they don't
              ;; seem to be indicative of real problems in practice.
@@ -217,9 +229,13 @@ (define-public node
                (setenv "CXX" ,(cxx-for-target))
                (setenv "PKG_CONFIG" ,(pkg-config-for-target))
                (apply invoke
-                      (string-append (assoc-ref (or native-inputs inputs)
-                                                "python")
-                                     "/bin/python")
+                      (let ((python
+                             (string-append (assoc-ref (or native-inputs inputs)
+                                                       "python")
+                                            "/bin/python")))
+                        (if (file-exists? python)
+                            python
+                            (string-append python "3")))
                       "configure" flags))))
          (add-after 'patch-shebangs 'patch-npm-shebang
            (lambda* (#:key outputs #:allow-other-keys)
@@ -256,7 +272,7 @@ (define-public node
             (variable "NODE_PATH")
             (files '("lib/node_modules")))))
     (inputs
-     `(("bash" ,bash)
+     `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
        ("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
@@ -712,66 +728,8 @@ (define-public node-lts
                                    libuv "/lib:"
                                    zlib "/lib"
                                    "'],"))))))
-           (replace 'configure
-             ;; Node's configure script is actually a python script, so we can't
-             ;; run it with bash.
-             (lambda* (#:key outputs (configure-flags '()) native-inputs inputs
-                       #:allow-other-keys)
-               (let* ((prefix (assoc-ref outputs "out"))
-                      (xflags ,(if (%current-target-system)
-                                   `'("--cross-compiling"
-                                     ,(string-append
-                                       "--dest-cpu="
-                                       (match (%current-target-system)
-                                         ((? (cut string-prefix? "arm" <>))
-                                          "arm")
-                                         ((? (cut string-prefix? "aarch64" <>))
-                                          "arm64")
-                                         ((? (cut string-prefix? "i686" <>))
-                                          "ia32")
-                                         ((? (cut string-prefix? "x86_64" <>))
-                                          "x64")
-                                         ((? (cut string-prefix? "powerpc64" <>))
-                                          "ppc64")
-                                         (_ "unsupported"))))
-                                   ''()))
-                      (flags (cons
-                               (string-append "--prefix=" prefix)
-                               (append xflags configure-flags))))
-                 (format #t "build directory: ~s~%" (getcwd))
-                 (format #t "configure flags: ~s~%" flags)
-                 ;; Node's configure script expects the CC environment variable to
-                 ;; be set.
-                 (setenv "CC_host" "gcc")
-                 (setenv "CXX_host" "g++")
-                 (setenv "CC" ,(cc-for-target))
-                 (setenv "CXX" ,(cxx-for-target))
-                 (setenv "PKG_CONFIG" ,(pkg-config-for-target))
-                 (apply invoke
-                        (string-append (assoc-ref (or native-inputs inputs)
-                                                  "python")
-                                       "/bin/python3")
-                        "configure" flags))))
-           (replace 'patch-files
+           (replace 'delete-problematic-tests
              (lambda* (#:key inputs #:allow-other-keys)
-               ;; Fix hardcoded /bin/sh references.
-               (substitute* '("lib/child_process.js"
-                              "lib/internal/v8_prof_polyfill.js"
-                              "test/parallel/test-child-process-spawnsync-shell.js"
-                              "test/parallel/test-fs-write-sigxfsz.js"
-                              "test/parallel/test-stdio-closed.js"
-                              "test/sequential/test-child-process-emfile.js")
-                 (("'/bin/sh'")
-                  (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
-
-               ;; Fix hardcoded /usr/bin/env references.
-               (substitute* '("test/parallel/test-child-process-default-options.js"
-                              "test/parallel/test-child-process-env.js"
-                              "test/parallel/test-child-process-exec-env.js")
-                 (("'/usr/bin/env'")
-                  (string-append "'" (assoc-ref inputs "coreutils")
-                                 "/bin/env'")))
-
                ;; FIXME: These tests fail in the build container, but they don't
                ;; seem to be indicative of real problems in practice.
                (for-each delete-file
@@ -810,8 +768,9 @@ (define-public node-lts
                ;; TODO: Regenerate certs instead.
                (for-each delete-file
                          '("test/parallel/test-tls-passphrase.js"
-                           "test/parallel/test-tls-server-verify.js"))
-
+                           "test/parallel/test-tls-server-verify.js"))))
+           (add-after 'delete-problematic-tests 'replace-llhttp-sources
+             (lambda* (#:key inputs #:allow-other-keys)
                ;; Replace pre-generated llhttp sources
                (let ((llhttp (assoc-ref inputs "llhttp")))
                  (copy-file (string-append llhttp "/src/llhttp.c")
@@ -838,7 +797,7 @@ (define-public node-lts
        ("python" ,python)
        ("util-linux" ,util-linux)))
     (inputs
-     `(("bash" ,bash)
+     `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
        ("c-ares" ,c-ares-for-node)
        ("icu4c" ,icu4c-67)
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 03/26] gnu: node: Patch shebangs in node_modules.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-3-philip@philipmcgrath.com
* gnu/packages/node.scm (node)[arguments]: Replace 'patch-npm-shebang
and 'patch-node-shebang with a new 'patch-nested-shebangs that also
handles node-gyp and other shebangs under "/lib/node_modules".
[inputs]: Add Python for node-gyp as "python-for-target".
(node-lts)[inputs]: Likewise.
(libnode)[arguments]: Adjust to delete 'patch-nested-shebangs rather
than 'patch-npm-shebang and 'patch-node-shebang.
---
gnu/packages/node.scm | 34 +++++++++++++++++-----------------
1 file changed, 17 insertions(+), 17 deletions(-)

Toggle diff (65 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 71b66774a6..a57a74fb81 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -237,21 +237,20 @@ (define-public node
                             python
                             (string-append python "3")))
                       "configure" flags))))
-         (add-after 'patch-shebangs 'patch-npm-shebang
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((bindir (string-append (assoc-ref outputs "out")
-                                           "/bin"))
-                    (npm    (string-append bindir "/npm"))
-                    (target (readlink npm)))
-               (with-directory-excursion bindir
-                 (patch-shebang target (list bindir))))))
-         (add-after 'install 'patch-node-shebang
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((bindir (string-append (assoc-ref outputs "out")
-                                           "/bin"))
-                    (npx    (readlink (string-append bindir "/npx"))))
-               (with-directory-excursion bindir
-                 (patch-shebang npx (list bindir)))))))))
+         (add-after 'patch-shebangs 'patch-nested-shebangs
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let* ((prefix (assoc-ref outputs "out"))
+                    (path (map (lambda (dir)
+                                 (string-append dir "/bin"))
+                               (list prefix
+                                     (assoc-ref inputs "python-for-target")))))
+               (for-each
+                (lambda (file)
+                  (patch-shebang file path))
+                (find-files (string-append prefix "/lib/node_modules")
+                            (lambda (file stat)
+                              (executable-file? file))
+                            #:stat lstat))))))))
     (native-inputs
      `(;; Runtime dependencies for binaries used as a bootstrap.
        ("c-ares" ,c-ares)
@@ -274,6 +273,7 @@ (define-public node
     (inputs
      `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
+       ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
        ("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
        ("icu4c" ,icu4c)
@@ -799,6 +799,7 @@ (define-public node-lts
     (inputs
      `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
+       ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
        ("c-ares" ,c-ares-for-node)
        ("icu4c" ,icu4c-67)
        ("libuv" ,libuv-for-node)
@@ -817,5 +818,4 @@ (define-public libnode
         `(cons* "--shared" "--without-npm" ,flags))
        ((#:phases phases '%standard-phases)
         `(modify-phases ,phases
-           (delete 'patch-npm-shebang)
-           (delete 'patch-node-shebang)))))))
+           (delete 'patch-nested-shebangs)))))))
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 04/26] gnu: node: Add an npmrc file to set nodedir.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-4-philip@philipmcgrath.com
* gnu/packages/node.scm (node, node-lis)[arguments]: Add a phase
'install-npmrc to create a "built-in" npmrc file that configures
"nodedir" to point to the output store path.
(libnode)[arguments]: Delete the 'install-npmrc phase.
---
gnu/packages/node.scm | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)

Toggle diff (35 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index a57a74fb81..0f67fe79c2 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -250,7 +250,21 @@ (define-public node
                 (find-files (string-append prefix "/lib/node_modules")
                             (lambda (file stat)
                               (executable-file? file))
-                            #:stat lstat))))))))
+                            #:stat lstat)))))
+         (add-after 'install 'install-npmrc
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (with-output-to-file
+                   ;; Use the config file "primarily for distribution
+                   ;; maintainers" rather than "{prefix}/etc/npmrc",
+                   ;; especially because node-build-system uses --prefix
+                   ;; to install things to their store paths:
+                   (string-append out "/lib/node_modules/npm/npmrc")
+                 (lambda ()
+                   ;; Tell npm (mostly node-gyp) where to find our
+                   ;; installed headers so it doesn't try to
+                   ;; download them from the internet:
+                   (format #t "nodedir=~a\n" out)))))))))
     (native-inputs
      `(;; Runtime dependencies for binaries used as a bootstrap.
        ("c-ares" ,c-ares)
@@ -818,4 +832,5 @@ (define-public libnode
         `(cons* "--shared" "--without-npm" ,flags))
        ((#:phases phases '%standard-phases)
         `(modify-phases ,phases
+           (delete 'install-npmrc)
            (delete 'patch-nested-shebangs)))))))
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-5-philip@philipmcgrath.com
Many of Guix's Node.js packages are built without some of the
dependencies they specify in their "package-lock.json" files,
either because we don't have them packaged yet (e.g. test
utilities) or because we don't want them (e.g. to reduce the
closure size). Previously, Guix package definitions would work
around this situation by deleting the `'configure`
phase (i.e. the initial `npm install`).

This commit adds an optional #:absent-dependencies argument to
`node-build-system` to list Node.js packages that should be
removed from the "package.json" file.Retaining the `'configure`
phase avoids skipping checks for the dependencies that are
intended to be present and other actions performed by `npm
install`, such as automatically building native add-ons with
`node-gyp` when the "gypfile" key is present.

* guix/build-system/node.scm (lower, node-build): Add optional
argument #:absent-dependencies with default of ''(). Pass it on
to the build-side code.
* guix/build/node-build-system.scm (patch-dependencies): Respect
the #:absent-dependencies argument, removing specified npm
packages from the "dependencies" or "devDependencies" tables
in "package.json". Also, strictly follow the linearity rules
for `assoc-set!` and friends.
---
guix/build-system/node.scm | 3 ++
guix/build/node-build-system.scm | 55 ++++++++++++++++++++++----------
2 files changed, 41 insertions(+), 17 deletions(-)

Toggle diff (103 lines)
diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 98f63f87ef..75ae34508f 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -44,6 +44,7 @@ (define (default-node)
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
                 (node (default-node))
+                (absent-dependencies ''())
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
@@ -73,6 +74,7 @@ (define* (node-build store name inputs
                      (tests? #t)
                      (phases '(@ (guix build node-build-system)
                                  %standard-phases))
+                     (absent-dependencies ''())
                      (outputs '("out"))
                      (search-paths '())
                      (system (%current-system))
@@ -94,6 +96,7 @@ (define builder
                    #:test-target ,test-target
                    #:tests? ,tests?
                    #:phases ,phases
+                   #:absent-dependencies ,absent-dependencies
                    #:outputs %outputs
                    #:search-paths ',(map search-path-specification->sexp
                                          search-paths)
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 70a367618e..32d6807e3e 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -69,30 +69,51 @@ (define (list-modules directory)
               input-paths)
     index))
 
-(define* (patch-dependencies #:key inputs #:allow-other-keys)
+(define* (patch-dependencies #:key inputs absent-dependencies
+                             #:allow-other-keys)
 
   (define index (index-modules (map cdr inputs)))
 
-  (define (resolve-dependencies package-meta meta-key)
-    (fold (lambda (key+value acc)
-            (match key+value
-              ('@ acc)
-              ((key . value) (acons key (hash-ref index key value) acc))))
+  (define (resolve-dependencies meta-alist meta-key)
+    (match (assoc-ref meta-alist meta-key)
+      (#f
+       '())
+      (('@ . orig-deps)
+       (fold (match-lambda*
+               (('@ acc)
+                acc)
+                (((key . value) acc)
+                 (if (member key absent-dependencies)
+                     acc
+                     (acons key (hash-ref index key value) acc))))
           '()
-          (or (assoc-ref package-meta meta-key) '())))
+          orig-deps))))
 
   (with-atomic-file-replacement "package.json"
     (lambda (in out)
-      (let ((package-meta (read-json in)))
-        (assoc-set! package-meta "dependencies"
-                    (append
-                     '(@)
-                     (resolve-dependencies package-meta "dependencies")
-                     (resolve-dependencies package-meta "peerDependencies")))
-        (assoc-set! package-meta "devDependencies"
-                    (append
-                     '(@)
-                     (resolve-dependencies package-meta "devDependencies")))
+      ;; It is unsafe to rely on 'assoc-set!' to update an
+      ;; existing assosciation list variable:
+      ;; see 'info "(guile)Adding or Setting Alist Entries"'.
+      (let* ((package-meta (read-json in))
+             (alist (match package-meta
+                      ((@ . alist) alist)))
+             ;; Other relevant keys may include peerDependenciesMeta
+             ;; and optionalDependencies, but it seems to work out fine
+             ;; just to leave those alone.
+             (alist
+              (assoc-set!
+               alist "dependencies"
+               (append
+                '(@)
+                (resolve-dependencies alist "dependencies")
+                (resolve-dependencies alist "peerDependencies"))))
+             (alist
+              (assoc-set!
+               alist "devDependencies"
+               (append
+                '(@)
+                (resolve-dependencies alist "devDependencies"))))
+             (package-meta (cons '@ alist)))
         (write-json package-meta out))))
   #t)
 
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 06/26] gnu: node-semver-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-6-philip@philipmcgrath.com
gnu/packages/node.scm (node-semver-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node.scm | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)

Toggle diff (18 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 0f67fe79c2..6c958eebec 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -330,9 +330,8 @@ (define-public node-semver-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       '("tap")))
     (home-page "https://github.com/npm/node-semver")
     (properties '((hidden? . #t)))
     (synopsis "Parses semantic versions strings")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 07/26] gnu: node-ms-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-7-philip@philipmcgrath.com
gnu/packages/node.scm (node-ms-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node.scm | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)

Toggle diff (22 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 6c958eebec..ca4bf1ad23 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -358,9 +358,12 @@ (define-public node-ms-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       `("eslint"
+         "expect.js"
+         "husky"
+         "lint-staged"
+         "mocha")))
     (home-page "https://github.com/zeit/ms#readme")
     (properties '((hidden? . #t)))
     (synopsis "Tiny millisecond conversion utility")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 08/26] gnu: node-binary-search-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-8-philip@philipmcgrath.com
gnu/packages/node.scm (node-binary-search-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node.scm | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

Toggle diff (19 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index ca4bf1ad23..2cdae34ae1 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -389,9 +389,9 @@ (define-public node-binary-search-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       `("chai"
+         "mocha")))
     (home-page "https://github.com/darkskyapp/binary-search#readme")
     (properties '((hidden? . #t)))
     (synopsis "Tiny binary search function with comparators")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 02/26] gnu: node: Update to 10.24.1 for bootstrapping.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-2-philip@philipmcgrath.com
* gnu/packages/node.scm (node): Update to 10.24.1.
---
gnu/packages/node.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (23 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 7ebbbc3060..71b66774a6 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -57,14 +57,14 @@ (define-module (gnu packages node)
 (define-public node
   (package
     (name "node")
-    (version "10.24.0")
+    (version "10.24.1")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://nodejs.org/dist/v" version
                                   "/node-v" version ".tar.xz"))
               (sha256
                (base32
-                "1k1srdis23782hnd1ymgczs78x9gqhv77v0am7yb54gqcspp70hm"))
+                "032801kg24j04xmf09m0vxzlcz86sv21s24lv9l4cfv08k1c4byp"))
               (modules '((guix build utils)))
               (snippet
                `(begin
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 09/26] gnu: node-debug-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-9-philip@philipmcgrath.com
gnu/packages/node.scm (node-debug-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node.scm | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)

Toggle diff (28 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 2cdae34ae1..98a51276e7 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -416,9 +416,18 @@ (define-public node-debug-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       `("brfs"
+         "browserify"
+         "coveralls"
+         "istanbul"
+         "karma"
+         "karma-browserify"
+         "karma-chrome-launcher"
+         "karma-mocha"
+         "mocha"
+         "mocha-lcov-reporter"
+         "xo")))
     (inputs `(("node-ms" ,node-ms-bootstrap)))
     (home-page "https://github.com/visionmedia/debug#readme")
     (properties '((hidden? . #t)))
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-10-philip@philipmcgrath.com
gnu/packages/node.scm (node-llparse-builder-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
Add a new phase `#:delete-package-lock` to remove the
problematic "package-lock.json".
---
gnu/packages/node.scm | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)

Toggle diff (29 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 98a51276e7..9d4903a8ca 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -479,9 +479,21 @@ (define-public node-llparse-builder-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
+       #:absent-dependencies
+       `("@types/mocha"
+         "@types/node"
+         "mocha"
+         "ts-node"
+         "tslint"
+         "typescript")
        #:phases
        (modify-phases %standard-phases
-         (delete 'configure)
+         (add-before 'configure 'remove-package-lock
+           ;; Having package-lock.json seems to cause npm
+           ;; to look for things on the internet in the configure phase,
+           ;; even if we have them properly installed.
+           (lambda args
+             (delete-file-recursively "package-lock.json")))
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 11/26] gnu: node-llparse-frontend-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-11-philip@philipmcgrath.com
gnu/packages/node.scm (node-llparse-frontend-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node.scm | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 9d4903a8ca..298b9376a8 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -547,9 +547,16 @@ (define-public node-llparse-frontend-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
+       #:absent-dependencies
+       `("@types/debug"
+         "@types/mocha"
+         "@types/node"
+         "mocha"
+         "ts-node"
+         "tslint"
+         "typescript")
        #:phases
        (modify-phases %standard-phases
-         (delete 'configure)
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 12/26] gnu: node-llparse-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-12-philip@philipmcgrath.com
gnu/packages/node.scm (node-llparse-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
Add a new phase `#:delete-package-lock` to remove the
problematic "package-lock.json".
---
gnu/packages/node.scm | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 298b9376a8..6d48816c77 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -609,9 +609,24 @@ (define-public node-llparse-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
+       #:absent-dependencies
+       `("@types/debug"
+         "@types/mocha"
+         "@types/node"
+         "esm"
+         "llparse-test-fixture"
+         "mocha"
+         "ts-node"
+         "tslint"
+         "typescript")
        #:phases
        (modify-phases %standard-phases
-         (delete 'configure)
+         (add-before 'configure 'remove-package-lock
+           ;; Having package-lock.json seems to cause npm
+           ;; to look for things on the internet in the configure phase,
+           ;; even if we have them properly installed.
+           (lambda args
+             (delete-file-recursively "package-lock.json")))
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 13/26] gnu: node-semver: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-13-philip@philipmcgrath.com
gnu/packages/node-xyz.scm (node-semver)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node-xyz.scm | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)

Toggle diff (22 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 98c611f227..93a033d33e 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -289,11 +289,10 @@ (define-public node-semver
                 "06biknqb05r9xsmcflm3ygh50pjvdk84x6r79w43kmck4fn3qn5p"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f ; FIXME: Tests depend on node-tap
-       #:phases
-       (modify-phases %standard-phases
-         ;; The only dependency to check for is tap, which we don't have.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       '("tap")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
     (home-page "https://github.com/npm/node-semver")
     (synopsis "Parses semantic versions strings")
     (description
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 14/26] gnu: node-wrappy: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-14-philip@philipmcgrath.com
gnu/packages/node-xyz.scm (node-wrappy)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node-xyz.scm | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)

Toggle diff (19 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 93a033d33e..77577cb315 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -316,10 +316,8 @@ (define-public node-wrappy
     (build-system node-build-system)
     (arguments
      '(#:tests? #f ; FIXME: Tests depend on node-tap
-       #:phases
-       (modify-phases %standard-phases
-         ;; The only dependency to check for is tap, which we don't have.
-         (delete 'configure))))
+       #:absent-dependencies
+       '("tap")))
     (home-page "https://github.com/npm/wrappy")
     (synopsis "Callback wrapping utility")
     (description "@code{wrappy} is a utility for Node.js to wrap callbacks.")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 15/26] gnu: node-once: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-15-philip@philipmcgrath.com
gnu/packages/node-xyz.scm (node-once)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node-xyz.scm | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 77577cb315..eb071320cd 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -338,13 +338,10 @@ (define-public node-once
                 "1z8dcbf28dqdcp4wb0c53wrs90a07nkrax2c9kk26dsk1dhrnxav"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f                      ; FIXME: tests depend on node-tap
-       #:phases
-       (modify-phases %standard-phases
-         ;; The default configure phase fails due to tap being missing, as we do
-         ;; not have tap packaged yet.  It is used only for tests.  This package
-         ;; still works as a dependency of node-glob and node-inflight.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       '("tap")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
     (inputs
      `(("node-wrappy" ,node-wrappy)))
     (home-page "https://github.com/isaacs/once")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 16/26] gnu: node-irc-colors: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-16-philip@philipmcgrath.com
gnu/packages/node-xyz.scm (node-irc-colors)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node-xyz.scm | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)

Toggle diff (23 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index eb071320cd..9646c14243 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -367,12 +367,10 @@ (define-public node-irc-colors
                 "0q3y34rbnlc55jcakmdxkicwazyvyph9r6gaf6hi8k7wj2nfwfli"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f                      ; FIXME: tests depend on node-istanbul
-       #:phases
-       (modify-phases %standard-phases
-         ;; The default configure phase fails due to various packages
-         ;; being missing, as we don't have them packaged yet.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       `("istanbul"
+         "vows")
+       #:tests? #f))
     (home-page "https://github.com/fent/irc-colors.js")
     (synopsis "Node.js module providing color and formatting for IRC")
     (description "@code{node-irc-colors} is a Node.js module that
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 18/26] guix: node-build-system: Add optional #:libuv? argument.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-18-philip@philipmcgrath.com
* guix/build-system/node.scm (lower): Add an optional #:libuv?
argument to tell the build system to add the version of libuv
used as an input to the #:node package as an additional implicit
input, so that packages needing libuv always get the correct version.
---
guix/build-system/node.scm | 15 +++++++++++++--
1 file changed, 13 insertions(+), 2 deletions(-)

Toggle diff (50 lines)
diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 75ae34508f..f83a7f64ce 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -1,6 +1,8 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
 ;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -44,12 +46,13 @@ (define (default-node)
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
                 (node (default-node))
+                (libuv? #f)
                 (absent-dependencies ''())
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
   (define private-keywords
-    '(#:source #:target #:node #:inputs #:native-inputs))
+    '(#:source #:target #:node #:libuv? #:inputs #:native-inputs))
 
   (and (not target)                    ;XXX: no cross-compilation
        (bag
@@ -59,10 +62,18 @@ (define private-keywords
                               `(("source" ,source))
                               '())
                         ,@inputs
-
                         ;; Keep the standard inputs of 'gnu-build-system'.
                         ,@(standard-packages)))
          (build-inputs `(("node" ,node)
+                         ;; Many packages with native addons need
+                         ;; libuv headers. The libuv version must
+                         ;; be exactly the same as for the node
+                         ;; package we are adding implicitly,
+                         ;; so we take care of adding libuv, too.
+                         ,@(if libuv?
+                               `(("libuv" ,@(assoc-ref (package-inputs node)
+                                                       "libuv")))
+                               '())
                          ,@native-inputs))
          (outputs outputs)
          (build node-build)
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 19/26] gnu: Add node-inherits.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-19-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-inherits): New variable.
---
gnu/packages/node-xyz.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (50 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 998d0a9a90..9d8bf27852 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2020 Giacomo Leidi <goodoldpaul@autistici.org>
 ;;; Copyright © 2021 Noisytoot <noisytoot@disroot.org>
 ;;; Copyright © 2021 Charles <charles.b.jackson@protonmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -352,6 +353,35 @@ (define-public node-once
 if desired.")
     (license license:isc)))
 
+(define-public node-inherits
+  (package
+    (name "node-inherits")
+    (version "2.0.4")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/isaacs/inherits")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0cpsr5yqwkxpbbbbl0rwk4mcby6zbx841k2zb4c3gb1579i5wq9p"))))
+    (build-system node-build-system)
+    (arguments
+     '(#:absent-dependencies
+       '("tap")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
+    (home-page
+     "https://github.com/isaacs/inherits")
+    (synopsis
+     "Browser-friendly inheritance Node.js")
+    (description
+     "Browser-friendly inheritance fully compatible with standard Node.js
+@code{inherits()}.")
+    (license license:isc)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:34
[PATCH v2 20/26] gnu: Add node-safe-buffer.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-20-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-safe-buffer): New variable.
---
gnu/packages/node-xyz.scm | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)

Toggle diff (41 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 9d8bf27852..6dff0ddba5 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -382,6 +382,34 @@ (define-public node-inherits
 @code{inherits()}.")
     (license license:isc)))
 
+(define-public node-safe-buffer
+  (package
+    (name "node-safe-buffer")
+    (version "5.2.1")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/feross/safe-buffer")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0r26m0nl41h90ihnl2xf0cqs6z9z7jb87dl5j8yqb7887r9jlbpi"))))
+    (build-system node-build-system)
+    (arguments
+     '(#:absent-dependencies
+       '("tape"
+         "standard")
+       #:tests? #f))
+    (home-page
+     "https://github.com/feross/safe-buffer")
+    (synopsis "Safer Node.js Buffer API")
+    (description "A safe drop-in replacement the Node.js @code{Buffer} API
+that works in all versions of Node.js, using the built-in implementation when
+available.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:34
[PATCH v2 21/26] gnu: Add node-string-decoder.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-21-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-string-decoder): New variable.
---
gnu/packages/node-xyz.scm | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)

Toggle diff (47 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 6dff0ddba5..c6cce64f7d 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -410,6 +410,40 @@ (define-public node-safe-buffer
 available.")
     (license license:expat)))
 
+(define-public node-string-decoder
+  (package
+    (name "node-string-decoder")
+    (version "1.3.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/string_decoder")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0xxvyya9fl9rlkqwmxzqzbz4rdr3jgw4vf37hff7cgscxkhg266k"))))
+    (build-system node-build-system)
+    (arguments
+     '(#:absent-dependencies
+       '("tap"
+         "core-util-is"
+         "babel-polyfill")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
+    (inputs
+     `(("node-safe-buffer" ,node-safe-buffer)
+       ("node-inherits" ,node-inherits)))
+    (home-page
+     "https://github.com/nodejs/string_decoder")
+    (synopsis
+     "Node.js core @code{string_decoder} for userland")
+    (description
+     "This package is a mirror of the @code{string_decoder} implementation in
+Node-core.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:34
[PATCH v2 22/26] gnu: Add node-readable-stream.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-22-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-readable-stream): New variable.
---
gnu/packages/node-xyz.scm | 54 +++++++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)

Toggle diff (67 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index c6cce64f7d..3e06413908 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -444,6 +444,60 @@ (define-public node-string-decoder
 Node-core.")
     (license license:expat)))
 
+(define-public node-readable-stream
+  (package
+    (name "node-readable-stream")
+    (version "3.6.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/readable-stream")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0ybl4cdgsm9c5jq3xq8s01201jk8w0yakh63hlclsfbcdfqhd9ri"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:absent-dependencies
+       `("@babel/cli"
+         "@babel/core"
+         "@babel/polyfill"
+         "@babel/preset-env"
+         "airtap"
+         "assert"
+         "bl"
+         "deep-strict-equal"
+         "events.once"
+         "glob"
+         "gunzip-maybe"
+         "hyperquest"
+         "lolex"
+         "nyc"
+         "pump"
+         "rimraf"
+         "tap"
+         "tape"
+         "tar-fs"
+         "util-promisify")
+       #:tests? #f))
+    (inputs
+     `(("node-util-deprecate" ,node-util-deprecate)
+       ("node-string-decoder" ,node-string-decoder)
+       ("node-inherits" ,node-inherits)))
+    (home-page
+     "https://github.com/nodejs/readable-stream")
+    (synopsis
+     "Node.js core streams for userland")
+    (description
+     "This package is a mirror of the streams implementations in Node.js.
+
+If you want to guarantee a stable streams base, regardless of what version of
+Node you (or the users of your libraries) are using, use
+@code{readable-stream} only and avoid the @code{stream} module in Node-core.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:34
[PATCH v2 24/26] gnu: Add node-openzwave-shared.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-24-philip@philipmcgrath.com
* gnu/packages/zwave.scm (node-openzwave-shared): New variable.
---
gnu/packages/zwave.scm | 67 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 67 insertions(+)

Toggle diff (95 lines)
diff --git a/gnu/packages/zwave.scm b/gnu/packages/zwave.scm
index 2019ec32df..8757ddecf2 100644
--- a/gnu/packages/zwave.scm
+++ b/gnu/packages/zwave.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2019 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -21,11 +22,14 @@ (define-module (gnu packages zwave)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system node)
   #:use-module (gnu packages)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages node-xyz)
   #:use-module (gnu packages libusb)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages perl)
+  #:use-module (gnu packages python)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages xml))
 
@@ -88,3 +92,66 @@ (define-public open-zwave
 and respond to devices on a Z-Wave network, without requiring in-depth
 knowledge of the Z-Wave protocol.")
     (license license:lgpl3+)))
+
+(define-public node-openzwave-shared
+  (package
+    (name "node-openzwave-shared")
+    (version "1.7.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/OpenZWave/node-openzwave-shared")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "1bqqy12dzqj05j9wsh50dmn84dddmhz0gjzvd3y20z4hpy1v8rsc"))))
+    (inputs
+     `(("open-zwave" ,open-zwave)
+       ("node-nan" ,node-nan)))
+    (native-inputs
+     `(("which" ,which)
+       ("python" ,python)
+       ("pkg-config" ,pkg-config)))
+    (build-system node-build-system)
+    (arguments
+     `(#:libuv?
+       #t
+       #:phases
+       (modify-phases %standard-phases
+         (replace 'build
+           (lambda* (#:key inputs #:allow-other-keys)
+             (define node-dir
+               (assoc-ref inputs "node"))
+             (invoke (string-append node-dir "/bin/npx")
+                     "--call"
+                     (string-append
+                      node-dir
+                      "/lib/node_modules/npm/bin/node-gyp-bin/node-gyp"
+                      " "
+                      "rebuild")))))))
+    (home-page "https://github.com/OpenZWave/node-openzwave-shared")
+    (synopsis "Node.js bindings for OpenZWave")
+    (description
+     "With the @code{node-openzwave-shared} package, you can easily control
+and manage your Z-Wave devices (lights, dimmers, blinds, you name it) from
+within Node.js applications.  This library also supports secure
+devices (e.g. door locks) that require encryption.  All widely used Node.js
+versions are supported with the help of @code{node-nan}.
+
+This library is currently able to:
+@itemize @bullet
+@item
+scan a Z-Wave network and report on connected devices;
+@item
+write values to Z-Wave nodes;
+@item
+monitor the network for changes;
+@item
+heal nodes and/or the network; and
+@item
+perform management tasks: add or remove nodes, replace failed nodes,
+manage their group associations, etc.
+@end itemize")
+    (license license:isc)))
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:34
[PATCH v2 25/26] gnu: Add node-addon-api.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-25-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-addon-api): New variable.
---
gnu/packages/node-xyz.scm | 63 +++++++++++++++++++++++++++++++++++++++
1 file changed, 63 insertions(+)

Toggle diff (80 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index ed169c0778..60dbfc163c 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -22,6 +22,9 @@
 
 (define-module (gnu packages node-xyz)
   #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (gnu packages sqlite)
+  #:use-module (gnu packages python)
+  #:use-module (guix gexp)
   #:use-module (guix packages)
   #:use-module (guix git-download)
   #:use-module (guix build-system node))
@@ -597,3 +600,63 @@ (define-public node-nan
 This project also contains some helper utilities that make addon development a
 bit more pleasant.")
     (license license:expat)))
+
+(define-public node-addon-api
+  (package
+    (name "node-addon-api")
+    (version "4.2.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/node-addon-api")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "1bhvfi2m9nxfz418s619914vmidcnrzbjv6l9nid476c3zlpazch"))))
+    (inputs
+     `(("python" ,python)))
+    (build-system node-build-system)
+    (arguments
+     `(#:absent-dependencies
+       `("benchmark"
+         "bindings"
+         "clang-format"
+         "eslint"
+         "eslint-config-semistandard"
+         "eslint-config-standard"
+         "eslint-plugin-import"
+         "eslint-plugin-node"
+         "eslint-plugin-promise"
+         "fs-extra"
+         "path"
+         "pre-commit"
+         "safe-buffer")
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'skip-js-tests
+           ;; We can't run the js-based tests,
+           ;; but we can still do the C++ parts
+           (lambda args
+             (substitute* "package.json"
+               (("\"test\": \"node test\"")
+                "\"test\": \"echo stopping after pretest on Guix\"")))))))
+    (home-page "https://github.com/nodejs/node-addon-api")
+    (synopsis "Node.js API (Node-API) header-only C++ wrappers")
+    (description "This module contains header-only C++ wrapper classes which
+simplify the use of the C based Node-API provided by Node.js when using C++.
+It provides a C++ object model and exception handling semantics with low
+overhead.
+
+Node-API is an ABI stable C interface provided by Node.js for building native
+addons.  It is intended to insulate native addons from changes in the
+underlying JavaScript engine and allow modules compiled for one version to run
+on later versions of Node.js without recompilation.  The @code{node-addon-api}
+module, which is not part of Node.js, preserves the benefits of the Node-API
+as it consists only of inline code that depends only on the stable API
+provided by Node-API.
+
+It is important to remember that @emph{other} Node.js interfaces such as
+@code{libuv} (included in a project via @code{#include <uv.h>}) are not
+ABI-stable across Node.js major versions.")
+    (license license:expat)))
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:34
[PATCH v2 26/26] gnu: Add node-sqlite3.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-26-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-sqlite3): New variable.
---
gnu/packages/node-xyz.scm | 107 ++++++++++++++++++++++++++++++++++++--
1 file changed, 104 insertions(+), 3 deletions(-)

Toggle diff (131 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 60dbfc163c..a56281fe18 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -615,7 +615,8 @@ (define-public node-addon-api
        (sha256
         (base32 "1bhvfi2m9nxfz418s619914vmidcnrzbjv6l9nid476c3zlpazch"))))
     (inputs
-     `(("python" ,python)))
+     `(("python" ,python)
+       ("node-safe-buffer" ,node-safe-buffer)))
     (build-system node-build-system)
     (arguments
      `(#:absent-dependencies
@@ -630,8 +631,7 @@ (define-public node-addon-api
          "eslint-plugin-promise"
          "fs-extra"
          "path"
-         "pre-commit"
-         "safe-buffer")
+         "pre-commit")
        #:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'skip-js-tests
@@ -660,3 +660,104 @@ (define-public node-addon-api
 @code{libuv} (included in a project via @code{#include <uv.h>}) are not
 ABI-stable across Node.js major versions.")
     (license license:expat)))
+
+(define-public node-sqlite3
+  (package
+    (name "node-sqlite3")
+    (version "5.0.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/mapbox/node-sqlite3")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "0sbbzzli282nxyfha10zx0k5m8hdp0sf3ipl59khjb7wm449j86h"))
+       (snippet
+        (with-imported-modules '((guix build utils))
+          #~(begin
+              (use-modules (guix build utils))
+              ;; unbundle sqlite
+              '(for-each delete-file-recursively
+                         (find-files "deps" gzip-file?)))))))
+    (inputs
+     `(("node-addon-api" ,node-addon-api)
+       ("python" ,python)
+       ("sqlite" ,sqlite)))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f ; FIXME: tests depend on node-mocha
+       #:modules
+       ((guix build node-build-system)
+        (guix build json)
+        (srfi srfi-1)
+        (ice-9 match)
+        (guix build utils))
+       #:libuv? #t
+       #:absent-dependencies
+       `(;; Normally, this is "built" using @mapbox/node-pre-gyp,
+         ;; which publishes or downloads pre-built binaries
+         ;; or falls back to building from source.
+         ;; Here, we patch out all of that and just build directly.
+         ;; It would be better to patch a version of @mapbox/node-pre-gyp
+         ;; that always builds from source, as Debian does,
+         ;; but there are a number of dependencies that need
+         ;; to be packaged or removed.
+         "@mapbox/node-pre-gyp"
+         "node-pre-gyp" ;; deprecated name still used in some places
+         "aws-sdk"
+         "@mapbox/cloudfriend"
+         ;; Confusingly, this is only a dependency beceuse of
+         ;; @mapbox/node-pre-gyp: with that removed,
+         ;; npm will use its own copy:
+         "node-gyp"
+         ;; These we'd like, we just don't have them yet:
+         "eslint"
+         "mocha")
+       #:phases
+       (modify-phases %standard-phases
+         ;; We need this step even if we do replace @mapbox/node-pre-gyp
+         ;; because the package expects to build its bundled sqlite
+         (add-before 'configure 'npm-config-sqlite
+           (lambda* (#:key inputs #:allow-other-keys)
+             (setenv "npm_config_sqlite" (assoc-ref inputs "sqlite"))))
+         (add-after 'patch-dependencies 'avoid-node-pre-gyp
+           (lambda args
+             (with-atomic-file-replacement "package.json"
+               (lambda (in out)
+                 (let* ((js (read-json in))
+                        (alist (match js
+                                 (('@ . alist) alist)))
+                        (scripts-alist (match (assoc-ref alist "scripts")
+                                         (('@ . alist) alist)))
+                        (scripts-alist
+                         ;; install script would use node-pre-gyp
+                         (assoc-remove! scripts-alist "install"))
+                        (alist
+                         (assoc-set! alist "scripts" (cons '@ scripts-alist)))
+                        (alist
+                         ;; causes `npm install` (our 'configure phase)
+                         ;; run the appropriate parts of node-gyp
+                         (assoc-set! alist "gypfile" #t))
+                        (binary-alist (match (assoc-ref alist "binary")
+                                        (('@ . alist) alist)))
+                        (js (cons '@ alist)))
+                   ;; compensate for lack of @mapbox/node-pre-gyp
+                   (setenv "GYP_DEFINES"
+                           (string-append
+                            "module_name="
+                            (assoc-ref binary-alist "module_name")
+                            " "
+                            "module_path="
+                            (assoc-ref binary-alist "module_path")))
+                   (write-json js
+                               out)))))))))
+    (home-page "https://github.com/mapbox/node-sqlite3")
+    (synopsis "Asynchronous, non-blocking SQLite3 bindings for Node.js")
+    (description
+     "The Node.js add-on @code{node-sqlite3} provides a set of a asynchronous,
+non-blocking bindings for SQLite3, written in modern C++ and tested for memory
+leaks.")
+     (license license:bsd-3)))
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:34
[PATCH v2 23/26] gnu: Add node-nan.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-23-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-nan): New variable.
---
gnu/packages/node-xyz.scm | 44 +++++++++++++++++++++++++++++++++++++++
1 file changed, 44 insertions(+)

Toggle diff (54 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 3e06413908..ed169c0778 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -553,3 +553,47 @@ (define-public node-irc
     (description "@code{node-irc} is an IRC client library for Node.js.
 It has functions for joining, parting, talking, and many other IRC commands.")
     (license license:gpl3+)))
+
+(define-public node-nan
+  (package
+    (name "node-nan")
+    (version "2.15.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/nan")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "18xslh9va5ld872scrp5y4251ax9s3c6qh0lnl1200lpzbsxy7yd"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       ;; tests need tap and other dependencies
+       #f
+       #:absent-dependencies
+       '("bindings"
+         "commander"
+         "glob"
+         "request"
+         "node-gyp" ;; would be needed for tests
+         "tap"
+         "xtend")))
+    (inputs
+     `(("readable-stream" ,node-readable-stream)))
+    (home-page "https://github.com/nodejs/nan")
+    (synopsis "Native Abstractions for Node.js")
+    (description "A header file filled with macro and utility goodness for
+making add-on development for Node.js easier across versions 0.8, 0.10, 0.12,
+1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 and 16.
+
+Thanks to the crazy changes in V8 (and some in Node core), keeping native
+addons compiling happily across versions, particularly 0.10 to 0.12 to 4.0, is
+a minor nightmare.  The goal of this project is to store all logic necessary
+to develop native Node.js addons without having to inspect
+@code{NODE_MODULE_VERSION} and get yourself into a macro-tangle.
+
+This project also contains some helper utilities that make addon development a
+bit more pleasant.")
+    (license license:expat)))
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 05:33
[PATCH v2 17/26] gnu: node-irc: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211120043406.952350-17-philip@philipmcgrath.com
gnu/packages/node-xyz.scm (node-irc)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node-xyz.scm | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)

Toggle diff (25 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 9646c14243..998d0a9a90 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -394,12 +394,12 @@ (define-public node-irc
                 "1ln4qfx20jbwg4cp8lp0vf27m5281z2sz16d15xd6150n26cbi4x"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f                      ; FIXME: tests depend on node-faucet
-       #:phases
-       (modify-phases %standard-phases
-         ;; The default configure phase fails due to various packages
-         ;; being missing, as we don't have them packaged yet.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       `("ansi-color"
+         "faucet"
+         "jscs"
+         "tape")
+       #:tests? #f))
     (inputs
      `(("node-irc-colors" ,node-irc-colors)))
     (home-page "https://github.com/martynsmith/node-irc")
-- 
2.32.0
P
P
Philip McGrath wrote on 20 Nov 2021 06:10
Re: [PATCH 03/11] guix: node-build-system: Support compiling add-ons with node-gyp.
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
cd45b636-33cd-880b-02a4-fbee854e8659@philipmcgrath.com
On 11/19/21 23:26, Philip McGrath wrote> In v2 of this series (which
will follow this email)

I'm not sure what went wrong, but I, at least, received these patches
out of order. Let me know if I should resend them as v3. They are
definitely in the right order here:

-Philip
L
L
Liliana Marie Prikler wrote on 20 Nov 2021 08:28
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
9da7a60ca57b06facf82142260fe44ba54096708.camel@gmail.com
Am Samstag, den 20.11.2021, 00:10 -0500 schrieb Philip McGrath:
Toggle quote (10 lines)
> On 11/19/21 23:26, Philip McGrath wrote> In v2 of this series (which
> will follow this email)
>
> I'm not sure what went wrong, but I, at least, received these
> patches
> out of order. Let me know if I should resend them as v3. They are
> definitely in the right order here:
> https://gitlab.com/philip1/guix-patches/-/tree/guix-issue-51838-v2
>
> -Philip
Our mumi frontend to debbugs should be able to deliver them to you as
one convenient mbox:

Cheers
L
L
Liliana Marie Prikler wrote on 20 Nov 2021 08:41
Re: [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument.
8c07272b588fc059f3c445fe1f33c63cce8f31f1.camel@gmail.com
Hi,

Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
Toggle quote (17 lines)
> Many of Guix's Node.js packages are built without some of the
> dependencies they specify in their "package-lock.json" files,
> either because we don't have them packaged yet (e.g. test
> utilities) or because we don't want them (e.g. to reduce the
> closure size). Previously, Guix package definitions would work
> around this situation by deleting the `'configure`
> phase (i.e. the initial `npm install`).
>
> This commit adds an optional #:absent-dependencies argument to
> `node-build-system` to list Node.js packages that should be
> removed from the "package.json" file.Retaining the `'configure`
> phase avoids skipping checks for the dependencies that are
> intended to be present and other actions performed by `npm
> install`, such as automatically building native add-ons with
> `node-gyp` when the "gypfile" key is present.
>
> [...]
This is a somewhat decent approach, but I wonder whether we could
improve this. Does nodejs complain if a dependency exists in the code,
but is not present in the packages.json?

In the resolve-dependencies subprocedure, we could check whether we
have a matching input somewhere and only include the dependency if we
do. WDYT?
L
L
Liliana Marie Prikler wrote on 20 Nov 2021 08:43
Re: [bug#51838] [PATCH v2 06/26] gnu: node-semver-bootstrap: Use #:absent-dependencies.
5be9db7833c1a64d84ac29205ea39c93fff18d24.camel@gmail.com
Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
Toggle quote (22 lines)
> gnu/packages/node.scm (node-semver-bootstrap)[arguments]: Add
> `#:absent-dependencies`. Stop deleting the `'configure` phase.
> ---
> gnu/packages/node.scm | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
> index 0f67fe79c2..6c958eebec 100644
> --- a/gnu/packages/node.scm
> +++ b/gnu/packages/node.scm
> @@ -330,9 +330,8 @@ (define-public node-semver-bootstrap
> (arguments
> `(#:node ,node-bootstrap
> #:tests? #f
> - #:phases
> - (modify-phases %standard-phases
> - (delete 'configure))))
> + #:absent-dependencies
> + '("tap")))
> (home-page "https://github.com/npm/node-semver")
> (properties '((hidden? . #t)))
> (synopsis "Parses semantic versions strings")
Not deleting configure SGTM, but recall my note about #:absent-
dependencies.
L
L
Liliana Marie Prikler wrote on 20 Nov 2021 08:44
Re: [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: Use #:absent-dependencies.
f4f2a0c7a6c6c8c650a2aec6698ce28765b321eb.camel@gmail.com
Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
Toggle quote (33 lines)
> gnu/packages/node.scm (node-llparse-builder-bootstrap)[arguments]:
> Add `#:absent-dependencies`. Stop deleting the `'configure` phase.
> Add a new phase `#:delete-package-lock` to remove the
> problematic "package-lock.json".
> ---
> gnu/packages/node.scm | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
> index 98a51276e7..9d4903a8ca 100644
> --- a/gnu/packages/node.scm
> +++ b/gnu/packages/node.scm
> @@ -479,9 +479,21 @@ (define-public node-llparse-builder-bootstrap
> (arguments
> `(#:node ,node-bootstrap
> #:tests? #f
> + #:absent-dependencies
> + `("@types/mocha"
> + "@types/node"
> + "mocha"
> + "ts-node"
> + "tslint"
> + "typescript")
> #:phases
> (modify-phases %standard-phases
> - (delete 'configure)
> + (add-before 'configure 'remove-package-lock
> + ;; Having package-lock.json seems to cause npm
> + ;; to look for things on the internet in the configure
> phase,
> + ;; even if we have them properly installed.
> + (lambda args
> + (delete-file-recursively "package-lock.json")))
I think a simple delete-file ought to work. This should also be done
by configure or similar, compare cargo-build-system.
L
L
Liliana Marie Prikler wrote on 20 Nov 2021 08:46
Re: [bug#51838] [PATCH v2 18/26] guix: node-build-system: Add optional #:libuv? argument.
15a8348b0c5b080d590f68cc14bca99afd5d7834.camel@gmail.com
Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
Toggle quote (4 lines)
> * guix/build-system/node.scm (lower): Add an optional #:libuv?
> argument to tell the build system to add the version of libuv
> used as an input to the #:node package as an additional implicit
> input, so that packages needing libuv always get the correct version.
Would it change something if we always had libuv as implicit input? I
don't think it'd do much for closure size, but it might impact implicit
package search as proposed by my solution to the #:absent-dependencies
thing.
L
L
Liliana Marie Prikler wrote on 20 Nov 2021 08:48
Re: [bug#51838] [PATCH v2 26/26] gnu: Add node-sqlite3.
76798c0422cb0e7eb0bfd11baa6af6a31688bf2f.camel@gmail.com
Warning, this patch does not just add node-sqlite3!

Am Freitag, den 19.11.2021, 23:34 -0500 schrieb Philip McGrath:
Toggle quote (148 lines)
> * gnu/packages/node-xyz.scm (node-sqlite3): New variable.
> ---
> gnu/packages/node-xyz.scm | 107
> ++++++++++++++++++++++++++++++++++++--
> 1 file changed, 104 insertions(+), 3 deletions(-)
>
> diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
> index 60dbfc163c..a56281fe18 100644
> --- a/gnu/packages/node-xyz.scm
> +++ b/gnu/packages/node-xyz.scm
> @@ -615,7 +615,8 @@ (define-public node-addon-api
> (sha256
> (base32
> "1bhvfi2m9nxfz418s619914vmidcnrzbjv6l9nid476c3zlpazch"))))
> (inputs
> - `(("python" ,python)))
> + `(("python" ,python)
> + ("node-safe-buffer" ,node-safe-buffer)))
> (build-system node-build-system)
> (arguments
> `(#:absent-dependencies
> @@ -630,8 +631,7 @@ (define-public node-addon-api
> "eslint-plugin-promise"
> "fs-extra"
> "path"
> - "pre-commit"
> - "safe-buffer")
> + "pre-commit")
> #:phases
> (modify-phases %standard-phases
> (add-after 'unpack 'skip-js-tests
> @@ -660,3 +660,104 @@ (define-public node-addon-api
> @code{libuv} (included in a project via @code{#include <uv.h>}) are
> not
> ABI-stable across Node.js major versions.")
> (license license:expat)))
> +
> +(define-public node-sqlite3
> + (package
> + (name "node-sqlite3")
> + (version "5.0.2")
> + (source
> + (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/mapbox/node-sqlite3")
> + (commit (string-append "v" version))))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> "0sbbzzli282nxyfha10zx0k5m8hdp0sf3ipl59khjb7wm449j86h"))
> + (snippet
> + (with-imported-modules '((guix build utils))
> + #~(begin
> + (use-modules (guix build utils))
> + ;; unbundle sqlite
> + '(for-each delete-file-recursively
> + (find-files "deps" gzip-file?)))))))
> + (inputs
> + `(("node-addon-api" ,node-addon-api)
> + ("python" ,python)
> + ("sqlite" ,sqlite)))
> + (build-system node-build-system)
> + (arguments
> + `(#:tests?
> + #f ; FIXME: tests depend on node-mocha
> + #:modules
> + ((guix build node-build-system)
> + (guix build json)
> + (srfi srfi-1)
> + (ice-9 match)
> + (guix build utils))
> + #:libuv? #t
> + #:absent-dependencies
> + `(;; Normally, this is "built" using @mapbox/node-pre-gyp,
> + ;; which publishes or downloads pre-built binaries
> + ;; or falls back to building from source.
> + ;; Here, we patch out all of that and just build directly.
> + ;; It would be better to patch a version of @mapbox/node-
> pre-gyp
> + ;; that always builds from source, as Debian does,
> + ;; but there are a number of dependencies that need
> + ;; to be packaged or removed.
> + "@mapbox/node-pre-gyp"
> + "node-pre-gyp" ;; deprecated name still used in some places
> + "aws-sdk"
> + "@mapbox/cloudfriend"
> + ;; Confusingly, this is only a dependency beceuse of
> + ;; @mapbox/node-pre-gyp: with that removed,
> + ;; npm will use its own copy:
> + "node-gyp"
> + ;; These we'd like, we just don't have them yet:
> + "eslint"
> + "mocha")
> + #:phases
> + (modify-phases %standard-phases
> + ;; We need this step even if we do replace @mapbox/node-
> pre-gyp
> + ;; because the package expects to build its bundled sqlite
> + (add-before 'configure 'npm-config-sqlite
> + (lambda* (#:key inputs #:allow-other-keys)
> + (setenv "npm_config_sqlite" (assoc-ref inputs
> "sqlite"))))
> + (add-after 'patch-dependencies 'avoid-node-pre-gyp
> + (lambda args
> + (with-atomic-file-replacement "package.json"
> + (lambda (in out)
> + (let* ((js (read-json in))
> + (alist (match js
> + (('@ . alist) alist)))
> + (scripts-alist (match (assoc-ref alist
> "scripts")
> + (('@ . alist) alist)))
> + (scripts-alist
> + ;; install script would use node-pre-gyp
> + (assoc-remove! scripts-alist "install"))
> + (alist
> + (assoc-set! alist "scripts" (cons '@
> scripts-alist)))
> + (alist
> + ;; causes `npm install` (our 'configure
> phase)
> + ;; run the appropriate parts of node-gyp
> + (assoc-set! alist "gypfile" #t))
> + (binary-alist (match (assoc-ref alist
> "binary")
> + (('@ . alist) alist)))
> + (js (cons '@ alist)))
> + ;; compensate for lack of @mapbox/node-pre-gyp
> + (setenv "GYP_DEFINES"
> + (string-append
> + "module_name="
> + (assoc-ref binary-alist "module_name")
> + " "
> + "module_path="
> + (assoc-ref binary-alist "module_path")))
> + (write-json js
> + out)))))))))
> + (home-page "https://github.com/mapbox/node-sqlite3")
> + (synopsis "Asynchronous, non-blocking SQLite3 bindings for
> Node.js")
> + (description
> + "The Node.js add-on @code{node-sqlite3} provides a set of a
> asynchronous,
> +non-blocking bindings for SQLite3, written in modern C++ and tested
> for memory
> +leaks.")
> + (license license:bsd-3)))
P
P
Philip McGrath wrote on 20 Nov 2021 18:04
Re: [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument.
37768a63-5a4a-b604-2542-e8890fd68a16@philipmcgrath.com
Hi,

On 11/20/21 02:41, Liliana Marie Prikler wrote:
Toggle quote (24 lines)
> Hi,
>
> Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
>> Many of Guix's Node.js packages are built without some of the
>> dependencies they specify in their "package-lock.json" files,
>> either because we don't have them packaged yet (e.g. test
>> utilities) or because we don't want them (e.g. to reduce the
>> closure size). Previously, Guix package definitions would work
>> around this situation by deleting the `'configure`
>> phase (i.e. the initial `npm install`).
>>
>> This commit adds an optional #:absent-dependencies argument to
>> `node-build-system` to list Node.js packages that should be
>> removed from the "package.json" file.Retaining the `'configure`
>> phase avoids skipping checks for the dependencies that are
>> intended to be present and other actions performed by `npm
>> install`, such as automatically building native add-ons with
>> `node-gyp` when the "gypfile" key is present.
>>
>> [...]
> This is a somewhat decent approach, but I wonder whether we could
> improve this. Does nodejs complain if a dependency exists in the code,
> but is not present in the packages.json?

To the best of my understanding, npm doesn't inspect the js files at all
by default, so no. (A package.json file might define a build script that
does more checks.) I think a missing and undeclared dependency manifests
itself as a runtime error when the require() function is evaluated.

Toggle quote (4 lines)
> In the resolve-dependencies subprocedure, we could check whether we
> have a matching input somewhere and only include the dependency if we
> do. WDYT?

I thought about this, but it seems to me there are two problems. First,
I'm very unsure about this, but, in the last few days, I've gotten the
impression there may be some "package.json" packages that don't
correspond directly to Guix packages. I don't really understand how that
works, but I think it may have something to do with npm packages that
can run either on Node.js or on the web and need to use some
functionality that's part of the Node.js core but not the web platform
(or vice versa?). I could be wrong about that, but I've tried,
especially in v2, to only have the build-side code do things I'm
confident are correct in all cases.

The other issue is that deleting packages with no matching input by
default would replicate some of the drawbacks of the current `(delete
'configure)` approach. I think it's better to have an explicit list of
dependencies that Guix is deleting. If eventually we package all of the
missing dependencies for Guix, it would be much easier to find the
packages that ought to use it. And, in the highly dynamic JavaScript
world, I'm reluctant to give up one of the few static checks we have. If
a missing package that really was required were automatically deleted
from "package.json", it seems the failure mode would by a mysterious
runtime error, potentially many steps down a dependency chain.
P
P
Philip McGrath wrote on 20 Nov 2021 18:09
Re: [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: Use #:absent-dependencies.
f5e6dc2c-6f71-ee6c-bad3-7aa744a4c6f9@philipmcgrath.com
On 11/20/21 02:44, Liliana Marie Prikler wrote:
Toggle quote (13 lines)
> Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
>> (modify-phases %standard-phases
>> - (delete 'configure)
>> + (add-before 'configure 'remove-package-lock
>> + ;; Having package-lock.json seems to cause npm
>> + ;; to look for things on the internet in the configure
>> phase,
>> + ;; even if we have them properly installed.
>> + (lambda args
>> + (delete-file-recursively "package-lock.json")))
> I think a simple delete-file ought to work. This should also be done
> by configure or similar, compare cargo-build-system.

Yes, I thought about having the build system automatically delete
"package-lock.json", but I'm not 100% sure it's always a problem to have
it, or if there might even be some circumstance where we want to keep
it. I'd prefer to wait until we see a significant number of node
packages deleting their "package-lock.json" files before trying to
abstract over the pattern.
P
P
Philip McGrath wrote on 20 Nov 2021 18:16
Re: [bug#51838] [PATCH v2 18/26] guix: node-build-system: Add optional #:libuv? argument.
ce2b636c-741e-99aa-2eaa-dfe9a13160d9@philipmcgrath.com
On 11/20/21 02:46, Liliana Marie Prikler wrote:
Toggle quote (10 lines)
> Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
>> * guix/build-system/node.scm (lower): Add an optional #:libuv?
>> argument to tell the build system to add the version of libuv
>> used as an input to the #:node package as an additional implicit
>> input, so that packages needing libuv always get the correct version.
> Would it change something if we always had libuv as implicit input? I
> don't think it'd do much for closure size, but it might impact implicit
> package search as proposed by my solution to the #:absent-dependencies
> thing.

From a Guix perspective, I don't think it would make much difference,
since libuv is always needed by node itself. Maybe it would require more
grafting than would be needed otherwise?

But I think the recommendation for authors of node add-ons is to avoid
libuv unless it's really needed, because it has fewer stability
guarantees that other Node.js APIs.
T
T
Timothy Sample wrote on 20 Nov 2021 21:08
Re: bug#51838: [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.
(name . Philip McGrath)(address . philip@philipmcgrath.com)
87tug6bnen.fsf@ngyro.com
Hi Philip,

There are some really valuable changes in here, thanks! I would like to
start understanding and reviewing the changes so that we can get some of
this good stuff merged in. However, I have one question that I couldn’t
answer from reading here or at https://issues.guix.gnu.org/49946: is
there an agreement between you and Pierre that these patches are the
“right” way to do native addons for Node.js. More importantly, is the
general plan that we merge these changes, and then Pierre rebases their
Tree-sitter changes on top these?

Pierre, maybe you could weigh in here?

Sorry if I missed something. I assume everything is OK, but I want to
be sure before I start digging into the details of the patches –
especially those first few more complicated ones. :)

Thanks!


-- Tim
T
T
Timothy Sample wrote on 20 Nov 2021 21:04
(name . Philip McGrath)(address . philip@philipmcgrath.com)
87o86ebnee.fsf@ngyro.com
Hello,

Philip McGrath <philip@philipmcgrath.com> writes:

Toggle quote (20 lines)
> On 11/20/21 02:41, Liliana Marie Prikler wrote:
>
>> In the resolve-dependencies subprocedure, we could check whether we
>> have a matching input somewhere and only include the dependency if we
>> do. WDYT?
>
> [...]
>
> The other issue is that deleting packages with no matching input by
> default would replicate some of the drawbacks of the current `(delete
> 'configure)` approach. I think it's better to have an explicit list of
> dependencies that Guix is deleting. If eventually we package all of
> the missing dependencies for Guix, it would be much easier to find the
> packages that ought to use it. And, in the highly dynamic JavaScript
> world, I'm reluctant to give up one of the few static checks we
> have. If a missing package that really was required were automatically
> deleted from "package.json", it seems the failure mode would by a
> mysterious runtime error, potentially many steps down a dependency
> chain.

This is well put. I actually experimented with a similar approach when
we updated the Node.js build system. This is a big improvement over
deleting the configure phase, which would never scale to more than a
handful of packages. Having a build-time check that ensures all the
developer-declared dependencies are available (save the “absent” ones)
will be very helpful when we are maintaining hundreds of JavaScript
packages. :)

For the patch itself, it would be better to move a lot of your commit
message into a comment somewhere in the build system code. If we had a
section in the manual for Node packages, it would go there, but alas....
I think most people would be happy to see a comment in the build system
code and be saved from having to poke around with ‘git blame’.


-- Tim
T
T
Timothy Sample wrote on 20 Nov 2021 20:55
(name . Philip McGrath)(address . philip@philipmcgrath.com)
87ilwmbne7.fsf@ngyro.com
Hello,

Philip McGrath <philip@philipmcgrath.com> writes:

Toggle quote (19 lines)
> On 11/20/21 02:46, Liliana Marie Prikler wrote:
>> Am Freitag, den 19.11.2021, 23:33 -0500 schrieb Philip McGrath:
>>> * guix/build-system/node.scm (lower): Add an optional #:libuv?
>>> argument to tell the build system to add the version of libuv
>>> used as an input to the #:node package as an additional implicit
>>> input, so that packages needing libuv always get the correct version.
>> Would it change something if we always had libuv as implicit input? I
>> don't think it'd do much for closure size, but it might impact implicit
>> package search as proposed by my solution to the #:absent-dependencies
>> thing.
>
> From a Guix perspective, I don't think it would make much difference,
> since libuv is always needed by node itself. Maybe it would require
> more grafting than would be needed otherwise?
>
> But I think the recommendation for authors of node add-ons is to avoid
> libuv unless it's really needed, because it has fewer stability
> guarantees that other Node.js APIs.

My assumption is that most packages would ignore the libuv headers, and
they wouldn’t retain a reference to it (except via Node.js itself).
Hence, I don’t think it would make any difference either to grafting or
closure size to just always add it.


-- Tim
T
T
Timothy Sample wrote on 20 Nov 2021 18:38
Re: [bug#51838] [PATCH v2 04/26] gnu: node: Add an npmrc file to set nodedir.
(name . Philip McGrath)(address . philip@philipmcgrath.com)
87czmubne1.fsf@ngyro.com
Hi!

Just a little typo:

Philip McGrath <philip@philipmcgrath.com> writes:

Toggle quote (2 lines)
> * gnu/packages/node.scm (node, node-lis)[arguments]: Add a phase

node-lis should be node-lts.


-- Tim
L
L
Liliana Marie Prikler wrote on 20 Nov 2021 21:24
Re: [bug#51838] [PATCH v2 05/26] guix: node-build-system: Add #:absent-dependencies argument.
62e2afb290ae88cc8bb42ccc26683f9d9435bbf5.camel@gmail.com
Hi,

Am Samstag, den 20.11.2021, 12:04 -0500 schrieb Philip McGrath:
Toggle quote (5 lines)
> To the best of my understanding, npm doesn't inspect the js files at
> all by default, so no. (A package.json file might define a build
> script that does more checks.) I think a missing and undeclared
> dependency manifests itself as a runtime error when the require()
> function is evaluated.
Sheesh.

Toggle quote (14 lines)
> > In the resolve-dependencies subprocedure, we could check whether we
> > have a matching input somewhere and only include the dependency if
> > we do. WDYT?
>
> I thought about this, but it seems to me there are two problems.
> First, I'm very unsure about this, but, in the last few days, I've
> gotten the impression there may be some "package.json" packages that
> don't correspond directly to Guix packages. I don't really understand
> how that works, but I think it may have something to do with npm
> packages that can run either on Node.js or on the web and need to use
> some functionality that's part of the Node.js core but not the web
> platform (or vice versa?). I could be wrong about that, but I've
> tried, especially in v2, to only have the build-side code do things
> I'm confident are correct in all cases.
What does "not directly correspond to Guix packages" mean here? At the
very least Node must have a way of finding them, probably somewhere in
lib/, no?

While I did say we may want to rewrite these dependencies in
‘configure’, perhaps we can find a middle ground in which we only do so
when a given argument is there. ant-build-system does that for
instance, so as to cope with packages that actually have no ant build
spec. WDYT?

Toggle quote (11 lines)
> The other issue is that deleting packages with no matching input by
> default would replicate some of the drawbacks of the current
> `(delete 'configure)` approach. I think it's better to have an
> explicit list of dependencies that Guix is deleting. If eventually we
> package all of the missing dependencies for Guix, it would be much
> easier to find the packages that ought to use it. And, in the highly
> dynamic JavaScript world, I'm reluctant to give up one of the few
> static checks we have. If a missing package that really was required
> were automatically deleted from "package.json", it seems the failure
> mode would by a mysterious runtime error, potentially many steps down
> a dependency chain.
I get where you're coming from, but OTOH if you were to change parts of
the package to suit your own needs – like adding or omitting some
inputs, you now have two places to change, and they're in a tricky
relationship with another. I think this is setting up some very error-
prone boilerplate.

I hope that in most cases the test suite will cover the important use
cases, but that's not always possible (e.g. with flaky tests,
networking, missing test inputs...) Furthermore, for better or for
worse, stack traces did become the gold standard of debugging long ago,
so even packages missing "deep down" would probably soon be dug out if
they lead to hard errors. Soft errors in Javascript though...
J
J
Jelle Licht wrote on 23 Nov 2021 12:04
Re: [bug#51838] [PATCH v2 10/26] gnu: node-llparse-builder-bootstrap: Use #:absent-dependencies.
(name . Liliana Marie Prikler)(address . liliana.prikler@gmail.com)
86tug39lps.fsf@fsfe.org
Hey Philip,

Philip McGrath <philip@philipmcgrath.com> writes:

Toggle quote (31 lines)
> gnu/packages/node.scm (node-llparse-builder-bootstrap)[arguments]: Add
> `#:absent-dependencies`. Stop deleting the `'configure` phase.
> Add a new phase `#:delete-package-lock` to remove the
> problematic "package-lock.json".
> ---
> gnu/packages/node.scm | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
> index 98a51276e7..9d4903a8ca 100644
> --- a/gnu/packages/node.scm
> +++ b/gnu/packages/node.scm
> @@ -479,9 +479,21 @@ (define-public node-llparse-builder-bootstrap
> (arguments
> `(#:node ,node-bootstrap
> #:tests? #f
> + #:absent-dependencies
> + `("@types/mocha"
> + "@types/node"
> + "mocha"
> + "ts-node"
> + "tslint"
> + "typescript")
> #:phases
> (modify-phases %standard-phases
> - (delete 'configure)
> + (add-before 'configure 'remove-package-lock
> + ;; Having package-lock.json seems to cause npm
> + ;; to look for things on the internet in the configure phase,
> + ;; even if we have them properly installed.

package-lock.json lists exact versions _and integrity hashes_; since it
seems unlikely that after node-build-system's finaggling we end up with
an identical hash, we will always have a mismatch and fetch 'proper'
sources online accordingly. As far as npm + package-lock.json are
concerned, we don't have them properly installed.

From what I have seen package-lock.json offers us no benefits (because
we track exact dependency information via the guix store) and can (as
you have seen) prevent builds from working. My 2c: always remove it in a
phase in the build system. It's simply a preference though, so your
please go with what you think is the right choice. You might want to
update the comment nonetheless so it's clear we know what's going on.

Toggle quote (7 lines)
> + (lambda args
> + (delete-file-recursively "package-lock.json")))
> (replace 'build
> (lambda* (#:key inputs #:allow-other-keys)
> (let ((esbuild (string-append (assoc-ref inputs "esbuild")
> --
> 2.32.0
P
P
Pierre Langlois wrote on 23 Nov 2021 21:54
Re: bug#51838: [PATCH 00/11] guix: node-build-system: Support compiling add-ons with node-gyp.
(name . Timothy Sample)(address . samplet@ngyro.com)
87czmqk29d.fsf@gmx.com
Hi Timothy,

Timothy Sample <samplet@ngyro.com> writes:

Toggle quote (13 lines)
> Hi Philip,
>
> There are some really valuable changes in here, thanks! I would like to
> start understanding and reviewing the changes so that we can get some of
> this good stuff merged in. However, I have one question that I couldn’t
> answer from reading here or at <https://issues.guix.gnu.org/49946>: is
> there an agreement between you and Pierre that these patches are the
> “right” way to do native addons for Node.js. More importantly, is the
> general plan that we merge these changes, and then Pierre rebases their
> Tree-sitter changes on top these?
>
> Pierre, maybe you could weigh in here?

The overall approach looks good to me, it's better than what I
originally proposed for sure :-). That being said, I'm not very
familiar with the Node.js ecosystem so I don't know if it's necessarily
the right way, but I suspect the correct way for node isn't very Guix-y
so I'm not too worried about that.

It's on my TODO list to take another look at the patches as well :-),
then yes, I'm planning on rebasing my tree-sitter series on top.

Toggle quote (4 lines)
> Sorry if I missed something. I assume everything is OK, but I want to
> be sure before I start digging into the details of the patches –
> especially those first few more complicated ones. :)

Thanks for taking a look!

Pierre
-----BEGIN PGP SIGNATURE-----

iQFMBAEBCgA2FiEEctU9gYy29KFyWDdMqPyeRH9PfVQFAmGdWI4YHHBpZXJyZS5s
YW5nbG9pc0BnbXguY29tAAoJEKj8nkR/T31UeigH/27lKcqCARHJuK4pZ6W1nT9M
Jn3dN6+VYNp2MZHlQ2OVQopytBCxMKuQoBnmakt9ifanZknzHWo1E5qOMuAJH16S
mV8/rmZphJTafMuq2ZVhE5iibt4ICgnDwtGi5s0jsl0Ku75I6AFImcC88NpXmkWC
PlesI3gG6IpSV5uMJ/Fk4ufCSzXA8V+zpTHmQo0fayplEVfLGYySdluLP3H3+LIV
4s+3L2zaIhl1jUNO1djieBOCXYHqXuBMEiDyNLqVGQqCRbW+CYb8odB0QfRPG544
pJSg04xhclJkhkfsbOX1+7UzsmWHB0kgHPJELEZnwzN0aCgh/YcXL8awHFUa/Jc=
=9JOK
-----END PGP SIGNATURE-----

T
T
Timothy Sample wrote on 28 Nov 2021 20:27
(name . Philip McGrath)(address . philip@philipmcgrath.com)
87tufwm66b.fsf_-_@ngyro.com
Hi Philip,

Philip McGrath <philip@philipmcgrath.com> writes:

Toggle quote (21 lines)
> - (define (resolve-dependencies package-meta meta-key)
> - (fold (lambda (key+value acc)
> - (match key+value
> - ('@ acc)
> - ((key . value) (acons key (hash-ref index key value) acc))))
> + (define (resolve-dependencies meta-alist meta-key)
> + (match (assoc-ref meta-alist meta-key)
> + (#f
> + '())
> + (('@ . orig-deps)
> + (fold (match-lambda*
> + (('@ acc)
> + acc)
> + (((key . value) acc)
> + (if (member key absent-dependencies)
> + acc
> + (acons key (hash-ref index key value) acc))))
> '()
> - (or (assoc-ref package-meta meta-key) '())))
> + orig-deps))))

There’s a lot of refactoring going here in addition to change at hand.
To me, it needlessly obscures the actual change (which is just the check
for membership in 'absent-dependencies', AFAICS).

Toggle quote (16 lines)
> (with-atomic-file-replacement "package.json"
> (lambda (in out)
> - (let ((package-meta (read-json in)))
> - (assoc-set! package-meta "dependencies"
> - (append
> - '(@)
> - (resolve-dependencies package-meta "dependencies")
> - (resolve-dependencies package-meta "peerDependencies")))
> - (assoc-set! package-meta "devDependencies"
> - (append
> - '(@)
> - (resolve-dependencies package-meta "devDependencies")))
> + ;; It is unsafe to rely on 'assoc-set!' to update an
> + ;; existing assosciation list variable:
> + ;; see 'info "(guile)Adding or Setting Alist Entries"'.

Good catch!

Toggle quote (4 lines)
> + (let* ((package-meta (read-json in))
> + (alist (match package-meta
> + ((@ . alist) alist)))

Why do we have to unwrap (and then re-wrap later on) the alist? It
would be nice to explain that in the changelog. Maybe it’s just me, but
I can’t figure it out! :)

Thanks!


-- Tim
T
T
Timothy Sample wrote on 28 Nov 2021 20:35
(name . Jelle Licht)(address . jlicht@fsfe.org)
87pmqkm5u1.fsf_-_@ngyro.com
Hello!

Jelle Licht <jlicht@fsfe.org> writes:

Toggle quote (16 lines)
> Philip McGrath <philip@philipmcgrath.com> writes:
>
>> Add a new phase `#:delete-package-lock` to remove the
>> problematic "package-lock.json".
>
> package-lock.json lists exact versions _and integrity hashes_; since it
> seems unlikely that after node-build-system's finaggling we end up with
> an identical hash, we will always have a mismatch and fetch 'proper'
> sources online accordingly. As far as npm + package-lock.json are
> concerned, we don't have them properly installed.
>
> From what I have seen package-lock.json offers us no benefits (because
> we track exact dependency information via the guix store) and can (as
> you have seen) prevent builds from working. My 2c: always remove it in a
> phase in the build system.

I’m inclined to agree with Jelle and Liliana. I can’t imagine a
situation in which we would want the lock files. We could be wrong, but
we can always adjust the build system later if something surprising
happens (e.g., ‘#:keep-lock-file?’ or whatever).


-- Tim
T
T
Timothy Sample wrote on 28 Nov 2021 20:59
(name . Pierre Langlois)(address . pierre.langlois@gmx.com)
87ilwcm4pm.fsf_-_@ngyro.com
Hello,

Pierre Langlois <pierre.langlois@gmx.com> writes:

Toggle quote (15 lines)
> Hi Timothy,
>
> Timothy Sample <samplet@ngyro.com> writes:
>
>> More importantly, is the general plan that we merge these changes,
>> and then Pierre rebases their Tree-sitter changes on top these?
>>
>> Pierre, maybe you could weigh in here?
>
> The overall approach looks good to me, it's better than what I
> originally proposed for sure :-). That being said, I'm not very
> familiar with the Node.js ecosystem so I don't know if it's necessarily
> the right way, but I suspect the correct way for node isn't very Guix-y
> so I'm not too worried about that.

The whole Node.js bundles NPM, which bundles node-gyp, which bundles a
fork of GYP [1] is not very Guix-y at all, no. :/ This is one of those
problems (like bootstrapping GCC) that will take years of incremental
improvements and side projects and all that.

[1] Not to get too off topic, but isn’t “gyp” a slur? How did Google
ever call something that?

Toggle quote (3 lines)
> It's on my TODO list to take another look at the patches as well :-),
> then yes, I'm planning on rebasing my tree-sitter series on top.

Excellent!

Toggle quote (6 lines)
>> Sorry if I missed something. I assume everything is OK, but I want to
>> be sure before I start digging into the details of the patches –
>> especially those first few more complicated ones. :)
>
> Thanks for taking a look!

I have an idea to simplify the patch series a bit: if we can answer my
question here https://issues.guix.gnu.org/51838#57 and come to a
conclusion about deleting lock files
‘#:absent-dependencies’ part of the patch series. I think this might
make future re-rolls easier and help rein in the scope a bit.

Thoughts? Philip?

Thanks!


-- Tim
P
P
Philip McGrath wrote on 2 Dec 2021 22:18
ade747ac-725f-81c4-d65e-c7c6fdfd2adb@philipmcgrath.com
On 11/28/21 14:35, Timothy Sample wrote:
Toggle quote (11 lines)
> Jelle Licht <jlicht@fsfe.org> writes:
>> From what I have seen package-lock.json offers us no benefits (because
>> we track exact dependency information via the guix store) and can (as
>> you have seen) prevent builds from working. My 2c: always remove it in a
>> phase in the build system.
>
> I’m inclined to agree with Jelle and Liliana. I can’t imagine a
> situation in which we would want the lock files. We could be wrong, but
> we can always adjust the build system later if something surprising
> happens (e.g., ‘#:keep-lock-file?’ or whatever).

This makes sense to me. Should we also delete "yarn.lock" (respected as
of npm v7)[1] and "npm-shrinkwrap.json"[2]? It seems like the same
reasons probably apply. We might want to handle
"node_modules/.package-lock.json"[3], too, but that seems less likely to
exist.

-Philip

[1]:
[2]:
[3]:
P
P
Philip McGrath wrote on 2 Dec 2021 22:50
(name . Timothy Sample)(address . samplet@ngyro.com)
b9a4fd02-ab9b-3445-b1d7-7236d0052d85@philipmcgrath.com
Hi!

On 11/28/21 14:27, Timothy Sample wrote:
Toggle quote (27 lines)
> Philip McGrath <philip@philipmcgrath.com> writes:
>
>> - (define (resolve-dependencies package-meta meta-key)
>> - (fold (lambda (key+value acc)
>> - (match key+value
>> - ('@ acc)
>> - ((key . value) (acons key (hash-ref index key value) acc))))
>> + (define (resolve-dependencies meta-alist meta-key)
>> + (match (assoc-ref meta-alist meta-key)
>> + (#f
>> + '())
>> + (('@ . orig-deps)
>> + (fold (match-lambda*
>> + (('@ acc)
>> + acc)
>> + (((key . value) acc)
>> + (if (member key absent-dependencies)
>> + acc
>> + (acons key (hash-ref index key value) acc))))
>> '()
>> - (or (assoc-ref package-meta meta-key) '())))
>> + orig-deps))))
>
> There’s a lot of refactoring going here in addition to change at hand.
> To me, it needlessly obscures the actual change (which is just the check
> for membership in 'absent-dependencies', AFAICS).

I could split it into two commits, if that would be more clear. But see
below.

Toggle quote (26 lines)
>> (with-atomic-file-replacement "package.json"
>> (lambda (in out)
>> - (let ((package-meta (read-json in)))
>> - (assoc-set! package-meta "dependencies"
>> - (append
>> - '(@)
>> - (resolve-dependencies package-meta "dependencies")
>> - (resolve-dependencies package-meta "peerDependencies")))
>> - (assoc-set! package-meta "devDependencies"
>> - (append
>> - '(@)
>> - (resolve-dependencies package-meta "devDependencies")))
>> + ;; It is unsafe to rely on 'assoc-set!' to update an
>> + ;; existing assosciation list variable:
>> + ;; see 'info "(guile)Adding or Setting Alist Entries"'.
>
> Good catch!
>
>> + (let* ((package-meta (read-json in))
>> + (alist (match package-meta
>> + ((@ . alist) alist)))
>
> Why do we have to unwrap (and then re-wrap later on) the alist? It
> would be nice to explain that in the changelog. Maybe it’s just me, but
> I can’t figure it out! :)

The (guix build json) module represents a JSON object as a Scheme pair
with the symbol @ in the car and an alist in the cdr. I'd guess this is
because, if JSON objects were represented as Scheme alists, it would be
impossible to distinguish the empty JSON object from the empty JSON
array. (Racket's json library instead represents JSON objects with
immutable hash tables, a disjoint type.)

I found it confusing that:

1. this format does not seem to be documented;

2. `resolve-dependencies` has the same shape as `assoc-ref`, but
returns an unwrapped alist where `assoc-ref` would return
a wrapped pair with @; and

3. prior to this patch, the implementation of `resolve-dependencies`
was written as though the symbol @ could appear at arbitrary
positions in the list, which made it less than obvious that it would
actually occur exactly once, at the head.

I tried to address the last point when refactoring, but I see that the
patch I sent made matters extra confusing by leaving in dead code from
the old approach (maybe due to a rebase conflict I vaguely remember?). I
should fix that, and probably also explain some of this in comments.

-Philip
P
P
Philip McGrath wrote on 2 Dec 2021 22:52
(name . Timothy Sample)(address . samplet@ngyro.com)
36cd2d26-f279-0bd9-b160-f03f0cce0f55@philipmcgrath.com
On 11/20/21 14:55, Timothy Sample wrote:
Toggle quote (4 lines)
> My assumption is that most packages would ignore the libuv headers, and
> they wouldn’t retain a reference to it (except via Node.js itself).
> Hence, I don’t think it would make any difference either to grafting or
> closure size to just always add it.
That makes sense to me.

-Philip
P
P
Philip McGrath wrote on 2 Dec 2021 23:02
(name . Timothy Sample)(address . samplet@ngyro.com)
e3b695ae-1afb-c6fb-76d7-652e8d56505a@philipmcgrath.com
Hi,

On 11/20/21 15:04, Timothy Sample wrote:
Toggle quote (32 lines)
> Hello,
>
> Philip McGrath <philip@philipmcgrath.com> writes:
>
>> On 11/20/21 02:41, Liliana Marie Prikler wrote:
>>
>>> In the resolve-dependencies subprocedure, we could check whether we
>>> have a matching input somewhere and only include the dependency if we
>>> do. WDYT?
>>
>> [...]
>>
>> The other issue is that deleting packages with no matching input by
>> default would replicate some of the drawbacks of the current `(delete
>> 'configure)` approach. I think it's better to have an explicit list of
>> dependencies that Guix is deleting. If eventually we package all of
>> the missing dependencies for Guix, it would be much easier to find the
>> packages that ought to use it. And, in the highly dynamic JavaScript
>> world, I'm reluctant to give up one of the few static checks we
>> have. If a missing package that really was required were automatically
>> deleted from "package.json", it seems the failure mode would by a
>> mysterious runtime error, potentially many steps down a dependency
>> chain.
>
> This is well put. I actually experimented with a similar approach when
> we updated the Node.js build system. This is a big improvement over
> deleting the configure phase, which would never scale to more than a
> handful of packages. Having a build-time check that ensures all the
> developer-declared dependencies are available (save the “absent” ones)
> will be very helpful when we are maintaining hundreds of JavaScript
> packages. :)

It's probably also worth noting that many (most?) of our Node.js
packages currently don't run tests: an unfortunate number don't have any
tests, and then there's the issue that we haven't packaged JavaScript
test frameworks for Guix yet.


Toggle quote (6 lines)
> For the patch itself, it would be better to move a lot of your commit
> message into a comment somewhere in the build system code. If we had a
> section in the manual for Node packages, it would go there, but alas....
> I think most people would be happy to see a comment in the build system
> code and be saved from having to poke around with ‘git blame’.

Yes, I'll add comments.

-Philip
P
P
Philip McGrath wrote on 2 Dec 2021 23:22
4b99d8f5-73af-b1bd-bcc8-be77d87f229b@philipmcgrath.com
Hi,

On 11/28/21 14:59, Timothy Sample wrote:
Toggle quote (12 lines)
> Pierre Langlois <pierre.langlois@gmx.com> writes:
>> The overall approach looks good to me, it's better than what I
>> originally proposed for sure :-). That being said, I'm not very
>> familiar with the Node.js ecosystem so I don't know if it's necessarily
>> the right way, but I suspect the correct way for node isn't very Guix-y
>> so I'm not too worried about that.
>
> The whole Node.js bundles NPM, which bundles node-gyp, which bundles a
> fork of GYP [1] is not very Guix-y at all, no. :/ This is one of those
> problems (like bootstrapping GCC) that will take years of incremental
> improvements and side projects and all that.

It's something of a tangent, but, as I discussed briefly with Pierre in
e.g. https://issues.guix.gnu.org/49946#73, I think it would make a lot
of sense, even if we can't unbundle/bootstrap everything, to make the
355 dependencies of npm we are already distributing available as Guix
packages. For example, I think I found a bootstrapping cycle with the
Unicode data packages: even if the version we're currently using is
messy, letting other packages use it, too, might unlock much more of the
JavaScript universe. I've experimented a bit (partially looking at what
Debian does), and I may look at that some time after this.


Toggle quote (7 lines)
> I have an idea to simplify the patch series a bit: if we can answer my
> question here <https://issues.guix.gnu.org/51838#57> and come to a
> conclusion about deleting lock files
> <https://issues.guix.gnu.org/51838#58>, I could merge the
> ‘#:absent-dependencies’ part of the patch series. I think this might
> make future re-rolls easier and help rein in the scope a bit.

That sounds like a great approach! I had not expected this patch series
to end up touching every JavaScript package in Guix :)

I was offline a bit for holidays here, and just before that I'd noticed
a problem with npm generating implicit build commands that tried to
rebuild node-gyp packages when they are installed. I'll page this stuff
back in and send a v3 soon that hopefully has at least the
'#:absent-dependencies' part ready to merge.

-Philip
L
L
Liliana Marie Prikler wrote on 3 Dec 2021 06:17
(address . 51838@debbugs.gnu.org)
46a042bc08eb72a068b1e8c69bfe28cf2d4b2e53.camel@gmail.com
Hi,

Am Donnerstag, den 02.12.2021, 16:18 -0500 schrieb Philip McGrath:
Toggle quote (15 lines)
> On 11/28/21 14:35, Timothy Sample wrote:
> > Jelle Licht <jlicht@fsfe.org> writes:
> > > From what I have seen package-lock.json offers us no benefits
> > > (because we track exact dependency information via the guix
> > > store) and can (as you have seen) prevent builds from working. My
> > > 2c: always remove it in a phase in the build system.
> >
> > I’m inclined to agree with Jelle and Liliana. I can’t imagine a
> > situation in which we would want the lock files. We could be
> > wrong, but we can always adjust the build system later if something
> > surprising happens (e.g., ‘#:keep-lock-file?’ or whatever).
>
> This makes sense to me. Should we also delete "yarn.lock" (respected
> as of npm v7)[1] and "npm-shrinkwrap.json"[2]? It seems like the
> same reasons probably apply.
If it breaks builds, then yes.

Toggle quote (2 lines)
> We might want to handle "node_modules/.package-lock.json"[3], too,
> but that seems less likely to exist.
What are node_modules in this context? To me that sounds like
"vendored packages" and if so, they ought to (ideally) be deleted in a
snippet way before unpack. Meaning you wouldn't have to handle that
case in configure.

Cheers
P
P
Philip McGrath wrote on 8 Dec 2021 21:27
[PATCH v3 01/43] gnu: node: Avoid duplicating build phases.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-2-philip@philipmcgrath.com
* gnu/packages/node.scm (node)[arguments]: Split 'patch-files phase
into 'patch-hardcoded-program-references and
'delete-problematic-tests. Adapt 'patch-hardcoded-program-references
and 'configure to work unmodified on node-lts, but don't try to
share 'delete-problematic-tests, because those details seem to change
too much between node versions.
(node, node-lts)[inputs]: Use bash-minimal rather than bash.
(node-lts)[arguments]: Inherit 'patch-hardcoded-program-references,
and 'configure phases from the bootstrap node.
Remove the 'patch-files phase, keeping its remaining
non-inherited work in a new 'replace-llhttp-sources phase.
---
gnu/packages/node.scm | 105 +++++++++++++-----------------------------
1 file changed, 32 insertions(+), 73 deletions(-)

Toggle diff (170 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index dccf871d2c..7ebbbc3060 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -9,6 +9,7 @@
 ;;; Copyright © 2020, 2021 Pierre Langlois <pierre.langlois@gmx.com>
 ;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2021 Simon Tournier <zimon.toutoune@gmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -105,14 +106,22 @@ (define-public node
        #:test-target "test-ci-js"
        #:phases
        (modify-phases %standard-phases
-         (add-before 'configure 'patch-files
+         (add-before 'configure 'patch-hardcoded-program-references
            (lambda* (#:key inputs #:allow-other-keys)
+
              ;; Fix hardcoded /bin/sh references.
-             (substitute* '("lib/child_process.js"
-                            "lib/internal/v8_prof_polyfill.js"
-                            "test/parallel/test-child-process-spawnsync-shell.js"
-                            "test/parallel/test-stdio-closed.js"
-                            "test/sequential/test-child-process-emfile.js")
+             (substitute*
+                 (let ((common
+                        '("lib/child_process.js"
+                          "lib/internal/v8_prof_polyfill.js"
+                          "test/parallel/test-child-process-spawnsync-shell.js"
+                          "test/parallel/test-stdio-closed.js"
+                          "test/sequential/test-child-process-emfile.js"))
+                       ;; not in bootstap node:
+                       (sigxfsz "test/parallel/test-fs-write-sigxfsz.js"))
+                   (if (file-exists? sigxfsz)
+                       (cons sigxfsz common)
+                       common))
                (("'/bin/sh'")
                 (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
 
@@ -122,7 +131,10 @@ (define-public node
                             "test/parallel/test-child-process-exec-env.js")
                (("'/usr/bin/env'")
                 (string-append "'" (assoc-ref inputs "coreutils")
-                               "/bin/env'")))
+                               "/bin/env'")))))
+         (add-after 'patch-hardcoded-program-references
+             'delete-problematic-tests
+           (lambda* (#:key inputs #:allow-other-keys)
 
              ;; FIXME: These tests fail in the build container, but they don't
              ;; seem to be indicative of real problems in practice.
@@ -217,9 +229,13 @@ (define-public node
                (setenv "CXX" ,(cxx-for-target))
                (setenv "PKG_CONFIG" ,(pkg-config-for-target))
                (apply invoke
-                      (string-append (assoc-ref (or native-inputs inputs)
-                                                "python")
-                                     "/bin/python")
+                      (let ((python
+                             (string-append (assoc-ref (or native-inputs inputs)
+                                                       "python")
+                                            "/bin/python")))
+                        (if (file-exists? python)
+                            python
+                            (string-append python "3")))
                       "configure" flags))))
          (add-after 'patch-shebangs 'patch-npm-shebang
            (lambda* (#:key outputs #:allow-other-keys)
@@ -256,7 +272,7 @@ (define-public node
             (variable "NODE_PATH")
             (files '("lib/node_modules")))))
     (inputs
-     `(("bash" ,bash)
+     `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
        ("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
@@ -712,66 +728,8 @@ (define-public node-lts
                                    libuv "/lib:"
                                    zlib "/lib"
                                    "'],"))))))
-           (replace 'configure
-             ;; Node's configure script is actually a python script, so we can't
-             ;; run it with bash.
-             (lambda* (#:key outputs (configure-flags '()) native-inputs inputs
-                       #:allow-other-keys)
-               (let* ((prefix (assoc-ref outputs "out"))
-                      (xflags ,(if (%current-target-system)
-                                   `'("--cross-compiling"
-                                     ,(string-append
-                                       "--dest-cpu="
-                                       (match (%current-target-system)
-                                         ((? (cut string-prefix? "arm" <>))
-                                          "arm")
-                                         ((? (cut string-prefix? "aarch64" <>))
-                                          "arm64")
-                                         ((? (cut string-prefix? "i686" <>))
-                                          "ia32")
-                                         ((? (cut string-prefix? "x86_64" <>))
-                                          "x64")
-                                         ((? (cut string-prefix? "powerpc64" <>))
-                                          "ppc64")
-                                         (_ "unsupported"))))
-                                   ''()))
-                      (flags (cons
-                               (string-append "--prefix=" prefix)
-                               (append xflags configure-flags))))
-                 (format #t "build directory: ~s~%" (getcwd))
-                 (format #t "configure flags: ~s~%" flags)
-                 ;; Node's configure script expects the CC environment variable to
-                 ;; be set.
-                 (setenv "CC_host" "gcc")
-                 (setenv "CXX_host" "g++")
-                 (setenv "CC" ,(cc-for-target))
-                 (setenv "CXX" ,(cxx-for-target))
-                 (setenv "PKG_CONFIG" ,(pkg-config-for-target))
-                 (apply invoke
-                        (string-append (assoc-ref (or native-inputs inputs)
-                                                  "python")
-                                       "/bin/python3")
-                        "configure" flags))))
-           (replace 'patch-files
+           (replace 'delete-problematic-tests
              (lambda* (#:key inputs #:allow-other-keys)
-               ;; Fix hardcoded /bin/sh references.
-               (substitute* '("lib/child_process.js"
-                              "lib/internal/v8_prof_polyfill.js"
-                              "test/parallel/test-child-process-spawnsync-shell.js"
-                              "test/parallel/test-fs-write-sigxfsz.js"
-                              "test/parallel/test-stdio-closed.js"
-                              "test/sequential/test-child-process-emfile.js")
-                 (("'/bin/sh'")
-                  (string-append "'" (assoc-ref inputs "bash") "/bin/sh'")))
-
-               ;; Fix hardcoded /usr/bin/env references.
-               (substitute* '("test/parallel/test-child-process-default-options.js"
-                              "test/parallel/test-child-process-env.js"
-                              "test/parallel/test-child-process-exec-env.js")
-                 (("'/usr/bin/env'")
-                  (string-append "'" (assoc-ref inputs "coreutils")
-                                 "/bin/env'")))
-
                ;; FIXME: These tests fail in the build container, but they don't
                ;; seem to be indicative of real problems in practice.
                (for-each delete-file
@@ -810,8 +768,9 @@ (define-public node-lts
                ;; TODO: Regenerate certs instead.
                (for-each delete-file
                          '("test/parallel/test-tls-passphrase.js"
-                           "test/parallel/test-tls-server-verify.js"))
-
+                           "test/parallel/test-tls-server-verify.js"))))
+           (add-after 'delete-problematic-tests 'replace-llhttp-sources
+             (lambda* (#:key inputs #:allow-other-keys)
                ;; Replace pre-generated llhttp sources
                (let ((llhttp (assoc-ref inputs "llhttp")))
                  (copy-file (string-append llhttp "/src/llhttp.c")
@@ -838,7 +797,7 @@ (define-public node-lts
        ("python" ,python)
        ("util-linux" ,util-linux)))
     (inputs
-     `(("bash" ,bash)
+     `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
        ("c-ares" ,c-ares-for-node)
        ("icu4c" ,icu4c-67)
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:27
[PATCH v3 00/43] guix: node-build-system: Support compiling add-ons with node-gyp.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-1-philip@philipmcgrath.com
Hi!

Here is a v3 of this patch series, hopefully incorporating all of the
suggestions. In particular, as Timothy suggested
in https://issues.guix.gnu.org/51838#59, I've tried to make sure the earlier
patches, at least through #:absent-dependencies, are able to be applied even
if there is more discussion needed on the later patches.

As I mentioned in https://issues.guix.gnu.org/51838#64, while experimenting
with v2 of this series, I discovered an issue with the install script
automatically generated by npm. Patch 21 ``guix: node-build-system: Add
avoid-node-gyp-rebuild phase.'' fixed the issue: I explain the details in
comments there.

This series also adds two additional leaf packages, `node-segfault-handler'
and `node-serialport', that confirm these patches can support additional
styles of native addons. In particular, because the native addon for
`node-serialport' is actually in one of the intermediate packages,
`node-serialport-bindings', it serves as a test case for the new
`avoid-node-gyp-rebuild' phase.

I've also put these patches up
if anyone finds that useful.

-Philip

Philip McGrath (43):
gnu: node: Avoid duplicating build phases.
gnu: node: Update to 10.24.1 for bootstrapping.
gnu: node: Patch shebangs in node_modules.
gnu: node: Add an npmrc file to set nodedir.
guix: node-build-system: Refactor patch-dependencies phase.
guix: node-build-system: Add #:absent-dependencies argument.
gnu: node-semver-bootstrap: Use #:absent-dependencies.
gnu: node-ms-bootstrap: Use #:absent-dependencies.
gnu: node-binary-search-bootstrap: Use #:absent-dependencies.
gnu: node-debug-bootstrap: Use #:absent-dependencies.
gnu: node-llparse-builder-bootstrap: Use #:absent-dependencies.
gnu: node-llparse-frontend-bootstrap: Use #:absent-dependencies.
gnu: node-llparse-bootstrap: Use #:absent-dependencies.
gnu: node-semver: Use #:absent-dependencies.
gnu: node-wrappy: Use #:absent-dependencies.
gnu: node-once: Use #:absent-dependencies.
gnu: node-irc-colors: Use #:absent-dependencies.
gnu: node-irc: Use #:absent-dependencies.
guix: node-build-system: Add implicit libuv input.
guix: node-build-system: Add delete-lockfiles phase.
guix: node-build-system: Add avoid-node-gyp-rebuild phase.
gnu: Add node-inherits.
gnu: Add node-safe-buffer.
gnu: Add node-string-decoder.
gnu: Add node-readable-stream.
gnu: Add node-nan.
gnu: Add node-openzwave-shared.
gnu: Add node-addon-api.
gnu: Add node-sqlite3.
gnu: Add node-file-uri-to-path.
gnu: Add node-bindings.
gnu: Add node-segfault-handler.
gnu: Add node-serialport-binding-abstract.
gnu: Add node-serialport-parser-delimiter.
gnu: Add node-serialport-parser-readling.
gnu: Add node-serialport-bindings.
gnu: Add node-serialport-parser-regex.
gnu: Add node-serialport-parser-ready.
gnu: Add node-serialport-parser-inter-byte-timeout.
gnu: Add node-serialport-parser-cctalk.
gnu: Add node-serialport-parser-byte-length.
gnu: Add node-serialport-stream.
gnu: Add node-serialport.

gnu/packages/node-xyz.scm | 833 +++++++++++++++++++++++++++++--
gnu/packages/node.scm | 219 ++++----
gnu/packages/zwave.scm | 65 +++
guix/build-system/node.scm | 28 +-
guix/build/node-build-system.scm | 129 ++++-
5 files changed, 1119 insertions(+), 155 deletions(-)

--
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:27
[PATCH v3 03/43] gnu: node: Patch shebangs in node_modules.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-4-philip@philipmcgrath.com
* gnu/packages/node.scm (node)[arguments]: Replace 'patch-npm-shebang
and 'patch-node-shebang with a new 'patch-nested-shebangs that also
handles node-gyp and other shebangs under "/lib/node_modules".
[inputs]: Add Python for node-gyp as "python-for-target".
(node-lts)[inputs]: Likewise.
(libnode)[arguments]: Adjust to delete 'patch-nested-shebangs rather
than 'patch-npm-shebang and 'patch-node-shebang.
---
gnu/packages/node.scm | 34 +++++++++++++++++-----------------
1 file changed, 17 insertions(+), 17 deletions(-)

Toggle diff (65 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 71b66774a6..a57a74fb81 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -237,21 +237,20 @@ (define-public node
                             python
                             (string-append python "3")))
                       "configure" flags))))
-         (add-after 'patch-shebangs 'patch-npm-shebang
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((bindir (string-append (assoc-ref outputs "out")
-                                           "/bin"))
-                    (npm    (string-append bindir "/npm"))
-                    (target (readlink npm)))
-               (with-directory-excursion bindir
-                 (patch-shebang target (list bindir))))))
-         (add-after 'install 'patch-node-shebang
-           (lambda* (#:key outputs #:allow-other-keys)
-             (let* ((bindir (string-append (assoc-ref outputs "out")
-                                           "/bin"))
-                    (npx    (readlink (string-append bindir "/npx"))))
-               (with-directory-excursion bindir
-                 (patch-shebang npx (list bindir)))))))))
+         (add-after 'patch-shebangs 'patch-nested-shebangs
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let* ((prefix (assoc-ref outputs "out"))
+                    (path (map (lambda (dir)
+                                 (string-append dir "/bin"))
+                               (list prefix
+                                     (assoc-ref inputs "python-for-target")))))
+               (for-each
+                (lambda (file)
+                  (patch-shebang file path))
+                (find-files (string-append prefix "/lib/node_modules")
+                            (lambda (file stat)
+                              (executable-file? file))
+                            #:stat lstat))))))))
     (native-inputs
      `(;; Runtime dependencies for binaries used as a bootstrap.
        ("c-ares" ,c-ares)
@@ -274,6 +273,7 @@ (define-public node
     (inputs
      `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
+       ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
        ("c-ares" ,c-ares)
        ("http-parser" ,http-parser)
        ("icu4c" ,icu4c)
@@ -799,6 +799,7 @@ (define-public node-lts
     (inputs
      `(("bash" ,bash-minimal)
        ("coreutils" ,coreutils)
+       ("python-for-target" ,python-wrapper) ;; for node-gyp (supports python3)
        ("c-ares" ,c-ares-for-node)
        ("icu4c" ,icu4c-67)
        ("libuv" ,libuv-for-node)
@@ -817,5 +818,4 @@ (define-public libnode
         `(cons* "--shared" "--without-npm" ,flags))
        ((#:phases phases '%standard-phases)
         `(modify-phases ,phases
-           (delete 'patch-npm-shebang)
-           (delete 'patch-node-shebang)))))))
+           (delete 'patch-nested-shebangs)))))))
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:27
[PATCH v3 04/43] gnu: node: Add an npmrc file to set nodedir.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-5-philip@philipmcgrath.com
* gnu/packages/node.scm (node, node-lts)[arguments]: Add a phase
'install-npmrc to create a "built-in" npmrc file that configures
"nodedir" to point to the output store path.
(libnode)[arguments]: Delete the 'install-npmrc phase.
---
gnu/packages/node.scm | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)

Toggle diff (35 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index a57a74fb81..0f67fe79c2 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -250,7 +250,21 @@ (define-public node
                 (find-files (string-append prefix "/lib/node_modules")
                             (lambda (file stat)
                               (executable-file? file))
-                            #:stat lstat))))))))
+                            #:stat lstat)))))
+         (add-after 'install 'install-npmrc
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (with-output-to-file
+                   ;; Use the config file "primarily for distribution
+                   ;; maintainers" rather than "{prefix}/etc/npmrc",
+                   ;; especially because node-build-system uses --prefix
+                   ;; to install things to their store paths:
+                   (string-append out "/lib/node_modules/npm/npmrc")
+                 (lambda ()
+                   ;; Tell npm (mostly node-gyp) where to find our
+                   ;; installed headers so it doesn't try to
+                   ;; download them from the internet:
+                   (format #t "nodedir=~a\n" out)))))))))
     (native-inputs
      `(;; Runtime dependencies for binaries used as a bootstrap.
        ("c-ares" ,c-ares)
@@ -818,4 +832,5 @@ (define-public libnode
         `(cons* "--shared" "--without-npm" ,flags))
        ((#:phases phases '%standard-phases)
         `(modify-phases ,phases
+           (delete 'install-npmrc)
            (delete 'patch-nested-shebangs)))))))
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 05/43] guix: node-build-system: Refactor patch-dependencies phase.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-6-philip@philipmcgrath.com
* guix/build/node-build-system.scm (patch-dependencies): Strictly
follow the linearity rules for `assoc-set!` and friends.
Clarify the types of the arguments to and return value from the
internal helper function `resolve-dependencies`.
---
guix/build/node-build-system.scm | 53 ++++++++++++++++++++++----------
1 file changed, 36 insertions(+), 17 deletions(-)

Toggle diff (69 lines)
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 70a367618e..6fade54670 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -73,26 +73,45 @@ (define* (patch-dependencies #:key inputs #:allow-other-keys)
 
   (define index (index-modules (map cdr inputs)))
 
-  (define (resolve-dependencies package-meta meta-key)
-    (fold (lambda (key+value acc)
-            (match key+value
-              ('@ acc)
-              ((key . value) (acons key (hash-ref index key value) acc))))
-          '()
-          (or (assoc-ref package-meta meta-key) '())))
+  (define (resolve-dependencies meta-alist meta-key)
+    ;; Given:
+    ;;  - The alist from "package.json", with the '@ unwrapped
+    ;;  - A string key, like "dependencies"
+    ;; Returns: an alist (without a wrapping '@) like the entry in
+    ;; meta-alist for meta-key, but with dependencies supplied
+    ;; by Guix packages mapped to the absolute store paths to use.
+    (match (assoc-ref meta-alist meta-key)
+      (#f
+       '())
+      (('@ . orig-deps)
+       (fold (match-lambda*
+               (((key . value) acc)
+                (acons key (hash-ref index key value) acc)))
+             '()
+             orig-deps))))
 
   (with-atomic-file-replacement "package.json"
     (lambda (in out)
-      (let ((package-meta (read-json in)))
-        (assoc-set! package-meta "dependencies"
-                    (append
-                     '(@)
-                     (resolve-dependencies package-meta "dependencies")
-                     (resolve-dependencies package-meta "peerDependencies")))
-        (assoc-set! package-meta "devDependencies"
-                    (append
-                     '(@)
-                     (resolve-dependencies package-meta "devDependencies")))
+      ;; It is unsafe to rely on 'assoc-set!' to update an
+      ;; existing assosciation list variable:
+      ;; see 'info "(guile)Adding or Setting Alist Entries"'.
+      (let* ((package-meta (read-json in))
+             (alist (match package-meta
+                      ((@ . alist) alist)))
+             (alist
+              (assoc-set!
+               alist "dependencies"
+               (append
+                '(@)
+                (resolve-dependencies alist "dependencies")
+                (resolve-dependencies alist "peerDependencies"))))
+             (alist
+              (assoc-set!
+               alist "devDependencies"
+               (append
+                '(@)
+                (resolve-dependencies alist "devDependencies"))))
+             (package-meta (cons '@ alist)))
         (write-json package-meta out))))
   #t)
 
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 06/43] guix: node-build-system: Add #:absent-dependencies argument.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-7-philip@philipmcgrath.com
* guix/build-system/node.scm (lower, node-build): Add optional
argument #:absent-dependencies with default of ''(). Pass it on
to the build-side code.
* guix/build/node-build-system.scm (patch-dependencies): Respect
the #:absent-dependencies argument, removing specified npm
packages from the "dependencies" or "devDependencies" tables
in "package.json".
---
guix/build-system/node.scm | 19 ++++++++++++++++++-
guix/build/node-build-system.scm | 7 +++++--
2 files changed, 23 insertions(+), 3 deletions(-)

Toggle diff (78 lines)
diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 98f63f87ef..4f437f9c0d 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -44,6 +44,7 @@ (define (default-node)
 (define* (lower name
                 #:key source inputs native-inputs outputs system target
                 (node (default-node))
+                (absent-dependencies ''())
                 #:allow-other-keys
                 #:rest arguments)
   "Return a bag for NAME."
@@ -73,6 +74,7 @@ (define* (node-build store name inputs
                      (tests? #t)
                      (phases '(@ (guix build node-build-system)
                                  %standard-phases))
+                     (absent-dependencies ''())
                      (outputs '("out"))
                      (search-paths '())
                      (system (%current-system))
@@ -80,7 +82,21 @@ (define* (node-build store name inputs
                      (imported-modules %node-build-system-modules)
                      (modules '((guix build node-build-system)
                                 (guix build utils))))
-  "Build SOURCE using NODE and INPUTS."
+  "Build SOURCE using NODE and INPUTS.
+
+The builder will remove Node.js packages listed in ABSENT-DEPENCENCIES from
+the 'package.json' file's 'dependencies' and 'devDependencies' tables.  This
+mechanism can be used both avoid dependencies we don't want (e.g. optional
+features that would increase closure size) and to work around dependencies
+that haven't been packaged for Guix yet (e.g. test utilities)."
+  ;; Before #:absent-dependencies existed, this scenario was often handled by
+  ;; deleting the 'configure phase. Using #:absent-dependencies, instead,
+  ;; retains the check that no dependencies are silently missing and other
+  ;; actions performed by 'npm install', such as building native
+  ;; addons. Having an explicit list of absent dependencies in the package
+  ;; definition should also facilitate future maintenence: for example, if we
+  ;; add a package for a test framework, it should be easy to find all the
+  ;; other packages that use it and enable their tests.
   (define builder
     `(begin
        (use-modules ,@modules)
@@ -94,6 +110,7 @@ (define builder
                    #:test-target ,test-target
                    #:tests? ,tests?
                    #:phases ,phases
+                   #:absent-dependencies ,absent-dependencies
                    #:outputs %outputs
                    #:search-paths ',(map search-path-specification->sexp
                                          search-paths)
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 6fade54670..249b3deee6 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -69,7 +69,8 @@ (define (list-modules directory)
               input-paths)
     index))
 
-(define* (patch-dependencies #:key inputs #:allow-other-keys)
+(define* (patch-dependencies #:key inputs absent-dependencies
+                             #:allow-other-keys)
 
   (define index (index-modules (map cdr inputs)))
 
@@ -86,7 +87,9 @@ (define (resolve-dependencies meta-alist meta-key)
       (('@ . orig-deps)
        (fold (match-lambda*
                (((key . value) acc)
-                (acons key (hash-ref index key value) acc)))
+                (if (member key absent-dependencies)
+                    acc
+                    (acons key (hash-ref index key value) acc))))
              '()
              orig-deps))))
 
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:27
[PATCH v3 02/43] gnu: node: Update to 10.24.1 for bootstrapping.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-3-philip@philipmcgrath.com
* gnu/packages/node.scm (node): Update to 10.24.1.
---
gnu/packages/node.scm | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

Toggle diff (23 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 7ebbbc3060..71b66774a6 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -57,14 +57,14 @@ (define-module (gnu packages node)
 (define-public node
   (package
     (name "node")
-    (version "10.24.0")
+    (version "10.24.1")
     (source (origin
               (method url-fetch)
               (uri (string-append "https://nodejs.org/dist/v" version
                                   "/node-v" version ".tar.xz"))
               (sha256
                (base32
-                "1k1srdis23782hnd1ymgczs78x9gqhv77v0am7yb54gqcspp70hm"))
+                "032801kg24j04xmf09m0vxzlcz86sv21s24lv9l4cfv08k1c4byp"))
               (modules '((guix build utils)))
               (snippet
                `(begin
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 08/43] gnu: node-ms-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-9-philip@philipmcgrath.com
gnu/packages/node.scm (node-ms-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node.scm | 9 ++++++---
1 file changed, 6 insertions(+), 3 deletions(-)

Toggle diff (22 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 6c958eebec..ca4bf1ad23 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -358,9 +358,12 @@ (define-public node-ms-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       `("eslint"
+         "expect.js"
+         "husky"
+         "lint-staged"
+         "mocha")))
     (home-page "https://github.com/zeit/ms#readme")
     (properties '((hidden? . #t)))
     (synopsis "Tiny millisecond conversion utility")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 09/43] gnu: node-binary-search-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-10-philip@philipmcgrath.com
gnu/packages/node.scm (node-binary-search-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node.scm | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

Toggle diff (19 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index ca4bf1ad23..2cdae34ae1 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -389,9 +389,9 @@ (define-public node-binary-search-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       `("chai"
+         "mocha")))
     (home-page "https://github.com/darkskyapp/binary-search#readme")
     (properties '((hidden? . #t)))
     (synopsis "Tiny binary search function with comparators")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 07/43] gnu: node-semver-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-8-philip@philipmcgrath.com
gnu/packages/node.scm (node-semver-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node.scm | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)

Toggle diff (18 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 0f67fe79c2..6c958eebec 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -330,9 +330,8 @@ (define-public node-semver-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       '("tap")))
     (home-page "https://github.com/npm/node-semver")
     (properties '((hidden? . #t)))
     (synopsis "Parses semantic versions strings")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 10/43] gnu: node-debug-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-11-philip@philipmcgrath.com
gnu/packages/node.scm (node-debug-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node.scm | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)

Toggle diff (28 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 2cdae34ae1..98a51276e7 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -416,9 +416,18 @@ (define-public node-debug-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
-       #:phases
-       (modify-phases %standard-phases
-         (delete 'configure))))
+       #:absent-dependencies
+       `("brfs"
+         "browserify"
+         "coveralls"
+         "istanbul"
+         "karma"
+         "karma-browserify"
+         "karma-chrome-launcher"
+         "karma-mocha"
+         "mocha"
+         "mocha-lcov-reporter"
+         "xo")))
     (inputs `(("node-ms" ,node-ms-bootstrap)))
     (home-page "https://github.com/visionmedia/debug#readme")
     (properties '((hidden? . #t)))
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 11/43] gnu: node-llparse-builder-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-12-philip@philipmcgrath.com
gnu/packages/node.scm (node-llparse-builder-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
Add a new phase `#:delete-package-lock` to remove the
problematic "package-lock.json".
---
gnu/packages/node.scm | 14 +++++++++++++-
1 file changed, 13 insertions(+), 1 deletion(-)

Toggle diff (29 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 98a51276e7..9d4903a8ca 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -479,9 +479,21 @@ (define-public node-llparse-builder-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
+       #:absent-dependencies
+       `("@types/mocha"
+         "@types/node"
+         "mocha"
+         "ts-node"
+         "tslint"
+         "typescript")
        #:phases
        (modify-phases %standard-phases
-         (delete 'configure)
+         (add-before 'configure 'remove-package-lock
+           ;; Having package-lock.json seems to cause npm
+           ;; to look for things on the internet in the configure phase,
+           ;; even if we have them properly installed.
+           (lambda args
+             (delete-file-recursively "package-lock.json")))
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 12/43] gnu: node-llparse-frontend-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-13-philip@philipmcgrath.com
gnu/packages/node.scm (node-llparse-frontend-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node.scm | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 9d4903a8ca..298b9376a8 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -547,9 +547,16 @@ (define-public node-llparse-frontend-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
+       #:absent-dependencies
+       `("@types/debug"
+         "@types/mocha"
+         "@types/node"
+         "mocha"
+         "ts-node"
+         "tslint"
+         "typescript")
        #:phases
        (modify-phases %standard-phases
-         (delete 'configure)
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 13/43] gnu: node-llparse-bootstrap: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-14-philip@philipmcgrath.com
gnu/packages/node.scm (node-llparse-bootstrap)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
Add a new phase `#:delete-package-lock` to remove the
problematic "package-lock.json".
---
gnu/packages/node.scm | 17 ++++++++++++++++-
1 file changed, 16 insertions(+), 1 deletion(-)

Toggle diff (32 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 298b9376a8..6d48816c77 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -609,9 +609,24 @@ (define-public node-llparse-bootstrap
     (arguments
      `(#:node ,node-bootstrap
        #:tests? #f
+       #:absent-dependencies
+       `("@types/debug"
+         "@types/mocha"
+         "@types/node"
+         "esm"
+         "llparse-test-fixture"
+         "mocha"
+         "ts-node"
+         "tslint"
+         "typescript")
        #:phases
        (modify-phases %standard-phases
-         (delete 'configure)
+         (add-before 'configure 'remove-package-lock
+           ;; Having package-lock.json seems to cause npm
+           ;; to look for things on the internet in the configure phase,
+           ;; even if we have them properly installed.
+           (lambda args
+             (delete-file-recursively "package-lock.json")))
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 14/43] gnu: node-semver: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-15-philip@philipmcgrath.com
gnu/packages/node-xyz.scm (node-semver)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node-xyz.scm | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)

Toggle diff (22 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 98c611f227..93a033d33e 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -289,11 +289,10 @@ (define-public node-semver
                 "06biknqb05r9xsmcflm3ygh50pjvdk84x6r79w43kmck4fn3qn5p"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f ; FIXME: Tests depend on node-tap
-       #:phases
-       (modify-phases %standard-phases
-         ;; The only dependency to check for is tap, which we don't have.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       '("tap")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
     (home-page "https://github.com/npm/node-semver")
     (synopsis "Parses semantic versions strings")
     (description
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 15/43] gnu: node-wrappy: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-16-philip@philipmcgrath.com
gnu/packages/node-xyz.scm (node-wrappy)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node-xyz.scm | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)

Toggle diff (19 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 93a033d33e..77577cb315 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -316,10 +316,8 @@ (define-public node-wrappy
     (build-system node-build-system)
     (arguments
      '(#:tests? #f ; FIXME: Tests depend on node-tap
-       #:phases
-       (modify-phases %standard-phases
-         ;; The only dependency to check for is tap, which we don't have.
-         (delete 'configure))))
+       #:absent-dependencies
+       '("tap")))
     (home-page "https://github.com/npm/wrappy")
     (synopsis "Callback wrapping utility")
     (description "@code{wrappy} is a utility for Node.js to wrap callbacks.")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 17/43] gnu: node-irc-colors: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-18-philip@philipmcgrath.com
gnu/packages/node-xyz.scm (node-irc-colors)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node-xyz.scm | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)

Toggle diff (23 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index eb071320cd..9646c14243 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -367,12 +367,10 @@ (define-public node-irc-colors
                 "0q3y34rbnlc55jcakmdxkicwazyvyph9r6gaf6hi8k7wj2nfwfli"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f                      ; FIXME: tests depend on node-istanbul
-       #:phases
-       (modify-phases %standard-phases
-         ;; The default configure phase fails due to various packages
-         ;; being missing, as we don't have them packaged yet.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       `("istanbul"
+         "vows")
+       #:tests? #f))
     (home-page "https://github.com/fent/irc-colors.js")
     (synopsis "Node.js module providing color and formatting for IRC")
     (description "@code{node-irc-colors} is a Node.js module that
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 18/43] gnu: node-irc: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-19-philip@philipmcgrath.com
gnu/packages/node-xyz.scm (node-irc)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node-xyz.scm | 12 ++++++------
1 file changed, 6 insertions(+), 6 deletions(-)

Toggle diff (25 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 9646c14243..998d0a9a90 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -394,12 +394,12 @@ (define-public node-irc
                 "1ln4qfx20jbwg4cp8lp0vf27m5281z2sz16d15xd6150n26cbi4x"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f                      ; FIXME: tests depend on node-faucet
-       #:phases
-       (modify-phases %standard-phases
-         ;; The default configure phase fails due to various packages
-         ;; being missing, as we don't have them packaged yet.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       `("ansi-color"
+         "faucet"
+         "jscs"
+         "tape")
+       #:tests? #f))
     (inputs
      `(("node-irc-colors" ,node-irc-colors)))
     (home-page "https://github.com/martynsmith/node-irc")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 16/43] gnu: node-once: Use #:absent-dependencies.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-17-philip@philipmcgrath.com
gnu/packages/node-xyz.scm (node-once)[arguments]: Add
`#:absent-dependencies`. Stop deleting the `'configure` phase.
---
gnu/packages/node-xyz.scm | 11 ++++-------
1 file changed, 4 insertions(+), 7 deletions(-)

Toggle diff (24 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 77577cb315..eb071320cd 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -338,13 +338,10 @@ (define-public node-once
                 "1z8dcbf28dqdcp4wb0c53wrs90a07nkrax2c9kk26dsk1dhrnxav"))))
     (build-system node-build-system)
     (arguments
-     '(#:tests? #f                      ; FIXME: tests depend on node-tap
-       #:phases
-       (modify-phases %standard-phases
-         ;; The default configure phase fails due to tap being missing, as we do
-         ;; not have tap packaged yet.  It is used only for tests.  This package
-         ;; still works as a dependency of node-glob and node-inflight.
-         (delete 'configure))))
+     '(#:absent-dependencies
+       '("tap")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
     (inputs
      `(("node-wrappy" ,node-wrappy)))
     (home-page "https://github.com/isaacs/once")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 20/43] guix: node-build-system: Add delete-lockfiles phase.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-21-philip@philipmcgrath.com
* guix/build/node-build-system.scm (delete-lockfiles): New function.
Remove 'package-lock.json', 'yarn.lock', and 'npm-shrinkwrap.json',
if they exist. Because these files specify dependency both exact
versions and integrity hashes, they only cause problems for Guix.
(%standard-phases): Add 'delete-lockfiles' after 'patch-dependencies'.
---
gnu/packages/node.scm | 12 ------------
guix/build/node-build-system.scm | 12 ++++++++++++
2 files changed, 12 insertions(+), 12 deletions(-)

Toggle diff (62 lines)
diff --git a/gnu/packages/node.scm b/gnu/packages/node.scm
index 6d48816c77..5289e2fe4f 100644
--- a/gnu/packages/node.scm
+++ b/gnu/packages/node.scm
@@ -488,12 +488,6 @@ (define-public node-llparse-builder-bootstrap
          "typescript")
        #:phases
        (modify-phases %standard-phases
-         (add-before 'configure 'remove-package-lock
-           ;; Having package-lock.json seems to cause npm
-           ;; to look for things on the internet in the configure phase,
-           ;; even if we have them properly installed.
-           (lambda args
-             (delete-file-recursively "package-lock.json")))
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
@@ -621,12 +615,6 @@ (define-public node-llparse-bootstrap
          "typescript")
        #:phases
        (modify-phases %standard-phases
-         (add-before 'configure 'remove-package-lock
-           ;; Having package-lock.json seems to cause npm
-           ;; to look for things on the internet in the configure phase,
-           ;; even if we have them properly installed.
-           (lambda args
-             (delete-file-recursively "package-lock.json")))
          (replace 'build
            (lambda* (#:key inputs #:allow-other-keys)
              (let ((esbuild (string-append (assoc-ref inputs "esbuild")
diff --git a/guix/build/node-build-system.scm b/guix/build/node-build-system.scm
index 249b3deee6..892104b6d2 100644
--- a/guix/build/node-build-system.scm
+++ b/guix/build/node-build-system.scm
@@ -118,6 +118,17 @@ (define (resolve-dependencies meta-alist meta-key)
         (write-json package-meta out))))
   #t)
 
+(define* (delete-lockfiles #:key inputs #:allow-other-keys)
+  "Delete 'package-lock.json', 'yarn.lock', and 'npm-shrinkwrap.json', if they
+exist."
+  (for-each (lambda (pth)
+              (when (file-exists? pth)
+                (delete-file pth)))
+            '("package-lock.json"
+              "yarn.lock"
+              "npm-shrinkwrap.json"))
+  #t)
+
 (define* (configure #:key outputs inputs #:allow-other-keys)
   (let ((npm (string-append (assoc-ref inputs "node") "/bin/npm")))
     (invoke npm "--offline" "--ignore-scripts" "install")
@@ -168,6 +179,7 @@ (define %standard-phases
   (modify-phases gnu:%standard-phases
     (add-after 'unpack 'set-home set-home)
     (add-before 'configure 'patch-dependencies patch-dependencies)
+    (add-after 'patch-dependencies 'delete-lockfiles delete-lockfiles)
     (replace 'configure configure)
     (replace 'build build)
     (replace 'check check)
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 19/43] guix: node-build-system: Add implicit libuv input.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-20-philip@philipmcgrath.com
* guix/build-system/node.scm (lower): Add the version of libuv
used as an input to the #:node package as an additional implicit
input, so that packages needing libuv always get the correct version.
---
guix/build-system/node.scm | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)

Toggle diff (32 lines)
diff --git a/guix/build-system/node.scm b/guix/build-system/node.scm
index 4f437f9c0d..179da65ee8 100644
--- a/guix/build-system/node.scm
+++ b/guix/build-system/node.scm
@@ -1,6 +1,8 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Jelle Licht <jlicht@fsfe.org>
 ;;; Copyright © 2019 Timothy Sample <samplet@ngyro.com>
+;;; Copyright © 2021 Pierre Langlois <pierre.langlois@gmx.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -59,10 +61,15 @@ (define private-keywords
                               `(("source" ,source))
                               '())
                         ,@inputs
-
                         ;; Keep the standard inputs of 'gnu-build-system'.
                         ,@(standard-packages)))
          (build-inputs `(("node" ,node)
+                         ;; Many packages with native addons need
+                         ;; libuv headers. The libuv version must
+                         ;; be exactly the same as for the node
+                         ;; package we are adding implicitly,
+                         ;; so we take care of adding libuv, too.
+                         ("libuv" ,@(assoc-ref (package-inputs node) "libuv"))
                          ,@native-inputs))
          (outputs outputs)
          (build node-build)
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 23/43] gnu: Add node-safe-buffer.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-24-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-safe-buffer): New variable.
---
gnu/packages/node-xyz.scm | 28 ++++++++++++++++++++++++++++
1 file changed, 28 insertions(+)

Toggle diff (41 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 9d8bf27852..6dff0ddba5 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -382,6 +382,34 @@ (define-public node-inherits
 @code{inherits()}.")
     (license license:isc)))
 
+(define-public node-safe-buffer
+  (package
+    (name "node-safe-buffer")
+    (version "5.2.1")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/feross/safe-buffer")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0r26m0nl41h90ihnl2xf0cqs6z9z7jb87dl5j8yqb7887r9jlbpi"))))
+    (build-system node-build-system)
+    (arguments
+     '(#:absent-dependencies
+       '("tape"
+         "standard")
+       #:tests? #f))
+    (home-page
+     "https://github.com/feross/safe-buffer")
+    (synopsis "Safer Node.js Buffer API")
+    (description "A safe drop-in replacement the Node.js @code{Buffer} API
+that works in all versions of Node.js, using the built-in implementation when
+available.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 24/43] gnu: Add node-string-decoder.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-25-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-string-decoder): New variable.
---
gnu/packages/node-xyz.scm | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)

Toggle diff (47 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 6dff0ddba5..c6cce64f7d 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -410,6 +410,40 @@ (define-public node-safe-buffer
 available.")
     (license license:expat)))
 
+(define-public node-string-decoder
+  (package
+    (name "node-string-decoder")
+    (version "1.3.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/string_decoder")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0xxvyya9fl9rlkqwmxzqzbz4rdr3jgw4vf37hff7cgscxkhg266k"))))
+    (build-system node-build-system)
+    (arguments
+     '(#:absent-dependencies
+       '("tap"
+         "core-util-is"
+         "babel-polyfill")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
+    (inputs
+     `(("node-safe-buffer" ,node-safe-buffer)
+       ("node-inherits" ,node-inherits)))
+    (home-page
+     "https://github.com/nodejs/string_decoder")
+    (synopsis
+     "Node.js core @code{string_decoder} for userland")
+    (description
+     "This package is a mirror of the @code{string_decoder} implementation in
+Node-core.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 22/43] gnu: Add node-inherits.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-23-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-inherits): New variable.
---
gnu/packages/node-xyz.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (50 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 998d0a9a90..9d8bf27852 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -3,6 +3,7 @@
 ;;; Copyright © 2020 Giacomo Leidi <goodoldpaul@autistici.org>
 ;;; Copyright © 2021 Noisytoot <noisytoot@disroot.org>
 ;;; Copyright © 2021 Charles <charles.b.jackson@protonmail.com>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -352,6 +353,35 @@ (define-public node-once
 if desired.")
     (license license:isc)))
 
+(define-public node-inherits
+  (package
+    (name "node-inherits")
+    (version "2.0.4")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/isaacs/inherits")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0cpsr5yqwkxpbbbbl0rwk4mcby6zbx841k2zb4c3gb1579i5wq9p"))))
+    (build-system node-build-system)
+    (arguments
+     '(#:absent-dependencies
+       '("tap")
+       ;; FIXME: Tests depend on node-tap
+       #:tests? #f))
+    (home-page
+     "https://github.com/isaacs/inherits")
+    (synopsis
+     "Browser-friendly inheritance Node.js")
+    (description
+     "Browser-friendly inheritance fully compatible with standard Node.js
+@code{inherits()}.")
+    (license license:isc)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 25/43] gnu: Add node-readable-stream.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-26-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-readable-stream): New variable.
---
gnu/packages/node-xyz.scm | 54 +++++++++++++++++++++++++++++++++++++++
1 file changed, 54 insertions(+)

Toggle diff (67 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index c6cce64f7d..3e06413908 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -444,6 +444,60 @@ (define-public node-string-decoder
 Node-core.")
     (license license:expat)))
 
+(define-public node-readable-stream
+  (package
+    (name "node-readable-stream")
+    (version "3.6.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/readable-stream")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "0ybl4cdgsm9c5jq3xq8s01201jk8w0yakh63hlclsfbcdfqhd9ri"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:absent-dependencies
+       `("@babel/cli"
+         "@babel/core"
+         "@babel/polyfill"
+         "@babel/preset-env"
+         "airtap"
+         "assert"
+         "bl"
+         "deep-strict-equal"
+         "events.once"
+         "glob"
+         "gunzip-maybe"
+         "hyperquest"
+         "lolex"
+         "nyc"
+         "pump"
+         "rimraf"
+         "tap"
+         "tape"
+         "tar-fs"
+         "util-promisify")
+       #:tests? #f))
+    (inputs
+     `(("node-util-deprecate" ,node-util-deprecate)
+       ("node-string-decoder" ,node-string-decoder)
+       ("node-inherits" ,node-inherits)))
+    (home-page
+     "https://github.com/nodejs/readable-stream")
+    (synopsis
+     "Node.js core streams for userland")
+    (description
+     "This package is a mirror of the streams implementations in Node.js.
+
+If you want to guarantee a stable streams base, regardless of what version of
+Node you (or the users of your libraries) are using, use
+@code{readable-stream} only and avoid the @code{stream} module in Node-core.")
+    (license license:expat)))
+
 (define-public node-irc-colors
   (package
     (name "node-irc-colors")
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 26/43] gnu: Add node-nan.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-27-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-nan): New variable.
---
gnu/packages/node-xyz.scm | 44 +++++++++++++++++++++++++++++++++++++++
1 file changed, 44 insertions(+)

Toggle diff (54 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 3e06413908..ed169c0778 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -553,3 +553,47 @@ (define-public node-irc
     (description "@code{node-irc} is an IRC client library for Node.js.
 It has functions for joining, parting, talking, and many other IRC commands.")
     (license license:gpl3+)))
+
+(define-public node-nan
+  (package
+    (name "node-nan")
+    (version "2.15.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/nan")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "18xslh9va5ld872scrp5y4251ax9s3c6qh0lnl1200lpzbsxy7yd"))))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       ;; tests need tap and other dependencies
+       #f
+       #:absent-dependencies
+       '("bindings"
+         "commander"
+         "glob"
+         "request"
+         "node-gyp" ;; would be needed for tests
+         "tap"
+         "xtend")))
+    (inputs
+     `(("readable-stream" ,node-readable-stream)))
+    (home-page "https://github.com/nodejs/nan")
+    (synopsis "Native Abstractions for Node.js")
+    (description "A header file filled with macro and utility goodness for
+making add-on development for Node.js easier across versions 0.8, 0.10, 0.12,
+1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15 and 16.
+
+Thanks to the crazy changes in V8 (and some in Node core), keeping native
+addons compiling happily across versions, particularly 0.10 to 0.12 to 4.0, is
+a minor nightmare.  The goal of this project is to store all logic necessary
+to develop native Node.js addons without having to inspect
+@code{NODE_MODULE_VERSION} and get yourself into a macro-tangle.
+
+This project also contains some helper utilities that make addon development a
+bit more pleasant.")
+    (license license:expat)))
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 28/43] gnu: Add node-addon-api.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-29-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-addon-api): New variable.
---
gnu/packages/node-xyz.scm | 63 +++++++++++++++++++++++++++++++++++++++
1 file changed, 63 insertions(+)

Toggle diff (80 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index ed169c0778..60dbfc163c 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -22,6 +22,9 @@
 
 (define-module (gnu packages node-xyz)
   #:use-module ((guix licenses) #:prefix license:)
+  #:use-module (gnu packages sqlite)
+  #:use-module (gnu packages python)
+  #:use-module (guix gexp)
   #:use-module (guix packages)
   #:use-module (guix git-download)
   #:use-module (guix build-system node))
@@ -597,3 +600,63 @@ (define-public node-nan
 This project also contains some helper utilities that make addon development a
 bit more pleasant.")
     (license license:expat)))
+
+(define-public node-addon-api
+  (package
+    (name "node-addon-api")
+    (version "4.2.0")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/nodejs/node-addon-api")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "1bhvfi2m9nxfz418s619914vmidcnrzbjv6l9nid476c3zlpazch"))))
+    (inputs
+     `(("python" ,python)))
+    (build-system node-build-system)
+    (arguments
+     `(#:absent-dependencies
+       `("benchmark"
+         "bindings"
+         "clang-format"
+         "eslint"
+         "eslint-config-semistandard"
+         "eslint-config-standard"
+         "eslint-plugin-import"
+         "eslint-plugin-node"
+         "eslint-plugin-promise"
+         "fs-extra"
+         "path"
+         "pre-commit"
+         "safe-buffer")
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'skip-js-tests
+           ;; We can't run the js-based tests,
+           ;; but we can still do the C++ parts
+           (lambda args
+             (substitute* "package.json"
+               (("\"test\": \"node test\"")
+                "\"test\": \"echo stopping after pretest on Guix\"")))))))
+    (home-page "https://github.com/nodejs/node-addon-api")
+    (synopsis "Node.js API (Node-API) header-only C++ wrappers")
+    (description "This module contains header-only C++ wrapper classes which
+simplify the use of the C based Node-API provided by Node.js when using C++.
+It provides a C++ object model and exception handling semantics with low
+overhead.
+
+Node-API is an ABI stable C interface provided by Node.js for building native
+addons.  It is intended to insulate native addons from changes in the
+underlying JavaScript engine and allow modules compiled for one version to run
+on later versions of Node.js without recompilation.  The @code{node-addon-api}
+module, which is not part of Node.js, preserves the benefits of the Node-API
+as it consists only of inline code that depends only on the stable API
+provided by Node-API.
+
+It is important to remember that @emph{other} Node.js interfaces such as
+@code{libuv} (included in a project via @code{#include <uv.h>}) are not
+ABI-stable across Node.js major versions.")
+    (license license:expat)))
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 27/43] gnu: Add node-openzwave-shared.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-28-philip@philipmcgrath.com
* gnu/packages/zwave.scm (node-openzwave-shared): New variable.
---
gnu/packages/zwave.scm | 65 ++++++++++++++++++++++++++++++++++++++++++
1 file changed, 65 insertions(+)

Toggle diff (93 lines)
diff --git a/gnu/packages/zwave.scm b/gnu/packages/zwave.scm
index 2019ec32df..586cd66e1d 100644
--- a/gnu/packages/zwave.scm
+++ b/gnu/packages/zwave.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2019 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2021 Philip McGrath <philip@philipmcgrath.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -21,11 +22,14 @@ (define-module (gnu packages zwave)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system node)
   #:use-module (gnu packages)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages node-xyz)
   #:use-module (gnu packages libusb)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages perl)
+  #:use-module (gnu packages python)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages xml))
 
@@ -88,3 +92,64 @@ (define-public open-zwave
 and respond to devices on a Z-Wave network, without requiring in-depth
 knowledge of the Z-Wave protocol.")
     (license license:lgpl3+)))
+
+(define-public node-openzwave-shared
+  (package
+    (name "node-openzwave-shared")
+    (version "1.7.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/OpenZWave/node-openzwave-shared")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32
+         "1bqqy12dzqj05j9wsh50dmn84dddmhz0gjzvd3y20z4hpy1v8rsc"))))
+    (inputs
+     `(("open-zwave" ,open-zwave)
+       ("node-nan" ,node-nan)))
+    (native-inputs
+     `(("which" ,which)
+       ("python" ,python)
+       ("pkg-config" ,pkg-config)))
+    (build-system node-build-system)
+    (arguments
+     `(#:phases
+       (modify-phases %standard-phases
+         (replace 'build
+           (lambda* (#:key inputs #:allow-other-keys)
+             (define node-dir
+               (assoc-ref inputs "node"))
+             (invoke (string-append node-dir "/bin/npx")
+                     "--call"
+                     (string-append
+                      node-dir
+                      "/lib/node_modules/npm/bin/node-gyp-bin/node-gyp"
+                      " "
+                      "rebuild")))))))
+    (home-page "https://github.com/OpenZWave/node-openzwave-shared")
+    (synopsis "Node.js bindings for OpenZWave")
+    (description
+     "With the @code{node-openzwave-shared} package, you can easily control
+and manage your Z-Wave devices (lights, dimmers, blinds, you name it) from
+within Node.js applications.  This library also supports secure
+devices (e.g. door locks) that require encryption.  All widely used Node.js
+versions are supported with the help of @code{node-nan}.
+
+This library is currently able to:
+@itemize @bullet
+@item
+scan a Z-Wave network and report on connected devices;
+@item
+write values to Z-Wave nodes;
+@item
+monitor the network for changes;
+@item
+heal nodes and/or the network; and
+@item
+perform management tasks: add or remove nodes, replace failed nodes,
+manage their group associations, etc.
+@end itemize")
+    (license license:isc)))
-- 
2.32.0
P
P
Philip McGrath wrote on 8 Dec 2021 21:28
[PATCH v3 29/43] gnu: Add node-sqlite3.
(address . 51838@debbugs.gnu.org)
20211208202838.752542-30-philip@philipmcgrath.com
* gnu/packages/node-xyz.scm (node-sqlite3): New variable.
---
gnu/packages/node-xyz.scm | 118 +++++++++++++++++++++++++++++++++++++-
1 file changed, 115 insertions(+), 3 deletions(-)

Toggle diff (142 lines)
diff --git a/gnu/packages/node-xyz.scm b/gnu/packages/node-xyz.scm
index 60dbfc163c..b979d0cd53 100644
--- a/gnu/packages/node-xyz.scm
+++ b/gnu/packages/node-xyz.scm
@@ -615,7 +615,8 @@ (define-public node-addon-api
        (sha256
         (base32 "1bhvfi2m9nxfz418s619914vmidcnrzbjv6l9nid476c3zlpazch"))))
     (inputs
-     `(("python" ,python)))
+     `(("python" ,python)
+       ("node-safe-buffer" ,node-safe-buffer)))
     (build-system node-build-system)
     (arguments
      `(#:absent-dependencies
@@ -630,8 +631,7 @@ (define-public node-addon-api
          "eslint-plugin-promise"
          "fs-extra"
          "path"
-         "pre-commit"
-         "safe-buffer")
+         "pre-commit")
        #:phases
        (modify-phases %standard-phases
          (add-after 'unpack 'skip-js-tests
@@ -660,3 +660,115 @@ (define-public node-addon-api
 @code{libuv} (included in a project via @code{#include <uv.h>}) are not
 ABI-stable across Node.js major versions.")
     (license license:expat)))
+
+(define-public node-sqlite3
+  (package
+    (name "node-sqlite3")
+    (version "5.0.2")
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url "https://github.com/mapbox/node-sqlite3")
+             (commit (string-append "v" version))))
+       (file-name (git-file-name name version))
+       (sha256
+        (base32 "0sbbzzli282nxyfha10zx0k5m8hdp0sf3ipl59khjb7wm449j86h"))
+       (snippet
+        (with-imported-modules '((guix build utils))
+          #~(begin
+              (use-modules (guix build utils))
+              ;; unbundle sqlite
+              (for-each delete-file-recursively
+                        (find-files "deps"
+                                    (lambda (pth stat)
+                                      (gzip-file? pth)))))))))
+    (inputs
+     `(("node-addon-api" ,node-addon-api)
+       ("python" ,python)
+       ("sqlite" ,sqlite)))
+    (build-system node-build-system)
+    (arguments
+     `(#:tests?
+       #f ; FIXME: tests depend on node-mocha
+       #:modules
+       ((guix build node-build-system)
+        (guix build json)
+        (srfi srfi-1)
+        (ice-9 match)
+        (guix build utils))
+       #:absent-dependencies
+       `(;; Normally, this is "built" using @mapbox/node-pre-gyp,
+         ;; which publishes or downloads pre-built binaries
+         ;; or falls back to building from source.
+         ;; Here, we patch out all of that and just build directly.
+         ;; It would be better to patch a version of @mapbox/node-pre-gyp
+         ;; that always builds from source, as Debian does,
+         ;; but there are a number of dependencies that need
+         ;; to be packaged or removed.
+         "@mapbox/node-pre-gyp"
+         "node-pre-gyp" ;; deprecated name still used in some places
+         "aws-sdk"
+         "@mapbox/cloudfriend"
+         ;; Confusingly, this is only a dependency beceuse of
+         ;; @mapbox/node-pre-gyp: with that removed,
+         ;; npm will use its own copy:
+         "node-gyp"
+         ;; These we'd like, we just don't have them yet:
+         "eslint"
+         "mocha")
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'configure 'npm-config-sqlite
+           ;; We need this step even if we do replace @mapbox/node-pre-gyp
+           ;; because the package expects to build its bundled sqlite
+           (lambda* (#:key inputs #:allow-other-keys)
+             (setenv "npm_config_sqlite" (assoc-ref inputs "sqlite"))))
+         (add-after 'install 'patch-binding-path
+           (lambda* (#:key outputs #:allow-other-keys)
+             (with-directory-excursion
+                 (string-append (assoc-ref outputs "out")
+                                "/lib/node_modules/sqlite3/lib")
+               (match (find-files "binding" "\\.node$")
+                 ((rel-path)
+                  (with-atomic-file-replacement "sqlite3-binding.js"
+                    (lambda (in out)
+                      (format out "var binding = require('./~a');\n" rel-path)
+                      (display "module.exports = exports = binding;\n" out))))))))
+         (add-after 'patch-dependencies 'avoid-node-pre-gyp
+           (lambda args
+             (substitute* ".npmignore"
+               (("lib/binding")
+                "#lib/binding # <- patched for Guix"))
+             (with-atomic-file-replacement "package.json"