[PATCH 1/2] gnu: Replace go-github-com-golang-protobuf-proto with go-github-com-golang-protobuf

OpenSubmitted by Dion Mendel.
Details
2 participants
  • Dion Mendel
  • Leo Famulari
Owner
unassigned
Severity
normal
D
D
Dion Mendel wrote on 6 May 2021 18:35
(address . guix-patches@gnu.org)
20210506163511.GB27260@dm9.info
Hi guix,

The github.com/golang/protobuf module contains several packages.
Only proto is currently packaged. This change updates the module from
1.3.1 to 1.5.2 and packages the entire module.

This is needed as I am preparing to package lxd (github.com/lxc/lxd),
which depends on the unpackaged packages from golang/protobuf.

This change replaces go-github-com-golang-protobuf-proto with
go-github-com-golang-protobuf.

There are currently two packages that depend on
go-github-com-golang-protobuf-proto. I have tested this changeset by
building both of these packages.

go-github-com-lucas-clemente-quic-go: this is a source only package that
no other package uses.

syncthing: depends on go-github-com-golang-protobuf-proto via several
intermediary packages. After building, I've run some of the syncthing
executables and they appear to work.
From 47de2a85a1ae1f0502a89d0097d3b36da7d20649 Mon Sep 17 00:00:00 2001
From: Dion Mendel <guix@dm9.info>
Date: Thu, 6 May 2021 23:25:22 +0800
Subject: [PATCH 1/2] gnu: Add go-golang-org-protobuf.

* gnu/packages/golang.scm (go-golang-org-protobuf): New variable.
---
gnu/packages/golang.scm | 30 ++++++++++++++++++++++++++++++
1 file changed, 30 insertions(+)

Toggle diff (50 lines)
diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index 8c3c81ceda..f0fa0e1d36 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -25,6 +25,7 @@
 ;;; Copyright © 2020 Martin Becze <mjbecze@riseup.net>
 ;;; Copyright © 2021 Ricardo Wurmus <rekado@elephly.net>
 ;;; Copyright © 2021 Guillaume Le Vaillant <glv@posteo.net>
+;;; Copyright © 2021 Dion Mendel <guix@dm9.info>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -4832,6 +4833,35 @@ data serialization format.")
     (home-page "https://github.com/golang/protobuf")
     (license license:bsd-3)))
 
+(define-public go-golang-org-protobuf
+  (package
+    (name "go-golang-org-protobuf")
+    (version "1.26.0")
+    (home-page "https://github.com/protocolbuffers/protobuf-go")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url home-page)
+                    (commit (string-append "v" version))))
+              (sha256
+               (base32
+                "0xq6phaps6d0vcv13ga59gzj4306l0ki9kikhmb52h6pq0iwfqlz"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "google.golang.org/protobuf"
+       ; Source-only package
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'build))))
+    (propagated-inputs
+     `(("go-github-com-google-go-cmp-cmp" ,go-github-com-google-go-cmp-cmp)))
+    (synopsis "Go support for Protocol Buffers")
+    (description "This project hosts the Go implementation for protocol buffers, which is a
+language-neutral, platform-neutral, extensible mechanism for serializing
+structured data.")
+    (license license:bsd-3)))
+
 (define-public go-github-com-mattn-go-zglob
   (package
     (name "go-github-com-mattn-go-zglob")
-- 
2.31.1
From 0f029698f12364a475211204498aeb3face1381e Mon Sep 17 00:00:00 2001
From: Dion Mendel <guix@dm9.info>
Date: Thu, 6 May 2021 23:39:16 +0800
Subject: [PATCH 2/2] gnu: Replace go-github-com-golang-protobuf-proto with
go-github-com-golang-protobuf

The github.com/golang/protobuf module contains several packages.
Only proto is currently packaged. This change updates the module from
1.3.1 to 1.5.2 and packages the entire module.

* gnu/packages/golang.scm
(go-github-com-golang-protobuf): New variable.
(go-github-com-golang-protobuf-proto): Removed variable.
(go-github-com-lucas-clemente-quic-go)[propagated-inputs]:
Replace go-github-com-golang-protobuf-proto@1.3.1
with go-github-com-golang-protobuf@1.5.2.
* gnu/packages/syncthing.scm
(go-github-com-matttproud-golang-protobuf-extensions-pbutil)
(go-github-com-prometheus-client-golang)
(go-github-com-prometheus-client-model)
(go-github-com-prometheus-common)
[propagated-inputs]: Replace go-github-com-golang-protobuf-proto@1.3.1
with go-github-com-golang-protobuf@1.5.2.
---
gnu/packages/golang.scm | 32 ++++++++++++++++++++++----------
gnu/packages/syncthing.scm | 16 ++++++++--------
2 files changed, 30 insertions(+), 18 deletions(-)

Toggle diff (112 lines)
diff --git a/gnu/packages/golang.scm b/gnu/packages/golang.scm
index f0fa0e1d36..9a689d225b 100644
--- a/gnu/packages/golang.scm
+++ b/gnu/packages/golang.scm
@@ -4237,7 +4237,7 @@ implementation of generics.")
        ("go-github-com-cheekybits-genny" ,go-github-com-cheekybits-genny)
        ("go-github-com-marten-seemann-chacha20" ,go-github-com-marten-seemann-chacha20)
        ("go-github-com-marten-seemann-qtls" ,go-github-com-marten-seemann-qtls)
-       ("go-github-com-golang-protobuf-proto" ,go-github-com-golang-protobuf-proto)))
+       ("go-github-com-golang-protobuf" ,go-github-com-golang-protobuf)))
     (synopsis "QUIC in Go")
     (description "This package provides a Go language implementation of the QUIC
 network protocol.")
@@ -4808,10 +4808,10 @@ test results.")
     (home-page "https://github.com/gotestyourself/gotestsum")
     (license license:asl2.0)))
 
-(define-public go-github-com-golang-protobuf-proto
+(define-public go-github-com-golang-protobuf
   (package
-    (name "go-github-com-golang-protobuf-proto")
-    (version "1.3.1")
+    (name "go-github-com-golang-protobuf")
+    (version "1.5.2")
     (source (origin
               (method git-fetch)
               (uri (git-reference
@@ -4820,16 +4820,28 @@ test results.")
               (file-name (git-file-name name version))
               (sha256
                (base32
-                "15am4s4646qy6iv0g3kkqq52rzykqjhm4bf08dk0fy2r58knpsyl"))))
+                "1mh5fyim42dn821nsd3afnmgscrzzhn3h8rag635d2jnr23r1zhk"))))
     (build-system go-build-system)
     (arguments
-     '(#:import-path "github.com/golang/protobuf/proto"
-       #:unpack-path "github.com/golang/protobuf"
-       ;; Requires unpackaged golang.org/x/sync/errgroup
-       #:tests? #f))
+     '(#:import-path "github.com/golang/protobuf"
+       ; Source-only package
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'build))))
+    (propagated-inputs
+     `(("go-golang-org-protobuf" ,go-golang-org-protobuf)))
     (synopsis "Go support for Protocol Buffers")
     (description "This package provides Go support for the Protocol Buffers
-data serialization format.")
+data serialization format.
+
+It has been superseded by go-golang-org-protobuf, which contains an updated and
+simplified API, support for protobuf reflection, and many other improvements.
+We recommend that new code use the google.golang.org/protobuf module.
+
+Versions v1.4 and later of go-github-com-golang-protobuf are implemented in
+terms of go-golang-org-protobuf.  Programs which use both modules must use
+at least version v1.4 of this one.")
     (home-page "https://github.com/golang/protobuf")
     (license license:bsd-3)))
 
diff --git a/gnu/packages/syncthing.scm b/gnu/packages/syncthing.scm
index e40f696720..67088ca6ca 100644
--- a/gnu/packages/syncthing.scm
+++ b/gnu/packages/syncthing.scm
@@ -838,8 +838,8 @@ bounds.")
            ;; Source-only package
            (delete 'build))))
       (propagated-inputs
-       `(("go-github-com-golang-protobuf-proto"
-          ,go-github-com-golang-protobuf-proto)))
+       `(("go-github-com-golang-protobuf"
+          ,go-github-com-golang-protobuf)))
       (synopsis "Data model artifacts for Prometheus")
       (description "This package provides data model artifacts for Prometheus.")
       (home-page "https://github.com/prometheus/client_model")
@@ -867,8 +867,8 @@ bounds.")
        '(#:import-path "github.com/matttproud/golang_protobuf_extensions/pbutil"
          #:unpack-path "github.com/matttproud/golang_protobuf_extensions"))
       (propagated-inputs
-       `(("go-github-com-golang-protobuf-proto"
-          ,go-github-com-golang-protobuf-proto)))
+       `(("go-github-com-golang-protobuf"
+          ,go-github-com-golang-protobuf)))
       (synopsis "Streaming Protocol Buffers in Go")
       (description "This package provides various Protocol Buffer
 extensions for the Go language, namely support for record length-delimited
@@ -907,8 +907,8 @@ message streaming.")
            ;; Source-only package
            (delete 'build))))
       (propagated-inputs
-       `(("go-github-com-golang-protobuf-proto"
-          ,go-github-com-golang-protobuf-proto)
+       `(("go-github-com-golang-protobuf"
+          ,go-github-com-golang-protobuf)
          ("go-github-com-matttproud-golang-protobuf-extensions-pbutil"
           ,go-github-com-matttproud-golang-protobuf-extensions-pbutil)
          ("go-github-com-prometheus-client-model"
@@ -968,8 +968,8 @@ system, kernel, and process metrics from the @file{/proc} pseudo file system.")
       (propagated-inputs
        `(("go-github-com-beorn7-perks-quantile"
           ,go-github-com-beorn7-perks-quantile)
-         ("go-github-com-golang-protobuf-proto"
-          ,go-github-com-golang-protobuf-proto)
+         ("go-github-com-golang-protobuf"
+          ,go-github-com-golang-protobuf)
          ("go-github-com-prometheus-client-model"
           ,go-github-com-prometheus-client-model)
          ("go-github-com-prometheus-common"
-- 
2.31.1
L
L
Leo Famulari wrote on 8 May 2021 17:50
(name . Dion Mendel)(address . guix@dm9.info)(address . 48259@debbugs.gnu.org)
YJazUoBEFdnU1OwY@jasmine.lan
On Fri, May 07, 2021 at 12:35:11AM +0800, Dion Mendel wrote:
Toggle quote (4 lines)
> syncthing: depends on go-github-com-golang-protobuf-proto via several
> intermediary packages. After building, I've run some of the syncthing
> executables and they appear to work.

Just to clarify, since commit ed3ef756f521a0df8596a88b66f65b7a1ad99252,
Syncthing does not depend on any Go packages, but instead uses its
bundled copies of all its dependencies.

D
D
Dion Mendel wrote on 9 May 2021 14:37
(address . 48259@debbugs.gnu.org)
20210509123700.GA5323@dm9.info
On Sat, May 08, 2021 at 11:50:42AM -0400, Leo Famulari wrote:
Toggle quote (5 lines)
>
>Just to clarify, since commit ed3ef756f521a0df8596a88b66f65b7a1ad99252,
>Syncthing does not depend on any Go packages, but instead uses its
>bundled copies of all its dependencies.

Is it acceptable to use bundled copies of dependencies? The manual
(section 16.6, checklist #6) says otherwise.

If it's okay, this would make packaging lxd much simpler. I've had to
package around 90 Go libraries so far.

If using lxd's bundled copies of dependencies would be accepted, then I
withdraw this patch request.
L
L
Leo Famulari wrote on 9 May 2021 17:17
(name . Dion Mendel)(address . guix@dm9.info)(address . 48259@debbugs.gnu.org)
YJf9B8AwFmJME2SJ@jasmine.lan
On Sun, May 09, 2021 at 08:37:00PM +0800, Dion Mendel wrote:
Toggle quote (3 lines)
> Is it acceptable to use bundled copies of dependencies? The manual (section
> 16.6, checklist #6) says otherwise.

This item, from the manual section Submitting Patches, says:

"Make sure the package does not use bundled copies of software already
available as separate packages."

So, already, there is some wiggle room based on whether or not things
are "already available as separate packages."

Using bundled dependnencies is strongly discouraged and you are expected
to try to avoid it.

With Syncthing, there were unfixed security vulnerabilities in our
package, and our go-build-system could not build later versions of
Syncthing. So, until our go-build-system is overhauled [0], we'll use
Syncthing's bundled dependencies. They are all free software anyways.

With Go, the old distro model of "no bundling" is not idiomatic, so the
future of Go in Guix may look different from how it is now. Go software
typically has no concept of "versions" and every program pins its
dependencies at specific Git commits. So, to unbundle everything would
leave us with hundreds or thousands of versions of popular libraries.
We'll have to be creative in how we deal with that.

But for now, we try to unbundle.

Toggle quote (6 lines)
> If it's okay, this would make packaging lxd much simpler. I've had to
> package around 90 Go libraries so far.
>
> If using lxd's bundled copies of dependencies would be accepted, then I
> withdraw this patch request.

Please, either try packaging them "by hand" or with the Go module
importer [1]. Code reviewers will ask you to explain why you are using the
bundled dependencies.

[0] The work to overhaul go-build-system is progressing.

I guess it would be a command like:
`guix import go github.com/lcx/lxd --recursive -pin-versions`
But, it depends on LXD using the standard Go module mechanisms. If it
doesn't, maybe each top-level dependency can be imported along with its
dependency graph.
D
D
Dion Mendel wrote on 10 May 2021 04:58
(address . 48259@debbugs.gnu.org)
20210510025838.GA7449@dm9.info
On Sun, May 09, 2021 at 11:17:27AM -0400, Leo Famulari wrote:
Toggle quote (2 lines)
>But for now, we try to unbundle.

Okay, thank you for clarification.

When this changeset passes review and is applied, I'll continue with
packaging lxd and its dependencies.

(Thanks for pointing me towards guix import).
?