[PATCH] Add Keybase

Open

Details

3 participants

Efraim Flashner
Jakub K?dzio?ka
Leo Famulari

Owner: unassigned

Submitted by: Jakub K?dzio?ka

Severity: normal

Jakub K?dzio?ka wrote on 7 Jan 2020 21:00

Recipients:(address . guix-patches@gnu.org)

Message-ID:20200107200039.jga75muq7excgpqu@zdrowyportier.kadziolka.net

Hi Guix,

please find attached a set of patches that adds the non-GUI parts of
Keybase (the GUI is an Electron application - as far as I am aware,
there are no similar packages in Guix, so packaging the GUI requires a
significant amount of further work).

Unresolved question: in other distributions, the package provides user
units for systemd. Should the Guix package provide Shepherd services to
be ran as the user? If so, are there any other examples I could look at
and draw inspiration from? Also, I haven't been able to find any
documentation on how to set up user shepherd, apart from a passing
mention of that being possible in the documentation. I would appreciate
any pointers on the topic.

Regards,
Jakub K?dzio?ka

From 0d744787c48c7184a70dda9fc1bb2c0d334fc080 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba@kadziolka.net>
Date: Sun, 5 Jan 2020 17:13:20 +0100
Subject: [PATCH 1/2] build-system/go: Allow providing additional build flags

* guix/build-system/go.scm (build-flags): New argument.
* guix/build/go-build-system.scm (build): Use apply to pass the
  additional arguments to invoke.
---
 guix/build-system/go.scm       | 3 +++
 guix/build/go-build-system.scm | 7 ++++---
 2 files changed, 7 insertions(+), 3 deletions(-)

Toggle diff (64 lines)diff --git a/guix/build-system/go.scm b/guix/build-system/go.scm
index 1b916af8f9..86f57079b1 100644
--- a/guix/build-system/go.scm
+++ b/guix/build-system/go.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Petter <petter@mykolab.ch>
 ;;; Copyright © 2017 Leo Famulari <leo@famulari.name>
+;;; Copyright © 2020 Jakub K?dzio?ka <kuba@kadziolka.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -82,6 +83,7 @@
                    (install-source? #t)
                    (import-path "")
                    (unpack-path "")
+                   (build-flags '())
                    (tests? #t)
                    (allow-go-reference? #f)
                    (system (%current-system))
@@ -109,6 +111,7 @@
                 #:install-source? ,install-source?
                 #:import-path ,import-path
                 #:unpack-path ,unpack-path
+                #:build-flags ,build-flags
                 #:tests? ,tests?
                 #:allow-go-reference? ,allow-go-reference?
                 #:inputs %build-inputs)))
diff --git a/guix/build/go-build-system.scm b/guix/build/go-build-system.scm
index 4bc0156a88..49887848f6 100644
--- a/guix/build/go-build-system.scm
+++ b/guix/build/go-build-system.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2016 Petter <petter@mykolab.ch>
 ;;; Copyright © 2017, 2019 Leo Famulari <leo@famulari.name>
 ;;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2020 Jakub K?dzio?ka <kuba@kadziolka.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -209,18 +210,18 @@ unpacking."
                 (_ #f))
               inputs))))
 
-(define* (build #:key import-path #:allow-other-keys)
+(define* (build #:key import-path build-flags #:allow-other-keys)
   "Build the package named by IMPORT-PATH."
   (with-throw-handler
     #t
     (lambda _
-      (invoke "go" "install"
+      (apply invoke "go" "install"
               "-v" ; print the name of packages as they are compiled
               "-x" ; print each command as it is invoked
               ;; Respectively, strip the symbol table and debug
               ;; information, and the DWARF symbol table.
               "-ldflags=-s -w"
-              import-path))
+              `(,@build-flags ,import-path)))
     (lambda (key . args)
       (display (string-append "Building '" import-path "' failed.\n"
                               "Here are the results of `go env`:\n"))
-- 
2.24.1

From 3de233a2d8e6bdb4723844337b69b6612616c9c5 Mon Sep 17 00:00:00 2001
From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba@kadziolka.net>
Date: Tue, 7 Jan 2020 20:29:21 +0100
Subject: [PATCH 2/2] gnu: Add keybase.

* gnu/packages/crypto.scm
  (keybase-component): New function.
  (keybase, git-remote-keybase, kbfs): New variables.
---
 gnu/packages/crypto.scm | 49 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

Toggle diff (73 lines)diff --git a/gnu/packages/crypto.scm b/gnu/packages/crypto.scm
index dd49c12e5b..c78515ae68 100644
--- a/gnu/packages/crypto.scm
+++ b/gnu/packages/crypto.scm
@@ -14,6 +14,7 @@
 ;;; Copyright © 2018 Tim Gesthuizen <tim.gesthuizen@yahoo.de>
 ;;; Copyright © 2019 Pierre Neidhardt <mail@ambrevar.xyz>
 ;;; Copyright © 2019 Tanguy Le Carrour <tanguy@bioneland.org>
+;;; Copyright © 2020 Jakub K?dzio?ka <kuba@kadziolka.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -69,6 +70,7 @@
   #:use-module (guix git-download)
   #:use-module (guix build-system cmake)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system perl)
   #:use-module (guix build utils)
   #:use-module (srfi srfi-1)
@@ -1040,3 +1042,50 @@ minisign uses a slightly different format to store secret keys.  Minisign
 signatures include trusted comments in addition to untrusted comments.
 Trusted comments are signed, thus verified, before being displayed.")
     (license license:isc)))
+
+(define* (keybase-component #:key name repo-path synopsis description)
+  (package
+    (name name)
+    (version "5.1.1")
+    (source (origin
+              (method url-fetch)
+              (uri
+                (string-append "https://github.com/keybase/client/releases/download/v"
+                               version "/keybase-v" version ".tar.xz"))
+              (sha256
+                (base32
+                  "1i6waaprdr89k3lrg36hn11f5z8j0n2iv6gpyrynbh2h128cfpwy"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path ,(string-append "github.com/keybase/client/go/" repo-path)
+       #:unpack-path "github.com/keybase/client"
+       #:build-flags '("-tags" "production")
+       #:install-source? #f))
+    (home-page "https://keybase.io")
+    (synopsis synopsis)
+    (description description)
+    (license license:bsd-3)))
+
+(define-public keybase
+  (keybase-component
+    #:name "keybase"
+    #:repo-path "keybase"
+    #:synopsis "Command-line client for Keybase"
+    #:description "Keybase is a key directory that maps social media identities
+to encryption keys in a publically auditable manner.  End-to-end encrypted chat,
+cloud storage and git hosting is also provided."))
+
+(define-public git-remote-keybase
+  (keybase-component
+    #:name "git-remote-keybase"
+    #:repo-path "kbfs/kbfsgit/git-remote-keybase"
+    #:synopsis "Git remote helper for Keybase encrypted repositories"
+    #:description "keybase-kbfsgit provides the git remote helper that handles
+@code{keybase://} URIs."))
+
+(define-public kbfs
+  (keybase-component
+    #:name "kbfs"
+    #:repo-path "kbfs/kbfsfuse"
+    #:synopsis "FUSE handler for the Keybase filesystem"
+    #:description "Handles mounting /keybase"))
-- 
2.24.1

Jakub K?dzio?ka wrote on 24 Jan 2020 19:34

go package rebuilds

Recipients:(address . 39021@debbugs.gnu.org)(address . me@tobias.gr)

Message-ID:20200124183428.iruqasedecsusbgw@zdrowyportier.kadziolka.net

With regards to the modification of go-build-system in this patchstack,

it seems that this would cause "only" <180 packages to be rebuilt:

~/guix$ rg -w go-build-system gnu/packages | wc -l

172

~/guix$ guix refresh --list-dependent go

Building the following 52 packages would ensure 176 dependent packages

are rebuilt: [...]

This has been discussed on IRC, and since some time has since passed, I

want to put the relevant link here, for future reference:

http://logs.guix.gnu.org/guix/2020-01-19.log#215122

Regards,

Jakub K?dzio?ka

Jakub K?dzio?ka wrote on 28 Jan 2020 22:54

[PATCH 1/2 v2] build-system/go: Allow providing additional build flags

Recipients:(address . 39021@debbugs.gnu.org)

Message-ID:20200128215405.b5nyh4tiewzxrjjq@zdrowyportier.kadziolka.net

* guix/build-system/go.scm (build-flags): New argument.
* guix/build/go-build-system.scm (build): Use apply to pass the
  additional arguments to invoke.
---
I have discovered a bug when testing v1 of this patch locally - some Go
packages have been failing to build due to a quoting mishap. This is now
fixed.

Diff between patches:
-                   (build-flags '())
+                   (build-flags ''())
[this note should be automatically removed by git am]
---
 guix/build-system/go.scm       | 3 +++
 guix/build/go-build-system.scm | 7 ++++---
 2 files changed, 7 insertions(+), 3 deletions(-)

Toggle diff (64 lines)diff --git a/guix/build-system/go.scm b/guix/build-system/go.scm
index 1b916af8f9..f8ebaefb27 100644
--- a/guix/build-system/go.scm
+++ b/guix/build-system/go.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2016 Petter <petter@mykolab.ch>
 ;;; Copyright © 2017 Leo Famulari <leo@famulari.name>
+;;; Copyright © 2020 Jakub K?dzio?ka <kuba@kadziolka.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -82,6 +83,7 @@
                    (install-source? #t)
                    (import-path "")
                    (unpack-path "")
+                   (build-flags ''())
                    (tests? #t)
                    (allow-go-reference? #f)
                    (system (%current-system))
@@ -109,6 +111,7 @@
                 #:install-source? ,install-source?
                 #:import-path ,import-path
                 #:unpack-path ,unpack-path
+                #:build-flags ,build-flags
                 #:tests? ,tests?
                 #:allow-go-reference? ,allow-go-reference?
                 #:inputs %build-inputs)))
diff --git a/guix/build/go-build-system.scm b/guix/build/go-build-system.scm
index 4bc0156a88..49887848f6 100644
--- a/guix/build/go-build-system.scm
+++ b/guix/build/go-build-system.scm
@@ -2,6 +2,7 @@
 ;;; Copyright © 2016 Petter <petter@mykolab.ch>
 ;;; Copyright © 2017, 2019 Leo Famulari <leo@famulari.name>
 ;;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer@gmail.com>
+;;; Copyright © 2020 Jakub K?dzio?ka <kuba@kadziolka.net>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -209,18 +210,18 @@ unpacking."
                 (_ #f))
               inputs))))
 
-(define* (build #:key import-path #:allow-other-keys)
+(define* (build #:key import-path build-flags #:allow-other-keys)
   "Build the package named by IMPORT-PATH."
   (with-throw-handler
     #t
     (lambda _
-      (invoke "go" "install"
+      (apply invoke "go" "install"
               "-v" ; print the name of packages as they are compiled
               "-x" ; print each command as it is invoked
               ;; Respectively, strip the symbol table and debug
               ;; information, and the DWARF symbol table.
               "-ldflags=-s -w"
-              import-path))
+              `(,@build-flags ,import-path)))
     (lambda (key . args)
       (display (string-append "Building '" import-path "' failed.\n"
                               "Here are the results of `go env`:\n"))
-- 
2.25.0

Leo Famulari wrote on 8 Feb 2020 01:20

Re: [bug#39021] [PATCH] Add Keybase

Recipients:(name . Jakub K?dzio?ka)(address . kuba@kadziolka.net)(address . 39021@debbugs.gnu.org)

Message-ID:20200208002020.GD12194@jasmine.lan

On Tue, Jan 07, 2020 at 09:00:39PM +0100, Jakub K?dzio?ka wrote:

Toggle quote (5 lines)

> please find attached a set of patches that adds the non-GUI parts of

> Keybase (the GUI is an Electron application - as far as I am aware,

> there are no similar packages in Guix, so packaging the GUI requires a

> significant amount of further work).

I don't *think* we have any Electron packages, but we do have Chromium

(and qtwebengine) and Node.js, so I hope it's possible.

Toggle quote (8 lines)

> Unresolved question: in other distributions, the package provides user

> units for systemd. Should the Guix package provide Shepherd services to

> be ran as the user? If so, are there any other examples I could look at

> and draw inspiration from? Also, I haven't been able to find any

> documentation on how to set up user shepherd, apart from a passing

> mention of that being possible in the documentation. I would appreciate

> any pointers on the topic.

We don't really do this kind of thing — adding features or service

manager files to upstream packages.

As for user shepherd, there's an example here:

https://git.dthompson.us/dotfiles.git/blob/HEAD:/dotfiles/.config/shepherd/init.scm

There's a lot going on there but the important parts are

(register-services ...), (make <service> ...), (action ...), and

(for-each start ...)

Then you just invoke `shepherd`.

Toggle quote (9 lines)

> From 0d744787c48c7184a70dda9fc1bb2c0d334fc080 Mon Sep 17 00:00:00 2001

> From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba@kadziolka.net>

> Date: Sun, 5 Jan 2020 17:13:20 +0100

> Subject: [PATCH 1/2] build-system/go: Allow providing additional build flags

> * guix/build-system/go.scm (build-flags): New argument.

> * guix/build/go-build-system.scm (build): Use apply to pass the

> additional arguments to invoke.

Nice, LGTM!

Toggle quote (9 lines)

> From 3de233a2d8e6bdb4723844337b69b6612616c9c5 Mon Sep 17 00:00:00 2001

> From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba@kadziolka.net>

> Date: Tue, 7 Jan 2020 20:29:21 +0100

> Subject: [PATCH 2/2] gnu: Add keybase.

> * gnu/packages/crypto.scm

> (keybase-component): New function.

> (keybase, git-remote-keybase, kbfs): New variables.

This is enough of it's own thing that we can make a new (gnu packages

keybase) module.

Toggle quote (2 lines)

> +(define* (keybase-component #:key name repo-path synopsis description)

We avoid abbreviations, so maybe "repository-path"? Bonus points if we

can make it more descriptive.

Can you take a look at the bundled ("vendored") dependencies:

https://github.com/keybase/client/tree/master/go/vendor

We strive to avoid using these, but sometimes we do, as in the Docker

package. It's not really idiomatic to unbundle things in Go. But we need

to at least make sure all the bundled dependencies are freely licensed.

Also, please run `guix lint` on these packages and make sure the

descriptions are written in complete sentences.

Jakub K?dzio?ka wrote on 11 Feb 2020 17:36

Recipients:(name . Leo Famulari)(address . leo@famulari.name)(address . 39021@debbugs.gnu.org)

Message-ID:20200211163654.v5jz5bf7audo7unh@gravity

Toggle quote (12 lines)> > From 3de233a2d8e6bdb4723844337b69b6612616c9c5 Mon Sep 17 00:00:00 2001
> > From: =?UTF-8?q?Jakub=20K=C4=85dzio=C5=82ka?= <kuba@kadziolka.net>
> > Date: Tue, 7 Jan 2020 20:29:21 +0100
> > Subject: [PATCH 2/2] gnu: Add keybase.
> > 
> > * gnu/packages/crypto.scm
> >   (keybase-component): New function.
> >   (keybase, git-remote-keybase, kbfs): New variables.
> 
> This is enough of it's own thing that we can make a new (gnu packages
> keybase) module.

Sure, will do.

Toggle quote (5 lines)

> > +(define* (keybase-component #:key name repo-path synopsis description)

> We avoid abbreviations, so maybe "repository-path"? Bonus points if we

> can make it more descriptive.

I can't think of anything more descriptive, as it's literally the path

in the repository the component is at.

Toggle quote (8 lines)

> Can you take a look at the bundled ("vendored") dependencies:

> https://github.com/keybase/client/tree/master/go/vendor

> We strive to avoid using these, but sometimes we do, as in the Docker

> package. It's not really idiomatic to unbundle things in Go. But we need

> to at least make sure all the bundled dependencies are freely licensed.

Apart from licensing concerns, what are the arguments for splitting this

into separate packages? I feel like this is just busywork...

Toggle quote (3 lines)

> Also, please run `guix lint` on these packages and make sure the

> descriptions are written in complete sentences.

Ah, sure, somehow I forgot to do this before.

Leo Famulari wrote on 11 Feb 2020 18:36

Recipients:(name . Jakub K?dzio?ka)(address . kuba@kadziolka.net)(address . 39021@debbugs.gnu.org)

Message-ID:20200211173634.GB9442@jasmine.lan

On Tue, Feb 11, 2020 at 05:36:54PM +0100, Jakub K?dzio?ka wrote:

Toggle quote (7 lines)

> > We strive to avoid using these, but sometimes we do, as in the Docker

> > package. It's not really idiomatic to unbundle things in Go. But we need

> > to at least make sure all the bundled dependencies are freely licensed.

> Apart from licensing concerns, what are the arguments for splitting this

> into separate packages? I feel like this is just busywork...

The question of licensing is unrelated to bundling, sorry if that wasn't
clear. The only thing you have to do here is make sure they are all
freely licensed.

To clarify, those bundled dependencies *are* separate packages,
developed by different organizations.

It's the standard in Guix (and every major GNU/Linux distro) to not
allow bundled dependencies because they make the graph of software
basically uninspectable and unmaintainable using the distro's normal
tools, as well as having the potential to waste time and space building
multiple versions of a package if it is bundled in more than one place
or already present as its own package. It negates all the advantages of
creating a distrubtion, especially for Go binaries, which can be
trivially deployed on any system, including Guix, without any extra
work.

But like I said, it's normal to bundle things in Go land, where there is
really no principled concept of dependency management or versioned
releases, and as time goes by changes to the Go compiler make it harder
and harder to unbundle. I did do it for Syncthing and I can confirm it
was a lot of work for no clear benefit. Excepting the standard library,
Go libraries do not even get security updates because nobody is looking
closely at them.

Efraim Flashner wrote on 31 May 2020 09:09

Recipients:(name . Leo Famulari)(address . leo@famulari.name)

Message-ID:20200531070938.GC7397@E5400

I have a working version of keybase, although I haven't gotten around to

unbundling all the go dependencies yet. I figured I'd post my WIP stuff

here.

Efraim Flashner <efraim@flashner.co.il> ????? ?????

GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351

Confidentiality cannot be guaranteed on emails sent or received unencrypted

Attachment: keybase.scm

;; https://github.com/keybase/client/blob/master/packaging/linux/systemd/keybase.service (define keybase (make <service> #:provides '(keybase) #:docstring "Provide access to Keybase™" #:start (make-forkexec-constructor '("keybase" "service") #:log-file (string-append (getenv "HOME") "/log/keybase.log")) #:stop (make-system-destructor "keybase ctl stop") #:actions (make-actions (restart (lambda _ (system* "keybase" "ctl" "restart"))) (reload (lambda _ (system* "keybase" "ctl" "reload")))) #:directory (string-append "/run/user/" (number->string (passwd:uid (getpw "efraim"))) "/keybase") #:respawn? #t)) ;; https://github.com/keybase/client/blob/master/packaging/linux/systemd/kbfs.service (define kbfs (make <service> #:provides '(kbfs) #:depends '(keybase) #:docstring "Provide access to Keybase™ fuse store" #:start (make-forkexec-constructor '("kbfsfuse" "-debug" "-log-to-file") #:log-file (string-append (getenv "HOME") "/log/kbfs.log")) #:stop (make-kill-destructor) #:respawn? #t)) (register-services keybase kbfs)

Jakub K?dzio?ka wrote on 31 May 2020 23:47

Recipients:(name . Efraim Flashner)(address . efraim@flashner.co.il)

Message-ID:20200531214738.vakvlerjdznt34on@gravity

On Sun, May 31, 2020 at 10:09:38AM +0300, Efraim Flashner wrote:

Toggle quote (4 lines)

> I have a working version of keybase, although I haven't gotten around to

> unbundling all the go dependencies yet. I figured I'd post my WIP stuff

> here.

Hi Efraim,

thanks for your message! I like how you consolidated the binaries into a

single package with a custom phase. I was going to ask whether you got

the GUI working, but after taking another look at the unbundling part,

it's clear that this is still not there.

When it comes to my own efforts for packaging keybase, I have

categorized all the vendored dependencies according to their licenses.

I have found 3 libraries without a license. For 2 of them, I prodded the

authors a bit and they now include a license. However, I haven't

received a response on github.com/keybase/golang-ico (note that despite

the /keybase/ in the URL, this is just a fork, the original author has

no relation to keybase).

I am considering learning enough Go to replace that library with a

freely-licensed alternative and submitting a PR upstream, however it is

quite low on my TODOs.

Regards,

Jakub K?dzio?ka

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 39021@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 39021

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date