[PATCH 0/3] Add a service for OpenDHT

OpenSubmitted by Maxim Cournoyer.
Details
One participant
  • Maxim Cournoyer
Owner
unassigned
Severity
normal
M
M
Maxim Cournoyer wrote on 13 Apr 06:14 +0200
(address . guix-patches@gnu.org)(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
20210413041500.1223-1-maxim.cournoyer@gmail.com
This series overhaul our opendht package a bit and adds a service for it. Youcan now for example set up your own Jami proxy server easily to use with Jamion your phone. Or simply to strengthen the distributed network byparticipating in it and adding new stable bootstrap nodes.
Thanks,
Maxim Cournoyer (3): gnu: opendht: Add Python bindings, tools. services: configuration: Add syntactic sugar to easily generate documentation. services: Add opendht.
doc/guix.texi | 84 ++++++++++++++++++++ gnu/packages/networking.scm | 75 +++++++++++++++--- gnu/services/configuration.scm | 13 ++++ gnu/services/networking.scm | 135 ++++++++++++++++++++++++++++++++- 4 files changed, 297 insertions(+), 10 deletions(-)
-- 2.31.1
M
M
Maxim Cournoyer wrote on 13 Apr 06:17 +0200
[PATCH 1/3] gnu: opendht: Add Python bindings, tools.
(address . 47741@debbugs.gnu.org)(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
20210413041710.1708-1-maxim.cournoyer@gmail.com
* gnu/packages/networking.scm (opendht)[outputs]: Add tools and debug outputs.[native-inputs]: Add python and python-cython.[imported-modules, modules, phases]: New arguments.[configure-flags]: Disable static compilation. Enable Python support and thetools.[synopsis]: Update synopsis.[description]: Expound description.--- gnu/packages/networking.scm | 75 ++++++++++++++++++++++++++++++++----- 1 file changed, 66 insertions(+), 9 deletions(-)
Toggle diff (103 lines)diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scmindex baaa9c5707..b3a51c4b82 100644--- a/gnu/packages/networking.scm+++ b/gnu/packages/networking.scm@@ -3436,6 +3436,7 @@ and targeted primarily for asynchronous processing of HTTP-requests.") (base32 "1wc0f6cnvnlmhxnx64nxqgsx93k4g7ljdaqjl40ml74jg3nqrzcl")))) ;; Since 2.0, the gnu-build-system does not seem to work anymore, upstream bug?+ (outputs '("out" "tools" "debug")) (build-system cmake-build-system) (inputs `(("argon2" ,argon2)@@ -3452,23 +3453,79 @@ and targeted primarily for asynchronous processing of HTTP-requests.") `(("autoconf" ,autoconf) ("automake" ,automake) ("pkg-config" ,pkg-config)+ ("python" ,python)+ ("python-cython" ,python-cython) ("libtool" ,libtool) ("cppunit" ,cppunit))) (arguments- `(#:tests? #f ; Tests require network connection.+ `(#:imported-modules ((guix build python-build-system) ;for site-packages+ ,@%cmake-build-system-modules)+ #:modules (((guix build python-build-system) #:prefix python:)+ (guix build cmake-build-system)+ (guix build utils))+ #:tests? #f ; Tests require network connection. #:configure-flags- '(;;"-DOPENDHT_TESTS=on"- "-DOPENDHT_TOOLS=off"- "-DOPENDHT_PYTHON=off"+ '( ;;"-DOPENDHT_TESTS=on"+ "-DOPENDHT_STATIC=off"+ "-DOPENDHT_TOOLS=on"+ "-DOPENDHT_PYTHON=on" "-DOPENDHT_PROXY_SERVER=on" "-DOPENDHT_PUSH_NOTIFICATIONS=on" "-DOPENDHT_PROXY_SERVER_IDENTITY=on"- "-DOPENDHT_PROXY_CLIENT=on")))+ "-DOPENDHT_PROXY_CLIENT=on")+ #:phases+ (modify-phases %standard-phases+ (add-after 'unpack 'fix-python-installation-prefix+ ;; Specify the installation prefix for the compiled Python module+ ;; that would otherwise attempt to installs itself to Python's own+ ;; site-packages directory.+ (lambda* (#:key inputs outputs #:allow-other-keys)+ (substitute* "python/CMakeLists.txt"+ (("--root=\\\\\\$ENV\\{DESTDIR\\}")+ (string-append "--root=/ --single-version-externally-managed "+ "--prefix=${CMAKE_INSTALL_PREFIX}")))))+ (add-after 'unpack 'specify-runpath-for-python-module+ (lambda* (#:key outputs #:allow-other-keys)+ (let ((out (assoc-ref outputs "out")))+ (substitute* "python/setup.py.in"+ (("extra_link_args=\\[(.*)\\]" _ args)+ (string-append "extra_link_args=[" args+ ", '-Wl,-rpath=" out "/lib']"))))))+ (add-after 'install 'move-and-wrap-tools+ (lambda* (#:key inputs outputs #:allow-other-keys)+ (let ((out (assoc-ref outputs "out"))+ (tools (assoc-ref outputs "tools"))+ (site-packages (python:site-packages inputs outputs)))+ (mkdir tools)+ (rename-file (string-append out "/bin")+ (string-append tools "/bin"))+ (wrap-program (string-append tools "/bin/dhtcluster")+ `("PYTHONPATH" prefix (,site-packages))))))))) (home-page "https://github.com/savoirfairelinux/opendht/")- (synopsis "Distributed Hash Table (DHT) library")- (description "OpenDHT is a Distributed Hash Table (DHT) library. It may-be used to manage peer-to-peer network connections as needed for real time-communication.")+ (synopsis "Lightweight Distributed Hash Table (DHT) library")+ (description "OpenDHT provides an easy to use distributed in-memory data+store. Every node in the network can read and write values to the store.+Values are distributed over the network, with redundancy. It includes the+following features:+@itemize+@item Lightweight and scalable, designed for large networks and small devices;+@item High resilience to network disruption;+@item Public key cryptography layer providing optional data signature and+encryption (using GnuTLS);+@item IPv4 and IPv6 support;+@item Clean and powerful C++14 map API;+@item Bindings for C, Rust & Python 3;+@item REST API with an optional HTTP client and server with push notification+support.+@end itemize+The following tools are also included:+@table @command+@item dhtnode+A command line tool to run a DHT node and perform operations supported by the+library (get, put, etc.) with text values.+@item dhtchat+A very simple IM client working over the DHT.+@end table") (license license:gpl3+))) (define-public frrouting-- 2.31.1
M
M
Maxim Cournoyer wrote on 13 Apr 06:17 +0200
[PATCH 2/3] services: configuration: Add syntactic sugar to easily generate documentation.
(address . 47741@debbugs.gnu.org)(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
20210413041710.1708-2-maxim.cournoyer@gmail.com
I found the original (undocumented) interface difficult to understand, whichwas the rationale for adding a simpler one on top of it.
* gnu/services/configuration.scm (configuration->documentation): New procedure.--- gnu/services/configuration.scm | 13 +++++++++++++ 1 file changed, 13 insertions(+)
Toggle diff (30 lines)diff --git a/gnu/services/configuration.scm b/gnu/services/configuration.scmindex 90f12a8d39..750535342d 100644--- a/gnu/services/configuration.scm+++ b/gnu/services/configuration.scm@@ -43,6 +43,7 @@ define-configuration validate-configuration generate-documentation+ configuration->documentation serialize-package)) ;;; Commentary:@@ -188,3 +189,15 @@ (or (assq-ref sub-documentation field-name) '()))))) fields))))) (stexi->texi `(*fragment* . ,(generate documentation-name))))++(define (configuration->documentation configuration-symbol)+ "Takes CONFIGURATION-SYMBOL, the symbol used when defining a configuration+record with DEFINE-CONFIGURATION, and outputs the Texinfo documentation of its+fields."+ ;; This is syntax sugar for a simple, straight-forward application of+ ;; GENERATE-DOCUMENTATION.+ (let ((fields-getter (module-ref (current-module)+ (symbol-append configuration-symbol+ '-fields))))+ (format #t (generate-documentation `((,configuration-symbol ,fields-getter))+ configuration-symbol))))-- 2.31.1
M
M
Maxim Cournoyer wrote on 13 Apr 06:17 +0200
[PATCH 3/3] services: Add opendht.
(address . 47741@debbugs.gnu.org)(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
20210413041710.1708-3-maxim.cournoyer@gmail.com
* gnu/services/networking.scm (serialize-boolean)(serialize-number, serialize-string): New dummy procedures.(maybe-number, opendht-configuration): New syntaxes.(%opendht-accounts): New variable.(opendht-configuration->command-line-arguments): Likewise.(opendht-shepherd-service, opendht-service-type): New variables.* doc/guix.texi: Document it.--- doc/guix.texi | 84 ++++++++++++++++++++++ gnu/services/networking.scm | 135 +++++++++++++++++++++++++++++++++++- 2 files changed, 218 insertions(+), 1 deletion(-)
Toggle diff (257 lines)diff --git a/doc/guix.texi b/doc/guix.texiindex 84d8bf50be..d2593b2180 100644--- a/doc/guix.texi+++ b/doc/guix.texi@@ -16653,6 +16653,90 @@ must be @code{'()} or @code{'("internal")}. detailed discussion of each configuration field. @end deftp +@cindex opendht, distributed hash table network service+@cindex dhtproxy, for use with jami+@defvr {Scheme Variable} opendht-service-type+This is the type of the service running a @uref{https://opendht.net,+OpenDHT} node, @command{dhtnode}. The daemon can be used to host your+own proxy service to the DHT, for example to connect to with Jami, among+other applications.++The value of this service is a @code{opendht-configuration} object, as+described below.+@end defvr++@deftp {Data Type} opendht-configuration+This is the data type for the OpenDHT service configuration.++@c The fields documentation has been auto-generated using the+@c configuration->documentation procedure from (gnu services+@c configuration).+@c TODO: Tweak the tool so that it generates doc that matches our+@c current standard.+Available @code{opendht-configuration} fields are:++@deftypevr {@code{opendht-configuration} parameter} package opendht+The @code{opendht} package to use.++@end deftypevr++@deftypevr {@code{opendht-configuration} parameter} boolean peer-discovery?+Whether to enable the multicast local peer discovery mechanism.++Defaults to @samp{#f}.++@end deftypevr++@deftypevr {@code{opendht-configuration} parameter} boolean enable-logging?+Whether to enable logging messages to syslog. It is disabled by default+as it is rather verbose.++Defaults to @samp{#f}.++@end deftypevr++@deftypevr {@code{opendht-configuration} parameter} boolean debug?+Whether to enable debug-level logging messages. This has no effect if+logging is disabled.++Defaults to @samp{#f}.++@end deftypevr++@deftypevr {@code{opendht-configuration} parameter} maybe-string bootstrap-host+The node host name that is used to make the first connection to the+network. A specific port value can be provided by appending the+@code{:PORT} suffix. By default, it uses the Jami bootstrap nodes, but+any host can be specified here. It's also possible to disable+bootsrapping by setting this to the @code{'disabled} symbol.++Defaults to @samp{"bootstrap.jami.net:4222"}.++@end deftypevr++@deftypevr {@code{opendht-configuration} parameter} maybe-number port+The UDP port to bind to. When set to @code{'disabled}, an available+port is automatically selected.++Defaults to @samp{4222}.++@end deftypevr++@deftypevr {@code{opendht-configuration} parameter} maybe-number proxy-server-port+Spawn a proxy server listening on the specified port.++Defaults to @samp{disabled}.++@end deftypevr++@deftypevr {@code{opendht-configuration} parameter} maybe-number proxy-server-port-tls+Spawn a proxy server listening to TLS connections on the specified port.++Defaults to @samp{disabled}.++@end deftypevr+@end deftp+ @cindex Tor @defvr {Scheme Variable} tor-service-type This is the type for a service that runs the @uref{https://torproject.org,diff --git a/gnu/services/networking.scm b/gnu/services/networking.scmindex 231a9f66c7..8c22f0eec3 100644--- a/gnu/services/networking.scm+++ b/gnu/services/networking.scm@@ -10,7 +10,7 @@ ;;; Copyright © 2018 Chris Marusich <cmmarusich@gmail.com> ;;; Copyright © 2018 Arun Isaac <arunisaac@systemreboot.net> ;;; Copyright © 2019 Florian Pelz <pelzflorian@pelzflorian.de>-;;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer@gmail.com>+;;; Copyright © 2019, 2021 Maxim Cournoyer <maxim.cournoyer@gmail.com> ;;; Copyright © 2019 Sou Bunnbu <iyzsong@member.fsf.org> ;;; Copyright © 2019 Alex Griffin <a@ajgrf.com> ;;; Copyright © 2020 Brice Waegeneire <brice@waegenei.re>@@ -108,6 +108,18 @@ inetd-entry inetd-service-type + opendht-configuration+ opendht-configuration-peer-discovery?+ opendht-configuration-verbose?+ opendht-configuration-bootstrap-host+ opendht-configuration-port+ opendht-configuration-proxy-server-port+ opendht-configuration-proxy-server-port-tls+ opendht-configuration->command-line-arguments++ opendht-shepherd-service+ opendht-service-type+ tor-configuration tor-configuration? tor-hidden-service@@ -730,6 +742,127 @@ daemon will keep the system clock synchronized with that of the given servers.") for listening on Internet sockets and spawning the corresponding services on demand."))) + +;;;+;;; OpenDHT, the distributed hash table network used by Jami+;;;++;;; XXX: These dummy definitions is because there's no way to disable the+;;; serialization code from define-configuration.+(define (serialize-boolean option value) "")+(define (serialize-number option value) "")+(define (serialize-string option value) "")++(define-maybe number)+(define-maybe string)++;;; To generate the documentation of the following configuration record, you+;;; can evaluate: (configuration->documentation 'opendht-configuration)+(define-configuration opendht-configuration+ (opendht+ (package opendht)+ "The @code{opendht} package to use.")+ (peer-discovery?+ (boolean #false)+ "Whether to enable the multicast local peer discovery mechanism.")+ (enable-logging?+ (boolean #false)+ "Whether to enable logging messages to syslog. It is disabled by default+as it is rather verbose.")+ (debug?+ (boolean #false)+ "Whether to enable debug-level logging messages. This has no effect if+logging is disabled.")+ (bootstrap-host+ (maybe-string "bootstrap.jami.net:4222")+ "The node host name that is used to make the first connection to the+network. A specific port value can be provided by appending the @code{:PORT}+suffix. By default, it uses the Jami bootstrap nodes, but any host can be+specified here. It's also possible to disable bootstrapping by setting this+to the @code{'disabled} symbol.")+ (port+ (maybe-number 4222)+ "The UDP port to bind to. When set to @code{'disabled}, an available port+is automatically selected.")+ (proxy-server-port+ (maybe-number 'disabled)+ "Spawn a proxy server listening on the specified port.")+ (proxy-server-port-tls+ (maybe-number 'disabled)+ "Spawn a proxy server listening to TLS connections on the specified+port."))++(define %opendht-accounts+ ;; User account and groups for Tor.+ (list (user-group (name "opendht") (system? #t))+ (user-account+ (name "opendht")+ (group "opendht")+ (system? #t)+ (comment "OpenDHT daemon user")+ (home-directory "/var/empty")+ (shell (file-append shadow "/sbin/nologin")))))++(define (opendht-configuration->command-line-arguments config)+ "Derive the command line arguments to used from CONFIG, an+<opendht-configuration> object."+ (match-record config <opendht-configuration>+ (opendht bootstrap-host enable-logging? port debug? peer-discovery?+ proxy-server-port proxy-server-port-tls)+ (let ((dhtnode #~(string-append #$opendht:tools "/bin/dhtnode")))+ `(,dhtnode+ "--service" ;non-forking mode+ ,@(if (string? bootstrap-host)+ (list "--bootstrap" bootstrap-host))+ ,@(if enable-logging?+ (list "--syslog")+ '())+ ,@(if (number? port)+ (list "--port" (number->string port))+ '())+ ,@(if debug?+ (list "--verbose")+ '())+ ,@(if peer-discovery?+ (list "--peer-discovery")+ '())+ ,@(if (number? proxy-server-port)+ (list "--proxyserver" (number->string proxy-server-port))+ '())+ ,@(if (number? proxy-server-port-tls)+ (list "--proxyserverssl" (number->string proxy-server-port-tls))+ '())))))++(define (opendht-shepherd-service config)+ "Return a <shepherd-service> running OpenDHT."+ (shepherd-service+ (documentation "Run an OpenDHT node.")+ (provision '(opendht dhtnode dhtproxy))+ (requirement '(user-processes syslogd))+ (start #~(make-forkexec-constructor/container+ (list #$@(opendht-configuration->command-line-arguments config))+ #:mappings (list (file-system-mapping+ (source "/dev/log") ;for syslog+ (target source)))+ #:user "opendht"))+ (stop #~(make-kill-destructor))))++(define opendht-service-type+ (service-type+ (name 'opendht)+ (default-value (opendht-configuration))+ (extensions+ (list (service-extension shepherd-root-service-type+ (compose list opendht-shepherd-service))+ (service-extension account-service-type+ (const %opendht-accounts))))+ (description "Run the OpenDHT @command{dhtnode} command that allows+participating in the distributed hash table based OpenDHT network. The+service can be configured to act as a proxy to the distributed network, which+can be useful for portable devices where minimizing energy consumption is+paramount. OpenDHT was originally based on Kademlia and adapted for+applications in communication. It is used by Jami, for example.")))+ ;;; ;;; Tor.-- 2.31.1
?