[PATCH 0/2] Add Elasticsearch package and service.

Christopher Baines wrote on 13 Mar 2018 20:12

Recipients:(address . guix-patches@gnu.org)

Message-ID:871sgn23ip.fsf@cbaines.net

Tags: moreinfo

I'm trying to reduce the number of patches I have sitting around in git

branches, so here is a bug about Elasticsearch.

I've had a really awful package definition for this for a while, which

just makes the built things in the release tarball run. While this isn't

suitable for inclusion in to Guix, it does still work.

Also included is a service, along with a really simple test.

Christopher Baines (2):

gnu: Add elasticsearch.

services: Add elasticsearch.

gnu/packages/databases.scm | 64 +++++++++++++++++++++++++++

gnu/services/databases.scm | 107 +++++++++++++++++++++++++++++++++++++++++++++

gnu/tests/databases.scm | 57 +++++++++++++++++++++++-

3 files changed, 227 insertions(+), 1 deletion(-)

Christopher Baines wrote on 13 Mar 2018 20:17

[PATCH 2/2] services: Add elasticsearch.

Recipients:(address . 30803@debbugs.gnu.org)

Message-ID:20180313191727.24508-2-mail@cbaines.net

---

gnu/services/databases.scm | 107 +++++++++++++++++++++++++++++++++++++++++++++

gnu/tests/databases.scm | 57 +++++++++++++++++++++++-

2 files changed, 163 insertions(+), 1 deletion(-)

Toggle diff (199 lines)diff --git a/gnu/services/databases.scm b/gnu/services/databases.scm
index 72927c453..cbb9e1699 100644
--- a/gnu/services/databases.scm
+++ b/gnu/services/databases.scm
@@ -51,6 +51,18 @@
             postgresql-service
             postgresql-service-type
 
+            <elasticsearch-configuration>
+            elasticsearch-configuration
+            elasticsearch-configuration?
+            elasticsearch-configuration-elasticsearch
+            elasticsearch-configuration-data-path
+            elasticsearch-configuration-logs-path
+            elasticsearch-configuration-port
+            elasticsearch-configuration-transport-port
+
+            elasticsearch-service
+            elasticsearch-service-type
+
             memcached-service-type
             <memcached-configuration>
             memcached-configuration
@@ -260,6 +272,101 @@ and stores the database cluster in @var{data-directory}."
             (data-directory data-directory))))
 
 
+;;;
+;;; Elasticsearch
+;;;
+
+(define-record-type* <elasticsearch-configuration>
+  elasticsearch-configuration make-elasticsearch-configuration
+  elasticsearch-configuration?
+  (elasticsearch     elasticsearch-configuration-elasticsearch
+                     (default elasticsearch))
+  (data-path         elasticsearch-configuration-data-path
+                     (default "/var/lib/"))
+  (logs-path         elasticsearch-configuration-logs-path
+                     (default "/var/log/elasticsearch"))
+  (http-port         elasticsearch-configuration-port
+                     (default 9200))
+  (transport-port    elasticsearch-configuration-transport-port
+                     (default 9300)))
+
+(define (elasticsearch-configuration-directory
+         data-path logs-path http-port transport-port)
+  (computed-file
+   "elasticsearch-config"
+   #~(begin
+       (mkdir #$output)
+       (mkdir (string-append #$output "/scripts"))
+       (call-with-output-file (string-append #$output "/elasticsearch.yml")
+         (lambda (port)
+           (display
+            (string-append
+             "path.data: " #$data-path "\n"
+             "path.logs: " #$logs-path "\n"
+             "http.port: " #$(number->string http-port) "\n"
+             "transport.tcp.port: " #$(number->string transport-port) "\n")
+            port))))))
+
+(define %elasticsearch-accounts
+  (list (user-group (name "elasticsearch") (system? #t))
+        (user-account
+         (name "elasticsearch")
+         (group "elasticsearch")
+         (system? #t)
+         (comment "Elasticsearch server user")
+         (home-directory "/var/empty")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define elasticsearch-activation
+  (match-lambda
+    (($ <elasticsearch-configuration> elasticsearch data-path logs-path
+                                      http-port transport-port)
+     #~(begin
+         (use-modules (guix build utils)
+                      (ice-9 match))
+
+         (let ((user (getpwnam "elasticsearch")))
+           ;; Create db state directory.
+           (for-each
+            (lambda (path)
+              (mkdir-p path)
+              (chown path (passwd:uid user) (passwd:gid user)))
+            '(#$data-path #$logs-path "/var/run/elasticsearch")))))))
+
+(define elasticsearch-shepherd-service
+  (match-lambda
+    (($ <elasticsearch-configuration> elasticsearch data-path logs-path
+                                      http-port transport-port)
+     (list (shepherd-service
+            (provision '(elasticsearch))
+            (documentation "Run the Elasticsearch daemon.")
+            (requirement '(user-processes syslogd))
+            (start #~(make-forkexec-constructor
+                      (list
+                       (string-append #$elasticsearch "/bin/elasticsearch")
+                       "-d"
+                       "-p" "/var/run/elasticsearch/pid"
+                       (string-append
+                        "-Dpath.conf="
+                        #$(elasticsearch-configuration-directory
+                           data-path logs-path http-port transport-port)))
+                      #:user "elasticsearch"
+                      #:pid-file "/var/run/elasticsearch/pid"
+                      #:log-file "/var/log/elasticsearch.log"))
+            (stop #~(make-kill-destructor)))))))
+
+(define elasticsearch-service-type
+  (service-type (name 'elasticsearch)
+                (extensions
+                 (list (service-extension shepherd-root-service-type
+                                          elasticsearch-shepherd-service)
+                       (service-extension activation-service-type
+                                          elasticsearch-activation)
+                       (service-extension account-service-type
+                                          (const %elasticsearch-accounts))))
+                (default-value (elasticsearch-configuration))))
+
+
 ;;;
 ;;; Memcached
 ;;;
diff --git a/gnu/tests/databases.scm b/gnu/tests/databases.scm
index 5c8ca85c1..e07a1f9dd 100644
--- a/gnu/tests/databases.scm
+++ b/gnu/tests/databases.scm
@@ -31,7 +31,8 @@
   #:export (%test-memcached
             %test-mongodb
             %test-postgresql
-            %test-mysql))
+            %test-mysql
+            %test-elasticsearch))
 
 (define %memcached-os
   (simple-operating-system
@@ -319,3 +320,57 @@
    (name "mysql")
    (description "Start the MySQL service.")
    (value (run-mysql-test))))
+
+
+;;;
+;;; The Elasticsearch service.
+;;;
+
+(define %elasticsearch-os
+  (simple-operating-system
+   (service elasticsearch-service-type)))
+
+(define (run-elasticsearch-test)
+  "Run tests in %ELASTICSEARCH-OS."
+  (define os
+    (marionette-operating-system
+     %elasticsearch-os
+     #:imported-modules '((gnu services herd)
+                          (guix combinators))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     (memory-size 512)))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-64)
+                       (gnu build marionette))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin "elasticsearch")
+
+          (test-assert "service running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (start-service 'elasticsearch))
+             marionette))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation "elasticsearch-test" test))
+
+(define %test-elasticsearch
+  (system-test
+   (name "elasticsearch")
+   (description "Start the Elasticsearch service.")
+   (value (run-elasticsearch-test))))
-- 
2.16.2

Christopher Baines wrote on 13 Mar 2018 20:17

[PATCH 1/2] gnu: Add elasticsearch.

Recipients:(address . 30803@debbugs.gnu.org)

Message-ID:20180313191727.24508-1-mail@cbaines.net

* gnu/packages/databases.scm (elasticsearch-2.4.6, elasticsearch): New
  variables.
---
 gnu/packages/databases.scm | 64 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

Toggle diff (84 lines)diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 87f65404f..661ee3a3e 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -69,6 +69,7 @@
   #:use-module (gnu packages guile)
   #:use-module (gnu packages time)
   #:use-module (gnu packages golang)
+  #:use-module (gnu packages java)
   #:use-module (gnu packages jemalloc)
   #:use-module (gnu packages language)
   #:use-module (gnu packages libevent)
@@ -355,6 +356,69 @@ ElasticSearch server")
     (home-page "https://github.com/patientslikeme/es_dump_restore")
     (license license:expat)))
 
+(define-public elasticsearch-2.4.6
+  (package
+   (name "elasticsearch")
+   (version "2.4.6")
+   (source
+    (origin
+     (method url-fetch)
+     (uri (string-append
+           "https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-"
+           version ".tar.gz"))
+     (sha256
+      (base32 "0vzw9kpyyyv3f1m5sy9zara6shc7xkgi5xm5qbzvfywijavlnzjz"))))
+   (build-system gnu-build-system)
+   (inputs
+    `(("jre" ,icedtea)
+      ("coreutils" ,coreutils)
+      ("inetutils" ,inetutils)
+      ("util-linux" ,util-linux)
+      ("grep" ,grep)))
+   (arguments
+    `(#:phases
+      (modify-phases %standard-phases
+        (delete 'check)
+        (delete 'configure)
+        (delete 'build)
+        (replace 'install
+          (lambda* (#:key inputs outputs #:allow-other-keys)
+            (let ((out (assoc-ref outputs "out")))
+              (for-each
+               (lambda (dir)
+                 (copy-recursively dir (string-append out "/" dir)
+                                   #:log (%make-void-port "w")))
+               '("bin" "config" "lib" "modules"))
+              (for-each
+               (lambda (dir)
+                 (mkdir (string-append out "/" dir)))
+               '("plugins"))
+              (for-each
+               delete-file
+               (find-files
+                (string-append out "/lib")
+                (lambda (name stat)
+                  (or (string-contains name "freebsd")
+                      (string-contains name "solaris")))))
+              (wrap-program
+                  (string-append out "/bin/elasticsearch")
+                `("PATH" = (,(string-append (assoc-ref inputs "util-linux")
+                                            "/bin")
+                            ,(string-append (assoc-ref inputs "coreutils")
+                                            "/bin")
+                            ,(string-append (assoc-ref inputs "inetutils")
+                                            "/bin")
+                            ,(string-append (assoc-ref inputs "grep")
+                                            "/bin")))
+                `("JAVA_HOME" = (,(assoc-ref inputs "jre"))))
+              #t))))))
+   (home-page "")
+   (synopsis "")
+   (description "")
+   (license "")))
+
+(define-public elasticsearch elasticsearch-2.4.6)
+
 (define-public leveldb
   (package
     (name "leveldb")
-- 
2.16.2

Ludovic Courtès wrote on 17 Mar 2018 22:06

Re: [bug#30803] [PATCH 0/2] Add Elasticsearch package and service.

Recipients:(name . Christopher Baines)(address . mail@cbaines.net)(address . 30803@debbugs.gnu.org)

Message-ID:874lleo1j1.fsf@gnu.org

Hello Christopher,

Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (3 lines)

> I'm trying to reduce the number of patches I have sitting around in git

> branches, so here is a bug about Elasticsearch.

Cool, thanks for sharing!

Toggle quote (4 lines)

> I've had a really awful package definition for this for a while, which

> just makes the built things in the release tarball run. While this isn't

> suitable for inclusion in to Guix, it does still work.

Do you know if we’re missing a lot of the Java dependencies to build it

from source?

Ludo’.

Christopher Baines wrote on 17 Mar 2018 23:36

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 30803@debbugs.gnu.org)

Message-ID:87woyaxrcc.fsf@cbaines.net

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (16 lines)> Hello Christopher,
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> I'm trying to reduce the number of patches I have sitting around in git
>> branches, so here is a bug about Elasticsearch.
>
> Cool, thanks for sharing!
>
>> I've had a really awful package definition for this for a while, which
>> just makes the built things in the release tarball run. While this isn't
>> suitable for inclusion in to Guix, it does still work.
>
> Do you know if we’re missing a lot of the Java dependencies to build it
> from source?

Unfortunately not, I haven't attempted to build it from source.

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date