[PATCH 0/2] Add Elasticsearch package and service.

Done

Details

4 participants

Ludovic Courtès
Christopher Baines
Maxim Cournoyer
zimoun

Owner: unassigned

Submitted by: Christopher Baines

Severity: normal

Christopher Baines wrote on 13 Mar 2018 20:12

Recipients:(address . guix-patches@gnu.org)

Message-ID:871sgn23ip.fsf@cbaines.net

Tags: moreinfo

I'm trying to reduce the number of patches I have sitting around in git

branches, so here is a bug about Elasticsearch.

I've had a really awful package definition for this for a while, which

just makes the built things in the release tarball run. While this isn't

suitable for inclusion in to Guix, it does still work.

Also included is a service, along with a really simple test.

Christopher Baines (2):

gnu: Add elasticsearch.

services: Add elasticsearch.

gnu/packages/databases.scm | 64 +++++++++++++++++++++++++++

gnu/services/databases.scm | 107 +++++++++++++++++++++++++++++++++++++++++++++

gnu/tests/databases.scm | 57 +++++++++++++++++++++++-

3 files changed, 227 insertions(+), 1 deletion(-)

Christopher Baines wrote on 13 Mar 2018 20:17

[PATCH 2/2] services: Add elasticsearch.

Recipients:(address . 30803@debbugs.gnu.org)

Message-ID:20180313191727.24508-2-mail@cbaines.net

---

gnu/services/databases.scm | 107 +++++++++++++++++++++++++++++++++++++++++++++

gnu/tests/databases.scm | 57 +++++++++++++++++++++++-

2 files changed, 163 insertions(+), 1 deletion(-)

Toggle diff (199 lines)diff --git a/gnu/services/databases.scm b/gnu/services/databases.scm
index 72927c453..cbb9e1699 100644
--- a/gnu/services/databases.scm
+++ b/gnu/services/databases.scm
@@ -51,6 +51,18 @@
             postgresql-service
             postgresql-service-type
 
+            <elasticsearch-configuration>
+            elasticsearch-configuration
+            elasticsearch-configuration?
+            elasticsearch-configuration-elasticsearch
+            elasticsearch-configuration-data-path
+            elasticsearch-configuration-logs-path
+            elasticsearch-configuration-port
+            elasticsearch-configuration-transport-port
+
+            elasticsearch-service
+            elasticsearch-service-type
+
             memcached-service-type
             <memcached-configuration>
             memcached-configuration
@@ -260,6 +272,101 @@ and stores the database cluster in @var{data-directory}."
             (data-directory data-directory))))
 
 
+;;;
+;;; Elasticsearch
+;;;
+
+(define-record-type* <elasticsearch-configuration>
+  elasticsearch-configuration make-elasticsearch-configuration
+  elasticsearch-configuration?
+  (elasticsearch     elasticsearch-configuration-elasticsearch
+                     (default elasticsearch))
+  (data-path         elasticsearch-configuration-data-path
+                     (default "/var/lib/"))
+  (logs-path         elasticsearch-configuration-logs-path
+                     (default "/var/log/elasticsearch"))
+  (http-port         elasticsearch-configuration-port
+                     (default 9200))
+  (transport-port    elasticsearch-configuration-transport-port
+                     (default 9300)))
+
+(define (elasticsearch-configuration-directory
+         data-path logs-path http-port transport-port)
+  (computed-file
+   "elasticsearch-config"
+   #~(begin
+       (mkdir #$output)
+       (mkdir (string-append #$output "/scripts"))
+       (call-with-output-file (string-append #$output "/elasticsearch.yml")
+         (lambda (port)
+           (display
+            (string-append
+             "path.data: " #$data-path "\n"
+             "path.logs: " #$logs-path "\n"
+             "http.port: " #$(number->string http-port) "\n"
+             "transport.tcp.port: " #$(number->string transport-port) "\n")
+            port))))))
+
+(define %elasticsearch-accounts
+  (list (user-group (name "elasticsearch") (system? #t))
+        (user-account
+         (name "elasticsearch")
+         (group "elasticsearch")
+         (system? #t)
+         (comment "Elasticsearch server user")
+         (home-directory "/var/empty")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define elasticsearch-activation
+  (match-lambda
+    (($ <elasticsearch-configuration> elasticsearch data-path logs-path
+                                      http-port transport-port)
+     #~(begin
+         (use-modules (guix build utils)
+                      (ice-9 match))
+
+         (let ((user (getpwnam "elasticsearch")))
+           ;; Create db state directory.
+           (for-each
+            (lambda (path)
+              (mkdir-p path)
+              (chown path (passwd:uid user) (passwd:gid user)))
+            '(#$data-path #$logs-path "/var/run/elasticsearch")))))))
+
+(define elasticsearch-shepherd-service
+  (match-lambda
+    (($ <elasticsearch-configuration> elasticsearch data-path logs-path
+                                      http-port transport-port)
+     (list (shepherd-service
+            (provision '(elasticsearch))
+            (documentation "Run the Elasticsearch daemon.")
+            (requirement '(user-processes syslogd))
+            (start #~(make-forkexec-constructor
+                      (list
+                       (string-append #$elasticsearch "/bin/elasticsearch")
+                       "-d"
+                       "-p" "/var/run/elasticsearch/pid"
+                       (string-append
+                        "-Dpath.conf="
+                        #$(elasticsearch-configuration-directory
+                           data-path logs-path http-port transport-port)))
+                      #:user "elasticsearch"
+                      #:pid-file "/var/run/elasticsearch/pid"
+                      #:log-file "/var/log/elasticsearch.log"))
+            (stop #~(make-kill-destructor)))))))
+
+(define elasticsearch-service-type
+  (service-type (name 'elasticsearch)
+                (extensions
+                 (list (service-extension shepherd-root-service-type
+                                          elasticsearch-shepherd-service)
+                       (service-extension activation-service-type
+                                          elasticsearch-activation)
+                       (service-extension account-service-type
+                                          (const %elasticsearch-accounts))))
+                (default-value (elasticsearch-configuration))))
+
+
 ;;;
 ;;; Memcached
 ;;;
diff --git a/gnu/tests/databases.scm b/gnu/tests/databases.scm
index 5c8ca85c1..e07a1f9dd 100644
--- a/gnu/tests/databases.scm
+++ b/gnu/tests/databases.scm
@@ -31,7 +31,8 @@
   #:export (%test-memcached
             %test-mongodb
             %test-postgresql
-            %test-mysql))
+            %test-mysql
+            %test-elasticsearch))
 
 (define %memcached-os
   (simple-operating-system
@@ -319,3 +320,57 @@
    (name "mysql")
    (description "Start the MySQL service.")
    (value (run-mysql-test))))
+
+
+;;;
+;;; The Elasticsearch service.
+;;;
+
+(define %elasticsearch-os
+  (simple-operating-system
+   (service elasticsearch-service-type)))
+
+(define (run-elasticsearch-test)
+  "Run tests in %ELASTICSEARCH-OS."
+  (define os
+    (marionette-operating-system
+     %elasticsearch-os
+     #:imported-modules '((gnu services herd)
+                          (guix combinators))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     (memory-size 512)))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-64)
+                       (gnu build marionette))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin "elasticsearch")
+
+          (test-assert "service running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (start-service 'elasticsearch))
+             marionette))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation "elasticsearch-test" test))
+
+(define %test-elasticsearch
+  (system-test
+   (name "elasticsearch")
+   (description "Start the Elasticsearch service.")
+   (value (run-elasticsearch-test))))
-- 
2.16.2

Christopher Baines wrote on 13 Mar 2018 20:17

[PATCH 1/2] gnu: Add elasticsearch.

Recipients:(address . 30803@debbugs.gnu.org)

Message-ID:20180313191727.24508-1-mail@cbaines.net

* gnu/packages/databases.scm (elasticsearch-2.4.6, elasticsearch): New
  variables.
---
 gnu/packages/databases.scm | 64 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 64 insertions(+)

Toggle diff (84 lines)diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 87f65404f..661ee3a3e 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -69,6 +69,7 @@
   #:use-module (gnu packages guile)
   #:use-module (gnu packages time)
   #:use-module (gnu packages golang)
+  #:use-module (gnu packages java)
   #:use-module (gnu packages jemalloc)
   #:use-module (gnu packages language)
   #:use-module (gnu packages libevent)
@@ -355,6 +356,69 @@ ElasticSearch server")
     (home-page "https://github.com/patientslikeme/es_dump_restore")
     (license license:expat)))
 
+(define-public elasticsearch-2.4.6
+  (package
+   (name "elasticsearch")
+   (version "2.4.6")
+   (source
+    (origin
+     (method url-fetch)
+     (uri (string-append
+           "https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-"
+           version ".tar.gz"))
+     (sha256
+      (base32 "0vzw9kpyyyv3f1m5sy9zara6shc7xkgi5xm5qbzvfywijavlnzjz"))))
+   (build-system gnu-build-system)
+   (inputs
+    `(("jre" ,icedtea)
+      ("coreutils" ,coreutils)
+      ("inetutils" ,inetutils)
+      ("util-linux" ,util-linux)
+      ("grep" ,grep)))
+   (arguments
+    `(#:phases
+      (modify-phases %standard-phases
+        (delete 'check)
+        (delete 'configure)
+        (delete 'build)
+        (replace 'install
+          (lambda* (#:key inputs outputs #:allow-other-keys)
+            (let ((out (assoc-ref outputs "out")))
+              (for-each
+               (lambda (dir)
+                 (copy-recursively dir (string-append out "/" dir)
+                                   #:log (%make-void-port "w")))
+               '("bin" "config" "lib" "modules"))
+              (for-each
+               (lambda (dir)
+                 (mkdir (string-append out "/" dir)))
+               '("plugins"))
+              (for-each
+               delete-file
+               (find-files
+                (string-append out "/lib")
+                (lambda (name stat)
+                  (or (string-contains name "freebsd")
+                      (string-contains name "solaris")))))
+              (wrap-program
+                  (string-append out "/bin/elasticsearch")
+                `("PATH" = (,(string-append (assoc-ref inputs "util-linux")
+                                            "/bin")
+                            ,(string-append (assoc-ref inputs "coreutils")
+                                            "/bin")
+                            ,(string-append (assoc-ref inputs "inetutils")
+                                            "/bin")
+                            ,(string-append (assoc-ref inputs "grep")
+                                            "/bin")))
+                `("JAVA_HOME" = (,(assoc-ref inputs "jre"))))
+              #t))))))
+   (home-page "")
+   (synopsis "")
+   (description "")
+   (license "")))
+
+(define-public elasticsearch elasticsearch-2.4.6)
+
 (define-public leveldb
   (package
     (name "leveldb")
-- 
2.16.2

Ludovic Courtès wrote on 17 Mar 2018 22:06

Re: [bug#30803] [PATCH 0/2] Add Elasticsearch package and service.

Recipients:(name . Christopher Baines)(address . mail@cbaines.net)(address . 30803@debbugs.gnu.org)

Message-ID:874lleo1j1.fsf@gnu.org

Hello Christopher,

Christopher Baines <mail@cbaines.net> skribis:

Toggle quote (3 lines)

> I'm trying to reduce the number of patches I have sitting around in git

> branches, so here is a bug about Elasticsearch.

Cool, thanks for sharing!

Toggle quote (4 lines)

> I've had a really awful package definition for this for a while, which

> just makes the built things in the release tarball run. While this isn't

> suitable for inclusion in to Guix, it does still work.

Do you know if we’re missing a lot of the Java dependencies to build it

from source?

Ludo’.

Christopher Baines wrote on 17 Mar 2018 23:36

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 30803@debbugs.gnu.org)

Message-ID:87woyaxrcc.fsf@cbaines.net

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (16 lines)> Hello Christopher,
>
> Christopher Baines <mail@cbaines.net> skribis:
>
>> I'm trying to reduce the number of patches I have sitting around in git
>> branches, so here is a bug about Elasticsearch.
>
> Cool, thanks for sharing!
>
>> I've had a really awful package definition for this for a while, which
>> just makes the built things in the release tarball run. While this isn't
>> suitable for inclusion in to Guix, it does still work.
>
> Do you know if we’re missing a lot of the Java dependencies to build it
> from source?

Unfortunately not, I haven't attempted to build it from source.

zimoun wrote on 17 Sep 2022 12:41

Re: bug#30803: [PATCH 0/2] Add Elasticsearch package and service.

Recipients:(name . Christopher Baines)(address . mail@cbaines.net)(address . 30803@debbugs.gnu.org)

Message-ID:86pmfu9tbl.fsf@gmail.com

Hi,

On Tue, 13 Mar 2018 at 19:12, Christopher Baines <mail@cbaines.net> wrote:

Toggle quote (2 lines)

> Also included is a service, along with a really simple test.

What about these patches?

Thanks,

simon

PS: From 10 Years event.

Maxim Cournoyer wrote on 21 Jul 2023 18:55

Recipients:(name . zimoun)(address . zimon.toutoune@gmail.com)

Message-ID:87bkg5i4qt.fsf_-_@gmail.com

Hello,

zimoun <zimon.toutoune@gmail.com> writes:

Toggle quote (13 lines)> Hi,
>
> On Tue, 13 Mar 2018 at 19:12, Christopher Baines <mail@cbaines.net> wrote:
>
>> Also included is a service, along with a really simple test.
>
> What about these patches?
>
> Thanks,
> simon
>
> PS: From 10 Years event.

I'm closing this stale patch issue (there was an issue about Java
binaries being required IIUC).

-- 
Thanks,
Maxim

Closed

Your comment

This issue is archived.

To comment on this conversation send an email to 30803@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 30803

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date