[PATCH] Add rsyslog

  • Done
  • quality assurance status badge
Details
3 participants
  • Katherine Cox-Buday
  • Ludovic Courtès
  • Ricardo Wurmus
Owner
unassigned
Submitted by
Katherine Cox-Buday
Severity
normal
K
K
Katherine Cox-Buday wrote on 3 Jun 2020 16:29
Add rsyslog
(address . guix-patches@gnu.org)
CA+TvSRigMNYAjB74v2=YnVhELzqNDFA8uojK3zZ06SRb+JWY+Q@mail.gmail.com

From 8d8d97441b397a48ae26761c2d826f35ae5d56e9 Mon Sep 17 00:00:00 2001
From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
Date: Tue, 2 Jun 2020 14:55:23 -0500
Subject: [PATCH 1/5] gnu: Add libestr.

* gnu/packages/c.scm (libestr): New variable.
---
gnu/packages/c.scm | 34 ++++++++++++++++++++++++++++++++++
1 file changed, 34 insertions(+)

Toggle diff (51 lines)
diff --git a/gnu/packages/c.scm b/gnu/packages/c.scm
index 3e227decc9..d3820c88b2 100644
--- a/gnu/packages/c.scm
+++ b/gnu/packages/c.scm
@@ -8,6 +8,7 @@
;;; Copyright © 2019 Andreas Enge <andreas@enge.fr>
;;; Copyright © 2020 Jan (janneke) Nieuwenhuizen <janneke@gnu.org>
;;; Copyright © 2020 Marius Bakke <marius@gnu.org>
+;;; Copyright @ 2020 Katherine Cox-Buday <cox.katherine.e@gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -339,3 +340,36 @@ address space pointers point to, or what locks a function acquires or
releases.")
(home-page "https://sparse.wiki.kernel.org/index.php/Main_Page")
(license license:expat)))
+
+(define-public libestr
+ (package
+ (name "libestr")
+ (version "0.1.11")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/rsyslog/libestr.git")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1ca4rj90c0dn7kqpbcchkflxjw88a7rxcnwbr0gply4a28i01nd8"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ ;; autogen.sh calls configure at the end of the script.
+ (replace 'bootstrap
+ (lambda _ (invoke "autoreconf" "-vfi"))))))
+ (native-inputs
+ `(("autoconf" ,autoconf)
+ ("automake" ,automake)
+ ("pkg-config" ,pkg-config)
+ ("libtool" ,libtool)))
+ (home-page "https://github.com/rsyslog/libestr")
+ (synopsis "Helper functions for handling strings")
+ (description
+ "A library which contains some essential string manipulation
+functions and more, like escaping special characters.")
+ (license license:lgpl2.1)))
--
2.26.2
Attachment: 0005-gnu-Add-rsyslog.patch
From 3667290c9d419ce5404d6a0631bf20ddbcf1c286 Mon Sep 17 00:00:00 2001
From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
Date: Tue, 2 Jun 2020 15:34:19 -0500
Subject: [PATCH 4/5] gnu: Add liblognorm.

* gnu/packages/c.scm (liblognorm): New variable.
---
gnu/packages/c.scm | 55 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 55 insertions(+)

Toggle diff (72 lines)
diff --git a/gnu/packages/c.scm b/gnu/packages/c.scm
index 98c939e76f..3a7914346e 100644
--- a/gnu/packages/c.scm
+++ b/gnu/packages/c.scm
@@ -46,6 +46,7 @@
#:use-module (gnu packages autotools)
#:use-module (gnu packages gettext)
#:use-module (gnu packages pkg-config)
+ #:use-module (gnu packages web)
#:use-module (gnu packages xml))
(define-public tcc
@@ -442,3 +443,57 @@ be very fast in processing.")
"Liblogging is an easy to use library for logging. It offers an enhanced
replacement for the syslog() call, but retains its ease of use.")
(license license:bsd-2)))
+
+(define-public liblognorm
+ (package
+ (name "liblognorm")
+ (version "2.0.6")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/rsyslog/liblognorm.git")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1pyy1swvq6jj12aqma42jimv71z8m66zy6ydd5v19cp2azm4krml"))))
+ (build-system gnu-build-system)
+ (arguments
+ ;; Bash scripts interact with the filesystem
+ `(#:tests? #f
+ #:configure-flags
+ (list (string-append "--includedir="
+ (assoc-ref %outputs "dev")
+ "/include"))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'install 'fix-circular-dependency
+ (lambda* (#:key outputs #:allow-other-keys)
+ (write (string-append "KT: " (assoc-ref outputs "lib")
+ "/lib/pkgconfig"))
+ (write (string-append "KT: " (assoc-ref outputs "dev")
+ "/lib/pkgconfig"))
+ (let ((pkgconfig (string-append (assoc-ref outputs "dev")
+ "/lib/pkgconfig")))
+ (mkdir-p pkgconfig)
+ (rename-file (string-append (assoc-ref outputs "lib")
+ "/lib/pkgconfig")
+ pkgconfig)))))))
+ (inputs
+ `(("libestr" ,libestr)
+ ("libfastjson" ,libfastjson)
+ ("json-c" ,json-c)))
+ (native-inputs
+ `(("autoconf" ,autoconf)
+ ("automake" ,automake)
+ ("libtool" ,libtool)
+ ("pkg-config" ,pkg-config)))
+ (outputs '("out" "lib" "dev"))
+ (home-page "https://www.liblognorm.com")
+ (synopsis
+ "Fast samples-based log normalization library")
+ (description
+ "Liblognorm normalizes event data into well-defined name-value
+pairs and a set of tags describing the message.")
+ (license license:lgpl2.1)))
--
2.26.2
From db4bcfc18e95fa39851c72414261b19b25c49db0 Mon Sep 17 00:00:00 2001
From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
Date: Tue, 2 Jun 2020 15:24:59 -0500
Subject: [PATCH 2/5] gnu: Add libfastjson.

* gnu/packages/c.scm (libfastjson): New variable.
---
gnu/packages/c.scm | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)

Toggle diff (42 lines)
diff --git a/gnu/packages/c.scm b/gnu/packages/c.scm
index d3820c88b2..307d8ca182 100644
--- a/gnu/packages/c.scm
+++ b/gnu/packages/c.scm
@@ -373,3 +373,35 @@ releases.")
"A library which contains some essential string manipulation
functions and more, like escaping special characters.")
(license license:lgpl2.1)))
+
+(define-public libfastjson
+ (package
+ (name "libfastjson")
+ (version "0.99.8")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/rsyslog/libfastjson.git")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0qhs0g9slj3p0v2z4s3cnsx44msrlb4k78ljg7714qiziqbrbwyl"))))
+ (build-system gnu-build-system)
+ (native-inputs
+ `(("autoconf" ,autoconf)
+ ("automake" ,automake)
+ ("libtool" ,libtool)))
+ (home-page "https://github.com/rsyslog/libfastjson")
+ (synopsis "Fast json library for C ")
+ (description
+ "libfastjson is a fork from json-c, and is currently under development.
+The aim of this project is not to provide a slightly modified clone of json-c.
+It's aim is to provide: a small library with essential json handling
+functions, sufficiently good json support (not 100% standards compliant), and
+be very fast in processing.")
+ (license
+ (license:non-copyleft
+ "https://github.com/rsyslog/libfastjson/blob/master/COPYING"
+ "It is a MIT license."))))
--
2.26.2
From 9e0c24cc6e0666581b11b45f831ab49486adbdf8 Mon Sep 17 00:00:00 2001
From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
Date: Tue, 2 Jun 2020 15:29:08 -0500
Subject: [PATCH 3/5] gnu: Add liblogging.

* gnu/packages/c.scm (liblogging): New variable.
---
gnu/packages/c.scm | 37 +++++++++++++++++++++++++++++++++++++
1 file changed, 37 insertions(+)

Toggle diff (54 lines)
diff --git a/gnu/packages/c.scm b/gnu/packages/c.scm
index 307d8ca182..98c939e76f 100644
--- a/gnu/packages/c.scm
+++ b/gnu/packages/c.scm
@@ -42,6 +42,7 @@
#:use-module (gnu packages multiprecision)
#:use-module (gnu packages pcre)
#:use-module (gnu packages python)
+ #:use-module (gnu packages python-xyz)
#:use-module (gnu packages autotools)
#:use-module (gnu packages gettext)
#:use-module (gnu packages pkg-config)
@@ -405,3 +406,39 @@ be very fast in processing.")
(license:non-copyleft
"https://github.com/rsyslog/libfastjson/blob/master/COPYING"
"It is a MIT license."))))
+
+(define-public liblogging
+ (package
+ (name "liblogging")
+ (version "1.0.6")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/rsyslog/liblogging.git")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "1l32m0y65svf5vxsgw935jnqs6842rcqr56dmzwqvr00yfrjhjkp"))))
+ (build-system gnu-build-system)
+ (arguments
+ `(#:phases
+ (modify-phases %standard-phases
+ ;; autogen.sh calls configure at the end of the script.
+ (replace 'bootstrap
+ (lambda _ (invoke "autoreconf" "-vfi"))))))
+ (native-inputs
+ `(("autoconf" ,autoconf)
+ ("automake" ,automake)
+ ("pkg-config" ,pkg-config)
+ ("libtool" ,libtool)
+ ;; For rst2man.py
+ ("python-docutils" ,python-docutils)))
+ (home-page "https://github.com/rsyslog/liblogging")
+ (synopsis
+ "Easy to use and lightweight signal-safe logging library")
+ (description
+ "Liblogging is an easy to use library for logging. It offers an enhanced
+replacement for the syslog() call, but retains its ease of use.")
+ (license license:bsd-2)))
--
2.26.2
K
K
Katherine Cox-Buday wrote on 4 Jun 2020 17:56
Adding patch tags
(address . control@debbugs.gnu.org)
CA+TvSRiEwzjjV4jWKU_NLurvH5Jn04_bVXLW=WsZ0XnJkD=hxA@mail.gmail.com
retitle 41670 [PATCH] Adds Multiple Common Lisp Packages
retitle 41677 [PATCH] Update apache-arrow to v0.17.1
retitle 41688 [PATCH] Add rsyslog
retitle 41689 [PATCH] Add cl-rdkafka
L
L
Ludovic Courtès wrote on 12 Jun 2020 18:45
Re: [bug#41688] Add rsyslog
(name . Katherine Cox-Buday)(address . cox.katherine.e@gmail.com)(address . 41688@debbugs.gnu.org)
87r1uk44ih.fsf@gnu.org
Hi Katherine,

Thanks for this long patch series! I’ve applied half of it. Here are
some comments:

Katherine Cox-Buday <cox.katherine.e@gmail.com> skribis:

Toggle quote (7 lines)
> From 8d8d97441b397a48ae26761c2d826f35ae5d56e9 Mon Sep 17 00:00:00 2001
> From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
> Date: Tue, 2 Jun 2020 14:55:23 -0500
> Subject: [PATCH 1/5] gnu: Add libestr.
>
> * gnu/packages/c.scm (libestr): New variable.

Applied with the changes below.
Toggle diff (14 lines)
diff --git a/gnu/packages/c.scm b/gnu/packages/c.scm
index d3820c88b2..12a9b56c51 100644
--- a/gnu/packages/c.scm
+++ b/gnu/packages/c.scm
@@ -370,6 +370,6 @@ releases.")
(home-page "https://github.com/rsyslog/libestr")
(synopsis "Helper functions for handling strings")
(description
- "A library which contains some essential string manipulation
-functions and more, like escaping special characters.")
- (license license:lgpl2.1)))
+ "This C library contains some essential string manipulation functions and
+more, like escaping special characters.")
+ (license license:lgpl2.1+)))
Toggle quote (7 lines)
> From 0e0e9cc550ff002d7bebb4f93b573c5ba527ac4d Mon Sep 17 00:00:00 2001
> From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
> Date: Tue, 2 Jun 2020 15:47:41 -0500
> Subject: [PATCH 5/5] gnu: Add rsyslog.
>
> * gnu/packages/logging.scm (rsyslog): New variable.

[...]

Toggle quote (27 lines)
> +(define-public rsyslog
> + (let ((modules (list "fmhash" "fmhttp" "imbatchreport"
> + "imczmq" "imdiag" "imdocker"
> + "imfile" "imgssapi" "imkafka"
> + "imklog" "imkmsg" "immark"
> + "improg" "impstats" "imptcp" "imtcp"
> + "imtuxedoulog" "imudp" "imuxsock"
> + "mmanon" "mmaudit" "mmcount"
> + "mmdblookup" "mmexternal"
> + "mmfields" "mmjsonparse"
> + "mmkubernetes" "mmnormalize"
> + "mmpstrucdata" "mmrfc5424addhmac"
> + "mmrm1stspace" "mmsequence"
> + "mmsnmptrapd" "mmtaghostname"
> + "mmutf8fix" "omclickhouse"
> + "omczmq" "omelasticsearch"
> + "omfile-hardened" "omgssapi"
> + "omhttpfs" "omhttp" "omkafka"
> + "omlibdbi" "ommail" "ommysql" "ompgsql"
> + "omprog" "omruleset" "omsnmp"
> + "omstdout" "omtcl" "omtesting"
> + "omudpspoof" "omuxsock"
> + "pmaixforwardedfrom" "pmciscoios"
> + "pmcisconames" "pmdb2diag"
> + "pmlastmsg" "pmnormalize"
> + "pmnull" "pmpanngfw" "pmsnare")))

Could you add a comment explaining where this list comes from?
Alternatively, do you think it could be generated at build time?

Toggle quote (8 lines)
> + #:configure-flags
> + (list
> + "--enable-largefile"
> + "--enable-inet"
> + "--enable-regexp"
> + "--enable-rsyslogrt"
> + "--enable-rsyslogd"

That’s also a very long list. :-)

Can we rely on defaults? Or, alternatively, could you add a comment
explaining how to come up with the list?

Toggle quote (2 lines)
> + (outputs `("out" ,@modules))

Oh. Is it a good idea to have one output per module? We’ve never done
such a thing before. Usually, extra outputs are justified if not having
them would lead to a big closure (as reported by ‘guix size’). The
solution is usually to have two or three outputs, but not 20ish. :-)

WDYT?

Toggle quote (5 lines)
> + (synopsis "RSYSLOG is the rocket-fast system for log processing")
> + (description
> + "It offers high-performance, great security features and a modular

s/It/Rsyslog/

Toggle quote (4 lines)
> + (license (list license:gpl3
> + license:asl2.0
> + license:lgpl3)))))

Please add a comment above explaining if it’s triple-licensing or a
combination, and possibly add missing ‘+’ signs.

Toggle quote (7 lines)
> From 3667290c9d419ce5404d6a0631bf20ddbcf1c286 Mon Sep 17 00:00:00 2001
> From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
> Date: Tue, 2 Jun 2020 15:34:19 -0500
> Subject: [PATCH 4/5] gnu: Add liblognorm.
>
> * gnu/packages/c.scm (liblognorm): New variable.

[...]

Toggle quote (4 lines)
> + (arguments
> + ;; Bash scripts interact with the filesystem
> + `(#:tests? #f

Could you clarify what this means with regards to running tests?

Toggle quote (13 lines)
> + #:configure-flags
> + (list (string-append "--includedir="
> + (assoc-ref %outputs "dev")
> + "/include"))
> + #:phases
> + (modify-phases %standard-phases
> + (add-after 'install 'fix-circular-dependency
> + (lambda* (#:key outputs #:allow-other-keys)
> + (write (string-append "KT: " (assoc-ref outputs "lib")
> + "/lib/pkgconfig"))
> + (write (string-append "KT: " (assoc-ref outputs "dev")
> + "/lib/pkgconfig"))

Leftovers. :-)

Toggle quote (7 lines)
> + (let ((pkgconfig (string-append (assoc-ref outputs "dev")
> + "/lib/pkgconfig")))
> + (mkdir-p pkgconfig)
> + (rename-file (string-append (assoc-ref outputs "lib")
> + "/lib/pkgconfig")
> + pkgconfig)))))))

Missing #t return value.

Toggle quote (2 lines)
> + (outputs '("out" "lib" "dev"))

“dev” is not part of the output names conventionally used. However,
there’s “include”, which is automatically recognized by gnu-build-system
and turned into a ‘--includedir’ flag. Perhaps you could try this?

Or you can just have “out” and “lib”: so far, unless headers are very
large, we just keep them alongside the library.

Toggle quote (8 lines)
> + (home-page "https://www.liblognorm.com")
> + (synopsis
> + "Fast samples-based log normalization library")
> + (description
> + "Liblognorm normalizes event data into well-defined name-value
> +pairs and a set of tags describing the message.")
> + (license license:lgpl2.1)))

Could it be lgpl2.1+? (I haven’t checked.)

Toggle quote (7 lines)
> From db4bcfc18e95fa39851c72414261b19b25c49db0 Mon Sep 17 00:00:00 2001
> From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
> Date: Tue, 2 Jun 2020 15:24:59 -0500
> Subject: [PATCH 2/5] gnu: Add libfastjson.
>
> * gnu/packages/c.scm (libfastjson): New variable.

Applied with minor changes.
Toggle diff (24 lines)
diff --git a/gnu/packages/c.scm b/gnu/packages/c.scm
index e935042572..2e62111d44 100644
--- a/gnu/packages/c.scm
+++ b/gnu/packages/c.scm
@@ -394,14 +394,9 @@ more, like escaping special characters.")
("automake" ,automake)
("libtool" ,libtool)))
(home-page "https://github.com/rsyslog/libfastjson")
- (synopsis "Fast json library for C ")
+ (synopsis "Fast JSON library for C")
(description
- "libfastjson is a fork from json-c, and is currently under development.
-The aim of this project is not to provide a slightly modified clone of json-c.
-It's aim is to provide: a small library with essential json handling
-functions, sufficiently good json support (not 100% standards compliant), and
-be very fast in processing.")
- (license
- (license:non-copyleft
- "https://github.com/rsyslog/libfastjson/blob/master/COPYING"
- "It is a MIT license."))))
+ "libfastjson is a fork from json-c aiming to provide: a small library
+with essential JSON handling functions, sufficiently good JSON support (not
+100% standards compliant), and very fast processing.")
+ (license license:expat)))
Toggle quote (7 lines)
> From 9e0c24cc6e0666581b11b45f831ab49486adbdf8 Mon Sep 17 00:00:00 2001
> From: Katherine Cox-Buday <cox.katherine.e@gmail.com>
> Date: Tue, 2 Jun 2020 15:29:08 -0500
> Subject: [PATCH 3/5] gnu: Add liblogging.
>
> * gnu/packages/c.scm (liblogging): New variable.

Applied.

Thank you!

Ludo’.
R
R
Ricardo Wurmus wrote on 31 Dec 2021 10:22
[PATCH] Add rsyslog
(address . 41688-done@debbugs.gnu.org)(name . Katherine Cox-Buday)(address . cox.katherine.e@gmail.com)
87fsq9gm66.fsf@elephly.net
Hi,

I’ve followed Ludo’s suggestions for the remaining package definitions
and pushed the result to the “master” branch.

Among the changes:

- fixed and clarified licenses
- removed per-module outputs
- removed default configure flags and added a comment about enabling
optional features
- enabled tests
- removed “dev” output
- used new input style

Now I guess we’ll need an rsyslog service :)

Thank you, Katherine, for the patches and your patience!

--
Ricardo
Closed
K
K
Katherine Cox-Buday wrote on 5 Jan 2022 21:04
(name . Ricardo Wurmus)(address . rekado@elephly.net)(address . 41688-done@debbugs.gnu.org)
87iluyuext.fsf@gmail.com
Ricardo Wurmus <rekado@elephly.net> writes:

Toggle quote (3 lines)
> I’ve followed Ludo’s suggestions for the remaining package definitions
> and pushed the result to the “master” branch.

Thank you so much, Ricardo!

It is my fault for letting this sit, but I think we should unbundle the outputs once more -- even if they must become separate packages.

As you know, rsyslog is meant to forward (egress) and receive (ingress) syslog logs, and it does so through these plug-ins. For ingress, I don't think this package is necessarily a problem, but systems that want to forward logs may have very limited resources (e.g. I have a wifi repeater that forwards logs). I didn't want to require things like PostgreSQL and MySQL, dependencies required by ingress, on egress.

A secondary concern is that a package like rsyslog becomes quite sticky since there are lots and lots of modules, and I imagine the Guix closure becomes quite big.

Toggle quote (12 lines)
> Among the changes:
>
> - fixed and clarified licenses
> - removed per-module outputs
> - removed default configure flags and added a comment about enabling
> optional features
> - enabled tests
> - removed “dev” output
> - used new input style
>
> Now I guess we’ll need an rsyslog service :)

I'm working[1] on it! Although it is frustrating at how little time I have to dedicate to things like this :(

Toggle quote (2 lines)
> Thank you, Katherine, for the patches and your patience!

Likewise, thank you for your contributions and getting this landed. Cheers!


--
Katherine
Closed
R
R
Ricardo Wurmus wrote on 5 Jan 2022 21:51
(name . Katherine Cox-Buday)(address . cox.katherine.e@gmail.com)(address . 41688-done@debbugs.gnu.org)
87bl0pdhsq.fsf@elephly.net
Katherine Cox-Buday <cox.katherine.e@gmail.com> writes:

Toggle quote (3 lines)
> I think we should unbundle the outputs once more -- even if they must
> become separate packages.

If we can build the plugins separately (and arrange for rsyslog to find
them via some search path) I’d be happy to see them moved to their own
packages. Having them as separate outputs would only help a little as
you’d still need to have *all* the inputs ready to build the thing, even
if you just want to use one of the many plugins.

Toggle quote (4 lines)
>> Now I guess we’ll need an rsyslog service :)
>
> I'm working[1] on it! Although it is frustrating at how little time I have to dedicate to things like this :(

Excellent, I’m looking forward to a service! Perhaps we could also use
the existing syslog service with it if that service allows us to pass a
plain configuration file. It should already let us swap out the syslog
package.

--
Ricardo
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 41688@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 41688
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch