[PATCH] gnu: Add jsonnet.

  • Done
  • quality assurance status badge
Details
4 participants
  • Leo Prikler
  • Ludovic Courtès
  • Maxime Devos
  • Vivien Kraus
Owner
unassigned
Submitted by
Vivien Kraus
Severity
normal
V
V
Vivien Kraus wrote on 2 Aug 2021 17:01
(address . guix-patches@gnu.org)
71fbbaa57a2c1dba8f0d956ea162c2e15d332e0c.camel@planete-kraus.eu
Hello,

Here is jsonnet. I don’t fully understand what it does, but thanks to
leoprikler, I know it is a dependency to package GraalJS, an
interpreter for JavaScript on the Java Virtual Machine.

For the sake of having "no dependencies", it bundles a custom
implementation of MD5. According to a comment in the associated
license, it is taken from the implementation of bzflag, but it does not
seem to use the same as of today.

I decided to use nettle, and add a few lines of C++ to shape it into
the required interface.

Best regards,

Vivien
From 7cfbec67974d03c6e7dd56e4e2ade95f38faad6a Mon Sep 17 00:00:00 2001
From: Vivien Kraus <vivien@planete-kraus.eu>
Date: Mon, 2 Aug 2021 16:07:08 +0200
Subject: [PATCH] gnu: Add jsonnet.

* gnu/packages/cpp.scm (jsonnet): New variable.
---
gnu/packages/cpp.scm | 103 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 103 insertions(+)

Toggle diff (120 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index 42e9d50687..4e51c4a0cc 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -63,6 +63,7 @@
#:use-module (gnu packages llvm)
#:use-module (gnu packages logging)
#:use-module (gnu packages maths)
+ #:use-module (gnu packages nettle)
#:use-module (gnu packages onc-rpc)
#:use-module (gnu packages perl)
#:use-module (gnu packages pkg-config)
@@ -1211,3 +1212,105 @@ of reading and writing XML.")
;; incompatible with the GPL v2. Refer to the file named FLOSSE for the
;; details.
(license license:gpl2+)))
+
+(define-public jsonnet
+ (package
+ (name "jsonnet")
+ (version "0.17.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/google/jsonnet")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1ddz14699v5lqx3dh0mb7hfffr6fk5zhmzn3z8yxkqqvriqnciim"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file-recursively "third_party")
+ (delete-file-recursively "doc/third_party")
+ (substitute*
+ '("core/vm.cpp")
+ (("#include \"json.hpp\"") "#include <nlohmann/json.hpp>"))
+ (mkdir-p "third_party/md5")
+ (call-with-output-file "third_party/md5/CMakeLists.txt"
+ (lambda (port)
+ (format port "add_library(md5 STATIC md5.cpp md5.h)
+find_package(PkgConfig REQUIRED)
+pkg_check_modules(NETTLE REQUIRED nettle)
+target_link_libraries(md5 ${NETTLE_LIBRARIES})
+target_include_directories(md5 PUBLIC ${NETTLE_INCLUDE_DIRS})
+")))
+ (call-with-output-file "third_party/md5/md5.h"
+ (lambda (port)
+ (format port "#ifndef BZF_MD5_H
+#define BZF_MD5_H
+#include <string>
+
+// Return the hexadecimal digest.
+std::string md5 (const std::string str);
+
+#endif
+")))
+ (call-with-output-file "third_party/md5/md5.cpp"
+ (lambda (port)
+ (format port "#include \"md5.h\"
+#include <nettle/md5.h>
+#include <nettle/base16.h>
+#include <string>
+#include <vector>
+
+#define OUTPUT_LENGTH BASE16_ENCODE_LENGTH (MD5_DIGEST_SIZE)
+
+std::string
+md5 (const std::string str)
+{
+ // Convert str to a byte array
+ std::vector<uint8_t> input (str.begin (), str.end ());
+
+ // Compute the digest
+ struct md5_ctx nettle_ctx;
+ md5_init (&nettle_ctx);
+ md5_update (&nettle_ctx, input.size (), input.data ());
+ uint8_t digest[MD5_DIGEST_SIZE];
+ md5_digest (&nettle_ctx, MD5_DIGEST_SIZE, digest);
+
+ // Encode it to base16
+ std::vector<char> encoded (BASE16_ENCODE_LENGTH (MD5_DIGEST_SIZE));
+ base16_encode_update (encoded.data (), MD5_DIGEST_SIZE, digest);
+ std::string final_digest (encoded.begin (), encoded.end ());
+ return final_digest;
+}
+")))))))
+ (build-system cmake-build-system)
+ (arguments
+ `(#:configure-flags '("-DUSE_SYSTEM_GTEST=ON" "-DUSE_SYSTEM_JSON=ON")))
+ (propagated-inputs
+ '())
+ (native-inputs
+ `(("googletest" ,googletest)
+ ("pkg-config" ,pkg-config)))
+ (inputs
+ `(("json-modern-cxx" ,json-modern-cxx)
+ ;; jsonnet uses a md5 implementation claiming to be from
+ ;; https://www.bzflag.org/, but they don’t use it anymore. We
+ ;; replace it with md5 from nettle.
+ ("nettle" ,nettle)))
+ (home-page "https://jsonnet.org/")
+ (synopsis "The data templating language")
+ (description "A data templating language for app and tool developers:
+- Generate config data
+- Side-effect free
+- Organize, simplify, unify
+- Manage sprawling config
+
+A simple extension of JSON
+- Open source (Apache 2.0)
+- Familiar syntax
+- Reformatter, linter
+- Editor & IDE integrations
+- Formally specified
+")
+ (license license:asl2.0)))
--
2.32.0
V
V
Vivien Kraus wrote on 2 Aug 2021 18:15
Using texinfo for the description
(address . 49823@debbugs.gnu.org)
b1abc32b7ea40f47e589a03fe3ac066f3e2ae228.camel@planete-kraus.eu
maximed told me I could use texinfo in the description.

(I also removed a couple of lines in the C++ source because the
constant was not used.)

Also, I was advised to put the added source code in a local-file such
as what is done by gnuzilla, but I could not figure out how it was done
for gnuzilla (the package is quite large). If I go with a patch, there
are a lot of things that get deleted, so it makes a rather large patch.

What do you think?

Vivien
From 7fb706951904f58cb3dd8963885024f7d9383c0b Mon Sep 17 00:00:00 2001
From: Vivien Kraus <vivien@planete-kraus.eu>
Date: Mon, 2 Aug 2021 16:07:08 +0200
Subject: [PATCH] gnu: Add jsonnet.

* gnu/packages/cpp.scm (jsonnet): New variable.
---
gnu/packages/cpp.scm | 105 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 105 insertions(+)

Toggle diff (122 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index 42e9d50687..318e387069 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -63,6 +63,7 @@
#:use-module (gnu packages llvm)
#:use-module (gnu packages logging)
#:use-module (gnu packages maths)
+ #:use-module (gnu packages nettle)
#:use-module (gnu packages onc-rpc)
#:use-module (gnu packages perl)
#:use-module (gnu packages pkg-config)
@@ -1211,3 +1212,107 @@ of reading and writing XML.")
;; incompatible with the GPL v2. Refer to the file named FLOSSE for the
;; details.
(license license:gpl2+)))
+
+(define-public jsonnet
+ (package
+ (name "jsonnet")
+ (version "0.17.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/google/jsonnet")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1ddz14699v5lqx3dh0mb7hfffr6fk5zhmzn3z8yxkqqvriqnciim"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file-recursively "third_party")
+ (delete-file-recursively "doc/third_party")
+ (substitute*
+ '("core/vm.cpp")
+ (("#include \"json.hpp\"") "#include <nlohmann/json.hpp>"))
+ (mkdir-p "third_party/md5")
+ (call-with-output-file "third_party/md5/CMakeLists.txt"
+ (lambda (port)
+ (format port "add_library(md5 STATIC md5.cpp md5.h)
+find_package(PkgConfig REQUIRED)
+pkg_check_modules(NETTLE REQUIRED nettle)
+target_link_libraries(md5 ${NETTLE_LIBRARIES})
+target_include_directories(md5 PUBLIC ${NETTLE_INCLUDE_DIRS})
+")))
+ (call-with-output-file "third_party/md5/md5.h"
+ (lambda (port)
+ (format port "#ifndef BZF_MD5_H
+#define BZF_MD5_H
+#include <string>
+
+// Return the hexadecimal digest.
+std::string md5 (const std::string str);
+
+#endif
+")))
+ (call-with-output-file "third_party/md5/md5.cpp"
+ (lambda (port)
+ (format port "#include \"md5.h\"
+#include <nettle/md5.h>
+#include <nettle/base16.h>
+#include <string>
+#include <vector>
+
+std::string
+md5 (const std::string str)
+{
+ // Convert str to a byte array
+ std::vector<uint8_t> input (str.begin (), str.end ());
+
+ // Compute the digest
+ struct md5_ctx nettle_ctx;
+ md5_init (&nettle_ctx);
+ md5_update (&nettle_ctx, input.size (), input.data ());
+ uint8_t digest[MD5_DIGEST_SIZE];
+ md5_digest (&nettle_ctx, MD5_DIGEST_SIZE, digest);
+
+ // Encode it to base16
+ std::vector<char> encoded (BASE16_ENCODE_LENGTH (MD5_DIGEST_SIZE));
+ base16_encode_update (encoded.data (), MD5_DIGEST_SIZE, digest);
+ std::string final_digest (encoded.begin (), encoded.end ());
+ return final_digest;
+}
+")))))))
+ (build-system cmake-build-system)
+ (arguments
+ `(#:configure-flags '("-DUSE_SYSTEM_GTEST=ON" "-DUSE_SYSTEM_JSON=ON")))
+ (propagated-inputs
+ '())
+ (native-inputs
+ `(("googletest" ,googletest)
+ ("pkg-config" ,pkg-config)))
+ (inputs
+ `(("json-modern-cxx" ,json-modern-cxx)
+ ;; jsonnet uses a md5 implementation claiming to be from
+ ;; https://www.bzflag.org/, but they don’t use it anymore. We
+ ;; replace it with md5 from nettle.
+ ("nettle" ,nettle)))
+ (home-page "https://jsonnet.org/")
+ (synopsis "The data templating language")
+ (description "A data templating language for app and tool developers:
+@itemize
+@item Generate config data
+@item Side-effect free
+@item Organize, simplify, unify
+@item Manage sprawling config
+@end itemize
+
+A simple extension of JSON:
+@itemize
+@item Open source (Apache 2.0)
+@item Familiar syntax
+@item Reformatter, linter
+@item Editor & IDE integrations
+@item Formally specified
+@end itemize
+")
+ (license license:asl2.0)))
--
2.32.0
M
M
Maxime Devos wrote on 2 Aug 2021 18:35
dcc495dd024e91ac850d925c83b30efee82db549.camel@telenet.be
Vivien Kraus via Guix-patches via schreef op ma 02-08-2021 om 18:15 [+0200]:
Toggle quote (9 lines)
> maximed told me I could use texinfo in the description.
>
> (I also removed a couple of lines in the C++ source because the
> constant was not used.)
>
> Also, I was advised to put the added source code in a local-file such
> as what is done by gnuzilla, but I could not figure out how it was done
> for gnuzilla (the package is quite large).

Something like this should work (untested):

(package
(origin
[...]
(snippet
#~(begin
[...]
(delete-file "third_party/md5/md5.cpp") ; maybe not necessary?
(copy-file #+(search-auxiliary-file "jsonnet-md5.cpp")
"third_party/md5/md5.cpp")
[...])))
[...])

Toggle quote (3 lines)
> If I go with a patch, there
> are a lot of things that get deleted, so it makes a rather large patch.

"git diff" has an option "--irreversible-deletes" and "--break-rewrites"
which should reduce the size of the patch, by not including the previous
version.

Toggle quote (3 lines)
> What do you think?
>
> Vivien
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYQgexhccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7vglAP9CU7VXrn0SPMhIXO537Q2pFmw4
0c0ngwWSloylqAw2JwD/enUj4vIxSIEXRp9C3ivff/Plsq/wVCthXMbBsweSeQU=
=8VgX
-----END PGP SIGNATURE-----


V
V
Vivien Kraus wrote on 2 Aug 2021 19:09
b8c23b18b9b032bce8bafb66c5dd8dd260e00015.camel@planete-kraus.eu
Le lundi 02 août 2021 à 18:35 +0200, Maxime Devos a écrit :
Toggle quote (8 lines)
> Something like this should work (untested):
>
> (package
> (origin
> [...]
> (snippet
> #~(begin

I didn’t know snippets could be Gexps. Thank you.
L
L
Leo Prikler wrote on 2 Aug 2021 19:52
80325791f76c740a20bd7ad04f9191da9299ec9a.camel@student.tugraz.at
Hi Vivien,

sorry for coaxing you into packaging this thing, I merely noticed
graaljs using jsonnet and I'm pretty sure those paths will at some
point of the build need to get parsed. ^^"

Am Montag, den 02.08.2021, 19:09 +0200 schrieb Vivien Kraus:
Toggle quote (1 lines)
> + (synopsis "The data templating language")
Should just be "Data templating language"
Toggle quote (18 lines)
> + (description "A data templating language for app and tool
> developers:
> +@itemize
> +@item Generate config data
> +@item Side-effect free
> +@item Organize, simplify, unify
> +@item Manage sprawling config
> +@end itemize
> +
> +A simple extension of JSON:
> +@itemize
> +@item Open source (Apache 2.0)
> +@item Familiar syntax
> +@item Reformatter, linter
> +@item Editor & IDE integrations
> +@item Formally specified
> +@end itemize
> +")
Sorry, but that's a little too much keyword soup. Perhaps "Jsonnet is
a templating language extending JSON syntax with variables, conditions,
functions and more."
Toggle quote (3 lines)
> + (license license:asl2.0)))

> + `(("json-modern-cxx" ,json-modern-cxx)
On a related note, we should imo rename this to nlohmann-json and patch
dependants accordingly. What do others think?
V
V
Vivien Kraus wrote on 3 Aug 2021 01:21
a6b05ab7a106bdaa7677edfcadda4a101fb9d850.camel@planete-kraus.eu
Here it is with the new wording.

Le lundi 02 août 2021 à 19:52 +0200, Leo Prikler a écrit :
Toggle quote (3 lines)
> On a related note, we should imo rename this to nlohmann-json and
> patch
> dependants accordingly. What do others think?
L
L
Ludovic Courtès wrote on 10 Aug 2021 14:58
Re: bug#49823: [PATCH] gnu: Add jsonnet.
(name . Vivien Kraus)(address . vivien@planete-kraus.eu)
878s19sbgf.fsf@gnu.org
Hi Vivien,

Vivien Kraus <vivien@planete-kraus.eu> skribis:

Toggle quote (12 lines)
> Here is jsonnet. I don’t fully understand what it does, but thanks to
> leoprikler, I know it is a dependency to package GraalJS, an
> interpreter for JavaScript on the Java Virtual Machine.
>
> For the sake of having "no dependencies", it bundles a custom
> implementation of MD5. According to a comment in the associated
> license, it is taken from the implementation of bzflag, but it does not
> seem to use the same as of today.
>
> I decided to use nettle, and add a few lines of C++ to shape it into
> the required interface.

I think using Nettle is a wise decision; however, it’s a decision for
upstream to make IMO. I’m not comfortable with shipping custom
md5.{cpp,h} and CMakeLists.txt; to me, we’d be taking a bit too much of
upstream’s burden and delivering something that’s quite different.

Apart from that, the last patch you sent LGTM.

Thoughts?

Thanks,
Ludo’.
V
V
Vivien Kraus wrote on 10 Aug 2021 16:28
(name . Ludovic Courtès)(address . ludo@gnu.org)
874kbx2x3e.fsf@planete-kraus.eu
Hi!

Ludovic Courtès writes:
Toggle quote (13 lines)
>> For the sake of having "no dependencies", it bundles a custom
>> implementation of MD5. According to a comment in the associated
>> license, it is taken from the implementation of bzflag, but it does not
>> seem to use the same as of today.
>>
>> I decided to use nettle, and add a few lines of C++ to shape it into
>> the required interface.
>
> I think using Nettle is a wise decision; however, it’s a decision for
> upstream to make IMO. I’m not comfortable with shipping custom
> md5.{cpp,h} and CMakeLists.txt; to me, we’d be taking a bit too much of
> upstream’s burden and delivering something that’s quite different.

OK. Should I keep the bundled MD5 implementation then?

Vivien
L
L
Ludovic Courtès wrote on 11 Aug 2021 11:56
(name . Vivien Kraus)(address . vivien@planete-kraus.eu)
87k0kspan5.fsf@gnu.org
Hi,

Vivien Kraus <vivien@planete-kraus.eu> skribis:

Toggle quote (16 lines)
> Ludovic Courtès writes:
>>> For the sake of having "no dependencies", it bundles a custom
>>> implementation of MD5. According to a comment in the associated
>>> license, it is taken from the implementation of bzflag, but it does not
>>> seem to use the same as of today.
>>>
>>> I decided to use nettle, and add a few lines of C++ to shape it into
>>> the required interface.
>>
>> I think using Nettle is a wise decision; however, it’s a decision for
>> upstream to make IMO. I’m not comfortable with shipping custom
>> md5.{cpp,h} and CMakeLists.txt; to me, we’d be taking a bit too much of
>> upstream’s burden and delivering something that’s quite different.
>
> OK. Should I keep the bundled MD5 implementation then?

Yeah, I think so. Clearly it’s a gray area, there are pros and cons,
but I’d lean towards minimal changes to upstream source.

Thanks,
Ludo’.
V
V
Vivien Kraus wrote on 11 Aug 2021 12:12
(name . Ludovic Courtès)(address . ludo@gnu.org)
87y2981e8x.fsf@planete-kraus.eu
Ludovic Courtès writes:
Toggle quote (5 lines)
>> OK. Should I keep the bundled MD5 implementation then?
>
> Yeah, I think so. Clearly it’s a gray area, there are pros and cons,
> but I’d lean towards minimal changes to upstream source.

Here you go :)
From 8ce461001ed14c3267d839334e4853feaed69b50 Mon Sep 17 00:00:00 2001
From: Vivien Kraus <vivien@planete-kraus.eu>
Date: Mon, 2 Aug 2021 16:07:08 +0200
Subject: [PATCH] gnu: Add jsonnet.

* gnu/packages/cpp.scm (jsonnet): New variable.
---
gnu/packages/cpp.scm | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)

Toggle diff (58 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index 42e9d50687..870c02dac3 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -45,6 +45,7 @@
#:use-module (guix build-system gnu)
#:use-module (guix build-system python)
#:use-module (guix modules)
+ #:use-module (guix gexp)
#:use-module (gnu packages)
#:use-module (gnu packages autotools)
#:use-module (gnu packages boost)
@@ -1211,3 +1212,43 @@ of reading and writing XML.")
;; incompatible with the GPL v2. Refer to the file named FLOSSE for the
;; details.
(license license:gpl2+)))
+
+(define-public jsonnet
+ (package
+ (name "jsonnet")
+ (version "0.17.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/google/jsonnet")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1ddz14699v5lqx3dh0mb7hfffr6fk5zhmzn3z8yxkqqvriqnciim"))
+ (modules '((guix build utils)))
+ (snippet
+ #~(begin
+ (rename-file "third_party/md5" ".md5")
+ (delete-file-recursively "third_party")
+ (delete-file-recursively "doc/third_party")
+ (substitute*
+ '("core/vm.cpp")
+ (("#include \"json.hpp\"") "#include <nlohmann/json.hpp>"))
+ (mkdir "third_party")
+ (rename-file ".md5" "third_party/md5")))))
+ (build-system cmake-build-system)
+ (arguments
+ `(#:configure-flags '("-DUSE_SYSTEM_GTEST=ON" "-DUSE_SYSTEM_JSON=ON")))
+ (propagated-inputs
+ '())
+ (native-inputs
+ `(("googletest" ,googletest)
+ ("pkg-config" ,pkg-config)))
+ (inputs
+ `(("json-modern-cxx" ,json-modern-cxx)))
+ (home-page "https://jsonnet.org/")
+ (synopsis "Data templating language")
+ (description "Jsonnet is a templating language extending JSON
+syntax with variables, conditions, functions and more.")
+ (license license:asl2.0)))
--
2.32.0
L
L
Ludovic Courtès wrote on 12 Aug 2021 09:53
(name . Vivien Kraus)(address . vivien@planete-kraus.eu)
87im0bhze6.fsf@gnu.org
Hi Vivien,

Vivien Kraus <vivien@planete-kraus.eu> skribis:

Toggle quote (7 lines)
> From 8ce461001ed14c3267d839334e4853feaed69b50 Mon Sep 17 00:00:00 2001
> From: Vivien Kraus <vivien@planete-kraus.eu>
> Date: Mon, 2 Aug 2021 16:07:08 +0200
> Subject: [PATCH] gnu: Add jsonnet.
>
> * gnu/packages/cpp.scm (jsonnet): New variable.

Applied with the tweak below.

Thank you, and thanks for taking the time for the extra work!

Ludo’.
Toggle diff (22 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index 870c02dac3..8c8109fb83 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -1232,16 +1232,13 @@ of reading and writing XML.")
(rename-file "third_party/md5" ".md5")
(delete-file-recursively "third_party")
(delete-file-recursively "doc/third_party")
- (substitute*
- '("core/vm.cpp")
+ (substitute* '("core/vm.cpp")
(("#include \"json.hpp\"") "#include <nlohmann/json.hpp>"))
(mkdir "third_party")
(rename-file ".md5" "third_party/md5")))))
(build-system cmake-build-system)
(arguments
`(#:configure-flags '("-DUSE_SYSTEM_GTEST=ON" "-DUSE_SYSTEM_JSON=ON")))
- (propagated-inputs
- '())
(native-inputs
`(("googletest" ,googletest)
("pkg-config" ,pkg-config)))
Closed
?