[PATCH] gnu: Add jsonnet.

  • Done
  • quality assurance status badge
Details
4 participants
  • Leo Prikler
  • Ludovic Courtès
  • Maxime Devos
  • Vivien Kraus
Owner
unassigned
Submitted by
Vivien Kraus
Severity
normal

Debbugs page

Vivien Kraus wrote 4 years ago
(address . guix-patches@gnu.org)
71fbbaa57a2c1dba8f0d956ea162c2e15d332e0c.camel@planete-kraus.eu
Hello,

Here is jsonnet. I don’t fully understand what it does, but thanks to
leoprikler, I know it is a dependency to package GraalJS, an
interpreter for JavaScript on the Java Virtual Machine.

For the sake of having "no dependencies", it bundles a custom
implementation of MD5. According to a comment in the associated
license, it is taken from the implementation of bzflag, but it does not
seem to use the same as of today.

I decided to use nettle, and add a few lines of C++ to shape it into
the required interface.

Best regards,

Vivien
From 7cfbec67974d03c6e7dd56e4e2ade95f38faad6a Mon Sep 17 00:00:00 2001
From: Vivien Kraus <vivien@planete-kraus.eu>
Date: Mon, 2 Aug 2021 16:07:08 +0200
Subject: [PATCH] gnu: Add jsonnet.

* gnu/packages/cpp.scm (jsonnet): New variable.
---
gnu/packages/cpp.scm | 103 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 103 insertions(+)

Toggle diff (120 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index 42e9d50687..4e51c4a0cc 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -63,6 +63,7 @@
#:use-module (gnu packages llvm)
#:use-module (gnu packages logging)
#:use-module (gnu packages maths)
+ #:use-module (gnu packages nettle)
#:use-module (gnu packages onc-rpc)
#:use-module (gnu packages perl)
#:use-module (gnu packages pkg-config)
@@ -1211,3 +1212,105 @@ of reading and writing XML.")
;; incompatible with the GPL v2. Refer to the file named FLOSSE for the
;; details.
(license license:gpl2+)))
+
+(define-public jsonnet
+ (package
+ (name "jsonnet")
+ (version "0.17.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/google/jsonnet")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1ddz14699v5lqx3dh0mb7hfffr6fk5zhmzn3z8yxkqqvriqnciim"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file-recursively "third_party")
+ (delete-file-recursively "doc/third_party")
+ (substitute*
+ '("core/vm.cpp")
+ (("#include \"json.hpp\"") "#include <nlohmann/json.hpp>"))
+ (mkdir-p "third_party/md5")
+ (call-with-output-file "third_party/md5/CMakeLists.txt"
+ (lambda (port)
+ (format port "add_library(md5 STATIC md5.cpp md5.h)
+find_package(PkgConfig REQUIRED)
+pkg_check_modules(NETTLE REQUIRED nettle)
+target_link_libraries(md5 ${NETTLE_LIBRARIES})
+target_include_directories(md5 PUBLIC ${NETTLE_INCLUDE_DIRS})
+")))
+ (call-with-output-file "third_party/md5/md5.h"
+ (lambda (port)
+ (format port "#ifndef BZF_MD5_H
+#define BZF_MD5_H
+#include <string>
+
+// Return the hexadecimal digest.
+std::string md5 (const std::string str);
+
+#endif
+")))
+ (call-with-output-file "third_party/md5/md5.cpp"
+ (lambda (port)
+ (format port "#include \"md5.h\"
+#include <nettle/md5.h>
+#include <nettle/base16.h>
+#include <string>
+#include <vector>
+
+#define OUTPUT_LENGTH BASE16_ENCODE_LENGTH (MD5_DIGEST_SIZE)
+
+std::string
+md5 (const std::string str)
+{
+ // Convert str to a byte array
+ std::vector<uint8_t> input (str.begin (), str.end ());
+
+ // Compute the digest
+ struct md5_ctx nettle_ctx;
+ md5_init (&nettle_ctx);
+ md5_update (&nettle_ctx, input.size (), input.data ());
+ uint8_t digest[MD5_DIGEST_SIZE];
+ md5_digest (&nettle_ctx, MD5_DIGEST_SIZE, digest);
+
+ // Encode it to base16
+ std::vector<char> encoded (BASE16_ENCODE_LENGTH (MD5_DIGEST_SIZE));
+ base16_encode_update (encoded.data (), MD5_DIGEST_SIZE, digest);
+ std::string final_digest (encoded.begin (), encoded.end ());
+ return final_digest;
+}
+")))))))
+ (build-system cmake-build-system)
+ (arguments
+ `(#:configure-flags '("-DUSE_SYSTEM_GTEST=ON" "-DUSE_SYSTEM_JSON=ON")))
+ (propagated-inputs
+ '())
+ (native-inputs
+ `(("googletest" ,googletest)
+ ("pkg-config" ,pkg-config)))
+ (inputs
+ `(("json-modern-cxx" ,json-modern-cxx)
+ ;; jsonnet uses a md5 implementation claiming to be from
+ ;; https://www.bzflag.org/, but they don’t use it anymore. We
+ ;; replace it with md5 from nettle.
+ ("nettle" ,nettle)))
+ (home-page "https://jsonnet.org/")
+ (synopsis "The data templating language")
+ (description "A data templating language for app and tool developers:
+- Generate config data
+- Side-effect free
+- Organize, simplify, unify
+- Manage sprawling config
+
+A simple extension of JSON
+- Open source (Apache 2.0)
+- Familiar syntax
+- Reformatter, linter
+- Editor & IDE integrations
+- Formally specified
+")
+ (license license:asl2.0)))
--
2.32.0
Vivien Kraus wrote 4 years ago
Using texinfo for the description
(address . 49823@debbugs.gnu.org)
b1abc32b7ea40f47e589a03fe3ac066f3e2ae228.camel@planete-kraus.eu
maximed told me I could use texinfo in the description.

(I also removed a couple of lines in the C++ source because the
constant was not used.)

Also, I was advised to put the added source code in a local-file such
as what is done by gnuzilla, but I could not figure out how it was done
for gnuzilla (the package is quite large). If I go with a patch, there
are a lot of things that get deleted, so it makes a rather large patch.

What do you think?

Vivien
From 7fb706951904f58cb3dd8963885024f7d9383c0b Mon Sep 17 00:00:00 2001
From: Vivien Kraus <vivien@planete-kraus.eu>
Date: Mon, 2 Aug 2021 16:07:08 +0200
Subject: [PATCH] gnu: Add jsonnet.

* gnu/packages/cpp.scm (jsonnet): New variable.
---
gnu/packages/cpp.scm | 105 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 105 insertions(+)

Toggle diff (122 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index 42e9d50687..318e387069 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -63,6 +63,7 @@
#:use-module (gnu packages llvm)
#:use-module (gnu packages logging)
#:use-module (gnu packages maths)
+ #:use-module (gnu packages nettle)
#:use-module (gnu packages onc-rpc)
#:use-module (gnu packages perl)
#:use-module (gnu packages pkg-config)
@@ -1211,3 +1212,107 @@ of reading and writing XML.")
;; incompatible with the GPL v2. Refer to the file named FLOSSE for the
;; details.
(license license:gpl2+)))
+
+(define-public jsonnet
+ (package
+ (name "jsonnet")
+ (version "0.17.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/google/jsonnet")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1ddz14699v5lqx3dh0mb7hfffr6fk5zhmzn3z8yxkqqvriqnciim"))
+ (modules '((guix build utils)))
+ (snippet
+ '(begin
+ (delete-file-recursively "third_party")
+ (delete-file-recursively "doc/third_party")
+ (substitute*
+ '("core/vm.cpp")
+ (("#include \"json.hpp\"") "#include <nlohmann/json.hpp>"))
+ (mkdir-p "third_party/md5")
+ (call-with-output-file "third_party/md5/CMakeLists.txt"
+ (lambda (port)
+ (format port "add_library(md5 STATIC md5.cpp md5.h)
+find_package(PkgConfig REQUIRED)
+pkg_check_modules(NETTLE REQUIRED nettle)
+target_link_libraries(md5 ${NETTLE_LIBRARIES})
+target_include_directories(md5 PUBLIC ${NETTLE_INCLUDE_DIRS})
+")))
+ (call-with-output-file "third_party/md5/md5.h"
+ (lambda (port)
+ (format port "#ifndef BZF_MD5_H
+#define BZF_MD5_H
+#include <string>
+
+// Return the hexadecimal digest.
+std::string md5 (const std::string str);
+
+#endif
+")))
+ (call-with-output-file "third_party/md5/md5.cpp"
+ (lambda (port)
+ (format port "#include \"md5.h\"
+#include <nettle/md5.h>
+#include <nettle/base16.h>
+#include <string>
+#include <vector>
+
+std::string
+md5 (const std::string str)
+{
+ // Convert str to a byte array
+ std::vector<uint8_t> input (str.begin (), str.end ());
+
+ // Compute the digest
+ struct md5_ctx nettle_ctx;
+ md5_init (&nettle_ctx);
+ md5_update (&nettle_ctx, input.size (), input.data ());
+ uint8_t digest[MD5_DIGEST_SIZE];
+ md5_digest (&nettle_ctx, MD5_DIGEST_SIZE, digest);
+
+ // Encode it to base16
+ std::vector<char> encoded (BASE16_ENCODE_LENGTH (MD5_DIGEST_SIZE));
+ base16_encode_update (encoded.data (), MD5_DIGEST_SIZE, digest);
+ std::string final_digest (encoded.begin (), encoded.end ());
+ return final_digest;
+}
+")))))))
+ (build-system cmake-build-system)
+ (arguments
+ `(#:configure-flags '("-DUSE_SYSTEM_GTEST=ON" "-DUSE_SYSTEM_JSON=ON")))
+ (propagated-inputs
+ '())
+ (native-inputs
+ `(("googletest" ,googletest)
+ ("pkg-config" ,pkg-config)))
+ (inputs
+ `(("json-modern-cxx" ,json-modern-cxx)
+ ;; jsonnet uses a md5 implementation claiming to be from
+ ;; https://www.bzflag.org/, but they don’t use it anymore. We
+ ;; replace it with md5 from nettle.
+ ("nettle" ,nettle)))
+ (home-page "https://jsonnet.org/")
+ (synopsis "The data templating language")
+ (description "A data templating language for app and tool developers:
+@itemize
+@item Generate config data
+@item Side-effect free
+@item Organize, simplify, unify
+@item Manage sprawling config
+@end itemize
+
+A simple extension of JSON:
+@itemize
+@item Open source (Apache 2.0)
+@item Familiar syntax
+@item Reformatter, linter
+@item Editor & IDE integrations
+@item Formally specified
+@end itemize
+")
+ (license license:asl2.0)))
--
2.32.0
Maxime Devos wrote 4 years ago
dcc495dd024e91ac850d925c83b30efee82db549.camel@telenet.be
Vivien Kraus via Guix-patches via schreef op ma 02-08-2021 om 18:15 [+0200]:
Toggle quote (9 lines)
> maximed told me I could use texinfo in the description.
>
> (I also removed a couple of lines in the C++ source because the
> constant was not used.)
>
> Also, I was advised to put the added source code in a local-file such
> as what is done by gnuzilla, but I could not figure out how it was done
> for gnuzilla (the package is quite large).

Something like this should work (untested):

(package
(origin
[...]
(snippet
#~(begin
[...]
(delete-file "third_party/md5/md5.cpp") ; maybe not necessary?
(copy-file #+(search-auxiliary-file "jsonnet-md5.cpp")
"third_party/md5/md5.cpp")
[...])))
[...])

Toggle quote (3 lines)
> If I go with a patch, there
> are a lot of things that get deleted, so it makes a rather large patch.

"git diff" has an option "--irreversible-deletes" and "--break-rewrites"
which should reduce the size of the patch, by not including the previous
version.

Toggle quote (3 lines)
> What do you think?
>
> Vivien
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYQgexhccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7vglAP9CU7VXrn0SPMhIXO537Q2pFmw4
0c0ngwWSloylqAw2JwD/enUj4vIxSIEXRp9C3ivff/Plsq/wVCthXMbBsweSeQU=
=8VgX
-----END PGP SIGNATURE-----


Vivien Kraus wrote 4 years ago
b8c23b18b9b032bce8bafb66c5dd8dd260e00015.camel@planete-kraus.eu
Le lundi 02 août 2021 à 18:35 +0200, Maxime Devos a écrit :
Toggle quote (8 lines)
> Something like this should work (untested):
>
> (package
> (origin
> [...]
> (snippet
> #~(begin

I didn’t know snippets could be Gexps. Thank you.
Leo Prikler wrote 4 years ago
80325791f76c740a20bd7ad04f9191da9299ec9a.camel@student.tugraz.at
Hi Vivien,

sorry for coaxing you into packaging this thing, I merely noticed
graaljs using jsonnet and I'm pretty sure those paths will at some
point of the build need to get parsed. ^^"

Am Montag, den 02.08.2021, 19:09 +0200 schrieb Vivien Kraus:
Toggle quote (1 lines)
> + (synopsis "The data templating language")
Should just be "Data templating language"
Toggle quote (18 lines)
> + (description "A data templating language for app and tool
> developers:
> +@itemize
> +@item Generate config data
> +@item Side-effect free
> +@item Organize, simplify, unify
> +@item Manage sprawling config
> +@end itemize
> +
> +A simple extension of JSON:
> +@itemize
> +@item Open source (Apache 2.0)
> +@item Familiar syntax
> +@item Reformatter, linter
> +@item Editor & IDE integrations
> +@item Formally specified
> +@end itemize
> +")
Sorry, but that's a little too much keyword soup. Perhaps "Jsonnet is
a templating language extending JSON syntax with variables, conditions,
functions and more."
Toggle quote (3 lines)
> + (license license:asl2.0)))

> + `(("json-modern-cxx" ,json-modern-cxx)
On a related note, we should imo rename this to nlohmann-json and patch
dependants accordingly. What do others think?
Vivien Kraus wrote 4 years ago
a6b05ab7a106bdaa7677edfcadda4a101fb9d850.camel@planete-kraus.eu
Here it is with the new wording.

Le lundi 02 août 2021 à 19:52 +0200, Leo Prikler a écrit :
Toggle quote (3 lines)
> On a related note, we should imo rename this to nlohmann-json and
> patch
> dependants accordingly. What do others think?
Ludovic Courtès wrote 4 years ago
Re: bug#49823: [PATCH] gnu: Add jsonnet.
(name . Vivien Kraus)(address . vivien@planete-kraus.eu)
878s19sbgf.fsf@gnu.org
Hi Vivien,

Vivien Kraus <vivien@planete-kraus.eu> skribis:

Toggle quote (12 lines)
> Here is jsonnet. I don’t fully understand what it does, but thanks to
> leoprikler, I know it is a dependency to package GraalJS, an
> interpreter for JavaScript on the Java Virtual Machine.
>
> For the sake of having "no dependencies", it bundles a custom
> implementation of MD5. According to a comment in the associated
> license, it is taken from the implementation of bzflag, but it does not
> seem to use the same as of today.
>
> I decided to use nettle, and add a few lines of C++ to shape it into
> the required interface.

I think using Nettle is a wise decision; however, it’s a decision for
upstream to make IMO. I’m not comfortable with shipping custom
md5.{cpp,h} and CMakeLists.txt; to me, we’d be taking a bit too much of
upstream’s burden and delivering something that’s quite different.

Apart from that, the last patch you sent LGTM.

Thoughts?

Thanks,
Ludo’.
Vivien Kraus wrote 4 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)
874kbx2x3e.fsf@planete-kraus.eu
Hi!

Ludovic Courtès writes:
Toggle quote (13 lines)
>> For the sake of having "no dependencies", it bundles a custom
>> implementation of MD5. According to a comment in the associated
>> license, it is taken from the implementation of bzflag, but it does not
>> seem to use the same as of today.
>>
>> I decided to use nettle, and add a few lines of C++ to shape it into
>> the required interface.
>
> I think using Nettle is a wise decision; however, it’s a decision for
> upstream to make IMO. I’m not comfortable with shipping custom
> md5.{cpp,h} and CMakeLists.txt; to me, we’d be taking a bit too much of
> upstream’s burden and delivering something that’s quite different.

OK. Should I keep the bundled MD5 implementation then?

Vivien
Ludovic Courtès wrote 4 years ago
(name . Vivien Kraus)(address . vivien@planete-kraus.eu)
87k0kspan5.fsf@gnu.org
Hi,

Vivien Kraus <vivien@planete-kraus.eu> skribis:

Toggle quote (16 lines)
> Ludovic Courtès writes:
>>> For the sake of having "no dependencies", it bundles a custom
>>> implementation of MD5. According to a comment in the associated
>>> license, it is taken from the implementation of bzflag, but it does not
>>> seem to use the same as of today.
>>>
>>> I decided to use nettle, and add a few lines of C++ to shape it into
>>> the required interface.
>>
>> I think using Nettle is a wise decision; however, it’s a decision for
>> upstream to make IMO. I’m not comfortable with shipping custom
>> md5.{cpp,h} and CMakeLists.txt; to me, we’d be taking a bit too much of
>> upstream’s burden and delivering something that’s quite different.
>
> OK. Should I keep the bundled MD5 implementation then?

Yeah, I think so. Clearly it’s a gray area, there are pros and cons,
but I’d lean towards minimal changes to upstream source.

Thanks,
Ludo’.
Vivien Kraus wrote 4 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)
87y2981e8x.fsf@planete-kraus.eu
Ludovic Courtès writes:
Toggle quote (5 lines)
>> OK. Should I keep the bundled MD5 implementation then?
>
> Yeah, I think so. Clearly it’s a gray area, there are pros and cons,
> but I’d lean towards minimal changes to upstream source.

Here you go :)
From 8ce461001ed14c3267d839334e4853feaed69b50 Mon Sep 17 00:00:00 2001
From: Vivien Kraus <vivien@planete-kraus.eu>
Date: Mon, 2 Aug 2021 16:07:08 +0200
Subject: [PATCH] gnu: Add jsonnet.

* gnu/packages/cpp.scm (jsonnet): New variable.
---
gnu/packages/cpp.scm | 41 +++++++++++++++++++++++++++++++++++++++++
1 file changed, 41 insertions(+)

Toggle diff (58 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index 42e9d50687..870c02dac3 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -45,6 +45,7 @@
#:use-module (guix build-system gnu)
#:use-module (guix build-system python)
#:use-module (guix modules)
+ #:use-module (guix gexp)
#:use-module (gnu packages)
#:use-module (gnu packages autotools)
#:use-module (gnu packages boost)
@@ -1211,3 +1212,43 @@ of reading and writing XML.")
;; incompatible with the GPL v2. Refer to the file named FLOSSE for the
;; details.
(license license:gpl2+)))
+
+(define-public jsonnet
+ (package
+ (name "jsonnet")
+ (version "0.17.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/google/jsonnet")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1ddz14699v5lqx3dh0mb7hfffr6fk5zhmzn3z8yxkqqvriqnciim"))
+ (modules '((guix build utils)))
+ (snippet
+ #~(begin
+ (rename-file "third_party/md5" ".md5")
+ (delete-file-recursively "third_party")
+ (delete-file-recursively "doc/third_party")
+ (substitute*
+ '("core/vm.cpp")
+ (("#include \"json.hpp\"") "#include <nlohmann/json.hpp>"))
+ (mkdir "third_party")
+ (rename-file ".md5" "third_party/md5")))))
+ (build-system cmake-build-system)
+ (arguments
+ `(#:configure-flags '("-DUSE_SYSTEM_GTEST=ON" "-DUSE_SYSTEM_JSON=ON")))
+ (propagated-inputs
+ '())
+ (native-inputs
+ `(("googletest" ,googletest)
+ ("pkg-config" ,pkg-config)))
+ (inputs
+ `(("json-modern-cxx" ,json-modern-cxx)))
+ (home-page "https://jsonnet.org/")
+ (synopsis "Data templating language")
+ (description "Jsonnet is a templating language extending JSON
+syntax with variables, conditions, functions and more.")
+ (license license:asl2.0)))
--
2.32.0
Ludovic Courtès wrote 4 years ago
(name . Vivien Kraus)(address . vivien@planete-kraus.eu)
87im0bhze6.fsf@gnu.org
Hi Vivien,

Vivien Kraus <vivien@planete-kraus.eu> skribis:

Toggle quote (7 lines)
> From 8ce461001ed14c3267d839334e4853feaed69b50 Mon Sep 17 00:00:00 2001
> From: Vivien Kraus <vivien@planete-kraus.eu>
> Date: Mon, 2 Aug 2021 16:07:08 +0200
> Subject: [PATCH] gnu: Add jsonnet.
>
> * gnu/packages/cpp.scm (jsonnet): New variable.

Applied with the tweak below.

Thank you, and thanks for taking the time for the extra work!

Ludo’.
Toggle diff (22 lines)
diff --git a/gnu/packages/cpp.scm b/gnu/packages/cpp.scm
index 870c02dac3..8c8109fb83 100644
--- a/gnu/packages/cpp.scm
+++ b/gnu/packages/cpp.scm
@@ -1232,16 +1232,13 @@ of reading and writing XML.")
(rename-file "third_party/md5" ".md5")
(delete-file-recursively "third_party")
(delete-file-recursively "doc/third_party")
- (substitute*
- '("core/vm.cpp")
+ (substitute* '("core/vm.cpp")
(("#include \"json.hpp\"") "#include <nlohmann/json.hpp>"))
(mkdir "third_party")
(rename-file ".md5" "third_party/md5")))))
(build-system cmake-build-system)
(arguments
`(#:configure-flags '("-DUSE_SYSTEM_GTEST=ON" "-DUSE_SYSTEM_JSON=ON")))
- (propagated-inputs
- '())
(native-inputs
`(("googletest" ,googletest)
("pkg-config" ,pkg-config)))
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 49823@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 49823
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help