Andreas Enge wrote 6 years ago
(address . bug-guix@gnu.org)
20190218200552.GB1881@jurong
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
- 6 years ago
Guix profile fails on Overdrive 1000
- Done
Details
- 5 participants
- Andreas Enge
- Danny Milosavljevic
- Ludovic Courtès
- Marius Bakke
- Ricardo Wurmus
- Owner
- unassigned
- Submitted by
- Andreas Enge
- Severity
- normal
Marius Bakke wrote 6 years ago
87mumsom9l.fsf@fastmail.com
Andreas Enge <andreas@enge.fr> writes:
Toggle quote (33 lines)
> Hello,
>
> on a newly installed Overdrive 1000 machine with 8 GB of memory, I am trying
> guix pull -n
>
> This results in
> Updating channel 'guix' from Git repository at 'https://git.savannah.gnu.org/git/guix.git'...
> Building from this channel:
> guix https://git.savannah.gnu.org/git/guix.git d0d207c
> The following derivations will be built:
> /gnu/store/qvljgqnsynw113h1062nws9i2zfcz15y-compute-guix-derivation.drv
> /gnu/store/rj824ibc8nhhl1j2h25sgw4lyxkxjggc-gdb-8.2.drv
> ...
> /gnu/store/w1jw283cdf6n03qnccsrvxgzy41z0hps-guile-gcrypt-0.1.0.drv
> building /gnu/store/wy37294j2rb6fg5fgpybki28vq60jz94-python-minimal-3.6.5.drv...
> starting phase `set-SOURCE-DATE-EPOCH'
> ...
>
> That a build of something starts although the "-n" flag is specified seems
> to be a first bug.
>
> Then, during the test phase, the build is killed since the machine runs out
> of memory. I also tried to pass "--cores=1" to "guix build", but this
> parameter is not honoured: "top" shows that all four cores are running.
> This might be a second bug.
>
> In any case, it becomes impossible to upgrade with the usual mechanism,
> which is a rather serious bug.
>
> My guix version:
> $ guix --version
> guix (GNU Guix) 0.15.0-8.71a78ba
The Python contained within this Guix snapshot has a known bug that
makes it leak memory on newer kernels. The Guix commit that works
around it is 589aca1c183ef1dfdef54d40fdd6a258bbcd39d0.
One thing you could try is bootstrap a Guix checkout through the usual
mechanisms (./bootstrap; ./configure; etc), using packages from the host
system if necessary.
From there you should be able to "./pre-inst-env guix pull".
A dirtier workaround is to delete
"/tmp/guix-build-python-minimal-3.6.5.drv-0/Lib/test/test_socket.py" at
some point during the build process, but you may have to do that a
couple of times.
Finally, installing 0.16.0 "from scratch" should also work.
HTH!
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAlxrGkYACgkQoqBt8qM6
VPpGXAgAyjE4PBegBmz7nZ9oqPkeRt7daBLQZzghxUcT6y3DiP9M9Qax4Tlc8dBo
SzuOm7jvtPMjt7diLZfWLbj/IS8HoDwNKmiqUw4RZRovvVV6j6WKc5u6Q9UGdakX
To9PoRgR6cmvOV+vb+cQm4xVPNouA6svw5PpZ3c+v67ii3iIxodtHb9jU6ACt9/j
kwN4IWOvYAw+ua9dL/Tbl6Hl/eP0+cAOwwnWNDlPsGCDODCmQaq++T72+E+d3odW
1Q7py0vNl6zs3SLDVjsmAf0JCcAyjnuayubIWNn9BvpxwPVWgw+5Bd4s9kTBKX75
c2IirUdAnaY/32xCr1RSrmR7HJmQVw==
=faAR
-----END PGP SIGNATURE-----
Andreas Enge wrote 6 years ago
(name . Marius Bakke)(address . mbakke@fastmail.com)(address . 34531@debbugs.gnu.org)
20190219082728.GA5650@jurong
Hello Marius,
thanks a lot for your quick and helpful reply!
On Mon, Feb 18, 2019 at 09:49:10PM +0100, Marius Bakke wrote:
Toggle quote (4 lines)
> The Python contained within this Guix snapshot has a known bug that
> makes it leak memory on newer kernels. The Guix commit that works
> around it is 589aca1c183ef1dfdef54d40fdd6a258bbcd39d0.
okay, so the main problem is fixed then.
Toggle quote (4 lines)
> One thing you could try is bootstrap a Guix checkout through the usual
> mechanisms (./bootstrap; ./configure; etc), using packages from the host
> system if necessary.
The host system is Guix, so this is complicated... But I realise that
there is also a guix-0.16.0 lying around, and with this I get beyond the
python point.
Before closing the bug, I would still like to ask whether there is a way
of getting around the need for python. And why "guix pull -n" builds
such a large number of packages, while one would expect it to just print
a number of packages to be built. But maybe this is also solved in a
later version already.
Andreas
Ricardo Wurmus wrote 6 years ago
(name . Andreas Enge)(address . andreas@enge.fr)
87lg2bvrne.fsf@elephly.net
Andreas Enge <andreas@enge.fr> writes:
Toggle quote (3 lines)
> Before closing the bug, I would still like to ask whether there is a way
> of getting around the need for python.
Guix pull needs “guile-git”, and “guile-git” needs “libgit2”, which
needs “python-wrapper”. To remove the need for Python in “guix pull” we
would need to build libgit2 without Python. I don’t know if anyone has
investigated whether this can be done.
Toggle quote (4 lines)
> And why "guix pull -n" builds
> such a large number of packages, while one would expect it to just print
> a number of packages to be built.
This is likely due to grafts. When you do “guix pull --no-grafts -n”
the behaviour is as expected.
--
Ricardo
Danny Milosavljevic wrote 6 years ago
(address . 34531@debbugs.gnu.org)
20190219161954.47234638@scratchpost.org
Hi Ricardo,
Hi Andreas,
On Tue, 19 Feb 2019 14:23:23 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:
Toggle quote (10 lines)
> Andreas Enge <andreas@enge.fr> writes:
>
> > Before closing the bug, I would still like to ask whether there is a way
> > of getting around the need for python.
>
> Guix pull needs “guile-git”, and “guile-git” needs “libgit2”, which
> needs “python-wrapper”. To remove the need for Python in “guix pull” we
> would need to build libgit2 without Python. I don’t know if anyone has
> investigated whether this can be done.
I have read it now--in the interest of simplicity and bootstrappability.
Python is only required to generate some libgit2 tests. It would be easy
to port the one Python program "generate.py" to some other language.
"generate.py" is from clar, which appears to be https://github.com/vmg/clar.
If someone were to replace it, that would be the correct place to contribute
it.
Python is listed as a regular input in libgit2.
I think it should be a native-input instead.
The dependency graph of libgit2 is very small after removing Python.
The languages it still depends on (possibly indirectly) are:
* Perl 5.28.0
* Guile 2.0.14
* bash 4.4.23
So writing the generate.py in one of those languages would not make the
dependency graph bigger than necessary.
generate.py uses regular expressions, simple file I/O, string templating
and that's pretty much it.
In any case, the first simplification would be just to move Python from
regular input to native-input.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlxsHpoACgkQ5xo1VCww
uqU5bwf/ZVdTP3KBqntQfeoImWzsRk8v0M3Z7YO3tBKl2gbPqxgsNZoswcukZRCe
T6Q1GVOAaZY+3/BdkrpCj5VC3tUgGvu/OV4BFGx/gnlx6RRzmYEUYpB7yhU2FdBO
cY7+/TwjSR+eXjKEyPF9Rjo//oglzMJ2vGuhHvD935R5V3/B36L9C3aXwR+OQeYc
3CBKhuqBWxVpk9unrwMNYGF81pH3pAQCcaQillgBhzWBSy2s2VdQoMn0TNAH5kQA
gc0ibM4+JVZemiXVaA/lmDv4boOniYdRj7BSRZYxPHz+iBAOxsTEX1DjJs/FbEOi
pbDoxXKiH8SHCvMhyPxxd0yfC23yzA==
=OiQs
-----END PGP SIGNATURE-----
Andreas Enge wrote 6 years ago
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20190219153553.GA4950@jurong
On Tue, Feb 19, 2019 at 02:23:23PM +0100, Ricardo Wurmus wrote:
Toggle quote (5 lines)
> Guix pull needs “guile-git”, and “guile-git” needs “libgit2”, which
> needs “python-wrapper”. To remove the need for Python in “guix pull” we
> would need to build libgit2 without Python. I don’t know if anyone has
> investigated whether this can be done.
I tried to simply drop python-wrapper from inputs. Actually it is not
referenced by the final output, so it should be in native-inputs. And
here is what happens:
-- Could NOT find PythonInterp (missing: PYTHON_EXECUTABLE)
CMake Error at tests/CMakeLists.txt:4 (MESSAGE):
Could not find a python interpeter, which is needed to build the tests.
Make sure python is available, or pass -DBUILD_CLAR=OFF to skip building
the tests
So we could create a separate package for internal guix use without running
the tests, which would save us from compiling and especially testing python.
Although this is not our usual style - but on the other hand, "guix pull"
is more or less the first command that we recommend to our users, and
it would be nice if it ran fast without requiring (many) additional packages.
Ironically, libgit2 advertises itself as needing "Zero Dependencies" on its
web site...
What do you think?
Andreas
Danny Milosavljevic wrote 6 years ago
(name . Andreas Enge)(address . andreas@enge.fr)
20190219164042.593a6990@scratchpost.org
Hi Andreas,
On Tue, 19 Feb 2019 16:35:53 +0100
Andreas Enge <andreas@enge.fr> wrote:
Toggle quote (10 lines)
> I tried to simply drop python-wrapper from inputs. Actually it is not
> referenced by the final output, so it should be in native-inputs. And
> here is what happens:
>
> -- Could NOT find PythonInterp (missing: PYTHON_EXECUTABLE)
> CMake Error at tests/CMakeLists.txt:4 (MESSAGE):
> Could not find a python interpeter, which is needed to build the tests.
> Make sure python is available, or pass -DBUILD_CLAR=OFF to skip building
> the tests
Toggle diff (51 lines)
diff --git a/gnu/packages/version-control.scm b/gnu/packages/version-control.scm
index b8e967bf6..7452f17b8 100644
--- a/gnu/packages/version-control.scm
+++ b/gnu/packages/version-control.scm
@@ -545,7 +545,8 @@ everything from small to very large projects with speed and efficiency.")
(build-system cmake-build-system)
(outputs '("out" "debug"))
(arguments
- `(#:configure-flags '("-DUSE_SHA1DC=ON") ; SHA-1 collision detection
+ `(#:configure-flags '("-DUSE_SHA1DC=ON" ; SHA-1 collision detection
+ "-DBUILD_CLAR=OFF")
#:phases
(modify-phases %standard-phases
(add-after 'unpack 'fix-hardcoded-paths
@@ -558,11 +559,13 @@ everything from small to very large projects with speed and efficiency.")
#t))
;; Run checks more verbosely.
(replace 'check
- (lambda _ (invoke "./libgit2_clar" "-v" "-Q"))))))
+ (const #t)
+ ))))
(inputs
`(("libssh2" ,libssh2)
("http-parser" ,http-parser)
- ("python" ,python-wrapper)))
+; ("python" ,python-wrapper)
+))
(native-inputs
`(("pkg-config" ,pkg-config)))
(propagated-inputs
> So we could create a separate package for internal guix use without running
> the tests, which would save us from compiling and especially testing python.
Please, let's just replace the test generator in this case. It's ridiculous
to complicate Guix that much for a 250 line test generator, most of which is
templates.
If necessary, I can do it, although a person who knows Perl or Guile better
can probably do it much faster.
> Although this is not our usual style - but on the other hand, "guix pull"
> is more or less the first command that we recommend to our users, and
> it would be nice if it ran fast without requiring (many) additional packages.
>
> Ironically, libgit2 advertises itself as needing "Zero Dependencies" on its
> web site...
>
> What do you think?
Just replace the one script and do run the tests, without depending on Python.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlxsI3oACgkQ5xo1VCww
uqUvggf/XeLw6unBrgyS1sT+T1ap1V5J2eWyL2S4BLDxeUEDeT/iuuMN55SshJ7O
XN0U+UTIQQ/R7BEYY8KeUIorLJFAfcKBFoIUdNmzaSPqU7fgb5uR3S10Rt1c2Ndd
dLe3gSBRE08qaAdIlZRmyi+25m2khw9QDeV93dYv8rDjA3AYiWYxbJkEw5LdNi0r
D7Q4gLNm4QDmaEW9VirzayLmXiEYVQjoKGEsE3nR/ilnilgJmcsJnIRanoAbMgZU
kkE/CUXkkXqB5a7iOTUdOdl30rJn8Gcc2tDib8qgUXexitPHbImZ4anEvoLLYg/Y
2y2Jp7WRPuxOv8is8VxD0xSSjWNM1Q==
=yeEl
-----END PGP SIGNATURE-----
Danny Milosavljevic wrote 6 years ago
(address . 34531@debbugs.gnu.org)
20190220115136.422bdf5b@scratchpost.org
I've started on implementing a replacement test generator, see attachment.
Usage is exactly the same as the original generate.py.
However, it hangs somewhere and I can't find how to step through a guile program, even in emacs.
Help?
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlxtMTgACgkQ5xo1VCww
uqX2bwgAgOH8T5Kvv/kuMQf+NJJKNq+SBKydQ3plUsCwRIb9Pecvo1B30OdfDSmz
YcMYYXANlFuqt45GcRiSO223/Vh2/a1Qu8GHNvM2/ycvzX/+IMAX7wh2SFpJs+pk
9IgaHU1Guxq8OSHzDN8suQXsfZByOY2Wmd+9bggxVw13RkBS/+iXy+mT5h99HG6i
6BKI430Y73pRi02UKFu/7Y253NKLVqgdHyuzwVqC5xViZUda0S/7YZ+wmZhF9iM3
lxkBKWr1dmdUEznsSz57udLU+HHVspT5jT7LCfKtJRQSsdigeHWhDt4m3njInoRh
YeijMGIAZzYpw2of0UY+F0BZfJ5C8A==
=/4nE
-----END PGP SIGNATURE-----
Ricardo Wurmus wrote 6 years ago
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
87va1eu1gp.fsf@elephly.net
Danny Milosavljevic <dannym@scratchpost.org> writes:
Toggle quote (2 lines)
> I've started on implementing a replacement test generator, see attachment.
Excellent!
Toggle quote (2 lines)
> Usage is exactly the same as the original generate.py.
Can you show us how to use it? What files does a proper test require?
--
Ricardo
Danny Milosavljevic wrote 6 years ago
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20190220142634.56868dba@scratchpost.org
Hi Ricardo,
unpack libgit2-0.27.7, then invoke
guile generate.scm -o /tmp -f -x online -x stress -x perf .
in there.
Reference:
./tests/CMakeLists.txt: COMMAND ${PYTHON_EXECUTABLE} generate.py -o "${CMAKE_CURRENT_BINARY_DIR}" -f -xonline -xstress -xperf .
(in libgit2)
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlxtVYoACgkQ5xo1VCww
uqUh5gf/f2cbfgCQYXuAfy+IdOSFX/pLR2L6JGg1qOnwsrRUve98kTzwbHDiPnqa
sYiaGemBu+0krTmDMHCQ5ILvKvSIWhTggmGbxwzsB8En6QDA/eLXGuFKFd9BeWQY
eyPvkbQV0K03uLW60LHZI4FCMt+O+BvQs9pMC6VYBlN/ExpYE2SPc6yw9I7JHcUT
Sz57bAn4KlojHg9vlZpoJHCfMPj7/icm6fQsxnxgBHhmQaQnykOuqGuHePZf34Xb
usLv0YpSPKkaCQoBLMAGYfO8Nvgx1eNKukBmB/kkK4M2JMGXW5cze/VoM1YmirW4
6gjWfTDDPlqGXIgQfWVg0fTp+POEsg==
=x9tW
-----END PGP SIGNATURE-----
Danny Milosavljevic wrote 6 years ago
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20190220142811.6add03bf@scratchpost.org
Toggle quote (2 lines)
> What files does a proper test require?
generate.scm scans the files with the glob "*.c" for functions test_... and
refers all of those in a new file "clar.suite" in the directory specified
by "-o".
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlxtVesACgkQ5xo1VCww
uqX4oAf7BiXK4olMgf/LmI56XN/W57NCc8rYQTvq/roA4i5zCymOvDk66LnRY2gf
yePbsrJIm/GzP0Mp9QbxrxE+rl4aIB7Cd4dY2v7K8Zf7wp+HV6xJNLcyLovMJ1dB
/nFAe3cgXcU8qI9C3aoDXDboCDzqfinK6gy84O9G/oHPvqVtZOunLThU9cLFPyfc
OWlXyE7ubp/Zxu5jr9b/iB1U2+LuCGo4IFnegexGU3gt7DmDcFVk4VlTE6WQAuUa
N59uSHgjm/TtGD0KKjx0vKIzjTdC3xYSaZKlZk/Bi5riClMXZeA3czl+P/ZL5GmS
yjVH3tkPkJLMLqJG0KBC30XHnnUodQ==
=e1vb
-----END PGP SIGNATURE-----
Ricardo Wurmus wrote 6 years ago
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
87k1hutpvw.fsf@elephly.net
Hi Danny,
Toggle quote (6 lines)
> unpack libgit2-0.27.7, then invoke
>
> guile generate.scm -o /tmp -f -x online -x stress -x perf .
>
> in there.
Thanks for the instructions.
One of the problems is that you use and also overwrite “write”,
which is a core Scheme procedure. I renamed it to “Write” and modified
the invocation to
(call-with-output-file (string-append output "/clar.suite") Write)
which leads to this error:
Toggle snippet (15 lines)
…
In ice-9/ftw.scm:
482:39 4 (loop _ _ #(64768 139272 16877 51 1000 1000 0 4096 1550677456 0 1550677457 4096 8 directory 493 818358318 0 1550677457) _ _)
In ice-9/eval.scm:
293:34 3 (_ #(#(#(#(#(#<directory (guile-user) 717140> "." ("perf" "stress" "online")) "./tests/clar_libgit2_timer.c" #(64768 132360 33188 1 1000 1000 0 …) …) …) …) …))
191:35 2 (_ #(#(#<directory (guile-user) 717140> #<variable 8e4eb0 value: #<procedure parse (a)>>) "./tests/clar_libgit2_timer.c"))
In ice-9/rdelim.scm:
160:18 1 (read-string _)
In unknown file:
0 (read-char "./tests/clar_libgit2_timer.c")
ERROR: In procedure read-char:
In procedure read-char: Wrong type argument in position 1 (expecting open input port): "./tests/clar_libgit2_timer.c"
Simple mistake: you called “get-string-all” on a string (stored in
“path”), but you need to use (call-with-input-file path get-string-all)
instead.
Next error is about a bad regex (forgot one escape character); and
a type error follows that.
The fixed version is attached.
--
Ricardo
Danny Milosavljevic wrote 6 years ago
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20190220172603.1a675f61@scratchpost.org
Hi Ricardo,
On Wed, 20 Feb 2019 16:56:35 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:
Toggle quote (2 lines)
> The fixed version is attached.
Thanks a lot!
It almost works, but it seems not to find the "test_" lines.
I've reduced the problem to:
scheme@(guile-user)> ,use (ice-9 regex)
scheme@(guile-user)> (fold-matches "^void" "blah\nvoid\n" '() cons)
$24 = ()
After reading the documentation, I've revised it to:
scheme@(guile-user)> ,use (ice-9 regex)
scheme@(guile-user)> (fold-matches "^void" "blah\nvoid\n" '() cons regexp/newline)
$25 = ()
Huh?
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlxtf5sACgkQ5xo1VCww
uqVeYwf/aUVQE+sMF0Pb1ShOd7rqfxul1KJ9Df2UUSTwJ9PihshZHcDALpEQ3M72
mEOBwNH5iY4kqaKB0gWHoCU+InUEp5Ggd3+4rfBbJvxxjBsXNWmegcUe/Tf7pEf0
t9JVgDXymPSZpW3e09E8EMCRvRViIp2pWh4BxDMzfjCR6iyhzmronReVh1Vx1VWU
tVZqnzAf0xHCwsUr1wMVLepLH6tM1hOrwRPAK/e3m18et2RGF0hPH2yU4C6jh2u0
oBtrleHB1sI29OsGffBSrWRURgXWbNurUknAcTWCEz111L+2TfQmQ/tnn1x1xnHh
tpOG3lyIIlEDlGo2edJJ19msOAdu7g==
=VnTa
-----END PGP SIGNATURE-----
Ricardo Wurmus wrote 6 years ago
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
87a7iq5gh8.fsf@elephly.net
Danny Milosavljevic <dannym@scratchpost.org> writes:
Toggle quote (14 lines)
> It almost works, but it seems not to find the "test_" lines.
>
> I've reduced the problem to:
>
> scheme@(guile-user)> ,use (ice-9 regex)
> scheme@(guile-user)> (fold-matches "^void" "blah\nvoid\n" '() cons)
> $24 = ()
>
> After reading the documentation, I've revised it to:
>
> scheme@(guile-user)> ,use (ice-9 regex)
> scheme@(guile-user)> (fold-matches "^void" "blah\nvoid\n" '() cons regexp/newline)
> $25 = ()
Try this instead:
(fold-matches (make-regexp "^void" regexp/newline)
"blah\nvoid\n" '() cons)
Or rather
(list-matches (make-regexp "^void" regexp/newline)
"blah\nvoid\n")
--
Ricardo
Danny Milosavljevic wrote 6 years ago
(address . 34531@debbugs.gnu.org)
20190220230857.2282d9b1@scratchpost.org
First somewhat working version attached...
It finds 1387 tests in 328 suites.
The original finds 2611 tests in 349 suites.
That's because skip-comments is somehow broken.
An example where it has an error very early is tests/core/futils.c :
(use-modules (ice-9 ftw))
(use-modules (ice-9 regex))
(use-modules (ice-9 getopt-long))
(use-modules (ice-9 rdelim))
(use-modules (ice-9 match))
(use-modules (ice-9 textual-ports))
(use-modules (srfi srfi-1))
(define fname "test/core/futils.c")
(define s (call-with-input-file fname get-string-all))
(define (replace needle replacement haystack)
"Replace all occurences of NEEDLE in HAYSTACK by REPLACEMENT.
NEEDLE is a regular expression."
(regexp-substitute/global #f needle haystack 'pre replacement 'post))
(define (skip-comments text)
(replace (string-append "//[^\n]*?|"
"/[*].*?[*]/|"
"'([.]|[^'])*?'|"
"\"([.]|[^\"])*?\"")
"" text))
(write (skip-comments s)) ; very short, for some reason
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlxtz/kACgkQ5xo1VCww
uqU1swf+Nx7zNqgtixg3I8YGIHyHJ0Ve1sUMOgoSDgQu/cyAZkVyeWefozSTen5P
3DS4IoYTbjg4FnZkqtX5udW80H6iyoxA82AxkXN9CT9nEXMDlLjwvxZzY8pUxIUp
KO5VCSAsNuj9LTfu/RC2gtGdzuw+h2QTejlhEYu7eiBtA8eih3WYu975ZtjdZyaO
J6xwmzpQauB6VlUmgn7TmUTm3jKkR2KlKGa5v26pZfb6eYQKv6sJSXt5FZkElfOz
+AU4KnTlqKuu03TmBtoUyBRQ0tL0GCwJHP5fbuW/Dq/PNoGE3WPViEgf4K6hGjuJ
bXDtpJm3e+l74rCco1FkR9W6iQr20A==
=1MH8
-----END PGP SIGNATURE-----
Danny Milosavljevic wrote 6 years ago
(address . 34531@debbugs.gnu.org)
20190220232833.3a437681@scratchpost.org
On Wed, 20 Feb 2019 23:08:57 +0100
Danny Milosavljevic <dannym@scratchpost.org> wrote:
Toggle quote (8 lines)
> First somewhat working version attached...
>
> It finds 1387 tests in 328 suites.
>
> The original finds 2611 tests in 349 suites.
>
> That's because skip-comments is somehow broken.
Another example is tests/repo/pathspec.c where the vast majority of the file is missing.
In any case, leaving the call of skip-comments off, it finds 2611 tests in 349 suites (see attachment).
So either we find the problem in skip-comments, or we leave it off. In both cases, this can be used in order to generate the test metadata for libgit2 now.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlxt1JEACgkQ5xo1VCww
uqXWAAf/TN3c93zRwvLvV3Udw4SkYcgo20Ln0LbJWmu7J6FRWyOsNxlvkqvpUEd1
1IFO231WI1vsQ/OkeEWeqYvHMSOm+6zeBwUZwNwOoxUX11m7Trm0WOCvV/TsfiWU
b3/8oifnKvYLfXgjP0AOZ5R/z+JVABPuOB8sGLLGf0JxrZ8+6/rVb2uyazsYm7T5
D7ITKmjb4UxcDB5Vva3pVBCRV2gNPjGps+THIvBM87Shm9caB/0jsFLEu/jntgjF
70ixuOd6frxFz3F07YuoXgYghJPnvdoiujAIZuFRRUTT+6y9Htrc/uVZeXSVaUFP
7jacGvkZDkkgDsBAaFGRCPcAbg+XAw==
=+gkR
-----END PGP SIGNATURE-----
Ricardo Wurmus wrote 6 years ago
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
87ef7y3g61.fsf@elephly.net
Danny Milosavljevic <dannym@scratchpost.org> writes:
Toggle quote (6 lines)
> (define (skip-comments text)
> (replace (string-append "//[^\n]*?|"
> "/[*].*?[*]/|"
> "'([.]|[^'])*?'|"
> "\"([.]|[^\"])*?\"")
This last part will remove anything between double quotes, such as every
string. Another problem seems to be the handling of /* comments */.
It’s a greedy regex that will swallow any character until it finally
hits */ — even if the characters in between may have been */.
I would not use regular expressions here but a read loop such as this:
Toggle snippet (34 lines)
(define (strip-comments port)
(let loop ((char (get-char port))
(balance 0)
(chars '()))
(cond
;; done!
((eof-object? char)
(list->string (reverse chars)))
;; line comment
((and (equal? char #\/)
(equal? #\/ (peek-char port)))
(begin (read-line port)
(loop (get-char port) balance chars)))
;; begin of block comment
((and (equal? char #\/)
(equal? #\* (peek-char port)))
(begin (read-char port)
(loop (get-char port) (1+ balance) chars)))
;; end of block comment
((and (positive? balance)
(equal? char #\*)
(equal? #\/ (peek-char port)))
(begin (read-char port)
(loop (get-char port) (1- balance) chars)))
;; inside block comment
((positive? balance)
(loop (get-char port) balance chars))
;; just a plain ol’ character
(else (loop (get-char port) balance (cons char chars))))))
;; Strip all comments from the file “fname” and show the result.
(display (call-with-input-file fname strip-comments))
--
Ricardo
Danny Milosavljevic wrote 6 years ago
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20190224114034.5f339ae3@scratchpost.org
Final version attached. Works fine now.
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlxydKIACgkQ5xo1VCww
uqWRrgf/ZVYQ0le+CZutV7yguT5yiC7B0RL5sA1v89Erkfli/B/phAnZPUnZ2skv
hVB2Ku8zIh7tZkmTf9bnHWlEp+H6SdsaxTGiJLjbNzIsu4NuBlKxiXHRo5qrmiTC
opk//7JgEEqO7ICVGt5PYH5i0zfJ5B6dtZ4V34JsaMZb4yWw1C9u1o30GfsqvWOK
JYuSu79d7D8KH2scMQzv1/BDYTa0lKrLtBYLsqbSZD5/VHfr0GS1lPYW75YYe69z
Zyg+HUcPNhM5ZmFsb1yIyLA4D9uB5NkNDAVVwgyKFoYhrbunX+ft9zg6JUfT5L8d
EiO7SKc4TGZ+QxVQ7fPwHs6kPfDElg==
=+pIw
-----END PGP SIGNATURE-----
Ricardo Wurmus wrote 6 years ago
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
878sy52yw1.fsf@elephly.net
Danny Milosavljevic <dannym@scratchpost.org> writes:
Toggle quote (2 lines)
> Final version attached. Works fine now.
The loop looks a bit more complicated than it needs to be, I think. Did
my version not work for you?
--
Ricardo
Danny Milosavljevic wrote 6 years ago
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20190224131204.21fdc1e5@scratchpost.org
Hi Ricardo,
On Sun, 24 Feb 2019 12:45:34 +0100
Ricardo Wurmus <rekado@elephly.net> wrote:
Toggle quote (7 lines)
> Danny Milosavljevic <dannym@scratchpost.org> writes:
>
> > Final version attached. Works fine now.
>
> The loop looks a bit more complicated than it needs to be, I think. Did
> my version not work for you?
It did, but I wanted to make sure the port did exactly the same as the
original generate.py--maybe I overdid it, but I didn't want to break it
by porting it.
What skip_comments in the original does is strip comments, but not strip
"comment-like things" that are in string literals ("/*blah*/").
(as far as I can tell, at least)
I agree your version is easier but does it do the same thing?
Attached a v2 where I fixed a bug in literal handling related to that (oops).
WDYT?
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAlxyihcACgkQ5xo1VCww
uqUs5Af/f7aY+mEgiIxPAqTB6eND4vLo4XF0JdSQI5ZUKiQLJS4AlZhMeBY9QMh+
BRzF0nQqx+uNSARZ4xicAnDSPrKb+8xEeDNBmpxzeGNn889J/mx2TtQDVRdFpZDh
B4edM97tpseoTKMZmKn9u+JrlZVIjA2VJ81iXF9z5WiCb0T6RUmDhszFk+V8+8Ce
xFnz6jZr3WuVM9aeqTu9ehiOIHyWdpMwdXS973+9PABYRl9bmOL9fw5iKGy1XJLm
v0r/yDLQYkYOmxLUXq4uEPBrCbuLAhh/Id1eQW6imW0orCVIVXg2zgJDnhv1l9ty
PRJopkHQ6En1PmO54t8qnl2sYAIqgA==
=RV1a
-----END PGP SIGNATURE-----
Ricardo Wurmus wrote 6 years ago
(name . Danny Milosavljevic)(address . dannym@scratchpost.org)
87mumiz2bp.fsf@elephly.net
Hi Danny,
Toggle quote (11 lines)
>> The loop looks a bit more complicated than it needs to be, I think. Did
>> my version not work for you?
>
> It did, but I wanted to make sure the port did exactly the same as the
> original generate.py--maybe I overdid it, but I didn't want to break it
> by porting it.
>
> What skip_comments in the original does is strip comments, but not strip
> "comment-like things" that are in string literals ("/*blah*/").
> (as far as I can tell, at least)
Ah, I see. If your version works fine I think it’s worth using it to
replace the need for Python here.
I’d probably implement this somewhat differently (to remove the
“almost*” symbols and to cut down on duplication), but I don’t think
it’s worth polishing this when the main purpose has been achieved.
Thanks for tackling this!
--
Ricardo
Ludovic Courtès wrote 6 years ago
(name . Ricardo Wurmus)(address . rekado@elephly.net)
87r2airour.fsf@gnu.org
Hello,
Ricardo Wurmus <rekado@elephly.net> skribis:
Toggle quote (10 lines)
> Andreas Enge <andreas@enge.fr> writes:
>
>> Before closing the bug, I would still like to ask whether there is a way
>> of getting around the need for python.
>
> Guix pull needs “guile-git”, and “guile-git” needs “libgit2”, which
> needs “python-wrapper”. To remove the need for Python in “guix pull” we
> would need to build libgit2 without Python. I don’t know if anyone has
> investigated whether this can be done.
Danny Milosavljevic <dannym@scratchpost.org> skribis:
Toggle quote (4 lines)
> I've started on implementing a replacement test generator, see attachment.
>
> Usage is exactly the same as the original generate.py.
I gather this bug was fixed with commit
03fb5ff6ae01a680c786d9ee148839543c519411. Closing!
Ludo’.
Closed
?