ant-bootstrap fails on core-updates (409 dependents)

  • Done
  • quality assurance status badge
Details
6 participants
  • Gábor Boskovits
  • Chris Marusich
  • Efraim Flashner
  • Julien Lepiller
  • Ludovic Courtès
  • Ricardo Wurmus
Owner
unassigned
Submitted by
Chris Marusich
Severity
important
C
C
Chris Marusich wrote on 16 Jul 2019 08:58
(address . bug-guix@gnu.org)
8736j61n57.fsf@gmail.com
Hi,

At commit 464a29d3d74e7d2f27042db6ab166bfdbe1f992e ('git branch --all
--contains 464a29d3d74e7d2f27042db6ab166bfdbe1f992e' tells me that only
core-updates has this commit at this time), the following error occurs
when trying to build ant-bootstrap, which causes the 409 dependent
packages to fail, also:

Toggle snippet (44 lines)
$ guix build -e '(@@ (gnu packages java) ant-bootstrap)'
...
Issued 1 semantic warning compiling "src/main/org/apache/tools/ant/filters/FixCrLfFilter.java":

<---------
665. case '\r':
. . .
684. }
-------------------->
*** Semantic Warning: This switch block can fall through to the next case. Did you forget a break statement?

Issued 1 semantic warning compiling "src/main/org/apache/tools/ant/taskdefs/Zip.java":

1555. Vector resources = new Vector();
^-------^
*** Semantic Warning: Local "resources" shadows a field of the same name in "org.apache.tools.ant.taskdefs.Zip".

Issued 1 semantic warning compiling "src/main/org/apache/tools/ant/taskdefs/Get.java":

633. URLConnection connection = aSource.openConnection();
^--------^
*** Semantic Warning: Local "connection" shadows a field of the same name in "org.apache.tools.ant.taskdefs.Get$GetThread".

Issued 1 semantic warning compiling "src/main/org/apache/tools/ant/taskdefs/rmic/XNewRmic.java":

34. public static final String COMPILER_NAME = "xnew";
^-----------^
*** Semantic Warning: Field "COMPILER_NAME" shadows a field of the same name in "org.apache.tools.ant.taskdefs.rmic.ForkingSunRmic".
... Copying Required Files
... Building Ant Distribution
Buildfile: /tmp/guix-build-ant-bootstrap-1.8.4.drv-0/apache-ant-1.8.4/build.xml

BUILD FAILED
Could not load the version information.

Total time: 0 seconds
... Failed Building Ant Distribution !
command "bash" "bootstrap.sh" "-Ddist.dir=/gnu/store/jd6jm79d0r5g59d0l2l3w445adykp5p9-ant-bootstrap-1.8.4" failed with status 1
builder for `/gnu/store/76apf0hpcdabpjy0839nhkwgfrz3m8z5-ant-bootstrap-1.8.4.drv' failed with exit code 1
build of /gnu/store/76apf0hpcdabpjy0839nhkwgfrz3m8z5-ant-bootstrap-1.8.4.drv failed
View build log at '/var/log/guix/drvs/76/apf0hpcdabpjy0839nhkwgfrz3m8z5-ant-bootstrap-1.8.4.drv.bz2'.
guix build: error: build of `/gnu/store/76apf0hpcdabpjy0839nhkwgfrz3m8z5-ant-bootstrap-1.8.4.drv' failed

It fails quite quickly, so you can easily get a copy failing locally if
you want to test it out.

--
Chris
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEy/WXVcvn5+/vGD+x3UCaFdgiRp0FAl0tdZQACgkQ3UCaFdgi
Rp1evA/9HUEfqG+JKJX3aPcVe3cBEuMsISgo2t7uLWUmReVsw9iBUDnN0gu/1Bqj
jgKKuwM6uTD9cA739j5lrJO5C9IXxDrMsy7AUH7KrIdPqxr8lDp7ECFxf1D0SuJr
io6BkHQXXgj+rYehe7hDJ7oiFivZdGRTznuubw9FN9+C4CrFGpXRzL+sum1TCi6U
fOilZf6TKgYVGUtIALcFYsmQmnk7UyfQyhQVu9b3rNXyg7bCc9HwwiaFVZBLAMIa
bHhADgP5AKIDNqjbMoV/aYFopb2BDl34wahVlOaiRS5+u/K0VtZqXsfM5vxhwIWI
ez63xzhkbf/QDqQ2JVGXSZQTAYbTsOBR3bTIJcbYktSAsJeBR1aqo3dOvl4ac0rE
vbjLgzIV35fYd4rfBrHCCNzQct6liKsnUTlJJFnh84i6otSESLUEYiwHxOz23reo
2IAUM3StWO3kLPVHfGnP17cbJS//DOpEZ6U4CxMl215hgvdtk/gEwnYcrLq960WS
Kyb/q3UhLW0+itGV4/I9sE0lZV0ox7/ST42u+RbrKFkwjnSlOG4sV3Y1gPnnei1b
YlY4KuWXKFpc3vZBjfQEyHGJd6a3zBh862x2RcrOvDaSH711hcg/WVT7AOxtVCSf
Cq0Mnsv82vyAW3NIWKsJveGoTXIc5jZ9/fW46bGEc4bgez4iN98=
=eiFS
-----END PGP SIGNATURE-----

G
G
Gábor Boskovits wrote on 16 Jul 2019 09:03
(name . Chris Marusich)(address . cmmarusich@gmail.com)(address . 36685@debbugs.gnu.org)
CAE4v=phnP0Gyj0P5E9ePc9Eec_hhRuKPCmyaEU-bpcBhnapaiw@mail.gmail.com
Hi,

Chris Marusich <cmmarusich@gmail.com> ezt írta (id?pont: 2019. júl. 16., Ke
8:59):

Toggle quote (72 lines)
> Hi,
>
> At commit 464a29d3d74e7d2f27042db6ab166bfdbe1f992e ('git branch --all
> --contains 464a29d3d74e7d2f27042db6ab166bfdbe1f992e' tells me that only
> core-updates has this commit at this time), the following error occurs
> when trying to build ant-bootstrap, which causes the 409 dependent
> packages to fail, also:
>
> --8<---------------cut here---------------start------------->8---
> $ guix build -e '(@@ (gnu packages java) ant-bootstrap)'
> ...
> Issued 1 semantic warning compiling
> "src/main/org/apache/tools/ant/filters/FixCrLfFilter.java":
>
> <---------
> 665. case '\r':
> . . .
> 684. }
> -------------------->
> *** Semantic Warning: This switch block can fall through to the next case.
> Did you forget a break statement?
>
> Issued 1 semantic warning compiling
> "src/main/org/apache/tools/ant/taskdefs/Zip.java":
>
> 1555. Vector resources = new Vector();
> ^-------^
> *** Semantic Warning: Local "resources" shadows a field of the same name
> in "org.apache.tools.ant.taskdefs.Zip".
>
> Issued 1 semantic warning compiling
> "src/main/org/apache/tools/ant/taskdefs/Get.java":
>
> 633. URLConnection connection = aSource.openConnection();
> ^--------^
> *** Semantic Warning: Local "connection" shadows a field of the same name
> in "org.apache.tools.ant.taskdefs.Get$GetThread".
>
> Issued 1 semantic warning compiling
> "src/main/org/apache/tools/ant/taskdefs/rmic/XNewRmic.java":
>
> 34. public static final String COMPILER_NAME = "xnew";
> ^-----------^
> *** Semantic Warning: Field "COMPILER_NAME" shadows a field of the same
> name in "org.apache.tools.ant.taskdefs.rmic.ForkingSunRmic".
> ... Copying Required Files
> ... Building Ant Distribution
> Buildfile:
> /tmp/guix-build-ant-bootstrap-1.8.4.drv-0/apache-ant-1.8.4/build.xml
>
> BUILD FAILED
> Could not load the version information.
>
> Total time: 0 seconds
> ... Failed Building Ant Distribution !
> command "bash" "bootstrap.sh"
> "-Ddist.dir=/gnu/store/jd6jm79d0r5g59d0l2l3w445adykp5p9-ant-bootstrap-1.8.4"
> failed with status 1
> builder for
> `/gnu/store/76apf0hpcdabpjy0839nhkwgfrz3m8z5-ant-bootstrap-1.8.4.drv'
> failed with exit code 1
> build of
> /gnu/store/76apf0hpcdabpjy0839nhkwgfrz3m8z5-ant-bootstrap-1.8.4.drv failed
> View build log at
> '/var/log/guix/drvs/76/apf0hpcdabpjy0839nhkwgfrz3m8z5-ant-bootstrap-1.8.4.drv.bz2'.
> guix build: error: build of
> `/gnu/store/76apf0hpcdabpjy0839nhkwgfrz3m8z5-ant-bootstrap-1.8.4.drv' failed
> --8<---------------cut here---------------end--------------->8---
>
> It fails quite quickly, so you can easily get a copy failing locally if
> you want to test it out.
>
Thanks for the report.
I will have a look at this soon.

Toggle quote (4 lines)
>
> --
> Chris
>
Best regards,
g_bor

Toggle quote (1 lines)
>
Attachment: file
E
E
Efraim Flashner wrote on 16 Jul 2019 10:08
(name . Chris Marusich)(address . cmmarusich@gmail.com)(address . 36685@debbugs.gnu.org)
20190716080816.GJ22158@macbook41
Building with gcc-5 didn't make it suddenly start working, before anyone
goes down that path.


--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAl0thesACgkQQarn3Mo9
g1HP5A//SHa0ezqV5L72mD7rrQSSCF+97/IdvlGdMLCaIauoZRShKsRVXIbZlR8O
zYCUHE6KhsAdIAgVfTa4onJLC/lcigJfaCnK5gd7YBG9PHBez2eTv73+2QJUtTnx
jQLVoMBrtZUkV0JixnJ52CsXGxHut0PzzTL70VSioN888vuRr01Qz+kL7J1Hpj/s
BFuGzqt4E5D07wvpX9CgQxVt1RdWgVLiVY7Fk3o7H/VK3jHOaNRCY6BcLBKB1inu
uIK9eFvwZ7EhD47RecFwAZSAdEmVx/0DKH4aHsF4PWsK5zpT44NmDURmhe7pT+Bv
UIIDO+mMSPWG+qyGXfbTf1iDri18qENXwzIMlQNG4CLoFa3ul8Gx7eTlH7RSXd4I
14QYZPz4TWgIZYNNBuYEMhhaYFS0RA63Qj4xpM8T8upYvXW8eypbwpMW+Hul7dsP
zzTg3X+rylrEabmQxMqoIzvMKgGVw4SXXbxr3JhuVQtdRlrxTg1NOvqNyS/yH7dz
Hr+sPi0o4SXmOPJb+Lfa6PlGnklidytKb5ymNjjeX3+hYNNPv4LaGZP7n1hkqJQ9
/5ORbe8rIbIcmeYUv3HALUm3/zB/uQXGYzUqdiNPhx4nrn6S6cGD8OfECZDoeWUA
NK5l1bAqu/OmyaBAbuK0TUdWWwa4ySjmPp4TdSDe1CtQ3NyKKx4=
=EiE4
-----END PGP SIGNATURE-----


R
R
Ricardo Wurmus wrote on 16 Jul 2019 18:13
(address . cmmarusich@gmail.com)(address . 36685@debbugs.gnu.org)
87o91ugdot.fsf@elephly.net
Chris Marusich <cmmarusich@gmail.com> writes:

Toggle quote (10 lines)
> Hi,
>
> At commit 464a29d3d74e7d2f27042db6ab166bfdbe1f992e ('git branch --all
> --contains 464a29d3d74e7d2f27042db6ab166bfdbe1f992e' tells me that only
> core-updates has this commit at this time), the following error occurs
> when trying to build ant-bootstrap, which causes the 409 dependent
> packages to fail, also:
>
> --8<---------------cut here---------------start------------->8---
> $ guix build -e '(@@ (gnu packages java) ant-bootstrap)'
[…]
Toggle quote (3 lines)
> BUILD FAILED
> Could not load the version information.

This error means that it couldn’t find the version.txt file. This is
strange because it’s right there in src/main, but not in build/classes.
So I thought I could just fix this by copying files from src/main to
build/classes and the bootstrap build does in fact begin:

Toggle snippet (33 lines)
diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
index 403c446a82..0ec582d10a 100644
--- a/gnu/packages/java.scm
+++ b/gnu/packages/java.scm
@@ -244,6 +244,8 @@ JNI.")
(setenv "HOME" "/tmp")
(with-output-to-file "/tmp/.ant.properties"
(lambda _ (display "")))
+ (with-output-to-file ".ant.properties"
+ (lambda _ (display "")))

;; Use jikes instead of javac for <javac ...> tags in build.xml
(setenv "ANT_OPTS" "-Dbuild.compiler=jikes")
@@ -260,6 +262,17 @@ JNI.")
`((string-append m "-Xnocompact "))
`((string-append m "-Xnocompact -Xnoinlining ")))))

+ (substitute* "src/main/org/apache/tools/ant/version.txt"
+ (("VERSION=.*") (string-append "VERSION=" ,version "\n"))
+ (("DATE=.*") "DATE=reproducible"))
+
+ ;; XXX: Copying the source files appears to be necessary because
+ ;; ant won't find the XML and txt resources in src/main for some
+ ;; reason. It really shouldn't be needed, so maybe this can be
+ ;; avoided by setting some environment variable.
+ (substitute* "bootstrap.sh"
+ (("cp src/script/antRun bin" m)
+ (string-append m "\ncp -ar src/main/* build/classes/")))
;; Disable tests because we are bootstrapping and thus don't have
;; any of the dependencies required to build and run the tests.
(substitute* "build.xml"

It’s possible we could get away with just adding these lines:

cp {src/main,build/classes}/org/apache/tools/ant/antlib.xml
cp {src/main,build/classes}/org/apache/tools/ant/version.txt

However, this doesn’t seem to help. Yes, the bootstrap script no longer
aborts but it gets stuck compiling things. I can’t get it to tell me
anything about the compilation progress, but strace shows me that it
keeps stat’ing for non-existent files like "/tmp/files16bfb86414e_b"
until the end of the day.

Judging by the name of the file I think this is ant’s
src/main/org/apache/tools/ant/taskdefs/compilers/DefaultCompilerAdapter.java
that creates them, which is used by Jikes.java. We should see if
there’s a race somewhere or if perhaps “jikes” crashes and thus gets
executed over and over again.

--
Ricardo
R
R
Ricardo Wurmus wrote on 17 Jul 2019 10:09
(address . cmmarusich@gmail.com)(address . 36685@debbugs.gnu.org)
87ftn5gjzw.fsf@elephly.net
Ricardo Wurmus <rekado@elephly.net> writes:

Toggle quote (12 lines)
> However, this doesn’t seem to help. Yes, the bootstrap script no longer
> aborts but it gets stuck compiling things. I can’t get it to tell me
> anything about the compilation progress, but strace shows me that it
> keeps stat’ing for non-existent files like "/tmp/files16bfb86414e_b"
> until the end of the day.
>
> Judging by the name of the file I think this is ant’s
> src/main/org/apache/tools/ant/taskdefs/compilers/DefaultCompilerAdapter.java
> that creates them, which is used by Jikes.java. We should see if
> there’s a race somewhere or if perhaps “jikes” crashes and thus gets
> executed over and over again.

The problem appears to be in GNU Classpath (classpath-bootstrap). Its
“native/jni/java-io/java_io_VMFile.c” defines
“Java_java_io_VMFile_exists”, which returns “1” when a file exists. It
does so by calling “cpio_isFileExists” which is defined in
“native/jni/native-lib/cpio.c”:

Toggle snippet (13 lines)
int cpio_isFileExists (const char *filename)
{
struct stat statbuf;

if (stat(filename, &statbuf) < 0)
{
return errno;
}

return 0;
}

I have confirmed with printf debugging that there’s a mismatch between
what the Java side thinks and what the C side tells it. On the C side
the temporary file is determined to not exist yet, but on the Java side
it is said to exist — this is in spite of the fact that the Java side
only converts the C side’s return value.

Here’s an example from the printf output:

exists? /tmp/files16bfd145b82_1 : 2 -- 0;
classpath trying to create: /tmp/files16bfd145b82_1 exists? true

The first line is from C, the 2 is ENOENT, and 0 is the return value.
The zero is taken to be a Java boolean, so it maps to false. The second
line comes from Java.

It hangs in “while (VMFile.exists(file.path))”:

Toggle snippet (26 lines)
File file;
if (!VMFile.IS_DOS_8_3)
{
do
{
long now = System.currentTimeMillis();
if (now > last_tmp)
{
// The last temporary file was created more than 1 ms ago.
last_tmp = now;
n_created = 0;
}
else
n_created++;

String name = Long.toHexString(now);
if (n_created > 0)
name += '_'+Integer.toHexString(n_created);
String filename = prefix + name + suffix;
file = new File(directory, filename);
}
while (VMFile.exists(file.path));
}
else …

I have confirmed that this is the problem by replacing “while
(VMFile.exists(file.path))” with “while (false)”. The build doesn’t
fully complete then either, but it gets past the compilation of the Ant
source files. This clears JamVM and Jikes.

I’m attaching my embarrassing printf debugging patches.

Any ideas?

--
Ricardo
Attachment: printf.diff
R
R
Ricardo Wurmus wrote on 18 Jul 2019 00:41
(address . cmmarusich@gmail.com)(address . 36685@debbugs.gnu.org)
871ryogu6j.fsf@elephly.net
Ricardo Wurmus <rekado@elephly.net> writes:

Toggle quote (5 lines)
> I have confirmed that this is the problem by replacing “while
> (VMFile.exists(file.path))” with “while (false)”. The build doesn’t
> fully complete then either, but it gets past the compilation of the Ant
> source files. This clears JamVM and Jikes.

It doesn’t.

VMFile.exists always returns “true”, no matter what. The C part is
fine, but the Java part is not, so I suspected the JVM to be at fault.

I managed to build ant-bootstrap successfully with this patch:
Toggle diff (44 lines)
diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
index 403c446a82..6ebfd0ffdf 100644
--- a/gnu/packages/java.scm
+++ b/gnu/packages/java.scm
@@ -152,6 +152,12 @@ and binary format defined in The Java Virtual Machine Specification.")
"--disable-gjdoc")
#:phases
(modify-phases %standard-phases
+ (add-after 'unpack 'foo
+ (lambda _
+ (substitute* "native/jni/java-io/java_io_VMFile.c"
+ (("result = cpio_isFileExists.*" m)
+ (string-append m "\nfprintf(stderr, \"exists? %s : %d -- %d;\\n\", filename, result, ((result == CPNATIVE_OK) ? 1 : 0));")))
+ #t))
(add-after 'install 'install-data
(lambda _ (invoke "make" "install-data"))))))
(native-inputs
@@ -186,10 +192,12 @@ language.")
(arguments
`(#:configure-flags
(list (string-append "--with-classpath-install-dir="
- (assoc-ref %build-inputs "classpath")))))
+ (assoc-ref %build-inputs "classpath"))
+ "--enable-ffi")))
(inputs
`(("classpath" ,classpath-bootstrap)
("jikes" ,jikes)
+ ("libffi" ,libffi)
("zlib" ,zlib)))
(home-page "http://jamvm.sourceforge.net/")
(synopsis "Small Java Virtual Machine")
@@ -258,7 +266,11 @@ JNI.")
,@(if (string-prefix? "armhf" (or (%current-system)
(%current-target-system)))
`((string-append m "-Xnocompact "))
- `((string-append m "-Xnocompact -Xnoinlining ")))))
+ `((string-append m "-Xnocompact ")))))
+
+ (substitute* "src/main/org/apache/tools/ant/version.txt"
+ (("VERSION=.*") (string-append "VERSION=" ,version "\n"))
+ (("DATE=.*") "DATE=reproducible"))
;; Disable tests because we are bootstrapping and thus don't have
;; any of the dependencies required to build and run the tests.
Note that the change to native/jni/java-io/java_io_VMFile.c is necessary
even though it just prints things and compares values. So this might be
time-related or it could be other stateful behaviour. Don’t know yet.

The changes in ant-bootstrap might not be needed.

Can someone confirm that this patch helps?

--
Ricardo
R
R
Ricardo Wurmus wrote on 18 Jul 2019 12:51
(address . cmmarusich@gmail.com)
87r26nfwes.fsf@elephly.net
Ricardo Wurmus <rekado@elephly.net> writes:

Toggle quote (14 lines)
> Ricardo Wurmus <rekado@elephly.net> writes:
>
>> I have confirmed that this is the problem by replacing “while
>> (VMFile.exists(file.path))” with “while (false)”. The build doesn’t
>> fully complete then either, but it gets past the compilation of the Ant
>> source files. This clears JamVM and Jikes.
>
> It doesn’t.
>
> VMFile.exists always returns “true”, no matter what. The C part is
> fine, but the Java part is not, so I suspected the JVM to be at fault.
>
> I managed to build ant-bootstrap successfully with this patch: […]

Here’s a shorter patch:

Toggle snippet (20 lines)
diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
index 403c446a82..bd98784232 100644
--- a/gnu/packages/java.scm
+++ b/gnu/packages/java.scm
@@ -152,6 +152,13 @@ and binary format defined in The Java Virtual Machine Specification.")
"--disable-gjdoc")
#:phases
(modify-phases %standard-phases
+ (add-after 'unpack 'foo
+ (lambda _
+ (substitute* "native/jni/java-io/java_io_VMFile.c"
+ (("result = cpio_isFileExists.*" m)
+ (string-append m "
+//Without a long comment the Java side will return \"true\" on x86_64 all the time.")))
+ #t))
(add-after 'install 'install-data
(lambda _ (invoke "make" "install-data"))))))
(native-inputs

This only adds a comment. If the comment is too short it won’t work.

I suspected some misguided optimizations, so I built jamvm-1-bootstrap
and classpath-bootstrap with -O0, -O1, and I disabled stripping of
binaries, but to no avail.

Any ideas?

--
Ricardo
R
R
Ricardo Wurmus wrote on 19 Jul 2019 10:09
(address . cmmarusich@gmail.com)
87a7dafntp.fsf@elephly.net
Ricardo Wurmus <rekado@elephly.net> writes:
Toggle quote (25 lines)
> Here’s a shorter patch:
>
> --8<---------------cut here---------------start------------->8---
> diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
> index 403c446a82..bd98784232 100644
> --- a/gnu/packages/java.scm
> +++ b/gnu/packages/java.scm
> @@ -152,6 +152,13 @@ and binary format defined in The Java Virtual Machine Specification.")
> "--disable-gjdoc")
> #:phases
> (modify-phases %standard-phases
> + (add-after 'unpack 'foo
> + (lambda _
> + (substitute* "native/jni/java-io/java_io_VMFile.c"
> + (("result = cpio_isFileExists.*" m)
> + (string-append m "
> +//Without a long comment the Java side will return \"true\" on x86_64 all the time.")))
> + #t))
> (add-after 'install 'install-data
> (lambda _ (invoke "make" "install-data"))))))
> (native-inputs
> --8<---------------cut here---------------end--------------->8---
>
> This only adds a comment. If the comment is too short it won’t work.

No, that’s wrong.

To my eternal embarrassement but also great relief this substitution has
the effect of commenting the *following* line which frees up previously
claimed resources (a bunch of characters making up the file name). The
other comments I tested must have ended on \n, so they did not have this
effect.

Thanks to Julien for pointing this out!

So! Creating a memory leak lets us successfully build ant-bootstrap.
It does not, however, get us all the way through the Java bootstrap.
When configuring the first icedtea I get this error:

Toggle snippet (5 lines)
checking if the VM and compiler work together... ./configure: line 9614: 697 Illegal instruction $JAVA -classpath . $BYTECODE 1>&5 2>&1
configure: error: VM failed to run compiled class.
command "/gnu/store/h9c5g3inn5zmkixk08m27zzpj58zbfgy-bash-minimal-5.0.7/bin/bash" "./configure" "CONFIG_SHELL=/gnu/store/h9c5g3inn5zmkixk08m27zzpj58zbfgy-bash-minimal-5.0.7/bin/bash" "SHELL=/gnu/store/h9c5g3inn5zmkixk08m27zzpj58zbfgy-bash-minimal-5.0.7/bin/bash" "--prefix=/gnu/store/802356lxpjkqk66kv35mdzxhvaw6rghp-icedtea-1.13.13" "--enable-fast-install" "--docdir=/gnu/store/d4c4w9bka2bnnrwrmph1ilgjss5i37h9-icedtea-1.13.13-doc/share/doc/icedtea" "--build=x86_64-unknown-linux-gnu" "--enable-bootstrap" "--enable-nss" "--without-rhino" "--with-parallel-jobs" "--disable-downloading" "--disable-tests" "--with-ecj=/gnu/store/6dijv9ynn5j2bya86dgjn8v0qfd1nv3j-ecj-bootstrap-3.2.2/share/java/ecj-bootstrap.jar" "--with-jar=/gnu/store/hw67b3w83cc2abbgrf0wqzra07iiz3a1-fastjar-0.98/bin/fastjar" "--with-jdk-home=/gnu/store/1agbz95p2ljcvbb88w7p7jn2hnd6z3gv-classpath-0.99-1.e7c13ee0c" "--with-java=/gnu/store/ril2kk63p1grib14vl88z3aladfs33gf-jamvm-2.0.0/bin/jamvm" failed with status 1

Illegal instruction? This uses JamVM 2.0.0 as the JVM. I’ll try to
figure out what instruction this is and where it comes from.

--
Ricardo
G
G
Gábor Boskovits wrote on 19 Jul 2019 14:37
(name . Ricardo Wurmus)(address . rekado@elephly.net)
CAE4v=pjmPSRCiqBauVMhK=CgfzLnUoN_z8UGfzu7A8uAdFMUbQ@mail.gmail.com
Hello,

Ricardo Wurmus <rekado@elephly.net> ezt írta (id?pont: 2019. júl. 19., P,
8:09):

Toggle quote (30 lines)
>
> Ricardo Wurmus <rekado@elephly.net> writes:
> > Here’s a shorter patch:
> >
> > --8<---------------cut here---------------start------------->8---
> > diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
> > index 403c446a82..bd98784232 100644
> > --- a/gnu/packages/java.scm
> > +++ b/gnu/packages/java.scm
> > @@ -152,6 +152,13 @@ and binary format defined in The Java Virtual
> Machine Specification.")
> > "--disable-gjdoc")
> > #:phases
> > (modify-phases %standard-phases
> > + (add-after 'unpack 'foo
> > + (lambda _
> > + (substitute* "native/jni/java-io/java_io_VMFile.c"
> > + (("result = cpio_isFileExists.*" m)
> > + (string-append m "
> > +//Without a long comment the Java side will return \"true\" on x86_64
> all the time.")))
> > + #t))
> > (add-after 'install 'install-data
> > (lambda _ (invoke "make" "install-data"))))))
> > (native-inputs
> > --8<---------------cut here---------------end--------------->8---
> >
> > This only adds a comment. If the comment is too short it won’t work.
>
>
I confirm this path works.
I tested a modified version, where I took out the comment text. It also
works that way.
We might contact the classpath devs to get a proper fix, and maybe a new
release, they
were super responsive the last time.


Toggle quote (40 lines)
> No, that’s wrong.
>
> To my eternal embarrassement but also great relief this substitution has
> the effect of commenting the *following* line which frees up previously
> claimed resources (a bunch of characters making up the file name). The
> other comments I tested must have ended on \n, so they did not have this
> effect.
>
> Thanks to Julien for pointing this out!
>
> So! Creating a memory leak lets us successfully build ant-bootstrap.
> It does not, however, get us all the way through the Java bootstrap.
> When configuring the first icedtea I get this error:
>
> --8<---------------cut here---------------start------------->8---
> checking if the VM and compiler work together... ./configure: line 9614:
> 697 Illegal instruction $JAVA -classpath . $BYTECODE 1>&5 2>&1
> configure: error: VM failed to run compiled class.
> command
> "/gnu/store/h9c5g3inn5zmkixk08m27zzpj58zbfgy-bash-minimal-5.0.7/bin/bash"
> "./configure"
> "CONFIG_SHELL=/gnu/store/h9c5g3inn5zmkixk08m27zzpj58zbfgy-bash-minimal-5.0.7/bin/bash"
> "SHELL=/gnu/store/h9c5g3inn5zmkixk08m27zzpj58zbfgy-bash-minimal-5.0.7/bin/bash"
> "--prefix=/gnu/store/802356lxpjkqk66kv35mdzxhvaw6rghp-icedtea-1.13.13"
> "--enable-fast-install"
> "--docdir=/gnu/store/d4c4w9bka2bnnrwrmph1ilgjss5i37h9-icedtea-1.13.13-doc/share/doc/icedtea"
> "--build=x86_64-unknown-linux-gnu" "--enable-bootstrap" "--enable-nss"
> "--without-rhino" "--with-parallel-jobs" "--disable-downloading"
> "--disable-tests"
> "--with-ecj=/gnu/store/6dijv9ynn5j2bya86dgjn8v0qfd1nv3j-ecj-bootstrap-3.2.2/share/java/ecj-bootstrap.jar"
> "--with-jar=/gnu/store/hw67b3w83cc2abbgrf0wqzra07iiz3a1-fastjar-0.98/bin/fastjar"
> "--with-jdk-home=/gnu/store/1agbz95p2ljcvbb88w7p7jn2hnd6z3gv-classpath-0.99-1.e7c13ee0c"
> "--with-java=/gnu/store/ril2kk63p1grib14vl88z3aladfs33gf-jamvm-2.0.0/bin/jamvm"
> failed with status 1
> --8<---------------cut here---------------end--------------->8---
>
> Illegal instruction? This uses JamVM 2.0.0 as the JVM. I’ll try to
> figure out what instruction this is and where it comes from.
>

I hit the same bug now.


Toggle quote (5 lines)
> --
> Ricardo
>
>
>
Best regards,
g_bor

--
OpenPGP Key Fingerprint: 7988:3B9F:7D6A:4DBF:3719:0367:2506:A96C:CF63:0B21
Attachment: file
R
R
Ricardo Wurmus wrote on 19 Jul 2019 14:56
(name . Gábor Boskovits)(address . boskovits@gmail.com)
878ssufajf.fsf@elephly.net
Hi Gábor,

Toggle quote (37 lines)
> Ricardo Wurmus <rekado@elephly.net> ezt írta (id?pont: 2019. júl. 19., P,
> 8:09):
>
>>
>> Ricardo Wurmus <rekado@elephly.net> writes:
>> > Here’s a shorter patch:
>> >
>> > --8<---------------cut here---------------start------------->8---
>> > diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
>> > index 403c446a82..bd98784232 100644
>> > --- a/gnu/packages/java.scm
>> > +++ b/gnu/packages/java.scm
>> > @@ -152,6 +152,13 @@ and binary format defined in The Java Virtual
>> Machine Specification.")
>> > "--disable-gjdoc")
>> > #:phases
>> > (modify-phases %standard-phases
>> > + (add-after 'unpack 'foo
>> > + (lambda _
>> > + (substitute* "native/jni/java-io/java_io_VMFile.c"
>> > + (("result = cpio_isFileExists.*" m)
>> > + (string-append m "
>> > +//Without a long comment the Java side will return \"true\" on x86_64
>> all the time.")))
>> > + #t))
>> > (add-after 'install 'install-data
>> > (lambda _ (invoke "make" "install-data"))))))
>> > (native-inputs
>> > --8<---------------cut here---------------end--------------->8---
>> >
>> > This only adds a comment. If the comment is too short it won’t work.
>>
>>
> I confirm this path works.
> I tested a modified version, where I took out the comment text. It also
> works that way.

Yes. It’s just commenting the *following* line (i.e. removal of the
call to “free”) that fixes it.

Toggle quote (4 lines)
> We might contact the classpath devs to get a proper fix, and maybe a new
> release, they
> were super responsive the last time.

I don’t think the problem is with GNU Classpath. Classpath devs are
welcome to look at this, of course, but we are not even using the latest
version at this point in the bootstrap.

I wonder what could have caused this problem in the first place. The
bootstrap hasn’t changed. Using the same compiler as before doesn’t fix
it, so it’s probably not the compiler either. So what is it?

Toggle quote (5 lines)
>> Illegal instruction? This uses JamVM 2.0.0 as the JVM. I’ll try to
>> figure out what instruction this is and where it comes from.
>
> I hit the same bug now.

Good. We need to figure out why – is it miscompilation? Tuning to a
different CPU type…?

--
Ricardo
R
R
Ricardo Wurmus wrote on 19 Jul 2019 18:30
(name . Gábor Boskovits)(address . boskovits@gmail.com)
875znyf0mr.fsf@elephly.net
Ricardo Wurmus <rekado@elephly.net> writes:

Toggle quote (8 lines)
>>> Illegal instruction? This uses JamVM 2.0.0 as the JVM. I’ll try to
>>> figure out what instruction this is and where it comes from.
>>
>> I hit the same bug now.
>
> Good. We need to figure out why – is it miscompilation? Tuning to a
> different CPU type…?

With this patch I can get past the configure phase in icedtea:

Toggle snippet (92 lines)
diff --git a/gnu/packages/java.scm b/gnu/packages/java.scm
index 403c446a82..14179f85e9 100644
--- a/gnu/packages/java.scm
+++ b/gnu/packages/java.scm
@@ -116,6 +116,9 @@
(base32
"1qqldrp74pzpy5ly421srqn30qppmm9cvjiqdngk8hf47dv2rc0c"))))
(build-system gnu-build-system)
+ (native-inputs
+ `(("libc" ,glibc-2.28)
+ ("gcc" ,gcc-5)))
(home-page "http://jikes.sourceforge.net/")
(synopsis "Compiler for the Java language")
(description "Jikes is a compiler that translates Java source files as
@@ -152,12 +155,20 @@ and binary format defined in The Java Virtual Machine Specification.")
"--disable-gjdoc")
#:phases
(modify-phases %standard-phases
+ (add-after 'unpack 'foo
+ (lambda _
+ (substitute* "native/jni/java-io/java_io_VMFile.c"
+ (("result = cpio_isFileExists.*" m)
+ (string-append m "\n//")))
+ #t))
(add-after 'install 'install-data
(lambda _ (invoke "make" "install-data"))))))
(native-inputs
`(("jikes" ,jikes)
("fastjar" ,fastjar)
("libltdl" ,libltdl)
+ ("gcc" ,gcc-5)
+ ("libc" ,glibc-2.28)
("pkg-config" ,pkg-config)))
(home-page "https://www.gnu.org/software/classpath/")
(synopsis "Essential libraries for Java")
@@ -191,6 +202,9 @@ language.")
`(("classpath" ,classpath-bootstrap)
("jikes" ,jikes)
("zlib" ,zlib)))
+ (native-inputs
+ `(("libc" ,glibc-2.28)
+ ("gcc" ,gcc-5)))
(home-page "http://jamvm.sourceforge.net/")
(synopsis "Small Java Virtual Machine")
(description "JamVM is a Java Virtual Machine conforming to the JVM
@@ -302,7 +316,9 @@ JNI.")
`(("jikes" ,jikes)
("jamvm" ,jamvm-1-bootstrap)
("unzip" ,unzip)
- ("zip" ,zip)))
+ ("zip" ,zip)
+ ("gcc" ,gcc-5)
+ ("libc" ,glibc-2.28)))
(home-page "http://ant.apache.org")
(synopsis "Build tool for Java")
(description
@@ -627,7 +643,9 @@ machine.")))
("fastjar" ,fastjar)
("jamvm" ,jamvm-1-bootstrap)
("libltdl" ,libltdl)
- ("pkg-config" ,pkg-config))))))
+ ("pkg-config" ,pkg-config)
+ ("gcc" ,gcc-5)
+ ("libc" ,glibc-2.28))))))

(define jamvm
(package (inherit jamvm-1-bootstrap)
@@ -656,7 +674,9 @@ machine.")))
`(("guile" ,guile-2.2)
("ecj-bootstrap" ,ecj-bootstrap)
("jamvm" ,jamvm)
- ("classpath" ,classpath-devel)))))
+ ("classpath" ,classpath-devel)
+ ("gcc" ,gcc-5)
+ ("libc" ,glibc-2.28)))))

;; The bootstrap JDK consisting of jamvm, classpath-devel,
;; ecj-javac-wrapper-final cannot build Icedtea 2.x directly, because it's
@@ -907,7 +927,10 @@ machine.")))
("fastjar" ,fastjar)
("fontconfig" ,fontconfig)
("freetype" ,freetype)
- ("gcc" ,gcc-4.9) ; there's a segmentation fault when compiling with gcc-5 or gcc-7
+ ;;("gcc" ,gcc-4.9) ; there's a segmentation fault when compiling with gcc-5 or gcc-7
+ ("gcc" ,gcc-5)
+ ("libc" ,glibc-2.28)
+
("gtk" ,gtk+-2)
("gawk" ,gawk)
("giflib" ,giflib)

The build aborts early with “ERROR: FreeType version 2.2.1 or higher is
required.” So I added this:

(substitute* "patches/freetypeversion.patch"
(("REQUIRED_FREETYPE_VERSION = 2.2.1")
"REQUIRED_FREETYPE_VERSION = 2.10.1"))

and icedtea is being built now.

We may want to check where the use of glibc-2.28 and gcc-5 is really
needed and where we can do without.

--
Ricardo
R
R
Ricardo Wurmus wrote on 20 Jul 2019 10:42
(name . Gábor Boskovits)(address . boskovits@gmail.com)
87zhl9drm6.fsf@elephly.net
Ricardo Wurmus <rekado@elephly.net> writes:

Toggle quote (9 lines)
> The build aborts early with “ERROR: FreeType version 2.2.1 or higher is
> required.” So I added this:
>
> (substitute* "patches/freetypeversion.patch"
> (("REQUIRED_FREETYPE_VERSION = 2.2.1")
> "REQUIRED_FREETYPE_VERSION = 2.10.1"))
>
> and icedtea is being built now.

Next problem: icedtea 2 fails its configure phase:

Toggle snippet (9 lines)
checking attr/xattr.h usability... no
checking attr/xattr.h presence... no
checking for attr/xattr.h... no
configure: error: Could not find required system headers; install the appropriate files from glibc-headers, libc6-dev and/or libattr-devel, libattr1-dev or use --disable-compile-against-syscalls if necessary.
command "/gnu/store/h9c5g3inn5zmkixk08m27zzpj58zbfgy-bash-minimal-5.0.7/bin/bash" "./configure" "CONFIG_SHELL=/gnu/store/h9c5g3inn5zmkixk08m27zzpj58zbfgy-bash-minimal-5.0.7/bin/bash" "SHELL=/gnu/store/h9c5g3inn5zmkixk08m27zzpj58zbfgy-bash-minimal-5.0.7/bin/bash" "--prefix=/gnu/store/wg3ywf88n2qd1hpc8fk39svm9ksah1l4-icedtea-2.6.13" "--enable-fast-install" "--docdir=/gnu/store/4l3xf1dmbx9494ica8cgllm6dvs3py2n-icedtea-2.6.13-doc/share/doc/icedtea" "--build=x86_64-unknown-linux-gnu" "--disable-system-pcsc" "--disable-system-sctp" "--enable-bootstrap" "--enable-nss" "--without-rhino" "--disable-downloading" "--disable-tests" "--with-openjdk-src-dir=./openjdk.src" "--with-jdk-home=/gnu/store/r4jz94v5xjxw95s0msxdxm36hlv07x64-icedtea-1.13.13-jdk" failed with status 1
builder for `/gnu/store/cda07il6lpg0j9l8blx5z8ah8d7wmjlz-icedtea-2.6.13.drv' failed with exit code 1
build of /gnu/store/cda07il6lpg0j9l8blx5z8ah8d7wmjlz-icedtea-2.6.13.drv failed

Of course the inputs include the attr package. Any ideas?

--
Ricardo
J
J
Julien Lepiller wrote on 20 Jul 2019 11:06
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20190720110612.3f33171f@sybil.lepiller.eu
Le Sat, 20 Jul 2019 10:42:25 +0200,
Ricardo Wurmus <rekado@elephly.net> a écrit :

Toggle quote (4 lines)
> configure: error: Could not find required system headers; install the
> appropriate files from glibc-headers, libc6-dev and/or libattr-devel,
> libattr1-dev or use --disable-compile-against-syscalls if necessary.

Can you check the content of config.log? It should tell you what test
it runs to determine whether attr is present or not.
R
R
Ricardo Wurmus wrote on 20 Jul 2019 23:07
(name . Julien Lepiller)(address . julien@lepiller.eu)
87sgr0e7ot.fsf@elephly.net
Julien Lepiller <julien@lepiller.eu> writes:

Toggle quote (10 lines)
> Le Sat, 20 Jul 2019 10:42:25 +0200,
> Ricardo Wurmus <rekado@elephly.net> a écrit :
>
>> configure: error: Could not find required system headers; install the
>> appropriate files from glibc-headers, libc6-dev and/or libattr-devel,
>> libattr1-dev or use --disable-compile-against-syscalls if necessary.
>
> Can you check the content of config.log? It should tell you what test
> it runs to determine whether attr is present or not.

It’s pretty straight-forward: xattr.h is no longer provided by the attr
package. sys/xattr.h from glibc should be used instead, so I’ve patched
it.

I fixed another problem in the icedtea 2 build and moved on. Let’s see
how far I’ll get this time. In the meantime it would be great to reach
consensus on whether we should shrug off the patch that introduces a
memory leak in GNU Classpath 0.93 as part of the bootstrap or if we
should spend more time trying to figure out how we can avoid it.

--
Ricardo
R
R
Ricardo Wurmus wrote on 21 Jul 2019 13:29
(name . Julien Lepiller)(address . julien@lepiller.eu)
87muh7eid5.fsf@elephly.net
So, with the following change I was able to build all the way up to the
latest openjdk. Should we use it despite the introduction of a memory
leak in a bootstrap JVM? Can we make the patch smaller (fewer uses of
glibc 2.28 or gcc-5)?

What do you think?
Attachment: java.diff
--
Ricardo
G
G
Gábor Boskovits wrote on 21 Jul 2019 17:57
(name . Ricardo Wurmus)(address . rekado@elephly.net)
CAE4v=phrw3vh9Y0vF4wV0Lmir4pmH6OwUTTPiMnaoi=dd6nD+A@mail.gmail.com
Hello,

Ricardo Wurmus <rekado@elephly.net> ezt írta (id?pont: 2019. júl. 21., Vas
13:29):

Toggle quote (8 lines)
> So, with the following change I was able to build all the way up to the
> latest openjdk. Should we use it despite the introduction of a memory
> leak in a bootstrap JVM? Can we make the patch smaller (fewer uses of
> glibc 2.28 or gcc-5)?
>
> What do you think?
>

I will have a look at reducing the patch later today. I will report back
tomorrow morning with the results.


Toggle quote (4 lines)
>
> --
> Ricardo
>
Attachment: file
R
R
Ricardo Wurmus wrote on 24 Jul 2019 23:07
(name . Gábor Boskovits)(address . boskovits@gmail.com)
87sgqv9m61.fsf@elephly.net
Hi Gábor,

Toggle quote (11 lines)
>> So, with the following change I was able to build all the way up to the
>> latest openjdk. Should we use it despite the introduction of a memory
>> leak in a bootstrap JVM? Can we make the patch smaller (fewer uses of
>> glibc 2.28 or gcc-5)?
>>
>> What do you think?
>>
>
> I will have a look at reducing the patch later today. I will report back
> tomorrow morning with the results.

Did you have any luck with this?

--
Ricardo
R
R
Ricardo Wurmus wrote on 6 Sep 2019 15:40
(name . Gábor Boskovits)(address . boskovits@gmail.com)
87v9u5ee6n.fsf@elephly.net
Ricardo Wurmus <rekado@elephly.net> writes:

Toggle quote (13 lines)
>>> So, with the following change I was able to build all the way up to the
>>> latest openjdk. Should we use it despite the introduction of a memory
>>> leak in a bootstrap JVM? Can we make the patch smaller (fewer uses of
>>> glibc 2.28 or gcc-5)?
>>>
>>> What do you think?
>>>
>>
>> I will have a look at reducing the patch later today. I will report back
>> tomorrow morning with the results.
>
> Did you have any luck with this?

We should decide soon, because core-updates is about to be merged
(finally!) – any objections to my earlier patch?

--
Ricardo
G
G
Gábor Boskovits wrote on 6 Sep 2019 15:41
(name . Ricardo Wurmus)(address . rekado@elephly.net)
CAE4v=pimHCg3G2AfAQUdM6qHV3XSe_fnjYn0LhempOrp3EpzcA@mail.gmail.com
Ricardo Wurmus <rekado@elephly.net> ezt írta (id?pont: 2019. szept. 6., Pén
15:40):

Toggle quote (20 lines)
>
> Ricardo Wurmus <rekado@elephly.net> writes:
>
> >>> So, with the following change I was able to build all the way up to the
> >>> latest openjdk. Should we use it despite the introduction of a memory
> >>> leak in a bootstrap JVM? Can we make the patch smaller (fewer uses of
> >>> glibc 2.28 or gcc-5)?
> >>>
> >>> What do you think?
> >>>
> >>
> >> I will have a look at reducing the patch later today. I will report back
> >> tomorrow morning with the results.
> >
> > Did you have any luck with this?
>
> We should decide soon, because core-updates is about to be merged
> (finally!) – any objections to my earlier patch?
>

No objection from here.

Toggle quote (4 lines)
>
> --
> Ricardo
>
g_bor

Toggle quote (1 lines)
>
Attachment: file
L
L
Ludovic Courtès wrote on 6 Sep 2019 15:52
control message for bug #36685
(address . control@debbugs.gnu.org)
874l1pcz1y.fsf@gnu.org
severity 36685 important
quit
R
R
Ricardo Wurmus wrote on 7 Sep 2019 16:28
Re: bug#36685: ant-bootstrap fails on core-updates (409 dependents)
(name . Gábor Boskovits)(address . boskovits@gmail.com)
875zm4uqne.fsf@elephly.net
I reduced the patch and built openjdk once more to test that it all
works. I pushed the changes in a series of commits ending with
6b7e09ae6b to core-updates.

--
Ricardo
Closed
?