java-testng: again nondeterministic test-failures?

  • Open
  • quality assurance status badge
Details
3 participants
  • Björn Höfling
  • Julien Lepiller
  • Konrad Hinsen
Owner
unassigned
Submitted by
Björn Höfling
Severity
normal
B
B
Björn Höfling wrote on 19 May 2020 11:27
(address . bug-guix@gnu.org)
20200519112745.6971271b@alma-ubu
Does anyone else have the same problems?

On my system java-testng sometimes fails nondeterministically within
the check phase:

===============================================
TestNG
Total tests run: 1633, Failures: 2, Skips: 0
===============================================

Failures in <suite> :TestNG, <test> :Regression2
test.thread.ParallelTestTest.verifyParallel()
StackTrace:
java.lang.AssertionError:
Expected size:<4> but was:<3> in:
<{59L=59L, 60L=60L, 62L=62L}>
at test.thread.ParallelTestTest.verifyExpected(Unknown Source)
at test.thread.ParallelTestTest.verifyParallel(Unknown Source)
... Removed 24 stack frames

test.thread.ParallelTestTest.verifyParallel()
StackTrace:
java.lang.AssertionError:
Expected size:<4> but was:<3> in:
<{55L=55L, 56L=56L, 58L=58L}>
at test.thread.ParallelTestTest.verifyExpected(Unknown Source)
at test.thread.ParallelTestTest.verifyParallel(Unknown Source)
... Removed 24 stack frames


Do others experience this too, or is it just my machine?

On berlin, everything looks fine, that makes we wonder:


We had some problems in the past with other parallel tests:


I also tried to update java-testng but that was not possible quickly,
because of new/updated dependency-graph.

Björn
-----BEGIN PGP SIGNATURE-----

iF0EAREKAB0WIQQiGUP0np8nb5SZM4K/KGy2WT5f/QUCXsOmkQAKCRC/KGy2WT5f
/WhtAJ0aCtxiA8F+Ted9WjJ75cU4YwkMNgCfa2pIUlhBKQmbsg81hsTldNiFALA=
=2B67
-----END PGP SIGNATURE-----


J
J
Julien Lepiller wrote on 19 May 2020 14:08
F14629F3-C125-4D86-B7DE-FF4D899E3C6D@lepiller.eu
Le 19 mai 2020 05:27:45 GMT-04:00, "Björn Höfling" <bjoern.hoefling@bjoernhoefling.de> a écrit :
Toggle quote (45 lines)
>Does anyone else have the same problems?
>
>On my system java-testng sometimes fails nondeterministically within
>the check phase:
>
>===============================================
>TestNG
>Total tests run: 1633, Failures: 2, Skips: 0
>===============================================
>
>Failures in <suite> :TestNG, <test> :Regression2
>test.thread.ParallelTestTest.verifyParallel()
>StackTrace:
> java.lang.AssertionError:
>Expected size:<4> but was:<3> in:
><{59L=59L, 60L=60L, 62L=62L}>
> at test.thread.ParallelTestTest.verifyExpected(Unknown Source)
> at test.thread.ParallelTestTest.verifyParallel(Unknown Source)
>... Removed 24 stack frames
>
>test.thread.ParallelTestTest.verifyParallel()
>StackTrace:
> java.lang.AssertionError:
>Expected size:<4> but was:<3> in:
><{55L=55L, 56L=56L, 58L=58L}>
> at test.thread.ParallelTestTest.verifyExpected(Unknown Source)
> at test.thread.ParallelTestTest.verifyParallel(Unknown Source)
>... Removed 24 stack frames
>
>
>Do others experience this too, or is it just my machine?
>
>On berlin, everything looks fine, that makes we wonder:
>
>https://ci.guix.gnu.org/search?query=%5Ejava-testng
>
>We had some problems in the past with other parallel tests:
>
>https://debbugs.gnu.org/cgi/bugreport.cgi?bug=30526
>
>I also tried to update java-testng but that was not possible quickly,
>because of new/updated dependency-graph.
>
>Björn

During my work on getting a maven-build-system, I recently rebuilt testng multiple times, with no issue at all.
K
K
Konrad Hinsen wrote on 24 Aug 2023 13:25
Re: java-testng: again nondeterministic test-failures?
(address . 41394@debbugs.gnu.org)
m1lee0k7gh.fsf@fastmail.net
During today's regular "guix pull", I ran into this problem: java-testng
failed to build twice (the failures comes from the test suite), but
built correctly the third time. That's for commit:
39fa1ef033fda82cb1d122e0d1675b51acb1db34

Cheers,
Konrad.
?