From debbugs-submit-bounces@debbugs.gnu.org Fri Jul 12 10:59:00 2019 Received: (at 35521) by debbugs.gnu.org; 12 Jul 2019 14:59:01 +0000 Received: from localhost ([127.0.0.1]:40520 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hlx0m-0002II-HR for submit@debbugs.gnu.org; Fri, 12 Jul 2019 10:59:00 -0400 Received: from out3-smtp.messagingengine.com ([66.111.4.27]:38949) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hlx0j-0002I3-EX for 35521@debbugs.gnu.org; Fri, 12 Jul 2019 10:58:58 -0400 Received: from compute5.internal (compute5.nyi.internal [10.202.2.45]) by mailout.nyi.internal (Postfix) with ESMTP id B2F5A21FE9; Fri, 12 Jul 2019 10:58:51 -0400 (EDT) Received: from mailfrontend2 ([10.202.2.163]) by compute5.internal (MEProxy); Fri, 12 Jul 2019 10:58:51 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fastmail.com; h= from:to:cc:subject:in-reply-to:references:date:message-id :mime-version:content-type; s=fm3; bh=SPAVuoYMuswYYCLCsiFRf9fRua w5jE+Rr6AKfPTcD78=; b=gX7A7qPTXq2ovZzbNpSYzCcGh6WRV7wW3IkJJaCvfa WtqPTDERSdAXVon+qCIzcb34HC+V2IWc9FegwQ1DgQ41g6i2ksi4siBWQoRWuvMc cQHji9+DPaJON5fCOrbsEHgkGMKp1zkzBpY6c4GYI61m9NGVDoOHDa95IEApspZU QbI+tcctnzUQJHguve0fbG3tLuaSjLW3aCn5i4FKnDH5xivNlV5ifRzxkYn1aZnT W3wKbkqrFAHnhmH3nbkAogOmsg8/+2PCo4DI5AwK8E+UT8ZosafaoRSbfn32CWxP V5guWtAO9kAbK0BsASpi94UkHIUrkfr6zOFLOFB8VbmA== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-me-proxy :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm3; bh=SPAVuo YMuswYYCLCsiFRf9fRuaw5jE+Rr6AKfPTcD78=; b=19RMpCKJsmAl3Q3etsMg/r 9ZX0etY29VmC3ps4hdY9Qj6ycW+fb48RPYMUKIFDD8ZS4aT+W63PM2wvs+jl3tVB X6Bzxkjj/R4txaK2eqhRcP1DB30i1CMyaRM8CTICPcfJsbUJslKF6x/Jcq1pus0W DxcoFXnGakIAWLLLMjsLGlyA5SNRsHdcX/fKnN5aaDHxfboLG8upwLRxttSDzY6a gNLYkAJpfhPbvLE8bz1Iskq1RCVrrkSGPNtZY+JVsJe1oos20Cr5mgu91Jn4DZ2/ BZJepJnS4hz24EKlGxJgP3Wvxzbu5d3Mp+azvCEOKJ74al6/Z5gk4F/ER9C4POAA == X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgeduvddrhedtgdekudcutefuodetggdotefrodftvf curfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfghnecu uegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmdenuc fjughrpefhvffujghffgffkfggtgesghdtreertdertdenucfhrhhomhepofgrrhhiuhhs uceurghkkhgvuceomhgsrghkkhgvsehfrghsthhmrghilhdrtghomheqnecuffhomhgrih hnpehgihhthhhusgdrtghomhenucfkphepiedvrdduiedrvddviedrudegtdenucfrrghr rghmpehmrghilhhfrhhomhepmhgsrghkkhgvsehfrghsthhmrghilhdrtghomhenucevlh hushhtvghrufhiiigvpedt X-ME-Proxy: Received: from localhost (140.226.16.62.customer.cdi.no [62.16.226.140]) by mail.messagingengine.com (Postfix) with ESMTPA id E742A380074; Fri, 12 Jul 2019 10:58:50 -0400 (EDT) From: Marius Bakke To: Mark H Weaver Subject: Re: bug#35521: Mariadb test suite failures on x86_64-linux In-Reply-To: <87ims8mds7.fsf@devup.no> References: <87tveemt19.fsf@netris.org> <87tveemt19.fsf@netris.org> <87h8aemrow.fsf@netris.org> <87pnmil8dq.fsf_-_@gmail.com> <87wogpn6f7.fsf@devup.no> <87ef2xlgmb.fsf@netris.org> <87ims8mds7.fsf@devup.no> User-Agent: Notmuch/0.29.1 (https://notmuchmail.org) Emacs/26.2 (x86_64-pc-linux-gnu) Date: Fri, 12 Jul 2019 16:58:48 +0200 Message-ID: <878st3mh93.fsf@devup.no> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="==-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" X-Spam-Score: -0.7 (/) X-Debbugs-Envelope-To: 35521 Cc: Platoxia , Chris Marusich , 35521@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.7 (-) --==-=-= Content-Type: multipart/mixed; boundary="=-=-=" --=-=-= Content-Type: text/plain Marius Bakke writes: > Mark H Weaver writes: > >> Hi, >> >> Marius Bakke writes: >> >>> Chris Marusich writes: >>> >>>> Hi, >>>> >>>> I've been encountering this failure off and on for a few weeks now, and >>>> I'd like to help fix it. In short, it seems like non-deterministic test >>>> failures, to me. I think we should gather data and report the issue >>>> upstream, and maybe disable the offending tests in the meantime. >>> >>> I agree. I notice many of these failing tests are for the TokuDB >>> backend, which I doubt anyone is using in Guix anyway. >>> >>> Here is a patch that disables all tests mentioned in this report. I >>> would like to push it to core-updates. Are there others? >> >> I'm concerned by how frequently and casually we simply disable failing >> tests. What is the utility of running test suites at all, if this is >> how we respond? > > I had no idea this issue was so widespread until I noticed Berlins > builders hit it more often than not. I have not been able to reproduce > these failures on my machines. So it was kind of a panic reaction, > being the person responsible for running these tests and all. > > Looking further into the changes between 10.1.37 and 10.1.38, I notice > the 'tokudb.*' tests were enabled: > > https://github.com/MariaDB/server/commit/4c490d6df63695dc97b2c808e59954e6877d3a51 > > Watching the build on Berlin in real time, I also see that the test > output grind nearly to a halt while running those. > 'tokudb.hotindex-insert-2' took 2700439 milliseconds, or 45 minutes, if > I'm reading the test output correctly. > > The default test case timeout is 40 minutes (as specified in the Guix > package), but I'm using 80 for this build (60 was insufficient). > > I suspect the problem is that the 'tokudb.*' tests put a lot of strain > on the file system, which causes these other tests to fail. It's > interesting that disabling parallel build was insufficient though. Update: Berlin built mariadb twice on core-updates with this patch: --=-=-= Content-Type: text/x-patch Content-Disposition: inline; filename=db.diff diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm index 6bfeaad9a2..64bc0938b6 100644 --- a/gnu/packages/databases.scm +++ b/gnu/packages/databases.scm @@ -753,7 +753,7 @@ Language.") (with-directory-excursion "mysql-test" (invoke "./mtr" "--verbose" "--retry=3" - "--testcase-timeout=40" + "--testcase-timeout=80" "--suite-timeout=600" "--parallel" (number->string (parallel-job-count)) "--skip-test-list=unstable-tests")) --=-=-= Content-Type: text/plain Mark, Chris: Can you try this change with MariaDB 10.1.40 and see if it works for you? --=-=-=-- --==-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAl0ooCgACgkQoqBt8qM6 VPq/FggAtPX7+OrcwlSI12IbUfUqhEZnQrocXIy9okA1Bl5NiV+Hy8k2rfg5/Q1K Rqix5TSkPCGLPtQYkW/IIyZiPaJZQuKWpUlphbFlZDQ1vGs2Q4WlbS0gOLJnEV3T EkRaj2rweRsTKel5OPMSKaV3VGTc+TchyRFvhjGfhSrBdokrBD2Z6cSWtf4rhss3 rge+11QhSBX3ZIqSghSSTd9GZ1D4XCIEdvUfxK0qoWH1bLhSmTQh/PNV8+wtl1os Xmrb8zXNXu1WP2FyaCnPeDS2vOEYBDogT0kDszadxWC0lNXzwxuRfbc4dJVhcI/Q SWQXWbvO88PyTjRep6zJ5iu6DjG1fw== =RYR1 -----END PGP SIGNATURE----- --==-=-=--