From debbugs-submit-bounces@debbugs.gnu.org Thu Apr 04 11:48:17 2019 Received: (at submit) by debbugs.gnu.org; 4 Apr 2019 15:48:17 +0000 Received: from localhost ([127.0.0.1]:44591 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hC4bA-0003BY-Ix for submit@debbugs.gnu.org; Thu, 04 Apr 2019 11:48:16 -0400 Received: from eggs.gnu.org ([209.51.188.92]:48168) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1hC4b7-0003BI-PE for submit@debbugs.gnu.org; Thu, 04 Apr 2019 11:48:14 -0400 Received: from lists.gnu.org ([209.51.188.17]:59700) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from ) id 1hC4b2-0001Rq-4t for submit@debbugs.gnu.org; Thu, 04 Apr 2019 11:48:08 -0400 Received: from eggs.gnu.org ([209.51.188.92]:34295) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1hC4az-0004id-Rp for bug-guix@gnu.org; Thu, 04 Apr 2019 11:48:08 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM, HTML_MESSAGE,URIBL_BLOCKED autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1hC4ay-0001OT-N2 for bug-guix@gnu.org; Thu, 04 Apr 2019 11:48:05 -0400 Received: from mail-pf1-x432.google.com ([2607:f8b0:4864:20::432]:39042) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1hC4at-00012z-PQ; Thu, 04 Apr 2019 11:48:00 -0400 Received: by mail-pf1-x432.google.com with SMTP id i17so1572665pfo.6; Thu, 04 Apr 2019 08:47:49 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:subject:from:in-reply-to:date:cc:message-id:references :to; bh=q+yRBi+kJPa0MqZjoaZwtE+95gOZNSZBIdBKnaMOiaE=; b=DdR2kzYZrv19KU61RHdjVp7XExFpnqlK2BmFdceYIv61lz3FPPVNp6TaHJN2sv4x2+ sMNZ3p1HYPDZpQDN6xdAe5N2pyfUAnsPxZztY8kQ9kkQChBoGsyGGmt+KFpMPAZnz8ip ktX4uQMTt+eiZH/PLDIZ1jLhS2eodduBsLXIy7Ld++LAdinVvtw4HwFmsIcGoCNrJ0JM 04fBDlhHlusQFMpa7FIycAi7bUXI0J9inE0bu7Ik69h5nJ+8aqFg0S7HBEwcYlQQHRS/ 2AYDxO1kQIgzWAL9pcLRKiabDijd5j/L7YAb5RlCtEh7xFerGL7MfeHZYfa6zwaxIP4j bGLA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc :message-id:references:to; bh=q+yRBi+kJPa0MqZjoaZwtE+95gOZNSZBIdBKnaMOiaE=; b=S6dJOXUblxUkA5bKn9CvQv1gfiejZtLRbTZyI/zrX98j/ImvwWT14t8dtAFs0DS5np qSUHUY13MF9Il7GjkgvhyPhbiXlmwFZ3uXsYbALsn1AUC8Y2nqFFWMnF8fIV1LtoJs3I xdCRmx09NOh7ncmI4B9F9vhzkLhsJO8E2OQrobJoXMQ8CFFaK6r6EOAiqDLymcOLy2uN VBV+QaSYPCwga86TSLvRHtxygRkCkUSoaGCNxk/E0woZm5DYiXF/Dv9ddj4LALra22P3 P1qHCzGpxQb8lvOH4nEa8+V+XuXNEZ9dwrsv5u/xesamhcLx7sgop+/LFcj8tskl8tXH Y4lw== X-Gm-Message-State: APjAAAWpsxHbr+fuCheXpDxotESatUDnAXeAd13fajwMi8k1FUkDEwpz kLSUR0mn4KQXC147PFQO2qI= X-Google-Smtp-Source: APXvYqyu6pByvdYdZb7fSC4p73unnmc9u/usL6PBT8S7gOgvgwpPu3WEfIV4IhJcbG0zzyYDy/i04Q== X-Received: by 2002:a62:1c07:: with SMTP id c7mr6619940pfc.159.1554392867934; Thu, 04 Apr 2019 08:47:47 -0700 (PDT) Received: from ?IPv6:2601:602:9a00:1784:9d2f:e342:1236:46d1? ([2601:602:9a00:1784:9d2f:e342:1236:46d1]) by smtp.gmail.com with ESMTPSA id y12sm56689223pgq.64.2019.04.04.08.47.46 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 04 Apr 2019 08:47:47 -0700 (PDT) Content-Type: multipart/alternative; boundary="Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D" Mime-Version: 1.0 (Mac OS X Mail 11.5 \(3445.9.1\)) Subject: Re: bug#35139: Rust builds systematically time out From: Ivan Petkov In-Reply-To: <87bm1mglus.fsf@gmx.com> Date: Thu, 4 Apr 2019 08:47:42 -0700 Message-Id: <101FBDE5-97FA-4449-9076-DD24C56B8715@gmail.com> References: <878swqtabb.fsf@gnu.org> <87bm1mglus.fsf@gmx.com> To: Pierre Langlois , =?utf-8?Q?Ludovic_Court=C3=A8s?= X-Mailer: Apple Mail (2.3445.9.1) X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2607:f8b0:4864:20::432 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Spam-Score: 1.0 (+) X-Debbugs-Envelope-To: submit Cc: bug-guix@gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -0.0 (/) --Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=utf-8 > On Apr 4, 2019, at 1:59 AM, Ludovic Court=C3=A8s wrote: >=20 > The build nodes may be slower than the front-end, but still, it seems > unlikely that it would take more than 6h there. (That could happen if > the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly = parallel=E2=80=9D, but > we=E2=80=99re running tests with =E2=80=98-j1=E2=80=99.) >=20 > To summarize, there are two problems: >=20 > 1. Rust takes too long to build. What can we do about it? Enable > parallel builds? Rust tests are designed to run in parallel, as long as you have enough RAM, file descriptors, etc. available on the machine for the amount of concurrency being used. The compiler test suite is largely just = compiling files, so the most important resource is probably available RAM/swap. > On Apr 4, 2019, at 2:28 AM, Pierre Langlois = wrote: >=20 > One thing I suggested in the past was to remove the check phase *only* > for rust packages used for bootstrapping. This way we still run the > tests for the final rust but not at every step in the chain. >=20 > Although, I wonder if we're more likely to miss a bug if we do this, = I'm > not sure. Although that definitely will speed the bootstrap chain, I=E2=80=99m = concerned that if a dependency package ever gets updated and breaks things we = wouldn=E2=80=99t know without running the test suite. Maybe if the bootstrapped versions don=E2=80=99t ever change skipping = the check phase will be safe, but I think we should try running parallel tests = first and see how far that gets us. =E2=80=94Ivan= --Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=utf-8

On Apr 4, 2019, at 1:59 AM, Ludovic Court=C3=A8= s <ludo@gnu.org> = wrote:

The build nodes may be slower = than the front-end, but still, it seems
unlikely that it = would take more than 6h there.  (That could happen if
the test suite, which lasts 2.1h, were =E2=80=9Cembarrassingly = parallel=E2=80=9D, but
we=E2=80=99re running tests with = =E2=80=98-j1=E2=80=99.)

To summarize, there = are two problems:

 1. Rust takes too = long to build.  What can we do about it?  Enable
    parallel builds?

Rust = tests are designed to run in parallel, as long as you have = enough
RAM, file descriptors, etc. available on the = machine for the amount of
concurrency being used. = The compiler test suite is largely just compiling
files, so the most important resource is probably available = RAM/swap.

On Apr 4, 2019, at 2:28 AM, Pierre Langlois = <pierre.langlois@gmx.com> wrote:
One thing I = suggested in the past was to remove the check phase *only*
for rust packages used for = bootstrapping. This way we still run the
tests for the final rust but not at every step in the = chain.

Although, I = wonder if we're more likely to miss a bug if we do this, I'm
not sure.

Although that definitely will speed the bootstrap chain, = I=E2=80=99m concerned that
if a dependency package = ever gets updated and breaks things we wouldn=E2=80=99t
know without running the test suite.

Maybe if the bootstrapped versions = don=E2=80=99t ever change skipping the check
phase = will be safe, but I think we should try running parallel tests = first
and see how far that gets us.

=E2=80=94Ivan
= --Apple-Mail=_57EEEA7D-F8AE-4F99-BEDF-E207A558071D--