From debbugs-submit-bounces@debbugs.gnu.org Tue Feb 08 07:55:54 2022 Received: (at 53463) by debbugs.gnu.org; 8 Feb 2022 12:55:54 +0000 Received: from localhost ([127.0.0.1]:45794 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nHQ2E-0002VL-2z for submit@debbugs.gnu.org; Tue, 08 Feb 2022 07:55:54 -0500 Received: from sender4-of-o51.zoho.com ([136.143.188.51]:21182) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1nHQ2C-0002VC-Rv for 53463@debbugs.gnu.org; Tue, 08 Feb 2022 07:55:53 -0500 ARC-Seal: i=1; a=rsa-sha256; t=1644324944; cv=none; d=zohomail.com; s=zohoarc; b=WMdIji/82VVd5HwtaGY6uh/HHU9xhIY91mPZhNnIda7iz4Ve+BrPWnqh8PMa0WHRIeeFTWmZuUfjDcy8/eWM2VocC2/o6NVjM0kQOgtS6NOrJm8AH+WaBlJdGzMLbnJvEfRz0jTibw69KKdG6sW6y/i9lYV3tsQBCmm24GgMCmY= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1644324944; h=Content-Type:Content-Transfer-Encoding:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:References:Subject:To; bh=K1IWJhjPN7Ml8sDC9T6p3HFftLtesOYkAyeyNuhMvSI=; b=FzlP+grqkaWZwsSNv7niM25LyL3TjDibFpjjnIXZsMa5lk5uawucSiGgfzoplGQiE9Tgn3mUT5P+iA7evG3bhz87Gg8BoNNvKBQ3ZuB1R/1Edr1+1dVT+zZ+Jn4pk+JnMecNZNF12kJgYrbavIB0SUVAQkI2Pn0gXEy4Ko9GSZs= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=elephly.net; spf=pass smtp.mailfrom=rekado@elephly.net; dmarc=pass header.from= DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1644324944; s=zoho; d=elephly.net; i=rekado@elephly.net; h=References:From:To:Cc:Subject:Date:In-reply-to:Message-ID:MIME-Version:Content-Type:Content-Transfer-Encoding; bh=K1IWJhjPN7Ml8sDC9T6p3HFftLtesOYkAyeyNuhMvSI=; b=GANElifdjM514f7c2C4WTFGuvXVA6J4QbrcXdUfpH9gwFUqoMiB0ynLbeJkhaipS vtAi1ZIGSOTbPGflHVnGtRRW8IcMVyCMogIDCiQJI7ofmillioGQf8DW/B0qB462dXB 4lz7AFJ0Tt7q1ZDCIoPTRD+JPc/roH/N5LhHPUsI= Received: from localhost (p54ad4e3e.dip0.t-ipconnect.de [84.173.78.62]) by mx.zohomail.com with SMTPS id 1644324927003948.9265188630875; Tue, 8 Feb 2022 04:55:27 -0800 (PST) References: <87leyt2jsr.fsf@gnu.org> <87sfsznh2z.fsf@gnu.org> <875ypv3qjo.fsf@gnu.org> <87zgn1aca4.fsf@gnu.org> User-agent: mu4e 1.6.10; emacs 27.2 From: Ricardo Wurmus To: Ludovic =?utf-8?Q?Court=C3=A8s?= Subject: Re: bug#53463: ci.guix.gnu.org not building the 'guix' job Date: Tue, 08 Feb 2022 13:52:21 +0100 In-reply-to: <87zgn1aca4.fsf@gnu.org> X-URL: https://elephly.net X-PGP-Key: https://elephly.net/rekado.pubkey X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC Message-ID: <87ee4dzfeu.fsf@elephly.net> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-ZohoMailClient: External X-Spam-Score: -0.0 (/) X-Debbugs-Envelope-To: 53463 Cc: Mathieu Othacehe , Tobias Geerinckx-Rice , 53463@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Ludovic Court=C3=A8s writes: > Hi, > > Mathieu Othacehe skribis: > >>> Oh! That indicates that it=E2=80=99s failing to offload to one of the >>> =E2=80=98localhost=E2=80=99 build machines specified in /etc/guix/machi= nes.scm. >>> Normally there=E2=80=99s an SSH tunnel set up for those, but I guess it= broke. >>> >>> Perhaps we can update /etc/guix/machines.scm to refer to armhf-linux >>> machines by their WireGuard IP? >> >> Seems like the right thing to do. This bit is also an unstaged change in >> the berlin maintenance repository, we should commit it. Tobias, could >> you have a look :) ? >> >> +(define powerpc64le >> + (list >> + ;; A VM donated/hosted by OSUOSL & administered by nckx. >> + ;; XXX: SSH tunnel via overdrive1: >> + ;; ssh -L 2224:p9.tobias.gr:22 hydra@10.0.0.3 >> + #;(build-machine >> + ;;(name "p9.tobias.gr") >> + (name "localhost") >> + (port 2224) >> + (user "hydra") >> + (systems '("powerpc64le-linux")) >> + (host-key "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIJEbRxJ6WqnNLYEMNDUK= FcdMtyZ9V/6oEfBFSHY8xE6A nckx")))) > > IIRC this machine is now running WireGuard, Tobias? If so, could you > change this to refer to its WireGuard IP and commit it? > >> I also found that other machines were unreachable and commented them: >> >> ;; CPU: 16 ARM Cortex-A72 cores >> ;; RAM: 32 GB >> - (list (build-machine >> + (list #;(build-machine >> ;;kreuzberg >> (name "10.0.0.9") >> (user "hydra") > > Ricardo, could you check what=E2=80=99s wrong with kreuzberg? Oh, the usual=E2=80=A6 --8<---------------cut here---------------start------------->8--- root@kreuzberg ~# guix shell wireguard-tools -- wg interface: wg0 public key: f9WGJTXp8bozJb0KxePjkOclF5pJUy1AomHWJHy80y4=3D private key: (hidden) listening port: 51820 peer: wOIfhHqQ+JQmskRS2qSvNRgZGh33UxFDi8uuSXOltF0=3D endpoint: 141.80.181.40:51820 allowed ips: 10.0.0.1/32 latest handshake: 2 days, 2 hours, 11 minutes, 13 seconds ago transfer: 292.79 MiB received, 6.05 GiB sent --8<---------------cut here---------------end--------------->8--- Whenever the build farm is awfully quiet (e.g. because of GC) the wireguard connection times out. I usually restart the cuirass-remote-worker and everything=E2=80=99s fine again. Today I got some additional SD cards for these machines, so I=E2=80=99m goi= ng to reconfigure them (locally, because of the =E2=80=9Cguix deploy=E2=80=9D bug= ) and then move them to the data centre. Once reconfigured they will keep the wireguard connection alive all by themselves, so no manual intervention is necessary. I didn=E2=80=99t reconfigure them locally because I hoped we would be able = to make time for the =E2=80=9Cguix deploy=E2=80=9D bug, but things turned out = differently. --=20 Ricardo