From debbugs-submit-bounces@debbugs.gnu.org Tue Jan 18 06:33:52 2022 Received: (at 48463) by debbugs.gnu.org; 18 Jan 2022 11:33:52 +0000 Received: from localhost ([127.0.0.1]:49115 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n9mkK-00041r-Eg for submit@debbugs.gnu.org; Tue, 18 Jan 2022 06:33:52 -0500 Received: from mail-wm1-f65.google.com ([209.85.128.65]:53043) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from ) id 1n9mkI-00041e-Mi for 48463@debbugs.gnu.org; Tue, 18 Jan 2022 06:33:51 -0500 Received: by mail-wm1-f65.google.com with SMTP id v123so26976412wme.2 for <48463@debbugs.gnu.org>; Tue, 18 Jan 2022 03:33:50 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=message-id:subject:from:to:cc:date:in-reply-to:references :user-agent:mime-version:content-transfer-encoding; bh=gMG1u0isEcr54Dajof74VgxUhwYhT4ewYlz3afAvhbU=; b=UTuCjGgWSmG5T2lyGOkCdiwDxRxcuEpmuVDEh4cucrmOb7eLdZ+MYcs4lcyt653unU lrcY4jRuSE4UYJGBmiv9/fzWbJx0CnoNobTLiT38kY7KkfYZ1nx9g6WN/MPrrao1Yi2y xl0raOJU/to4taSzLLuO7yMI0AaqPS1dShmBYMSF8uhfVTOBO+X6s4tQTbPwgcQZV9MH h6ZFbnVBRZggc/poF2wrMVWWeYvUe9VMX5hDZIwravZ4L2WisnwroF3m6Simc3/Qqasp 6hMSVOrmKFqEfjdewMEBUD5WpM7+6umaD40Eloee7dzoIsEIIUPNpVQH47fRKVuvIKhk ca1Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:subject:from:to:cc:date:in-reply-to :references:user-agent:mime-version:content-transfer-encoding; bh=gMG1u0isEcr54Dajof74VgxUhwYhT4ewYlz3afAvhbU=; b=fB6ztLUVXUtnEGnEFS5XS+9H2PHyeaI3pMaUe+8+EckHsd36hnZ9VzNNc9lcwKPlNo lOqHidHNxL/hP7Ys6Hao3jeBr09+14djbC7bykBD4pWs5yTNO8bKMsRt54hE+9bpHEY6 LEmn7NrHMul1LgGCXKsuyOZtCW70grdwqvibj57XfemIXgIfO49hmxibpK6Hrdonbi4c BGFMnCqF+InpWmqx7vbVgbkW8xZMT9CxGCCKCq1yiOwJ9n9UR5NZe2e2RnjdfohqYIOU YWVfAXPirkoLwQG7yztFl5ZlC4RYtbHKm3QMUSmByUuM/Ch77j6QtJeWFsoVDLt07+gq W6ew== X-Gm-Message-State: AOAM5339v3HHhfXeld3H2UQzEgBQuW5HkTnPzg9aEnQWAu7SvuvtP8lf P8COT0IoknH2s9OlVHTvlbA= X-Google-Smtp-Source: ABdhPJzDwTR2aRR0zyVcqDC29i6U8+iLee9/KWIe8o/L0n/oDtLt8FktQbHU0Xd+IB/IFFwxjlKVqQ== X-Received: by 2002:a5d:6f14:: with SMTP id ay20mr24427127wrb.164.1642505624585; Tue, 18 Jan 2022 03:33:44 -0800 (PST) Received: from nijino.fritz.box (85-127-52-93.dsl.dynamic.surfer.at. [85.127.52.93]) by smtp.gmail.com with ESMTPSA id h11sm2117561wmb.12.2022.01.18.03.33.43 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 18 Jan 2022 03:33:44 -0800 (PST) Message-ID: Subject: Re: [bug#48463] gnu: Add j. From: Liliana Marie Prikler To: elaexuotee@wilsonb.com Date: Tue, 18 Jan 2022 12:33:38 +0100 In-Reply-To: <2GLLHI6YET96V.279HCOFQSZLLF@wilsonb.com> References: <3LOAUDT0FLL4U.2SOD925YP915T@wilsonb.com> <8b853d0585505ce29c9afc638b644fa34805e6c0.camel@student.tugraz.at> <293L8YPQS4CLB.3VK1B1A36XNAY@wilsonb.com> <5d30160bd2a4592459cd407f99cbd3edadb1db1b.camel@student.tugraz.at> <27DCD25Y68ZWJ.2HRC4G65PWIA7@wilsonb.com> <94f4625dcb0479d873cf60449631527e841fd457.camel@gmail.com> <2JQJMV0O718S1.31FZE8GKCTLPF@wilsonb.com> <90704c2259f576a14fb1268219e8c0dc2b3bf289.camel@telenet.be> <2P322C327XW0Q.21O5A4IFGMNDI@wilsonb.com> <72aff035c93f9f91afa54ef5b51c7381b0b02ccb.camel@gmail.com> <3MMTDZQJQ8IR6.334ZWY8AD0487@wilsonb.com> <62d37956f16c08bc4ce26e44da16dce704ddd0f8.camel@gmail.com> <24ZUUMG4QYSHN.2OS7YAMCKREUA@wilsonb.com> <83aba994536bec60f79900d551d4801c967742bd.camel@gmail.com> <25Z6NGGGNJYD1.3S7A1QLFX7I9Y@wilsonb.com> <24FU0VP6N4ZZ7.3PE5LG30BSNUQ@wilsonb.com> <2EZU214MJAIBY.3EXSPSUMS5WW5@wilsonb.com> <3T4YAXDRKZCAW.2LXSUL7VDA46J@wilsonb.com> <2GLLHI6YET96V.279HCOFQSZLLF@wilsonb.com> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.42.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 48463 Cc: Maxime Devos , 48463@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -1.0 (-) Am Dienstag, dem 18.01.2022 um 19:40 +0900 schrieb elaexuotee@wilsonb.com: > > FWIW I don't quite think that fat binaries are the issue here, but > > that building AVX blows up the check phase in the way we currently > > do. It's a similar issue w.r.t. check? for cross-compiling.  In my > > opinion only the base feature set can be checked unless the CPU > > supports the same features as the target. > > Oh, good. Apparently, I mis-interpreted your original concerns. > > > I think if we want to do fat binaries, the solution would be to > > build all three variants from make-j and tests only the base > > variants, and then have a non-substitutable wrapper package, that > > takes that as an input and simply provides a wrapper tuned to the > > current CPU features. > > If you want, you can then rerun the correct tests in the wrapper > > package.  The package returned by make-j could itself also provide > > three binaries just in case someone doesn't want to generate the > > wrapper package, but knows they can run ijconsole-avx2 just fine. > > WDYT? > > Slick. Let me check my understanding against yours with some > specifics: > > - Name make-j's package 'jsoftware-j-lib' which >   1. Builds all upstream stuff, and >   2. Only runs libj.so (non-avx) tests; > > - Create non-substitutable 'jsoftware-j' wrapper package which >   1. Detects SSE extensions at build time and specializes the > 'ijconsole' >      script accordingly, and >   2. Runs tests if and only if an avx or avx2 variant. > > Does those points jibe with your thoughts on the fat binaries > approach? That's roughly what I was suggesting, yes. > Glancing around at the CPU tuning stuff, it looks like tunable > packages end up getting a 'cpu-tuning' property which holds the > microartecture name passed to -march. AVX first landed in Sandy > Bridge, and AVX2 in Haswell, so assuming cpu-tuning is accessible at > build time, it should be easy enough to condition the build phase on > those. > > Regarding the check phase, here's a relevant comment in > guix/transformations.scm(tuned-package):552: > >     ;; The machine building this package may or may not be able to > run code >     ;; for MICRO-ARCHITECTURE.  Because of that, skip tests; they are > run for >     ;; the "baseline" variant anyway. > > Which I read to mean that the check phase should explicitly use > libj.so, ignoring libj-avx.so and libj-avx2.so. Running specialized > tests in a non-substitutable wrapper seems potentially better in this > particular case. > > Thoughts? I can't say that I agree with your reasoning here. Essentially, what you (or rather the J authors) are doing here is trying to outsmart the compiler, while Guix assumes, that -march=WHATEVER ought to both suffice in terms of performance and still be correct. I'm leaning more towards the assumption that Guix makes here. If you want to enable tests on your local machine, you can still do so by inheriting the transformed package. Perhaps we should add a "-- with-tests" transformation as a counterpart to the already existing -- without-tests, which would forcefully enable checks that have been disabled elsewhere. Furthermore, tuned-package would still support substitution, you just need to tell your CI to build for particular micro-architectures. If you're part of a sufficiently large dev team that uses J with Guix tooling, that makes a difference between everyone running checks Monday in the morning or simply fetching a build that has already succeeded on Sunday. Cheers