From debbugs-submit-bounces@debbugs.gnu.org Thu Feb 04 04:36:01 2016 Received: (at 20272) by debbugs.gnu.org; 4 Feb 2016 09:36:01 +0000 Received: from localhost ([127.0.0.1]:58927 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aRGKX-0006xg-CL for submit@debbugs.gnu.org; Thu, 04 Feb 2016 04:36:01 -0500 Received: from eggs.gnu.org ([208.118.235.92]:58696) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from ) id 1aRGKW-0006xN-Ab for 20272@debbugs.gnu.org; Thu, 04 Feb 2016 04:36:00 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1aRGKN-00058l-Sz for 20272@debbugs.gnu.org; Thu, 04 Feb 2016 04:35:55 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.3 required=5.0 tests=BAYES_50,RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:46234) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1aRGKN-00058h-PH; Thu, 04 Feb 2016 04:35:51 -0500 Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:32966 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from ) id 1aRGKM-0008SG-QF; Thu, 04 Feb 2016 04:35:51 -0500 From: ludo@gnu.org (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: Mark H Weaver Subject: Re: bug#20272: Support reproducible builds References: <87twws88hr.fsf@gnu.org> <87d1sdqjs2.fsf@netris.org> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 16 =?utf-8?Q?Pluvi=C3=B4se?= an 224 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x3D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Thu, 04 Feb 2016 10:35:48 +0100 In-Reply-To: <87d1sdqjs2.fsf@netris.org> (Mark H. Weaver's message of "Wed, 03 Feb 2016 21:41:33 -0500") Message-ID: <878u30g6mj.fsf@gnu.org> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Received-From: 2001:4830:134:3::e X-Spam-Score: -5.5 (-----) X-Debbugs-Envelope-To: 20272 Cc: 20272@debbugs.gnu.org X-BeenThere: debbugs-submit@debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: debbugs-submit-bounces@debbugs.gnu.org Sender: "Debbugs-submit" X-Spam-Score: -5.5 (-----) Mark H Weaver skribis: > ludo@gnu.org (Ludovic Court=C3=A8s) writes: > >> Currently .go files embed randomly-generated symbols stemming from >> =E2=80=98syntax-session-id=E2=80=99, which prevents reproducible builds = (see >> .) >> >> One way to fix it would be to allow users to specify a random seed used >> when generating session ids, and to make that available as a >> command-line option to =E2=80=98guild compile=E2=80=99. (GCC does somet= hing similar >> with its =E2=80=98-frandom-seed=E2=80=99 option.) > > We could add this, but it is not analogous to the -frandom-seed option > where it is okay to give it the same value everywhere. Users would need > to ensure that distinct session-ids are used for every invocation of > Guile. With GCC the common idiom is to use =E2=80=98-frandom-seed=3D$source_file= =E2=80=99. However, it would be best if =E2=80=98guild compile=E2=80=99 would choose t= he seed deterministically by default somehow, because we cannot expect all users to add the new flag and use properly. What about having =E2=80=98guild compile=E2=80=99 use the canonical file na= me of the source being compiled (or a hash thereof) as the seed? > More precisely, users of this feature would need to observe the > following restriction, or else unspecified behavior may result: > > If A.go is generated by a Guile session with session-id A, and > B.go is generated by a Guile session with session-id B, and > they are both loaded into a Guile session with session-id C, then > A, B, and C must all be distinct session-ids. Right. I wonder if we could detect collisions. Ideally each .go could record its session ID, but that=E2=80=99s probably not feasible in 2.0. > One more thing: even with a deterministic session-id, the multi-threaded > compiling of *.go files recently added to Guix will lead to > non-deterministic outputs. I'm not sure how to make this work > deterministically with concurrency. Even if we switched to a fixed > number of concurrent processes with one thread each, when process A > attempts to load a .go file that is produced by process B, it is not > deterministic whether it will be there. If it is there, the macros in > that .go file will contain B's session-id, and if not, they will contain > A's session-id. Hmm, OK. Well, let=E2=80=99s keep this use case aside for now. >> Probably, =E2=80=98syntax-session-id=E2=80=99 would have to be a SRFI-39= parameter > > This might adversely affect the efficiency of our macro expander on > platforms with slow thread local variables, and I'm not sure what it > would buy us. If the idea is that it would allow us to build things in > multiple threads, I think that won't work anyway, for the reasons given > above. I was just thinking that it would be a convenient interface for (scripts compile) to specify the session ID to use. I think the first thing to do is to change =E2=80=98fresh_syntax_session_id= =E2=80=99 so that they can use a user-specified seed, when available, instead of =E2=80=98scm_i_random_bytes_from_platform=E2=80=99. WDYT? Thanks for working on it! :-) Ludo=E2=80=99.