Gettext introduces timestamps in .mo files

OpenSubmitted by Ludovic Courtès.
Details
4 participants
  • Julien Lepiller
  • Ludovic Courtès
  • Miguel Ángel Arruga Vivas
  • zimoun
Owner
unassigned
Severity
normal
L
L
Ludovic Courtès wrote on 24 Mar 2017 23:54
Download
(address . bug-guix@gnu.org)
8760iyxp3t.fsf@gnu.org
Download
Gettext 0.19.8.1 (current core-updates,77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mofile, without honoring SOURCE_DATE_EPOCH, which leads tonon-reproducible builds (for example ‘guix’).
Ludo’.
Z
Z
zimoun wrote on 1 Dec 2020 19:46
Download
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 26247@debbugs.gnu.org)
865z5luzxx.fsf@gmail.com
Download
Hi Ludo,
This old bug #26247 about timestamp in .mo files from Gettext is stillopen:
http://issues.guix.gnu.org/issue/26247
On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:
Toggle quote (5 lines)> Gettext 0.19.8.1 (current core-updates,> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo> file, without honoring SOURCE_DATE_EPOCH, which leads to> non-reproducible builds (for example ‘guix’).
Is it still relevant? Since Gettext is now at 0.20.1. How can Ireproduce the issue? Because the usual:
guix build gettext --no-grafts --check
works fine.

Cheers,simon
L
L
Ludovic Courtès wrote on 3 Dec 2020 11:32
Download
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 26247@debbugs.gnu.org)
87v9djgox5.fsf@gnu.org
Download
Hi Simon,
zimoun <zimon.toutoune@gmail.com> skribis:
Toggle quote (10 lines)> On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:>>> Gettext 0.19.8.1 (current core-updates,>> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo>> file, without honoring SOURCE_DATE_EPOCH, which leads to>> non-reproducible builds (for example ‘guix’).>> Is it still relevant? Since Gettext is now at 0.20.1. How can I> reproduce the issue?
I still see this:
Toggle snippet (19 lines)$ guix challenge guix --substitute-urls="https://ci.guix.gnu.org https://bayfront.guix.gnu.org"updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0%/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe contents differ: no local build for '/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe' https://ci.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe: 13wvxga668grzs0p6sp0ghvdiy96nc9w71vs11djjkypsaf7wpw1 https://bayfront.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe: 1rpwil9h2whjd9dbwpikxn8prkg924nhljglwj9yjijh578nlfr8 differing files: /share/locale/en@quot/LC_MESSAGES/guix.mo /share/locale/en@quot/LC_MESSAGES/guix-packages.mo /share/locale/en@boldquot/LC_MESSAGES/guix.mo /share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo /share/info/guix-cookbook.de.info.gz /lib/guile/3.0/site-ccache/guix/workers.go /lib/guile/3.0/site-ccache/guix/ui.go /lib/guile/3.0/site-ccache/guix/swh.go /lib/guile/3.0/site-ccache/guix/svn-download.go[…]
‘--diff=diffoscope’ is not an option here because it takes too longlooking at all the .go files…
A focused diff shows this:
Toggle snippet (41 lines)$ diffoscope --exclude-directory-metadata=yes /tmp/{t1,t2}/share/locale/en@quot/--- /tmp/t1/share/locale/en@quot/+++ /tmp/t2/share/locale/en@quot/│ --- /tmp/t1/share/locale/en@quot/LC_MESSAGES├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES│ │ --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix-packages.mo│ ├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix-packages.mo│ │ ├── msgunfmt {}│ │ │ @@ -1,12 +1,12 @@│ │ │ msgid ""│ │ │ msgstr ""│ │ │ "Project-Id-Version: guix 1.2.0-3.35a32fe\n"│ │ │ "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"│ │ │ -"PO-Revision-Date: 2020-11-29 18:33+0000\n"│ │ │ +"PO-Revision-Date: 2020-12-02 10:10+0000\n"│ │ │ "Last-Translator: Automatically generated\n"│ │ │ "Language-Team: none\n"│ │ │ "Language: en@quot\n"│ │ │ "MIME-Version: 1.0\n"│ │ │ "Content-Type: text/plain; charset=UTF-8\n"│ │ │ "Content-Transfer-Encoding: 8bit\n"│ │ │ "Plural-Forms: nplurals=2; plural=(n != 1);\n"│ │ --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix.mo│ ├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix.mo│ │ ├── msgunfmt {}│ │ │ @@ -1,12 +1,12 @@│ │ │ msgid ""│ │ │ msgstr ""│ │ │ "Project-Id-Version: guix 1.2.0-3.35a32fe\n"│ │ │ "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"│ │ │ -"PO-Revision-Date: 2020-11-29 18:33+0000\n"│ │ │ +"PO-Revision-Date: 2020-12-02 10:10+0000\n"│ │ │ "Last-Translator: Automatically generated\n"│ │ │ "Language-Team: none\n"│ │ │ "Language: en@quot\n"│ │ │ "MIME-Version: 1.0\n"│ │ │ "Content-Type: text/plain; charset=UTF-8\n"│ │ │ "Content-Transfer-Encoding: 8bit\n"│ │ │ "Plural-Forms: nplurals=2; plural=(n != 1);\n"
Ludo’.
J
J
Julien Lepiller wrote on 3 Dec 2020 13:07
Download
(address . 26247@debbugs.gnu.org)
EB5BE5DC-EF6C-4311-9A07-B2A06E71E65A@lepiller.eu
Download
So it's not gettext itself, but our build system that generates the en@quote and en@boldquote files. Do we really need them? If so, we should find a way to generate them reproducibly.
Le 3 décembre 2020 05:32:54 GMT-05:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
Toggle quote (91 lines)>Hi Simon,>>zimoun <zimon.toutoune@gmail.com> skribis:>>> On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:>>>>> Gettext 0.19.8.1 (current core-updates,>>> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in>.mo>>> file, without honoring SOURCE_DATE_EPOCH, which leads to>>> non-reproducible builds (for example ‘guix’).>>>> Is it still relevant? Since Gettext is now at 0.20.1. How can I>> reproduce the issue?>>I still see this:>>--8<---------------cut here---------------start------------->8--->$ guix challenge guix --substitute-urls="https://ci.guix.gnu.org>https://bayfront.guix.gnu.org">updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0%>/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe>contents differ:>no local build for>'/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe'>https://ci.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe:>13wvxga668grzs0p6sp0ghvdiy96nc9w71vs11djjkypsaf7wpw1>https://bayfront.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe:>1rpwil9h2whjd9dbwpikxn8prkg924nhljglwj9yjijh578nlfr8> differing files:> /share/locale/en@quot/LC_MESSAGES/guix.mo> /share/locale/en@quot/LC_MESSAGES/guix-packages.mo> /share/locale/en@boldquot/LC_MESSAGES/guix.mo> /share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo> /share/info/guix-cookbook.de.info.gz> /lib/guile/3.0/site-ccache/guix/workers.go> /lib/guile/3.0/site-ccache/guix/ui.go> /lib/guile/3.0/site-ccache/guix/swh.go> /lib/guile/3.0/site-ccache/guix/svn-download.go>[…]>--8<---------------cut here---------------end--------------->8--->>‘--diff=diffoscope’ is not an option here because it takes too long>looking at all the .go files…>>A focused diff shows this:>>--8<---------------cut here---------------start------------->8--->$ diffoscope --exclude-directory-metadata=yes>/tmp/{t1,t2}/share/locale/en@quot/>--- /tmp/t1/share/locale/en@quot/>+++ /tmp/t2/share/locale/en@quot/>│ --- /tmp/t1/share/locale/en@quot/LC_MESSAGES>├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES>│ │ --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix-packages.mo>│ ├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix-packages.mo>│ │ ├── msgunfmt {}>│ │ │ @@ -1,12 +1,12 @@>│ │ │ msgid "">│ │ │ msgstr "">│ │ │ "Project-Id-Version: guix 1.2.0-3.35a32fe\n">│ │ │ "Report-Msgid-Bugs-To: bug-guix@gnu.org\n">│ │ │ -"PO-Revision-Date: 2020-11-29 18:33+0000\n">│ │ │ +"PO-Revision-Date: 2020-12-02 10:10+0000\n">│ │ │ "Last-Translator: Automatically generated\n">│ │ │ "Language-Team: none\n">│ │ │ "Language: en@quot\n">│ │ │ "MIME-Version: 1.0\n">│ │ │ "Content-Type: text/plain; charset=UTF-8\n">│ │ │ "Content-Transfer-Encoding: 8bit\n">│ │ │ "Plural-Forms: nplurals=2; plural=(n != 1);\n">│ │ --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix.mo>│ ├── +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix.mo>│ │ ├── msgunfmt {}>│ │ │ @@ -1,12 +1,12 @@>│ │ │ msgid "">│ │ │ msgstr "">│ │ │ "Project-Id-Version: guix 1.2.0-3.35a32fe\n">│ │ │ "Report-Msgid-Bugs-To: bug-guix@gnu.org\n">│ │ │ -"PO-Revision-Date: 2020-11-29 18:33+0000\n">│ │ │ +"PO-Revision-Date: 2020-12-02 10:10+0000\n">│ │ │ "Last-Translator: Automatically generated\n">│ │ │ "Language-Team: none\n">│ │ │ "Language: en@quot\n">│ │ │ "MIME-Version: 1.0\n">│ │ │ "Content-Type: text/plain; charset=UTF-8\n">│ │ │ "Content-Transfer-Encoding: 8bit\n">│ │ │ "Plural-Forms: nplurals=2; plural=(n != 1);\n">--8<---------------cut here---------------end--------------->8--->>Ludo’.
Attachment: file
L
L
Ludovic Courtès wrote on 3 Dec 2020 15:41
Download
(name . Julien Lepiller)(address . julien@lepiller.eu)
87blfbdka0.fsf@gnu.org
Download
Julien Lepiller <julien@lepiller.eu> skribis:
Toggle quote (2 lines)> So it's not gettext itself, but our build system that generates the en@quote and en@boldquote files. Do we really need them? If so, we should find a way to generate them reproducibly.
They’re generated automatically by the gettext Makefilery that we use,so I don’t think there’s much we can do?
We could remove them from ‘LINGUAS’, but we’d be losing something,wouldn’t we?
Ludo’.
M
M
Miguel Ángel Arruga Vivas wrote on 11 Dec 2020 14:26
Download
(name . Ludovic Courtès)(address . ludo@gnu.org)
875z58fp8e.fsf@gmail.com
Download
Hi!
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (9 lines)> Julien Lepiller <julien@lepiller.eu> skribis:>>> So it's not gettext itself, but our build system that generates the>> en@quote and en@boldquote files. Do we really need them? If so, we>> should find a way to generate them reproducibly.>> They’re generated automatically by the gettext Makefilery that we use,> so I don’t think there’s much we can do?
The issue isn't on those files but on POT generation. Currentlyxgettext doesn't honor SOURCE_DATE_EPOCH to fill POT-Creation-Date,which is used to fill PO-Revision-Date for auto-generated po files.These files are included into tarballs generated by make dist, thereforeits date is already fixed, but they aren't present on our git tree---forobvious reasons.
This bug has already been reported upstream[1] and probably I'll push itas soon as I have more test cases prepared and receive a brief review,but I can prepare a patch for previous versions if it's needed too.
Best regards,Miguel
[1] https://savannah.gnu.org/bugs/?59658
L
L
Ludovic Courtès wrote on 14 Dec 2020 10:09
Download
(name . Miguel ÁngelArrugaVivas)(address . rosen644835@gmail.com)
87sg88wy7p.fsf@gnu.org
Download
Hi Miguel,
Miguel Ángel Arruga Vivas <rosen644835@gmail.com> skribis:
Toggle quote (18 lines)> Ludovic Courtès <ludo@gnu.org> writes:>>> Julien Lepiller <julien@lepiller.eu> skribis:>>>>> So it's not gettext itself, but our build system that generates the>>> en@quote and en@boldquote files. Do we really need them? If so, we>>> should find a way to generate them reproducibly.>>>> They’re generated automatically by the gettext Makefilery that we use,>> so I don’t think there’s much we can do?>> The issue isn't on those files but on POT generation. Currently> xgettext doesn't honor SOURCE_DATE_EPOCH to fill POT-Creation-Date,> which is used to fill PO-Revision-Date for auto-generated po files.> These files are included into tarballs generated by make dist, therefore> its date is already fixed, but they aren't present on our git tree---for> obvious reasons.
OK.
Toggle quote (4 lines)> This bug has already been reported upstream[1] and probably I'll push it> as soon as I have more test cases prepared and receive a brief review,> but I can prepare a patch for previous versions if it's needed too.
[...]
Toggle quote (2 lines)> [1] https://savannah.gnu.org/bugs/?59658
Thanks for getting to the bottom of this and for preparing a patchupstream!
In ‘core-updates’, we could either wait for the next Gettext release orapply your patch in the meantime.
Ludo’.
?
Your may also send email to 26247@debbugs.gnu.org to comment.