Gettext introduces timestamps in .mo files

  • Open
  • quality assurance status badge
Details
4 participants
  • Julien Lepiller
  • Ludovic Courtès
  • Miguel Ángel Arruga Vivas
  • zimoun
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 24 Mar 2017 23:54
(address . bug-guix@gnu.org)
8760iyxp3t.fsf@gnu.org
Gettext 0.19.8.1 (current core-updates,
77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo
file, without honoring SOURCE_DATE_EPOCH, which leads to
non-reproducible builds (for example ‘guix’).

Ludo’.
Z
Z
zimoun wrote on 1 Dec 2020 19:46
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 26247@debbugs.gnu.org)
865z5luzxx.fsf@gmail.com
Hi Ludo,

This old bug #26247 about timestamp in .mo files from Gettext is still
open:


On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:

Toggle quote (5 lines)
> Gettext 0.19.8.1 (current core-updates,
> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo
> file, without honoring SOURCE_DATE_EPOCH, which leads to
> non-reproducible builds (for example ‘guix’).

Is it still relevant? Since Gettext is now at 0.20.1. How can I
reproduce the issue? Because the usual:

guix build gettext --no-grafts --check

works fine.


Cheers,
simon
L
L
Ludovic Courtès wrote on 3 Dec 2020 11:32
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 26247@debbugs.gnu.org)
87v9djgox5.fsf@gnu.org
Hi Simon,

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (10 lines)
> On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Gettext 0.19.8.1 (current core-updates,
>> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo
>> file, without honoring SOURCE_DATE_EPOCH, which leads to
>> non-reproducible builds (for example ‘guix’).
>
> Is it still relevant? Since Gettext is now at 0.20.1. How can I
> reproduce the issue?

I still see this:

Toggle snippet (19 lines)
$ guix challenge guix --substitute-urls="https://ci.guix.gnu.org https://bayfront.guix.gnu.org"
updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0%
/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe contents differ:
no local build for '/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe'
https://ci.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe: 13wvxga668grzs0p6sp0ghvdiy96nc9w71vs11djjkypsaf7wpw1
https://bayfront.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe: 1rpwil9h2whjd9dbwpikxn8prkg924nhljglwj9yjijh578nlfr8
differing files:
/share/locale/en@quot/LC_MESSAGES/guix.mo
/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
/share/locale/en@boldquot/LC_MESSAGES/guix.mo
/share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
/share/info/guix-cookbook.de.info.gz
/lib/guile/3.0/site-ccache/guix/workers.go
/lib/guile/3.0/site-ccache/guix/ui.go
/lib/guile/3.0/site-ccache/guix/swh.go
/lib/guile/3.0/site-ccache/guix/svn-download.go
[…]

‘--diff=diffoscope’ is not an option here because it takes too long
looking at all the .go files…

A focused diff shows this:

Toggle snippet (41 lines)
$ diffoscope --exclude-directory-metadata=yes /tmp/{t1,t2}/share/locale/en@quot/
--- /tmp/t1/share/locale/en@quot/
+++ /tmp/t2/share/locale/en@quot/
? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES
??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES
? ? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
? ??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
? ? ??? msgunfmt {}
? ? ? @@ -1,12 +1,12 @@
? ? ? msgid ""
? ? ? msgstr ""
? ? ? "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
? ? ? "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
? ? ? -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
? ? ? +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
? ? ? "Last-Translator: Automatically generated\n"
? ? ? "Language-Team: none\n"
? ? ? "Language: en@quot\n"
? ? ? "MIME-Version: 1.0\n"
? ? ? "Content-Type: text/plain; charset=UTF-8\n"
? ? ? "Content-Transfer-Encoding: 8bit\n"
? ? ? "Plural-Forms: nplurals=2; plural=(n != 1);\n"
? ? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix.mo
? ??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix.mo
? ? ??? msgunfmt {}
? ? ? @@ -1,12 +1,12 @@
? ? ? msgid ""
? ? ? msgstr ""
? ? ? "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
? ? ? "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
? ? ? -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
? ? ? +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
? ? ? "Last-Translator: Automatically generated\n"
? ? ? "Language-Team: none\n"
? ? ? "Language: en@quot\n"
? ? ? "MIME-Version: 1.0\n"
? ? ? "Content-Type: text/plain; charset=UTF-8\n"
? ? ? "Content-Transfer-Encoding: 8bit\n"
? ? ? "Plural-Forms: nplurals=2; plural=(n != 1);\n"

Ludo’.
J
J
Julien Lepiller wrote on 3 Dec 2020 13:07
(address . 26247@debbugs.gnu.org)
EB5BE5DC-EF6C-4311-9A07-B2A06E71E65A@lepiller.eu
So it's not gettext itself, but our build system that generates the en@quote and en@boldquote files. Do we really need them? If so, we should find a way to generate them reproducibly.

Le 3 décembre 2020 05:32:54 GMT-05:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
Toggle quote (91 lines)
>Hi Simon,
>
>zimoun <zimon.toutoune@gmail.com> skribis:
>
>> On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:
>>
>>> Gettext 0.19.8.1 (current core-updates,
>>> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in
>.mo
>>> file, without honoring SOURCE_DATE_EPOCH, which leads to
>>> non-reproducible builds (for example ‘guix’).
>>
>> Is it still relevant? Since Gettext is now at 0.20.1. How can I
>> reproduce the issue?
>
>I still see this:
>
>--8<---------------cut here---------------start------------->8---
>$ guix challenge guix --substitute-urls="https://ci.guix.gnu.org
>https://bayfront.guix.gnu.org"
>updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0%
>/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe
>contents differ:
>no local build for
>'/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe'
>https://ci.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe:
>13wvxga668grzs0p6sp0ghvdiy96nc9w71vs11djjkypsaf7wpw1
>https://bayfront.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe:
>1rpwil9h2whjd9dbwpikxn8prkg924nhljglwj9yjijh578nlfr8
> differing files:
> /share/locale/en@quot/LC_MESSAGES/guix.mo
> /share/locale/en@quot/LC_MESSAGES/guix-packages.mo
> /share/locale/en@boldquot/LC_MESSAGES/guix.mo
> /share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
> /share/info/guix-cookbook.de.info.gz
> /lib/guile/3.0/site-ccache/guix/workers.go
> /lib/guile/3.0/site-ccache/guix/ui.go
> /lib/guile/3.0/site-ccache/guix/swh.go
> /lib/guile/3.0/site-ccache/guix/svn-download.go
>[…]
>--8<---------------cut here---------------end--------------->8---
>
>‘--diff=diffoscope’ is not an option here because it takes too long
>looking at all the .go files…
>
>A focused diff shows this:
>
>--8<---------------cut here---------------start------------->8---
>$ diffoscope --exclude-directory-metadata=yes
>/tmp/{t1,t2}/share/locale/en@quot/
>--- /tmp/t1/share/locale/en@quot/
>+++ /tmp/t2/share/locale/en@quot/
>? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES
>??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES
>? ? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
>? ??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
>? ? ??? msgunfmt {}
>? ? ? @@ -1,12 +1,12 @@
>? ? ? msgid ""
>? ? ? msgstr ""
>? ? ? "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
>? ? ? "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
>? ? ? -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
>? ? ? +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
>? ? ? "Last-Translator: Automatically generated\n"
>? ? ? "Language-Team: none\n"
>? ? ? "Language: en@quot\n"
>? ? ? "MIME-Version: 1.0\n"
>? ? ? "Content-Type: text/plain; charset=UTF-8\n"
>? ? ? "Content-Transfer-Encoding: 8bit\n"
>? ? ? "Plural-Forms: nplurals=2; plural=(n != 1);\n"
>? ? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix.mo
>? ??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix.mo
>? ? ??? msgunfmt {}
>? ? ? @@ -1,12 +1,12 @@
>? ? ? msgid ""
>? ? ? msgstr ""
>? ? ? "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
>? ? ? "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
>? ? ? -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
>? ? ? +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
>? ? ? "Last-Translator: Automatically generated\n"
>? ? ? "Language-Team: none\n"
>? ? ? "Language: en@quot\n"
>? ? ? "MIME-Version: 1.0\n"
>? ? ? "Content-Type: text/plain; charset=UTF-8\n"
>? ? ? "Content-Transfer-Encoding: 8bit\n"
>? ? ? "Plural-Forms: nplurals=2; plural=(n != 1);\n"
>--8<---------------cut here---------------end--------------->8---
>
>Ludo’.
Attachment: file
L
L
Ludovic Courtès wrote on 3 Dec 2020 15:41
(name . Julien Lepiller)(address . julien@lepiller.eu)
87blfbdka0.fsf@gnu.org
Julien Lepiller <julien@lepiller.eu> skribis:

Toggle quote (2 lines)
> So it's not gettext itself, but our build system that generates the en@quote and en@boldquote files. Do we really need them? If so, we should find a way to generate them reproducibly.

They’re generated automatically by the gettext Makefilery that we use,
so I don’t think there’s much we can do?

We could remove them from ‘LINGUAS’, but we’d be losing something,
wouldn’t we?

Ludo’.
M
M
Miguel Ángel Arruga Vivas wrote on 11 Dec 2020 14:26
(name . Ludovic Courtès)(address . ludo@gnu.org)
875z58fp8e.fsf@gmail.com
Hi!

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (9 lines)
> Julien Lepiller <julien@lepiller.eu> skribis:
>
>> So it's not gettext itself, but our build system that generates the
>> en@quote and en@boldquote files. Do we really need them? If so, we
>> should find a way to generate them reproducibly.
>
> They’re generated automatically by the gettext Makefilery that we use,
> so I don’t think there’s much we can do?

The issue isn't on those files but on POT generation. Currently
xgettext doesn't honor SOURCE_DATE_EPOCH to fill POT-Creation-Date,
which is used to fill PO-Revision-Date for auto-generated po files.
These files are included into tarballs generated by make dist, therefore
its date is already fixed, but they aren't present on our git tree---for
obvious reasons.

This bug has already been reported upstream[1] and probably I'll push it
as soon as I have more test cases prepared and receive a brief review,
but I can prepare a patch for previous versions if it's needed too.

Best regards,
Miguel

L
L
Ludovic Courtès wrote on 14 Dec 2020 10:09
(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)
87sg88wy7p.fsf@gnu.org
Hi Miguel,

Miguel Ángel Arruga Vivas <rosen644835@gmail.com> skribis:

Toggle quote (18 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Julien Lepiller <julien@lepiller.eu> skribis:
>>
>>> So it's not gettext itself, but our build system that generates the
>>> en@quote and en@boldquote files. Do we really need them? If so, we
>>> should find a way to generate them reproducibly.
>>
>> They’re generated automatically by the gettext Makefilery that we use,
>> so I don’t think there’s much we can do?
>
> The issue isn't on those files but on POT generation. Currently
> xgettext doesn't honor SOURCE_DATE_EPOCH to fill POT-Creation-Date,
> which is used to fill PO-Revision-Date for auto-generated po files.
> These files are included into tarballs generated by make dist, therefore
> its date is already fixed, but they aren't present on our git tree---for
> obvious reasons.

OK.

Toggle quote (4 lines)
> This bug has already been reported upstream[1] and probably I'll push it
> as soon as I have more test cases prepared and receive a brief review,
> but I can prepare a patch for previous versions if it's needed too.

[...]

Toggle quote (2 lines)
Thanks for getting to the bottom of this and for preparing a patch
upstream!

In ‘core-updates’, we could either wait for the next Gettext release or
apply your patch in the meantime.

Ludo’.
Z
Z
zimoun wrote on 8 Oct 2022 16:25
(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)
868rlqtmsu.fsf@gmail.com
Hi Miguel,

It is about an old patch [2] and on old message from Dec. 2020.

On Fri, 11 Dec 2020 at 14:26, Miguel Ángel Arruga Vivas <rosen644835@gmail.com> wrote:

Toggle quote (9 lines)
> This bug has already been reported upstream[1] and probably I'll push it
> as soon as I have more test cases prepared and receive a brief review,
> but I can prepare a patch for previous versions if it's needed too.
>
> Best regards,
> Miguel
>
> [1] https://savannah.gnu.org/bugs/?59658

Are commits 4343ca8ba5b02c8fe09e5bd681abbc0440ab8b08 and following the
ones mentioned here?

Well, I am not sure to understand if it had been fixed upstream [1].


Cheers,
simon

S
S
Simon Tournier wrote on 17 Oct 2023 01:48
(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)
86edhup1rb.fsf@gmail.com
Hi,

It is about this old bug#26247:


On Sat, 08 Oct 2022 at 16:25, zimoun <zimon.toutoune@gmail.com> wrote:
Toggle quote (16 lines)
> On Fri, 11 Dec 2020 at 14:26, Miguel Ángel Arruga Vivas <rosen644835@gmail.com> wrote:
>
>> This bug has already been reported upstream[1] and probably I'll push it
>> as soon as I have more test cases prepared and receive a brief review,
>> but I can prepare a patch for previous versions if it's needed too.
>>
>> Best regards,
>> Miguel
>>
>> [1] https://savannah.gnu.org/bugs/?59658
>
> Are commits 4343ca8ba5b02c8fe09e5bd681abbc0440ab8b08 and following the
> ones mentioned here?
>
> Well, I am not sure to understand if it had been fixed upstream [1].

The issue is still not fixed in Guix.

Toggle snippet (20 lines)
$ guix time-machine --commit=b437896e87a51cc610388d4c462893652dd773e6 -- challenge guix --substitute-urls="https://ci.guix.gnu.org https://bordeaux.guix.gnu.org"
/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274 contents differ:
no local build for '/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274'
https://ci.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 1inv5ri4z35xz6cv9laiaf4nv9v9km7ggvbwcdhxpv5hkabsbjia
https://bordeaux.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 13njz5kn402g5larsbmbi25dx4w8azpbffqlkhyagc52hnbpqxaf
differing files:

[...]

/share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
/share/locale/en@boldquot/LC_MESSAGES/guix.mo
/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
/share/locale/en@quot/LC_MESSAGES/guix.mo

1 store items were analyzed:
- 0 (0.0%) were identical
- 1 (100.0%) differed
- 0 (0.0%) were inconclusive

Cheers,
simon
J
J
Julien Lepiller wrote on 17 Oct 2023 12:11
A5FA904C-F79B-40BC-A70C-F57EB5A8C826@lepiller.eu
It's interesting to see that only the generate po files are compiled to non-reproducible mo files. All other translations seem fine, right? Could it be related to the way these po files are created? It might introduce a timestamp at that point, but I can't check right now.

Le 17 octobre 2023 01:48:40 GMT+02:00, Simon Tournier <zimon.toutoune@gmail.com> a écrit :
Toggle quote (48 lines)
>Hi,
>
>It is about this old bug#26247:
>
> https://issues.guix.gnu.org/issue/26247
>
>On Sat, 08 Oct 2022 at 16:25, zimoun <zimon.toutoune@gmail.com> wrote:
>> On Fri, 11 Dec 2020 at 14:26, Miguel Ángel Arruga Vivas <rosen644835@gmail.com> wrote:
>>
>>> This bug has already been reported upstream[1] and probably I'll push it
>>> as soon as I have more test cases prepared and receive a brief review,
>>> but I can prepare a patch for previous versions if it's needed too.
>>>
>>> Best regards,
>>> Miguel
>>>
>>> [1] https://savannah.gnu.org/bugs/?59658
>>
>> Are commits 4343ca8ba5b02c8fe09e5bd681abbc0440ab8b08 and following the
>> ones mentioned here?
>>
>> Well, I am not sure to understand if it had been fixed upstream [1].
>
>The issue is still not fixed in Guix.
>
>--8<---------------cut here---------------start------------->8---
>$ guix time-machine --commit=b437896e87a51cc610388d4c462893652dd773e6 -- challenge guix --substitute-urls="https://ci.guix.gnu.org https://bordeaux.guix.gnu.org"
>/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274 contents differ:
> no local build for '/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274'
> https://ci.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 1inv5ri4z35xz6cv9laiaf4nv9v9km7ggvbwcdhxpv5hkabsbjia
> https://bordeaux.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 13njz5kn402g5larsbmbi25dx4w8azpbffqlkhyagc52hnbpqxaf
> differing files:
>
>[...]
>
> /share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
> /share/locale/en@boldquot/LC_MESSAGES/guix.mo
> /share/locale/en@quot/LC_MESSAGES/guix-packages.mo
> /share/locale/en@quot/LC_MESSAGES/guix.mo
>
>1 store items were analyzed:
> - 0 (0.0%) were identical
> - 1 (100.0%) differed
> - 0 (0.0%) were inconclusive
>--8<---------------cut here---------------end--------------->8---
>
>Cheers,
>simon
?