Gettext introduces timestamps in .mo files

  • Open
  • quality assurance status badge
Details
4 participants
  • Julien Lepiller
  • Ludovic Courtès
  • Miguel Ángel Arruga Vivas
  • zimoun
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 24 Mar 2017 23:54
(address . bug-guix@gnu.org)
8760iyxp3t.fsf@gnu.org
Gettext 0.19.8.1 (current core-updates,
77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo
file, without honoring SOURCE_DATE_EPOCH, which leads to
non-reproducible builds (for example ‘guix’).

Ludo’.
Z
Z
zimoun wrote on 1 Dec 2020 19:46
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 26247@debbugs.gnu.org)
865z5luzxx.fsf@gmail.com
Hi Ludo,

This old bug #26247 about timestamp in .mo files from Gettext is still
open:


On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:

Toggle quote (5 lines)
> Gettext 0.19.8.1 (current core-updates,
> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo
> file, without honoring SOURCE_DATE_EPOCH, which leads to
> non-reproducible builds (for example ‘guix’).

Is it still relevant? Since Gettext is now at 0.20.1. How can I
reproduce the issue? Because the usual:

guix build gettext --no-grafts --check

works fine.


Cheers,
simon
L
L
Ludovic Courtès wrote on 3 Dec 2020 11:32
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 26247@debbugs.gnu.org)
87v9djgox5.fsf@gnu.org
Hi Simon,

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (10 lines)
> On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:
>
>> Gettext 0.19.8.1 (current core-updates,
>> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in .mo
>> file, without honoring SOURCE_DATE_EPOCH, which leads to
>> non-reproducible builds (for example ‘guix’).
>
> Is it still relevant? Since Gettext is now at 0.20.1. How can I
> reproduce the issue?

I still see this:

Toggle snippet (19 lines)
$ guix challenge guix --substitute-urls="https://ci.guix.gnu.org https://bayfront.guix.gnu.org"
updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0%
/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe contents differ:
no local build for '/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe'
https://ci.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe: 13wvxga668grzs0p6sp0ghvdiy96nc9w71vs11djjkypsaf7wpw1
https://bayfront.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe: 1rpwil9h2whjd9dbwpikxn8prkg924nhljglwj9yjijh578nlfr8
differing files:
/share/locale/en@quot/LC_MESSAGES/guix.mo
/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
/share/locale/en@boldquot/LC_MESSAGES/guix.mo
/share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
/share/info/guix-cookbook.de.info.gz
/lib/guile/3.0/site-ccache/guix/workers.go
/lib/guile/3.0/site-ccache/guix/ui.go
/lib/guile/3.0/site-ccache/guix/swh.go
/lib/guile/3.0/site-ccache/guix/svn-download.go
[…]

‘--diff=diffoscope’ is not an option here because it takes too long
looking at all the .go files…

A focused diff shows this:

Toggle snippet (41 lines)
$ diffoscope --exclude-directory-metadata=yes /tmp/{t1,t2}/share/locale/en@quot/
--- /tmp/t1/share/locale/en@quot/
+++ /tmp/t2/share/locale/en@quot/
? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES
??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES
? ? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
? ??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
? ? ??? msgunfmt {}
? ? ? @@ -1,12 +1,12 @@
? ? ? msgid ""
? ? ? msgstr ""
? ? ? "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
? ? ? "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
? ? ? -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
? ? ? +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
? ? ? "Last-Translator: Automatically generated\n"
? ? ? "Language-Team: none\n"
? ? ? "Language: en@quot\n"
? ? ? "MIME-Version: 1.0\n"
? ? ? "Content-Type: text/plain; charset=UTF-8\n"
? ? ? "Content-Transfer-Encoding: 8bit\n"
? ? ? "Plural-Forms: nplurals=2; plural=(n != 1);\n"
? ? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix.mo
? ??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix.mo
? ? ??? msgunfmt {}
? ? ? @@ -1,12 +1,12 @@
? ? ? msgid ""
? ? ? msgstr ""
? ? ? "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
? ? ? "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
? ? ? -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
? ? ? +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
? ? ? "Last-Translator: Automatically generated\n"
? ? ? "Language-Team: none\n"
? ? ? "Language: en@quot\n"
? ? ? "MIME-Version: 1.0\n"
? ? ? "Content-Type: text/plain; charset=UTF-8\n"
? ? ? "Content-Transfer-Encoding: 8bit\n"
? ? ? "Plural-Forms: nplurals=2; plural=(n != 1);\n"

Ludo’.
J
J
Julien Lepiller wrote on 3 Dec 2020 13:07
(address . 26247@debbugs.gnu.org)
EB5BE5DC-EF6C-4311-9A07-B2A06E71E65A@lepiller.eu
So it's not gettext itself, but our build system that generates the en@quote and en@boldquote files. Do we really need them? If so, we should find a way to generate them reproducibly.

Le 3 décembre 2020 05:32:54 GMT-05:00, "Ludovic Courtès" <ludo@gnu.org> a écrit :
Toggle quote (91 lines)
>Hi Simon,
>
>zimoun <zimon.toutoune@gmail.com> skribis:
>
>> On Fri, 24 Mar 2017 at 23:54, ludo@gnu.org (Ludovic Courtès) wrote:
>>
>>> Gettext 0.19.8.1 (current core-updates,
>>> 77ab6983a19ef307558ab2607920158d6bb94ba8) introduces timestamps in
>.mo
>>> file, without honoring SOURCE_DATE_EPOCH, which leads to
>>> non-reproducible builds (for example ‘guix’).
>>
>> Is it still relevant? Since Gettext is now at 0.20.1. How can I
>> reproduce the issue?
>
>I still see this:
>
>--8<---------------cut here---------------start------------->8---
>$ guix challenge guix --substitute-urls="https://ci.guix.gnu.org
>https://bayfront.guix.gnu.org"
>updating substitutes from 'https://bayfront.guix.gnu.org'... 100.0%
>/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe
>contents differ:
>no local build for
>'/gnu/store/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe'
>https://ci.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe:
>13wvxga668grzs0p6sp0ghvdiy96nc9w71vs11djjkypsaf7wpw1
>https://bayfront.guix.gnu.org/nar/lzip/babcmx68gkfxwzr3rmccan88dqjiqzb4-guix-1.2.0-3.35a32fe:
>1rpwil9h2whjd9dbwpikxn8prkg924nhljglwj9yjijh578nlfr8
> differing files:
> /share/locale/en@quot/LC_MESSAGES/guix.mo
> /share/locale/en@quot/LC_MESSAGES/guix-packages.mo
> /share/locale/en@boldquot/LC_MESSAGES/guix.mo
> /share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
> /share/info/guix-cookbook.de.info.gz
> /lib/guile/3.0/site-ccache/guix/workers.go
> /lib/guile/3.0/site-ccache/guix/ui.go
> /lib/guile/3.0/site-ccache/guix/swh.go
> /lib/guile/3.0/site-ccache/guix/svn-download.go
>[…]
>--8<---------------cut here---------------end--------------->8---
>
>‘--diff=diffoscope’ is not an option here because it takes too long
>looking at all the .go files…
>
>A focused diff shows this:
>
>--8<---------------cut here---------------start------------->8---
>$ diffoscope --exclude-directory-metadata=yes
>/tmp/{t1,t2}/share/locale/en@quot/
>--- /tmp/t1/share/locale/en@quot/
>+++ /tmp/t2/share/locale/en@quot/
>? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES
>??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES
>? ? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
>? ??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
>? ? ??? msgunfmt {}
>? ? ? @@ -1,12 +1,12 @@
>? ? ? msgid ""
>? ? ? msgstr ""
>? ? ? "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
>? ? ? "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
>? ? ? -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
>? ? ? +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
>? ? ? "Last-Translator: Automatically generated\n"
>? ? ? "Language-Team: none\n"
>? ? ? "Language: en@quot\n"
>? ? ? "MIME-Version: 1.0\n"
>? ? ? "Content-Type: text/plain; charset=UTF-8\n"
>? ? ? "Content-Transfer-Encoding: 8bit\n"
>? ? ? "Plural-Forms: nplurals=2; plural=(n != 1);\n"
>? ? --- /tmp/t1/share/locale/en@quot/LC_MESSAGES/guix.mo
>? ??? +++ /tmp/t2/share/locale/en@quot/LC_MESSAGES/guix.mo
>? ? ??? msgunfmt {}
>? ? ? @@ -1,12 +1,12 @@
>? ? ? msgid ""
>? ? ? msgstr ""
>? ? ? "Project-Id-Version: guix 1.2.0-3.35a32fe\n"
>? ? ? "Report-Msgid-Bugs-To: bug-guix@gnu.org\n"
>? ? ? -"PO-Revision-Date: 2020-11-29 18:33+0000\n"
>? ? ? +"PO-Revision-Date: 2020-12-02 10:10+0000\n"
>? ? ? "Last-Translator: Automatically generated\n"
>? ? ? "Language-Team: none\n"
>? ? ? "Language: en@quot\n"
>? ? ? "MIME-Version: 1.0\n"
>? ? ? "Content-Type: text/plain; charset=UTF-8\n"
>? ? ? "Content-Transfer-Encoding: 8bit\n"
>? ? ? "Plural-Forms: nplurals=2; plural=(n != 1);\n"
>--8<---------------cut here---------------end--------------->8---
>
>Ludo’.
Attachment: file
L
L
Ludovic Courtès wrote on 3 Dec 2020 15:41
(name . Julien Lepiller)(address . julien@lepiller.eu)
87blfbdka0.fsf@gnu.org
Julien Lepiller <julien@lepiller.eu> skribis:

Toggle quote (2 lines)
> So it's not gettext itself, but our build system that generates the en@quote and en@boldquote files. Do we really need them? If so, we should find a way to generate them reproducibly.

They’re generated automatically by the gettext Makefilery that we use,
so I don’t think there’s much we can do?

We could remove them from ‘LINGUAS’, but we’d be losing something,
wouldn’t we?

Ludo’.
M
M
Miguel Ángel Arruga Vivas wrote on 11 Dec 2020 14:26
(name . Ludovic Courtès)(address . ludo@gnu.org)
875z58fp8e.fsf@gmail.com
Hi!

Ludovic Courtès <ludo@gnu.org> writes:

Toggle quote (9 lines)
> Julien Lepiller <julien@lepiller.eu> skribis:
>
>> So it's not gettext itself, but our build system that generates the
>> en@quote and en@boldquote files. Do we really need them? If so, we
>> should find a way to generate them reproducibly.
>
> They’re generated automatically by the gettext Makefilery that we use,
> so I don’t think there’s much we can do?

The issue isn't on those files but on POT generation. Currently
xgettext doesn't honor SOURCE_DATE_EPOCH to fill POT-Creation-Date,
which is used to fill PO-Revision-Date for auto-generated po files.
These files are included into tarballs generated by make dist, therefore
its date is already fixed, but they aren't present on our git tree---for
obvious reasons.

This bug has already been reported upstream[1] and probably I'll push it
as soon as I have more test cases prepared and receive a brief review,
but I can prepare a patch for previous versions if it's needed too.

Best regards,
Miguel

L
L
Ludovic Courtès wrote on 14 Dec 2020 10:09
(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)
87sg88wy7p.fsf@gnu.org
Hi Miguel,

Miguel Ángel Arruga Vivas <rosen644835@gmail.com> skribis:

Toggle quote (18 lines)
> Ludovic Courtès <ludo@gnu.org> writes:
>
>> Julien Lepiller <julien@lepiller.eu> skribis:
>>
>>> So it's not gettext itself, but our build system that generates the
>>> en@quote and en@boldquote files. Do we really need them? If so, we
>>> should find a way to generate them reproducibly.
>>
>> They’re generated automatically by the gettext Makefilery that we use,
>> so I don’t think there’s much we can do?
>
> The issue isn't on those files but on POT generation. Currently
> xgettext doesn't honor SOURCE_DATE_EPOCH to fill POT-Creation-Date,
> which is used to fill PO-Revision-Date for auto-generated po files.
> These files are included into tarballs generated by make dist, therefore
> its date is already fixed, but they aren't present on our git tree---for
> obvious reasons.

OK.

Toggle quote (4 lines)
> This bug has already been reported upstream[1] and probably I'll push it
> as soon as I have more test cases prepared and receive a brief review,
> but I can prepare a patch for previous versions if it's needed too.

[...]

Toggle quote (2 lines)
Thanks for getting to the bottom of this and for preparing a patch
upstream!

In ‘core-updates’, we could either wait for the next Gettext release or
apply your patch in the meantime.

Ludo’.
Z
Z
zimoun wrote on 8 Oct 2022 16:25
(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)
868rlqtmsu.fsf@gmail.com
Hi Miguel,

It is about an old patch [2] and on old message from Dec. 2020.

On Fri, 11 Dec 2020 at 14:26, Miguel Ángel Arruga Vivas <rosen644835@gmail.com> wrote:

Toggle quote (9 lines)
> This bug has already been reported upstream[1] and probably I'll push it
> as soon as I have more test cases prepared and receive a brief review,
> but I can prepare a patch for previous versions if it's needed too.
>
> Best regards,
> Miguel
>
> [1] https://savannah.gnu.org/bugs/?59658

Are commits 4343ca8ba5b02c8fe09e5bd681abbc0440ab8b08 and following the
ones mentioned here?

Well, I am not sure to understand if it had been fixed upstream [1].


Cheers,
simon

S
S
Simon Tournier wrote on 17 Oct 2023 01:48
(name . Miguel Ángel Arruga Vivas)(address . rosen644835@gmail.com)
86edhup1rb.fsf@gmail.com
Hi,

It is about this old bug#26247:


On Sat, 08 Oct 2022 at 16:25, zimoun <zimon.toutoune@gmail.com> wrote:
Toggle quote (16 lines)
> On Fri, 11 Dec 2020 at 14:26, Miguel Ángel Arruga Vivas <rosen644835@gmail.com> wrote:
>
>> This bug has already been reported upstream[1] and probably I'll push it
>> as soon as I have more test cases prepared and receive a brief review,
>> but I can prepare a patch for previous versions if it's needed too.
>>
>> Best regards,
>> Miguel
>>
>> [1] https://savannah.gnu.org/bugs/?59658
>
> Are commits 4343ca8ba5b02c8fe09e5bd681abbc0440ab8b08 and following the
> ones mentioned here?
>
> Well, I am not sure to understand if it had been fixed upstream [1].

The issue is still not fixed in Guix.

Toggle snippet (20 lines)
$ guix time-machine --commit=b437896e87a51cc610388d4c462893652dd773e6 -- challenge guix --substitute-urls="https://ci.guix.gnu.org https://bordeaux.guix.gnu.org"
/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274 contents differ:
no local build for '/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274'
https://ci.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 1inv5ri4z35xz6cv9laiaf4nv9v9km7ggvbwcdhxpv5hkabsbjia
https://bordeaux.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 13njz5kn402g5larsbmbi25dx4w8azpbffqlkhyagc52hnbpqxaf
differing files:

[...]

/share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
/share/locale/en@boldquot/LC_MESSAGES/guix.mo
/share/locale/en@quot/LC_MESSAGES/guix-packages.mo
/share/locale/en@quot/LC_MESSAGES/guix.mo

1 store items were analyzed:
- 0 (0.0%) were identical
- 1 (100.0%) differed
- 0 (0.0%) were inconclusive

Cheers,
simon
J
J
Julien Lepiller wrote on 17 Oct 2023 12:11
A5FA904C-F79B-40BC-A70C-F57EB5A8C826@lepiller.eu
It's interesting to see that only the generate po files are compiled to non-reproducible mo files. All other translations seem fine, right? Could it be related to the way these po files are created? It might introduce a timestamp at that point, but I can't check right now.

Le 17 octobre 2023 01:48:40 GMT+02:00, Simon Tournier <zimon.toutoune@gmail.com> a écrit :
Toggle quote (48 lines)
>Hi,
>
>It is about this old bug#26247:
>
> https://issues.guix.gnu.org/issue/26247
>
>On Sat, 08 Oct 2022 at 16:25, zimoun <zimon.toutoune@gmail.com> wrote:
>> On Fri, 11 Dec 2020 at 14:26, Miguel Ángel Arruga Vivas <rosen644835@gmail.com> wrote:
>>
>>> This bug has already been reported upstream[1] and probably I'll push it
>>> as soon as I have more test cases prepared and receive a brief review,
>>> but I can prepare a patch for previous versions if it's needed too.
>>>
>>> Best regards,
>>> Miguel
>>>
>>> [1] https://savannah.gnu.org/bugs/?59658
>>
>> Are commits 4343ca8ba5b02c8fe09e5bd681abbc0440ab8b08 and following the
>> ones mentioned here?
>>
>> Well, I am not sure to understand if it had been fixed upstream [1].
>
>The issue is still not fixed in Guix.
>
>--8<---------------cut here---------------start------------->8---
>$ guix time-machine --commit=b437896e87a51cc610388d4c462893652dd773e6 -- challenge guix --substitute-urls="https://ci.guix.gnu.org https://bordeaux.guix.gnu.org"
>/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274 contents differ:
> no local build for '/gnu/store/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274'
> https://ci.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 1inv5ri4z35xz6cv9laiaf4nv9v9km7ggvbwcdhxpv5hkabsbjia
> https://bordeaux.guix.gnu.org/nar/lzip/0znvqzij8lvf4lv7xxs126wxzgmx0zsw-guix-1.4.0-13.e863274: 13njz5kn402g5larsbmbi25dx4w8azpbffqlkhyagc52hnbpqxaf
> differing files:
>
>[...]
>
> /share/locale/en@boldquot/LC_MESSAGES/guix-packages.mo
> /share/locale/en@boldquot/LC_MESSAGES/guix.mo
> /share/locale/en@quot/LC_MESSAGES/guix-packages.mo
> /share/locale/en@quot/LC_MESSAGES/guix.mo
>
>1 store items were analyzed:
> - 0 (0.0%) were identical
> - 1 (100.0%) differed
> - 0 (0.0%) were inconclusive
>--8<---------------cut here---------------end--------------->8---
>
>Cheers,
>simon
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 26247@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 26247
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch