Tests (tests/channels.scm) assume UTF-8 locale

  • Open
  • quality assurance status badge
Details
3 participants
  • Ludovic Courtès
  • Tobias Geerinckx-Rice
  • Vagrant Cascadian
Owner
unassigned
Submitted by
Vagrant Cascadian
Severity
normal
Merged with
V
V
Vagrant Cascadian wrote on 22 Jan 2021 17:56
guix 1.2.0 test failure: channels-news, one entry
(address . bug-guix@gnu.org)
87lfck6hbl.fsf@yucca
I've uploaded guix 1.2.0 built against guile-2.2 to Debian, and while it
builds fine on the official buildd.debian.org infrastructure, on amd64
and arm64 the "channel-news, one entry" test from tests/channels.scm
fails on tests.reproducible-builds.org.

There are likely a few differences in the two build environments,
possibly including network access.

Does the "channel-news, one entry" test indirectly depend on network or
bootstrap binaries?

Could a difference in locale related variables affect the result of the
test (e.g. LANGUAGE=en:en_US vs. LANGUAGE unset, LC_ALL unset
vs. LC_ALL=C or LC_ALL=C.UTF-8)?


test-name: channel-news, one entry
location: /build/1st/guix-1.2.0/tests/channels.scm:324
source:
+ (test-assert
+ "channel-news, one entry"
+ (with-temporary-git-repository
+ directory
+ `((add ".guix-channel"
+ ,(object->string
+ '(channel (version 0) (news-file "news.scm"))))
+ (commit "first commit")
+ (add "src/a.txt" "A")
+ (commit "second commit")
+ (tag "tag-for-first-news-entry")
+ (add "news.scm"
+ ,(lambda (repository)
+ (let ((previous
+ (reference-name->oid repository "HEAD")))
+ (object->string
+ `(channel-news
+ (version 0)
+ (entry (commit ,(oid->string previous))
+ (title (en "New file!") (eo "Nova dosiero!"))
+ (body (en "Yeah, a.txt."))))))))
+ (commit "third commit")
+ (add "src/b.txt" "B")
+ (commit "fourth commit")
+ (add hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /tmp/guix-directory.6SfxEu/.git/
[master (root-commit) 8b5d0e8] first commit
1 file changed, 1 insertion(+)
create mode 100644 .guix-channel
[master b8dd467] second commit
1 file changed, 1 insertion(+)
create mode 100644 src/a.txt
[master 324d7bc] third commit
1 file changed, 1 insertion(+)
create mode 100644 news.scm
[master 2cd62e1] fourth commit
1 file changed, 1 insertion(+)
create mode 100644 src/b.txt
[master d0e63c3] fifth commit
1 file changed, 1 insertion(+), 1 deletion(-)
hint: Using 'master' as the name for the initial branch. This default branch name
hint: is subject to change. To configure the initial branch name to use in all
hint: of your new repositories, which will suppress this warning, call:
hint:
hint: git config --global init.defaultBranch <name>
hint:
hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
hint: 'development'. The just-created branch can be renamed via this command:
hint:
hint: git branch -m <name>
Initialized empty Git repository in /tmp/guix-directory.M2UpCv/.git/
[master (root-commit) f84a5c3] first commit
1 file changed, 1 insertion(+)
create mode 100644 a.txt
[master b1e63da] second commit
1 file changed, 1 insertion(+)
create mode 100644 b.scm
"news.scm"
+ ,(lambda (repository)
+ (let ((second
+ (commit-id
+ (find-commit repository "second commit")))
+ (previous
+ (reference-name->oid repository "HEAD")))
+ (object->string
+ `(channel-news
+ (version 0)
+ (entry (commit ,(oid->string previous))
+ (title (en "Another file!"))
+ (body (en "Yeah, b.txt.")))
+ (entry (tag "tag-for-first-news-entry")
+ (title (en "Old news.") (eo "Malnova?oj."))
+ (body (en "For a.txt"))))))))
+ (commit "fifth commit"))
+ (with-repository
+ directory
+ repository
+ (define (find-commit* message)
+ (oid->string
+ (commit-id (find-commit repository message))))
+ (let ((channel
+ (channel
+ (url (string-append "file://" directory))
+ (name 'foo)))
+ (commit1 (find-commit* "first commit"))
+ (commit2 (find-commit* "second commit"))
+ (commit3 (find-commit* "third commit"))
+ (commit4 (find-commit* "fourth commit"))
+ (commit5 (find-commit* "fifth commit")))
+ (and (null? (channel-news-for-commit channel commit2))
+ (lset= string=?
+ (map channel-news-entry-commit
+ (channel-news-for-commit channel commit5))
+ (list commit2 commit4))
+ (lset= equal?
+ (map channel-news-entry-title
+ (channel-news-for-commit channel commit5))
+ '((("en" . "Another file!"))
+ (("en" . "Old news.") ("eo" . "Malnova?oj."))))
+ (lset= string=?
+ (map channel-news-entry-commit
+ (channel-news-for-commit channel commit3))
+ (list commit2))
+ (lset= string=?
+ (map channel-news-entry-commit
+ (channel-news-for-commit channel commit3 commit1))
+ (list commit2))
+ (lset= string=?
+ (map channel-news-entry-commit
+ (channel-news-for-commit channel commit5 commit3))
+ (list commit4))
+ (lset= string=?
+ (map channel-news-entry-commit
+ (channel-news-for-commit channel commit5 commit1))
+ (list commit4 commit2))
+ (lset= equal?
+ (map channel-news-entry-tag
+ (channel-news-for-commit channel commit5 commit1))
+ '(#f "tag-for-first-news-entry")))))))
actual-value: #f
result: FAIL


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYAsDzwAKCRDcUY/If5cW
qgGYAP4ndX3GCLNV/hSA2z9pMtDBUAffBv+zSSl7AqtE4sEBAQEA/s8ZTZBt9IZw
EbMykzn6hSLgTM3Se4XRPLLwlrdU3QM=
=7up1
-----END PGP SIGNATURE-----

V
V
Vagrant Cascadian wrote on 2 May 2021 04:23
guix 1.3.0rc1 test failure: channels-news, one entry
(address . 46038@debbugs.gnu.org)
87bl9tvppm.fsf@yucca
On 2021-01-22, Vagrant Cascadian wrote:
Toggle quote (17 lines)
> I've uploaded guix 1.2.0 built against guile-2.2 to Debian, and while it
> builds fine on the official buildd.debian.org infrastructure, on amd64
> and arm64 the "channel-news, one entry" test from tests/channels.scm
> fails on tests.reproducible-builds.org.
>
> There are likely a few differences in the two build environments,
> possibly including network access.
>
> Does the "channel-news, one entry" test indirectly depend on network or
> bootstrap binaries?
>
> Could a difference in locale related variables affect the result of the
> test (e.g. LANGUAGE=en:en_US vs. LANGUAGE unset, LC_ALL unset
> vs. LC_ALL=C or LC_ALL=C.UTF-8)?
>
> https://tests.reproducible-builds.org/debian/rb-pkg/experimental/amd64/guix.html

Still basically the same story with 1.3.0rc1, in some cases this test
fails, but I haven't consistently figured out what triggers it.


Toggle quote (135 lines)
> test-name: channel-news, one entry
> location: /build/1st/guix-1.2.0/tests/channels.scm:324
> source:
> + (test-assert
> + "channel-news, one entry"
> + (with-temporary-git-repository
> + directory
> + `((add ".guix-channel"
> + ,(object->string
> + '(channel (version 0) (news-file "news.scm"))))
> + (commit "first commit")
> + (add "src/a.txt" "A")
> + (commit "second commit")
> + (tag "tag-for-first-news-entry")
> + (add "news.scm"
> + ,(lambda (repository)
> + (let ((previous
> + (reference-name->oid repository "HEAD")))
> + (object->string
> + `(channel-news
> + (version 0)
> + (entry (commit ,(oid->string previous))
> + (title (en "New file!") (eo "Nova dosiero!"))
> + (body (en "Yeah, a.txt."))))))))
> + (commit "third commit")
> + (add "src/b.txt" "B")
> + (commit "fourth commit")
> + (add hint: Using 'master' as the name for the initial branch. This default branch name
> hint: is subject to change. To configure the initial branch name to use in all
> hint: of your new repositories, which will suppress this warning, call:
> hint:
> hint: git config --global init.defaultBranch <name>
> hint:
> hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
> hint: 'development'. The just-created branch can be renamed via this command:
> hint:
> hint: git branch -m <name>
> Initialized empty Git repository in /tmp/guix-directory.6SfxEu/.git/
> [master (root-commit) 8b5d0e8] first commit
> 1 file changed, 1 insertion(+)
> create mode 100644 .guix-channel
> [master b8dd467] second commit
> 1 file changed, 1 insertion(+)
> create mode 100644 src/a.txt
> [master 324d7bc] third commit
> 1 file changed, 1 insertion(+)
> create mode 100644 news.scm
> [master 2cd62e1] fourth commit
> 1 file changed, 1 insertion(+)
> create mode 100644 src/b.txt
> [master d0e63c3] fifth commit
> 1 file changed, 1 insertion(+), 1 deletion(-)
> hint: Using 'master' as the name for the initial branch. This default branch name
> hint: is subject to change. To configure the initial branch name to use in all
> hint: of your new repositories, which will suppress this warning, call:
> hint:
> hint: git config --global init.defaultBranch <name>
> hint:
> hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and
> hint: 'development'. The just-created branch can be renamed via this command:
> hint:
> hint: git branch -m <name>
> Initialized empty Git repository in /tmp/guix-directory.M2UpCv/.git/
> [master (root-commit) f84a5c3] first commit
> 1 file changed, 1 insertion(+)
> create mode 100644 a.txt
> [master b1e63da] second commit
> 1 file changed, 1 insertion(+)
> create mode 100644 b.scm
> "news.scm"
> + ,(lambda (repository)
> + (let ((second
> + (commit-id
> + (find-commit repository "second commit")))
> + (previous
> + (reference-name->oid repository "HEAD")))
> + (object->string
> + `(channel-news
> + (version 0)
> + (entry (commit ,(oid->string previous))
> + (title (en "Another file!"))
> + (body (en "Yeah, b.txt.")))
> + (entry (tag "tag-for-first-news-entry")
> + (title (en "Old news.") (eo "Malnova?oj."))
> + (body (en "For a.txt"))))))))
> + (commit "fifth commit"))
> + (with-repository
> + directory
> + repository
> + (define (find-commit* message)
> + (oid->string
> + (commit-id (find-commit repository message))))
> + (let ((channel
> + (channel
> + (url (string-append "file://" directory))
> + (name 'foo)))
> + (commit1 (find-commit* "first commit"))
> + (commit2 (find-commit* "second commit"))
> + (commit3 (find-commit* "third commit"))
> + (commit4 (find-commit* "fourth commit"))
> + (commit5 (find-commit* "fifth commit")))
> + (and (null? (channel-news-for-commit channel commit2))
> + (lset= string=?
> + (map channel-news-entry-commit
> + (channel-news-for-commit channel commit5))
> + (list commit2 commit4))
> + (lset= equal?
> + (map channel-news-entry-title
> + (channel-news-for-commit channel commit5))
> + '((("en" . "Another file!"))
> + (("en" . "Old news.") ("eo" . "Malnova?oj."))))
> + (lset= string=?
> + (map channel-news-entry-commit
> + (channel-news-for-commit channel commit3))
> + (list commit2))
> + (lset= string=?
> + (map channel-news-entry-commit
> + (channel-news-for-commit channel commit3 commit1))
> + (list commit2))
> + (lset= string=?
> + (map channel-news-entry-commit
> + (channel-news-for-commit channel commit5 commit3))
> + (list commit4))
> + (lset= string=?
> + (map channel-news-entry-commit
> + (channel-news-for-commit channel commit5 commit1))
> + (list commit4 commit2))
> + (lset= equal?
> + (map channel-news-entry-tag
> + (channel-news-for-commit channel commit5 commit1))
> + '(#f "tag-for-first-news-entry")))))))
> actual-value: #f
> result: FAIL


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYI4NFgAKCRDcUY/If5cW
qrldAQD+BqPUJDBGtukLOZq+IQLxXKIIYgZQKhmOZCK6OupYDgD/XcWmIv7TguzL
YBqXj4yTUtx3PEuEFE0NaBXgL5LALww=
=R028
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 4 May 2021 21:53
(name . Vagrant Cascadian)(address . vagrant@reproducible-builds.org)(address . 46038@debbugs.gnu.org)
875yzymg1h.fsf@gnu.org
Hi Vagrant,

Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:

Toggle quote (26 lines)
> On 2021-01-22, Vagrant Cascadian wrote:
>> I've uploaded guix 1.2.0 built against guile-2.2 to Debian, and while it
>> builds fine on the official buildd.debian.org infrastructure, on amd64
>> and arm64 the "channel-news, one entry" test from tests/channels.scm
>> fails on tests.reproducible-builds.org.
>>
>> There are likely a few differences in the two build environments,
>> possibly including network access.
>>
>> Does the "channel-news, one entry" test indirectly depend on network or
>> bootstrap binaries?
>>
>> Could a difference in locale related variables affect the result of the
>> test (e.g. LANGUAGE=en:en_US vs. LANGUAGE unset, LC_ALL unset
>> vs. LC_ALL=C or LC_ALL=C.UTF-8)?
>>
>> https://tests.reproducible-builds.org/debian/rb-pkg/experimental/amd64/guix.html
>
> Still basically the same story with 1.3.0rc1, in some cases this test
> fails, but I haven't consistently figured out what triggers it.
>
>
>> test-name: channel-news, one entry
>> location: /build/1st/guix-1.2.0/tests/channels.scm:324
>> source:

[...]

Toggle quote (6 lines)
>> + (lset= equal?
>> + (map channel-news-entry-title
>> + (channel-news-for-commit channel commit5))
>> + '((("en" . "Another file!"))
>> + (("en" . "Old news.") ("eo" . "Malnova?oj."))))

The culprit is right here: it should read “Malnova?oj”, but there’s a
question mark instead of ‘?’.

Could it be that you’re not running tests in a UTF-8 locale?

Thanks,
Ludo’.
V
V
Vagrant Cascadian wrote on 5 May 2021 00:34
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 46038@debbugs.gnu.org)
871ramt9ff.fsf@yucca
On 2021-05-04, Ludovic Courtès wrote:
Toggle quote (41 lines)
> Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:
>
>> On 2021-01-22, Vagrant Cascadian wrote:
>>> I've uploaded guix 1.2.0 built against guile-2.2 to Debian, and while it
>>> builds fine on the official buildd.debian.org infrastructure, on amd64
>>> and arm64 the "channel-news, one entry" test from tests/channels.scm
>>> fails on tests.reproducible-builds.org.
>>>
>>> There are likely a few differences in the two build environments,
>>> possibly including network access.
>>>
>>> Does the "channel-news, one entry" test indirectly depend on network or
>>> bootstrap binaries?
>>>
>>> Could a difference in locale related variables affect the result of the
>>> test (e.g. LANGUAGE=en:en_US vs. LANGUAGE unset, LC_ALL unset
>>> vs. LC_ALL=C or LC_ALL=C.UTF-8)?
>>>
>>> https://tests.reproducible-builds.org/debian/rb-pkg/experimental/amd64/guix.html
>>
>> Still basically the same story with 1.3.0rc1, in some cases this test
>> fails, but I haven't consistently figured out what triggers it.
>>
>>
>>> test-name: channel-news, one entry
>>> location: /build/1st/guix-1.2.0/tests/channels.scm:324
>>> source:
>
> [...]
>
>>> + (lset= equal?
>>> + (map channel-news-entry-title
>>> + (channel-news-for-commit channel commit5))
>>> + '((("en" . "Another file!"))
>>> + (("en" . "Old news.") ("eo" . "Malnova?oj."))))
>
> The culprit is right here: it should read “Malnova?oj”, but there’s a
> question mark instead of ‘?’.
>
> Could it be that you’re not running tests in a UTF-8 locale?

Thanks for taking a deeper look!

Yes, on tests.reproducible-builds.org, one build is run in the C locale,
the other in various UTF-8 locales (somewhat arbitrarily tied to
architecture exactly which UTF-8 locale is used). I'm guessing
buildd.debian.org use C.UTF-8, since it builds fine there.

So now the question is what to do; should tests be able to assume a
UTF-8 locale?

Should I try to adapt the test to work in C?

Should I workaround it in the Debian packaging by forcing to use a UTF-8
locale (on Debian, the only one definitely available is C.UTF-8, which
isn't in upstream glibc, and thus not in guix itself).


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCYJHMBAAKCRDcUY/If5cW
qqQDAP9kMwfAOx1eDo/+OGZydmkGRjDxwHkSAM9htT3Fhb7f/wEA9sWmJj8hMmuE
BjjFH33rWIzeofrSHZ0I+moziOEbAQo=
=L1QL
-----END PGP SIGNATURE-----

L
L
Ludovic Courtès wrote on 6 May 2021 12:46
(name . Vagrant Cascadian)(address . vagrant@reproducible-builds.org)(address . 46038@debbugs.gnu.org)
87eeekgmw7.fsf@gnu.org
Hi,

Vagrant Cascadian <vagrant@reproducible-builds.org> skribis:

Toggle quote (9 lines)
> So now the question is what to do; should tests be able to assume a
> UTF-8 locale?
>
> Should I try to adapt the test to work in C?
>
> Should I workaround it in the Debian packaging by forcing to use a UTF-8
> locale (on Debian, the only one definitely available is C.UTF-8, which
> isn't in upstream glibc, and thus not in guix itself).

Since it’s the only test that makes this assumption, you can choose
whichever option sounds best to you. In Guix proper, I’d rather assume
we’re running in a UTF-8 locale. If that’s not a good option for you,
you could patch this one non-ASCII character.

HTH!

Ludo’.
T
T
Tobias Geerinckx-Rice wrote on 27 Nov 2021 14:42
(no subject)
(name . GNU bug tracker automated control server)(address . control@debbugs.gnu.org)
87pmqlemse.fsf@nckx
merge 50512 46038 50123
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYaI2IQ0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15mz0A/A0rKwsXmwBzfqRDON1DbVf1OLF1CoUtIh0WV2Fn
11EAAQD+//y6XzP04eKOjzq2tLFPQUcId1IHwLPmHQyvqg3DAQ==
=qh5g
-----END PGP SIGNATURE-----

T
T
Tobias Geerinckx-Rice wrote on 27 Nov 2021 14:44
(name . GNU bug tracker automated control server)(address . control@debbugs.gnu.org)
87lf19emo8.fsf@nckx
retitle 46038 Tests assume UTF-8 locale (specifically
tests/channels.scm)
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYaI2tw0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW15EEoA/3qxXm1mPe8GfWVzKTtBnbejRhX9/fEhF7SFwzq/
Z9DkAQDjtyonF5c9Nr2UegR8UVZxiIsDnsqw2XRGtuhCoFA6Dg==
=H+zU
-----END PGP SIGNATURE-----

T
T
Tobias Geerinckx-Rice wrote on 27 Nov 2021 14:57
(name . GNU bug tracker automated control server)(address . control@debbugs.gnu.org)
87h7bxem53.fsf@nckx
retitle 46038 Tests (tests/channels.scm) assume UTF-8 locale
?