OpenSSL 1.1.1n test failures due to expired certificates (time bomb)

  • Open
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Maxime Devos
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
important
Merged with
L
L
Ludovic Courtès wrote on 22 Jun 2022 11:58
OpenSSL 3.0.3/1.1.1n includes a time-dependent test
(address . bug-guix@gnu.org)
87r13h3tqr.fsf@gnu.org
Hello,

As reported by phodina in https://issues.guix.gnu.org/53581, OpenSSL
1.1.1n and 3.0.3 include a time-dependent test that now fails due to an
expired certificate:


The log looks like this:

Toggle snippet (91 lines)
80-test_ocsp.t ..................... ok
80-test_pkcs12.t ................... ok

# ERROR: (int) 'result->result == test_ctx->expected_result' failed @ test/ssl_test.c:36
# [2] compared to [0]
# INFO: @ test/ssl_test.c:37
# ExpectedResult mismatch: expected Success, got ClientFail.
# 40B78AF7FF7F0000:error:0A000415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired:ssl/record/rec_layer_s3.c:1584:SSL alert number 45
# OPENSSL_TEST_RAND_ORDER=1655844368
not ok 2 - iteration 2
# ------------------------------------------------------------------------------
# ERROR: (int) 'result->result == test_ctx->expected_result' failed @ test/ssl_test.c:36
# [2] compared to [0]
# INFO: @ test/ssl_test.c:37
# ExpectedResult mismatch: expected Success, got ClientFail.
# 40B78AF7FF7F0000:error:0A000415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired:ssl/record/rec_layer_s3.c:1584:SSL alert number 45
# OPENSSL_TEST_RAND_ORDER=1655844368
not ok 4 - iteration 4
# ------------------------------------------------------------------------------
# ERROR: (int) 'result->result == test_ctx->expected_result' failed @ test/ssl_test.c:36
# [4] compared to [0]
# INFO: @ test/ssl_test.c:37
# ExpectedResult mismatch: expected Success, got FirstHandshakeFailed.
# 40B78AF7FF7F0000:error:0A000415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired:ssl/record/rec_layer_s3.c:1584:SSL alert number 45
# OPENSSL_TEST_RAND_ORDER=1655844368
not ok 5 - iteration 5
# ------------------------------------------------------------------------------
# ERROR: (int) 'result->result == test_ctx->expected_result' failed @ test/ssl_test.c:36
# [4] compared to [0]
# INFO: @ test/ssl_test.c:37
# ExpectedResult mismatch: expected Success, got FirstHandshakeFailed.
# 40B78AF7FF7F0000:error:0A000415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired:ssl/record/rec_layer_s3.c:1584:SSL alert number 45
# OPENSSL_TEST_RAND_ORDER=1655844368
not ok 6 - iteration 6
# ------------------------------------------------------------------------------
# OPENSSL_TEST_RAND_ORDER=1655844368
not ok 1 - test_handshake
# ------------------------------------------------------------------------------
../../util/wrap.pl ../../test/ssl_test 12-ct.cnf.none none => 1
not ok 3 - running ssl_test 12-ct.cnf
# ------------------------------------------------------------------------------
# ERROR: (int) 'result->result == test_ctx->expected_result' failed @ test/ssl_test.c:36
# [2] compared to [0]
# INFO: @ test/ssl_test.c:37
# ExpectedResult mismatch: expected Success, got ClientFail.
# 40B78AF7FF7F0000:error:0A000415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired:ssl/record/rec_layer_s3.c:1584:SSL alert number 45
# OPENSSL_TEST_RAND_ORDER=1655844369
not ok 2 - iteration 2
# ------------------------------------------------------------------------------
# ERROR: (int) 'result->result == test_ctx->expected_result' failed @ test/ssl_test.c:36
# [2] compared to [0]
# INFO: @ test/ssl_test.c:37
# ExpectedResult mismatch: expected Success, got ClientFail.
# 40B78AF7FF7F0000:error:0A000415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired:ssl/record/rec_layer_s3.c:1584:SSL alert number 45
# OPENSSL_TEST_RAND_ORDER=1655844369
not ok 4 - iteration 4
# ------------------------------------------------------------------------------
# ERROR: (int) 'result->result == test_ctx->expected_result' failed @ test/ssl_test.c:36
# [4] compared to [0]
# INFO: @ test/ssl_test.c:37
# ExpectedResult mismatch: expected Success, got FirstHandshakeFailed.
# 40B78AF7FF7F0000:error:0A000415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired:ssl/record/rec_layer_s3.c:1584:SSL alert number 45
# OPENSSL_TEST_RAND_ORDER=1655844369
not ok 5 - iteration 5
# ------------------------------------------------------------------------------
# ERROR: (int) 'result->result == test_ctx->expected_result' failed @ test/ssl_test.c:36
# [4] compared to [0]
# INFO: @ test/ssl_test.c:37
# ExpectedResult mismatch: expected Success, got FirstHandshakeFailed.
# 40B78AF7FF7F0000:error:0A000415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired:ssl/record/rec_layer_s3.c:1584:SSL alert number 45
# OPENSSL_TEST_RAND_ORDER=1655844369
not ok 6 - iteration 6
# ------------------------------------------------------------------------------
# OPENSSL_TEST_RAND_ORDER=1655844369
not ok 1 - test_handshake
# ------------------------------------------------------------------------------
../../util/wrap.pl ../../test/ssl_test 12-ct.cnf.default default => 1
not ok 6 - running ssl_test 12-ct.cnf
# ------------------------------------------------------------------------------
# Failed test 'running ssl_test 12-ct.cnf'
# at test/recipes/80-test_ssl_new.t line 171.
# Looks like you failed 2 tests of 6.
not ok 12 - Test configuration 12-ct.cnf
# ------------------------------------------------------------------------------
# Looks like you failed 1 test of 30.80-test_ssl_new.t ..................
Dubious, test returned 1 (wstat 256, 0x100)
Failed 1/30 subtests
80-test_ssl_old.t .................. ok
80-test_ssl_test_ctx.t ............. ok

That means that ‘openssl’ on current master (ca.
73761d8049f483e6685c2c736872d0366e03238a) now fails to build.

Ludo’.
L
L
Ludovic Courtès wrote on 22 Jun 2022 12:35
control message for bug #56137
(address . control@debbugs.gnu.org)
87pmj13s0g.fsf@gnu.org
severity 56137 important
quit
L
L
Ludovic Courtès wrote on 22 Jun 2022 12:39
Re: bug#56137: OpenSSL 3.0.3/1.1.1n includes a time-dependent test
(address . 56137-done@debbugs.gnu.org)(name . phodina)(address . phodina@protonmail.com)
87ilot3ru7.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (6 lines)
> As reported by phodina in https://issues.guix.gnu.org/53581, OpenSSL
> 1.1.1n and 3.0.3 include a time-dependent test that now fails due to an
> expired certificate:
>
> https://github.com/openssl/openssl/issues/18441

Fixed on ‘core-updates’ with 6cd438c4c2beb016a821143cdfdd12892aa9fd5f.

That commit skips the test. I tried another approach with ‘datefudge’,
which has the advantage of being more explicit and future-proof (should
there be similar issues lying around):

(invoke "datefudge" "2022-01-01"
"make" test-target
#$@(if (or (target-arm?) (target-riscv64?))
#~("TESTS=-test_afalg")
#~()))

For some reason it didn’t work.

Note that we cannot use libfaketime because:

Toggle snippet (7 lines)
$ guix graph -t derivation --path libfaketime openssl@1
/gnu/store/a4jcd4h7nvn97a2mw4n1yydgbh0i2wmz-libfaketime-0.9.9.drv
/gnu/store/hf5arq562aiisycnjcnhgfwzrl8lwrbc-libfaketime-0.9.9-checkout.drv
/gnu/store/xpnrk8hjfh7rvgqfsjwkjrb9cz1ws626-git-minimal-2.36.1.drv
/gnu/store/gavjhl823bhd95rijqf3iw3vl32ix494-openssl-1.1.1l.drv

Ludo’.
Closed
M
M
Maxime Devos wrote on 22 Jun 2022 12:49
(name . phodina)(address . phodina@protonmail.com)
80d9565a7af986075ecc93bc64ce6a48d1381efc.camel@telenet.be
Ludovic Courtès schreef op wo 22-06-2022 om 12:39 [+0200]:
Toggle quote (10 lines)
> That commit skips the test.  I tried another approach with ‘datefudge’,
> which has the advantage of being more explicit and future-proof (should
> there be similar issues lying around):
>
>                (invoke "datefudge" "2022-01-01"
>                        "make" test-target
>                        #$@(if (or (target-arm?) (target-riscv64?))
>                               #~("TESTS=-test_afalg")
>                               #~()))

upsteam just replaces the certificates when these things happen, so
there could easily be more time bombs. As such, WDYT of removing _all_
the certs in tests/certs for robustness, maybe generating them locally
with test/smime-certs/mksmime-certs.sh?

Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYrLzzxccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7rqGAQDU9WQl/p/I4RP+F3fy8N0NEauy
hNlQzLCwuqCkAWCk4AEAoF42URzn8EwJqe6bR3uC5sfdgXK9bJyefaKDgAe8jQU=
=C8h6
-----END PGP SIGNATURE-----


Closed
L
L
Ludovic Courtès wrote on 24 Jun 2022 16:47
(name . Maxime Devos)(address . maximedevos@telenet.be)
87fsjuunhy.fsf@gnu.org
Maxime Devos <maximedevos@telenet.be> skribis:
Toggle quote (16 lines)
> Ludovic Courtès schreef op wo 22-06-2022 om 12:39 [+0200]:
>> That commit skips the test.  I tried another approach with ‘datefudge’,
>> which has the advantage of being more explicit and future-proof (should
>> there be similar issues lying around):
>>
>>                (invoke "datefudge" "2022-01-01"
>>                        "make" test-target
>>                        #$@(if (or (target-arm?) (target-riscv64?))
>>                               #~("TESTS=-test_afalg")
>>                               #~()))
>
> Looking at <https://github.com/openssl/openssl/issues/15179>,
> upsteam just replaces the certificates when these things happen, so
> there could easily be more time bombs. As such, WDYT of removing _all_
> the certs in tests/certs for robustness, maybe generating them locally
> with test/smime-certs/mksmime-certs.sh?
That’s an option, but it might be trickier than it seems? Or is it
really just about running that script?
I thought it’d be easier and more robust to use ‘datefudge’ or similar
because it’d amount to freezing things in time (GnuTLS does that in its
test suite). It didn’t work for some reason but it might be worth
investigating.
Ludo’.
Closed
M
M
Maxime Devos wrote on 24 Jun 2022 17:00
(name . Ludovic Courtès)(address . ludo@gnu.org)
a76013c8bfcdff3d09b5e651f694a76f0ce4a558.camel@telenet.be
Ludovic Courtès schreef op vr 24-06-2022 om 16:47 [+0200]:
Toggle quote (3 lines)
> That’s an option, but it might be trickier than it seems?  Or is it
> really just about running that script?

I don't know, Someone(™) would need to try it out. Though to be 100%
correct, it's not sufficient, IIRC there was something about TLS
certificates only supporting years up to 9999, so we would need to
check that the year isn't to big and if so skip tests or something.


Greetings,
Maxime.
-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYrXRjxccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7traAP9vSs8c6B1sXoXlw7b3KDsy+Gi9
r9FNhntymGOoKhR2eAEAxqNaoC01q5Mctsc4K9rIz2rbMqmIrd+fJDAAfFeZug4=
=FpYZ
-----END PGP SIGNATURE-----


Closed
M
M
Maxime Devos wrote on 8 Nov 2022 02:56
Re: Processed (with 1 errors): Re: bug#58650: build of /gnu/store/mw6ax0gk33gh082anrdrxp2flrbskxv6-openssl-1.1.1n.drv failed
(name . GNU bug tracker automated control server)(address . control@debbugs.gnu.org)(address . tracker@debbugs.gnu.org)
981a5aeb-9125-7cfe-4bbf-4cceef86932b@telenet.be
unarchive 56137
reopen 56137
merge 56137 58650
thanks
Attachment: OpenPGP_signature
M
L
L
Ludovic Courtès wrote on 15 Nov 2022 17:15
control message for bug #58650
(address . control@debbugs.gnu.org)
87k03wkxdg.fsf@gnu.org
retitle 58650 OpenSSL 1.1.1n test failures due to expired certificates (time bomb)
quit
?