vdirsyncer fails to verify certificates

  • Open
  • quality assurance status badge
Details
5 participants
  • Ethan Blanton
  • Giovanni Biscuolo
  • Leo Famulari
  • Tobias Geerinckx-Rice
  • Michael Albinus
Owner
unassigned
Submitted by
Ethan Blanton
Severity
normal
E
E
Ethan Blanton wrote on 16 Feb 2023 21:29
(address . bug-guix@gnu.org)
Y+6SIw5S64Rodiyi@colt.lan
Package: vdirsyncer
Version: 0.19.0

I am using Guix on a foreign distro of Debian GNU/Linux 11 (bullseye).

I have the following manifest installed in particular profile:

(specifications->manifest
(list "go"
"sbcl"
"khal"
"mutt"
"nss-certs"
"protobuf"
"vdirsyncer"))

Since vdirsyncer updated to 0.19.0, I cannot sync with any remote host
using CalDAV or HTTPS iCalendar files. This is reproducible with my
private servers, Microsoft Outlook 365 calendars, Google Calendars,
and others. I have moset recently verified it with Guix 312f1f4 and a
vdirsyncer producing
/gnu/store/9aa2bj3likla61zqbsim1a1c99k3jk93-vdirsyncer-0.19.0 (I don't
know how to give a more precise or useful install, please let me know
if I should, and how I would), but I have narrowed the breaking change
down to Guix revision f635f725778f86abaa77f674f8f670f74bffd7be.
Revision ed18b697c4783f139e23731f5bd0b0ed197997bb, which is vdirsyncer
0.18.0, works as expected.

The lightly redacted error that vdirsyncer produces is:

error: Unknown error occurred for [config entry]/calendarname: Cannot connect to host cloud.kb8ojh.net:443 ssl:True [SSLCertVerificationError: (1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate
(_ssl.c:1129)')]

An example configuration that causes this is:

[storage samplecalendar_public]
type = "http"

[storage localcalendar_public]
type = "filesystem"
path = "~/.calendars/public"
fileext = ".ics"

[pair public_calendar]
a = "samplecalendar_public"
b = "localcalendar public"
collections = [ "from a" ]

It appears that the root cause is in Python aiohttp, as starting the
python3 interpreter invoked by the vdirsyncer binary in the installed
profile with the GUIX_PYTHONPATH provided, then attempting to fetch an
HTTPS URL using aiohttp, will fail with an SSL error. I cannot tell
if the root configuration problem is in vdirsyncer and its
dependencies or in aiohttp, so I am reporting it against vdirsyncer,
which I can confirm is broken.

I have tried installing various certificate packages and other
packages that seemed like they might be related (such as nss-certs,
nss itself, gnutls, etc.), but not found anything that seemed to
resolve the issue.

This bug that I have reported upstream is related, but I think the
problem is with the Guix packaging and/or dependencies, not with
vdirsyncer itself:


Ethan
E
E
Ethan Blanton wrote on 25 Feb 2023 03:30
bug database indexing problem for bug #61557
(address . bug-guix@gnu.org)
Y/ly3+gvZbQuM7Wc@colt.lan
Bug #61557, filed against the vdirsyncer package in Guix and having
the title "vdirsyncer fails to verify certificates", does not show up
in the Guix bug database at issues.guix.gnu.org when searching by
keywords such as "vdirsyncer" or "certificates", although it does
appear when searching for "61557".

There may be an indexing problem.

(Filing at the request of nckx/irc)
E
E
Ethan Blanton wrote on 25 Feb 2023 03:41
reassign 61557 guix
(address . control@debbugs.gnu.org)
Y/l1RESjlrE2l4kj@colt.lan
reassign 61557 guix
thanks
T
T
Tobias Geerinckx-Rice wrote on 25 Feb 2023 03:44
vdirsyncer fails to verify certificates
c1ac8e44749e6264761e1654977a7e98@tobias.gr
reassign 61557 guix
thanks

Hi,

I had missed the Package: pseudo-header because I shouldn't be alive at
this point.

All Guix bugs should be filed against the ‘guix’ package, no matter what
the package—confusing, I know. Luckily, sending mail to bug-guix@ does
this for you, so you don't usually need to think about it.

Thanks again!

Kind regards,

T G-R

Sent from a Web browser. Excuse or enjoy my brevity.
M
M
Michael Albinus wrote on 25 Feb 2023 09:58
Re: bug#61557: bug database indexing problem for bug #61557
(name . Ethan Blanton via General discussion for the tracker at debbugs.gnu.org)(address . help-debbugs@gnu.org)
87r0uew27i.fsf@gmx.de
Ethan Blanton via "General discussion for the tracker at
debbugs.gnu.org" <help-debbugs@gnu.org> writes:

Hi Ethan,

Toggle quote (6 lines)
> Bug #61557, filed against the vdirsyncer package in Guix and having
> the title "vdirsyncer fails to verify certificates", does not show up
> in the Guix bug database at issues.guix.gnu.org when searching by
> keywords such as "vdirsyncer" or "certificates", although it does
> appear when searching for "61557".

When I search for vdirsyncer, bug#61557 appears in the hit list.

Searching for certificates does not show such a hit. However, in the bug
messages this word appears only as certificates" or "certificates". This
seems to prevent the word to be indexed. The HyperEstraier search engine
counts only words, and a leading or trailing apostrophe seems to
suppress a string to be regarded as word. Just a guess, but it is the
most plausible explanation.

Best regards, Michael.
L
L
Leo Famulari wrote on 25 Feb 2023 22:52
Re: vdirsyncer fails to verify certificates
(address . 61557@debbugs.gnu.org)
Y/qDCFVxLqOiuXrh@jasmine.lan
Thanks for the report!

Did you follow the instructions about X.509 Certificates in the manual
section Application Setup? That section is about using Guix on other
distros.

I use vdirsyncer from Guix on Debian and it works fine when validating
X.509 / TLS / HTTPS certificates.
E
E
Ethan Blanton wrote on 27 Mar 2023 00:05
(address . 61557@debbugs.gnu.org)
ZCDBpX9L2+FHO3qK@colt.lan
(Pardon the delay, for some reason I do not get email notifications
for this bug.)

I had read the X.509 Certificates section of the manual, but since my
certificates ARE in the default location of /etc/ssl/certs, and
vdirsyncer had previously worked, for some reason I did not dig into
it deeply enough, or perhaps I attempted to set it up wrongly at some
point in the past.

Setting SSL_CERT_DIR=/etc/ssl/certs in my environment fixes the
vdirsyncer package, and it syncs correctly.

I have also discovered that python aiohttp will correctly verify
certificates WITHOUT this environment variable with:

guix shell -P -C -N python python-aiohttp nss-certs openssl

Leaving out EITHER nss-certs OR openssl causes aiohttp to exhibit the
same behavior as vdirsyncer.

However, including both of these packages in the same (foreign distro)
profile that includes vdirsyncer does NOT cause vdirsyncer to
correctly verify certificates.

I am not sure what this means for this bug; certainly the change from
"working without extra configuration" to "broken without extra
configuration" is a regression in user experience, but it may be that
it is working as intended. It seems to me that the principle of least
astonishment for foreign distro users would suggest that python
aiohttp defaults to loading /etc/ssl/certs from the foreign distro, if
present.
G
G
Giovanni Biscuolo wrote on 27 Mar 2023 14:50
Re: bug#61557: vdirsyncer fails to verify certificates
(name . Leo Famulari)(address . leo@famulari.name)
87mt3y2war.fsf@xelera.eu
Hi Ethan,

I'm also using Guix on a foreign distribution (Debian)

Ethan Blanton via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

Toggle quote (6 lines)
> I had read the X.509 Certificates section of the manual, but since my
> certificates ARE in the default location of /etc/ssl/certs, and
> vdirsyncer had previously worked, for some reason I did not dig into
> it deeply enough, or perhaps I attempted to set it up wrongly at some
> point in the past.

I'm pretty sure my default profile vdirsyncer was working in the past
but stopped working while ago for this very same issue [1]; vdirsyncer
it's not working in my default profile but it's working in my "emacs"
profile

Reading (again) the "X.509 Certificates" section [2] I realized that I
had not set up SSL_CERT_DIR and SSL_CERT_FILE env variables in my
.profile

After adding this to my .profile:

Toggle snippet (6 lines)
export SSL_CERT_DIR="$HOME/.guix-profile/etc/ssl/certs"
export SSL_CERT_FILE="$HOME/.guix-profile/etc/ssl/certs/ca-certificates.crt"


now "vdirsyncer sync" is working (in my default profile, including cron
jobs)

As I said before, the SSL_CERT_* variables setting was/is not necessary
in my emacs profile and it depends on this:

within a shell in my emacs profile I have:

Toggle snippet (9 lines)
$: cat $GUIX_PROFILE/etc/profile | grep -i ssl
export CURL_CA_BUNDLE="${GUIX_PROFILE:-/gnu/store/hwc2pm42r2xg3mv0f7jlkf7dlvi6rpxh-profile}/etc/ssl/certs/ca-certificates.crt"
export SSL_CERT_FILE="${GUIX_PROFILE:-/gnu/store/hwc2pm42r2xg3mv0f7jlkf7dlvi6rpxh-profile}/etc/ssl/certs/ca-certificates.crt"
export SSL_CERT_DIR="${GUIX_PROFILE:-/gnu/store/hwc2pm42r2xg3mv0f7jlkf7dlvi6rpxh-profile}/etc/ssl/certs"
export GIT_SSL_CAINFO="${GUIX_PROFILE:-/gnu/store/hwc2pm42r2xg3mv0f7jlkf7dlvi6rpxh-profile}/etc/ssl/certs/ca-certificates.crt"


within a shell in my default profile:

Toggle snippet (6 lines)
$: cat $GUIX_PROFILE/etc/profile | grep -i ssl
export GIT_SSL_CAINFO="${GUIX_PROFILE:-/gnu/store/ylycvfsnm1gkzhph39g62bwbc9lbh3g7-profile}/etc/ssl/certs/ca-certificates.crt"


For sure it depends on the fact that an installed package in my emacs
profile (curl, not installed in my default profile) is adding
"SSL_CERT_FILE" and "SSL_CERT_DIR" in $GUIX_PROFILE/etc/profile

Since I usually source the latter when I switch to my "emacs" profile
before starting emacs in a shell:

Toggle snippet (5 lines)
GUIX_PROFILE="$GUIX_EXTRA_PROFILES"/emacs/emacs; . "$GUIX_PROFILE"/etc/profile


I get the two env variables defined in my "emacs" profile, while in my
default profile I don't

Toggle quote (3 lines)
> Setting SSL_CERT_DIR=/etc/ssl/certs in my environment fixes the
> vdirsyncer package, and it syncs correctly.

I'd use the Guix certs installed via nss-certs, but both dirs works
obviously

Please note that you should set SSL_CERT_FILE for other software

Toggle quote (12 lines)
> I have also discovered that python aiohttp will correctly verify
> certificates WITHOUT this environment variable with:
>
> guix shell -P -C -N python python-aiohttp nss-certs openssl
>
> Leaving out EITHER nss-certs OR openssl causes aiohttp to exhibit the
> same behavior as vdirsyncer.
>
> However, including both of these packages in the same (foreign distro)
> profile that includes vdirsyncer does NOT cause vdirsyncer to
> correctly verify certificates.

Strange behaviour, please can you tell us what is the output of this
command:

Toggle snippet (5 lines)
guix shell --pure --container coreutils grep nss-certs openssl -- env | grep -i ssl


I get this (meaning that both SSL env variables are defined):

Toggle snippet (6 lines)
SSL_CERT_DIR=/gnu/store/1ghginmnzplmp3nbv2jsavjgdjhgq4i3-profile/etc/ssl/certs
SSL_CERT_FILE=/gnu/store/1ghginmnzplmp3nbv2jsavjgdjhgq4i3-profile/etc/ssl/certs/ca-certificates.crt


while with

Toggle snippet (5 lines)
guix shell --pure --container coreutils grep nss-certs -- env | grep -i ssl


I get no output (meaning that env is missing SSL_CERT_* variables)

So in my tests openssl (and curl) are defining "SSL_CERT_FILE" and
"SSL_CERT_DIR" in $GUIX_PROFILE/etc/profile

I guess that also nss-certs package could add both "SSL_CERT_FILE" and
"SSL_CERT_DIR" in $GUIX_PROFILE/etc/profile but I don't know the ratio
for this choiche

Toggle quote (5 lines)
> I am not sure what this means for this bug; certainly the change from
> "working without extra configuration" to "broken without extra
> configuration" is a regression in user experience, but it may be that
> it is working as intended.

The bug I see here is that X.509 certificates are "working without extra
configuration" **depending** on installed packages.

If possible I'd patch nss-certs in order to add "SSL_CERT_FILE" and
"SSL_CERT_DIR" to $GUIX_PROFILE/etc/profile, this would also avoid the
extra step of "manually" defining X.509 related variables on foreign
distros

I'd also investigate this "meta-issue" for other packages, e.g. for R
that needs "CURL_CA_BUNDLE", added when installing curl but not
r-minimal

Toggle quote (4 lines)
> It seems to me that the principle of least astonishment for foreign
> distro users would suggest that python aiohttp defaults to loading
> /etc/ssl/certs from the foreign distro, if present.

IMHO it's better to use the nss-certs installed via Guix than the
foreign distro ones

HTH! Gio'


[1] maybe I was using a Guix package able to add "SSL_CERT_FILE" and
"SSL_CERT_DIR" to $GUIX_PROFILE/etc/profile and then I removed it


--
Giovanni Biscuolo

Xelera IT Infrastructures
-----BEGIN PGP SIGNATURE-----

iQJABAEBCgAqFiEERcxjuFJYydVfNLI5030Op87MORIFAmQhkP0MHGdAeGVsZXJh
LmV1AAoJENN9DqfOzDkST8QP/3C+zDL9KIeZZqjN8iLIFfFDz5gqyN5z30EpWXID
du/CGki6GiiGOlQ78i4cByIk9bskLD7pLGUgsU66gmMOrP3JYUa+e8CM/XPKtkmH
t8j7wvWO0Na+liSVk7zqzmaw7lOzycy9QToK3XC2sidlh4eDThs62x6L3VCCtHq7
CoyomJYO0XM9sewnkR3+GNJr5RZjZP1TzPVYCMf4ziJKnZ8c6VHq2PepOJJn8rLv
D5UeRmB4z0HG9vwmnFax6zm0joV0Bn4hOZLBpbrJZd5v5qDBwocwTUDk4BffLH3D
1QzORklqkE4+10WVfxsl3gPi7GEkVxPsD8LigiiEDLo48dG7X0952GQcA1bEGbof
ZawD3hFjQyDzQNTuDOt5L4ZYBcw8aD31cg4vKCLGvbOq9b0BiOk5+KD2MozikbBs
5rVR7XaUwLsuQwK4WHSi4xbCWsLfWprKpESt1w+qd9LH3rX3QW2926kv/6oq9SiN
pES8EZi5HEXOmyNQonGb4sxCMa2UZaEoMZG7nKdKT+h4ngxXpMDmB8Bp+3uEqVoS
oODseq4yvc0ygU+mOb+r6rpcmLyjZBoPIvfS36B6Xh5JimkZEUZn5I4JNh+4WrN3
lN9FqAdDwqNvPoEA/WjoIMvkrAU0GRH7ibrpKxMWfmWe1oI1gBMDPF3AA747NbrK
BOG4
=4rvy
-----END PGP SIGNATURE-----

?