(guix build download) leaks file descriptor on TLS connections

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Valentin Ignatev
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
Merged with
L
L
Ludovic Courtès wrote on 19 Mar 2015 19:16
(address . bug-guix@gnu.org)
87619w6efb.fsf@gnu.org
When opening an HTTPS connection, the file descriptor beneath the port
returned by ‘tls-wrap’ is leaked.

This is not a problem in most cases (downloads) because the process is
left as soon as the download is over.

This is more problematic for ‘guix lint’, which may open a large number
of HTTPS connections for the ‘source’ and ‘home-page’ checkers when
working on all the packages.

Ludo’.
L
L
Ludovic Courtès wrote on 17 Mar 2016 23:57
(address . 20145-done@debbugs.gnu.org)
87oaac1zme.fsf@gnu.org
ludo@gnu.org (Ludovic Courtès) skribis:

Toggle quote (10 lines)
> When opening an HTTPS connection, the file descriptor beneath the port
> returned by ‘tls-wrap’ is leaked.
>
> This is not a problem in most cases (downloads) because the process is
> left as soon as the download is over.
>
> This is more problematic for ‘guix lint’, which may open a large number
> of HTTPS connections for the ‘source’ and ‘home-page’ checkers when
> working on all the packages.

This is essentially solved by commits
14d6ca3e4dd23ee92adb5e2fcf58546e67534631 and
097a951e96718a037dbfa6d579e2d26f7dab3e82.

One still needs to be careful, though, for instance because closing a
chunked encoding port (which is a custom binary input port wrapped
around the real socket port) still fails to close the raw socket port
that’s behind the TLS session record port.

Ludo’.
Closed
L
L
Ludovic Courtès wrote on 3 Jan 2020 00:05
unarchive 20145
(address . request@debbugs.gnu.org)
87y2upqx1j.fsf@gnu.org
unarchive 20145
reopen 20145
thanks
L
L
Ludovic Courtès wrote on 3 Jan 2020 00:08
control message for bug #20145
(address . control@debbugs.gnu.org)
87v9ptqwxa.fsf@gnu.org
merge 20145 38836
quit
L
L
Ludovic Courtès wrote on 3 Jan 2020 00:19
Re: bug#20145: (guix build download) leaks file descriptor on TLS connections
(address . 20145@debbugs.gnu.org)
87png1qweg.fsf_-_@gnu.org
Hi,

Back in 2015, I closed https://issues.guix.gnu.org/issue/20145 saying:

Toggle quote (21 lines)
> ludo@gnu.org (Ludovic Courtès) skribis:
>
>> When opening an HTTPS connection, the file descriptor beneath the port
>> returned by ‘tls-wrap’ is leaked.
>>
>> This is not a problem in most cases (downloads) because the process is
>> left as soon as the download is over.
>>
>> This is more problematic for ‘guix lint’, which may open a large number
>> of HTTPS connections for the ‘source’ and ‘home-page’ checkers when
>> working on all the packages.
>
> This is essentially solved by commits
> 14d6ca3e4dd23ee92adb5e2fcf58546e67534631 and
> 097a951e96718a037dbfa6d579e2d26f7dab3e82.
>
> One still needs to be careful, though, for instance because closing a
> chunked encoding port (which is a custom binary input port wrapped
> around the real socket port) still fails to close the raw socket port
> that’s behind the TLS session record port.

Unfortunately, the bug just reported by Valentin and by Ricardo are
instances of this problem (at least I checked with crates.io and it
uses chunked encoding, leading to a file descriptor leak):


To be continued…

Ludo’.
V
V
Valentin Ignatev wrote on 3 Jan 2020 13:25
(name . Ludovic Courtès)(address . ludo@gnu.org)
CADXmXkjq+JbUsWcy1a+_Hq=n6fd_ZdJ6tYDZjf7z0w6502ScVw@mail.gmail.com
Hey Ludo, thanks for providing more info! I understand that the best
way is to fix the leak for good, but I wonder if there's some possible
quick workaround to mitigate the issue in case of a recursive import?
Like giving package definitions for packages that were followed before
exception happened so the person who packages something can go on from
that point, or something like that.

Valentin.
L
L
Ludovic Courtès wrote on 3 Jan 2020 16:12
(address . 20145-done@debbugs.gnu.org)
87png0poac.fsf@gnu.org
Hello again!

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (30 lines)
> Back in 2015, I closed https://issues.guix.gnu.org/issue/20145 saying:
>
>> ludo@gnu.org (Ludovic Courtès) skribis:
>>
>>> When opening an HTTPS connection, the file descriptor beneath the port
>>> returned by ‘tls-wrap’ is leaked.
>>>
>>> This is not a problem in most cases (downloads) because the process is
>>> left as soon as the download is over.
>>>
>>> This is more problematic for ‘guix lint’, which may open a large number
>>> of HTTPS connections for the ‘source’ and ‘home-page’ checkers when
>>> working on all the packages.
>>
>> This is essentially solved by commits
>> 14d6ca3e4dd23ee92adb5e2fcf58546e67534631 and
>> 097a951e96718a037dbfa6d579e2d26f7dab3e82.
>>
>> One still needs to be careful, though, for instance because closing a
>> chunked encoding port (which is a custom binary input port wrapped
>> around the real socket port) still fails to close the raw socket port
>> that’s behind the TLS session record port.
>
> Unfortunately, the bug just reported by Valentin and by Ricardo are
> instances of this problem (at least I checked with crates.io and it
> uses chunked encoding, leading to a file descriptor leak):
>
> https://issues.guix.gnu.org/issue/38857
> https://issues.guix.gnu.org/issue/38836

Commit f4cde9ac4aedb516c050a30fd999673da434bfa0 fixes it for good it
seems! (You can monitor /proc/PID/fd while ‘guix refresh’ or ‘guix
import crate -r’ is running. :-))

There was also a CRAN-specific FD leak fixed in
af0aefd8c10701fa32341506e36297e5105f6143.

Let me know is anything is amiss!

Ludo’.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 20145@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 20145
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch