nondeterministic Broken pipe

Open

Details

6 participants

Alex Vong
Andreas Enge
Oleg Pykhalov
Ludovic Courtès
Mark H Weaver
Ricardo Wurmus

Owner: unassigned

Submitted by: Alex Vong

Severity: normal

Merged with

34209

Alex Vong wrote on 23 Dec 2017 21:23

Recipients:(address . bug-guix@gnu.org)

Message-ID:874lohdwhb.fsf@gmail.com

Hello,

I get the following error when running ``guix --version | head -n 1''. I
can get similar after replacing ``--version'' with ``--help''. Also, the
error is nondeterministic. Any idea?


$ LC_ALL=C guix --version | head -n 1
guix (GNU Guix) 91213c384b62bc422896d39005efa922e4fa75f2
Backtrace:
In ice-9/boot-9.scm:
 160: 14 [catch #t #<catch-closure 5628460c6140> ...]
In unknown file:
   ?: 13 [apply-smob/1 #<catch-closure 5628460c6140>]
In ice-9/boot-9.scm:
  66: 12 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
 432: 11 [eval # #]
In ice-9/boot-9.scm:
2404: 10 [save-module-excursion #<procedure 5628460e8900 at ice-9/boot-9.scm:4051:3 ()>]
4056: 9 [#<procedure 5628460e8900 at ice-9/boot-9.scm:4051:3 ()>]
1727: 8 [%start-stack load-stack ...]
1732: 7 [#<procedure 5628460fdb40 ()>]
In unknown file:
   ?: 6 [primitive-load "/usr/local/bin/guix"]
In guix/ui.scm:
 390: 5 [show-version-and-exit "guix"]
In ice-9/format.scm:
1593: 4 [format #t "Copyright ~a 2017 ~a" "(C)" "the Guix authors\n"]
 766: 3 [format:format-work "Copyright ~a 2017 ~a" ("(C)" "the Guix authors\n")]
 264: 2 [tilde-dispatch]
  73: 1 [format:out-obj-padded #f "the Guix authors\n" #f ()]
In unknown file:
   ?: 0 [display "the Guix authors\n" #<output: file 1>]

ERROR: In procedure display:
ERROR: In procedure scm_flush: Broken pipe
Backtrace:
In ice-9/boot-9.scm:
 160: 1 [catch #t #<catch-closure 562846cd3020> ...]
In unknown file:
   ?: 0 [apply-smob/1 #<catch-closure 562846cd3020>]

ERROR: In procedure apply-smob/1:
ERROR: In procedure scm_flush: Broken pipe


Cheers,
Alex

Andreas Enge wrote on 23 Dec 2017 21:48

Recipients:(name . Alex Vong)(address . alexvong1995@gmail.com)(address . 29826@debbugs.gnu.org)

Message-ID:20171223204828.GA3290@jurong

On Sun, Dec 24, 2017 at 04:23:44AM +0800, Alex Vong wrote:

Toggle quote (3 lines)

> ERROR: In procedure display:

> ERROR: In procedure scm_flush: Broken pipe

Interesting! I get a different error message:

ERROR: In procedure display:

In procedure fport_write: Broken pipe

Andreas

Alex Vong wrote on 24 Dec 2017 09:37

Recipients:(name . Andreas Enge)(address . andreas@enge.fr)(address . 29826@debbugs.gnu.org)

Message-ID:87zi68cyhq.fsf@gmail.com

Andreas Enge <andreas@enge.fr> writes:

Toggle quote (11 lines)> On Sun, Dec 24, 2017 at 04:23:44AM +0800, Alex Vong wrote:
>> ERROR: In procedure display:
>> ERROR: In procedure scm_flush: Broken pipe
>
> Interesting! I get a different error message:
>
> ERROR: In procedure display:
> In procedure fport_write: Broken pipe
>
> Andreas

Hmm... This may due to the fact I am using Guile 2.0 instead of 2.2

Mark H Weaver wrote on 24 Dec 2017 23:11

Recipients:(name . Alex Vong)(address . alexvong1995@gmail.com)(address . 29826@debbugs.gnu.org)

Message-ID:87d133lqsb.fsf@netris.org

Alex Vong <alexvong1995@gmail.com> writes:

Toggle quote (4 lines)

> I get the following error when running ``guix --version | head -n 1''. I

> can get similar after replacing ``--version'' with ``--help''. Also, the

> error is nondeterministic. Any idea?

Attempts to write to a pipe that has already been closed on the other

end results in EPIPE. From the write(2) man page:

EPIPE fd is connected to a pipe or socket whose reading end is closed.

When this happens the writing process will also receive a

SIGPIPE signal. (Thus, the write return value is seen only if

the program catches, blocks or ignores this signal.)

In this case, there's a race condition. The result depends on whether

"head -n 1" closes its end of the pipe before or after "guix --version"

is finished writing all of its output. If "head -n 1" closes the pipe

first, then "guix --version" will receive EPIPE while attempting to

write to it.

What normally happens is that the sending process receives SIGPIPE,

which simply causes it to exit prematurely without ever receiving this

error. However, since Guix arranges to ignore SIGPIPE in

'initialize-guix' in guix/ui.scm, we receive EPIPE.

That's what's happening here. I'll need to think on how best to fix it.

Regards,

Mark

Alex Vong wrote on 25 Dec 2017 15:02

Recipients:(name . Mark H Weaver)(address . mhw@netris.org)(address . 29826@debbugs.gnu.org)

Message-ID:87608uaorx.fsf@gmail.com

Mark H Weaver <mhw@netris.org> writes:

Toggle quote (30 lines)> Alex Vong <alexvong1995@gmail.com> writes:
>
>> I get the following error when running ``guix --version | head -n 1''. I
>> can get similar after replacing ``--version'' with ``--help''. Also, the
>> error is nondeterministic. Any idea?
>
> Attempts to write to a pipe that has already been closed on the other
> end results in EPIPE.  From the write(2) man page:
>
>   EPIPE fd is connected to a pipe or socket whose reading end is closed.
>         When this happens the writing process will also receive a
>         SIGPIPE signal.  (Thus, the write return value is seen only if
>         the program catches, blocks or ignores this signal.)
>
> In this case, there's a race condition.  The result depends on whether
> "head -n 1" closes its end of the pipe before or after "guix --version"
> is finished writing all of its output.  If "head -n 1" closes the pipe
> first, then "guix --version" will receive EPIPE while attempting to
> write to it.
>
> What normally happens is that the sending process receives SIGPIPE,
> which simply causes it to exit prematurely without ever receiving this
> error.  However, since Guix arranges to ignore SIGPIPE in
> 'initialize-guix' in guix/ui.scm, we receive EPIPE.
>
> That's what's happening here.  I'll need to think on how best to fix it.
>
>      Regards,
>        Mark

Nice explaination as always! I forget to mention that I reported a bug
of similar flavour before http://bugs.gnu.org/27017. I agree that
thought is needed to fix all instances of this type of bug.

Cheers,
Alex

Ludovic Courtès wrote on 31 Dec 2017 11:11

Recipients:(name . Alex Vong)(address . alexvong1995@gmail.com)

Message-ID:87k1x32on0.fsf@gnu.org

Hi,

Alex Vong <alexvong1995@gmail.com> skribis:

Toggle quote (36 lines)> Mark H Weaver <mhw@netris.org> writes:
>
>> Alex Vong <alexvong1995@gmail.com> writes:
>>
>>> I get the following error when running ``guix --version | head -n 1''. I
>>> can get similar after replacing ``--version'' with ``--help''. Also, the
>>> error is nondeterministic. Any idea?
>>
>> Attempts to write to a pipe that has already been closed on the other
>> end results in EPIPE.  From the write(2) man page:
>>
>>   EPIPE fd is connected to a pipe or socket whose reading end is closed.
>>         When this happens the writing process will also receive a
>>         SIGPIPE signal.  (Thus, the write return value is seen only if
>>         the program catches, blocks or ignores this signal.)
>>
>> In this case, there's a race condition.  The result depends on whether
>> "head -n 1" closes its end of the pipe before or after "guix --version"
>> is finished writing all of its output.  If "head -n 1" closes the pipe
>> first, then "guix --version" will receive EPIPE while attempting to
>> write to it.
>>
>> What normally happens is that the sending process receives SIGPIPE,
>> which simply causes it to exit prematurely without ever receiving this
>> error.  However, since Guix arranges to ignore SIGPIPE in
>> 'initialize-guix' in guix/ui.scm, we receive EPIPE.
>>
>> That's what's happening here.  I'll need to think on how best to fix it.
>>
>>      Regards,
>>        Mark
>
> Nice explaination as always! I forget to mention that I reported a bug
> of similar flavour before <http://bugs.gnu.org/27017>. I agree that
> thought is needed to fix all instances of this type of bug.

Not sure! We specifically ignore EPIPE in cases where it matters, such

as for the output of ‘guix package --search’, ‘guix package -A’, etc.

In other cases, it’s probably an error, so it’s worth reporting.

WDYT?

In C such errors are usually ignored, which is nice for shell hackery

but otherwise not so great.

Ludo’.

Alex Vong wrote on 2 Jan 2018 13:07

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87po6s31ma.fsf@gmail.com

Hello,

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (51 lines)> Hi,
>
> Alex Vong <alexvong1995@gmail.com> skribis:
>
>> Mark H Weaver <mhw@netris.org> writes:
>>
>>> Alex Vong <alexvong1995@gmail.com> writes:
>>>
>>>> I get the following error when running ``guix --version | head -n 1''. I
>>>> can get similar after replacing ``--version'' with ``--help''. Also, the
>>>> error is nondeterministic. Any idea?
>>>
>>> Attempts to write to a pipe that has already been closed on the other
>>> end results in EPIPE.  From the write(2) man page:
>>>
>>>   EPIPE fd is connected to a pipe or socket whose reading end is closed.
>>>         When this happens the writing process will also receive a
>>>         SIGPIPE signal.  (Thus, the write return value is seen only if
>>>         the program catches, blocks or ignores this signal.)
>>>
>>> In this case, there's a race condition.  The result depends on whether
>>> "head -n 1" closes its end of the pipe before or after "guix --version"
>>> is finished writing all of its output.  If "head -n 1" closes the pipe
>>> first, then "guix --version" will receive EPIPE while attempting to
>>> write to it.
>>>
>>> What normally happens is that the sending process receives SIGPIPE,
>>> which simply causes it to exit prematurely without ever receiving this
>>> error.  However, since Guix arranges to ignore SIGPIPE in
>>> 'initialize-guix' in guix/ui.scm, we receive EPIPE.
>>>
>>> That's what's happening here.  I'll need to think on how best to fix it.
>>>
>>>      Regards,
>>>        Mark
>>
>> Nice explaination as always! I forget to mention that I reported a bug
>> of similar flavour before <http://bugs.gnu.org/27017>. I agree that
>> thought is needed to fix all instances of this type of bug.
>
> Not sure!  We specifically ignore EPIPE in cases where it matters, such
> as for the output of ‘guix package --search’, ‘guix package -A’, etc.
> In other cases, it’s probably an error, so it’s worth reporting.
>
> WDYT?
>
> In C such errors are usually ignored, which is nice for shell hackery
> but otherwise not so great.
>
> Ludo’.

Do you mean there are use-cases where the EPIPE signal really means
there is an error? What I think is that the 'guix' command is meant to
be used in a shell script, so it should work nice with other shell tools
in a pipe, including head & tail. But maybe it will cause other problems
if we always ignore EPIPE, I don't know...

Mark H Weaver wrote on 2 Jan 2018 20:04

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:87inckxetr.fsf@netris.org

Hi,

ludo@gnu.org (Ludovic Courtès) writes:

Toggle quote (44 lines)> Alex Vong <alexvong1995@gmail.com> skribis:
>
>> Mark H Weaver <mhw@netris.org> writes:
>>
>>> Alex Vong <alexvong1995@gmail.com> writes:
>>>
>>>> I get the following error when running ``guix --version | head -n 1''. I
>>>> can get similar after replacing ``--version'' with ``--help''. Also, the
>>>> error is nondeterministic. Any idea?
>>>
>>> Attempts to write to a pipe that has already been closed on the other
>>> end results in EPIPE.  From the write(2) man page:
>>>
>>>   EPIPE fd is connected to a pipe or socket whose reading end is closed.
>>>         When this happens the writing process will also receive a
>>>         SIGPIPE signal.  (Thus, the write return value is seen only if
>>>         the program catches, blocks or ignores this signal.)
>>>
>>> In this case, there's a race condition.  The result depends on whether
>>> "head -n 1" closes its end of the pipe before or after "guix --version"
>>> is finished writing all of its output.  If "head -n 1" closes the pipe
>>> first, then "guix --version" will receive EPIPE while attempting to
>>> write to it.
>>>
>>> What normally happens is that the sending process receives SIGPIPE,
>>> which simply causes it to exit prematurely without ever receiving this
>>> error.  However, since Guix arranges to ignore SIGPIPE in
>>> 'initialize-guix' in guix/ui.scm, we receive EPIPE.
>>>
>>> That's what's happening here.  I'll need to think on how best to fix it.
>>>
>>>      Regards,
>>>        Mark
>>
>> Nice explaination as always! I forget to mention that I reported a bug
>> of similar flavour before <http://bugs.gnu.org/27017>. I agree that
>> thought is needed to fix all instances of this type of bug.
>
> Not sure!  We specifically ignore EPIPE in cases where it matters, such
> as for the output of ‘guix package --search’, ‘guix package -A’, etc.
> In other cases, it’s probably an error, so it’s worth reporting.
>
> WDYT?

I see from the comment in (guix ui) where SIGPIPE is ignored, the

rationale:

;; Ignore SIGPIPE. If the daemon closes the connection, we prefer to be

;; notified via an EPIPE later.

(sigaction SIGPIPE SIG_IGN)

Instead of unconditionally ignoring SIGPIPE here in (initialize-guix),

it might be better to ignore SIGPIPE only if we open a connection to the

daemon with the intent of mutating the store, and perhaps in some other

cases where we're mutating information on disk (e.g. switching

generations). In those cases, we have a job to do that should ideally

be completed regardless of whether anyone is still listening to our

STDOUT.

However, in many other cases, we don't mutate anything on disk, and our

*only* job is printing information to the user, e.g. when showing

version/usage information, the list of available packages, the list of

generations, etc. In those cases, I think it would be better to let

SIGPIPE kill us, because there is no reason to keep the 'guix' process

alive if its output is going nowhere. These are also the cases where

it's most useful to pipe 'guix' output into other commands.

So, I think we should consider removing (sigaction SIGPIPE SIG_IGN) from

(initialize-guix), and instead putting it in various other selected

places.

What do you think?

Mark

Ludovic Courtès wrote on 2 Jan 2018 23:17

Recipients:(name . Mark H Weaver)(address . mhw@netris.org)

Message-ID:874lo3ykgr.fsf@gnu.org

Hello,

Mark H Weaver <mhw@netris.org> skribis:

Toggle quote (2 lines)

> ludo@gnu.org (Ludovic Courtès) writes:

[...]

Toggle quote (35 lines)>> Not sure!  We specifically ignore EPIPE in cases where it matters, such
>> as for the output of ‘guix package --search’, ‘guix package -A’, etc.
>> In other cases, it’s probably an error, so it’s worth reporting.
>>
>> WDYT?
>
> I see from the comment in (guix ui) where SIGPIPE is ignored, the
> rationale:
>
>   ;; Ignore SIGPIPE.  If the daemon closes the connection, we prefer to be
>   ;; notified via an EPIPE later.
>   (sigaction SIGPIPE SIG_IGN)
>
> Instead of unconditionally ignoring SIGPIPE here in (initialize-guix),
> it might be better to ignore SIGPIPE only if we open a connection to the
> daemon with the intent of mutating the store, and perhaps in some other
> cases where we're mutating information on disk (e.g. switching
> generations).  In those cases, we have a job to do that should ideally
> be completed regardless of whether anyone is still listening to our
> STDOUT.
>
> However, in many other cases, we don't mutate anything on disk, and our
> *only* job is printing information to the user, e.g. when showing
> version/usage information, the list of available packages, the list of
> generations, etc.  In those cases, I think it would be better to let
> SIGPIPE kill us, because there is no reason to keep the 'guix' process
> alive if its output is going nowhere.  These are also the cases where
> it's most useful to pipe 'guix' output into other commands.
>
> So, I think we should consider removing (sigaction SIGPIPE SIG_IGN) from
> (initialize-guix), and instead putting it in various other selected
> places.
>
> What do you think?

Why not. An option would be to move (sigaction SIGPIPE SIG_IGN) to

‘open-connection’, though that’s not following “library design best

practices.”

If we do that, can we really remove the ‘leave-on-EPIPE’ uses that we

have in (guix scripts package) for instance? At first sight they are in

‘process-query’, which corresponds to operations that don’t rely on the

store, so that should be safe.

There are a few other uses of ‘leave-on-EPIPE’ that happen while the

store is opened (in ‘guix size’, ‘guix challenge’). We’d have to keep

these.

Thoughts?

Ludo’.

Oleg Pykhalov wrote on 14 Feb 2018 13:21

Recipients:(name . Alex Vong)(address . alexvong1995@gmail.com)(address . 29826@debbugs.gnu.org)

Message-ID:87a7wbiwx9.fsf@gmail.com

Hello Alex,

Alex Vong <alexvong1995@gmail.com> writes:

Toggle quote (7 lines)

> I get the following error when running ``guix --version | head -n 1''. I

> can get similar after replacing ``--version'' with ``--help''. Also, the

> error is nondeterministic. Any idea?

> $ LC_ALL=C guix --version | head -n 1

> guix (GNU Guix) 91213c384b62bc422896d39005efa922e4fa75f2

As a workaround ‘LC_ALL=C head -n 1 <(guix --version)’.

[…]

Oleg.

Ricardo Wurmus wrote on 31 Jan 2019 15:27

Re: bug#34209: guix --version: ERROR: In procedure display: In procedure fport_write: Broken pipe

Recipients:(address . control@debbugs.gnu.org)

Message-ID:875zu4oqaz.fsf@elephly.net

merge 34209 29826

thanks

Ricardo

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 29826@debbugs.gnu.org

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date