nginx: Fails to start on boot while upstream service is not yet running

  • Done
  • quality assurance status badge
Details
3 participants
  • Jonathan Brielmaier
  • Ludovic Courtès
  • mirai
Owner
unassigned
Submitted by
Jonathan Brielmaier
Severity
normal
J
J
Jonathan Brielmaier wrote on 23 Nov 2022 16:34
(address . bug-guix@gnu.org)
82b04209-e3ec-a854-314e-d3c77ad14ac4@web.de
When I start my personal server with a radicale service behind a nginx
reverse proxy, nginx fails to start.

The relevant part in the log:
Nov 23 16:02:56 localhost shepherd[1]: Service networking has been started.
Nov 23 16:02:56 localhost shepherd[1]: Service radicale has been started.
Nov 23 16:02:56 localhost shepherd[1]: Service ssh-daemon has been started.
Nov 23 16:02:56 localhost shepherd[1]: [nginx] nginx: [emerg] host not
found in upstream "localhost:5232" in
/gnu/store/y29zl57pprwxbcxfx593s16456kxk99y-nginx.conf:15
Nov 23 16:02:56 localhost shepherd[1]: Failed to start nginx in the
background.

The config can be found here:
```
(upstream-blocks (list
(nginx-upstream-configuration
(name "radicale")
(servers (list "localhost:5232")))))
```

I wonder whats going wrong here. Is there a way to define that nginx
service should wait until radicale service is started?

~Jonathan
M
M
mirai wrote on 23 Nov 2022 18:35
(address . 59515@debbugs.gnu.org)
61612e6d-0ac1-8129-917c-2e72d629c3be@makinata.eu
nginx-configuration has a 'shepherd-requirement' parameter that can be used here
J
J
Jonathan Brielmaier wrote on 25 Nov 2022 09:56
(address . 59515@debbugs.gnu.org)
ca94c57f-b01e-8201-da00-718991dda7f0@web.de
Ah thanks.

I added `(shepherd-requirement '(radicale))` to my nginx-configuration.
Sadly the issue is not fixed, the situation is the same as before.

I assume that radicale takes a fracture of a second to start. And nginx
starts faster up. See this small experiment:

$ sudo herd stop radicale
$ sudo herd start radicale; curl localhost:5232
Service radicale has been started.
curl: (7) Failed to connect to localhost port 5232 after 0 ms:
Verbindungsaufbau abgelehnt

$ sudo herd stop radicale
$ sudo herd start radicale; sleep 0.2; curl localhost:5232
Service radicale has been started.
Redirected to /.web
L
L
Ludovic Courtès wrote on 27 Nov 2022 22:14
(name . Jonathan Brielmaier)(address . jonathan.brielmaier@web.de)(address . 59515@debbugs.gnu.org)
87a64c5cc7.fsf@gnu.org
Jonathan Brielmaier <jonathan.brielmaier@web.de> skribis:

Toggle quote (6 lines)
> I added `(shepherd-requirement '(radicale))` to my nginx-configuration.
> Sadly the issue is not fixed, the situation is the same as before.
>
> I assume that radicale takes a fracture of a second to start. And nginx
> starts faster up. See this small experiment:

That looks like a bug in the radicale service: there’s no
synchronization upon startup:

Toggle snippet (12 lines)
(list (shepherd-service
(provision '(radicale))
(documentation "Run the radicale daemon.")
(requirement '(networking))
(start #~(make-forkexec-constructor
(list #$(file-append package "/bin/radicale")
"-C" #$config-file)
#:user "radicale"
#:group "radicale"))
(stop #~(make-kill-destructor))))

It should wait for a PID file or be started as a systemd or inetd
service. That way, we’d be sure that shepherd considers it started once
it’s really listening for input. A quick grep in the source suggests it
creates a PID file somewhere.

HTH!

Ludo’.
J
J
Jonathan Brielmaier wrote on 2 Jan 2023 18:09
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 59515@debbugs.gnu.org)
ce8875fb-515e-9714-d1c4-49c7a4e7e946@web.de
On 27.11.22 22:14, Ludovic Courtès wrote:> That looks like a bug in the
radicale service: there’s no
Toggle quote (2 lines)
> synchronization upon startup:

I think its rather an issue with nscd, networking or some other base
service.

Changing the nginx configuration from
(servers (list "localhost:5232"))
to
(servers (list "127.0.0.1:5232"))
resolves the issue for me :)

Toggle quote (5 lines)
> It should wait for a PID file or be started as a systemd or inetd
> service. That way, we’d be sure that shepherd considers it started once
> it’s really listening for input. A quick grep in the source suggests it
> creates a PID file somewhere.

I couldn't find any PID file on my server of radicale. But there seems
to by a .lock file: /var/lib/radicale/collections/.Radicale.lock
No idea if that would be helpful...

~Jonathan
J
J
Jonathan Brielmaier wrote on 22 Mar 2023 00:02
nginx: Fails to start on boot while upstream service is not yet running
(address . 59515-done@debbugs.gnu.org)
adb1b5b7-f016-cf7c-0f98-aac29e5ae25a@web.de
Closing.
Workaround with using "127.0.0.1" instead of "localhost" provided.
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 59515@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 59515
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch