nginx: Fails to start on boot while upstream service is not yet running

  • Done
  • quality assurance status badge
Details
3 participants
  • Jonathan Brielmaier
  • Ludovic Courtès
  • mirai
Owner
unassigned
Submitted by
Jonathan Brielmaier
Severity
normal
J
J
Jonathan Brielmaier wrote on 23 Nov 2022 16:34
(address . bug-guix@gnu.org)
82b04209-e3ec-a854-314e-d3c77ad14ac4@web.de
When I start my personal server with a radicale service behind a nginx
reverse proxy, nginx fails to start.

The relevant part in the log:
Nov 23 16:02:56 localhost shepherd[1]: Service networking has been started.
Nov 23 16:02:56 localhost shepherd[1]: Service radicale has been started.
Nov 23 16:02:56 localhost shepherd[1]: Service ssh-daemon has been started.
Nov 23 16:02:56 localhost shepherd[1]: [nginx] nginx: [emerg] host not
found in upstream "localhost:5232" in
/gnu/store/y29zl57pprwxbcxfx593s16456kxk99y-nginx.conf:15
Nov 23 16:02:56 localhost shepherd[1]: Failed to start nginx in the
background.

The config can be found here:
```
(upstream-blocks (list
(nginx-upstream-configuration
(name "radicale")
(servers (list "localhost:5232")))))
```

I wonder whats going wrong here. Is there a way to define that nginx
service should wait until radicale service is started?

~Jonathan
M
(address . 59515@debbugs.gnu.org)
61612e6d-0ac1-8129-917c-2e72d629c3be@makinata.eu
nginx-configuration has a 'shepherd-requirement' parameter that can be used here
J
J
Jonathan Brielmaier wrote on 25 Nov 2022 09:56
(address . 59515@debbugs.gnu.org)
ca94c57f-b01e-8201-da00-718991dda7f0@web.de
Ah thanks.

I added `(shepherd-requirement '(radicale))` to my nginx-configuration.
Sadly the issue is not fixed, the situation is the same as before.

I assume that radicale takes a fracture of a second to start. And nginx
starts faster up. See this small experiment:

$ sudo herd stop radicale
$ sudo herd start radicale; curl localhost:5232
Service radicale has been started.
curl: (7) Failed to connect to localhost port 5232 after 0 ms:
Verbindungsaufbau abgelehnt

$ sudo herd stop radicale
$ sudo herd start radicale; sleep 0.2; curl localhost:5232
Service radicale has been started.
Redirected to /.web
L
L
Ludovic Courtès wrote on 27 Nov 2022 22:14
(name . Jonathan Brielmaier)(address . jonathan.brielmaier@web.de)(address . 59515@debbugs.gnu.org)
87a64c5cc7.fsf@gnu.org
Jonathan Brielmaier <jonathan.brielmaier@web.de> skribis:

Toggle quote (6 lines)
> I added `(shepherd-requirement '(radicale))` to my nginx-configuration.
> Sadly the issue is not fixed, the situation is the same as before.
>
> I assume that radicale takes a fracture of a second to start. And nginx
> starts faster up. See this small experiment:

That looks like a bug in the radicale service: there’s no
synchronization upon startup:

Toggle snippet (12 lines)
(list (shepherd-service
(provision '(radicale))
(documentation "Run the radicale daemon.")
(requirement '(networking))
(start #~(make-forkexec-constructor
(list #$(file-append package "/bin/radicale")
"-C" #$config-file)
#:user "radicale"
#:group "radicale"))
(stop #~(make-kill-destructor))))

It should wait for a PID file or be started as a systemd or inetd
service. That way, we’d be sure that shepherd considers it started once
it’s really listening for input. A quick grep in the source suggests it
creates a PID file somewhere.

HTH!

Ludo’.
J
J
Jonathan Brielmaier wrote on 2 Jan 2023 18:09
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 59515@debbugs.gnu.org)
ce8875fb-515e-9714-d1c4-49c7a4e7e946@web.de
On 27.11.22 22:14, Ludovic Courtès wrote:> That looks like a bug in the
radicale service: there’s no
Toggle quote (2 lines)
> synchronization upon startup:

I think its rather an issue with nscd, networking or some other base
service.

Changing the nginx configuration from
(servers (list "localhost:5232"))
to
(servers (list "127.0.0.1:5232"))
resolves the issue for me :)

Toggle quote (5 lines)
> It should wait for a PID file or be started as a systemd or inetd
> service. That way, we’d be sure that shepherd considers it started once
> it’s really listening for input. A quick grep in the source suggests it
> creates a PID file somewhere.

I couldn't find any PID file on my server of radicale. But there seems
to by a .lock file: /var/lib/radicale/collections/.Radicale.lock
No idea if that would be helpful...

~Jonathan
J
J
Jonathan Brielmaier wrote on 22 Mar 2023 00:02
nginx: Fails to start on boot while upstream service is not yet running
(address . 59515-done@debbugs.gnu.org)
adb1b5b7-f016-cf7c-0f98-aac29e5ae25a@web.de
Closing.
Workaround with using "127.0.0.1" instead of "localhost" provided.
Closed
?