Shepherd is not able to run simple networked programs as services

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • Lars Rustand
Owner
unassigned
Submitted by
Lars Rustand
Severity
normal
L
L
Lars Rustand wrote on 12 Dec 2023 18:43
(address . bug-guix@gnu.org)
877clje05x.fsf@yoga.mail-host-address-is-not-set
Hello, I have created two very simple shepherd services for two
different mail programs (offlineimap and davmail). Both of the programs
are able to run without problem when ran from the commandline, but both
of them fail with networking related errors when I try to run them as
shepherd services.

I have read all the relevant sections of the manual and looked at
similar shepherd services in the source code, but I can't find anything
that I am missing in my services. I am either missing something obvious
here or shepherd is doing something weird that messes up these programs.

Here is my service definition for offlineimap:

Toggle snippet (83 lines)
(define-module (lrustand services offlineimap)
#:use-module (gnu)
#:use-module (gnu services)
#:use-module (gnu packages mail)
#:use-module (gnu services shepherd)
#:use-module (gnu services configuration)
#:use-module (guix gexp)
#:use-module (guix records)
#:use-module (ice-9 curried-definitions)
#:use-module (gnu home services)
;; For the 'home-shepherd-service-type' mapping.
#:use-module (gnu home services shepherd)
#:export (offlineimap-configuration
offlineimap-configuration?

offlineimap-configuration-log-file
offlineimap-configuration-pid-file

offlineimap-shepherd-service
offlineimap-service-type
home-offlineimap-service-type))

(define-configuration/no-serialization offlineimap-configuration
(pid-file
(string "/var/run/offlineimap.pid")
"Where to store the PID file.")
(config-file
(string "/home/lars/.config/offlineimap/config")
"Configuration file to use.")
(log-file
(string "/home/lars/offlineimap.log")
"File where ‘offlineimap’ writes its log to.")
(user
(string "lars")
"")
(extra-options
(list-of-strings '())
"This option provides an “escape hatch” for the user to provide
arbitrary command-line arguments to ‘offlineimap’ as a list of strings.")
(home-service?
(boolean for-home?)
""))

(define offlineimap-shepherd-service
(match-record-lambda <offlineimap-configuration>
(pid-file config-file log-file user extra-options home-service?)
(list (shepherd-service
(provision '(offlineimap))
(documentation "")
(requirement (if home-service? '() '(networking user-processes)))
(start #~(make-forkexec-constructor
(list (string-append #$offlineimap
"/bin/offlineimap")
#$@extra-options
"-c" #$config-file
"-l" #$log-file)
#:user #$user
#:environment-variables
(list (string-append "HOME=" (passwd:dir (getpw #$user))))
#:log-file #$log-file
#:pid-file #$pid-file))
(stop #~(make-kill-destructor))
(one-shot? #f)
(respawn? #t)))))

(define offlineimap-service-type
(service-type
(name 'offlineimap)
(extensions
(list (service-extension shepherd-root-service-type
offlineimap-shepherd-service)))
(default-value (offlineimap-configuration))
(description
"Synchronize remote IMAP mail with local Maildir.")))

(define home-offlineimap-service-type
(service-type
(inherit (system->home-service-type offlineimap-service-type))
(default-value (for-home (offlineimap-configuration)))))



And here is my service definition for davmail:


Toggle snippet (75 lines)
(define-module (lrustand services davmail)
#:use-module (gnu)
#:use-module (gnu services)
#:use-module (gnu packages mail)
#:use-module (gnu services shepherd)
#:use-module (gnu services configuration)
#:use-module (guix gexp)
#:use-module (guix records)
#:use-module (ice-9 curried-definitions)
#:use-module (gnu home services)
;; For the 'home-shepherd-service-type' mapping.
#:use-module (gnu home services shepherd)
#:export (davmail-configuration
davmail-configuration?

davmail-configuration-log-file
davmail-configuration-pid-file

davmail-shepherd-service
davmail-service-type
home-davmail-service-type))

(define-configuration/no-serialization davmail-configuration
(pid-file
(string "/var/run/davmail.pid")
"Where to store the PID file.")
(config-file
(string "/home/lars/.config/davmail/davmail.properties")
"Configuration file to use.")
(log-file
(string "/home/lars/davmail.log")
"File where ‘davmail’ writes its log to.")
(extra-options
(list-of-strings '())
"This option provides an “escape hatch” for the user to provide
arbitrary command-line arguments to ‘davmail’ as a list of strings.")
(home-service?
(boolean for-home?)
""))

(define davmail-shepherd-service
(match-record-lambda <davmail-configuration>
(pid-file config-file log-file extra-options home-service?)
(list (shepherd-service
(provision '(davmail))
(documentation "")
(requirement (if home-service? '() '(user-processes)))
(start #~(make-forkexec-constructor
(list (string-append #$davmail
"/bin/davmail")
#$@extra-options
"-c" #$config-file)
#:log-file #$log-file
#:pid-file #$pid-file))
(stop #~(make-kill-destructor))
(one-shot? #f)
(respawn? #t)))))

(define davmail-service-type
(service-type
(name 'davmail)
(extensions
(list (service-extension shepherd-root-service-type
davmail-shepherd-service)))
(default-value (davmail-configuration))
(description
"Gateway for Outlook mail to regular IMAP and SMTP")))

(define home-davmail-service-type
(service-type
(inherit (system->home-service-type davmail-service-type))
(default-value (for-home (davmail-configuration)))))


Offlineimap fails with a socket error:

Toggle snippet (14 lines)
2023-12-12 19:34:33
2023-12-12 19:34:33 Traceback:
2023-12-12 19:34:33 File "/gnu/store/g2231vkzbnncyj72085n1ylkh8hq16lm-offlineimap-8.0.0/lib/python3.10/site-packages/offlineimap/accounts.py", line 298, in syncrunner
2023-12-12 19:34:33 self.__sync()
2023-12-12 19:34:33 File "/gnu/store/g2231vkzbnncyj72085n1ylkh8hq16lm-offlineimap-8.0.0/lib/python3.10/site-packages/offlineimap/accounts.py", line 374, in __sync
2023-12-12 19:34:33 remoterepos.getfolders()
2023-12-12 19:34:33 File "/gnu/store/g2231vkzbnncyj72085n1ylkh8hq16lm-offlineimap-8.0.0/lib/python3.10/site-packages/offlineimap/repository/IMAP.py", line 681, in getfolders
2023-12-12 19:34:33 imapobj = self.imapserver.acquireconnection()
2023-12-12 19:34:33 File "/gnu/store/g2231vkzbnncyj72085n1ylkh8hq16lm-offlineimap-8.0.0/lib/python3.10/site-packages/offlineimap/imapserver.py", line 683, in acquireconnection
2023-12-12 19:34:33 e.args[0][:35] == 'IMAP4 protocol error: socket error:':
2023-12-12 19:34:33


And the davmail service fails with:

Toggle snippet (20 lines)
2023-12-12 19:01:01 openjdk version "21" 2023-09-19
2023-12-12 19:01:01 OpenJDK Runtime Environment (build 21+-adhoc.nixbld.source)
2023-12-12 19:01:01 OpenJDK 64-Bit Server VM (build 21+-adhoc.nixbld.source, mixed mode, sharing)
2023-12-12 19:01:01,734 DEBUG [main] davmail.DavGateway - Start DavMail in server mode
2023-12-12 19:01:01 2023-12-12 19:01:01,734 DEBUG [main] davmail.DavGateway - Start DavMail in server mode
2023-12-12 19:01:01,759 INFO [main] davmail - DavMail Gateway 6.2.0-3464 listening on SMTP port 1025 IMAP port 1143 CALDAV port 1080 LDAP port 1389
2023-12-12 19:01:01 2023-12-12 19:01:01,759 INFO [main] davmail - DavMail Gateway 6.2.0-3464 listening on SMTP port 1025 IMAP port 1143 CALDAV port 1080 LDAP port 1389
2023-12-12 19:01:01,828 DEBUG [CheckRelease] davmail.http.HttpClientAdapter - GET https://davmail.sourceforge.net/version.txt
2023-12-12 19:01:01 2023-12-12 19:01:01,828 DEBUG [CheckRelease] davmail.http.HttpClientAdapter - GET https://davmail.sourceforge.net/version.txt
2023-12-12 19:01:01,849 DEBUG [CheckRelease] davmail.http.DavGatewaySSLSocketFactory - createSocket davmail.sourceforge.net 443
2023-12-12 19:01:01 2023-12-12 19:01:01,849 DEBUG [CheckRelease] davmail.http.DavGatewaySSLSocketFactory - createSocket davmail.sourceforge.net 443
2023-12-12 19:01:02,404 DEBUG [CheckRelease] davmail.DavGateway - DavMail released version: 6.2.0-3464
2023-12-12 19:01:02 2023-12-12 19:01:02,404 DEBUG [CheckRelease] davmail.DavGateway - DavMail released version: 6.2.0-3464
2023-12-12 19:01:06,532 DEBUG [Shutdown] davmail - Stopping DavMail gateway
2023-12-12 19:01:06 2023-12-12 19:01:06,532 DEBUG [Shutdown] davmail - Stopping DavMail gateway
2023-12-12 19:01:06,539 INFO [Shutdown] davmail - DavMail gateway stopped
2023-12-12 19:01:06 2023-12-12 19:01:06,539 INFO [Shutdown] davmail - DavMail gateway stopped


They both seem to fail when opening an *outgoing* socket, but davmail
seems to be able to start *listening* on several ports just fine.
L
L
Lars Rustand wrote on 15 Dec 2023 19:00
(address . bug-guix@gnu.org)
87ttoj1gdv.fsf@yoga.mail-host-address-is-not-set
Lars Rustand <rustand.lars@gmail.com> writes:

Toggle quote (5 lines)
> Hello, I have created two very simple shepherd services for two
> different mail programs (offlineimap and davmail). Both of the programs
> are able to run without problem when ran from the commandline, but both
> of them fail with networking related errors when I try to run them as
> shepherd services.
...
Toggle quote (3 lines)
> They both seem to fail when opening an *outgoing* socket, but davmail
> seems to be able to start *listening* on several ports just fine.

So, I figured this out. It had nothing to do with networking even though
it looked like it. The problem was that I had cargo cult-copied a
#:pid-file parameter from another service, believing that this was just
a path were Shepherd could create a pid-file for the service, but
Shepherd was in fact expecting the program to create the pid-file. So
when the program did not create this pid-file then Shepherd killed the
program.

So the original bug I reported is in fact not a bug at all and can be
closed. However, the error handling in Shepherd could be improved in
order to make it more clear what is happening.
L
L
Ludovic Courtès wrote on 17 Dec 2023 23:02
Re: bug#67802: Shepherd is not able to run simple networked programs as services
(name . Lars Rustand)(address . rustand.lars@gmail.com)(address . 67802@debbugs.gnu.org)
87le9sqyhw.fsf@gnu.org
Hi,

Lars Rustand <rustand.lars@gmail.com> skribis:

Toggle quote (12 lines)
> So, I figured this out. It had nothing to do with networking even though
> it looked like it. The problem was that I had cargo cult-copied a
> #:pid-file parameter from another service, believing that this was just
> a path were Shepherd could create a pid-file for the service, but
> Shepherd was in fact expecting the program to create the pid-file. So
> when the program did not create this pid-file then Shepherd killed the
> program.
>
> So the original bug I reported is in fact not a bug at all and can be
> closed. However, the error handling in Shepherd could be improved in
> order to make it more clear what is happening.

Good that you found the issue! I improved logging for this particular
case as a result of this report:


Thanks for reporting it,
Ludo’.
L
L
Ludovic Courtès wrote on 17 Dec 2023 23:02
control message for bug #67802
(address . control@debbugs.gnu.org)
87jzpcqyhp.fsf@gnu.org
tags 67802 notabug
close 67802
quit
?