Shepherd parallel make help2man issue

Hi

I got the build error below. Presumably it happens because the jobs to

create the tools has not finished before the help2man jobs has started.

I will patch around this in the Debian packaging to serialize the build:

override_dh_auto_build:

# build tools that help2man needs first

dh_auto_build $(DH_BUILD_OPTS) -- shepherd herd halt reboot

dh_auto_build $(DH_BUILD_OPTS)

But it seems like a upstream problem. I know how annoying these kind of

bugs are, sorry for another report of this type :)

/Simon

dh_auto_build

make -j42

make[1]: Entering directory '/build/reproducible-path/shepherd-1.0.2'

/usr/bin/mkdir -p "`dirname modules/shepherd/config.scm`"

/usr/bin/sed -e 's,%PREFIX%,/usr,g' -e 's,%modsrcdir%,/build/reproducible-path/shepherd-1.0.2/modules,g' -e 's,%modbuilddir%,/build/reproducible-path/shepherd-1.0.2/modules,g' -e 's,%localstatedir%,/var,g' -e 's,%runstatedir%,/run,g' -e 's,%pkglibdir%,/usr/lib/x86_64-linux-gnu/shepherd,g' -e 's,%sysconfdir%,/etc,g' -e 's,%localedir%,/usr/share/locale,g' -e 's,%VERSION%,1.0.2,g' -e 's,%PACKAGE_BUGREPORT%,bug-guix@gnu.org,g' -e 's,%PACKAGE_NAME%,GNU Shepherd,g' -e 's,%PACKAGE_URL%,https://www.gnu.org/software/shepherd/,g'-e 's,%FIBERS_SOURCE_DIRECTORY%,/usr/share/guile/site/3.0,g' -e 's,%FIBERS_OBJECT_DIRECTORY%,/usr/lib/x86_64-linux-gnu/guile/3.0/site-ccache,g' -e 's,%GUILE%,/usr/bin/guile-3.0,g' modules/shepherd/config.scm.in modules/shepherd/config.scm

make all-recursive

make[2]: Entering directory '/build/reproducible-path/shepherd-1.0.2'

Making all in po

make[3]: Entering directory '/build/reproducible-path/shepherd-1.0.2/po'

make[3]: Nothing to be done for 'all'.

make[3]: Leaving directory '/build/reproducible-path/shepherd-1.0.2/po'

make[3]: Entering directory '/build/reproducible-path/shepherd-1.0.2'

/usr/bin/mkdir -p "`dirname shepherd`" ; \

/usr/bin/sed -e 's,%PREFIX%,/usr,g' -e 's,%modsrcdir%,/build/reproducible-path/shepherd-1.0.2/modules,g' -e 's,%modbuilddir%,/build/reproducible-path/shepherd-1.0.2/modules,g' -e 's,%localstatedir%,/var,g' -e 's,%runstatedir%,/run,g' -e 's,%pkglibdir%,/usr/lib/x86_64-linux-gnu/shepherd,g' -e 's,%sysconfdir%,/etc,g' -e 's,%localedir%,/usr/share/locale,g' -e 's,%VERSION%,1.0.2,g' -e 's,%PACKAGE_BUGREPORT%,bug-guix@gnu.org,g' -e 's,%PACKAGE_NAME%,GNU Shepherd,g' -e 's,%PACKAGE_URL%,https://www.gnu.org/software/shepherd/,g'-e 's,%FIBERS_SOURCE_DIRECTORY%,/usr/share/guile/site/3.0,g' -e 's,%FIBERS_OBJECT_DIRECTORY%,/usr/lib/x86_64-linux-gnu/guile/3.0/site-ccache,g' -e 's,%GUILE%,/usr/bin/guile-3.0,g' shepherd.in shepherd ; \

chmod +x shepherd

/usr/bin/mkdir -p "`dirname herd`" ; \

/usr/bin/sed -e 's,%PREFIX%,/usr,g' -e 's,%modsrcdir%,/build/reproducible-path/shepherd-1.0.2/modules,g' -e 's,%modbuilddir%,/build/reproducible-path/shepherd-1.0.2/modules,g' -e 's,%localstatedir%,/var,g' -e 's,%runstatedir%,/run,g' -e 's,%pkglibdir%,/usr/lib/x86_64-linux-gnu/shepherd,g' -e 's,%sysconfdir%,/etc,g' -e 's,%localedir%,/usr/share/locale,g' -e 's,%VERSION%,1.0.2,g' -e 's,%PACKAGE_BUGREPORT%,bug-guix@gnu.org,g' -e 's,%PACKAGE_NAME%,GNU Shepherd,g' -e 's,%PACKAGE_URL%,https://www.gnu.org/software/shepherd/,g'-e 's,%FIBERS_SOURCE_DIRECTORY%,/usr/share/guile/site/3.0,g' -e 's,%FIBERS_OBJECT_DIRECTORY%,/usr/lib/x86_64-linux-gnu/guile/3.0/site-ccache,g' -e 's,%GUILE%,/usr/bin/guile-3.0,g' herd.in herd ; \

chmod +x herd

/usr/bin/mkdir -p "`dirname halt`" ; \

/usr/bin/sed -e 's,%PREFIX%,/usr,g' -e 's,%modsrcdir%,/build/reproducible-path/shepherd-1.0.2/modules,g' -e 's,%modbuilddir%,/build/reproducible-path/shepherd-1.0.2/modules,g' -e 's,%localstatedir%,/var,g' -e 's,%runstatedir%,/run,g' -e 's,%pkglibdir%,/usr/lib/x86_64-linux-gnu/shepherd,g' -e 's,%sysconfdir%,/etc,g' -e 's,%localedir%,/usr/share/locale,g' -e 's,%VERSION%,1.0.2,g' -e 's,%PACKAGE_BUGREPORT%,bug-guix@gnu.org,g' -e 's,%PACKAGE_NAME%,GNU Shepherd,g' -e 's,%PACKAGE_URL%,https://www.gnu.org/software/shepherd/,g'-e 's,%FIBERS_SOURCE_DIRECTORY%,/usr/share/guile/site/3.0,g' -e 's,%FIBERS_OBJECT_DIRECTORY%,/usr/lib/x86_64-linux-gnu/guile/3.0/site-ccache,g' -e 's,%GUILE%,/usr/bin/guile-3.0,g' halt.in halt ; \

chmod +x halt

/usr/bin/mkdir -p "`dirname reboot`" ; \

/usr/bin/sed -e 's,%PREFIX%,/usr,g' -e 's,%modsrcdir%,/build/reproducible-path/shepherd-1.0.2/modules,g' -e 's,%modbuilddir%,/build/reproducible-path/shepherd-1.0.2/modules,g' -e 's,%localstatedir%,/var,g' -e 's,%runstatedir%,/run,g' -e 's,%pkglibdir%,/usr/lib/x86_64-linux-gnu/shepherd,g' -e 's,%sysconfdir%,/etc,g' -e 's,%localedir%,/usr/share/locale,g' -e 's,%VERSION%,1.0.2,g' -e 's,%PACKAGE_BUGREPORT%,bug-guix@gnu.org,g' -e 's,%PACKAGE_NAME%,GNU Shepherd,g' -e 's,%PACKAGE_URL%,https://www.gnu.org/software/shepherd/,g'-e 's,%FIBERS_SOURCE_DIRECTORY%,/usr/share/guile/site/3.0,g' -e 's,%FIBERS_OBJECT_DIRECTORY%,/usr/lib/x86_64-linux-gnu/guile/3.0/site-ccache,g' -e 's,%GUILE%,/usr/bin/guile-3.0,g' reboot.in reboot ; \

chmod +x reboot

LANGUAGE= /bin/bash '/build/reproducible-path/shepherd-1.0.2/build-aux/missing' help2man --source=GNU --info-page=shepherd --output="doc/shepherd.1" "./`basename "doc/shepherd.1" .1`"

LANGUAGE= /bin/bash '/build/reproducible-path/shepherd-1.0.2/build-aux/missing' help2man --source=GNU --info-page=shepherd --output="doc/herd.1" "./`basename "doc/herd.1" .1`"

make[4]: Entering directory '/build/reproducible-path/shepherd-1.0.2'

help2man: can't get `--help' info from ./herd

Try `--no-discard-stderr' if option outputs to stderr

help2man: can't get `--help' info from ./shepherd

Try `--no-discard-stderr' if option outputs to stderr

WARNING: 'help2man' is missing on your system.

You should only need it if you modified a dependency of a man page.

You may want to install the GNU Help2man package:

https://www.gnu.org/software/help2man/

WARNING: 'help2man' is missing on your system.

You should only need it if you modified a dependency of a man page.

You may want to install the GNU Help2man package:

https://www.gnu.org/software/help2man/

make[3]: *** [Makefile:2252: doc/shepherd.1] Error 127

make[3]: *** Waiting for unfinished jobs....

make[3]: *** [Makefile:2255: doc/herd.1] Error 127

make[4]: Leaving directory '/build/reproducible-path/shepherd-1.0.2'

Updating ./doc/version.texi

make[3]: Leaving directory '/build/reproducible-path/shepherd-1.0.2'

make[2]: *** [Makefile:1470: all-recursive] Error 1

make[2]: Leaving directory '/build/reproducible-path/shepherd-1.0.2'

make[1]: *** [Makefile:871: all] Error 2

make[1]: Leaving directory '/build/reproducible-path/shepherd-1.0.2'

dh_auto_build: error: make -j42 returned exit code 2

Hi Simon,

Simon Josefsson <simon@josefsson.org> skribis:

Am I right that you’re building from a Git checkout?

The problem is that the Makefile rules for man pages are meant for

developers building the tarball; the tarball already includes man pages,

so someone building from the tarball doesn’t trigger any of these rules.

The changes below fix dependencies, but it also means that now everyone

will be building man pages from source, even if they’re already included

in the tarball. That’s probably the right thing to do longer term, but

I’d rather not do it in the middle of a stable series.

WDYT?

Ludo’.

Ludovic Courtès <ludo@gnu.org> writes:

This was during build of Debian packages of Shepherd. They use the

tarballs.

Ah. The Debian packaging intentionally removes the prebuilt manpages

because we want to know that we can rebuild them from source. That is

what is triggering this problem!

execute_after_dh_auto_clean:

rm -fv doc/shepherd.info doc/herd.1 doc/shepherd.1

rm -fv doc/stamp-vti doc/version.texi

So maybe this is not so much of an upstream problem after all, and my

fault for trying to rebuild the manpages.

Maybe we should build from a 'git-archive' snapshot instead... but those

doesn't come with PGP signatures right now, although you could do that:

essentially just:

git archive --prefix=shepherd-vX.Y/ -o shepherd-vX.Y-src.tar.gz vX.Y

gpg --sign shepherd-vX.Y-src.tar.gz

Thanks for developing that patch. This problem is complex, and I'm not

sure what the best solution really is. Your patch makes the man pages

depend on the binary, and I've had bad experiences with that. If

someone is cross-compile Shepherd (not sure if that's a realistic

situation) then help2man can't run the binary, and in this situation it

would be preferable to fall back on the prebuilt manpages. That doesn't

sound ideal.

Maybe an uglier solution is to try to run help2man, and if it works,

then fine, but if it doesn't, re-use the prebuilt manpage or fail.

/Simon

Hi Simon,

Simon Josefsson <simon@josefsson.org> skribis:

Oh. That makes perfect sense from a reproducible build perspective.

Right, I had forgotten about cross-compilation (we do rely on it in

Guix).

So in practice, we have no choice but to build the tarball natively and

then (cross-)build from the tarball.

Yeah.

Or, longer-term, extract command-line metadata from the Guile modules

and generate man pages without resorting to ‘help2man’.

Ludo’.