[core-updates-frozen] lualatex needs additional setup

  • Done
  • quality assurance status badge
Details
5 participants
  • Thiago Jung Bauermann
  • John Kehayias
  • Ludovic Courtès
  • Ricardo Wurmus
  • Robin Templeton
Owner
unassigned
Submitted by
John Kehayias
Severity
normal
J
J
John Kehayias wrote on 17 Oct 2021 17:25
(name . bug-guix@gnu.org)(address . bug-guix@gnu.org)
OEeidAceWLbJcLHC0q3L-6Lm6opZ41cpE5uSV3K88dIxQeUydodwhmFVnYVOFMKwF-vFC-g--S9uXYwGrrdKgGWXBHgaLEhEjuAdNoKiw_w=@protonmail.com
Hello,

This is my first time trying out TeX on Guix, but ran into a problem trying to run lualatex on core-updates-frozen. Since I wasn't sure what I needed, I tried the huge full texlive package in case that solved the problem I was seeing. It did not.

Running lualatex on anything gives the output

This is LuaTeX, Version 1.13.0 (TeX Live 2021/GNU Guix)
restricted system commands enabled.

kpathsea: Running mktexfmt lualatex.fmt
/gnu/store/cpflshy9z2yzd8dybm0j9a7i0bscpxa2-texlive-bin-20210325/share/texmf-dist/scripts/texlive/fmtutil.pl: Unexpected non-option argument(s): lualatex.fmt
Try "fmtutil --help" for more information.
I can't find the format file `lualatex.fmt'!

xelatex on the same file and environment worked fine. After some digging, I could get the lualatex.fmt file by installing texlive-tex-ini-files and running

lualatex -ini /gnu/store/skznnf2n2f4dwg44gm8bvcn78157z69p-texlive-tex-ini-files-59745/share/texmf-dist/tex/generic/tex-ini-files/lualatex.ini

With this lualatex.fmt file present, lualatex runs and produces the correct output. I'm guessing this is not the expected procedure, though I haven't been able to test on master to compare yet. I wasn't sure if I missed a texlive- package, but xelatex working as is makes me think not.

John
J
J
John Kehayias wrote on 17 Oct 2021 18:05
(name . 51252@debbugs.gnu.org)(address . 51252@debbugs.gnu.org)
BKtS6WUmC0Ib2dhYGQjJfBvjIDIbrve5PDynShSH0liSqEcTBgU_HRjOSQlrJC3po7-uo8zhpm89Bm4qHPZhIPuolUFhDipJqwjFXq65KJs=@protonmail.com
With just texlive-base I get:

---! lualatex.fmt was written by luahbtex
(Fatal format file error; I'm stymied)%

On IRC it was reported lualatex works with texlive installed without additional setup, so this does seem like a core-updates-frozen bug. Unless I've done something I didn't realize that would do this?
R
R
Robin Templeton wrote on 17 Oct 2021 18:05
Re: bug#51252: [core-updates-frozen] lualatex needs additional setup
(name . John Kehayias)(address . john.kehayias@protonmail.com)(address . 51252@debbugs.gnu.org)
877debr61d.fsf@terpri.org
John Kehayias <john.kehayias@protonmail.com> writes:

Toggle quote (6 lines)
> With this lualatex.fmt file present, lualatex runs and produces the
> correct output. I'm guessing this is not the expected procedure,
> though I haven't been able to test on master to compare yet. I wasn't
> sure if I missed a texlive- package, but xelatex working as is makes
> me think not.

I tested this on master, and lualatex (installed via texlive) does
indeed work without any additional setup.

Robin
R
R
Ricardo Wurmus wrote on 20 Oct 2021 23:45
[core-updates-frozen] lualatex needs additional setup
(address . 51252@debbugs.gnu.org)
87r1cf9xl9.fsf@elephly.net
“texlive-latex-base” provides that file. We disable a whole bunch
of formats that we cannot build that early in the process, and
then we run fmtutil-sys on the patched file.
The cause for trouble lies in texlive-kpathsea, which provides
share/texmf-dist/web2c/fmtutil.cnf
. That file states how to build the various fmt files.

The file in the earlier version of Tex Live contains this line for the lualatex format:

lualatex luatex language.dat,language.dat.lua lualatex.ini

the new file (on core-updates-frozen) says this:

lualatex luahbtex language.dat,language.dat.lua lualatex.ini

i.e. it will try to build the lualatex fmt file with luahbtex
instead of luatex. I suppose at this point in the build we don’t
actually have a working luahbtex, so fmtutil-sys doesn’t generate
the correct lualatex.fmt.

I’m not sure how to fix this, but I guess we can avoid a world
rebuild by adding an extra package that installs a working
lualatex.fmt file.

--
Ricardo
T
T
Thiago Jung Bauermann wrote on 23 Nov 2021 02:40
(address . 51252@debbugs.gnu.org)
3440649.fVeeSc1VS3@popigai
Hello,

I did some investigation about this problem today. Sorry for the trouble.

Em quarta-feira, 20 de outubro de 2021, às 18:45:35 -03, Ricardo Wurmus escreveu:
Toggle quote (19 lines)
> “texlive-latex-base” provides that file. We disable a whole bunch
> of formats that we cannot build that early in the process, and
> then we run fmtutil-sys on the patched file.
> The cause for trouble lies in texlive-kpathsea, which provides
> share/texmf-dist/web2c/fmtutil.cnf
> . That file states how to build the various fmt files.
>
> The file in the earlier version of Tex Live contains this line for the
> lualatex format:
>
> lualatex luatex language.dat,language.dat.lua lualatex.ini
>
> the new file (on core-updates-frozen) says this:
>
> lualatex luahbtex language.dat,language.dat.lua lualatex.ini
>
> i.e. it will try to build the lualatex fmt file with luahbtex
> instead of luatex.

Thank you for this analysis!

Toggle quote (4 lines)
> I suppose at this point in the build we don’t
> actually have a working luahbtex, so fmtutil-sys doesn’t generate
> the correct lualatex.fmt.

Looking at the build log of ‘texlive-latex-base’ from the core-updates-
frozen branch, LuaHBTeX seems to be functional at the time it is used to
generate ‘lualatex.fmt’. Comparing it to the build log of the same package
on master, there are some differences though:

• core-updates-frozen shows the following messages:
• “No file TS1lmr.fd.”
• “No file latex2e-first-aid-for-external-files.ltx.”
• core-updates-frozen lists some fonts with strange names and sizes:
• \font\c__fp_exp_intarray=cmr10 at 0.00002pt
• \font\c__fp_trig_intarray=cmr10 at 0.00003pt
• \font\g__regex_state_active_intarray=cmr10 at 0.00005pt
• \font\g__regex_thread_info_intarray=cmr10 at 0.00006pt
• \font\g__regex_submatch_prev_intarray=cmr10 at 0.00008pt
• \font\g__regex_submatch_begin_intarray=cmr10 at 0.00009pt
• \font\g__regex_submatch_end_intarray=cmr10 at 0.0001pt
• \font\g__regex_balance_intarray=cmr10 at 0.00012pt
• master instals file at web2c/luatex/lualatex.fmt, while
core-updates-frozen installs it at web2c/luahbtex/lualatex.fmt.

This last difference coupled with the following excerpt from the TexLive
news section¹:

“LuaTeX: Integration with HarfBuzz library, available as new engines
luahbtex (used for lualatex) and luajithbtex.”

suggests that the command ‘lualatex’ is supposed to invoke the LuaHBTeX
engine rather than the LuaTeX engine. Indeed, when using LuaHBTeX
explicitly, there’s no error about the format file. Unfortunately, there’s an
error about font loading:

Toggle snippet (19 lines)
popigai ?env?: luahbtex '&lualatex' hello.tex
This is LuaHBTeX, Version 1.13.0 (TeX Live 2021/GNU Guix)
restricted system commands enabled.
(./hello.tex
LaTeX2e <2020-10-01> patch level 4
L3 programming layer <2021-02-18> (/gnu/store/nx4jih5xnm6hzfgvi04w4wkp4pbma8bm-profile/share/texmf-dist/tex/latex/base/article.cls
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class
(/gnu/store/nx4jih5xnm6hzfgvi04w4wkp4pbma8bm-profile/share/texmf-dist/tex/latex/base/size10.clo
luaotfload | db : Font names database not found, generating new one.
luaotfload | db : This can take several minutes; please be patient.
luaotfload | db : Reload initiated (formats: otf,ttf,ttc); reason: File not found: "lmroman10-regular".
! Font \TU/lmr/m/n/10=[lmroman10-regular]:+tlig; at 10pt not loadable: metric data not found or bad.
<to be read again>
relax
l.54 \normalsize
?

I tried running `luaotfload-tool --update` as suggested on the interwebs
for a similar problem, but I ran into a separate issue with that tool:

Toggle snippet (8 lines)
popigai ?env?: luaotfload-tool --update
...ih5xnm6hzfgvi04w4wkp4pbma8bm-profile/bin/luaotfload-tool:183: module 'alt_getopt' not found:
no field package.preload['alt_getopt']
[kpse lua searcher] file not found: 'alt_getopt'
[kpse C searcher] file not found: 'alt_getopt'
popigai ?env?:

T
T
Thiago Jung Bauermann wrote on 4 Dec 2021 03:16
[PATCH core-updates-frozen] gnu: Add texlive-latex-luatex.
(address . guix-patches@gnu.org)
20211204021611.80688-1-bauermann@kolabnow.com
When TeX Live was updated to version 2021 the ‘lualatex’ format started
being generated with the LuaHBTeX engine, but the ‘lualatex’ command still
uses the LuaTeX engine. This causes the command to fail:

user@popigai:~$ lualatex hello.tex
This is LuaTeX, Version 1.13.0 (TeX Live 2021/GNU Guix)
restricted system commands enabled.

---! lualatex.fmt was written by luahbtex
(Fatal format file error; I'm stymied)user@popigai:~$

The correct way to fix this problem would be either to change texlive-bin
to make ‘lualatex’ use the LuaHBTeX engine, or to change texlive-latex-base
to generate ‘lualatex.fmt’ with LuaTeX. Both options would rebuild large
parts of the world.

Ricardo Wurmus suggested creating a new package to provide ‘lualatex.fmt’
for LuaTeX as a workaround. This is what this package does.


* gnu/packages/tex.scm (texlive-latex-luatex): New variable.
---

Hello,

Strictly speaking, this patch solves the problem reported in issue 51252, but
I still cannot use ‘lualatex’ because of a problem with fonts:

Toggle snippet (29 lines)
user@popigai:~$ lualatex hello.tex
This is LuaTeX, Version 1.13.0 (TeX Live 2021/GNU Guix)
restricted system commands enabled.
(./hello.tex
LaTeX2e <2020-10-01> patch level 4
L3 programming layer <2021-02-18> (/home/user/.guix-profile/share/texmf-dist/tex/latex/base/article.cls
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class
(/home/user/.guix-profile/share/texmf-dist/tex/latex/base/size10.clo
luaotfload | db : Font names database not found, generating new one.
luaotfload | db : This can take several minutes; please be patient.
luaotfload | db : Reload initiated (formats: otf,ttf,ttc); reason: File not found: "lmroman10-regular".
! Font \TU/lmr/m/n/10=[lmroman10-regular]:+tlig; at 10pt not loadable: metric data not found or bad.
<to be read again>
relax
l.54 \normalsize

?
! Emergency stop.
<to be read again>
relax
l.54 \normalsize

307 words of node memory still in use:
1 hlist, 1 dir, 3 kern, 1 glyph, 1 attribute, 39 glue_spec, 1 attribute_list, 3 if_stack nodes
avail lists: 2:7,3:3,4:1,5:1
! ==> Fatal error occurred, no output PDF file produced!
Transcript written on hello.log.

This happens even though I have the packages texlive-base, texlive-lm and
texlive-fonts-latex installed in the profile. I suspect this is a problem with
luaotfload. I’ll poke at it a bit to see if I can find something out.

The reporter of 51252 doesn’t seem to hit this font issue, so perhaps this patch
will be enough for them.

In the (hopefuly) near future, I’ll submit a patch for core-updates changing
‘lualatex’ to use the LuaHBTeX engine and deprecating this package.

Thanks,
Thiago


gnu/packages/tex.scm | 65 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 65 insertions(+)

Toggle diff (78 lines)
diff --git a/gnu/packages/tex.scm b/gnu/packages/tex.scm
index 910be212ac17..256fe7da90bc 100644
--- a/gnu/packages/tex.scm
+++ b/gnu/packages/tex.scm
@@ -3746,6 +3746,71 @@ (define-public texlive-luaotfload
(define-deprecated-package texlive-luatex-luaotfload texlive-luaotfload)
+;; FIXME: This package is a temporary workaround to provide ‘lualatex.fmt’ for
+;; the LuaTeX engine. It is needed because it was discovered too late in the
+;; core-updates-frozen cycle that texlive-latex-base only provides it for
+;; LuaHBTeX. See https://issues.guix.gnu.org/51252.
+(define-public texlive-latex-luatex
+ (package
+ (name "texlive-latex-luatex")
+ (version (number->string %texlive-revision))
+ (source #f)
+ (build-system gnu-build-system)
+ (arguments
+ `(#:modules ((guix build gnu-build-system)
+ (guix build utils)
+ (ice-9 rdelim)
+ (ice-9 string-fun))
+ #:phases
+ (modify-phases %standard-phases
+ (delete 'unpack)
+ (delete 'bootstrap)
+ (delete 'configure)
+ (delete 'check)
+ (replace 'build
+ (lambda* (#:key inputs #:allow-other-keys)
+ (mkdir "web2c")
+ (let ((fmtutil.cnf-in (open-file
+ (string-append
+ (assoc-ref inputs "texlive-kpathsea")
+ "/share/texmf-dist/web2c/fmtutil.cnf")
+ "r"))
+ (fmtutil.cnf-out (open-file "web2c/fmtutil.cnf" "w")))
+
+ ;; Copy ‘lualatex’ format lines to the new fmtutil.cnf, changing
+ ;; the engine from ‘luahbtex’ to ‘luatex’.
+ (do ((line "" (read-line fmtutil.cnf-in 'concat)))
+ ((eof-object? line))
+ (when (string-prefix? "lualatex" line)
+ (display (string-replace-substring line "luahbtex" "luatex")
+ fmtutil.cnf-out)))
+ (close-port fmtutil.cnf-out)
+ (close-port fmtutil.cnf-in)
+
+ (invoke "fmtutil" "--sys" "--all" "--fmtdir=web2c"
+ "--cnffile=web2c/fmtutil.cnf")
+
+ ;; Don't risk this file interfering with anything else.
+ (delete-file "web2c/fmtutil.cnf"))))
+ (replace 'install
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let ((web2c (string-append %output "/share/texmf-dist/web2c")))
+ (mkdir-p web2c)
+ (copy-recursively "web2c" web2c)))))))
+ (native-inputs
+ `(("texlive-bin" ,texlive-bin)
+ ("texlive-babel" ,texlive-babel)
+ ("texlive-cm" ,texlive-cm)
+ ("texlive-fonts-latex" ,texlive-fonts-latex)
+ ("texlive-kpathsea" ,texlive-kpathsea)
+ ("texlive-latex-base" ,texlive-latex-base)
+ ("texlive-lm" ,texlive-lm)
+ ("texlive-tex-ini-files" ,texlive-tex-ini-files)))
+ (home-page (package-home-page texlive-latex-base))
+ (synopsis "LuaLaTeX format files for LuaTeX")
+ (description "This package is necessary to use LaTeX with the LuaTeX engine.")
+ (license (package-license texlive-latex-base))))
+
(define-public texlive-latex-amsmath
(package
(name "texlive-latex-amsmath")

base-commit: 6d4f260166eb499564fa31ce487ecf301f445e4e
L
L
Ludovic Courtès wrote on 5 Dec 2021 16:40
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
871r2rukjy.fsf@gnu.org
Hello,

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (23 lines)
> When TeX Live was updated to version 2021 the ‘lualatex’ format started
> being generated with the LuaHBTeX engine, but the ‘lualatex’ command still
> uses the LuaTeX engine. This causes the command to fail:
>
> user@popigai:~$ lualatex hello.tex
> This is LuaTeX, Version 1.13.0 (TeX Live 2021/GNU Guix)
> restricted system commands enabled.
>
> ---! lualatex.fmt was written by luahbtex
> (Fatal format file error; I'm stymied)user@popigai:~$
>
> The correct way to fix this problem would be either to change texlive-bin
> to make ‘lualatex’ use the LuaHBTeX engine, or to change texlive-latex-base
> to generate ‘lualatex.fmt’ with LuaTeX. Both options would rebuild large
> parts of the world.
>
> Ricardo Wurmus suggested creating a new package to provide ‘lualatex.fmt’
> for LuaTeX as a workaround. This is what this package does.
>
> Fixes https://issues.guix.gnu.org/51252.
>
> * gnu/packages/tex.scm (texlive-latex-luatex): New variable.

Applied, thanks!

I don’t fully understand the issue but it looks like a reasonable and
non-intrusive fix.

Ludo’.
Closed
T
T
Thiago Jung Bauermann wrote on 6 Dec 2021 23:53
(name . Ludovic Courtès)(address . ludo@gnu.org)
42037117.LzcUNmiDBR@popigai
Hello Ludo,

Em domingo, 5 de dezembro de 2021, às 12:40:33 -03, Ludovic Courtès
escreveu:
Toggle quote (14 lines)
> Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:
> > Ricardo Wurmus suggested creating a new package to provide
> > ‘lualatex.fmt’ for LuaTeX as a workaround. This is what this package
> > does.
> >
> > Fixes https://issues.guix.gnu.org/51252.
> >
> > * gnu/packages/tex.scm (texlive-latex-luatex): New variable.
>
> Applied, thanks!
>
> I don’t fully understand the issue but it looks like a reasonable and
> non-intrusive fix.

Thank you!

--
Thanks,
Thiago
Closed
?