texlive-* packages fail to build non-deterministically

  • Open
  • quality assurance status badge
Details
5 participants
  • Thiago Jung Bauermann
  • Ludovic Courtès
  • Ludovic Courtès
  • luigi scarso
  • Thiago Jung Bauermann
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
important
Merged with
L
L
Ludovic Courtès wrote on 27 Apr 2021 17:39
(address . bug-guix@gnu.org)
871rav68km.fsf@inria.fr
Some texlive- packages fail to build non-deterministically. This one is
on powerpc64le-linux from current master (commit
d904abe0768293b2322dbf355b6e41d94e769d78), but ISTR we saw the same kind
of problem on other architectures before:

Toggle snippet (37 lines)
Processing file fileerr.dtx (help) -> h.tex
(scroll) -> s.tex
(edit) -> e.tex
(batch) -> q.tex
(run) -> r.tex
(exit) -> x.tex
File fileerr.dtx ended by \endinput.
Lines processed: 119
Comments removed: 100
Comments passed: 0
Codelines passed: 10

***********************************************************
*
* To finish the installation you have to move the following
* files into a directory searched by TeX:
*
* All the files with extension `.sty' and `.tex'
* Note there also may be a file .tex which is `invisible'
* on some operating systems.
*
* To produce the documentation run the .dtx files through LaTeX.
*
* Happy TeXing
***********************************************************

* Finally trying to make a file `.tex'.
* (Placed at the end of this run, as this
* may fail on some operating systems.)

Generating file(s) .tex

realloc(): invalid next size
command "luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "tools.ins" failed with signal 6
builder for `/gnu/store/k9v4w4vc9q22yrrl5ggmpcymidwcbamf-texlive-latex-tools-51265.drv' failed with exit code 1

Notice the “realloc” message (from glibc) suggesting heap corruption.

Often, simply retrying yields a successful build.

Ludo’.
L
L
Ludovic Courtès wrote on 21 May 2021 23:37
control message for bug #48064
(address . control@debbugs.gnu.org)
87cztj7oq6.fsf@gnu.org
merge 48064 48455
quit
T
T
Thiago Jung Bauermann wrote on 6 Jun 2021 03:36
texlive-* packages fail to build non-deterministically
(address . 48064@debbugs.gnu.org)
5446817.TUF6V9tvpM@popigai
I just tried building core-updates after the upgrade to TeX Live 2020
(core-updates commit f72a125) and texlive-amscls failed with a slightly
different error coming from glibc, but still indicative of heap corruption:
“corrupted size vs. prev_size”

Full build output attached, and relevant snippet below:

```
This is LuaTeX, Version 1.12.0 (TeX Live 2020)
restricted system commands enabled.
(./amsdtx.ins (/gnu/store/6w056ryry98gw1h6hl1xnh725l7k5jg9-texlive-latex-base-54632/share/texmf-dist/tex/latex/base/docstrip.tex
Utility: `docstrip' 2.5g <2018/05/03>
English documentation <2018/05/03>

**********************************************************
* This program converts documented macro-files into fast *
* loadable files by stripping off (nearly) all comments! *
**********************************************************

********************************************************
* No Configuration file found, using default settings. *
********************************************************

)

Generating file(s) amsdtx.cls amsldoc.cls

Processing file amsdtx.dtx (amsdtx) -> amsdtx.cls
(amsldoc) -> amsldoc.cls
File amsdtx.dtx ended by \endinput.
Lines processed: 1196
Comments removed: 573
Comments passed: 1
Codelines passed: 584

)
warning (pdf backend): no pages of output.
Transcript written on amsdtx.log.
corrupted size vs. prev_size
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "luatex" arguments: ("-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsdtx.ins") exit-status: #f term-signal: 6 stop-signal: #f>
phase `build' failed after 6.5 seconds
command "luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsdtx.ins" failed with signal 6
```

I’ll keep trying to run valgrind on it to see where the corruption happens.
--
Thanks,
Thiago
T
T
Thiago Jung Bauermann wrote on 9 Jun 2021 02:28
(address . 48064@debbugs.gnu.org)
2686819.PDXOTiNPFx@popigai
Hello,

I was able to do a build of texlive-amscls with `export MALLOC_CHECK_=2`,
and a core dump was generated. I managed to get a guix environment with
debuginfo for texlive-bin, but for some reason it still doesn’t have
debug info for glibc available. FYI, this was the command line I used:

$ guix environment texlive-amscls --with-debug-info=texlive-bin \
--ad-hoc valgrind gdb glibc:debug

Valgrind isn’t working because of the lack of glibc debug info, but I was
able to get the backtrace below from the core file, using GDB. I am yet to
analyse it and see if it provides any clue:

Core was generated by `luatex amsclass.ins'.
Program terminated with signal SIGABRT, Aborted.
#0 0x00007f50ad3d5a7a in raise () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
(gdb) backtrace
#0 0x00007f50ad3d5a7a in raise () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#1 0x00007f50ad3c0523 in abort () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#2 0x00007f50ad415d28 in ?? () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#3 0x00007f50ad41d54a in ?? () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#4 0x00007f50ad421bd7 in ?? () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#5 0x0000000000486c7e in my_luaalloc (ud=<optimized out>, ptr=<optimized out>, osize=1104, nsize=<optimized out>)
at ../../../texlive-20190410-source/texk/web2c/luatexdir/lua/luastuff.c:115
#6 0x00007f50ad754b81 in luaM_realloc_ (L=L@entry=0x12212a8, block=block@entry=0x1316cc0,
osize=osize@entry=1104, nsize=nsize@entry=2208)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lmem.c:86
#7 0x00007f50ad74d8b3 in luaD_reallocstack (L=0x12212a8, newsize=138)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:182
#8 0x00007f50ad74e121 in luaD_precall (L=L@entry=0x12212a8, func=<optimized out>, nresults=0)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:424
#9 0x00007f50ad74e393 in luaD_precall (nresults=<optimized out>, func=<optimized out>, L=0x12212a8)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:413
#10 luaD_call (L=L@entry=0x12212a8, func=<optimized out>, nResults=<optimized out>)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:498
#11 0x00007f50ad74e3f1 in luaD_callnoyield (L=0x12212a8, func=<optimized out>, nResults=<optimized out>)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:509
#12 0x00007f50ad74d83f in luaD_rawrunprotected (L=L@entry=0x12212a8, f=f@entry=0x7f50ad74f6d0 <dothecall>,
ud=ud@entry=0x0) at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:142
#13 0x00007f50ad74e6fb in luaD_pcall (L=L@entry=0x12212a8, func=func@entry=0x7f50ad74f6d0 <dothecall>,
u=u@entry=0x0, old_top=1200, ef=ef@entry=0)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:729
#14 0x00007f50ad74f5dd in GCTM (L=L@entry=0x12212a8, propagateerrors=propagateerrors@entry=0)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lgc.c:823
#15 0x00007f50ad750eca in callallpendingfinalizers (L=<optimized out>)
--Type <RET> for more, q to quit, c to continue without paging--
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lgc.c:862
#16 luaC_freeallobjects (L=L@entry=0x12212a8)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lgc.c:971
#17 0x00007f50ad75a93e in close_state (L=0x12212a8)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lstate.c:245
#18 0x00007f50ad75ae20 in lua_close (L=<optimized out>)
at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lstate.c:344
#19 0x00000000004999a6 in do_final_end ()
at ../../../texlive-20190410-source/texk/web2c/luatexdir/tex/errors.c:257
#20 0x000000000044ea9e in main (ac=<optimized out>, av=<optimized out>)
at ../../../texlive-20190410-source/texk/web2c/luatexdir/luatex.c:609
(gdb)
--
Thanks,
Thiago
L
L
Ludovic Courtès wrote on 29 Jun 2021 12:42
control message for bug #48064
(address . control@debbugs.gnu.org)
87zgv9rly7.fsf@gnu.org
severity 48064 important
quit
L
L
Ludovic Courtès wrote on 29 Jun 2021 16:02
Re: bug#48064: texlive-* packages fail to build non-deterministically
(address . dev-luatex@ntg.nl)(address . 48064@debbugs.gnu.org)
874kdgsr9h.fsf@gnu.org
Hello,

While investigating luatex crashes in the TeX Live 2020 package of
GNU Guix¹, we identified the following heap corruption reported by
Valgrind (this is on GNU/Linux, with glibc 2.33):

Toggle snippet (61 lines)
sh-5.0$ ~ludo/.guix-profile/bin/valgrind --extra-debuginfo-path=/gnu/store/f933bvd6ab6l2lg6xl6k1a6jwvcls0z6-glibc-2.33-debug/lib/debug "luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsbsy.dtx"
==28531== Memcheck, a memory error detector
==28531== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==28531== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==28531== Command: luatex -interaction=nonstopmode -output-directory=build &luatex amsbsy.dtx
==28531==
This is LuaTeX, Version 1.12.0 (TeX Live 2020)
restricted system commands enabled.
==28531== Invalid write of size 8
==28531== at 0x485C691: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x45118D: main (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== Address 0xae14170 is 0 bytes after a block of size 1,168 alloc'd
==28531== at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==28531== by 0x46695D: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x486D932: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x48660F2: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x4868BE7: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x4868FCF: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x486988B: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x485C6BB: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==
==28531== Invalid write of size 4
==28531== at 0x485C6A2: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x45118D: main (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== Address 0xae14178 is 8 bytes after a block of size 1,168 alloc'd
==28531== at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==28531== by 0x46695D: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x486D932: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x48660F2: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x4868BE7: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x4868FCF: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x486988B: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x485C6BB: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531== by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==

[...]

valgrind: m_mallocfree.c:305 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 1232, hi = 68.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata. If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away. Please try that before reporting this as a bug.

Does that ring a bell? Is there a chance this problem was fixed in the
meantime?

Thanks in advance,
Ludovic.

L
L
Ludovic Courtès wrote on 29 Jun 2021 17:10
(address . dev-luatex@ntg.nl)(address . 48064@debbugs.gnu.org)
877dicr9k7.fsf@gnu.org
Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (4 lines)
> While investigating luatex crashes in the TeX Live 2020 package of
> GNU Guix¹, we identified the following heap corruption reported by
> Valgrind (this is on GNU/Linux, with glibc 2.33):

This time with debug info for luatex:

Toggle snippet (81 lines)
sh-5.0$ ~ludo/.guix-profile/bin/valgrind --extra-debuginfo-path=/gnu/store/f933bvd6ab6l2lg6xl6k1a6jwvcls0z6-glibc-2.33-debug/lib/debug "/gnu/store/00addvl34y6qj8k9k0bnx9jrgxqwry6q-texlive-bin-20200406/bin/luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsbsy.dtx"
==21562== Memcheck, a memory error detector
==21562== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==21562== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==21562== Command: /gnu/store/00addvl34y6qj8k9k0bnx9jrgxqwry6q-texlive-bin-20200406/bin/luatex -interaction=nonstopmode -output-directory=build &luatex amsbsy.dtx
==21562==
This is LuaTeX, Version 1.12.0 (TeX Live 2020)
restricted system commands enabled.
==21562== Invalid write of size 8
==21562== at 0x485C691: lua_pushlstring (lapi.c:483)
==21562== by 0x568E03: load_hyphenation (texlang.c:306)
==21562== by 0x56B41C: undump_one_language (texlang.c:1259)
==21562== by 0x56B41C: undump_language_data (texlang.c:1272)
==21562== by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562== by 0x4EF0ED: main_body (mainbody.c:530)
==21562== by 0x45118D: main (luatex.c:609)
==21562== Address 0xac0fc30 is 0 bytes after a block of size 1,168 alloc'd
==21562== at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21562== by 0x46695D: my_luaalloc (luastuff.c:115)
==21562== by 0x486D932: luaM_realloc_ (lmem.c:86)
==21562== by 0x48660F2: luaD_reallocstack (ldo.c:182)
==21562== by 0x4868BE7: traversethread (lgc.c:549)
==21562== by 0x4868BE7: propagatemark (lgc.c:588)
==21562== by 0x4868FCF: singlestep (lgc.c:1057)
==21562== by 0x486988B: luaC_step (lgc.c:1137)
==21562== by 0x485C6BB: lua_pushlstring (lapi.c:485)
==21562== by 0x568E03: load_hyphenation (texlang.c:306)
==21562== by 0x56B41C: undump_one_language (texlang.c:1259)
==21562== by 0x56B41C: undump_language_data (texlang.c:1272)
==21562== by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562== by 0x4EF0ED: main_body (mainbody.c:530)
==21562==
==21562== Invalid write of size 4
==21562== at 0x485C6A2: lua_pushlstring (lapi.c:483)
==21562== by 0x568E03: load_hyphenation (texlang.c:306)
==21562== by 0x56B41C: undump_one_language (texlang.c:1259)
==21562== by 0x56B41C: undump_language_data (texlang.c:1272)
==21562== by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562== by 0x4EF0ED: main_body (mainbody.c:530)
==21562== by 0x45118D: main (luatex.c:609)
==21562== Address 0xac0fc38 is 8 bytes after a block of size 1,168 alloc'd
==21562== at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21562== by 0x46695D: my_luaalloc (luastuff.c:115)
==21562== by 0x486D932: luaM_realloc_ (lmem.c:86)
==21562== by 0x48660F2: luaD_reallocstack (ldo.c:182)
==21562== by 0x4868BE7: traversethread (lgc.c:549)
==21562== by 0x4868BE7: propagatemark (lgc.c:588)
==21562== by 0x4868FCF: singlestep (lgc.c:1057)
==21562== by 0x486988B: luaC_step (lgc.c:1137)
==21562== by 0x485C6BB: lua_pushlstring (lapi.c:485)
==21562== by 0x568E03: load_hyphenation (texlang.c:306)
==21562== by 0x56B41C: undump_one_language (texlang.c:1259)
==21562== by 0x56B41C: undump_language_data (texlang.c:1272)
==21562== by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562== by 0x4EF0ED: main_body (mainbody.c:530)
==21562==
==21562== Invalid read of size 16
==21562== at 0x485D269: lua_rawset (lapi.c:809)
==21562== by 0x568E14: load_hyphenation (texlang.c:307)
==21562== by 0x56B41C: undump_one_language (texlang.c:1259)
==21562== by 0x56B41C: undump_language_data (texlang.c:1272)
==21562== by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562== by 0x4EF0ED: main_body (mainbody.c:530)
==21562== by 0x45118D: main (luatex.c:609)
==21562== Address 0xac0fc30 is 0 bytes after a block of size 1,168 alloc'd
==21562== at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21562== by 0x46695D: my_luaalloc (luastuff.c:115)
==21562== by 0x486D932: luaM_realloc_ (lmem.c:86)
==21562== by 0x48660F2: luaD_reallocstack (ldo.c:182)
==21562== by 0x4868BE7: traversethread (lgc.c:549)
==21562== by 0x4868BE7: propagatemark (lgc.c:588)
==21562== by 0x4868FCF: singlestep (lgc.c:1057)
==21562== by 0x486988B: luaC_step (lgc.c:1137)
==21562== by 0x485C6BB: lua_pushlstring (lapi.c:485)
==21562== by 0x568E03: load_hyphenation (texlang.c:306)
==21562== by 0x56B41C: undump_one_language (texlang.c:1259)
==21562== by 0x56B41C: undump_language_data (texlang.c:1272)
==21562== by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562== by 0x4EF0ED: main_body (mainbody.c:530)

Ludo’.
T
T
Thiago Jung Bauermann wrote on 29 Jun 2021 21:59
texlive-* packages fail to build non-deterministically
(address . 48064@debbugs.gnu.org)
3617506.ZDHn73UUgp@popigai
I have bad news and good news. :-)

Unfortunately, TeX Live 2021 still has this bug. I tested version 20210325
(which is the latest on the historic TeX Live FTP site), with subversion
tag texlive-2021.3 (which is the latest tag in the TeX Live repo).

The good news is that I found a simple workaround: use pdftex instead of
luatex to build the affected packages. I am currently building all packages
matching ‘^texlive’ a few times to find the ones needing this workaround.
So far, I found these:

• texlive-amsfonts
• texlive-amscls
• texlive-babel
• texlive-babel-swedish
• texlive-latex-amsmath
• texlive-generic-babel-english
• texlive-latex-cyrillic
• texlive-latex-graphics
• texlive-latex-tools

I will submit a patch to the guix-patches mailing list (with this bug in
Cc:) as soon as the process finishes.
L
L
Ludovic Courtès wrote on 30 Jun 2021 12:05
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)(address . 48064@debbugs.gnu.org)
87zgv7oegg.fsf@gnu.org
Hi Thiago,

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (8 lines)
> I was able to do a build of texlive-amscls with `export MALLOC_CHECK_=2`,
> and a core dump was generated. I managed to get a guix environment with
> debuginfo for texlive-bin, but for some reason it still doesn’t have
> debug info for glibc available. FYI, this was the command line I used:
>
> $ guix environment texlive-amscls --with-debug-info=texlive-bin \
> --ad-hoc valgrind gdb glibc:debug

FWIW I managed to get a Valgrind report for TeX Live on core-updates,
using ‘--extra-debuginfo-path’ to help it a bit:


Not that it helps a whole lot yet!

Ludo’.
L
L
luigi scarso wrote on 30 Jun 2021 13:53
Re: [Dev-luatex] bug#48064: texlive-* packages fail to build non-deterministically
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAG5iGsDpnaRLqU2CvA_sNSJcdTHX44YdiJfEoY1GAc5Qs1XXFw@mail.gmail.com
On Wed, Jun 30, 2021 at 8:20 AM Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (11 lines)
> Hi,
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
> > While investigating luatex crashes in the TeX Live 2020 package of
> > GNU Guix¹, we identified the following heap corruption reported by
> > Valgrind (this is on GNU/Linux, with glibc 2.33):
>
> This time with debug info for luatex:
>

Thank you for the report, I will check asap.


--
luigi
Attachment: file
T
T
Thiago Jung Bauermann wrote on 30 Jun 2021 16:54
Re: bug#48064: texlive-* packages fail to build non-deterministically
4019408.j1JhQaI1tB@popigai
Em quarta-feira, 30 de junho de 2021, às 09:46:01 -03, Thiago Jung
Bauermann escreveu:
Toggle quote (3 lines)
> I will use your command line to get a Valgrind report for TeX Live 2021
> and post it to the dev-luatex mailing list as well.

Unfortunately I’m still having problems with glibc debug info and thus
Valgrind doesn’t work for me.

When I use this command:

$ guix environment texlive-amscls --pure \
--with-debug-info=texlive-bin \
--ad-hoc valgrind glibc:debug gdb

I get luatex linked with this glibc:

$ /bin/which luatex
/gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex
$ ldd /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex | grep libc.so
libc.so.6 => /gnu/store/rhlm1nixi98x09xbyjc4i38gl9xs01f2-glibc-2.33/lib/libc.so.6 (0x00007fec0ac3d000)

But for some reason I get debug info for a different glibc:

$ ls -1 /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/lib/debug/gnu/store/
5ahv6zfv64jm5y7sficll0k2scr7cxvz-glibc-2.33
d9r40z2klf9l25pjk9k71z3dvxrsifzs-glibc-2.33-static

--
Thanks,
Thiago
T
T
Thiago Jung Bauermann wrote on 30 Jun 2021 14:46
(address . 48064@debbugs.gnu.org)
D07DE924-B776-465F-9217-3F9C1D471F49@kolabnow.com
Hello Ludo,

Em 30 de junho de 2021 07:05:03 BRT, "Ludovic Courtès" <ludo@gnu.org> escreveu:
Toggle quote (9 lines)
>Hi Thiago,
>
>FWIW I managed to get a Valgrind report for TeX Live on core-updates,
>using ‘--extra-debuginfo-path’ to help it a bit:
>
> https://issues.guix.gnu.org/48064#6
>
>Not that it helps a whole lot yet!

I saw it, but after I posted my message above. Thank you!

I will use your command line to get a Valgrind report for TeX Live 2021 and post it to the dev-luatex mailing list as well.

--
Enviado de meu dispositivo Android com K-9 mail. Desculpe-me pela brevidade.
L
L
Ludovic Courtès wrote on 1 Jul 2021 15:07
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)(address . 48064@debbugs.gnu.org)
87a6n6mbcg.fsf@gnu.org
Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (27 lines)
> Em quarta-feira, 30 de junho de 2021, às 09:46:01 -03, Thiago Jung
> Bauermann escreveu:
>> I will use your command line to get a Valgrind report for TeX Live 2021
>> and post it to the dev-luatex mailing list as well.
>
> Unfortunately I’m still having problems with glibc debug info and thus
> Valgrind doesn’t work for me.
>
> When I use this command:
>
> $ guix environment texlive-amscls --pure \
> --with-debug-info=texlive-bin \
> --ad-hoc valgrind glibc:debug gdb
>
> I get luatex linked with this glibc:
>
> $ /bin/which luatex
> /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex
> $ ldd /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex | grep libc.so
> libc.so.6 => /gnu/store/rhlm1nixi98x09xbyjc4i38gl9xs01f2-glibc-2.33/lib/libc.so.6 (0x00007fec0ac3d000)
>
> But for some reason I get debug info for a different glibc:
>
> $ ls -1 /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/lib/debug/gnu/store/
> 5ahv6zfv64jm5y7sficll0k2scr7cxvz-glibc-2.33
> d9r40z2klf9l25pjk9k71z3dvxrsifzs-glibc-2.33-static

Yes, you need debug info for (@@ (gnu packages commencement)
glibc-final), which is not what you get when you type “glibc:debug”.

HTH!

Ludo’.
T
T
Thiago Jung Bauermann wrote on 2 Jul 2021 17:11
Re: [Dev-luatex] bug#48064: texlive-* packages fail to build non-deterministically
(name . luigi scarso)(address . luigi.scarso@gmail.com)
2640745.LZQS8YVrSi@popigai
Em quarta-feira, 30 de junho de 2021, às 08:53:41 -03, luigi scarso escreveu:
Toggle quote (11 lines)
> On Wed, Jun 30, 2021 at 8:20 AM Ludovic Courtès <ludo@gnu.org> wrote:
> > Hi,
> >
> > Ludovic Courtès <ludo@gnu.org> skribis:
> > > While investigating luatex crashes in the TeX Live 2020 package of
> > > GNU Guix¹, we identified the following heap corruption reported by
> >
> > > Valgrind (this is on GNU/Linux, with glibc 2.33):
> > This time with debug info for luatex:
> Thank you for the report, I will check asap.

Thanks! I was able to run Valgrind on LuaTeX 1.13.0, which is the latest
one in TeX Live 2021.

The invalid reads and writes don’t happen on every run. I had to re-run the
command 3 or 4 times until I got the result below (which matches our
experience with the build failures in Guix packages)

--
Thanks,
Thiago


$ valgrind --extra-debuginfo-path=/gnu/store/rkhx3pj1qi7fx6pi9p2cg2sb9zn59qmg-profile/lib/debug luatex amsclass.ins
==239904== Memcheck, a memory error detector
==239904== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==239904== Using Valgrind-3.17.0 and LibVEX; rerun with -h for copyright info
==239904== Command: luatex amsclass.ins
==239904==
This is LuaTeX, Version 1.13.0 (TeX Live 2021)
restricted system commands enabled.
==239904== Invalid write of size 8
==239904== at 0x4860691: lua_pushlstring (lapi.c:483)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904== by 0x45118D: main (luatex.c:609)
==239904== Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd
==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904== by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904== by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904== by 0x486CC17: traversethread (lgc.c:549)
==239904== by 0x486CC17: propagatemark (lgc.c:588)
==239904== by 0x486CFFF: singlestep (lgc.c:1057)
==239904== by 0x486D8BB: luaC_step (lgc.c:1137)
==239904== by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 4
==239904== at 0x48606A2: lua_pushlstring (lapi.c:483)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904== by 0x45118D: main (luatex.c:609)
==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904== by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904== by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904== by 0x486CC17: traversethread (lgc.c:549)
==239904== by 0x486CC17: propagatemark (lgc.c:588)
==239904== by 0x486CFFF: singlestep (lgc.c:1057)
==239904== by 0x486D8BB: luaC_step (lgc.c:1137)
==239904== by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 16
==239904== at 0x4861269: lua_rawset (lapi.c:809)
==239904== by 0x56A974: load_hyphenation (texlang.c:307)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904== by 0x45118D: main (luatex.c:609)
==239904== Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd
==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904== by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904== by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904== by 0x486CC17: traversethread (lgc.c:549)
==239904== by 0x486CC17: propagatemark (lgc.c:588)
==239904== by 0x486CFFF: singlestep (lgc.c:1057)
==239904== by 0x486D8BB: luaC_step (lgc.c:1137)
==239904== by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 1
==239904== at 0x486127D: lua_rawset (lapi.c:811)
==239904== by 0x56A974: load_hyphenation (texlang.c:307)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904== by 0x45118D: main (luatex.c:609)
==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904== by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904== by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904== by 0x486CC17: traversethread (lgc.c:549)
==239904== by 0x486CC17: propagatemark (lgc.c:588)
==239904== by 0x486CFFF: singlestep (lgc.c:1057)
==239904== by 0x486D8BB: luaC_step (lgc.c:1137)
==239904== by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 8
==239904== at 0x485F068: auxgetstr (lapi.c:596)
==239904== by 0x463955: check_texconfig_init (luainit.c:1198)
==239904== by 0x4F0507: main_body (mainbody.c:565)
==239904== by 0x45118D: main (luatex.c:609)
==239904== Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd
==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904== by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904== by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904== by 0x486CC17: traversethread (lgc.c:549)
==239904== by 0x486CC17: propagatemark (lgc.c:588)
==239904== by 0x486CFFF: singlestep (lgc.c:1057)
==239904== by 0x486D8BB: luaC_step (lgc.c:1137)
==239904== by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 4
==239904== at 0x485F07A: auxgetstr (lapi.c:596)
==239904== by 0x463955: check_texconfig_init (luainit.c:1198)
==239904== by 0x4F0507: main_body (mainbody.c:565)
==239904== by 0x45118D: main (luatex.c:609)
==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904== by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904== by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904== by 0x486CC17: traversethread (lgc.c:549)
==239904== by 0x486CC17: propagatemark (lgc.c:588)
==239904== by 0x486CFFF: singlestep (lgc.c:1057)
==239904== by 0x486D8BB: luaC_step (lgc.c:1137)
==239904== by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 4
==239904== at 0x4880608: luaV_finishget (lvm.c:176)
==239904== by 0x485F089: auxgetstr (lapi.c:598)
==239904== by 0x463955: check_texconfig_init (luainit.c:1198)
==239904== by 0x4F0507: main_body (mainbody.c:565)
==239904== by 0x45118D: main (luatex.c:609)
==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904== by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904== by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904== by 0x486CC17: traversethread (lgc.c:549)
==239904== by 0x486CC17: propagatemark (lgc.c:588)
==239904== by 0x486CFFF: singlestep (lgc.c:1057)
==239904== by 0x486D8BB: luaC_step (lgc.c:1137)
==239904== by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 4
==239904== at 0x485F092: auxgetstr (lapi.c:601)
==239904== by 0x463955: check_texconfig_init (luainit.c:1198)
==239904== by 0x4F0507: main_body (mainbody.c:565)
==239904== by 0x45118D: main (luatex.c:609)
==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904== by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904== by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904== by 0x486CC17: traversethread (lgc.c:549)
==239904== by 0x486CC17: propagatemark (lgc.c:588)
==239904== by 0x486CFFF: singlestep (lgc.c:1057)
==239904== by 0x486D8BB: luaC_step (lgc.c:1137)
==239904== by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 4
==239904== at 0x485F6D9: lua_type (lapi.c:253)
==239904== by 0x463966: check_texconfig_init (luainit.c:1199)
==239904== by 0x4F0507: main_body (mainbody.c:565)
==239904== by 0x45118D: main (luatex.c:609)
==239904== Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904== by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904== by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904== by 0x486CC17: traversethread (lgc.c:549)
==239904== by 0x486CC17: propagatemark (lgc.c:588)
==239904== by 0x486CFFF: singlestep (lgc.c:1057)
==239904== by 0x486D8BB: luaC_step (lgc.c:1137)
==239904== by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904==
(./amsclass.ins (/gnu/store/rkhx3pj1qi7fx6pi9p2cg2sb9zn59qmg-profile/share/texmf-dist/tex/latex/base/docstrip.tex==239904== Conditional jump or move depends on uninitialised value(s)
==239904== at 0x4647BD: tprint (printing.c:484)
==239904== by 0x4E5E6B: write_out (extensions.c:583)
==239904== by 0x4E62EA: wrapup_leader (extensions.c:1324)
==239904== by 0x4E62EA: do_extension (extensions.c:423)
==239904== by 0x4F4860: main_control (maincontrol.c:1030)
==239904== by 0x4F0537: main_body (mainbody.c:577)
==239904== by 0x45118D: main (luatex.c:609)
==239904==

==239904== Conditional jump or move depends on uninitialised value(s)
==239904== at 0x4647BD: tprint (printing.c:484)
==239904== by 0x4E5DED: write_out (extensions.c:585)
==239904== by 0x4E62EA: wrapup_leader (extensions.c:1324)
==239904== by 0x4E62EA: do_extension (extensions.c:423)
==239904== by 0x4F4860: main_control (maincontrol.c:1030)
==239904== by 0x4F0537: main_body (mainbody.c:577)
==239904== by 0x45118D: main (luatex.c:609)
==239904==
Utility: `docstrip' v2.6a <2020-07-07>
English documentation <2020-07-11>
==239904== Conditional jump or move depends on uninitialised value(s)
==239904== at 0x464523: tprint (printing.c:512)
==239904== by 0x4E5DED: write_out (extensions.c:585)
==239904== by 0x4E62EA: wrapup_leader (extensions.c:1324)
==239904== by 0x4E62EA: do_extension (extensions.c:423)
==239904== by 0x4F4860: main_control (maincontrol.c:1030)
==239904== by 0x4F0537: main_body (mainbody.c:577)
==239904== by 0x45118D: main (luatex.c:609)
==239904==

**********************************************************
* This program converts documented macro-files into fast *
* loadable files by stripping off (nearly) all comments! *
**********************************************************

********************************************************
* No Configuration file found, using default settings. *
********************************************************

)

Generating file(s) amsthm.sty amsart.cls amsbook.cls amsproc.cls

Processing file amsclass.dtx (amsthm) -> amsthm.sty
(amsart,classes) -> amsart.cls
(amsbook,classes) -> amsbook.cls
(amsproc,classes) -> amsproc.cls
File amsclass.dtx ended by \endinput.
Lines processed: 5197
Comments removed: 2926
Comments passed: 21
Codelines passed: 2062

)
warning (pdf backend): no pages of output.
Transcript written on amsclass.log.
==239904== Invalid write of size 8
==239904== at 0x486C013: GCTM (lgc.c:819)
==239904== by 0x486D779: callallpendingfinalizers (lgc.c:862)
==239904== by 0x486D779: luaC_freeallobjects (lgc.c:971)
==239904== by 0x4877A0B: close_state (lstate.c:245)
==239904== by 0x4E33A5: do_final_end (errors.c:257)
==239904== by 0x45118D: main (luatex.c:609)
==239904== Address 0x894aa40 is 16 bytes after a block of size 1,184 alloc'd
==239904== at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904== by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904== by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904== by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904== by 0x486CC17: traversethread (lgc.c:549)
==239904== by 0x486CC17: propagatemark (lgc.c:588)
==239904== by 0x486CFFF: singlestep (lgc.c:1057)
==239904== by 0x486D8BB: luaC_step (lgc.c:1137)
==239904== by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904== by 0x56A963: load_hyphenation (texlang.c:306)
==239904== by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904== by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904== by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904== by 0x4F03DD: main_body (mainbody.c:540)
==239904==

valgrind: m_mallocfree.c:303 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 1248, hi = 102.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata. If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away. Please try that before reporting this as a bug.


host stacktrace:
==239904== at 0x5803F050: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904== by 0x5803F157: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904== by 0x5803F2DE: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904== by 0x58048742: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904== by 0x58037DCB: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904== by 0x58036637: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904== by 0x5803AAB2: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904== by 0x58035988: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904== by 0x100417A3ED: ???
==239904== by 0x1002CB9F2F: ???
==239904== by 0xBF0E: ???

sched status:
running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 239904)
==239904== at 0x486C01B: GCTM (lgc.c:820)
==239904== by 0x486D779: callallpendingfinalizers (lgc.c:862)
==239904== by 0x486D779: luaC_freeallobjects (lgc.c:971)
==239904== by 0x4877A0B: close_state (lstate.c:245)
==239904== by 0x4E33A5: do_final_end (errors.c:257)
==239904== by 0x45118D: main (luatex.c:609)
client stack range: [0x1FFEFB0000 0x1FFF000FFF] client SP: 0x1FFF000130
valgrind stack range: [0x1002BBA000 0x1002CB9FFF] top usage: 9624 of 1048576


Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using. Thanks.

$ echo $?
1
T
T
Thiago Jung Bauermann wrote on 2 Jul 2021 17:17
Re: bug#48064: texlive-* packages fail to build non-deterministically
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 48064@debbugs.gnu.org)
2539205.lm1NgOegKz@popigai
Em quinta-feira, 1 de julho de 2021, às 10:07:27 -03, Ludovic Courtès
escreveu:
Toggle quote (5 lines)
> Yes, you need debug info for (@@ (gnu packages commencement)
> glibc-final), which is not what you get when you type “glibc:debug”.
>
> HTH!

It did, thanks for the tip! For the record, this is the command line that
worked for me:

$ guix environment texlive-amscls --pure \
--with-debug-info=texlive-bin \
--ad-hoc \
-e '(list (@@ (gnu packages commencement) glibc-final) “debug”)’ \
valgrind gdb

Then I pointed valgrind at the /lib/debug directory in the environment’s
profile, as seen in my previous message.

--
Thanks,
Thiago
T
T
Thiago Jung Bauermann wrote on 2 Jul 2021 18:00
[PATCH core-updates] build-system/texlive: Change default format to pdftex
(address . 48064@debbugs.gnu.org)(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
20210702160010.246888-1-bauermann@kolabnow.com
LuaTeX has a bug where sometimes it corrupts the heap and aborts. This
causes the build of texlive packages to fail at random. The problem is

While a fix isn't found, switch the default TeX format (and consequently
also the engine) to pdftex to avoid the issue.

* guix/build-system/texlive.scm (texlive-build): Change default value of
the ‘tex-format’ key parameter to “pdftex”.
---
guix/build-system/texlive.scm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

Hello,

Originally I had a bigger patch which changed packages individually to
build with pdfTeX, because I thought the bug in LuaTeX was triggered only
by specific packages. As I changed more and more packages though, I
realized that any package can trigger the problem and the heap corruption
happens at random.

Therefore, to work around the LuaTeX bug we need to build all packages with
pdfTeX. I tested the patch with this script:

Toggle snippet (38 lines)
#!/bin/bash

set -e

LOG_FILE="$1"
ROUNDS=$2

function verify_package() {
local package="$1"
local log_file="$2"

if ! guix build "$package"; then
echo "failure while building $package" >> "$log_file"
return
fi

echo "success while building $package" >> "$log_file"

if ! guix build --check --rounds=$ROUNDS "$package"; then
echo "failure while checking $package" >> "$log_file"
return
fi

echo "success while checking $package" >> "$log_file"

return
}

guix describe >> "$LOG_FILE"
echo rounds = "$ROUNDS" >> "$LOG_FILE"
echo >> "$LOG_FILE"

for package in $(guix package --list-available='^texlive' | cut -f1)
do
verify_package $package "$LOG_FILE"
done

and called it with `./verify-texlive-packages.sh verify-packages.log 5`.

I see some packages fail during `guix build --check` because they don't
build deterministically for some reason (which is a separate problem), but
I don't see any package failing to build anymore.

Toggle diff (13 lines)
diff --git a/guix/build-system/texlive.scm b/guix/build-system/texlive.scm
index 0efa139fc124..00a36d5862d4 100644
--- a/guix/build-system/texlive.scm
+++ b/guix/build-system/texlive.scm
@@ -128,7 +128,7 @@ level package ID."
(tests? #f)
tex-directory
(build-targets #f)
- (tex-format "luatex")
+ (tex-format "pdftex")
(phases '(@ (guix build texlive-build-system)
%standard-phases))
(outputs '("out"))
L
L
Ludovic Courtès wrote on 5 Jul 2021 10:42
Re: bug#48064: texlive-* packages fail to build non-deterministically
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)(address . 48064@debbugs.gnu.org)
87k0m5w3qo.fsf@gnu.org
Hello!

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (6 lines)
> I have bad news and good news. :-)
>
> Unfortunately, TeX Live 2021 still has this bug. I tested version 20210325
> (which is the latest on the historic TeX Live FTP site), with subversion
> tag texlive-2021.3 (which is the latest tag in the TeX Live repo).

Bah. Still, if you have the whole texlive upgrade, we should apply it!

Toggle quote (15 lines)
> The good news is that I found a simple workaround: use pdftex instead of
> luatex to build the affected packages. I am currently building all packages
> matching ‘^texlive’ a few times to find the ones needing this workaround.
> So far, I found these:
>
> • texlive-amsfonts
> • texlive-amscls
> • texlive-babel
> • texlive-babel-swedish
> • texlive-latex-amsmath
> • texlive-generic-babel-english
> • texlive-latex-cyrillic
> • texlive-latex-graphics
> • texlive-latex-tools

We haven’t heard from dev-luatex yet. I think we should go with this
workaround for now (those build failures are frequently preventing
completing, which is a real bummer). Does using ‘pdftex’ rather than
‘luatex’ have an impact on the output of these packages?

Thanks!

Ludo’.
L
L
Ludovic Courtès wrote on 5 Jul 2021 11:20
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)(address . 48064@debbugs.gnu.org)
871r8dw20b.fsf_-_@gnu.org
Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (10 lines)
> LuaTeX has a bug where sometimes it corrupts the heap and aborts. This
> causes the build of texlive packages to fail at random. The problem is
> being tracked at https://issues.guix.gnu.org/48064.
>
> While a fix isn't found, switch the default TeX format (and consequently
> also the engine) to pdftex to avoid the issue.
>
> * guix/build-system/texlive.scm (texlive-build): Change default value of
> the ‘tex-format’ key parameter to “pdftex”.

Pushed as 04f9f9158da348e8299e9ab90ec389ba81be46b0 with the text above
inlined as a FIXME comment.

On IRC there were concerns about Unicode support, which LuaTeX provides
but pdftex doesn’t (IIUC), but it would seem that the worst that can
happen is that documentation of the packages themselves would be
mangled, which is okay.

Anyway, it’d be ideal to get feedback from the LuaTeX folks!

Thanks,
Ludo’.
T
T
Thiago Jung Bauermann wrote on 5 Jul 2021 19:27
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 48064@debbugs.gnu.org)
2706677.GvGBn1mmWc@popigai
Hi Ludo,

Em segunda-feira, 5 de julho de 2021, às 06:20:20 -03, Ludovic Courtès
escreveu:
Toggle quote (16 lines)
> Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:
> > LuaTeX has a bug where sometimes it corrupts the heap and aborts. This
> > causes the build of texlive packages to fail at random. The problem is
> > being tracked at https://issues.guix.gnu.org/48064.
> >
> > While a fix isn't found, switch the default TeX format (and
> > consequently
> > also the engine) to pdftex to avoid the issue.
> >
> > * guix/build-system/texlive.scm (texlive-build): Change default value
> > of
> > the ‘tex-format’ key parameter to “pdftex”.
>
> Pushed as 04f9f9158da348e8299e9ab90ec389ba81be46b0 with the text above
> inlined as a FIXME comment.

Thank you!

Toggle quote (5 lines)
> On IRC there were concerns about Unicode support, which LuaTeX provides
> but pdftex doesn’t (IIUC), but it would seem that the worst that can
> happen is that documentation of the packages themselves would be
> mangled, which is okay.

I chose pdfTeX for the workaround because it’s the direct “predecessor” to
LuaTeX, so I thought it would behave most similarly to it. But it’s just an
uneducated guess.

Searching around a bit¹²³, XeTeX also has native Unicode support, so we
could also switch to it. Either as the default, or for specific packages
that need it.

NB: I last used TeX more than 15 years ago, and even then just lightly and
sporadically. Don’t trust my judgement on TeX-related issues. :-)

Toggle quote (2 lines)
> Anyway, it’d be ideal to get feedback from the LuaTeX folks!

L
L
luigi scarso wrote on 31 Dec 2021 18:40
Re: [Dev-luatex] bug#48064: texlive-* packages fail to build non-deterministically
(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)
CAG5iGsDQYfYEShOEqLGYGPYc4Lf-S-qsCPOV-ZZ0xKnVK+QpFg@mail.gmail.com
On Fri, Jul 2, 2021 at 5:11 PM Thiago Jung Bauermann <bauermann@kolabnow.com>
wrote:

Toggle quote (26 lines)
> Em quarta-feira, 30 de junho de 2021, às 08:53:41 -03, luigi scarso
> escreveu:
> > On Wed, Jun 30, 2021 at 8:20 AM Ludovic Courtès <ludo@gnu.org> wrote:
> > > Hi,
> > >
> > > Ludovic Courtès <ludo@gnu.org> skribis:
> > > > While investigating luatex crashes in the TeX Live 2020 package of
> > > > GNU Guix¹, we identified the following heap corruption reported by
> > >
> > > > Valgrind (this is on GNU/Linux, with glibc 2.33):
> > > This time with debug info for luatex:
> > Thank you for the report, I will check asap.
>
> Thanks! I was able to run Valgrind on LuaTeX 1.13.0, which is the latest
> one in TeX Live 2021.
>
> The invalid reads and writes don’t happen on every run. I had to re-run
> the
> command 3 or 4 times until I got the result below (which matches our
> experience with the build failures in Guix packages)
>
> --
> Thanks,
> Thiago
>

Until now I was not able to reproduce the issue.

--
luigi
Attachment: file
?