texlive-* packages fail to build non-deterministically

Open

Details

6 participants

Thiago Jung Bauermann
Ludovic Courtès
Ludovic Courtès
luigi scarso
Maxim Cournoyer
Thiago Jung Bauermann

Owner: unassigned

Submitted by: Ludovic Courtès

Severity: important

Merged with

Ludovic Courtès wrote on 27 Apr 2021 17:39

Recipients:(address . bug-guix@gnu.org)

Message-ID:871rav68km.fsf@inria.fr

Some texlive- packages fail to build non-deterministically. This one is

on powerpc64le-linux from current master (commit

d904abe0768293b2322dbf355b6e41d94e769d78), but ISTR we saw the same kind

of problem on other architectures before:

Toggle snippet (37 lines)Processing file fileerr.dtx (help) -> h.tex
                            (scroll) -> s.tex
                            (edit) -> e.tex
                            (batch) -> q.tex
                            (run) -> r.tex
                            (exit) -> x.tex
File fileerr.dtx ended by \endinput.
Lines  processed: 119
Comments removed: 100
Comments  passed: 0
Codelines passed: 10

***********************************************************
*
* To finish the installation you have to move the following
* files into a directory searched by TeX:
*
* All the files with extension `.sty' and `.tex'
* Note there also may be a file .tex which is `invisible'
* on some operating systems.
*
* To produce the documentation run the .dtx files through LaTeX.
*
* Happy TeXing
***********************************************************

* Finally trying to make a file `.tex'.
* (Placed at the end of this run, as this
* may fail on some operating systems.)

Generating file(s) .tex

realloc(): invalid next size
command "luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "tools.ins" failed with signal 6
builder for `/gnu/store/k9v4w4vc9q22yrrl5ggmpcymidwcbamf-texlive-latex-tools-51265.drv' failed with exit code 1

Notice the “realloc” message (from glibc) suggesting heap corruption.

Often, simply retrying yields a successful build.

Ludo’.

Ludovic Courtès wrote on 21 May 2021 23:37

control message for bug #48064

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87cztj7oq6.fsf@gnu.org

merge 48064 48455

quit

Thiago Jung Bauermann wrote on 6 Jun 2021 03:36

texlive-* packages fail to build non-deterministically

Recipients:(address . 48064@debbugs.gnu.org)

Message-ID:5446817.TUF6V9tvpM@popigai

I just tried building core-updates after the upgrade to TeX Live 2020
(core-updates commit f72a125) and texlive-amscls failed with a slightly
different error coming from glibc, but still indicative of heap corruption:
“corrupted size vs. prev_size”

Full build output attached, and relevant snippet below:

```
This is LuaTeX, Version 1.12.0 (TeX Live 2020) 
 restricted system commands enabled.
(./amsdtx.ins (/gnu/store/6w056ryry98gw1h6hl1xnh725l7k5jg9-texlive-latex-base-54632/share/texmf-dist/tex/latex/base/docstrip.tex
Utility: `docstrip' 2.5g <2018/05/03>
English documentation    <2018/05/03>

**********************************************************
* This program converts documented macro-files into fast *
* loadable files by stripping off (nearly) all comments! *
**********************************************************

********************************************************
* No Configuration file found, using default settings. *
********************************************************

)

Generating file(s) amsdtx.cls amsldoc.cls 

Processing file amsdtx.dtx (amsdtx) -> amsdtx.cls
                           (amsldoc) -> amsldoc.cls
File amsdtx.dtx ended by \endinput.
Lines  processed: 1196
Comments removed: 573 
Comments  passed: 1
Codelines passed: 584 

)
warning  (pdf backend): no pages of output.
Transcript written on amsdtx.log.
corrupted size vs. prev_size
error: in phase 'build': uncaught exception:
%exception #<&invoke-error program: "luatex" arguments: ("-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsdtx.ins") exit-status: #f term-signal: 6 stop-signal: #f> 
phase `build' failed after 6.5 seconds
command "luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsdtx.ins" failed with signal 6
```

I’ll keep trying to run valgrind on it to see where the corruption happens.
-- 
Thanks,
Thiago

Attachment: kl1j9qpdpplhavyjwpk1sgrafcl5kx-texlive-amscls-54632.drv

Thiago Jung Bauermann wrote on 9 Jun 2021 02:28

Recipients:(address . 48064@debbugs.gnu.org)

Message-ID:2686819.PDXOTiNPFx@popigai

Hello,

I was able to do a build of texlive-amscls with `export MALLOC_CHECK_=2`, 
and a core dump was generated. I managed to get a guix environment with 
debuginfo for texlive-bin, but for some reason it still doesn’t have 
debug info for glibc available. FYI, this was the command line I used:

$ guix environment texlive-amscls --with-debug-info=texlive-bin \
	--ad-hoc valgrind gdb glibc:debug

Valgrind isn’t working because of the lack of glibc debug info, but I was 
able to get the backtrace below from the core file, using GDB. I am yet to
analyse it and see if it provides any clue:

Core was generated by `luatex amsclass.ins'.
Program terminated with signal SIGABRT, Aborted.
#0  0x00007f50ad3d5a7a in raise () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
(gdb) backtrace
#0  0x00007f50ad3d5a7a in raise () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#1  0x00007f50ad3c0523 in abort () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#2  0x00007f50ad415d28 in ?? () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#3  0x00007f50ad41d54a in ?? () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#4  0x00007f50ad421bd7 in ?? () from /gnu/store/vp1q2ymbfzr52zxqs5z3r3nd0yprbg7h-glibc-2.33/lib/libc.so.6
#5  0x0000000000486c7e in my_luaalloc (ud=<optimized out>, ptr=<optimized out>, osize=1104, nsize=<optimized out>)
    at ../../../texlive-20190410-source/texk/web2c/luatexdir/lua/luastuff.c:115
#6  0x00007f50ad754b81 in luaM_realloc_ (L=L@entry=0x12212a8, block=block@entry=0x1316cc0,
    osize=osize@entry=1104, nsize=nsize@entry=2208)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lmem.c:86
#7  0x00007f50ad74d8b3 in luaD_reallocstack (L=0x12212a8, newsize=138)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:182
#8  0x00007f50ad74e121 in luaD_precall (L=L@entry=0x12212a8, func=<optimized out>, nresults=0)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:424
#9  0x00007f50ad74e393 in luaD_precall (nresults=<optimized out>, func=<optimized out>, L=0x12212a8)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:413
#10 luaD_call (L=L@entry=0x12212a8, func=<optimized out>, nResults=<optimized out>)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:498
#11 0x00007f50ad74e3f1 in luaD_callnoyield (L=0x12212a8, func=<optimized out>, nResults=<optimized out>)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:509
#12 0x00007f50ad74d83f in luaD_rawrunprotected (L=L@entry=0x12212a8, f=f@entry=0x7f50ad74f6d0 <dothecall>,
    ud=ud@entry=0x0) at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:142
#13 0x00007f50ad74e6fb in luaD_pcall (L=L@entry=0x12212a8, func=func@entry=0x7f50ad74f6d0 <dothecall>,
    u=u@entry=0x0, old_top=1200, ef=ef@entry=0)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/ldo.c:729
#14 0x00007f50ad74f5dd in GCTM (L=L@entry=0x12212a8, propagateerrors=propagateerrors@entry=0)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lgc.c:823
#15 0x00007f50ad750eca in callallpendingfinalizers (L=<optimized out>)
--Type <RET> for more, q to quit, c to continue without paging--
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lgc.c:862
#16 luaC_freeallobjects (L=L@entry=0x12212a8)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lgc.c:971
#17 0x00007f50ad75a93e in close_state (L=0x12212a8)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lstate.c:245
#18 0x00007f50ad75ae20 in lua_close (L=<optimized out>)
    at ../../../texlive-20190410-source/libs/lua53/lua53-src/src/lstate.c:344
#19 0x00000000004999a6 in do_final_end ()
    at ../../../texlive-20190410-source/texk/web2c/luatexdir/tex/errors.c:257
#20 0x000000000044ea9e in main (ac=<optimized out>, av=<optimized out>)
    at ../../../texlive-20190410-source/texk/web2c/luatexdir/luatex.c:609
(gdb)
-- 
Thanks,
Thiago

Ludovic Courtès wrote on 29 Jun 2021 12:42

control message for bug #48064

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87zgv9rly7.fsf@gnu.org

severity 48064 important

quit

Ludovic Courtès wrote on 29 Jun 2021 16:02

Re: bug#48064: texlive-* packages fail to build non-deterministically

Recipients:(address . dev-luatex@ntg.nl)(address . 48064@debbugs.gnu.org)

Message-ID:874kdgsr9h.fsf@gnu.org

Hello,

While investigating luatex crashes in the TeX Live 2020 package of

GNU�Guix¹, we identified the following heap corruption reported by

Valgrind (this is on GNU/Linux, with glibc 2.33):

Toggle snippet (61 lines)sh-5.0$  ~ludo/.guix-profile/bin/valgrind --extra-debuginfo-path=/gnu/store/f933bvd6ab6l2lg6xl6k1a6jwvcls0z6-glibc-2.33-debug/lib/debug "luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsbsy.dtx"
==28531== Memcheck, a memory error detector
==28531== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==28531== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==28531== Command: luatex -interaction=nonstopmode -output-directory=build &luatex amsbsy.dtx
==28531== 
This is LuaTeX, Version 1.12.0 (TeX Live 2020) 
 restricted system commands enabled.
==28531== Invalid write of size 8
==28531==    at 0x485C691: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x45118D: main (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==  Address 0xae14170 is 0 bytes after a block of size 1,168 alloc'd
==28531==    at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==28531==    by 0x46695D: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x486D932: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x48660F2: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x4868BE7: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x4868FCF: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x486988B: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x485C6BB: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== 
==28531== Invalid write of size 4
==28531==    at 0x485C6A2: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x45118D: main (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==  Address 0xae14178 is 8 bytes after a block of size 1,168 alloc'd
==28531==    at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==28531==    by 0x46695D: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x486D932: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x48660F2: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x4868BE7: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x4868FCF: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x486988B: ??? (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x485C6BB: lua_pushlstring (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/lib/libtexlua53.so.5.3.5)
==28531==    by 0x568E03: load_hyphenation (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x56B41C: undump_language_data (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4DFB9F: load_fmt_file (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531==    by 0x4EF0ED: main_body (in /gnu/store/w20xxg8p0wksbrxxvj3y46fngvr3954w-texlive-bin-20200406/bin/luatex)
==28531== 

[...]

valgrind: m_mallocfree.c:305 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 1232, hi = 68.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata.  If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away.  Please try that before reporting this as a bug.

Does that ring a bell? Is there a chance this problem was fixed in the

meantime?

Thanks in advance,

Ludovic.

¹ https://issues.guix.gnu.org/48064

Ludovic Courtès wrote on 29 Jun 2021 17:10

Recipients:(address . dev-luatex@ntg.nl)(address . 48064@debbugs.gnu.org)

Message-ID:877dicr9k7.fsf@gnu.org

Hi,

Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (4 lines)

> While investigating luatex crashes in the TeX Live 2020 package of

> GNU�Guix¹, we identified the following heap corruption reported by

> Valgrind (this is on GNU/Linux, with glibc 2.33):

This time with debug info for luatex:

Toggle snippet (81 lines)sh-5.0$  ~ludo/.guix-profile/bin/valgrind --extra-debuginfo-path=/gnu/store/f933bvd6ab6l2lg6xl6k1a6jwvcls0z6-glibc-2.33-debug/lib/debug "/gnu/store/00addvl34y6qj8k9k0bnx9jrgxqwry6q-texlive-bin-20200406/bin/luatex" "-interaction=nonstopmode" "-output-directory=build" "&luatex" "amsbsy.dtx"
==21562== Memcheck, a memory error detector
==21562== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==21562== Using Valgrind-3.16.1 and LibVEX; rerun with -h for copyright info
==21562== Command: /gnu/store/00addvl34y6qj8k9k0bnx9jrgxqwry6q-texlive-bin-20200406/bin/luatex -interaction=nonstopmode -output-directory=build &luatex amsbsy.dtx
==21562== 
This is LuaTeX, Version 1.12.0 (TeX Live 2020) 
 restricted system commands enabled.
==21562== Invalid write of size 8
==21562==    at 0x485C691: lua_pushlstring (lapi.c:483)
==21562==    by 0x568E03: load_hyphenation (texlang.c:306)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
==21562==    by 0x45118D: main (luatex.c:609)
==21562==  Address 0xac0fc30 is 0 bytes after a block of size 1,168 alloc'd
==21562==    at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21562==    by 0x46695D: my_luaalloc (luastuff.c:115)
==21562==    by 0x486D932: luaM_realloc_ (lmem.c:86)
==21562==    by 0x48660F2: luaD_reallocstack (ldo.c:182)
==21562==    by 0x4868BE7: traversethread (lgc.c:549)
==21562==    by 0x4868BE7: propagatemark (lgc.c:588)
==21562==    by 0x4868FCF: singlestep (lgc.c:1057)
==21562==    by 0x486988B: luaC_step (lgc.c:1137)
==21562==    by 0x485C6BB: lua_pushlstring (lapi.c:485)
==21562==    by 0x568E03: load_hyphenation (texlang.c:306)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
==21562== 
==21562== Invalid write of size 4
==21562==    at 0x485C6A2: lua_pushlstring (lapi.c:483)
==21562==    by 0x568E03: load_hyphenation (texlang.c:306)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
==21562==    by 0x45118D: main (luatex.c:609)
==21562==  Address 0xac0fc38 is 8 bytes after a block of size 1,168 alloc'd
==21562==    at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21562==    by 0x46695D: my_luaalloc (luastuff.c:115)
==21562==    by 0x486D932: luaM_realloc_ (lmem.c:86)
==21562==    by 0x48660F2: luaD_reallocstack (ldo.c:182)
==21562==    by 0x4868BE7: traversethread (lgc.c:549)
==21562==    by 0x4868BE7: propagatemark (lgc.c:588)
==21562==    by 0x4868FCF: singlestep (lgc.c:1057)
==21562==    by 0x486988B: luaC_step (lgc.c:1137)
==21562==    by 0x485C6BB: lua_pushlstring (lapi.c:485)
==21562==    by 0x568E03: load_hyphenation (texlang.c:306)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
==21562== 
==21562== Invalid read of size 16
==21562==    at 0x485D269: lua_rawset (lapi.c:809)
==21562==    by 0x568E14: load_hyphenation (texlang.c:307)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)
==21562==    by 0x45118D: main (luatex.c:609)
==21562==  Address 0xac0fc30 is 0 bytes after a block of size 1,168 alloc'd
==21562==    at 0x483EBE1: realloc (in /gnu/store/jlmh0jbgr6zn4iyvhfbvxps8pykzj5ry-valgrind-3.16.1/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==21562==    by 0x46695D: my_luaalloc (luastuff.c:115)
==21562==    by 0x486D932: luaM_realloc_ (lmem.c:86)
==21562==    by 0x48660F2: luaD_reallocstack (ldo.c:182)
==21562==    by 0x4868BE7: traversethread (lgc.c:549)
==21562==    by 0x4868BE7: propagatemark (lgc.c:588)
==21562==    by 0x4868FCF: singlestep (lgc.c:1057)
==21562==    by 0x486988B: luaC_step (lgc.c:1137)
==21562==    by 0x485C6BB: lua_pushlstring (lapi.c:485)
==21562==    by 0x568E03: load_hyphenation (texlang.c:306)
==21562==    by 0x56B41C: undump_one_language (texlang.c:1259)
==21562==    by 0x56B41C: undump_language_data (texlang.c:1272)
==21562==    by 0x4DFB9F: load_fmt_file (dumpdata.c:520)
==21562==    by 0x4EF0ED: main_body (mainbody.c:530)

Ludo’.

Thiago Jung Bauermann wrote on 29 Jun 2021 21:59

texlive-* packages fail to build non-deterministically

Recipients:(address . 48064@debbugs.gnu.org)

Message-ID:3617506.ZDHn73UUgp@popigai

I have bad news and good news. :-)

Unfortunately, TeX Live 2021 still has this bug. I tested version 20210325 
(which is the latest on the historic TeX Live FTP site), with subversion 
tag texlive-2021.3 (which is the latest tag in the TeX Live repo).

The good news is that I found a simple workaround: use pdftex instead of 
luatex to build the affected packages. I am currently building all packages 
matching ‘^texlive’ a few times to find the ones needing this workaround.
So far, I found these:

• texlive-amsfonts
• texlive-amscls
• texlive-babel
• texlive-babel-swedish
• texlive-latex-amsmath
• texlive-generic-babel-english
• texlive-latex-cyrillic
• texlive-latex-graphics
• texlive-latex-tools

I will submit a patch to the guix-patches mailing list (with this bug in 
Cc:) as soon as the process finishes.

Ludovic Courtès wrote on 30 Jun 2021 12:05

Recipients:(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)(address . 48064@debbugs.gnu.org)

Message-ID:87zgv7oegg.fsf@gnu.org

Hi Thiago,

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (8 lines)

> I was able to do a build of texlive-amscls with `export MALLOC_CHECK_=2`,

> and a core dump was generated. I managed to get a guix environment with

> debuginfo for texlive-bin, but for some reason it still doesn’t have

> debug info for glibc available. FYI, this was the command line I used:

> $ guix environment texlive-amscls --with-debug-info=texlive-bin \

> --ad-hoc valgrind gdb glibc:debug

FWIW I managed to get a Valgrind report for TeX Live on core-updates,

using ‘--extra-debuginfo-path’ to help it a bit:

https://issues.guix.gnu.org/48064#6

Not that it helps a whole lot yet!

Ludo’.

luigi scarso wrote on 30 Jun 2021 13:53

Re: [Dev-luatex] bug#48064: texlive-* packages fail to build non-deterministically

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)

Message-ID:CAG5iGsDpnaRLqU2CvA_sNSJcdTHX44YdiJfEoY1GAc5Qs1XXFw@mail.gmail.com

On Wed, Jun 30, 2021 at 8:20 AM Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (11 lines)> Hi,
>
> Ludovic Courtès <ludo@gnu.org> skribis:
>
> > While investigating luatex crashes in the TeX Live 2020 package of
> > GNU Guix¹, we identified the following heap corruption reported by
> > Valgrind (this is on GNU/Linux, with glibc 2.33):
>
> This time with debug info for luatex:
>

Thank you for the report, I will check asap.

luigi

Attachment: file

Thiago Jung Bauermann wrote on 30 Jun 2021 16:54

Re: bug#48064: texlive-* packages fail to build non-deterministically

Recipients:

Message-ID:4019408.j1JhQaI1tB@popigai

Em quarta-feira, 30 de junho de 2021, às 09:46:01 -03, Thiago Jung

Bauermann escreveu:

Toggle quote (3 lines)

> I will use your command line to get a Valgrind report for TeX Live 2021

> and post it to the dev-luatex mailing list as well.

Unfortunately I’m still having problems with glibc debug info and thus

Valgrind doesn’t work for me.

When I use this command:

$ guix environment texlive-amscls --pure \

--with-debug-info=texlive-bin \

--ad-hoc valgrind glibc:debug gdb

I get luatex linked with this glibc:

$ /bin/which luatex

/gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex

$ ldd /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex | grep libc.so

libc.so.6 => /gnu/store/rhlm1nixi98x09xbyjc4i38gl9xs01f2-glibc-2.33/lib/libc.so.6 (0x00007fec0ac3d000)

But for some reason I get debug info for a different glibc:

$ ls -1 /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/lib/debug/gnu/store/

5ahv6zfv64jm5y7sficll0k2scr7cxvz-glibc-2.33

d9r40z2klf9l25pjk9k71z3dvxrsifzs-glibc-2.33-static

Thanks,

Thiago

Thiago Jung Bauermann wrote on 30 Jun 2021 14:46

Recipients:(address . 48064@debbugs.gnu.org)

Message-ID:D07DE924-B776-465F-9217-3F9C1D471F49@kolabnow.com

Hello Ludo,

Em 30 de junho de 2021 07:05:03 BRT, "Ludovic Courtès" <ludo@gnu.org> escreveu:

Toggle quote (9 lines)

>Hi Thiago,

>FWIW I managed to get a Valgrind report for TeX Live on core-updates,

>using ‘--extra-debuginfo-path’ to help it a bit:

> https://issues.guix.gnu.org/48064#6

>Not that it helps a whole lot yet!

I saw it, but after I posted my message above. Thank you!

I will use your command line to get a Valgrind report for TeX Live 2021 and post it to the dev-luatex mailing list as well.

Enviado de meu dispositivo Android com K-9 mail. Desculpe-me pela brevidade.

Ludovic Courtès wrote on 1 Jul 2021 15:07

Recipients:(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)(address . 48064@debbugs.gnu.org)

Message-ID:87a6n6mbcg.fsf@gnu.org

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (27 lines)> Em quarta-feira, 30 de junho de 2021, às 09:46:01 -03, Thiago Jung
> Bauermann escreveu:
>> I will use your command line to get a Valgrind report for TeX Live 2021
>> and post it to the dev-luatex mailing list as well.
>
> Unfortunately I’m still having problems with glibc debug info and thus
> Valgrind doesn’t work for me.
>
> When I use this command:
>
> $ guix environment texlive-amscls --pure	\
> 	--with-debug-info=texlive-bin	\
> 	--ad-hoc valgrind glibc:debug gdb 
>
> I get luatex linked with this glibc:
>
> $ /bin/which luatex
> /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex
> $ ldd /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/bin/luatex | grep libc.so
>         libc.so.6 => /gnu/store/rhlm1nixi98x09xbyjc4i38gl9xs01f2-glibc-2.33/lib/libc.so.6 (0x00007fec0ac3d000)
>
> But for some reason I get debug info for a different glibc:
>
> $ ls -1 /gnu/store/s44lh4hqbklplqm3cpaih3rxsv8i1dx1-profile/lib/debug/gnu/store/
> 5ahv6zfv64jm5y7sficll0k2scr7cxvz-glibc-2.33
> d9r40z2klf9l25pjk9k71z3dvxrsifzs-glibc-2.33-static

Yes, you need debug info for (@@ (gnu packages commencement)

glibc-final), which is not what you get when you type “glibc:debug”.

HTH!

Ludo’.

Thiago Jung Bauermann wrote on 2 Jul 2021 17:11

Re: [Dev-luatex] bug#48064: texlive-* packages fail to build non-deterministically

Recipients:(name . luigi scarso)(address . luigi.scarso@gmail.com)

Message-ID:2640745.LZQS8YVrSi@popigai

Em quarta-feira, 30 de junho de 2021, às 08:53:41 -03, luigi scarso escreveu:

Toggle quote (11 lines)> On Wed, Jun 30, 2021 at 8:20 AM Ludovic Courtès <ludo@gnu.org> wrote:
> > Hi,
> > 
> > Ludovic Courtès <ludo@gnu.org> skribis:
> > > While investigating luatex crashes in the TeX Live 2020 package of
> > > GNU Guix¹, we identified the following heap corruption reported by
> > 
> > > Valgrind (this is on GNU/Linux, with glibc 2.33):
> > This time with debug info for luatex:
> Thank you for the report, I will check asap.

Thanks! I was able to run Valgrind on LuaTeX 1.13.0, which is the latest 
one in TeX Live 2021.

The invalid reads and writes don’t happen on every run. I had to re-run the 
command 3 or 4 times until I got the result below (which matches our 
experience with the build failures in Guix packages)

-- 
Thanks,
Thiago


$ valgrind --extra-debuginfo-path=/gnu/store/rkhx3pj1qi7fx6pi9p2cg2sb9zn59qmg-profile/lib/debug luatex amsclass.ins
==239904== Memcheck, a memory error detector
==239904== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==239904== Using Valgrind-3.17.0 and LibVEX; rerun with -h for copyright info
==239904== Command: luatex amsclass.ins
==239904==
This is LuaTeX, Version 1.13.0 (TeX Live 2021)
 restricted system commands enabled.
==239904== Invalid write of size 8
==239904==    at 0x4860691: lua_pushlstring (lapi.c:483)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 4
==239904==    at 0x48606A2: lua_pushlstring (lapi.c:483)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 16
==239904==    at 0x4861269: lua_rawset (lapi.c:809)
==239904==    by 0x56A974: load_hyphenation (texlang.c:307)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 1
==239904==    at 0x486127D: lua_rawset (lapi.c:811)
==239904==    by 0x56A974: load_hyphenation (texlang.c:307)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 8
==239904==    at 0x485F068: auxgetstr (lapi.c:596)
==239904==    by 0x463955: check_texconfig_init (luainit.c:1198)
==239904==    by 0x4F0507: main_body (mainbody.c:565)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa30 is 0 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 4
==239904==    at 0x485F07A: auxgetstr (lapi.c:596)
==239904==    by 0x463955: check_texconfig_init (luainit.c:1198)
==239904==    by 0x4F0507: main_body (mainbody.c:565)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid write of size 4
==239904==    at 0x4880608: luaV_finishget (lvm.c:176)
==239904==    by 0x485F089: auxgetstr (lapi.c:598)
==239904==    by 0x463955: check_texconfig_init (luainit.c:1198)
==239904==    by 0x4F0507: main_body (mainbody.c:565)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 4
==239904==    at 0x485F092: auxgetstr (lapi.c:601)
==239904==    by 0x463955: check_texconfig_init (luainit.c:1198)
==239904==    by 0x4F0507: main_body (mainbody.c:565)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
==239904== Invalid read of size 4
==239904==    at 0x485F6D9: lua_type (lapi.c:253)
==239904==    by 0x463966: check_texconfig_init (luainit.c:1199)
==239904==    by 0x4F0507: main_body (mainbody.c:565)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa38 is 8 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==
(./amsclass.ins (/gnu/store/rkhx3pj1qi7fx6pi9p2cg2sb9zn59qmg-profile/share/texmf-dist/tex/latex/base/docstrip.tex==239904== Conditional jump or move depends on uninitialised value(s)
==239904==    at 0x4647BD: tprint (printing.c:484)
==239904==    by 0x4E5E6B: write_out (extensions.c:583)
==239904==    by 0x4E62EA: wrapup_leader (extensions.c:1324)
==239904==    by 0x4E62EA: do_extension (extensions.c:423)
==239904==    by 0x4F4860: main_control (maincontrol.c:1030)
==239904==    by 0x4F0537: main_body (mainbody.c:577)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==

==239904== Conditional jump or move depends on uninitialised value(s)
==239904==    at 0x4647BD: tprint (printing.c:484)
==239904==    by 0x4E5DED: write_out (extensions.c:585)
==239904==    by 0x4E62EA: wrapup_leader (extensions.c:1324)
==239904==    by 0x4E62EA: do_extension (extensions.c:423)
==239904==    by 0x4F4860: main_control (maincontrol.c:1030)
==239904==    by 0x4F0537: main_body (mainbody.c:577)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==
Utility: `docstrip' v2.6a <2020-07-07>
English documentation    <2020-07-11>
==239904== Conditional jump or move depends on uninitialised value(s)
==239904==    at 0x464523: tprint (printing.c:512)
==239904==    by 0x4E5DED: write_out (extensions.c:585)
==239904==    by 0x4E62EA: wrapup_leader (extensions.c:1324)
==239904==    by 0x4E62EA: do_extension (extensions.c:423)
==239904==    by 0x4F4860: main_control (maincontrol.c:1030)
==239904==    by 0x4F0537: main_body (mainbody.c:577)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==

**********************************************************
* This program converts documented macro-files into fast *
* loadable files by stripping off (nearly) all comments! *
**********************************************************

********************************************************
* No Configuration file found, using default settings. *
********************************************************

)

Generating file(s) amsthm.sty amsart.cls amsbook.cls amsproc.cls

Processing file amsclass.dtx (amsthm) -> amsthm.sty
                             (amsart,classes) -> amsart.cls
                             (amsbook,classes) -> amsbook.cls
                             (amsproc,classes) -> amsproc.cls
File amsclass.dtx ended by \endinput.
Lines  processed: 5197
Comments removed: 2926
Comments  passed: 21
Codelines passed: 2062

)
warning  (pdf backend): no pages of output.
Transcript written on amsclass.log.
==239904== Invalid write of size 8
==239904==    at 0x486C013: GCTM (lgc.c:819)
==239904==    by 0x486D779: callallpendingfinalizers (lgc.c:862)
==239904==    by 0x486D779: luaC_freeallobjects (lgc.c:971)
==239904==    by 0x4877A0B: close_state (lstate.c:245)
==239904==    by 0x4E33A5: do_final_end (errors.c:257)
==239904==    by 0x45118D: main (luatex.c:609)
==239904==  Address 0x894aa40 is 16 bytes after a block of size 1,184 alloc'd
==239904==    at 0x484242B: realloc (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/vgpreload_memcheck-amd64-linux.so)
==239904==    by 0x466BCD: my_luaalloc (luastuff.c:115)
==239904==    by 0x48719C2: luaM_realloc_ (lmem.c:86)
==239904==    by 0x486A122: luaD_reallocstack (ldo.c:182)
==239904==    by 0x486CC17: traversethread (lgc.c:549)
==239904==    by 0x486CC17: propagatemark (lgc.c:588)
==239904==    by 0x486CFFF: singlestep (lgc.c:1057)
==239904==    by 0x486D8BB: luaC_step (lgc.c:1137)
==239904==    by 0x48606BB: lua_pushlstring (lapi.c:485)
==239904==    by 0x56A963: load_hyphenation (texlang.c:306)
==239904==    by 0x56D0CC: undump_one_language (texlang.c:1277)
==239904==    by 0x56D0CC: undump_language_data (texlang.c:1290)
==239904==    by 0x4E0D7F: load_fmt_file (dumpdata.c:520)
==239904==    by 0x4F03DD: main_body (mainbody.c:540)
==239904==

valgrind: m_mallocfree.c:303 (get_bszB_as_is): Assertion 'bszB_lo == bszB_hi' failed.
valgrind: Heap block lo/hi size mismatch: lo = 1248, hi = 102.
This is probably caused by your program erroneously writing past the
end of a heap block and corrupting heap metadata.  If you fix any
invalid writes reported by Memcheck, this assertion failure will
probably go away.  Please try that before reporting this as a bug.


host stacktrace:
==239904==    at 0x5803F050: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x5803F157: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x5803F2DE: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x58048742: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x58037DCB: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x58036637: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x5803AAB2: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x58035988: ??? (in /gnu/store/a4xjjppiw7x0vgd2jimmzssj9i22jf5f-valgrind-3.17.0/libexec/valgrind/memcheck-amd64-linux)
==239904==    by 0x100417A3ED: ???
==239904==    by 0x1002CB9F2F: ???
==239904==    by 0xBF0E: ???

sched status:
  running_tid=1

Thread 1: status = VgTs_Runnable (lwpid 239904)
==239904==    at 0x486C01B: GCTM (lgc.c:820)
==239904==    by 0x486D779: callallpendingfinalizers (lgc.c:862)
==239904==    by 0x486D779: luaC_freeallobjects (lgc.c:971)
==239904==    by 0x4877A0B: close_state (lstate.c:245)
==239904==    by 0x4E33A5: do_final_end (errors.c:257)
==239904==    by 0x45118D: main (luatex.c:609)
client stack range: [0x1FFEFB0000 0x1FFF000FFF] client SP: 0x1FFF000130
valgrind stack range: [0x1002BBA000 0x1002CB9FFF] top usage: 9624 of 1048576


Note: see also the FAQ in the source distribution.
It contains workarounds to several common problems.
In particular, if Valgrind aborted or crashed after
identifying problems in your program, there's a good chance
that fixing those problems will prevent Valgrind aborting or
crashing, especially if it happened in m_mallocfree.c.

If that doesn't help, please report this bug to: www.valgrind.org

In the bug report, send all the above text, the valgrind
version, and what OS and version you are using.  Thanks.

$ echo $?
1

Thiago Jung Bauermann wrote on 2 Jul 2021 17:17

Re: bug#48064: texlive-* packages fail to build non-deterministically

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 48064@debbugs.gnu.org)

Message-ID:2539205.lm1NgOegKz@popigai

Em quinta-feira, 1 de julho de 2021, às 10:07:27 -03, Ludovic Courtès

escreveu:

Toggle quote (5 lines)

> Yes, you need debug info for (@@ (gnu packages commencement)

> glibc-final), which is not what you get when you type “glibc:debug”.

> HTH!

It did, thanks for the tip! For the record, this is the command line that

worked for me:

$ guix environment texlive-amscls --pure \

--with-debug-info=texlive-bin \

--ad-hoc \

-e '(list (@@ (gnu packages commencement) glibc-final) “debug”)’ \

valgrind gdb

Then I pointed valgrind at the /lib/debug directory in the environment’s

profile, as seen in my previous message.

Thanks,

Thiago

Thiago Jung Bauermann wrote on 2 Jul 2021 18:00

[PATCH core-updates] build-system/texlive: Change default format to pdftex

Recipients:(address . 48064@debbugs.gnu.org)(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)

Message-ID:20210702160010.246888-1-bauermann@kolabnow.com

LuaTeX has a bug where sometimes it corrupts the heap and aborts. This

causes the build of texlive packages to fail at random. The problem is

being tracked at https://issues.guix.gnu.org/48064.

While a fix isn't found, switch the default TeX format (and consequently

also the engine) to pdftex to avoid the issue.

* guix/build-system/texlive.scm (texlive-build): Change default value of

the ‘tex-format’ key parameter to “pdftex”.

---

guix/build-system/texlive.scm | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)

Hello,

Originally I had a bigger patch which changed packages individually to

build with pdfTeX, because I thought the bug in LuaTeX was triggered only

by specific packages. As I changed more and more packages though, I

realized that any package can trigger the problem and the heap corruption

happens at random.

Therefore, to work around the LuaTeX bug we need to build all packages with

pdfTeX. I tested the patch with this script:

Toggle snippet (38 lines)

#!/bin/bash

set -e

LOG_FILE="$1"

ROUNDS=$2

function verify_package() {

local package="$1"

local log_file="$2"

if ! guix build "$package"; then

echo "failure while building $package" >> "$log_file"

return

echo "success while building $package" >> "$log_file"

if ! guix build --check --rounds=$ROUNDS "$package"; then

echo "failure while checking $package" >> "$log_file"

return

echo "success while checking $package" >> "$log_file"

return

}

guix describe >> "$LOG_FILE"

echo rounds = "$ROUNDS" >> "$LOG_FILE"

echo >> "$LOG_FILE"

for package in $(guix package --list-available='^texlive' | cut -f1)

verify_package $package "$LOG_FILE"

done

and called it with `./verify-texlive-packages.sh verify-packages.log 5`.

I see some packages fail during `guix build --check` because they don't

build deterministically for some reason (which is a separate problem), but

I don't see any package failing to build anymore.

Toggle diff (13 lines)diff --git a/guix/build-system/texlive.scm b/guix/build-system/texlive.scm
index 0efa139fc124..00a36d5862d4 100644
--- a/guix/build-system/texlive.scm
+++ b/guix/build-system/texlive.scm
@@ -128,7 +128,7 @@ level package ID."
                         (tests? #f)
                         tex-directory
                         (build-targets #f)
-                        (tex-format "luatex")
+                        (tex-format "pdftex")
                         (phases '(@ (guix build texlive-build-system)
                                     %standard-phases))
                         (outputs '("out"))

Ludovic Courtès wrote on 5 Jul 2021 10:42

Re: bug#48064: texlive-* packages fail to build non-deterministically

Recipients:(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)(address . 48064@debbugs.gnu.org)

Message-ID:87k0m5w3qo.fsf@gnu.org

Hello!

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (6 lines)

> I have bad news and good news. :-)

> Unfortunately, TeX Live 2021 still has this bug. I tested version 20210325

> (which is the latest on the historic TeX Live FTP site), with subversion

> tag texlive-2021.3 (which is the latest tag in the TeX Live repo).

Bah. Still, if you have the whole texlive upgrade, we should apply it!

Toggle quote (15 lines)> The good news is that I found a simple workaround: use pdftex instead of 
> luatex to build the affected packages. I am currently building all packages 
> matching ‘^texlive’ a few times to find the ones needing this workaround.
> So far, I found these:
>
> • texlive-amsfonts
> • texlive-amscls
> • texlive-babel
> • texlive-babel-swedish
> • texlive-latex-amsmath
> • texlive-generic-babel-english
> • texlive-latex-cyrillic
> • texlive-latex-graphics
> • texlive-latex-tools

We haven’t heard from dev-luatex yet. I think we should go with this

workaround for now (those build failures are frequently preventing

evaluations at https://ci.guix.gnu.org/jobset/core-updates from

completing, which is a real bummer). Does using ‘pdftex’ rather than

‘luatex’ have an impact on the output of these packages?

Thanks!

Ludo’.

Ludovic Courtès wrote on 5 Jul 2021 11:20

Recipients:(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)(address . 48064@debbugs.gnu.org)

Message-ID:871r8dw20b.fsf_-_@gnu.org

Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:

Toggle quote (10 lines)> LuaTeX has a bug where sometimes it corrupts the heap and aborts. This
> causes the build of texlive packages to fail at random. The problem is
> being tracked at https://issues.guix.gnu.org/48064.
>
> While a fix isn't found, switch the default TeX format (and consequently
> also the engine) to pdftex to avoid the issue.
>
> * guix/build-system/texlive.scm (texlive-build): Change default value of
> the ‘tex-format’ key parameter to “pdftex”.

Pushed as 04f9f9158da348e8299e9ab90ec389ba81be46b0 with the text above

inlined as a FIXME comment.

On IRC there were concerns about Unicode support, which LuaTeX provides

but pdftex doesn’t (IIUC), but it would seem that the worst that can

happen is that documentation of the packages themselves would be

mangled, which is okay.

Anyway, it’d be ideal to get feedback from the LuaTeX folks!

Thanks,

Ludo’.

Thiago Jung Bauermann wrote on 5 Jul 2021 19:27

Recipients:(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 48064@debbugs.gnu.org)

Message-ID:2706677.GvGBn1mmWc@popigai

Hi Ludo,

Em segunda-feira, 5 de julho de 2021, às 06:20:20 -03, Ludovic Courtès

escreveu:

Toggle quote (16 lines)> Thiago Jung Bauermann <bauermann@kolabnow.com> skribis:
> > LuaTeX has a bug where sometimes it corrupts the heap and aborts. This
> > causes the build of texlive packages to fail at random. The problem is
> > being tracked at https://issues.guix.gnu.org/48064.
> > 
> > While a fix isn't found, switch the default TeX format (and
> > consequently
> > also the engine) to pdftex to avoid the issue.
> > 
> > * guix/build-system/texlive.scm (texlive-build): Change default value
> > of
> > the ‘tex-format’ key parameter to “pdftex”.
> 
> Pushed as 04f9f9158da348e8299e9ab90ec389ba81be46b0 with the text above
> inlined as a FIXME comment.

Thank you!

Toggle quote (5 lines)

> On IRC there were concerns about Unicode support, which LuaTeX provides

> but pdftex doesn’t (IIUC), but it would seem that the worst that can

> happen is that documentation of the packages themselves would be

> mangled, which is okay.

I chose pdfTeX for the workaround because it’s the direct “predecessor” to

LuaTeX, so I thought it would behave most similarly to it. But it’s just an

uneducated guess.

Searching around a bit¹²³, XeTeX also has native Unicode support, so we

could also switch to it. Either as the default, or for specific packages

that need it.

NB: I last used TeX more than 15 years ago, and even then just lightly and

sporadically. Don’t trust my judgement on TeX-related issues. :-)

Toggle quote (2 lines)

> Anyway, it’d be ideal to get feedback from the LuaTeX folks!

Agreed!

Thanks,

Thiago

¹ https://tex.stackexchange.com/questions/13593/the-differences-between-tex-engines

² https://tex.stackexchange.com/questions/36/differences-between-luatex-context-and-xetex

³ https://www.tug.org/texlive/doc/texlive-en/texlive-en.html#x1-120002.4

luigi scarso wrote on 31 Dec 2021 18:40

Re: [Dev-luatex] bug#48064: texlive-* packages fail to build non-deterministically

Recipients:(name . Thiago Jung Bauermann)(address . bauermann@kolabnow.com)

Message-ID:CAG5iGsDQYfYEShOEqLGYGPYc4Lf-S-qsCPOV-ZZ0xKnVK+QpFg@mail.gmail.com

On Fri, Jul 2, 2021 at 5:11 PM Thiago Jung Bauermann <bauermann@kolabnow.com>

wrote:

Toggle quote (26 lines)> Em quarta-feira, 30 de junho de 2021, às 08:53:41 -03, luigi scarso
> escreveu:
> > On Wed, Jun 30, 2021 at 8:20 AM Ludovic Courtès <ludo@gnu.org> wrote:
> > > Hi,
> > >
> > > Ludovic Courtès <ludo@gnu.org> skribis:
> > > > While investigating luatex crashes in the TeX Live 2020 package of
> > > > GNU Guix¹, we identified the following heap corruption reported by
> > >
> > > > Valgrind (this is on GNU/Linux, with glibc 2.33):
> > > This time with debug info for luatex:
> > Thank you for the report, I will check asap.
>
> Thanks! I was able to run Valgrind on LuaTeX 1.13.0, which is the latest
> one in TeX Live 2021.
>
> The invalid reads and writes don’t happen on every run. I had to re-run
> the
> command 3 or 4 times until I got the result below (which matches our
> experience with the build failures in Guix packages)
>
> --
> Thanks,
> Thiago
>

Until now I was not able to reproduce the issue.

luigi

Attachment: file

Maxim Cournoyer wrote on 4 Oct 02:25 +0200

control message for bug #73613

Recipients:(address . control@debbugs.gnu.org)

Message-ID:87jzeoemto.fsf@gmail.com

merge 73613 48455

quit

Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 48064@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it

mumi current 48064

Then, you may apply the latest patchset in this issue (with sign off)

mumi am -- -s

Or, compose a reply to this issue

mumi compose

Or, send patches to this issue

mumi send-email *.patch

is:open	open issues
is:done	closed issues
submitter:<who>	search issue submitter
author:<who>	search by message author
date:yesterday..now	search by issue date
mdate:3m..2d	search by message date