Bootstrap packages fail to build due to mes-libc lacking 'stat64' etc. syscalls

OpenSubmitted by Mathieu Othacehe.
Details
3 participants
  • Jan Nieuwenhuizen
  • Ludovic Courtès
  • Mathieu Othacehe
Owner
unassigned
Severity
important
Merged with
M
M
Mathieu Othacehe wrote on 14 May 2020 17:17
Bootstrap packages fail to build.
(address . bug-guix@gnu.org)(address . janneke@gnu.org)
87h7wik0kk.fsf@gnu.org
Hello,
This command fails on one of my systems:
Toggle snippet (3 lines)guix build -e "(@@ (gnu packages commencement) glibc-mesboot0)"
with the following error:
Toggle snippet (11 lines)phase `unpack' succeeded after 11.8 secondsstarting phase `apply-boot-patch'patch: **** fstatsterror: unknown error: command "patch" "--force" "-p1" "-i" "/gnu/store/pfz4y5i7krlvam2m8lpddmg9vi44rpqh-glibc-boot-2.2.5.patch" failed with status 2note: keeping build directory `/tmp/guix-build-glibc-mesboot0-2.2.5.drv-1'builder for `/gnu/store/jcqggqckhiq43y2ivlfhpkbfbp2vyjlc-glibc-mesboot0-2.2.5.drv' failed with exit code 1build of /gnu/store/jcqggqckhiq43y2ivlfhpkbfbp2vyjlc-glibc-mesboot0-2.2.5.drv failedView build log at '/var/log/guix/drvs/jc/qggqckhiq43y2ivlfhpkbfbp2vyjlc-glibc-mesboot0-2.2.5.drv.bz2'.guix build: error: build of `/gnu/store/jcqggqckhiq43y2ivlfhpkbfbp2vyjlc-glibc-mesboot0-2.2.5.drv' failed
Here's a stracing of the failing "patch" command:
Toggle snippet (5 lines)open("/gnu/store/pfz4y5i7krlvam2m8lpddmg9vi44rpqh-glibc-boot-2.2.5.patch", O_RDONLY) = 3brk(0x9377913) = 0x9377913fstat(3, 0xffb29328) = -1 EOVERFLOW (Value too large for defined data type)
"patch-mesboot" is built for 32 bits. Hence, it can be using"__ia32_sys_fstat", "__ia32_compat_sys_newfstat" or"__ia32_compat_sys_x86_fstat64" syscall for "fstat". Here, according toperf, it's using __ia32_compat_sys_newfstat which is overflowing on myfile system (inode count to high or so).
There's a little demonstration program attached. When built with `gcc-m32 test.c', I have:
Toggle snippet (5 lines)fstat(3, 0xffad5874) = -1 EOVERFLOW (Value too large for defined data type)fstat64(3, {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0fstat64(3, {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0
So I think somehow, bootstrap packages use the legacy "fstat" syscall,which may overflow on a 64 bits system.
WDYT,
Thanks,
Mathieu
#define _GNU_SOURCE#include <unistd.h>#include <sys/syscall.h>
#include <sys/types.h>#include <sys/stat.h>#include <time.h>#include <stdio.h>#include <stdlib.h>#include <fcntl.h>#include <sys/sysmacros.h>
intmain(int argc, char *argv[]){ struct stat sb;
if (argc != 2) { fprintf(stderr, "Usage: %s <pathname>\n", argv[0]); exit(EXIT_FAILURE); }
int fd = open(argv[1], O_RDONLY); syscall(__NR_fstat, fd, &sb); syscall(__NR_fstat64, fd, &sb); fstat(fd, &sb);

exit(EXIT_SUCCESS);}
M
M
Mathieu Othacehe wrote on 15 May 2020 14:11
(address . 41264@debbugs.gnu.org)
87v9kxe6tl.fsf@gnu.org
Hello,
Toggle quote (7 lines)> fstat(3, 0xffad5874) = -1 EOVERFLOW (Value too large for defined data type)> fstat64(3, {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0> fstat64(3, {st_mode=S_IFDIR|0555, st_size=4096, ...}) = 0>> So I think somehow, bootstrap packages use the legacy "fstat" syscall,> which may overflow on a 64 bits system.
More info on that one. Linux syscall "newstat", will call"cp_compat_stat". This function starts by checking the device id:
Toggle snippet (6 lines) struct compat_stat tmp;
if (!old_valid_dev(stat->dev) || !old_valid_dev(stat->rdev)) return -EOVERFLOW;
Here, stat->dev is 66308 (major: 259, minor 4).
"old_valid_dev" checks that:
Toggle snippet (17 lines)static inline bool old_valid_dev(dev_t dev){ return MAJOR(dev) < 256 && MINOR(dev) < 256;}--8<---------------cut here---------------end--------------->8---
Which is false here, because my NVME disk has a BLOCK_EXT_MAJOR (259).
So stat, lstat, and all other related function will return -EOVERFLOWunless their 64 bits stat64, lstat64 counterpart is used.
So I think this means that one cannot build the Guix bootstrap toolchain on anNVME disk.
Thanks,
Mathieu
M
M
Mathieu Othacehe wrote on 19 May 2020 10:52
(address . 41264@debbugs.gnu.org)
87ftbwtigj.fsf@gnu.org
Hello,
Toggle quote (6 lines)> So stat, lstat, and all other related function will return -EOVERFLOW> unless their 64 bits stat64, lstat64 counterpart is used.>> So I think this means that one cannot build the Guix bootstrap toolchain on an> NVME disk.
After further investigations I think that it is needed to patch GNU Mesto fix this bug.
We would need to add three new syscalls for x86:
Toggle snippet (5 lines)#define SYS_stat64 0xc3#define SYS_lstat64 0xc4#define SYS_fstat64 0xc5
lib/linux/stat.c should be modified this way:
Toggle snippet (24 lines)#if __i386__#define STAT_SYSCALL SYS_stat64#else#define STAT_SYSCALL SYS_stat#endif
intstat (char const *file_name, struct stat *statbuf){ struct stat64 statbuf64; int ret;
ret = _sys_call2 (STAT_SYSCALL, (long) file_name, (long) &statbuf64);
#if __i386__ stat64_to_32(&statbuf64, statbuf);#else *statbuf = statbuf64;#endif
return ret;}
Then we would need to create stat64_to_32 which could be inspired from__xstat64_conv from the glibc. Then, lstat and fstat64 would need to bepatched the same way.
This way, we would replicate the glibc behavior.
Thanks,
Mathieu
J
J
Jan Nieuwenhuizen wrote on 19 May 2020 18:52
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 41264@debbugs.gnu.org)
87sgfvq32v.fsf@gnu.org
Mathieu Othacehe writes:
Hello Mathieu,
Toggle quote (9 lines)>> So stat, lstat, and all other related function will return -EOVERFLOW>> unless their 64 bits stat64, lstat64 counterpart is used.>>>> So I think this means that one cannot build the Guix bootstrap toolchain on an>> NVME disk.>> After further investigations I think that it is needed to patch GNU Mes> to fix this bug.
Hmm, we need to patch Mes after all. That's unfortunate.
Toggle quote (15 lines)> We would need to add three new syscalls for x86:>> #define SYS_stat64 0xc3> #define SYS_lstat64 0xc4> #define SYS_fstat64 0xc5>>> lib/linux/stat.c should be modified this way:>> #if __i386__> #define STAT_SYSCALL SYS_stat64> #else> #define STAT_SYSCALL SYS_stat> #endif
Ah...the stat64 syscall is meant for i386; now it at starts making atleast some sense to me.
Toggle quote (17 lines)> int> stat (char const *file_name, struct stat *statbuf)> {> struct stat64 statbuf64;> int ret;>> ret = _sys_call2 (STAT_SYSCALL, (long) file_name, (long) &statbuf64);>> #if __i386__> stat64_to_32(&statbuf64, statbuf);> #else> *statbuf = statbuf64;> #endif>> return ret;> }
That looks OK...
Toggle quote (6 lines)> Then we would need to create stat64_to_32 which could be inspired from> __xstat64_conv from the glibc. Then, lstat and fstat64 would need to be> patched the same way.>> This way, we would replicate the glibc behavior.
Beautiful, thanks for getting to the bottom of this. Now that youalready have gone this far, would you like to whip-up a full patch forGNU Mes?
To test it we may have to provide a tarball as we don't want to use XZand we don't have patch yet. Or possibly we can download someindividual files ande overwrite them.
Greetings,Janneke
-- Jan Nieuwenhuizen <janneke@gnu.org> | GNU LilyPond http://lilypond.orgFreelance IT http://JoyofSource.com| Avatar® http://AvatarAcademy.com
L
L
Ludovic Courtès wrote on 1 Sep 15:56 +0200
control message for bug #41264
(address . control@debbugs.gnu.org)
87wno0761w.fsf@gnu.org
merge 41264 49985quit
L
L
Ludovic Courtès wrote on 1 Sep 15:59 +0200
(address . control@debbugs.gnu.org)
87v93k75xu.fsf@gnu.org
retitle 41264 Bootstrap packages fail to build due to mes-libc lacking 'stat64' etc. syscallsquit
L
L
Ludovic Courtès wrote on 1 Sep 15:59 +0200
(address . control@debbugs.gnu.org)
87tuj475xp.fsf@gnu.org
severity 41264 importantquit
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 41264@debbugs.gnu.org