numpy CPU dispatch probably prevents builds of python-numpy from being reproducible

Numpy accelerates operations with native code. Early in the build

it probes the CPU / compiler for supported instruction extensions

and then proceeds to build extensions for different features. I’m

guessing that this won’t work when building on older CPUs, so the

results of two builds on different CPUs may differ.

It would be helpful if someone could confirm this by building

numpy on an older CPU and then compare the build result with that

provided by ci.guix.gnu.org.

--

Ricardo

Ricardo Wurmus schreef op za 20-11-2021 om 15:36 [+0000]:

I built it locally, and have the same result as ci.guix.gnu.org.

However, the bordeaux substitute is different:

$ LANGUAGE=en guix challenge python-numpy

/gnu/store/k311v0843hgq5frcbscylwimqssl1h54-python-numpy-1.17.3

contents differ:

local hash: 07var0wn8fywxchldz5pjqpnlavrlbc8s81aqwsqyi0i7qlh6ka7

: 07var0wn8fywxchldz5pjqpnlavrlbc8s81aqwsqyi0i7qlh6ka7

: 0cbl3q19bshb6ddz8xkcrjzkcmillsqii4z852ybzixyp7rg40qa

bordeaux.guix.gnu.org 3.9MiB 6.7MiB/s

00:01 [##################] 100.0%

1 store items were analyzed:

- 0 (0.0%) were identical

- 1 (100.0%) differed

- 0 (0.0%) were inconclusive

Diffoscope output:

bordeaux.guix.gnu.org 3.9MiB 4.2MiB/s 00:01 [##################] 100.0%

[...]

? ? ? ? ? ? ? ? --- /tmp/guix-directory.6ivbVt/lib/python3.8/site-packages/numpy/distutils/fcompiler/__pycache__/vast.cpython-38.pyc

? ? ? ? ? ? ? ??? +++ /gnu/store/k311v0843hgq5frcbscylwimqssl1h54-python-numpy-1.17.3/lib/python3.8/site-packages/numpy/distutils/fcompiler/__pycache__/vast.cpython-38.pyc

? ? ? ? ? ? ? ? @@ -106,15 +106,15 @@

? ? ? ? ? ? ? ? 00000690: 0104 0106 0104 0104 0106 0104 f906 0904 ................

? ? ? ? ? ? ? ? 000006a0: 0104 0208 0308 06da 085f 5f6d 6169 6e5f .........__main_

? ? ? ? ? ? ? ? 000006b0: 5f29 01da 036c 6f67 e902 0000 0029 01da _)...log.....)..

? ? ? ? ? ? ? ? 000006c0: 1463 7573 746f 6d69 7a65 645f 6663 6f6d .customized_fcom

? ? ? ? ? ? ? ? 000006d0: 7069 6c65 7272 0700 0000 2901 da08 636f pilerr....)...co

? ? ? ? ? ? ? ? 000006e0: 6d70 696c 6572 2911 da0a 5f5f 6675 7475 mpiler)...__futu

? ? ? ? ? ? ? ? 000006f0: 7265 5f5f 7202 0000 0072 0300 0000 7204 re__r....r....r.

? ? ? ? ? ? ? ? -00000700: 0000 0072 1900 0000 da1d 6e75 6d70 792e ...r......numpy.

? ? ? ? ? ? ? ? +00000700: 0000 0072 1900 0000 5a1d 6e75 6d70 792e ...r....Z.numpy.

? ? ? ? ? ? ? ? 00000710: 6469 7374 7574 696c 732e 6663 6f6d 7069 distutils.fcompi

? ? ? ? ? ? ? ? 00000720: 6c65 722e 676e 7572 0500 0000 da09 636f ler.gnur......co

? ? ? ? ? ? ? ? 00000730: 6d70 696c 6572 7372 0600 0000 7227 0000 mpilersr....r'..

? ? ? ? ? ? ? ? 00000740: 00da 0964 6973 7475 7469 6c73 7233 0000 ...distutilsr3..

? ? ? ? ? ? ? ? 00000750: 00da 0d73 6574 5f76 6572 626f 7369 7479 ...set_verbosity

? ? ? ? ? ? ? ? 00000760: da0f 6e75 6d70 792e 6469 7374 7574 696c ..numpy.distutil

? ? ? ? ? ? ? ? 00000770: 7372 3500 0000 da05 7072 696e 7472 2000 sr5.....printr .

1 bit is different.

Greetings,

Maxime.

Hi,

Ricardo Wurmus <rekado@elephly.net> skribis:

I don’t see anything in the build log that suggests it might be building

for a specific ISA extension (on x86_64).

On the contrary, it seems to do the right thing by dispatching to the

right optimized implementation at run time, in numpy/core/src/umath.

Apparently this directory contains templates for home-made function

multi-versioning, with variants and dispatch code generated by

numpy/core/code_generators/generate_umath.py.

Besides, I see the same .pyc difference as Maxime, but I don’t think it

relates to ISA extensions.

Ludo’.