Reprotest fails to run due to missing execute bit

  • Done
  • quality assurance status badge
Details
2 participants
  • Lars-Dominik Braun
  • Vagrant Cascadian
Owner
unassigned
Submitted by
Vagrant Cascadian
Severity
normal

Debbugs page

Vagrant Cascadian wrote 2 weeks ago
(address . guix-patches@gnu.org)
87jz9c73wo.fsf@wireframe
Reprotest fails to run correctly due to files that are not marked
executable:

reprotest --verbose --vary=-fileordering,-domain_host,-user_group,-time -c 'date > date' . date -- null
WARNING:reprotest:The control build runs on 1 CPU by default, give --min-cpus to increase this.
WARNING:reprotest:diffoscope not available, falling back to regular diff
INFO:reprotest:STARTING VIRTUAL SERVER ['/gnu/store/7cglj9r39y4hfbwzd3rbh2rgl1yg343v-reprotest-0.7.28/lib/python3.10/site-packages/reprotest/virt/autopkgtest-virt-null']
Traceback (most recent call last):
File "/gnu/store/7cglj9r39y4hfbwzd3rbh2rgl1yg343v-reprotest-0.7.28/lib/python3.10/site-packages/reprotest/__init__.py", line 862, in run
return 0 if check_func(*check_args) else 1
File "/gnu/store/7cglj9r39y4hfbwzd3rbh2rgl1yg343v-reprotest-0.7.28/lib/python3.10/site-packages/reprotest/__init__.py", line 370, in check
proc = test_args._replace(result_dir=result_dir).corun_builds(testbed_args)
File "/gnu/store/7cglj9r39y4hfbwzd3rbh2rgl1yg343v-reprotest-0.7.28/lib/python3.10/site-packages/reprotest/__init__.py", line 114, in start
next(cr)
File "/gnu/store/7cglj9r39y4hfbwzd3rbh2rgl1yg343v-reprotest-0.7.28/lib/python3.10/site-packages/reprotest/__init__.py", line 322, in corun_builds
with start_testbed(virtual_server_args, temp_dir, no_clean_on_error,
File "/gnu/store/50047r6f9chb6dwd01f9hai90y9arwb4-python-3.10.7/lib/python3.10/contextlib.py", line 135, in __enter__
return next(self.gen)
File "/gnu/store/7cglj9r39y4hfbwzd3rbh2rgl1yg343v-reprotest-0.7.28/lib/python3.10/site-packages/reprotest/__init__.py", line 85, in start_testbed
testbed.start()
File "/gnu/store/7cglj9r39y4hfbwzd3rbh2rgl1yg343v-reprotest-0.7.28/lib/python3.10/site-packages/reprotest/lib/adt_testbed.py", line 134, in start
self.sp = subprocess.Popen(self.vserver_argv,
File "/gnu/store/50047r6f9chb6dwd01f9hai90y9arwb4-python-3.10.7/lib/python3.10/subprocess.py", line 969, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/gnu/store/50047r6f9chb6dwd01f9hai90y9arwb4-python-3.10.7/lib/python3.10/subprocess.py", line 1845, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
PermissionError: [Errno 13] Permission denied: '/gnu/store/7cglj9r39y4hfbwzd3rbh2rgl1yg343v-reprotest-0.7.28/lib/python3.10/site-packages/reprotest/virt/autopkgtest-virt-null'

The reprotest version 0.7.21 included in guix v1.4.0 did have these
files marked executable, but for some reason in recent versions (maybe
changes in pyproject-build-system?), they are shipped without the
execute bit set. I have not git bisected between all the versions of
guix to see exactly where it starts failing.

CCed members of the python team if they have ideas about changes in the
build system which might cause the issue.

The attached patch marks the necessary files are executable with an
added phase. Open to refactorring the patch, but my guile skills are
quite limited and this works!


live well,
vagrant
From 5ed6bdda83e9e06f5984faac3d347b91438e9b04 Mon Sep 17 00:00:00 2001
From: Vagrant Cascadian <vagrant@reproducible-builds.org>
Date: Wed, 26 Feb 2025 11:49:48 -0800
Subject: [PATCH] gnu: reprotest: Add phase fixing executable bit on virt
files.

* gnu/packages/diffoscope.scm (reprotest): Add 'make-virt-files-executable
phase.
---
gnu/packages/diffoscope.scm | 7 +++++++
1 file changed, 7 insertions(+)

Toggle diff (22 lines)
diff --git a/gnu/packages/diffoscope.scm b/gnu/packages/diffoscope.scm
index d0e911363d..90be6e68ec 100644
--- a/gnu/packages/diffoscope.scm
+++ b/gnu/packages/diffoscope.scm
@@ -273,6 +273,13 @@ (define-public reprotest
#:tests? #f
#:phases
#~(modify-phases %standard-phases
+ (add-after 'compress-documentation 'make-virt-files-executable
+ ;; The autopkgtest-virt- files need to be marked executable for
+ ;; reprotest to function correctly.
+ (lambda _
+ (for-each (lambda (file)
+ (invoke "chmod" "+x" file))
+ (find-files #$output "autopkgtest-virt-.*"))))
(add-after 'install 'install-doc
(lambda _
(let* ((mandir1 (string-append

base-commit: 54ff5d33739ae95c19f4ed5bea38d2613f6f7d3c
--
2.39.5
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCZ7+ReAAKCRDcUY/If5cW
qn3WAP9KkbpEZlSTUpQw5apcFCtQkhhj7GS/BGqfqw4xZWguowEA+9TPtpJ0Gqxi
5Z00KPmo3Nwqx9d6iaaviYHT4p5utQg=
=h8ZM
-----END PGP SIGNATURE-----

Lars-Dominik Braun wrote 2 weeks ago
(name . Vagrant Cascadian)(address . vagrant@reproducible-builds.org)
Z8HIeQGvVEI_Eiar@noor.fritz.box
Hi,

Toggle quote (6 lines)
> The reprotest version 0.7.21 included in guix v1.4.0 did have these
> files marked executable, but for some reason in recent versions (maybe
> changes in pyproject-build-system?), they are shipped without the
> execute bit set. I have not git bisected between all the versions of
> guix to see exactly where it starts failing.

my best guess is that the package previously used python-build-system
instead of pyproject-build-system. The latter basically builds a zip file
(wheel) of everything and extracts it to site-packages. Zip files don’t
include the +x bit and so it gets lost in the process.

I believe this is a bug in the package. It should either indirectly call
these files via the python3 executable or mark them as entry points[1],
so pyproject-build-system can put an executable into /bin.


Toggle quote (4 lines)
> The attached patch marks the necessary files are executable with an
> added phase. Open to refactorring the patch, but my guile skills are
> quite limited and this works!

Perhaps (chmod file #o755) instead of (invoke …)? Otherwise should be
fine to get it working again.

Cheers,
Lars
Vagrant Cascadian wrote 2 weeks ago
(name . Lars-Dominik Braun)(address . lars@6xq.net)
87bjul7n8g.fsf@wireframe
On 2025-02-28, Lars-Dominik Braun wrote:
Toggle quote (11 lines)
>> The reprotest version 0.7.21 included in guix v1.4.0 did have these
>> files marked executable, but for some reason in recent versions (maybe
>> changes in pyproject-build-system?), they are shipped without the
>> execute bit set. I have not git bisected between all the versions of
>> guix to see exactly where it starts failing.
>
> my best guess is that the package previously used python-build-system
> instead of pyproject-build-system. The latter basically builds a zip file
> (wheel) of everything and extracts it to site-packages. Zip files don’t
> include the +x bit and so it gets lost in the process.

Good hint! It was switched to pyproject-build-system in
d3bad46f990fbd884ca114dddd5af0eae434aecc ... and reverting that commit
does get it running again!

I have had a git bisect running in the background for days now to try
and identify the commit, as most of the builds fail, but can stop that
now! *sigh*


Toggle quote (6 lines)
> I believe this is a bug in the package. It should either indirectly call
> these files via the python3 executable or mark them as entry points[1],
> so pyproject-build-system can put an executable into /bin.
>
> [1] https://setuptools.pypa.io/en/latest/userguide/entry_point.html

Will try to wrap my head around that and get it fixed upstream...


Toggle quote (7 lines)
>> The attached patch marks the necessary files are executable with an
>> added phase. Open to refactorring the patch, but my guile skills are
>> quite limited and this works!
>
> Perhaps (chmod file #o755) instead of (invoke …)? Otherwise should be
> fine to get it working again.

Will try it! Thanks!


live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCZ8IvnwAKCRDcUY/If5cW
qoyPAQDEKQKXdPxfM18k1311LTTTgqJVSmSOatbYBBbNCFdTJwD7BSyCQKX1L1oN
MKfzqM+76jqHUcURdxnqcRdj5lsGsws=
=YsnO
-----END PGP SIGNATURE-----

Vagrant Cascadian wrote 2 weeks ago
(name . Lars-Dominik Braun)(address . lars@6xq.net)
8734fw767c.fsf@wireframe
On 2025-02-28, Vagrant Cascadian wrote:
Toggle quote (29 lines)
> On 2025-02-28, Lars-Dominik Braun wrote:
>>> The reprotest version 0.7.21 included in guix v1.4.0 did have these
>>> files marked executable, but for some reason in recent versions (maybe
>>> changes in pyproject-build-system?), they are shipped without the
>>> execute bit set. I have not git bisected between all the versions of
>>> guix to see exactly where it starts failing.
>>
>> my best guess is that the package previously used python-build-system
>> instead of pyproject-build-system. The latter basically builds a zip file
>> (wheel) of everything and extracts it to site-packages. Zip files don’t
>> include the +x bit and so it gets lost in the process.
>
> Good hint! It was switched to pyproject-build-system in
> d3bad46f990fbd884ca114dddd5af0eae434aecc ... and reverting that commit
> does get it running again!
>
> I have had a git bisect running in the background for days now to try
> and identify the commit, as most of the builds fail, but can stop that
> now! *sigh*
>
>
>> I believe this is a bug in the package. It should either indirectly call
>> these files via the python3 executable or mark them as entry points[1],
>> so pyproject-build-system can put an executable into /bin.
>>
>> [1] https://setuptools.pypa.io/en/latest/userguide/entry_point.html
>
> Will try to wrap my head around that and get it fixed upstream...

Still should do this at some point...


Toggle quote (9 lines)
>>> The attached patch marks the necessary files are executable with an
>>> added phase. Open to refactorring the patch, but my guile skills are
>>> quite limited and this works!
>>
>> Perhaps (chmod file #o755) instead of (invoke …)? Otherwise should be
>> fine to get it working again.
>
> Will try it! Thanks!

Pushed as:

2558392803663f4381fa9c09a53d329b719152bd gnu: reprotest: Add phase fixing executable bit on virt files.

Thanks for your help!

live well,
vagrant
-----BEGIN PGP SIGNATURE-----

iHUEARYKAB0WIQRlgHNhO/zFx+LkXUXcUY/If5cWqgUCZ8OF1wAKCRDcUY/If5cW
qgRnAQCiV9rMG07o1fOPlKHkPCJCL7BoJL+m/r/O/RkyrNotEgD9FMasOcXjvhCZ
mRB9WBgbVp5xGWHFDIgYkIz2Qt5AkA8=
=r4XN
-----END PGP SIGNATURE-----

Closed
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send an email to 76602@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 76602
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help