[core-updates] Python sitecustomize issue.

OpenSubmitted by Mathieu Othacehe.
Details
2 participants
  • Maxim Cournoyer
  • Mathieu Othacehe
Owner
unassigned
Severity
important
M
M
Mathieu Othacehe wrote on 18 Aug 11:53 +0200
(address . bug-guix@gnu.org)(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)
878s0zoz98.fsf@gnu.org
Hello, I tried to upgrade glade to the 3.38.2 release on core-update and encountered the following test issue:
Toggle snippet (13 lines)2/5 modules FAIL 0.29s killed by signal 5 SIGTRAP >>> GLADE_MODULE_SEARCH_PATH=/home/mathieu/glade-3.38.2/build/plugins/gtk+:/home/mathieu/glade-3.38.2/build/plugins/python:/home/mathieu/glade-3.38.2/build/plugins/gjs:/home/mathieu/glade-3.38.2/tests/modules MALLOC_PERTURB_=195 GLADE_TESTING=1 GLADE_CATALOG_SEARCH_PATH=/home/mathieu/glade-3.38.2/plugins/gtk+:/home/mathieu/glade-3.38.2/plugins/python:/home/mathieu/glade-3.38.2/plugins/gjs:/home/mathieu/glade-3.38.2/tests/catalogs GLADE_PIXMAP_DIR=/home/mathieu/glade-3.38.2/data/icons GLADE_ICON_THEME_PATH=/home/mathieu/glade-3.38.2/plugins/gtk+/icons/22x22 /home/mathieu/glade-3.38.2/build/tests/modules ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― stdout: # random seed: R02Sd222538bc9b76b6e424ec0cb17ee887c # GLib-GIO-DEBUG: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ?gio-vfs? Bail out! GladeUI-PYTHON-FATAL-WARNING: Error initializing Python interpreter: could not import pygobject stderr: Error in sitecustomize; set PYTHONVERBOSE for traceback: ValueError: '/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages' is not in list (/home/mathieu/glade-3.38.2/build/tests/modules:4898): GladeUI-PYTHON-WARNING **: 09:43:58.605: Error initializing Python interpreter: could not import pygobject
I think the issue here lies in the sitecustomize.py file introduced with cb72f9a773e0931ee3758c851d96007ded034e4c.
Toggle snippet (2 lines)python_root = os.path.realpath(sys.executable).split('/bin/')[0]
When using the Python C library, sys.executable is not the expected python binary but the test binary. So when we try to find the associated libraries in the Python search path, this way:
Toggle snippet (3 lines)index = sys_path_absolute.index(python_site) sys.path = sys.path[:index] + matching_sites + sys.path[index:]
we get:
Toggle snippet (2 lines)ValueError: '/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages' is not in list
Running the test with the PYTHONPATH set to:
Toggle snippet (2 lines)PYTHONPATH=/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages:$PYTHONPATH
is a way to work around this issue. The fix here would be to replace sys.executable in the sitecustomize.py with something more adequate, but I have no idea what could it be. Thanks, Mathieu
M
M
Maxim Cournoyer wrote on 31 Aug 06:03 +0200
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 50105@debbugs.gnu.org)
87zgsyi7lg.fsf@gmail.com
Hi Mathieu,
[...]
Toggle quote (30 lines)> I think the issue here lies in the sitecustomize.py file introduced with> cb72f9a773e0931ee3758c851d96007ded034e4c.>> python_root = os.path.realpath(sys.executable).split('/bin/')[0]>>> When using the Python C library, sys.executable is not the expected> python binary but the test binary.>> So when we try to find the associated libraries in the Python search> path, this way:>> index = sys_path_absolute.index(python_site)> sys.path = sys.path[:index] + matching_sites + sys.path[index:]>>> we get:>> ValueError: '/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages' is not in list>>> Running the test with the PYTHONPATH set to:>> PYTHONPATH=/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages:$PYTHONPATH >> is a way to work around this issue.>> The fix here would be to replace sys.executable in the sitecustomize.py> with something more adequate, but I have no idea what could it be.
From the doc [0], sys.prefix seems to be more correct. This had beensuggested by Hartmut some time ago and I had this patch ready recently.Perhaps it already addresses the issue at hand?
You'll find it attached. It hasn't gotten much testing yet.
[0] https://docs.python.org/3/library/sys.html#sys.prefix
From e693b38a6d5c370e13f795c6303f50871ec78ae4 Mon Sep 17 00:00:00 2001From: Maxim Cournoyer <maxim.cournoyer@gmail.com>Date: Tue, 3 Aug 2021 20:41:20 -0400Subject: [PATCH] aux-files: sitecustomize: Cleanup and add explanatory comments.
* gnu/packages/aux-files/python/sitecustomize.py: Add a comment explaining thegeneral idea.
(major_minor): Use the unpacking operator (*) to provide the arguments.(site_packages_prefix): Use os.path.join to form the path.(python_site): Likewise.(all_sites_raw): Split on os.path.pathsep.(sys.path): Directly splice the result in the list.
Suggested-by: Hartmut Goebel <h.goebel@crazy-compilers.com>--- .../aux-files/python/sitecustomize.py | 28 ++++++++++++++----- 1 file changed, 21 insertions(+), 7 deletions(-)
Toggle diff (49 lines)diff --git a/gnu/packages/aux-files/python/sitecustomize.py b/gnu/packages/aux-files/python/sitecustomize.pyindex 65d3c7d554..71e328b9ac 100644--- a/gnu/packages/aux-files/python/sitecustomize.py+++ b/gnu/packages/aux-files/python/sitecustomize.py@@ -20,13 +20,26 @@ import os import sys -python_root = os.path.realpath(sys.executable).split('/bin/')[0]-major_minor = '{}.{}'.format(sys.version_info[0], sys.version_info[1])-site_packages_prefix = 'lib/python' + major_minor + '/site-packages'-python_site = python_root + '/' + site_packages_prefix+# Commentary:+#+# Site-specific customization for Guix.+#+# The program below honors the GUIX_PYTHONPATH environment variable to+# discover Python packages. File names appearing in this variable that match+# a predefined versioned installation prefix are added to the sys.path. To be+# considered, a Python package must be installed under the+# 'lib/pythonX.Y/site-packages' directory, where X and Y are the major and+# minor version numbers of the Python interpreter.+#+# Code:++major_minor = '{}.{}'.format(*sys.version_info)+site_packages_prefix = os.path.join(+ 'lib', 'python' + major_minor, 'site-packages')+python_site = os.path.join(sys.prefix, site_packages_prefix) try:- all_sites_raw = os.environ['GUIX_PYTHONPATH'].split(':')+ all_sites_raw = os.environ['GUIX_PYTHONPATH'].split(os.path.pathsep) except KeyError: all_sites_raw = [] # Normalize paths, otherwise a trailing slash would cause it to not match.@@ -35,7 +48,8 @@ matching_sites = [p for p in all_sites_norm if p.endswith(site_packages_prefix)] # Insert sites matching the current version into sys.path, right before-# Python's own site.+# Python's own site. This way, the user can override the libraries provided+# by Python itself. sys_path_absolute = [os.path.realpath(p) for p in sys.path] index = sys_path_absolute.index(python_site)-sys.path = sys.path[:index] + matching_sites + sys.path[index:]+sys.path[index:index] = matching_sites-- 2.32.0
M
M
Maxim Cournoyer wrote on 31 Aug 06:03 +0200
control message for bug #50105
(address . control@debbugs.gnu.org)
87y28ii7ks.fsf@gmail.com
severity 50105 importantquit
M
M
Maxim Cournoyer wrote on 31 Aug 15:47 +0200
(address . control@debbugs.gnu.org)
87wno1iv4x.fsf@gmail.com
tags 50105 + patchquit
M
M
Maxim Cournoyer wrote on 3 Sep 16:41 +0200
Re: bug#50105: [core-updates] Python sitecustomize issue.
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 50105@debbugs.gnu.org)
87y28dbu1z.fsf@gmail.com
Hi Mathieu, Mathieu Othacehe <othacehe@gnu.org> writes:
Toggle quote (21 lines)> Hello, > > I tried to upgrade glade to the 3.38.2 release on core-update and > encountered the following test issue: > > 2/5 modules FAIL 0.29s killed by signal 5 SIGTRAP >>>> GLADE_MODULE_SEARCH_PATH=/home/mathieu/glade-3.38.2/build/plugins/gtk+:/home/mathieu/glade-3.38.2/build/plugins/python:/home/mathieu/glade-3.38.2/build/plugins/gjs:/home/mathieu/glade-3.38.2/tests/modules MALLOC_PERTURB_=195 GLADE_TESTING=1 GLADE_CATALOG_SEARCH_PATH=/home/mathieu/glade-3.38.2/plugins/gtk+:/home/mathieu/glade-3.38.2/plugins/python:/home/mathieu/glade-3.38.2/plugins/gjs:/home/mathieu/glade-3.38.2/tests/catalogs GLADE_PIXMAP_DIR=/home/mathieu/glade-3.38.2/data/icons GLADE_ICON_THEME_PATH=/home/mathieu/glade-3.38.2/plugins/gtk+/icons/22x22 /home/mathieu/glade-3.38.2/build/tests/modules > ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀ ――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― > stdout: > # random seed: R02Sd222538bc9b76b6e424ec0cb17ee887c > # GLib-GIO-DEBUG: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ?gio-vfs? > Bail out! GladeUI-PYTHON-FATAL-WARNING: Error initializing Python interpreter: could not import pygobject > stderr: > Error in sitecustomize; set PYTHONVERBOSE for traceback: > ValueError: '/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages' is not in list > > (/home/mathieu/glade-3.38.2/build/tests/modules:4898): GladeUI-PYTHON-WARNING **: 09:43:58.605: Error initializing Python interpreter: could not import pygobject > > > I think the issue here lies in the sitecustomize.py file introduced with > cb72f9a773e0931ee3758c851d96007ded034e4c.
[...] Is there a simple reproducer to test my fix? Would just building current Glade and its test suite trigger it? Thanks, Maxim
M
M
Maxim Cournoyer wrote on 3 Sep 17:21 +0200
(name . Mathieu Othacehe)(address . othacehe@gnu.org)(address . 50105@debbugs.gnu.org)
87tuj1bs6p.fsf@gmail.com
Hi again,
Maxim Cournoyer <maxim.cournoyer@gmail.com> writes:
[...]
Toggle quote (3 lines)> Is there a simple reproducer to test my fix? Would just building> current Glade and its test suite trigger it?
OK, nevermind, I found that the test case was deleting the 'fix-tests'build phase from glade3, and building the package. I confirm the fixworks!
I'll send that patch series I've been talking about.
Thanks,
Maxim
M
M
Mathieu Othacehe wrote on 3 Sep 20:46 +0200
(name . Maxim Cournoyer)(address . maxim.cournoyer@gmail.com)(address . 50105@debbugs.gnu.org)
87fsulqyyn.fsf@gnu.org
Hello Maxim,
Toggle quote (4 lines)> OK, nevermind, I found that the test case was deleting the 'fix-tests'> build phase from glade3, and building the package. I confirm the fix> works!
Great, that's exactly what I was trying to test but you beat me to it:). I'll be afk for a month so I won't be able to confirm that I havethe same result locally. However, this fix looks good to me.
Thanks,
Mathieu
?
Your comment

Commenting via the web interface is currently disabled.

To comment on this conversation send email to 50105@debbugs.gnu.org