[PATCH] python: Honor '--cores=...' in tests.

  • Done
  • quality assurance status badge
Details
3 participants
  • Eric Bavier
  • Mathieu Othacehe
  • Marius Bakke
Owner
unassigned
Submitted by
Eric Bavier
Severity
normal

Debbugs page

Eric Bavier wrote 6 years ago
(address . guix-patches@gnu.org)
20181105202344.0cab8e1e@centurylink.net
Hello Guix,

I noticed building 'python-minimal' that its tests seemed to have poor
memory performance, i.e. using *a lot* of RAM. I thought maybe
dropping --cores=... would help, but it did not; the tests use all
available cores by default. The attached patch fixes this issue,
though, the package still uses all cores while building some extension
libraries.

I suppose this patch would go to core-updates?

Cheers,
`~Eric
From 076d2f7f32a51f90f85be3da836d208987e9c678 Mon Sep 17 00:00:00 2001
From: Eric Bavier <bavier@member.fsf.org>
Date: Thu, 1 Nov 2018 21:18:41 -0500
Subject: [PATCH] python: Honor '--cores=...' in tests.

* gnu/packages/python.scm (python-2.7)[arguments]: Add #:make-flags.
---
gnu/packages/python.scm | 3 +++
1 file changed, 3 insertions(+)

Toggle diff (16 lines)
diff --git a/gnu/packages/python.scm b/gnu/packages/python.scm
index fbb280224..9a8b9dfcf 100644
--- a/gnu/packages/python.scm
+++ b/gnu/packages/python.scm
@@ -206,6 +206,9 @@
"--enable-unicode=ucs4"
(string-append "LDFLAGS=-Wl,-rpath="
(assoc-ref %outputs "out") "/lib"))
+ ;; With no -j argument tests use all available cpus, so provide one.
+ #:make-flags
+ (list (format #f "EXTRATESTOPTS=-j~d" (parallel-job-count)))
#:modules ((ice-9 ftw) (ice-9 match)
(guix build utils) (guix build gnu-build-system))
--
2.19.1
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEoMXjUi7471xkzbfw/XPKxxnTJWYFAlvg+zAACgkQ/XPKxxnT
JWaQJhAAnK19/Gseg2vroo+GPgnUIrKK61Nsa55pbr0G0h0fPZTxSOABO2QkTt0W
ZYqAwFtaSpxbebwRWUrZEAJpxqHST4KDsAaiHAryDyue/oxojNIhUvZfpgpVG1Dh
sNxkkS4uX0ru/yVtPwpRhw3SXmZb62EKAGT09R4Vn3oC8GRVTpfKeRjQMDY42zPD
dLfSN2lUx/WbwE0oAfnDOUQtX0JwuFdErHJ4wXNiiRqedw0FWNrdeMS0mImSK0ol
2tIHUUMC57CmozcXuemyB2o6Acn6UTyie1XjqpDlfKKbuheI9THLKcGrmha/6UU/
epgtjcH/2K+Q0aoepr7obCocAUuIEtJwiRyLstsyGK5SqVOfQUGhC9BEGrj83uJy
BswnD9iT5UNkNfBxQDBvdEhKAInvJDjNbt5XR2OGcdzGKF9ePrfahpWP/cijIKOG
JD069Zezv8bPaKTXh58pMHEu/+4RE+ClimdJqMdseJX6vOg6xFhpV0xO8xW4jnD8
SZYkqlP3StATyLit8Lro89ZrbJ6HRjmqiDFjFJz7f7yW3Zen4LyFmFrigWzr0Dub
qMtfUASji+87tRiAe+qezilOWy2TER2Ly0SfzVz90LbD4kFUFYALIKDSgc0ABZ+R
JQnEBHrlivwjWGWh4dchqhB6MW/f3d6Te1f1BUC8KUHZ0tZIswE=
=MFCC
-----END PGP SIGNATURE-----


Mathieu Othacehe wrote 6 years ago
(name . Eric Bavier)(address . ericbavier@centurylink.net)(address . 33284@debbugs.gnu.org)
CANVeeZyvpnaNa5YULLM0Vsv08weu3J_j0uuYBNFf1-NM+_vEYQ@mail.gmail.com
Hi Eric,

Your patch seems go to me, even if I think the root cause of your problem
is fixed by:


I think that both could be applied to core-updates.

Thanks,

Mathieu

Le mar. 6 nov. 2018 11:27, Eric Bavier <ericbavier@centurylink.net> a
écrit :

Toggle quote (14 lines)
> Hello Guix,
>
> I noticed building 'python-minimal' that its tests seemed to have poor
> memory performance, i.e. using *a lot* of RAM. I thought maybe
> dropping --cores=... would help, but it did not; the tests use all
> available cores by default. The attached patch fixes this issue,
> though, the package still uses all cores while building some extension
> libraries.
>
> I suppose this patch would go to core-updates?
>
> Cheers,
> `~Eric
>
Attachment: file
Eric Bavier wrote 6 years ago
(name . Mathieu Othacehe)(address . m.othacehe@gmail.com)(address . 33284@debbugs.gnu.org)
20181106085202.5e119586@centurylink.net
Hello Mathieu,

On Tue, 6 Nov 2018 14:07:47 +0900
Mathieu Othacehe <m.othacehe@gmail.com> wrote:

Toggle quote (9 lines)
> Hi Eric,
>
> Your patch seems go to me, even if I think the root cause of your problem
> is fixed by:
>
> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33186#26
>
> I think that both could be applied to core-updates.

Thanks for the link; it looks like it should help. But, yes, I think
both can be applied.

Toggle quote (18 lines)
>
> Le mar. 6 nov. 2018 11:27, Eric Bavier <ericbavier@centurylink.net> a
> écrit :
>
> > Hello Guix,
> >
> > I noticed building 'python-minimal' that its tests seemed to have poor
> > memory performance, i.e. using *a lot* of RAM. I thought maybe
> > dropping --cores=... would help, but it did not; the tests use all
> > available cores by default. The attached patch fixes this issue,
> > though, the package still uses all cores while building some extension
> > libraries.
> >
> > I suppose this patch would go to core-updates?
> >
> > Cheers,
> > `~Eric
> >
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEoMXjUi7471xkzbfw/XPKxxnTJWYFAlvhqpIACgkQ/XPKxxnT
JWbBLQ/+IAJFHxqXjYiLUfAhEJh5lpE9N6xzOyh3DgPDRe8bOQG+n5BdSz8PWtxl
830Q+xm9UVxu8zQ3l0W38+Wm2ifheHcyBnXSfSY/bB8Z4HkZGB3s6FeAt5Bb7Mj3
GH27mrdsVcIrreXwW+jcQbthbrodMCcpWtEf74g1YOEUmMrPvyHPKJ42WfxPuTFF
PVhYHraYXEfaUVZcH/Ym1+/xUMX9HNM3ZNmCEOfCjSBxd3n8rJF669tbDrNJd0VI
e71HVazJZ2F/mdlGaour/WFbbuIQk5mCMqyIYBM9DFCHsprTF5A6Rd1BNc5ZfX4F
gTbUS+YH80KnrGHb+tvx20+XnrikBsunL1wRuUoAMy2zXOCffZzzxcd43aghcKqY
ZqXVj9krWXhFDEKxttxGPJfo2eQvnpageET6FrSxR0yqUkGMafDVPipWAARauRen
Xp+L12RwoJalZXjKj2VKLKgDqIeBxiPPonmptXg9Mtdm/P/RD21Dri5pfLpZn1Yl
X+kA+fq6CK4jYCXK0C5F+RpTk9HDorNprETzDTuITHJ6ZYuVFcePQkUXSdPV7vAJ
qs8DI4YpDcuqeCVkbKMeIJK+Lk6pj2aTUeiOddvb7fwVLfnZNI5dRBryzAaTEUOy
urEBrkadJJANP+ZqTY1TGb7yeDYp6eB7OrrUBipQXePd9Pg18vg=
=7cPB
-----END PGP SIGNATURE-----


Marius Bakke wrote 6 years ago
(address . 33284@debbugs.gnu.org)
87o9b2p56m.fsf@fastmail.com
Eric Bavier <ericbavier@centurylink.net> writes:

Toggle quote (17 lines)
> Hello Mathieu,
>
> On Tue, 6 Nov 2018 14:07:47 +0900
> Mathieu Othacehe <m.othacehe@gmail.com> wrote:
>
>> Hi Eric,
>>
>> Your patch seems go to me, even if I think the root cause of your problem
>> is fixed by:
>>
>> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33186#26
>>
>> I think that both could be applied to core-updates.
>
> Thanks for the link; it looks like it should help. But, yes, I think
> both can be applied.

Note that the broken test has already been removed on 'core-updates'.

Eric: can you push a 'python-updates' branch? I wonder if we have time
for a "half-rebuild" before the next 'core-updates' round; I've got some
other heavy-impact patches in my pipeline (Glib, Cairo, ++).

The patch LGTM.
-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAlvhuoEACgkQoqBt8qM6
VPpTBgf/VLk6QCF2Uv2puv3EQ9HAZt+zvxcOfcIs24OJ8N7Im443ACMFlPF49yjx
SP4fFTyF88LI/2xn87iYBu2Y0cCo00KFXkUwkFkxTKvi4l3hNHtP1j31sB409aB/
DoCL3qE7ZE5DNqy3HyLNkENiOeud82PbNSlOKvA88HDJdXoiHlfGrwONzcC5YUKw
r1c3Sci5bAjncNeYNDFzkqCc1BqG3FsEyDy/uXzeNj2kMKRcMvzXERdyeTKhP7+C
8P/lMioYiPnGlZFntBfOwj/7xs8QQijIaxCNdpQqMVr18z9K51kXHBLUzUlMRYhp
8EKkPsOfKmIwem0POY2dz55cwJohXQ==
=Ci0K
-----END PGP SIGNATURE-----

Eric Bavier wrote 6 years ago
(name . Marius Bakke)(address . mbakke@fastmail.com)
20181106192237.79683848@centurylink.net
On Tue, 06 Nov 2018 17:00:01 +0100
Marius Bakke <mbakke@fastmail.com> wrote:

Toggle quote (27 lines)
> Eric Bavier <ericbavier@centurylink.net> writes:
>
> > Hello Mathieu,
> >
> > On Tue, 6 Nov 2018 14:07:47 +0900
> > Mathieu Othacehe <m.othacehe@gmail.com> wrote:
> >
> >> Hi Eric,
> >>
> >> Your patch seems go to me, even if I think the root cause of your problem
> >> is fixed by:
> >>
> >> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33186#26
> >>
> >> I think that both could be applied to core-updates.
> >
> > Thanks for the link; it looks like it should help. But, yes, I think
> > both can be applied.
>
> Note that the broken test has already been removed on 'core-updates'.
>
> Eric: can you push a 'python-updates' branch? I wonder if we have time
> for a "half-rebuild" before the next 'core-updates' round; I've got some
> other heavy-impact patches in my pipeline (Glib, Cairo, ++).
>
> The patch LGTM.

Sure, I can push this to a new branch. Thanks for the review.

`~Eric
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEoMXjUi7471xkzbfw/XPKxxnTJWYFAlviPl0ACgkQ/XPKxxnT
JWYIRw//WYeuFlU0WRoT95tKswY9qYKLGoMQhgxDNiXYn1OpvA4eh51NcUbZZB/w
jxL8V+AzcXldY9IIc7J/lOI3V5L1/tUjRgAZTBuvF7nKrB03bhHHGCGR6iLbe+XI
BAh91VrQ7O97NmoM+mSAzN7uexqd6HTJrWB6etAFgqmeEl0UV6Vzp2GsQoUMQTum
i7oBSt2JMcU2oh2hxN00Ru5mrEn+Pbq98oTTQQ8O+JXDmUdlAcol2q07AfkPGY3d
Bs4BiwijLenQZ5cO4yKtZzrif0UVhmaC4hduEFTWFIKwJoO8/ZH+pMlb9K7A/X59
xhpi/FyzoaN5DPz2333x2J55YLestsbtmZspRJ6f38THDJ7qQ+sCMs5eKiSFp3bZ
Gc+zEfbgaBxNwSDCFbpn0A9qpnYa81QQtVJ9gKGZy9Wdxu5C3atatD3E9gLknLxj
a99M8GGAJNp09XCKqrrue6guQ9nhmupGH3BLd16Fo23eGRy5mXs7wvmYFMn3arb6
Q3L9Dz/1wD1BF1lIpPdCagjiJttZAu3xHVmICzA+dN9lqCXdkKmDl2ZGZd62S3CF
2TtYa3hLGtCAYrcJ3+4eoBF4VPG7zr30nAvfSmzCahDDTkuyyhCJaLfUCNhV2ERl
RwZb+wMLSVFIBOszgl6eqZ48ntXfImqW812sOYS3h83jKqr5qhw=
=QG5E
-----END PGP SIGNATURE-----


Eric Bavier wrote 6 years ago
(name . Marius Bakke)(address . mbakke@fastmail.com)(address . 33284@debbugs.gnu.org)
20181106201822.3e82b810@centurylink.net
On Tue, 6 Nov 2018 19:22:37 -0600
Eric Bavier <ericbavier@centurylink.net> wrote:

Toggle quote (11 lines)
> On Tue, 06 Nov 2018 17:00:01 +0100
> Marius Bakke <mbakke@fastmail.com> wrote:
>
> > Eric: can you push a 'python-updates' branch? I wonder if we have time
> > for a "half-rebuild" before the next 'core-updates' round; I've got some
> > other heavy-impact patches in my pipeline (Glib, Cairo, ++).
> >
> > The patch LGTM.
>
> Sure, I can push this to a new branch.

Oops, there's already a 'python-updates' branch on savannah. Should
that branch be deleted, or force-pushed? (sorry, haven't paid much
attention to our branch management...)

`~Eric
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEoMXjUi7471xkzbfw/XPKxxnTJWYFAlviS24ACgkQ/XPKxxnT
JWbGWQ//exU973nMzRE5SaKVFjhVzw/JEWSlJFFHruquxQPG4SSda8iW8OVI3zM7
sdTP02au0khPAGA8AQ8JGCMitfFCRBf7jo9DuTD8cQUmCrSsMKXPx01KFLwcX4iw
+w2HMjqAnnNGYEwdAj63XRKJMuOi5xse9NgsRDHd6NRnkk2ulat7zEWr9ZjW8KpP
7ZFProhHsvRDO1M7tbbgbJ/s0tKfRWyHU35+4nH5E0R4IlUbWNHh7meJOd+VGaPm
EMJdkFP9ZqrAk9byjng4IgXipoEDTJ18xvc++uAxpsVCQHorWzP2RrUGyd2sMxkA
pg0FI+y4xbJTLiLglouVPBe75dDuxh/MCTNnL8735pbNMn+9DtbI8J3Cc9EgpMug
fKiE4nyWhSRxLBe8WWDgQ2/MTcu6t7hl34oBGvjjCzrv+z88EeQpzXdjdT7dFcvp
7798TSf+FBNK4bBd+Np9UuWE059gMxVz0htGo6URSD+WNLyNTHAU7G377n0+nwLR
HAcRnK7hdHzRgPelTUQPsLE8o32pyY1wdZ/1SKpgXbMe0JCZw7whlfAffqca04r3
L4kg9+vt8bmblib99z6yH5MSFKC+2iZcMLcoJAg+/OvtgVitce0qUmZbmvh00/ou
Nqy1Z297v3HYZx2g135pYlZbMt5aVDkWYFr61ymat6A5+S30Ttw=
=jTsd
-----END PGP SIGNATURE-----


Eric Bavier wrote 6 years ago
(name . Marius Bakke)(address . mbakke@fastmail.com)
20181117004515.1111b65a@centurylink.net
On Tue, 06 Nov 2018 17:00:01 +0100
Marius Bakke <mbakke@fastmail.com> wrote:

Toggle quote (27 lines)
> Eric Bavier <ericbavier@centurylink.net> writes:
>
> > Hello Mathieu,
> >
> > On Tue, 6 Nov 2018 14:07:47 +0900
> > Mathieu Othacehe <m.othacehe@gmail.com> wrote:
> >
> >> Hi Eric,
> >>
> >> Your patch seems go to me, even if I think the root cause of your problem
> >> is fixed by:
> >>
> >> https://debbugs.gnu.org/cgi/bugreport.cgi?bug=33186#26
> >>
> >> I think that both could be applied to core-updates.
> >
> > Thanks for the link; it looks like it should help. But, yes, I think
> > both can be applied.
>
> Note that the broken test has already been removed on 'core-updates'.
>
> Eric: can you push a 'python-updates' branch? I wonder if we have time
> for a "half-rebuild" before the next 'core-updates' round; I've got some
> other heavy-impact patches in my pipeline (Glib, Cairo, ++).
>
> The patch LGTM.

I've pushed this to the core-updates-next branch in commit
5b01b6034aeab32a5011c5757f18bd9772d3497d

`~Eric
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEoMXjUi7471xkzbfw/XPKxxnTJWYFAlvvuPsACgkQ/XPKxxnT
JWbsqxAAgjJLrou6Yn5X7XI+PvKvOnvuBlY2TWCGA1Lz6L4lnQ43ZkVWlatCH1KR
+fknDjt5+WjEPK80H7Jzb6xM8IRGADwTwipwuFBgOtLDLoD9TznQLZGeeJNz+U/c
AQ9+vDLuzskZiGXy6bgkL13hG+l3X91T3ewzsK3RkX/vpL1sQQI1/UlgPEXHDmae
To87NHug3XMv+FCfv+iF9vj69ZdXxZLEYQiFRjfQlObmFU4/4to0E8Eu3x8sf/5e
zeeVzMZXvWPxk2qHmN4frEcKkhAMjaPUOJDUCy3O8kNDErMcMejPHXoHjq2hmhho
sqbsnCPBcEFACDH7L/GicbmXuyKzCLt1B/QSrHpfnxIo3dWZ2kh3PjHMnXh6lwS6
mhB7K/hSPkE69ZL2LPnZCL+6gj4LvU7tS2BaNZUpRqXz7pY2tXfMtbH2j2SPncM+
/Wj9jwvW4WvHT18j1nyMkxlYVp0IiogzER1kFmNu+CbW6jFJ2o7WNhzvnj5d+mot
/dgbhWst9WtORDjsdSFdYGOfNbEbBg7b4MxpK3oO2PnRKpe0Jdj4RyCqePZUss8A
UitodlkYx1kvejnliFggwBbr6lb4oM16+iuui+aqqWtFayuoqd62FKOYi59c4yuw
1AVS0sBt7NDd/eWL5XivQGw4zsfrapLAA2ViLQvqxlkz9r5qAOg=
=zhTU
-----END PGP SIGNATURE-----


?
Your comment

This issue is archived.

To comment on this conversation send an email to 33284@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 33284
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help