Request for merging "mesa-updates" branch

Hello Guix,

The mesa-updates branch I think is just almost ready for merging. Besides some other fixes and updates, the main series is tracked at https://issues.guix.gnu.org/73071. There is an update to add NVK support to mesa for x86_64-linux which I need to review and push (and rebase to get more fixes from master).

Coverage looks good for x86_64 and i686 on QA, with powerpc64le as well on Berlin. I worry that aarch64 and others may have stalled out on Bordeaux. Perhaps Efraim can chime in there.

With an update for NVK for x86_64, that will take maybe a day to catch up again in builds but tends to be pretty quick. I'm not aware of other blockers.

Thanks!

John

On Mon, Sep 16, 2024 at 02:38:16AM +0000, John Kehayias via Guix-patches via wrote:

I built out to gtk+@3 and gtk on aarch64 without any problems, and I

also built mesa on riscv64 and armhf without any problems.

I haven't tested running any programs on those architectures.

--

Efraim Flashner <efraim@flashner.co.il> ????? ?????

GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351

Confidentiality cannot be guaranteed on emails sent or received unencrypted

Hello,

Thanks for the report and testing, Efraim!

I'm cc'ing guix-devel to see if anyone else wants to weigh in here:

On Mon, Sep 23, 2024 at 08:34 AM, Efraim Flashner wrote:

Progress on QA/Bordeaux is, from what I hear, waiting in line behind

other branch merge requests (one is from many months ago and I don't

think will be ready soon). I think this branch is ready to merge, the

only potential issue is lower substitute coverage on

non-i686/x86_64-linux architectures. (Note that although QA shows only

in the 80% range, it was about the same as master before the more

recent rebases. No idea why as I can't find new failures that would

cause this.)

So, what shall we do? Personally, I would merge it now with the

understanding that substitutes will take time (weeks? months?) to

catch up. I don't think we have the capacity to be quicker even if

there was only one active non-master branch for these architectures.

Is this correct?

While at times issues crop up, in my experience the mesa update part

of mesa-updates (which is almost entirely what is in this current

branch) rarely causes many issues, just lots of rebuilds. We can also

always revert if something was missed. I would be happy to add a news

entry as a warning to anyone relying on substitutes for other

architectures, if that is helpful.

Thoughts? Concerns? Guidance we can solidify going forward?

Thanks!

John

Hi,

The current queue of branches awaiting for review and merge:

| 71408 | python-team | Fri Jun 07 10:55:25+0200 2024 | Done |

| 72959 | fonts-split-outputs | Mon Sep 02 12:55:25+0200 2024 | Open |

| 73104 | r-team | Sat Sep 07 17:55:24+0200 2024 | Open |

| 73288 | mesa-updates | Mon Sep 16 04:38:25+0200 2024 | Open |

| 73502 | go-team | Thu Sep 26 23:40:25+0200 2024 | Open |

| 73515 | qt-team | Fri Sep 27 14:46:24+0200 2024 | Open |

| 73558 | wip-gsl-upgrade | Sun Sep 29 22:33:24+0200 2024 | Open |

| 73567 | lisp-team | Mon Sep 30 15:43:28+0200 2024 | Open |

- https://qa.guix.gnu.org/branch/python-team

#71408 It was closed due to a large amount of merge conflicts which

can't be resolved quick enough not blocking other changes (CC Steve who

started cherry pick process)

- https://qa.guix.gnu.org/branch/fonts-split-outputs

#72959 Needs to be rebased and place new evaluation (Cc Ludovic)

- https://qa.guix.gnu.org/branch/r-team

#73104 I've pinged R team members if that branch may be merged, the

changes touch just R packages from CRAN and Bioconductor. QA passed.

- https://qa.guix.gnu.org/branch/mesa-updates

#73288 I'd love to have that branch merged, it would help to pack more

projects for Astro* soft =).

- https://qa.guix.gnu.org/branch/go-team

#73502 I've rebased it recently and pushed again it looks like most of

the major builds passed successfully, some architectures (aarch64) are

lagged behind. The branch is ready and would let me complete packaging

Prometheus and start large unbundle task.

- https://qa.guix.gnu.org/branch/qt-team

#73515 QA is unhappy: "Unable to check changes between branch and

master. Merge base has not be processed by the data service yet."

- https://qa.guix.gnu.org/branch/wip-gsl-upgrade

#73558 not picked up yet: "Unable to check changes between branch and master.

Merge base has not be processed by the data service yet."

- https://qa.guix.gnu.org/branch/lisp-team

#73567 not picked up yet: "Information unavailable"

--

Oleg

Hi,

On 02/10/2024 21:53, Sharlatan Hellseher wrote:

(...)

(...)

(...)

What's the definition of when a branch looks good for merging? Does some

% of substitutes have to be achieved, and for which architectures?

https://qa.guix.gnu.org/branch/r-teamshows 96% for x86_64 which is .4 %

higher than current master [0]. So it's a win by merging it! ;-)

Seriously, it's also at 96% for aarch64-linux (bordeaux). So "it looks

good to me".

If that's the case, what prevents this "just" being merged?

Presumably r-team demonstrated their desire for it to be merged by

opening the merge request ticket. Is it a break in process if someone

else does it? (rather than waiting for them to respond).

https://qa.guix.gnu.org/branch/mesa-updatesdoesn't look as good to me.

Fine on x86_64, but aarch64-linux is lower than master. Unsure if this

is due to the build farms still trying to catch-up?

This looks about the same as Mesa-upates to me.

Is there a way to compare master<-->go-team to see if different packages

are failing?

We shouldn't make master break in new ways by merging right!

Steve / Futurile

[0] https://qa.guix.gnu.org/branch/master

On 02/10/2024 21:53, Sharlatan Hellseher wrote:

There are 231 commits ahead of master.

I propose to break this into groups of 30-50 commits at a time.

Excluding the build-system changes that Lars [0] said they would do.

Rebase on current master and test that they don't make master 'worse' [1]

Each 30-50 can be put into a new branch for a smaller 'merge-train', to

see if it makes merging faster overall.

I did the first 31 and tested them. A new branch and merge-request could

be created for them [2].

I need someone to collaborate with as I can't do that myself.

However, looks like everyone in python-team is busy atm.

I'll cherry-pick the next tranche if someone has the time to work with

me on merging them - otherwise I'm wasting my time :-)

Steve / Futurile

[0] https://issues.guix.gnu.org/71408#8

[1] no additional breakage, though there are quite a few broken packages

[2] https://issues.guix.gnu.org/71408#13

Hello!

Steve George <steve@futurile.net> skribis:

My take is that by filing a “request to merge”, you claim responsibility

for carrying out the work until it’s merged, unless otherwise stated.

To me, R team folks are responsible for merging ‘r-team’ because they’re

the one who know and they haven’t expressed the desire to get it merged

when it’s good on qa.guix (they didn’t click on “auto-merge”, in GitLab

parlance ;-)).

That said, if there’s no feedback from the R team in a timely fashion,

maybe it’s OK to move to the next branch in the queue.

FWIW, I also pinged ??? regarding ‘fonts-split-outputs’. If we can’t

get it merged real soon, we should probably skip it.

Thoughts?

Ludo’.

Hello,

I have just pushed the fonts-split-outputs branch, which had been built

on QA, after rebasing it on master.

Now I have rebased mesa-updates on the (new) master branch and pushed it;

normally it should be picked up and built automatically by QA.

Thanks to you all for your patience!

Andreas

Hello!

Advance in queue

Hope it's ready to proceed.

--

Oleg

John Kehayias via Guix-patches via <guix-patches@gnu.org> writes:

maybe is time to merge?

see https://qa.guix.gnu.org/branch/mesa-updates

ci have x86_64-linux 96.3%, i686-linux 87.7%, powerpc64le-linux 85.5%

bordeaux have x86_64-linux 91.5%, i686-linux 77.8%, armhf-linux 79.4%, aarch64-linux 89.0%.

Is there anything else in the way?

On Sun, Nov 03, 2024 at 06:04:08PM +0800, Z572 wrote:

Comparing them against master and against each other:

x86_64: comparable on ci, slight regression on bordeaux

i686: comparable on ci, regression on bordeaux (91.8 -> 77.8)

aarch64: comparable on ci, regression on bordeaux (97.0 -> 89.0)

armhf: slight regression on bordeaux

ppc64le: comparable on ci and bordeaux

riscv64: regression on bordeaux (62.0 -> 28.2)

I feel like bordeaux will catch-up fairly quickly post merge. However,

we do now have the regression page for bordeaux of master vs

mesa-updates:

However, after spot-checking a few of them to see if there are

substitutes (including gnome and openjdk) it looks like it probably just

needs to be sent through again.

It looks okay to me

--

Efraim Flashner <efraim@flashner.co.il> ????? ?????

GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351

Confidentiality cannot be guaranteed on emails sent or received unencrypted

Hi all,

On Sun, Nov 03, 2024 at 03:02 PM, Efraim Flashner wrote:

I had been keeping a close eye some weeks ago during the initial batch of patches I pushed and I also think everything looks good. I was just waiting for non-x86 substitute coverage which seems to finally be there as noted above after waiting for other branches and recent Berlin issues. I have been running my system on this branch for a couple weeks as well.

However, the other day on IRC there was a comment about (if I remember) Sway hardware acceleration needing newer libva...? I think it was Josselin (cc'ed); apologies if I misremembered as I was traveling.

Is that a blocker? If so, it would be good to have that update (plus likely yet another mesa version bump) so substitutes can be rebuilt. But I also don't want to hold up any other branches longer than necessary as this has already been waiting for some weeks.

John

Efraim Flashner <efraim@flashner.co.il> writes:

bordeaux substitute availability has been going up for mesa-updates, but

it seems like a large number of changes hit master over the last week or

so which are taking priority, so that'll slow any work on mesa-updates.

John Kehayias via Guix-patches via <guix-patches@gnu.org> writes:

i think we can merge this branch first, and setup a new branch to fix/update

libva and mesa, people can use inferior to get have hardware

acceleration package on new branch. WDYT?

On Tue, Nov 05, 2024 at 12:50 AM, Z572 wrote:

Yes, sounds good. I did a rebase and deleted/pushed to mesa-updates. I'll check in about 12 hours and will do the merge assuming it still looks good and no sudden issues.

Ah, did find the log I missed about Sway, seems it will be for the new version https://logs.guix.gnu.org/guix/2024-10-31.log#120948. So, I will do the merge and then we can do a new branch for libva, sway, etc. (and keep up with mesa). Probably will just put it in the queue once build slots open up as I would like to (every time) keep it small and quick to keep it from becoming harder to manage.

Thanks all!

John

On Tue, Nov 05, 2024 at 04:15:54AM +0000, John Kehayias wrote:

As expected, no change in the derivation for

aarch64/armhf/riscv64/ppc64le

--

Efraim Flashner <efraim@flashner.co.il> ????? ?????

GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351

Confidentiality cannot be guaranteed on emails sent or received unencrypted

On Mon, Nov 04, 2024 at 11:15 PM, John Kehayias wrote:

Merged! Closing. Thanks all.