Request for merging "python-team" branch

I haven't been involved in this branch, but I'm opening the request for

merge issue just to keep track of things.

Hello,

(Cc’ing the Python team.)

Christopher Baines <mail@cbaines.net> skribis:

I haven’t been involved either. My understanding is that the main

blocker before merging is assessing the amount of breakage introduced in

this branch¹.

Right now it’s built at 89% on x86_64-linux only, which is a bit less

than on ‘master’ but there are still pending builds. Interested readers

can browse failed builds at:

https://ci.guix.gnu.org/eval/latest/dashboard?spec=python-team

(If you find builds that failed due to transient errors like the

infamous “missing derivation”, you can click on “Action -> Restart”—this

is now available in most cases without privileges.)

An important issue seems to be test failures of ‘python-responses’:

https://ci.guix.gnu.org/build/4738398/details

Excerpt:

Ideas, Pythonistas? :-)

Ludo’.

¹ https://lists.gnu.org/archive/html/guix-devel/2024-05/msg00227.html

Hi,

All failed tests have the same reason:

It looks like the bump to the latest version would resolve it 0.22.0 -> 0.25.2

There are some mentioned in GitHub issues with the same simptoms:

On Sat, 8 Jun 2024 at 15:27, Ludovic Courtès <ludo@gnu.org> wrote:

--

VCS: https://github.incerto.xyz/;https://git.sr.ht/~hellseher/

GPG: 9847 81DE 689C 21C2 6418 0867 76D7 27BF F62C D2B5

… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????

????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??

?????????? ?? ? ????????? ??????? ????? ? ?????????????????.

Hi Python Team, Guix Team at large, and Parenphilic Pythonistas,

This long lived python-team branch as of today has a lot of git conflicts if you try to rebase and/or merge it on to master.

What do you think if we discuss an alternative team branch policy for the future for feature branches that target master?

Here's a tentative proposal, with an example:

Instead of having a single python-team branch, with a wide variety of new Python features, what if we had, a python-team feature branch that we work on relatively quickly?

In other words, we avoid long lived branches but try to merge for example, a new python-team-sphinx branch as soon as the "Sphinx feature" is ready. This python-team-sphinx branch will only contain the work required to bump sphinx to the latest version that we'd like to support. The reason I use python-sphinx as an example is because the python-sphinx package requires a lot of rebuilds across many language ecosystems that use Sphinx for documentation purposes.

I think that keeping the team branches focused on a particular team sub-feature within that team's scope and not using long-lived and largely scoped branches will avoid a ton of frustration trying to fix merge conflicts when/before we announce a request to merge.

We can then focus our focused efforts on iterating over preparing major Python features and packages that require large amounts rebuilds.

WDYT?

jgart

Hi jgart,

Quoting jgart (2024-06-18 22:46:47)

IM(V)HO… the smaller and the more-frequently-rebased the branches, the better!

I still haven’t wrapped my head around the consequences for the CI of rebasing

often —can a simple rebase trigger a world rebuild?—, but this is definitively

something that I do —quite successfully!— on a daily basis in my professional life.

I guess it’s just a matter of giving it a try and see? It’s always possible to

go back to the one-branch-to-rule-them-all `python-team` branch if managing

multiple branches ends up being a pain, right?

I also have no clue how the other teams are managing their branch and their

rebase/merge policy?

--

Tanguy

"jgart" <jgart@dismail.de> writes:

This aligns with the current (and previous) guidance on managing

branches [1]. Providing there's a consistent topic for the branch, any

name is fine (e.g. python-team-sphinx is fine).

1: https://guix.gnu.org/manual/devel/en/html_node/Managing-Patches-and-Branches.html

I'm also hoping the requirement to create a guix-patches issue

requesting to merge the branch when it's created will help avoid the

long lived branches that we've had over the last year.

Hi Christopher,

That's great that my proposal already aligns with the current guidance.

I am going to try to start cherry picking uncontroversial and non failing patches on the python-team branch on to master and removing them from the python-team branch in order to reduce the size of the branch. I can then remove those commits from the python-team branch and force push to update it. What do you think of this approach before I attempt it? I'll wait on your guidance before doing this.

Another approach I can take is to break the branch apart into 2 or 3 smaller topic branches and make requests for those branches to merge one at a time.

Note that I have an underpowered old Thinkpad X230 laptop and will only be able to test branches that are not too large. I tried using QA/CI but it seemed to be in an unknown state the last time that I tried. Any suggestions here for how to more effectively use that QA/CI infrastructure would be much appreciated.

I tried spending a bit of time on trying to resolve the merge conflicts but it quickly lead down a long rabbit hole and I only have a limited time to fix so many merge conflicts. The process is also currently very tedious to do with high amounts of accuracy due to their being

1. Lots of commits that I am not familiar

2. Lots of commits

3. The branch has diverged alot since latest master so things don't cleanly apply anymore

What do you think of this current approach of cherry picking good commits off of the python-team branch in order to reduce the size?

Another issue that I ran into on the branch is that there are commits that are failing. For example, at commit fdef5662cce6a57aa6bfa682c7260b65ce3669b8 (the borgmatic changes), borgmatic fails. Should I remove this commit since it is broken?

all the best,

jgart

Hi,

@jgart Please let me know if you need a quick test of changes, I've got some

computer recourses at home.

Thanks

Oleg

On Tue, 25 Jun 2024 at 18:04, jgart <jgart@dismail.de> wrote:

--

VCS: https://github.incerto.xyz/;https://git.sr.ht/~hellseher/

GPG: 9847 81DE 689C 21C2 6418 0867 76D7 27BF F62C D2B5

… ??? ????? - ???????????? ?????????????? ?????? ??????? ????????

????? ????? ????? ? ??? ??????, ??????????? ????? ???????, ??

?????????? ?? ? ????????? ??????? ????? ? ?????????????????.

Hi,

please go ahead. Divide and conquer is imo a good approach to get this branch merged.

I’m splitting of the build-system-related changes into the

pyproject-build-system branch (again), because these are relatively

self-contained and thus easy to merge into master.

Lars

"jgart" <jgart@dismail.de> writes:

If there are commits on the branch that have been submitted as patches,

you might just be able to look at pushing them to master. If there are

commits that haven't been submitted as patches yet, but could be pushed

to master, it would be good to send them as patches so that QA can take

a look.

And splitting off some commits in to a more specific branch is fine as

well.

QA has been performing recently as it's taking too long to process

revisions. Let me know if things aren't working or if you have questions

though and I can try and help.

I think making sure commits that can go to master, do go to master, is

really helpful to avoid merge conflicts. It's so much more likely to

have conflicts if there are changes that can be made on master, but are

made on a branch, or changes that should be made on a branch but are

pushed to master.

While ideally we'd care about the state at each commit, this doesn't

really happen so it's up to you what you do there.