<#secure method=pgpmime mode=sign>
Ludovic Courtès <ludo@gnu.org> writes:
Toggle quote (85 lines)
> Hello Guix!
>
> This is the formal submission of “Migrating repositories, issues, and
> patches to Codeberg” (GCD 002), a preliminary draft of which I posted
> before the Guix Days⁰.
>
> In accordance with the GCD Process, discussion will end on April 23rd at
> the latest.
>
> I would like to remind everyone that you can try out Codeberg either by
> contributing to one of the Guix-Science repositories¹, or by reviewing
> or making a pull request for a trivial packaging change (and nothing
> more!) in my Guix clone at Codeberg:
>
> https://lists.gnu.org/archive/html/guix-devel/2025-02/msg00313.html
>
> Please do try it especially if you feel reluctant or wonder what the
> workflow would be like.
>
> Ludo’.
>
> ⁰ https://lists.gnu.org/archive/html/guix-devel/2025-01/msg00218.html
> ¹ https://codeberg.org/guix-science
>
> title: Migrating repositories, issues, and patches to Codeberg
> id: 002
> status: submitted
> discussion: https://issues.guix.gnu.org/<number assigned by issue tracker>
> authors: Ludovic Courtès
> sponsors: Tobias Geerinckx-Rice, Ricardo Wurmus
> date-submitted: 2025-02-23
> date: 2025-02-23
> SPDX-License-Identifier: CC-BY-SA-4.0 OR GFDL-1.3-no-invariants-or-later
> ---
>
> # Summary
>
> The contribution workflow in Guix has been facing several challenges:
> difficult onboarding, lack of legibility, complex, unreliable, and
> labor-intensive infrastructure, and lack of automation. All these lead
> to an experience that contributors often find frustrating and hinders
> quality assurance efforts. We propose to address these limitations by
> migrating repositories, issue tracking, and patch tracking to Codeberg,
> a “modern” forge hosted by a non-profit.
>
> # Motivation
>
> To keep track of bug reports and patches, Guix historically chose tools
> that were *simple* in their design:
>
> - bug reports and patches can be sent by plain email, without having
> to create an account or even subscribe to a mailing list;
> - discussion and patch review happen naturally by email, without
> requiring special tools;
> - the Debbugs instance at https://bugs.gnu.org keeps track of bug
> reports and patches by assigning them an identifier and creating a
> mailing list specifically for each bug or patch.
>
> However, to overcome several limitations, the project developed
> processes and tools, which can be characterized as *incidental
> complexity*:
>
> - because the Debbugs web interface is crude by today’s standards and
> hard to search and navigate, the project developed
> [mumi](https://git.savannah.gnu.org/cgit/guix/mumi.git/), the web
> interface running at https://issues.guix.gnu.org;
> - to navigate bugs and patches more conveniently than what an email
> client supports, contributors were
> [encouraged](https://guix.gnu.org/manual/devel/en/html_node/Debbugs-User-Interfaces.html)
> to use interfaces like `debbugs.el` or `b4`;
> - sending patch series by email does not play well with Debbugs’
> automatic identifier assignment, so [contributors were told to send
> their “cover letter”, wait for an identifier to be assigned, and
> then send the
> rest](https://guix.gnu.org/manual/devel/en/html_node/Sending-a-Patch-Series.html#Multiple-Patches-1);
> - to help sending and applying patch series, mumi was extended to
> provide a command line interface;
> - to build patch series submitted by email, the [QA
> service](https://qa.guix.gnu.org) has to rely on a [Patchwork
> instance](https://patches.guix-patches.cbaines.net/project/guix-patches/list/)
> that is subscribed to the `guix-patches` mailing list, coupled with
> its own [parsing of incoming
> email](https://git.savannah.gnu.org/gitweb/?p=guix/data-service.git;a=blob;f=guix-data-service/branch-updated-emails.scm;h=aeb1570dfda725864a77780d0541f26c090b0e55;hb=c886685e9284da4bbed9377f70dd70da9e7ca29f);
> - the project added a commit hook to create add unique `Change-Id`
Possible typo in "create add"? :)
Toggle quote (6 lines)
> headers in commit messages in an attempt to correlate commits in the
> repository with messages send to `guix-patches`; none of the
> existing tools takes advantage of it though, and it is up to
> contributors to manually close entries in the bug/patch tracker once
> they have been fixed/applied.
This seems reasonably simple to do, if this GCD does not pass, I will
take a stab at that.
Toggle quote (7 lines)
>
> Developing and maintaining this software and infrastructure is
> time-consuming. Worse, it leaves contributors largely dissatisfied for
> a variety of reasons:
>
> - the process is unfamiliar to most newcomers;
If the AGit flow will be mandatory, new process will also be unfamiliar.
Just saying.
Toggle quote (52 lines)
> - the tools and infrastructure in Guix have become a maze;
> - apart from the happy few using `debbugs.el` in Emacs, navigating
> open issues and patches is hard; filtering incoming messages is
> equally hard, even for those with 10+ years of experience with
> advanced email tools (Gnus, mu4e, notmuch, b4, etc.);
> - because the various parts of the development process (repository,
> issue tracking, QA automation, `etc/teams.scm`) are largely
> disconnected, even long-time contributors can hardly follow issues
> relevant to them; issues may remain open after they’ve been fixed,
> new activity on an issue may go unnoticed, cross-references among
> issues are not visible in any of the interfaces, etc.
>
> All this contributes to a [poor
> experience](https://guix.gnu.org/en/blog/2025/guix-user-and-contributor-survey-2024-the-results-part-3/)
> for those who choose to contribute despite the barrier to entry,
> probably discourages many to even start contributing, and adds to the
> load of committers and infrastructure maintainers.
>
> # Detailed Design
>
> This section explains the chosen solution among the available options,
> the scope of the proposed migration, a migration path, and an outlook on
> automation.
>
> ## Choice of a Forge
>
> We set out to choose a “modern forge” that supports a pull-request style
> workflow and provides good integration between the repository, the issue
> tracker, and the merge request tracker. Such a system is necessarily
> more *complex* at first glance than the email-based tools we have but
> (1) the increase in complexity is reasonable once we consider the
> incidental complexity of the existing services, as mentioned above, and
> (2) we think the added usage benefits outweigh this increase in
> complexity.
>
> The software behind the forge has to be free software that is
> *plausibly* self-hosted on Guix System—this probably rules out GitLab
> Community Edition and makes [Forgejo](https://forgejo.org/) the main
> contender.
>
> [SourceHut](https://sourcehut.org/), the other interesting option, does
> not offer the same convenience when it comes to dealing with patches and
> runs the risk of reproducing onboarding and integration issues
> surrounding an email-based workflow and “read-only” web interface that
> Guix is already experiencing.
>
> Forgejo has several features to support collaboration among a large
> number of people and on a large code base, including
> [teams](https://docs.codeberg.org/collaborating/create-organization/#teams)
> and [issue and pull request
> templates](https://forgejo.org/docs/latest/user/issue-pull-request-templates/).
How do pull request templates work together with AGit flow that (as far
as I understand it) is being considered to be mandatory?
Toggle quote (27 lines)
> Support for
> [federation](https://forgejo.org/2023-01-10-answering-forgejo-federation-questions/)
> is also under development and is a promising way to avoid
> centralization.
>
> Instead of self-hosting, this GCD suggests using the Forgejo instance on
> codeberg.org, run by the [Codeberg e.V.](https://codeberg.org/about)
> non-profit, registered in Germany. The non-profit has a good track
> record of running codeberg.org with minimal downtime, is [committed to
> supporting free software
> development](https://codeberg.org/Codeberg/org/src/branch/main/en/bylaws.md#preamble),
> [transparent](https://codeberg.org/Codeberg/org), and has governance set
> up to achieve its mission.
>
> The Guix-Science umbrella project [has been using Codeberg for several
> months
> now](https://hpc.guix.info/blog/2025/01/join-the-guix-science-community/),
> which has allowed us to gain confidence in its suitability for a project
> like Guix.
>
> ## Rights and Privileges
>
> Migration should preserve rights and privileges regarding access to the
> repositories. To that end, we propose the following rules:
>
> - Committers to several of the repositories listed above and [Savannah
Maybe "listed below" instead?
Toggle quote (71 lines)
> membership in the [“Owners”
> team](https://docs.codeberg.org/collaborating/create-organization/#teams)
> of the [Guix *organization*](https://codeberg.org/guix). As of this
> writing, only three people are members.
> - Anyone listed the `.guix-authorizations` file of Guix can request
> membership of the https://codeberg.org/guix/guix once it is created.
> - Committers to one of the other repositories can request membership
> of that repository.
>
> In the future, we should extend the [“Commit
> Rights”](https://guix.gnu.org/manual/devel/en/html_node/Commit-Access.html)
> section of the manual to clarify the distinction between being a member
> of the organization and being a member of a specific repository, in a
> specific team.
>
> ## Repository Migration Path
>
> The Guix project at Savannah contains the following repositories:
>
> - [Guix itself](https://git.savannah.gnu.org/git/guix.git);
> - [the bootstrappable.org web
> site](https://git.savannah.gnu.org/git/guix/bootstrappable.git);
> - [the DHCP client in
> Guile](https://git.savannah.gnu.org/git/guix/dhcp.git) (forgotten
> 2015 Google Summer of Code project);
> - [Guile bindings to
> GNUnet](https://git.savannah.gnu.org/git/guix/gnunet.git) (forgotten
> 2015 Google Summer of Code project);
> - [Guix artwork and web
> site](https://git.savannah.gnu.org/git/guix/guix-artwork.git);
> - [Cuirass](https://git.savannah.gnu.org/git/guix/guix-cuirass.git);
> - [“maintenance”
> repository](https://git.savannah.gnu.org/git/guix/maintenance.git)
> (includes Guix System infrastructure configuration, talks, and other
> documents);
> - [scripts for videos presenting
> Guix](https://git.savannah.gnu.org/git/guix/videos.git);
> - [Guix Data
> Service](https://git.savannah.gnu.org/git/guix/data-service.git);
> - [Emacs-Guix](https://git.savannah.gnu.org/git/guix/emacs-guix.git);
> - [Guix Build
> Coordinator](https://git.savannah.gnu.org/git/guix/build-coordinator.git);
> - [nar-herder](https://git.savannah.gnu.org/git/guix/nar-herder.git);
> - [QA
> Frontpage](https://git.savannah.gnu.org/git/guix/qa-frontpage.git);
> - [mumi](https://git.savannah.gnu.org/cgit/guix/mumi.git);
> - [Guix Consensus
> Documents](https://git.savannah.gnu.org/git/guix/guix-consensus-documents.git).
>
> Within **30 days** following acceptance of this GCD, committers would
> migrate all these repositories to https://codeberg.org/guix.
>
> For Guix itself, we would decide on a **flag day** 14 days after
> acceptance of this GCD at the earliest, and 30 days at the latest. On
> that day, the official URL of the Guix repository would become
> https://codeberg.org/guix/guix.git. A commit would reflect that by
> updating:
>
> 1. the `url` field in `.guix-channel`;
> 2. the `%default-channel-url` variable in `(guix channels)`;
> 3. any other reference to the URL that may appear in the repository,
> in particular in the manual.
>
> To ease this migration and possibly future migration, we may add a new
> `git.guix.gnu.org` DNS entry with HTTP redirects to
> `git.savannah.gnu.org` (before migration) and `codeberg.org` (after
> migration); a [patch](https://issues.guix.gnu.org/76296) implementing
> this has been submitted. The `%default-channel-url` variable would
> refer to `https://git.guix.gnu.org/guix.git`.
of whether this GCD is accepted or not. It would allow us to switch the
hosting on moments notice in case the underlying infrastructure goes
down.
Toggle quote (32 lines)
>
> Following this commit, an entry in `etc/news.scm` would explain the
> migration. See [this entry in
> Guix-Science](https://codeberg.org/guix-science/guix-science/commit/fd1b2dacd8d37c9d1939f9dc5a5b74256171ccbd)
> for an example.
>
> The Savannah `guix.git` repository would become a mirror of the one at
> Codeberg, with a script periodically updating it for **at least one
> year** after the switch, as a way to ease migration to the new
> repository for users. Other repositories would be deleted from Savannah
> once migrated, to avoid confusion.
>
> ## Issue Tracker Migration Path
>
> Importing all the issues and patches from Debbugs/mumi into Codeberg
> would be impractical: it would require the development of specific
> tools, would be a lossy process due to the fundamental mismatch between
> plain text email threads and Forgejo issues and pull requests, and would
> bring little in return.
>
> Our proposal is the following:
>
> - https://issues.guix.gnu.org will remain up and running for at least
> **two years** following acceptance of this GCD. Note that once
> issues.guix.gnu.org is down, issues will remain visible at
> https://bugs.gnu.org and email archives will remain visible at
> https://mail.gnu.org.
> - Within **30 days** after acceptance of this GCD, mailing list
> administrators will set up the `bug-guix` and `guix-patches` mailing
> lists in “Emergency Moderation” mode in the Mailman
> interface—meaning that messages will not get through anymore.
This contradicts GCD 001 no? The 001 requires the GCD to be sent as
patch to guix-patches@gnu.org. How will this be handled? Once 002 is
accepted, will we rush out 004 replacing the 001? Will there be enough
time for that? Once you close the guix-patches, you cannot propose a
*new* GCD per the current rules (and you cannot change the rules without
a GCD), so at least the initial posting *must* happen within those 30
days.
On separate note, there are other projects (well, I know just of
Shepherd, are there others?) than Guix using bug-guix and guix-patches,
so maybe this GCD should go a bit into how that will be handled.
Toggle quote (19 lines)
> It
> will still be possible to interact on individual issues via
> `NNN@debbugs.gnu.org`.
> - The switchover will be advertised before it takes place with a post
> to `info-guix@gnu.org`, to `guix-devel@gnu.org`, as well as through
> a blog post.
> - The “Contributing” section of the manual will be updated accordingly
> at that time.
>
> ## User Interfaces
>
> For many contributors, a strength of the email-based workflow is that it
> works out of the browser, possibly offline; we want to preserve that
> comfort as much as possible.
>
> Everything that can be done through Forgejo’s web interface can be done
> *via* its [HTTP
> interface](https://forgejo.org/docs/latest/user/api-usage/).
This is not true. There is no API for getting the Terms of Use.
Changes in those are announced only as a banner on the website (as far
as I can tell).
Toggle quote (13 lines)
> This has
> given rise to several Emacs and command-line interfaces that existing
> contributors may find convenient.
>
> [forgejo-cli](https://codeberg.org/Cyborus/forgejo-cli/) and
> [codeberg-cli](https://codeberg.org/Aviac/codeberg-cli) provide rather
> comprehensive command-line interfaces, as [reported by
> Efraim](https://lists.gnu.org/archive/html/guix-devel/2025-02/msg00057.html).
>
> [fj.el](https://codeberg.org/martianh/fj.el/) is an Emacs interface
> similar to `mastodon.el` that lets you view and comment on issues and
> pull requests, list repositories, view notifications, and so on.
One thing it does not support (based on the README) is the ability to do
an actual code review, which is a bummer.
Toggle quote (46 lines)
> It
> does not support off-line access. It can be set up with something like:
>
> ```lisp
> (with-eval-after-load 'fj
> (setq fj-host "https://codeberg.org")
> (setq fj-user "civodul")
> (setq fj-token
> (funcall (plist-get
> (car
> (auth-source-search :host "codeberg.org/api/v1"
> :user fj-user
> :type 'netrc))
> :secret))))
> ```
>
> … and a line like this one to `~/.authinfo.gpg`:
>
> ```
> machine codeberg.org/api/v1 login civodul password TOKEN
> ```
>
> … where `TOKEN` is the [token obtained from
> Codeberg](https://docs.codeberg.org/advanced/access-token/).
>
> [Magit-Forge](https://github.com/magit/forge/) is another Emacs
> interface to forges, with Magit integration and support for working
> off-line. However, Forgejo support is currently next to nonexistent:
> only `forge-browse` is supported (allowing users to open a browser on a
> forge page).
>
> Besides these interfaces, there is a couple of tricks that can simplify
> the life of contributors and reviewers, out of the browser.
>
> As a contributor, you can create pull requests without first creating a
> fork and then a merge request thanks to the [AGit
> workflow](https://forgejo.org/docs/latest/user/agit-support/). This
> works by passing `git push` the relevant *options*, as in this example:
>
> ```
> git push origin HEAD:refs/for/main \
> -o topic="update-hello" \
> -o title="gnu: hello: Update to 42." \
> -o description='This updates the `hello` package."
> ```
Does this still require a Codeberg account? Can anyone update other
people's pull requests, the way I can currently sent n+1 version of a
patch started by someone else?
Toggle quote (46 lines)
>
> As a reviewer, it is possible to pull references of pending pull
> requests by adding something like this to `.git/config`:
>
> ```
> [remote "pulls"]
> url = git@codeberg.org:org/guix-science/guix-science.git
> fetch = +refs/pull/*/head:refs/remotes/pulls/pr/*
> ```
>
> Running `git fetch pulls` then retrieves references to branches
> corresponding to all the pull requests.
>
> ## Teams
>
> All the teams currently defined in `etc/teams.scm` will be reified as
> [teams](https://docs.codeberg.org/collaborating/create-organization/#teams)
> in the [Guix organization](https://codeberg.org/guix).
>
> All these teams would have read-only access to the repositories, with
> the exception of a new *Committers* team, with read-write access to the
> repository, which would contain all the people who already have [commit
> rights on
> Savannah](https://savannah.gnu.org/project/memberlist.php?group=guix)
> (“on-duty members”).
>
> Team scopes in `etc/teams.scm` will be converted to a `CODEOWNERS` file
> similar to [that found in
> Guix-Science](https://codeberg.org/guix-science/guix-science/src/branch/master/CODEOWNERS).
> That way, pull requests will automatically have them suggested as
> reviewers for changes in their scope.
>
> ## Continuous Integration
>
> Forgejo supports
> [*webhooks*](https://forgejo.org/docs/latest/user/webhooks/), `POST`
> requests that are sent to the server of one’s choice upon events such as
> pull request creation. Cuirass (running at ci.guix.gnu.org) already
> [supports](https://hpc.guix.info/blog/2025/01/join-the-guix-science-community/)
> them and automatically creates a *jobset* when a pull request is made.
> The [QA frontpage](https://qa.guix.gnu.org) and its [Data
> Service](https://data.qa.guix.gnu.org) does not support Forgejo webhooks
> yet but can be extended to do so without too much effort, possibly
> sharing or reusing the Forgejo interface code from Cuirass.
>
> In the Guix repository, we will set up webhooks to trig