Request for merging "mesa-updates" branch

  • Done
  • quality assurance status badge
Details
3 participants
  • Efraim Flashner
  • John Kehayias
  • Christopher Baines
Owner
unassigned
Submitted by
John Kehayias
Severity
normal
J
J
John Kehayias wrote on 18 Dec 2023 06:55
request for merging mesa-updates
(name . guix-patches@gnu.org)(address . guix-patches@gnu.org)
87wmtcqckr.fsf@protonmail.com
As mentioned on guix-devel,
hopefully quick branch to update mesa (new stable release as current
one we have ended up without updates) and get security updates for
xorg-server-xwayland, via https://issues.guix.gnu.org/67136.

That requires a newer xorgproto which rebuilds (most of) the world. As
with libx11 ungrafting, this is due mostly to python as far as I can
tell, via the tk dependency. Unfortunately, it seems that can't be
easily separated out:

Anyway, there are currently those 3 commits (mesa, xorgproto,
xorg-server-xwayland) building away. I'm seeing lots of builds
"failing" with "missing derivation" which I've been manually
restarting. The biggest real failure I've spotted just clicking around
on Cuirass is gtk on i686-linux. A test fails and I have no idea why
and don't see how to disable just that test so far. I tried to see
about updating glib and gtk from the gnome branch to see if that fixes
it, but it required too much local building so far. So, that is one to
keep an eye out. At least the dependencies number around 100.

Still waiting to hear from other people about current branches for
what the order will be in merging, but figured I'd get things building
either way with the CI looking idle.

Thanks!
John

(this message may have gone out earlier from a different address,
please ignore/I'll close that one if it appears later)
C
C
Christopher Baines wrote on 18 Dec 2023 12:41
(no subject)
(address . control@debbugs.gnu.org)
875y0vybxw.fsf@cbaines.net
retitle 67875 Request for merging "mesa-updates" branch
thanks
J
J
John Kehayias wrote on 4 Jan 06:13 +0100
Re: xwayland security updates, to mesa- or core-updates or ?
(name . Efraim Flashner)(address . efraim@flashner.co.il)
87a5pl4r5o.fsf@protonmail.com
Hi Efraim and guix-devel

On Mon, Dec 25, 2023 at 08:44 AM, Efraim Flashner wrote:

Toggle quote (6 lines)
> On Fri, Dec 22, 2023 at 09:19:27AM +0200, Efraim Flashner wrote:
>> On Thu, Dec 21, 2023 at 09:18:50PM +0000, John Kehayias wrote:
>> > Hi all,
>> >
>> > On Mon, Dec 18, 2023 at 12:57 AM, John Kehayias wrote:
>> >
[snip]
Toggle quote (25 lines)
>> >
>> > I haven't seen QA process this branch, so I'm just going with what I
>> > see on Berlin. From the branches overview it shows about 61% last I
>> > saw, compared to 72% for master. Unfortunately, non x86 architectures
>> > are usually better covered by Bordeaux, but I don't know where to get
>> > a sense of that coverage. For what it is worth, Efraim has manually
>> > built xorgproto and mesa at least on powerpc64le, riscv64, without
>> > issues.
>>
>> I had berlin build for powerpc64le and that went without any problems.
>> Locally I built for riscv64 and powerpc and those both built fine. I
>> ran into an issue locally with curl on aarch64 and test 1477(?) which is
>> weird since it's supposed to be skipped but I'm sending it through
>> again. Haven't started armhf yet.
>>
>> > Coverage on x86_64 and i686 seems good from what I can tell. I also
>> > don't think there are any other branches ready to merge, and would
>> > like to give them time to rebuild once these changes hit.
>> >
>> > Any thoughts on when to merge?
>> >
>> > Thanks everyone!
>> > John
>

Coming back to this point, seems Berlin is doing better with building
but I don't see mesa-updates on QA so I'm not sure about non
x86_64/i686-linux coverage. Anyone have any thoughts?

I don't know that I've seen real new failures, as still lots of
"missing derivation" or other transient issues that resolve on forcing
a rebuild.

I don't want to merge to master and have issues with substitute
coverage, but do have to call it at some point or will end up keep
scheduling/waiting for rebuilds to happen anyway.

Thoughts?

Toggle quote (27 lines)
> I've been having trouble with curl on aarch64 again. Looking at this
> snippet from the build log:
>
> test 1477...[Verify that error codes in headers and libcurl-errors.3 are in sync]
>
> 1477: stdout FAILED:
> --- log/1/check-expected 2023-12-22 10:53:51.658667071 +0000
> +++ log/1/check-generated 2023-12-22 10:53:51.658667071 +0000
> @@ -1 +0,0 @@
> -Result[LF]
>
> - abort tests
> test 1475...[-f and 416 with Content-Range: */size]
> --pd---e--- OK (1247 out of 1472, remaining: 00:45, took 5.310s, duration: 04:11)
> test 1474...[HTTP PUT with Expect: 100-continue and 417 response during upload]
> --pd---e--- OK (1246 out of 1472, remaining: 00:48, took 22.794s, duration: 04:29)
> Warning: test1474 result is ignored, but passed!
> ...
> TESTFAIL: These test cases failed: 1477
>
> It looks like 1474 is passing locally and the ~1474 is telling the test
> suite to ignore the result. If that's how ~<number> is interpreted then
> I'd suggest that 1477 is failing hard enough that it's taking the test
> suite with it, not merely ignoring the result. I'll continue poking it
> but right now I'm starting to like the hurd plan of disabling the test
> instead of merely ignoring the result.

Thanks for looking at this Efraim. Looks like a good chunk of the curl
rebuilds did get through, did it look okay on aarch64 and anywhere
else you checked?

John
E
E
Efraim Flashner wrote on 4 Jan 08:34 +0100
(name . John Kehayias)(address . john.kehayias@protonmail.com)
ZZZfiYEI9Z0FWane@3900XT
On Thu, Jan 04, 2024 at 05:13:46AM +0000, John Kehayias wrote:
Toggle quote (50 lines)
> Hi Efraim and guix-devel
>
> On Mon, Dec 25, 2023 at 08:44 AM, Efraim Flashner wrote:
>
> > On Fri, Dec 22, 2023 at 09:19:27AM +0200, Efraim Flashner wrote:
> >> On Thu, Dec 21, 2023 at 09:18:50PM +0000, John Kehayias wrote:
> >> > Hi all,
> >> >
> >> > On Mon, Dec 18, 2023 at 12:57 AM, John Kehayias wrote:
> >> >
> [snip]
> >> >
> >> > I haven't seen QA process this branch, so I'm just going with what I
> >> > see on Berlin. From the branches overview it shows about 61% last I
> >> > saw, compared to 72% for master. Unfortunately, non x86 architectures
> >> > are usually better covered by Bordeaux, but I don't know where to get
> >> > a sense of that coverage. For what it is worth, Efraim has manually
> >> > built xorgproto and mesa at least on powerpc64le, riscv64, without
> >> > issues.
> >>
> >> I had berlin build for powerpc64le and that went without any problems.
> >> Locally I built for riscv64 and powerpc and those both built fine. I
> >> ran into an issue locally with curl on aarch64 and test 1477(?) which is
> >> weird since it's supposed to be skipped but I'm sending it through
> >> again. Haven't started armhf yet.
> >>
> >> > Coverage on x86_64 and i686 seems good from what I can tell. I also
> >> > don't think there are any other branches ready to merge, and would
> >> > like to give them time to rebuild once these changes hit.
> >> >
> >> > Any thoughts on when to merge?
> >> >
> >> > Thanks everyone!
> >> > John
> >
>
> Coming back to this point, seems Berlin is doing better with building
> but I don't see mesa-updates on QA so I'm not sure about non
> x86_64/i686-linux coverage. Anyone have any thoughts?
>
> I don't know that I've seen real new failures, as still lots of
> "missing derivation" or other transient issues that resolve on forcing
> a rebuild.
>
> I don't want to merge to master and have issues with substitute
> coverage, but do have to call it at some point or will end up keep
> scheduling/waiting for rebuilds to happen anyway.
>
> Thoughts?

I've been massaging the aarch64 builds to try to build out to rust,
currently I'm still around cmake. Last time we relied on bayfront for
substitutes, which I'd be okay with again, as long as we can tell that
it's doing alright.

Toggle quote (31 lines)
> > I've been having trouble with curl on aarch64 again. Looking at this
> > snippet from the build log:
> >
> > test 1477...[Verify that error codes in headers and libcurl-errors.3 are in sync]
> >
> > 1477: stdout FAILED:
> > --- log/1/check-expected 2023-12-22 10:53:51.658667071 +0000
> > +++ log/1/check-generated 2023-12-22 10:53:51.658667071 +0000
> > @@ -1 +0,0 @@
> > -Result[LF]
> >
> > - abort tests
> > test 1475...[-f and 416 with Content-Range: */size]
> > --pd---e--- OK (1247 out of 1472, remaining: 00:45, took 5.310s, duration: 04:11)
> > test 1474...[HTTP PUT with Expect: 100-continue and 417 response during upload]
> > --pd---e--- OK (1246 out of 1472, remaining: 00:48, took 22.794s, duration: 04:29)
> > Warning: test1474 result is ignored, but passed!
> > ...
> > TESTFAIL: These test cases failed: 1477
> >
> > It looks like 1474 is passing locally and the ~1474 is telling the test
> > suite to ignore the result. If that's how ~<number> is interpreted then
> > I'd suggest that 1477 is failing hard enough that it's taking the test
> > suite with it, not merely ignoring the result. I'll continue poking it
> > but right now I'm starting to like the hurd plan of disabling the test
> > instead of merely ignoring the result.
>
> Thanks for looking at this Efraim. Looks like a good chunk of the curl
> rebuilds did get through, did it look okay on aarch64 and anywhere
> else you checked?

Looks like I got it working on whichever systems I tested it on and I
today saw it build correctly on Berlin.

--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmWWX4YACgkQQarn3Mo9
g1Hp/g//UBJ313bP+/4vChJZnU4bX2Q3YgTq+a749fJUCgKPqNC5fp2uXsjxUZnF
dh0nbvfg4DJ5dmiPOqNjvzPZ7YurO3kigWn8VPvVAS9tiGNZQRSiRkofADZvW1wS
zbTb4aeAOzQW2uq47y18K8ZuAbx4hYdYTR5M2qQqsQ8aNAPzjBF06H5WOUVigtMY
cxkw0AenmtSxduwrssDdipaAnyMBp1TUck5rtzsonWNuzDvM1E0lkXhR8UcwZ0RI
vBAIKlIugIkd48fUKlx0301YoL4/1hf6Ley9aSmMehsk2O/Cu+5sP1EdfPKLdo5I
GcKYN8Pq5S3DfaLNHej19pw5sqG+vkSYLmi8XauXXQCCfk3WhoNmb32vcHuaKlmE
ARfdycXELS7Gi2hrxtDHu8kk9Z4OEQMiaLYOx/d9lTZU9LMDJc4xsj+eEidHPnjK
1M4I3t4otcGjgGVv6vz12VVKM8G8HQMYSvYFitRfxjUrBm/5GzYjNQseTzXcc3W+
Q+W/xNWZMoK0OlWRTvwxs/3+tVdAwnGLT4TDDuaGrYPOFOkmfIN+OWKwoR9KWp8i
mmm6GqwnhbqxeSpXBFpnBEF1i5SOqEIFdMvaxj5pbE/9a/wgnQXJUkJWCXrRg6js
f9Bk4T3VnG+LRQ7zYzpNI5649H4ELUASk8N0xL1wt9lqaE7olb4=
=KOP9
-----END PGP SIGNATURE-----


J
J
John Kehayias wrote on 8 Jan 06:43 +0100
(name . guix-devel)(address . guix-devel@gnu.org)
87mstg1it6.fsf@protonmail.com
Hi all,

Forgive the top post and please see below/previous messages for
previous updates.

TL;DR: I plan to merge mesa-updates into master today-ish (well,
tomorrow for me at this point).

I've been checking in with Efraim who's been very helpful at trying to
nudge along substitute coverage on non-x86_64 platforms. Unfortunately
looks like we have plateaued a bit on, e.g., aarch64. We haven't been
getting stats from QA for this round, and Berlin looks good for what
it covers (x86) but other architectures are down from what we can
tell.

I don't think there are any fundamental failures at this point but
just lots of "missing derivation" errors (I've restarted so many
manually for x86_64/i686) and builds not completing without restarts.
Or unknown reasons. Given the few weeks I've given this and the risk
of just perpetually doing rebuilds to keep catching up (with then more
updates to push) I think it would be best to merge to master. Mesa and
other bits will continue to move forward as well, so I think it is
time so we can be somewhat timely.

I'd rather not without complete substitute coverage, but given recent
build farm difficulties, and the tools we do have for users (pinning,
weather checks, etc.) I think it is best to call this branch so we can
move on. Gnome has some updates that will need (re)building as well as
trying to move forward with core-updates now too.

This is a case where having some better sense of our users and actual
substitute needs/wants would be helpful (yes, Guix survey!) as well as
recognizing our current infrastructure limits. Here's another vote for
prioritizing infrastructure and making sure QA lives and expands.

Feel free to object to this merge timing, though with the relative
silence in each previous message I take it I can make a call here.

Thanks everyone and hope 2024 is off to a good start! Enjoy the new
mesa with curl and xwayland security updates (no new grafts!).

John

On Thu, Jan 04, 2024 at 12:09 AM, John Kehayias wrote:

Toggle quote (82 lines)
> Hi Efraim and guix-devel
>
> On Mon, Dec 25, 2023 at 08:44 AM, Efraim Flashner wrote:
>
>> On Fri, Dec 22, 2023 at 09:19:27AM +0200, Efraim Flashner wrote:
>>> On Thu, Dec 21, 2023 at 09:18:50PM +0000, John Kehayias wrote:
>>> > Hi all,
>>> >
>>> > On Mon, Dec 18, 2023 at 12:57 AM, John Kehayias wrote:
>>> >
> [snip]
>>> >
>>> > I haven't seen QA process this branch, so I'm just going with what I
>>> > see on Berlin. From the branches overview it shows about 61% last I
>>> > saw, compared to 72% for master. Unfortunately, non x86 architectures
>>> > are usually better covered by Bordeaux, but I don't know where to get
>>> > a sense of that coverage. For what it is worth, Efraim has manually
>>> > built xorgproto and mesa at least on powerpc64le, riscv64, without
>>> > issues.
>>>
>>> I had berlin build for powerpc64le and that went without any problems.
>>> Locally I built for riscv64 and powerpc and those both built fine. I
>>> ran into an issue locally with curl on aarch64 and test 1477(?) which is
>>> weird since it's supposed to be skipped but I'm sending it through
>>> again. Haven't started armhf yet.
>>>
>>> > Coverage on x86_64 and i686 seems good from what I can tell. I also
>>> > don't think there are any other branches ready to merge, and would
>>> > like to give them time to rebuild once these changes hit.
>>> >
>>> > Any thoughts on when to merge?
>>> >
>>> > Thanks everyone!
>>> > John
>>
>
> Coming back to this point, seems Berlin is doing better with building
> but I don't see mesa-updates on QA so I'm not sure about non
> x86_64/i686-linux coverage. Anyone have any thoughts?
>
> I don't know that I've seen real new failures, as still lots of
> "missing derivation" or other transient issues that resolve on forcing
> a rebuild.
>
> I don't want to merge to master and have issues with substitute
> coverage, but do have to call it at some point or will end up keep
> scheduling/waiting for rebuilds to happen anyway.
>
> Thoughts?
>
>> I've been having trouble with curl on aarch64 again. Looking at this
>> snippet from the build log:
>>
>> test 1477...[Verify that error codes in headers and libcurl-errors.3 are in sync]
>>
>> 1477: stdout FAILED:
>> --- log/1/check-expected 2023-12-22 10:53:51.658667071 +0000
>> +++ log/1/check-generated 2023-12-22 10:53:51.658667071 +0000
>> @@ -1 +0,0 @@
>> -Result[LF]
>>
>> - abort tests
>> test 1475...[-f and 416 with Content-Range: */size]
>> --pd---e--- OK (1247 out of 1472, remaining: 00:45, took 5.310s, duration: 04:11)
>> test 1474...[HTTP PUT with Expect: 100-continue and 417 response during upload]
>> --pd---e--- OK (1246 out of 1472, remaining: 00:48, took 22.794s, duration: 04:29)
>> Warning: test1474 result is ignored, but passed!
>> ...
>> TESTFAIL: These test cases failed: 1477
>>
>> It looks like 1474 is passing locally and the ~1474 is telling the test
>> suite to ignore the result. If that's how ~<number> is interpreted then
>> I'd suggest that 1477 is failing hard enough that it's taking the test
>> suite with it, not merely ignoring the result. I'll continue poking it
>> but right now I'm starting to like the hurd plan of disabling the test
>> instead of merely ignoring the result.
>
> Thanks for looking at this Efraim. Looks like a good chunk of the curl
> rebuilds did get through, did it look okay on aarch64 and anywhere
> else you checked?
>
> John
E
E
Efraim Flashner wrote on 8 Jan 09:32 +0100
(name . John Kehayias)(address . john.kehayias@protonmail.com)
ZZuzMs0mdntz9ooz@3900XT
On Mon, Jan 08, 2024 at 05:43:40AM +0000, John Kehayias wrote:
Toggle quote (41 lines)
> Hi all,
>
> Forgive the top post and please see below/previous messages for
> previous updates.
>
> TL;DR: I plan to merge mesa-updates into master today-ish (well,
> tomorrow for me at this point).
>
> I've been checking in with Efraim who's been very helpful at trying to
> nudge along substitute coverage on non-x86_64 platforms. Unfortunately
> looks like we have plateaued a bit on, e.g., aarch64. We haven't been
> getting stats from QA for this round, and Berlin looks good for what
> it covers (x86) but other architectures are down from what we can
> tell.
>
> I don't think there are any fundamental failures at this point but
> just lots of "missing derivation" errors (I've restarted so many
> manually for x86_64/i686) and builds not completing without restarts.
> Or unknown reasons. Given the few weeks I've given this and the risk
> of just perpetually doing rebuilds to keep catching up (with then more
> updates to push) I think it would be best to merge to master. Mesa and
> other bits will continue to move forward as well, so I think it is
> time so we can be somewhat timely.
>
> I'd rather not without complete substitute coverage, but given recent
> build farm difficulties, and the tools we do have for users (pinning,
> weather checks, etc.) I think it is best to call this branch so we can
> move on. Gnome has some updates that will need (re)building as well as
> trying to move forward with core-updates now too.
>
> This is a case where having some better sense of our users and actual
> substitute needs/wants would be helpful (yes, Guix survey!) as well as
> recognizing our current infrastructure limits. Here's another vote for
> prioritizing infrastructure and making sure QA lives and expands.
>
> Feel free to object to this merge timing, though with the relative
> silence in each previous message I take it I can make a call here.
>
> Thanks everyone and hope 2024 is off to a good start! Enjoy the new
> mesa with curl and xwayland security updates (no new grafts!).

To record here more or less what I said on IRC, we're currently at
rust-1.56 or 1.57 on the mesa-teams branch, and we're looking at
probably more than a week to build out to rust itself, and then the
packages which depend on it. Currently, on master, Berlin already is
running behind on building rust, and it wasn't until after the previous
mesa-updates merge that it caught up with building rust.


--
Efraim Flashner <efraim@flashner.co.il> ????? ?????
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEoov0DD5VE3JmLRT3Qarn3Mo9g1EFAmWbsy8ACgkQQarn3Mo9
g1HARBAAiSX9kQxkMpVXRXiVLOneEPVoIhua+XENY+T34G0yXxhIFSERYOZBEwPh
lcwR+iNB7VERB+zhs7ogqnAz1itZNfPCm9ks1Tw5urjMhZepCgTKl4iqOGq3kPbn
y5SjEYhil94w95QNjxVhEB06elFSoiPKbXmX59fB470mJoPXVy1Qbc3QIGAsV856
VhPROiO7yR7JtByQu/NV+Z+e+8sNjoMflLTjV7baes355v41fNwbLGsq4kuMjXQ3
Cb6HIcdS9Q3dKsd/8jaY5QEY4bHq4dl3+g0moWTjqdbYPqjZJnJkmo/vWp41Dn3B
32FS55i3w9EJSDEWX8o1/TQkGsdqFegjlS6qpL5og2zVfAkpSrZgFxGQ08OLNtUp
8StkjHSuEPQ602CKx/JVfDWP3fKtbRNit1CKHO08t7mf1NsUJb91dRs6ucccYRQI
CbXYC672OJiloxpXDJnWE4gHsALFl3i9OUA9KXZwWsl9u49PkOstChZKV2cl95Bu
7HB9evFbGR2dmCdJk9k41J1QPvBws7vGOBTsGjdkvxeVb9PSRjAA0q5JgWzYjgUf
O+76FYUWUWeEzGUdtd2ZlhSQ98D4XTZAQfQDu4E0b5+x0i4sEjjGZGOXcVvKjxCk
qFyr3YfeKQPuMAzSNgWBezfYAFR7TecL9WA4xNH1vHGR1Rxdtl0=
=2mPz
-----END PGP SIGNATURE-----


J
J
John Kehayias wrote on 8 Jan 18:24 +0100
(name . Efraim Flashner)(address . efraim@flashner.co.il)
87le8z20xf.fsf@protonmail.com
On Mon, Jan 08, 2024 at 10:32 AM, Efraim Flashner wrote:

Toggle quote (49 lines)
> On Mon, Jan 08, 2024 at 05:43:40AM +0000, John Kehayias wrote:
>> Hi all,
>>
>> Forgive the top post and please see below/previous messages for
>> previous updates.
>>
>> TL;DR: I plan to merge mesa-updates into master today-ish (well,
>> tomorrow for me at this point).
>>
>> I've been checking in with Efraim who's been very helpful at trying to
>> nudge along substitute coverage on non-x86_64 platforms. Unfortunately
>> looks like we have plateaued a bit on, e.g., aarch64. We haven't been
>> getting stats from QA for this round, and Berlin looks good for what
>> it covers (x86) but other architectures are down from what we can
>> tell.
>>
>> I don't think there are any fundamental failures at this point but
>> just lots of "missing derivation" errors (I've restarted so many
>> manually for x86_64/i686) and builds not completing without restarts.
>> Or unknown reasons. Given the few weeks I've given this and the risk
>> of just perpetually doing rebuilds to keep catching up (with then more
>> updates to push) I think it would be best to merge to master. Mesa and
>> other bits will continue to move forward as well, so I think it is
>> time so we can be somewhat timely.
>>
>> I'd rather not without complete substitute coverage, but given recent
>> build farm difficulties, and the tools we do have for users (pinning,
>> weather checks, etc.) I think it is best to call this branch so we can
>> move on. Gnome has some updates that will need (re)building as well as
>> trying to move forward with core-updates now too.
>>
>> This is a case where having some better sense of our users and actual
>> substitute needs/wants would be helpful (yes, Guix survey!) as well as
>> recognizing our current infrastructure limits. Here's another vote for
>> prioritizing infrastructure and making sure QA lives and expands.
>>
>> Feel free to object to this merge timing, though with the relative
>> silence in each previous message I take it I can make a call here.
>>
>> Thanks everyone and hope 2024 is off to a good start! Enjoy the new
>> mesa with curl and xwayland security updates (no new grafts!).
>
> To record here more or less what I said on IRC, we're currently at
> rust-1.56 or 1.57 on the mesa-teams branch, and we're looking at
> probably more than a week to build out to rust itself, and then the
> packages which depend on it. Currently, on master, Berlin already is
> running behind on building rust, and it wasn't until after the previous
> mesa-updates merge that it caught up with building rust.

Thanks again for your help and watchful eye on this Efraim!

Merged in 7a7c8920aeddaf9ab8d68c572780bc34b404711b.

Thanks everyone, apologies for anyone that needs to wait for
substitutes. Feel free to CC me directly on any breakages due to this
merge but hopefully I didn't miss anything major.

John
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 67875@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 67875
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch