Mumi inserts spurious underscore in bug title

  • Done
  • quality assurance status badge
Details
4 participants
  • Arun Isaac
  • Felix Lechner
  • Tobias Geerinckx-Rice
  • Ricardo Wurmus
Owner
unassigned
Submitted by
Tobias Geerinckx-Rice
Severity
normal
T
T
Tobias Geerinckx-Rice wrote on 19 Jun 2021 13:37
(name . Bug Guix)(address . bug-guix@gnu.org)
87bl823x0x.fsf@nckx
Guix,

See http://issues.guix.gnu.org/49114, which Mumi currently
titles

‘guix_ lint’ should catch certificate validation exceptions

There's no underscore in the original subject. It's also rendered
correctly in the grey ‘mail view’ div:

Tobias Geerinckx-Rice wrote seconds ago
‘guix lint’ should catch certificate validation exceptions

Perhaps due to my use of Unicode quotes. There's nothing
suspicious in the HTML.

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYM3XDw0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW153YYBALQ4Y3RVrumeTsKRYMtAVG7rWERFDR4V9SanRTCU
LtxFAQCcuHLMQrkZCH66EWTA3PwemLVyynnF7EtfZ3QZPj7KCg==
=5di5
-----END PGP SIGNATURE-----

T
T
Tobias Geerinckx-Rice wrote on 19 Jun 2021 13:43
878s363wrk.fsf@nckx
Tobias Geerinckx-Rice via Bug reports for GNU Guix ???
Toggle quote (2 lines)
> It's also rendered correctly in the grey ‘mail view’ div:

Otherwise I'd suspect guile-email but now I'm not so sure.

Debbugs doesn't do this:

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYM3YXw0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW1531YBAIn9E8CNLJy01PprUpZZbxBy47b83TalykIsbsPg
fbh2AP4gDjO7y6ueUdjolGOhpfxs4CGblTne1iK3bNVEo+TMCw==
=nDd2
-----END PGP SIGNATURE-----

R
R
Ricardo Wurmus wrote on 24 Dec 2022 23:53
Mumi inserts spurious underscore in bug title
(address . 49115@debbugs.gnu.org)(address . guile-email@systemreboot.net)
87y1qwbdlr.fsf@elephly.net
Toggle quote (7 lines)
> See http://issues.guix.gnu.org/49114,which Mumi currently
> titles
>
> ‘guix_ lint’ should catch certificate validation exceptions
>
> There's no underscore in the original subject.

The debbugs “.log” file for issue 49114 contains this line:

Subject: bug#49114: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?lint=E2=80=99?= should catch certificate validation exceptions

This is MIME Q encoding (similar to “quoted printable” encoding), as
specified in RFC 2047.

The underscore is mentioned in 4.2 (2):

(2) The 8-bit hexadecimal value 20 (e.g., ISO-8859-1 SPACE) may be
represented as "_" (underscore, ASCII 95.). (This character may
not pass through some internetwork mail gateways, but its use
will greatly enhance readability of "Q" encoded data with mail
readers that do not support this encoding.) Note that the "_"
always represents hexadecimal 20, even if the SPACE character
occupies a different code position in the character set in use.

So, the underscore above ought to be replaced with a space. This seems
to be a bug in guile-email.

--
Ricardo
A
A
Arun Isaac wrote on 29 Dec 2022 20:36
(address . guile-email@systemreboot.net)
87ilhungt4.fsf@systemreboot.net
Hi Ricardo,

Thanks for the bug report!

It's very strange, but I'm not able to reproduce this. I tried the
following. bug_49114.mbox is downloaded from

Toggle snippet (13 lines)
(use-modules (email email))

(pk
(parse-email-headers "Subject: bug#49114: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?lint=E2=80=99?= should catch certificate validation exceptions
"))

(for-each (lambda (bv)
(pk (assoc-ref (email-headers (parse-email bv))
'subject)))
(call-with-input-file "bug_49114.mbox"
mbox->emails))

Even at https://issues.guix.gnu.org/49114, only the "bug title" has the
spurious underscore. The subject of the first message does not. Is the
bug title something stored in the xapian index? Could it be that this
was an older bug that has corrupted the xapian index?

If I understand correctly, mumi does not rebuild its xapian index. I
think it should do so from time to time. It would help prevent old bugs
from getting persisted in storage.

Cheers, and wish you a Happy New Year! :-)
Arun
R
R
Ricardo Wurmus wrote on 1 Jan 2023 13:15
(name . Arun Isaac)(address . arunisaac@systemreboot.net)
87ilhq5lyw.fsf@elephly.net
Hi Arun,

Toggle quote (4 lines)
> Thanks for the bug report!
>
> It's very strange, but I'm not able to reproduce this.

Thank you for your analysis!

Toggle quote (5 lines)
> Even at https://issues.guix.gnu.org/49114, only the "bug title" has the
> spurious underscore. The subject of the first message does not. Is the
> bug title something stored in the xapian index? Could it be that this
> was an older bug that has corrupted the xapian index?

As part of #60410 I rebuilt the xapian database (deleted it and then ran
“mumi fetch”), so it’s not that.

[… time passes …]

I think I found the culprit in mumi/debbugs.scm:

(define qp-pattern "=\\?UTF-8\\?Q\\?([^?]+)\\?=")
…
(let ((subject (or (assoc-ref properties "Subject") "")))
(if (string-contains subject "=?UTF-8?Q?")
(or (false-if-exception
(utf8->string
(quoted-printable-decode
(regexp-substitute/global #f qp-pattern
subject 'pre 1 'post))))
subject)
subject))

I’m probably not using quoted-printable-decode correctly. My apologies
for assuming a bug in Guile Email. I changed this with commit
9d7eb3c5efe6427a1a89fb3e4c7165259edff4dd, using parse-email-headers.

Toggle quote (4 lines)
> If I understand correctly, mumi does not rebuild its xapian index. I
> think it should do so from time to time. It would help prevent old bugs
> from getting persisted in storage.

True. I’ll be sure to rebuild the index on the next upgrade.

Toggle quote (2 lines)
> Cheers, and wish you a Happy New Year! :-)

Thank you, and a Happy New Year to you too!

--
Ricardo
Closed
A
A
Arun Isaac wrote on 2 Jan 2023 18:43
[PATCH 0/1] Do not check for MIME encoded words
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20230102174312.9685-1-arunisaac@systemreboot.net
Hi Ricardo,

I have an additional small improvement. guile-email transparently
handles MIME encoded words. We need not check for them. Patch follows.

Regards,
Arun

Arun Isaac (1):
debbugs: Do not check for MIME encoded words in subject.

mumi/debbugs.scm | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)

--
2.38.1
Closed
A
A
Arun Isaac wrote on 2 Jan 2023 18:43
[PATCH 1/1] debbugs: Do not check for MIME encoded words in subject.
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20230102174312.9685-2-arunisaac@systemreboot.net
guile-email transparently handles MIME encoded words. We do not have
to check for them.

* mumi/debbugs.scm (bug-status): Do not check for MIME encoded words
in subject.
---
mumi/debbugs.scm | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)

Toggle diff (30 lines)
diff --git a/mumi/debbugs.scm b/mumi/debbugs.scm
index 16bff8e..7e95ddf 100644
--- a/mumi/debbugs.scm
+++ b/mumi/debbugs.scm
@@ -1,5 +1,6 @@
;;; mumi -- Mediocre, uh, mail interface
;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2023 Arun Isaac <arunisaac@systemreboot.net>
;;;
;;; This program is free software: you can redistribute it and/or
;;; modify it under the terms of the GNU Affero General Public License
@@ -280,10 +281,10 @@ defaults to 30 days."
(assoc-ref properties "Submitter")
(assoc-ref properties "Owner")
(or (assoc-ref properties "Severity") "normal")
- (let ((subject (or (assoc-ref properties "Subject") "")))
- (if (string-contains subject "=?UTF-8?Q?")
- (match (parse-email-headers (string-append "Subject: " subject "\n"))
- ((('subject . sub) . rest) sub)
- (other subject))
- subject))
+ (assq-ref
+ (parse-email-headers
+ (string-append "Subject: "
+ (or (assoc-ref properties "Subject") "")
+ "\n"))
+ 'subject)
(assoc-ref properties "Tags"))))
--
2.38.1
Closed
R
R
Ricardo Wurmus wrote on 2 Jan 2023 20:34
(name . Arun Isaac)(address . arunisaac@systemreboot.net)
87sfgs4tn8.fsf@elephly.net
Arun Isaac <arunisaac@systemreboot.net> writes:

Toggle quote (6 lines)
> guile-email transparently handles MIME encoded words. We do not have
> to check for them.
>
> * mumi/debbugs.scm (bug-status): Do not check for MIME encoded words
> in subject.

Thank you, I applied it.

--
Ricardo
Closed
F
F
Felix Lechner wrote on 8 Feb 18:25 +0100
(no subject)
(address . control@debbugs.gnu.org)
87plx66f7u.fsf@lease-up.com
unarchive 49115
reassign 49115 mumi
archive 49115

unarchive 41906
reassign 41906 mumi
archive 41906

unarchive 60410
reassign 60410 mumi
archive 60410

unarchive 63215
reassign 63215 mumi
archive 63215

unarchive 41098
reassign 41098 mumi
archive 41098

thanks
F
F
Felix Lechner wrote on 23 Feb 14:23 +0100
(address . control@debbugs.gnu.org)
875xyf1fhb.fsf@lease-up.com
unarchive 68680
reassign 68680 mumi
archive 68680

unarchive 63802
reassign 63802 mumi
archive 63802

unarchive 63215
reassign 63215 mumi
archive 63215

unarchive 61645
reassign 61645 mumi
archive 61645

unarchive 60410
reassign 60410 mumi
archive 60410

unarchive 60292
reassign 60292 mumi
archive 60292

unarchive 60292
reassign 60292 mumi
archive 60292

unarchive 58573
reassign 58573 mumi
archive 58573

unarchive 54024
reassign 54024 mumi
archive 54024

unarchive 49115
reassign 49115 mumi
archive 49115

unarchive 48160
reassign 48160 mumi
archive 48160

unarchive 47739
reassign 47739 mumi
archive 47739

unarchive 47520
reassign 47520 mumi
archive 47520

unarchive 47121
reassign 47121 mumi
archive 47121

unarchive 45015
reassign 45015 mumi
archive 45015

unarchive 43661
reassign 43661 mumi
archive 43661

unarchiv 43166
reassign 43166 mumi
archive 43166

unarchive 41906
reassign 41906 mumi
archive 41906

unarchive 41098
reassign 41098 mumi
archive 41098

unarchive 39924
reassign 39924 mumi
archive 39924

unarchive 39924
reassign 39924 mumi
archive 39924

unarchive 39924
reassign 39924 mumi
archive 39924

thanks
?