Mumi inserts spurious underscore in bug title

  • Done
  • quality assurance status badge
Details
4 participants
  • Arun Isaac
  • Felix Lechner
  • Tobias Geerinckx-Rice
  • Ricardo Wurmus
Owner
unassigned
Submitted by
Tobias Geerinckx-Rice
Severity
normal

Debbugs page

Tobias Geerinckx-Rice wrote 4 years ago
(name . Bug Guix)(address . bug-guix@gnu.org)
87bl823x0x.fsf@nckx
Guix,

See http://issues.guix.gnu.org/49114, which Mumi currently
titles

‘guix_ lint’ should catch certificate validation exceptions

There's no underscore in the original subject. It's also rendered
correctly in the grey ‘mail view’ div:

Tobias Geerinckx-Rice wrote seconds ago
‘guix lint’ should catch certificate validation exceptions

Perhaps due to my use of Unicode quotes. There's nothing
suspicious in the HTML.

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYM3XDw0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW153YYBALQ4Y3RVrumeTsKRYMtAVG7rWERFDR4V9SanRTCU
LtxFAQCcuHLMQrkZCH66EWTA3PwemLVyynnF7EtfZ3QZPj7KCg==
=5di5
-----END PGP SIGNATURE-----

Tobias Geerinckx-Rice wrote 4 years ago
878s363wrk.fsf@nckx
Tobias Geerinckx-Rice via Bug reports for GNU Guix 写道：
Toggle quote (2 lines)
> It's also rendered correctly in the grey ‘mail view’ div:

Otherwise I'd suspect guile-email but now I'm not so sure.

Debbugs doesn't do this:

Kind regards,

T G-R
-----BEGIN PGP SIGNATURE-----

iIMEARYKACsWIQT12iAyS4c9C3o4dnINsP+IT1VteQUCYM3YXw0cbWVAdG9iaWFz
LmdyAAoJEA2w/4hPVW1531YBAIn9E8CNLJy01PprUpZZbxBy47b83TalykIsbsPg
fbh2AP4gDjO7y6ueUdjolGOhpfxs4CGblTne1iK3bNVEo+TMCw==
=nDd2
-----END PGP SIGNATURE-----

Ricardo Wurmus wrote 2 years ago
Mumi inserts spurious underscore in bug title
(address . 49115@debbugs.gnu.org)(address . guile-email@systemreboot.net)
87y1qwbdlr.fsf@elephly.net
Toggle quote (7 lines)
> See http://issues.guix.gnu.org/49114,which Mumi currently
> titles
>
> ‘guix_ lint’ should catch certificate validation exceptions
>
> There's no underscore in the original subject.

The debbugs “.log” file for issue 49114 contains this line:

Subject: bug#49114: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?lint=E2=80=99?= should catch certificate validation exceptions

This is MIME Q encoding (similar to “quoted printable” encoding), as
specified in RFC 2047.

The underscore is mentioned in 4.2 (2):

(2) The 8-bit hexadecimal value 20 (e.g., ISO-8859-1 SPACE) may be
represented as "_" (underscore, ASCII 95.). (This character may
not pass through some internetwork mail gateways, but its use
will greatly enhance readability of "Q" encoded data with mail
readers that do not support this encoding.) Note that the "_"
always represents hexadecimal 20, even if the SPACE character
occupies a different code position in the character set in use.

So, the underscore above ought to be replaced with a space. This seems
to be a bug in guile-email.

--
Ricardo
Arun Isaac wrote 2 years ago
(address . guile-email@systemreboot.net)
87ilhungt4.fsf@systemreboot.net
Hi Ricardo,

Thanks for the bug report!

It's very strange, but I'm not able to reproduce this. I tried the
following. bug_49114.mbox is downloaded from

Toggle snippet (13 lines)
(use-modules (email email))

(pk
(parse-email-headers "Subject: bug#49114: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?lint=E2=80=99?= should catch certificate validation exceptions
"))

(for-each (lambda (bv)
(pk (assoc-ref (email-headers (parse-email bv))
'subject)))
(call-with-input-file "bug_49114.mbox"
mbox->emails))

Even at https://issues.guix.gnu.org/49114, only the "bug title" has the
spurious underscore. The subject of the first message does not. Is the
bug title something stored in the xapian index? Could it be that this
was an older bug that has corrupted the xapian index?

If I understand correctly, mumi does not rebuild its xapian index. I
think it should do so from time to time. It would help prevent old bugs
from getting persisted in storage.

Cheers, and wish you a Happy New Year! :-)
Arun
Ricardo Wurmus wrote 2 years ago
(name . Arun Isaac)(address . arunisaac@systemreboot.net)
87ilhq5lyw.fsf@elephly.net
Hi Arun,

Toggle quote (4 lines)
> Thanks for the bug report!
>
> It's very strange, but I'm not able to reproduce this.

Thank you for your analysis!

Toggle quote (5 lines)
> Even at https://issues.guix.gnu.org/49114, only the "bug title" has the
> spurious underscore. The subject of the first message does not. Is the
> bug title something stored in the xapian index? Could it be that this
> was an older bug that has corrupted the xapian index?

As part of #60410 I rebuilt the xapian database (deleted it and then ran
“mumi fetch”), so it’s not that.

[… time passes …]

I think I found the culprit in mumi/debbugs.scm:

(define qp-pattern "=\\?UTF-8\\?Q\\?([^?]+)\\?=")
…
(let ((subject (or (assoc-ref properties "Subject") "")))
(if (string-contains subject "=?UTF-8?Q?")
(or (false-if-exception
(utf8->string
(quoted-printable-decode
(regexp-substitute/global #f qp-pattern
subject 'pre 1 'post))))
subject)
subject))

I’m probably not using quoted-printable-decode correctly. My apologies
for assuming a bug in Guile Email. I changed this with commit
9d7eb3c5efe6427a1a89fb3e4c7165259edff4dd, using parse-email-headers.

Toggle quote (4 lines)
> If I understand correctly, mumi does not rebuild its xapian index. I
> think it should do so from time to time. It would help prevent old bugs
> from getting persisted in storage.

True. I’ll be sure to rebuild the index on the next upgrade.

Toggle quote (2 lines)
> Cheers, and wish you a Happy New Year! :-)

Thank you, and a Happy New Year to you too!

--
Ricardo
Closed
Arun Isaac wrote 2 years ago
[PATCH 0/1] Do not check for MIME encoded words
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20230102174312.9685-1-arunisaac@systemreboot.net
Hi Ricardo,

I have an additional small improvement. guile-email transparently
handles MIME encoded words. We need not check for them. Patch follows.

Regards,
Arun

Arun Isaac (1):
debbugs: Do not check for MIME encoded words in subject.

mumi/debbugs.scm | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)

--
2.38.1
Closed
Arun Isaac wrote 2 years ago
[PATCH 1/1] debbugs: Do not check for MIME encoded words in subject.
(name . Ricardo Wurmus)(address . rekado@elephly.net)
20230102174312.9685-2-arunisaac@systemreboot.net
guile-email transparently handles MIME encoded words. We do not have
to check for them.

* mumi/debbugs.scm (bug-status): Do not check for MIME encoded words
in subject.
---
mumi/debbugs.scm | 13 +++++++------
1 file changed, 7 insertions(+), 6 deletions(-)

Toggle diff (30 lines)
diff --git a/mumi/debbugs.scm b/mumi/debbugs.scm
index 16bff8e..7e95ddf 100644
--- a/mumi/debbugs.scm
+++ b/mumi/debbugs.scm
@@ -1,5 +1,6 @@
;;; mumi -- Mediocre, uh, mail interface
;;; Copyright © 2020 Ricardo Wurmus <rekado@elephly.net>
+;;; Copyright © 2023 Arun Isaac <arunisaac@systemreboot.net>
;;;
;;; This program is free software: you can redistribute it and/or
;;; modify it under the terms of the GNU Affero General Public License
@@ -280,10 +281,10 @@ defaults to 30 days."
(assoc-ref properties "Submitter")
(assoc-ref properties "Owner")
(or (assoc-ref properties "Severity") "normal")
- (let ((subject (or (assoc-ref properties "Subject") "")))
- (if (string-contains subject "=?UTF-8?Q?")
- (match (parse-email-headers (string-append "Subject: " subject "\n"))
- ((('subject . sub) . rest) sub)
- (other subject))
- subject))
+ (assq-ref
+ (parse-email-headers
+ (string-append "Subject: "
+ (or (assoc-ref properties "Subject") "")
+ "\n"))
+ 'subject)
(assoc-ref properties "Tags"))))
--
2.38.1
Closed
Ricardo Wurmus wrote 2 years ago
(name . Arun Isaac)(address . arunisaac@systemreboot.net)
87sfgs4tn8.fsf@elephly.net
Arun Isaac <arunisaac@systemreboot.net> writes:

Toggle quote (6 lines)
> guile-email transparently handles MIME encoded words. We do not have
> to check for them.
>
> * mumi/debbugs.scm (bug-status): Do not check for MIME encoded words
> in subject.

Thank you, I applied it.

--
Ricardo
Closed
Felix Lechner wrote 1 years ago
(no subject)
(address . control@debbugs.gnu.org)
87plx66f7u.fsf@lease-up.com
unarchive 49115
reassign 49115 mumi
archive 49115

unarchive 41906
reassign 41906 mumi
archive 41906

unarchive 60410
reassign 60410 mumi
archive 60410

unarchive 63215
reassign 63215 mumi
archive 63215

unarchive 41098
reassign 41098 mumi
archive 41098

thanks
Felix Lechner wrote 1 years ago
(address . control@debbugs.gnu.org)
875xyf1fhb.fsf@lease-up.com
unarchive 68680
reassign 68680 mumi
archive 68680

unarchive 63802
reassign 63802 mumi
archive 63802

unarchive 63215
reassign 63215 mumi
archive 63215

unarchive 61645
reassign 61645 mumi
archive 61645

unarchive 60410
reassign 60410 mumi
archive 60410

unarchive 60292
reassign 60292 mumi
archive 60292

unarchive 60292
reassign 60292 mumi
archive 60292

unarchive 58573
reassign 58573 mumi
archive 58573

unarchive 54024
reassign 54024 mumi
archive 54024

unarchive 49115
reassign 49115 mumi
archive 49115

unarchive 48160
reassign 48160 mumi
archive 48160

unarchive 47739
reassign 47739 mumi
archive 47739

unarchive 47520
reassign 47520 mumi
archive 47520

unarchive 47121
reassign 47121 mumi
archive 47121

unarchive 45015
reassign 45015 mumi
archive 45015

unarchive 43661
reassign 43661 mumi
archive 43661

unarchiv 43166
reassign 43166 mumi
archive 43166

unarchive 41906
reassign 41906 mumi
archive 41906

unarchive 41098
reassign 41098 mumi
archive 41098

unarchive 39924
reassign 39924 mumi
archive 39924

unarchive 39924
reassign 39924 mumi
archive 39924

unarchive 39924
reassign 39924 mumi
archive 39924

thanks
?
Your comment

This issue is archived.

To comment on this conversation send an email to 49115@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 49115
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help