Download all issues so I can search?

  • Done
  • quality assurance status badge
Details
8 participants
  • Bengt Richter
  • Felix Lechner
  • Maxim Cournoyer
  • Tobias Geerinckx-Rice
  • Michael Rohleder
  • Ricardo Wurmus
  • Yasuaki Kudo
  • zimoun
Owner
unassigned
Submitted by
Yasuaki Kudo
Severity
normal
Y
Y
Yasuaki Kudo wrote on 20 Sep 2020 09:30
(address . bug-guix@gnu.org)
2C07B06D-C525-413D-B105-5A103474ABA3@yasuaki.com
Hi,

If possible, I would like to download all guix issues in the tracker (in other words, database dump?) so I can just use full text search locally. Is it possible?

Cheers,
Yasu
T
T
Tobias Geerinckx-Rice wrote on 20 Sep 2020 19:33
9c8d46eecb649c5bce585926d4aaf322@tobias.gr
Hi!

On 2020-09-20 9:30, Yasuaki Kudo wrote:
Toggle quote (6 lines)
> Hi,
>
> If possible, I would like to download all guix issues in the tracker
> (in other words, database dump?) so I can just use full text search
> locally. Is it possible?

Probably! This is basically what Mumi, the software behind
issues.guix.gnu.org, does: pull all bug-guix and guix-patches mail from
debbugs.gnu.org and index it locally using Mu.

I don't *think* it uses any secret API/authentication/whitelisted IPs
etc. but am not very familiar with its innards.

If you can read Scheme, you can probably figure it out from

Hoping to've been of some help,

T G-R

Sent from a Web browser. Excuse or enjoy my brevity.
R
R
Ricardo Wurmus wrote on 20 Sep 2020 21:32
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
878sd4tfam.fsf@elephly.net
Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

Toggle quote (15 lines)
> Hi!
>
> On 2020-09-20 9:30, Yasuaki Kudo wrote:
>> Hi,
>> If possible, I would like to download all guix issues in the tracker
>> (in other words, database dump?) so I can just use full text search
>> locally. Is it possible?
>
> Probably! This is basically what Mumi, the software behind
> issues.guix.gnu.org, does: pull all bug-guix and guix-patches mail
> from debbugs.gnu.org and index it locally using Mu.
>
> I don't *think* it uses any secret API/authentication/whitelisted IPs
> etc. but am not very familiar with its innards.

Actually it does exactly that (whitelisted IP doing rsync), ever since
The Indicident. I used to just fetch all mbox files for all Guix bugs,
but this put the Debbugs servers under excessive load. After the
server’s IP was blocked from accessing Debbugs we negotiated rsync
access.

We do have full text search on issues.guix.gnu.org. It was brought to
my attention that it’s not yielding expected results since a while,
though, so this would have to be investigated.

--
Ricardo
M
M
Michael Rohleder wrote on 22 Sep 2020 00:27
(name . Yasuaki Kudo)(address . yasu@yasuaki.com)(address . 43528@debbugs.gnu.org)
87r1quhikf.fsf@rohleder.de
Here's [1] an archive of the ML in mbox format.
Maybe this helps if you want to grep for things.



--
Das Leben kann nur rückwärts verstanden werden,
aber es muss vorwärts gelebt werden.
– Søren Kierkegaard
-----BEGIN PGP SIGNATURE-----

iQFFBAEBCAAvFiEEdV4t5dDVhcUueCgwfHr/vv7yyyUFAl9pKMARHG1pa2VAcm9o
bGVkZXIuZGUACgkQfHr/vv7yyyWevggAv7cmbYNHvL+u1Hzi16qGWxalpv4gULXu
SYWMhkLZ9Ys8FWT6iPfuWJwzhjQag1NC91pt03ItJoaNPBmXD1DLY13LTnZ1IHfE
kSBW48x73Unmvwrkx5N7FV6MtdVnT3mMaaZQ5H1UceugpNDXgvKJ/ZrJQLyk4j/n
+CtDAyoDyEzO7CGiXPQp21UDnKIIyMlcxbfmDOsFrBhre004jn6VKmMdf13gkGz5
1VWsOohfP3UK6mY3t8l77DzO9U/FQ//R0wb0qSM+KyoqbvHibsMttGegt9P/pH/g
Sg21JKoZ7LlGcDlOUFzgxpdzZ6IisMLBjCQDC2XlDAw0cJU5Rg11hg==
=r7o+
-----END PGP SIGNATURE-----

Z
Z
zimoun wrote on 22 Sep 2020 00:54
(name . Yasuaki Kudo)(address . yasu@yasuaki.com)(address . 43528@debbugs.gnu.org)
CAJ3okZ0g25+-wsL2OepE+xMR5QWShmS73hSHsjEmEgYBiOWwAg@mail.gmail.com
Dear,

On Sun, 20 Sep 2020 at 11:17, Yasuaki Kudo <yasu@yasuaki.com> wrote:

Toggle quote (2 lines)
> If possible, I would like to download all guix issues in the tracker (in other words, database dump?) so I can just use full text search locally. Is it possible?

Here [1], unofficial public-inbox mirrors so you can "git clone" all
the issues and search offline. Otherwise, please report the searching
issue you could hit with the searching facilities of the official
<issues.guix.gnu.org>.



All the best,
simon
Z
Z
zimoun wrote on 7 Oct 2020 18:21
tags 43528 notabug
(address . control@debbugs.gnu.org)
877ds280s5.fsf@gmail.com
tags 43528 notabug
thanks
M
M
Maxim Cournoyer wrote on 9 Oct 2020 03:41
Re: bug#43528: Download all issues so I can search?
(name . Michael Rohleder)(address . mike@rohleder.de)
878scgtbut.fsf@gmail.com
Michael Rohleder <mike@rohleder.de> writes:

Toggle quote (6 lines)
> Here's [1] an archive of the ML in mbox format.
> Maybe this helps if you want to grep for things.
>
>
> [1] https://lists.gnu.org/archive/mbox/bug-guix/

That is indeed a good suggestion.

Closing.

Thanks,

Maxim
Closed
B
B
Bengt Richter wrote on 9 Oct 2020 05:37
20201009033753.GA8374@LionPure
On +2020-10-08 21:41:14 -0400, Maxim Cournoyer wrote:
Toggle quote (19 lines)
> Michael Rohleder <mike@rohleder.de> writes:
>
> > Here's [1] an archive of the ML in mbox format.
> > Maybe this helps if you want to grep for things.
> >
> >
> > [1] https://lists.gnu.org/archive/mbox/bug-guix/
>
> That is indeed a good suggestion.
>
> Closing.
>
> Thanks,
>
> Maxim
>
>
>

I discovered that the server supports incremental download using the '-c'
option of wget (so it only downloads what was added to the archive
since you last did it), e.g.,
then
mutt -f 2020-10 # (or gnus equivalent I guess) looks normal.

Probably not news to most devs, but I thought it nice that it works.

(For me anyway :)

--
Regards,
Bengt Richter
F
F
Felix Lechner wrote on 8 Feb 18:07 +0100
(no subject)
(address . control@debbugs.gnu.org)
87sf226g1u.fsf@lease-up.com
unarchive 43528
reassign 43528 mumi
archive 43528
thanks
?
Your comment

This issue is archived.

To comment on this conversation send an email to 43528@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 43528
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch