Download all issues so I can search?

  • Done
  • quality assurance status badge
Details
8 participants
  • Bengt Richter
  • Felix Lechner
  • Maxim Cournoyer
  • Tobias Geerinckx-Rice
  • Michael Rohleder
  • Ricardo Wurmus
  • Yasuaki Kudo
  • zimoun
Owner
unassigned
Submitted by
Yasuaki Kudo
Severity
normal

Debbugs page

Yasuaki Kudo wrote 4 years ago
(address . bug-guix@gnu.org)
2C07B06D-C525-413D-B105-5A103474ABA3@yasuaki.com
Hi,

If possible, I would like to download all guix issues in the tracker (in other words, database dump?) so I can just use full text search locally. Is it possible?

Cheers,
Yasu
Tobias Geerinckx-Rice wrote 4 years ago
9c8d46eecb649c5bce585926d4aaf322@tobias.gr
Hi!

On 2020-09-20 9:30, Yasuaki Kudo wrote:
Toggle quote (6 lines)
> Hi,
>
> If possible, I would like to download all guix issues in the tracker
> (in other words, database dump?) so I can just use full text search
> locally. Is it possible?

Probably! This is basically what Mumi, the software behind
issues.guix.gnu.org, does: pull all bug-guix and guix-patches mail from
debbugs.gnu.org and index it locally using Mu.

I don't *think* it uses any secret API/authentication/whitelisted IPs
etc. but am not very familiar with its innards.

If you can read Scheme, you can probably figure it out from

Hoping to've been of some help,

T G-R

Sent from a Web browser. Excuse or enjoy my brevity.
Ricardo Wurmus wrote 4 years ago
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
878sd4tfam.fsf@elephly.net
Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org> writes:

Toggle quote (15 lines)
> Hi!
>
> On 2020-09-20 9:30, Yasuaki Kudo wrote:
>> Hi,
>> If possible, I would like to download all guix issues in the tracker
>> (in other words, database dump?) so I can just use full text search
>> locally. Is it possible?
>
> Probably! This is basically what Mumi, the software behind
> issues.guix.gnu.org, does: pull all bug-guix and guix-patches mail
> from debbugs.gnu.org and index it locally using Mu.
>
> I don't *think* it uses any secret API/authentication/whitelisted IPs
> etc. but am not very familiar with its innards.

Actually it does exactly that (whitelisted IP doing rsync), ever since
The Indicident. I used to just fetch all mbox files for all Guix bugs,
but this put the Debbugs servers under excessive load. After the
server’s IP was blocked from accessing Debbugs we negotiated rsync
access.

We do have full text search on issues.guix.gnu.org. It was brought to
my attention that it’s not yielding expected results since a while,
though, so this would have to be investigated.

--
Ricardo
Michael Rohleder wrote 4 years ago
(name . Yasuaki Kudo)(address . yasu@yasuaki.com)(address . 43528@debbugs.gnu.org)
87r1quhikf.fsf@rohleder.de
Here's [1] an archive of the ML in mbox format.
Maybe this helps if you want to grep for things.



--
Das Leben kann nur rückwärts verstanden werden,
aber es muss vorwärts gelebt werden.
– Søren Kierkegaard
-----BEGIN PGP SIGNATURE-----

iQFFBAEBCAAvFiEEdV4t5dDVhcUueCgwfHr/vv7yyyUFAl9pKMARHG1pa2VAcm9o
bGVkZXIuZGUACgkQfHr/vv7yyyWevggAv7cmbYNHvL+u1Hzi16qGWxalpv4gULXu
SYWMhkLZ9Ys8FWT6iPfuWJwzhjQag1NC91pt03ItJoaNPBmXD1DLY13LTnZ1IHfE
kSBW48x73Unmvwrkx5N7FV6MtdVnT3mMaaZQ5H1UceugpNDXgvKJ/ZrJQLyk4j/n
+CtDAyoDyEzO7CGiXPQp21UDnKIIyMlcxbfmDOsFrBhre004jn6VKmMdf13gkGz5
1VWsOohfP3UK6mY3t8l77DzO9U/FQ//R0wb0qSM+KyoqbvHibsMttGegt9P/pH/g
Sg21JKoZ7LlGcDlOUFzgxpdzZ6IisMLBjCQDC2XlDAw0cJU5Rg11hg==
=r7o+
-----END PGP SIGNATURE-----

zimoun wrote 4 years ago
(name . Yasuaki Kudo)(address . yasu@yasuaki.com)(address . 43528@debbugs.gnu.org)
CAJ3okZ0g25+-wsL2OepE+xMR5QWShmS73hSHsjEmEgYBiOWwAg@mail.gmail.com
Dear,

On Sun, 20 Sep 2020 at 11:17, Yasuaki Kudo <yasu@yasuaki.com> wrote:

Toggle quote (2 lines)
> If possible, I would like to download all guix issues in the tracker (in other words, database dump?) so I can just use full text search locally. Is it possible?

Here [1], unofficial public-inbox mirrors so you can "git clone" all
the issues and search offline. Otherwise, please report the searching
issue you could hit with the searching facilities of the official
<issues.guix.gnu.org>.



All the best,
simon
zimoun wrote 4 years ago
tags 43528 notabug
(address . control@debbugs.gnu.org)
877ds280s5.fsf@gmail.com
tags 43528 notabug
thanks
Maxim Cournoyer wrote 4 years ago
Re: bug#43528: Download all issues so I can search?
(name . Michael Rohleder)(address . mike@rohleder.de)
878scgtbut.fsf@gmail.com
Michael Rohleder <mike@rohleder.de> writes:

Toggle quote (6 lines)
> Here's [1] an archive of the ML in mbox format.
> Maybe this helps if you want to grep for things.
>
>
> [1] https://lists.gnu.org/archive/mbox/bug-guix/

That is indeed a good suggestion.

Closing.

Thanks,

Maxim
Closed
Bengt Richter wrote 4 years ago
20201009033753.GA8374@LionPure
On +2020-10-08 21:41:14 -0400, Maxim Cournoyer wrote:
Toggle quote (19 lines)
> Michael Rohleder <mike@rohleder.de> writes:
>
> > Here's [1] an archive of the ML in mbox format.
> > Maybe this helps if you want to grep for things.
> >
> >
> > [1] https://lists.gnu.org/archive/mbox/bug-guix/
>
> That is indeed a good suggestion.
>
> Closing.
>
> Thanks,
>
> Maxim
>
>
>

I discovered that the server supports incremental download using the '-c'
option of wget (so it only downloads what was added to the archive
since you last did it), e.g.,
then
mutt -f 2020-10 # (or gnus equivalent I guess) looks normal.

Probably not news to most devs, but I thought it nice that it works.

(For me anyway :)

--
Regards,
Bengt Richter
Felix Lechner wrote 1 years ago
(no subject)
(address . control@debbugs.gnu.org)
87sf226g1u.fsf@lease-up.com
unarchive 43528
reassign 43528 mumi
archive 43528
thanks
?
Your comment

This issue is archived.

To comment on this conversation send an email to 43528@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 43528
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help