Download all issues so I can search?

DoneSubmitted by Yasuaki Kudo.
Details
7 participants
  • Bengt Richter
  • Maxim Cournoyer
  • Tobias Geerinckx-Rice
  • Michael Rohleder
  • Ricardo Wurmus
  • Yasuaki Kudo
  • zimoun
Owner
unassigned
Severity
normal
Y
Y
Yasuaki Kudo wrote on 20 Sep 2020 09:30
(address . bug-guix@gnu.org)
2C07B06D-C525-413D-B105-5A103474ABA3@yasuaki.com
Hi,
If possible, I would like to download all guix issues in the tracker (in other words, database dump?) so I can just use full text search locally. Is it possible?
Cheers,Yasu
T
T
Tobias Geerinckx-Rice wrote on 20 Sep 2020 19:33
9c8d46eecb649c5bce585926d4aaf322@tobias.gr
Hi!
On 2020-09-20 9:30, Yasuaki Kudo wrote:
Toggle quote (6 lines)> Hi,> > If possible, I would like to download all guix issues in the tracker> (in other words, database dump?) so I can just use full text search> locally. Is it possible?
Probably! This is basically what Mumi, the software behind issues.guix.gnu.org, does: pull all bug-guix and guix-patches mail from debbugs.gnu.org and index it locally using Mu.
I don't *think* it uses any secret API/authentication/whitelisted IPs etc. but am not very familiar with its innards.
If you can read Scheme, you can probably figure it out from https://git.elephly.net/gitweb.cgi?p=software/mumi.git.
Hoping to've been of some help,
T G-R
Sent from a Web browser. Excuse or enjoy my brevity.
R
R
Ricardo Wurmus wrote on 20 Sep 2020 21:32
(name . Tobias Geerinckx-Rice)(address . me@tobias.gr)
878sd4tfam.fsf@elephly.net
Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix@gnu.org> writes:
Toggle quote (15 lines)> Hi!>> On 2020-09-20 9:30, Yasuaki Kudo wrote:>> Hi,>> If possible, I would like to download all guix issues in the tracker>> (in other words, database dump?) so I can just use full text search>> locally. Is it possible?>> Probably! This is basically what Mumi, the software behind> issues.guix.gnu.org, does: pull all bug-guix and guix-patches mail> from debbugs.gnu.org and index it locally using Mu.>> I don't *think* it uses any secret API/authentication/whitelisted IPs> etc. but am not very familiar with its innards.
Actually it does exactly that (whitelisted IP doing rsync), ever sinceThe Indicident. I used to just fetch all mbox files for all Guix bugs,but this put the Debbugs servers under excessive load. After theserver’s IP was blocked from accessing Debbugs we negotiated rsyncaccess.
We do have full text search on issues.guix.gnu.org. It was brought tomy attention that it’s not yielding expected results since a while,though, so this would have to be investigated.
-- Ricardo
M
M
Michael Rohleder wrote on 22 Sep 2020 00:27
(name . Yasuaki Kudo)(address . yasu@yasuaki.com)(address . 43528@debbugs.gnu.org)
87r1quhikf.fsf@rohleder.de
Here's [1] an archive of the ML in mbox format.Maybe this helps if you want to grep for things.

[1] https://lists.gnu.org/archive/mbox/bug-guix/
-- Das Leben kann nur rückwärts verstanden werden,aber es muss vorwärts gelebt werden. – Søren Kierkegaard
-----BEGIN PGP SIGNATURE-----
iQFFBAEBCAAvFiEEdV4t5dDVhcUueCgwfHr/vv7yyyUFAl9pKMARHG1pa2VAcm9obGVkZXIuZGUACgkQfHr/vv7yyyWevggAv7cmbYNHvL+u1Hzi16qGWxalpv4gULXuSYWMhkLZ9Ys8FWT6iPfuWJwzhjQag1NC91pt03ItJoaNPBmXD1DLY13LTnZ1IHfEkSBW48x73Unmvwrkx5N7FV6MtdVnT3mMaaZQ5H1UceugpNDXgvKJ/ZrJQLyk4j/n+CtDAyoDyEzO7CGiXPQp21UDnKIIyMlcxbfmDOsFrBhre004jn6VKmMdf13gkGz51VWsOohfP3UK6mY3t8l77DzO9U/FQ//R0wb0qSM+KyoqbvHibsMttGegt9P/pH/gSg21JKoZ7LlGcDlOUFzgxpdzZ6IisMLBjCQDC2XlDAw0cJU5Rg11hg===r7o+-----END PGP SIGNATURE-----
Z
Z
zimoun wrote on 22 Sep 2020 00:54
(name . Yasuaki Kudo)(address . yasu@yasuaki.com)(address . 43528@debbugs.gnu.org)
CAJ3okZ0g25+-wsL2OepE+xMR5QWShmS73hSHsjEmEgYBiOWwAg@mail.gmail.com
Dear,
On Sun, 20 Sep 2020 at 11:17, Yasuaki Kudo <yasu@yasuaki.com> wrote:
Toggle quote (2 lines)> If possible, I would like to download all guix issues in the tracker (in other words, database dump?) so I can just use full text search locally. Is it possible?
Here [1], unofficial public-inbox mirrors so you can "git clone" allthe issues and search offline. Otherwise, please report the searchingissue you could hit with the searching facilities of the official<issues.guix.gnu.org>.
[1] https://yhetil.org/

All the best,simon
Z
Z
zimoun wrote on 7 Oct 2020 18:21
tags 43528 notabug
(address . control@debbugs.gnu.org)
877ds280s5.fsf@gmail.com
tags 43528 notabugthanks
M
M
Maxim Cournoyer wrote on 9 Oct 2020 03:41
Re: bug#43528: Download all issues so I can search?
(name . Michael Rohleder)(address . mike@rohleder.de)
878scgtbut.fsf@gmail.com
Michael Rohleder <mike@rohleder.de> writes:
Toggle quote (6 lines)> Here's [1] an archive of the ML in mbox format.> Maybe this helps if you want to grep for things.>>> [1] https://lists.gnu.org/archive/mbox/bug-guix/
That is indeed a good suggestion.
Closing.
Thanks,
Maxim
Closed
B
B
Bengt Richter wrote on 9 Oct 2020 05:37
20201009033753.GA8374@LionPure
On +2020-10-08 21:41:14 -0400, Maxim Cournoyer wrote:
Toggle quote (19 lines)> Michael Rohleder <mike@rohleder.de> writes:> > > Here's [1] an archive of the ML in mbox format.> > Maybe this helps if you want to grep for things.> >> >> > [1] https://lists.gnu.org/archive/mbox/bug-guix/> > That is indeed a good suggestion.> > Closing.> > Thanks,> > Maxim> > >
I discovered that the server supports incremental download using the '-c'option of wget (so it only downloads what was added to the archivesince you last did it), e.g., wget -c https://lists.gnu.org/archive/mbox/bug-guix/2020-10then mutt -f 2020-10 # (or gnus equivalent I guess) looks normal.
Probably not news to most devs, but I thought it nice that it works.
(For me anyway :)
-- Regards,Bengt Richter
?