[PATCH] gnu: packages: Add grokmirror.

  • Done
  • quality assurance status badge
Details
2 participants
  • Kyle Meyer
  • Ludovic Courtès
Owner
unassigned
Submitted by
Kyle Meyer
Severity
normal

Debbugs page

Kyle Meyer wrote 4 years ago
(address . guix-patches@gnu.org)
20200926204819.30602-1-kyle@kyleam.com
* gnu/packages/version-control.scm (grokmirror): New variable.
---
gnu/packages/version-control.scm | 39 +++++++++++++++++++++++++++++++-
1 file changed, 38 insertions(+), 1 deletion(-)

Toggle diff (61 lines)
diff --git a/gnu/packages/version-control.scm b/gnu/packages/version-control.scm
index 2ee8247071..68971c60f9 100644
--- a/gnu/packages/version-control.scm
+++ b/gnu/packages/version-control.scm
@@ -7,7 +7,7 @@
;;; Copyright © 2014, 2015, 2016 Mark H Weaver <mhw@netris.org>
;;; Copyright © 2014, 2016, 2019 Eric Bavier <bavier@member.fsf.org>
;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Efraim Flashner <efraim@flashner.co.il>
-;;; Copyright © 2015, 2018 Kyle Meyer <kyle@kyleam.com>
+;;; Copyright © 2015, 2018, 2020 Kyle Meyer <kyle@kyleam.com>
;;; Copyright © 2015, 2017, 2018, 2020 Ricardo Wurmus <rekado@elephly.net>
;;; Copyright © 2016, 2017 Leo Famulari <leo@famulari.name>
;;; Copyright © 2016, 2017, 2018 Nikita <nikita@n0.is>
@@ -2269,6 +2269,43 @@ (define-public myrepos
Mercurial, Bazaar, Darcs, CVS, Fossil, and Veracity.")
(license license:gpl2+)))
+(define-public grokmirror
+ (package
+ (name "grokmirror")
+ (version "2.0.0")
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url (string-append "https://git.kernel.org/pub/scm/"
+ "utils/grokmirror/grokmirror.git"))
+ (commit (string-append "v" version))))
+ (file-name (string-append name "-" version "-checkout"))
+ (sha256
+ (base32 "1cs43vf87x8x5k5ncgiwiclc92a1dvxpg2z6lh6psaiip808gylp"))))
+ (build-system python-build-system)
+ (arguments
+ `(#:tests? #f ; no test suite
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'install 'install-manpages
+ (lambda* (#:key outputs #:allow-other-keys)
+ (let* ((man (string-append (assoc-ref outputs "out")
+ "/man/man1/")))
+ (mkdir-p man)
+ (for-each (lambda (file) (install-file file man))
+ (find-files "." "\\.1$")))
+ #t)))))
+ (propagated-inputs
+ `(("python-requests" ,python-requests)))
+ (home-page
+ "https://git.kernel.org/pub/scm/utils/grokmirror/grokmirror.git")
+ (synopsis "Framework to smartly mirror git repositories")
+ (description "Grokmirror enables replicating large git repository
+collections efficiently. Mirrors decide to clone and update repositories
+based on a manifest file published by servers.")
+ (license license:gpl3+)))
+
(define-public git-annex-remote-rclone
(package
(name "git-annex-remote-rclone")

base-commit: 3876ed32f25e160c47ba046d987ad9097b2bf0cb
--
2.28.0.849.g665beee440
Kyle Meyer wrote 4 years ago
(address . 43637@debbugs.gnu.org)(name . zimoun)(address . zimon.toutoune@gmail.com)
878scww903.fsf@kyleam.com
A recent message on bug-guix [1] mentioned cloning the archives at
https://yhetil.org. public-inbox archives, including those at
yhetil.org, publish a grokmirror-compatible manifest. So, if you're
interested in mirroring the Guix-related archives at yhetil.org for any
reason [2], you can use grokmirror to do so in a way that's nicer than
periodically hitting yhetil.org with a git-fetch call for each
Guix-related inbox.

You can clone (and then later update) Guix-related inboxes with

$ grok-pull -vc /path/to/config

and a config file that looks something like this:

Toggle snippet (16 lines)
[core]
toplevel = /path/to/inboxes
manifest = ${toplevel}/manifest.js.gz
log = ${toplevel}/grok.log
loglevel = info
objstore = ${toplevel}/objstore

[pull]
pull_threads = 2
include = /guix-*

[remote]
site = https://yhetil.org
manifest = ${site}/manifest.js.gz

CAJ3okZ0g25+-wsL2OepE+xMR5QWShmS73hSHsjEmEgYBiOWwAg@mail.gmail.com

[2] to have a fast local copy, to publish your own mirror, to feed the
archives into some other tool...
Ludovic Courtès wrote 4 years ago
Re: [bug#43637] [PATCH] gnu: packages: Add grokmirror.
(name . Kyle Meyer)(address . kyle@kyleam.com)(address . 43637-done@debbugs.gnu.org)
87wo0fdjtn.fsf@gnu.org
Hi,

Kyle Meyer <kyle@kyleam.com> skribis:

Toggle quote (2 lines)
> * gnu/packages/version-control.scm (grokmirror): New variable.

Applied, thanks!

Toggle quote (8 lines)
> A recent message on bug-guix [1] mentioned cloning the archives at
> <https://yhetil.org>. public-inbox archives, including those at
> yhetil.org, publish a grokmirror-compatible manifest. So, if you're
> interested in mirroring the Guix-related archives at yhetil.org for any
> reason [2], you can use grokmirror to do so in a way that's nicer than
> periodically hitting yhetil.org with a git-fetch call for each
> Guix-related inbox.

So yhetil.org stores mailing list archives as a Git repo?

Sounds interesting.

Thanks,
Ludo’.
Closed
Kyle Meyer wrote 4 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 43637-done@debbugs.gnu.org)
87wo0eexbn.fsf@kyleam.com
Ludovic Courtès writes:

Toggle quote (8 lines)
> Hi,
>
> Kyle Meyer <kyle@kyleam.com> skribis:
>
>> * gnu/packages/version-control.scm (grokmirror): New variable.
>
> Applied, thanks!

Thanks for reviewing/applying!

Toggle quote (12 lines)
>> A recent message on bug-guix [1] mentioned cloning the archives at
>> <https://yhetil.org>. public-inbox archives, including those at
>> yhetil.org, publish a grokmirror-compatible manifest. So, if you're
>> interested in mirroring the Guix-related archives at yhetil.org for any
>> reason [2], you can use grokmirror to do so in a way that's nicer than
>> periodically hitting yhetil.org with a git-fetch call for each
>> Guix-related inbox.
>
> So yhetil.org stores mailing list archives as a Git repo?
>
> Sounds interesting.

Yep, messages are stored in a Git repo or, for archives greater than
around one gigabyte, a collection of repos [*]. (A freshly gc'd
guix-devel weighs in at ~140M.)

[*] documentation on public-inbox's v2 format:
Closed
?
Your comment

This issue is archived.

To comment on this conversation send an email to 43637@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 43637
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help