[PATCH] refresh: Account for overlapping updater coverage.

  • Done
  • quality assurance status badge
Details
3 participants
  • ericbavier
  • Ludovic Courtès
  • swedebugia
Owner
unassigned
Submitted by
ericbavier
Severity
normal

Debbugs page

ericbavier wrote 6 years ago
(address . guix-patches@gnu.org)(name . Eric Bavier)(address . bavier@member.fsf.org)
20181128022737.18091-1-ericbavier@centurylink.net
From: Eric Bavier <bavier@member.fsf.org>

* guix/scripts/refresh.scm (list-updaters-and-exit): Do not assume updater
predicates are disjoint. Track covered packages directly.
---
Hello Guix,

Some of our packages are covered by more than one of our updaters:

scheme@(guile-user)> ,use(gnu packages)(guix packages)(guix upstream)
scheme@(guile-user)> ,use(srfi srfi-1)(srfi srfi-26)
scheme@(guile-user)> (define updaters (force (@@ (guix upstream) %updaters)))
scheme@(guile-user)> (define predicates (map upstream-updater-predicate updaters))
scheme@(guile-user)> (define doubles
(fold-packages
(lambda (pkg result)
(if (> (count (cut <> pkg) predicates) 1)
(cons pkg result)
result))
'()))
scheme@(guile-user)> (length doubles)
$1 = 469
scheme@(guile-user)> (map package-name doubles)
$2 = ("agda" "emacs-agda2-mode" "fribidi" "raincat" "ghc-tasty-quickcheck" ... "ghc-hxt")

It seams mostly packages covered by both the "hackage" and "stackage"
updaters. And Fribidi is a GNU package but hosted on github.

Currently this leads to double-counting while computing total package
coverage and a too optimistic result (by about 5.4%).

The below patch fixes it by tracking the (un)covered packages directly.


guix/scripts/refresh.scm | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)

Toggle diff (39 lines)
diff --git a/guix/scripts/refresh.scm b/guix/scripts/refresh.scm
index 58fc64db1..f7d2cffb7 100644
--- a/guix/scripts/refresh.scm
+++ b/guix/scripts/refresh.scm
@@ -179,24 +179,24 @@ specified with `--select'.\n"))
(let* ((packages (fold-packages cons '()))
(total (length packages)))
- (define covered
- (fold (lambda (updater covered)
- (let ((matches (count (upstream-updater-predicate updater)
- packages)))
+ (define uncovered
+ (fold (lambda (updater uncovered)
+ (let ((matches (filter (upstream-updater-predicate updater)
+ packages)))
;; TRANSLATORS: The parenthetical expression here is rendered
;; like "(42% coverage)" and denotes the fraction of packages
;; covered by the given updater.
(format #t (G_ " - ~a: ~a (~2,1f% coverage)~%")
(upstream-updater-name updater)
(G_ (upstream-updater-description updater))
- (* 100. (/ matches total)))
- (+ covered matches)))
- 0
+ (* 100. (/ (length matches) total)))
+ (lset-difference eq? uncovered matches)))
+ packages
(force %updaters)))
(newline)
(format #t (G_ "~2,1f% of the packages are covered by these updaters.~%")
- (* 100. (/ covered total))))
+ (* 100. (/ (- total (length uncovered)) total))))
(exit 0))
(define (warn-no-updater package)
--
2.19.1
Ludovic Courtès wrote 6 years ago
(address . ericbavier@centurylink.net)
87bm67zsoh.fsf@gnu.org
Hello!

ericbavier@centurylink.net skribis:

Toggle quote (33 lines)
> From: Eric Bavier <bavier@member.fsf.org>
>
> * guix/scripts/refresh.scm (list-updaters-and-exit): Do not assume updater
> predicates are disjoint. Track covered packages directly.
> ---
> Hello Guix,
>
> Some of our packages are covered by more than one of our updaters:
>
> scheme@(guile-user)> ,use(gnu packages)(guix packages)(guix upstream)
> scheme@(guile-user)> ,use(srfi srfi-1)(srfi srfi-26)
> scheme@(guile-user)> (define updaters (force (@@ (guix upstream) %updaters)))
> scheme@(guile-user)> (define predicates (map upstream-updater-predicate updaters))
> scheme@(guile-user)> (define doubles
> (fold-packages
> (lambda (pkg result)
> (if (> (count (cut <> pkg) predicates) 1)
> (cons pkg result)
> result))
> '()))
> scheme@(guile-user)> (length doubles)
> $1 = 469
> scheme@(guile-user)> (map package-name doubles)
> $2 = ("agda" "emacs-agda2-mode" "fribidi" "raincat" "ghc-tasty-quickcheck" ... "ghc-hxt")
>
> It seams mostly packages covered by both the "hackage" and "stackage"
> updaters. And Fribidi is a GNU package but hosted on github.
>
> Currently this leads to double-counting while computing total package
> coverage and a too optimistic result (by about 5.4%).
>
> The below patch fixes it by tracking the (un)covered packages directly.

Oh, good catch. LGTM!

That also means we’ll have to work on our updaters… :-)

Thanks,
Ludo’.
Eric Bavier wrote 6 years ago
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 33535-done@debbugs.gnu.org)
20181210225941.033fb854@centurylink.net
On Thu, 29 Nov 2018 18:42:38 +0100
ludo@gnu.org (Ludovic Courtès) wrote:

Toggle quote (16 lines)
> Hello!
>
> ericbavier@centurylink.net skribis:
>
> > From: Eric Bavier <bavier@member.fsf.org>
> >
> > It seams mostly packages covered by both the "hackage" and "stackage"
> > updaters. And Fribidi is a GNU package but hosted on github.
> >
> > Currently this leads to double-counting while computing total package
> > coverage and a too optimistic result (by about 5.4%).
> >
> > The below patch fixes it by tracking the (un)covered packages directly.
>
> Oh, good catch. LGTM!

Applied in cba7ddcf603455c6692eb50c8bbf203a6bf17ab1

Toggle quote (4 lines)
>
> That also means we’ll have to work on our updaters… :-)
>

Yup.

I have a prototype for an "arch" importer that parses the Archlinux
PKGBUILD files (32% coverage even without any fancy package name
mapping).

I also have a patch to the github updater so that it can update
packages that use git-fetch. This brings the coverage up to 15.9% from
9.6%. I think this will be useful if we keep moving packages away from
github's generated tarballs.

`~Eric
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEoMXjUi7471xkzbfw/XPKxxnTJWYFAlwPRD0ACgkQ/XPKxxnT
JWYEnBAAj+DY+IyomUNHLbzrK36UOHBCJCiSbq/O4reJx53s0Y3+eNxc0Se7S0MG
SxMB0Am/ZUNj70m1bZHOwPG8F/EX1qKwzwQDTONU3TF/vJsJRw2CI4ecq82266aY
7tKSWimf96QYvYdpIEHNwSThk1itkO6w24ffxyrTWwYum/AucDyrP6aWnNWdRP7V
2Kn5hHmRIj1NVS3SLbhJgBVSSYjjMO9llpcc0z3VIoNiDJOe6/H+EnjWGcRKsYqQ
hifhyBTEUEZcuUKTgtxBD0Oto0HNGakgGjlYlCdj8BWMPCwUM2uFG4nKUiGkiiB1
Y3U1rvIMI0evGkp2tSKNoK8SmbYz4BHXTi3cJz08KjIZV0j8FOkfU/g1238n7u0t
zCM0AMU4UAa0qMvvJd+wbjbnuv8hvTtIdli9Fg+tR0yXuiYy4J/22Ro0kmE0OMKC
lH3XLMzHLRVB7+/VP6gY65HmEuutzA/cLppxkJc3uhfC3Raoo87oRoxNlTv+HN11
zhLrR7NRceWGo4wZGLiTY+Fe/UMJdUpiqnuS2RyULDt7rIjStHbdCyQ9ESsdQoT8
ggME6uy297mE2vmQot8T0OCbneuFJzwYGPhck343SF103A90lYoHt5tzbkvFjXqP
FxfAR4V31aT8JcZhPDK2wP/3nGbkIevbTt/qJxRUXXAnkOWg7lo=
=PIYV
-----END PGP SIGNATURE-----


Closed
swedebugia wrote 6 years ago
Re: bug#33535: [PATCH] refresh: Account for overlapping updater coverage.
4f2f820d39d6cbef14ab3f4f68b6d130@riseup.net
On 2018-12-11 05:59, Eric Bavier wrote:
snip

Toggle quote (4 lines)
> I have a prototype for an "arch" importer that parses the Archlinux
> PKGBUILD files (32% coverage even without any fancy package name
> mapping).

That sounds nice Eric. Can I see code somewhere? :)

--
Cheers
Swedebugia
Eric Bavier wrote 6 years ago
(address . swedebugia@riseup.net)(address . 33535@debbugs.gnu.org)
20181212085504.4cee54a5@centurylink.net
On Wed, 12 Dec 2018 03:11:44 -0800
swedebugia@riseup.net wrote:

Toggle quote (10 lines)
> On 2018-12-11 05:59, Eric Bavier wrote:
> snip
>
> > I have a prototype for an "arch" importer that parses the Archlinux
> > PKGBUILD files (32% coverage even without any fancy package name
> > mapping).
>
> That sounds nice Eric. Can I see code somewhere? :)
>

It's on my "feature/arch-updater" branch:


The current commit:


I'm open to suggestions :)

`~Eric
-----BEGIN PGP SIGNATURE-----

iQIzBAEBCgAdFiEEoMXjUi7471xkzbfw/XPKxxnTJWYFAlwRIUgACgkQ/XPKxxnT
JWakeA/9FsKutHzVDsGZQvkr7+cqcMzPJUkvQLLfR5v91G3nRu8jh7gNQzFTHXr3
ekV2Dl45DDdFhnEGTwfCz4bMPhfbgiic93YXlRq3tqzOH41v14mUfJvUR7SZ/T6w
9Y1WkMdLCztIzKpgKbQoCXmnnIQeUdW974eKhB8pA7RONYPH5k8900pRwtcwXKDo
BhHeGuDet5h9BHcXzdJ/hy/VQaqzFUw2bFV35UuhFtiJO+zz208kBxkDHHoTXLCY
0+KlGw3YPHAtgIZpecFwQH8H9Z2QijKv0DQSVtZThpuiRvbYn4s8Sfdq5ySFuZXv
8cRkBWJBxulWLVlgyflo1G7qVlv9uo4CxpGLU9bZvD6ztWz6cxi0/g64Czu5znwC
IjM9El0rjtsMX8KNHyVGE80KP19FdX14u4uCioG2Pefaa9eOSRTtP9cc5tEeYxWC
59Di4z0idJnUM4fQzdAaD5KjcpH5MA2OucN7j5gnPfIAZ3hfl9btCwLxh+JynaxV
5lzA7B1utGzciSeYG6ocWaKmR9fxdT+FV8w+wbp/Ia9++zleETLj9XVXk0qgtDFy
AJTdU4ZjyupXAW77tGFuVaAFbW+Ef1xSEp2OsJ4anOXQOVgsTdiw0TaJlnbGo7C7
OrJ8py+4ROW8tplfAO8/5oyOs2RPP+EvgIqAF2KOGblm8ByZ5qE=
=DN1m
-----END PGP SIGNATURE-----


?
Your comment

This issue is archived.

To comment on this conversation send an email to 33535@debbugs.gnu.org

To respond to this issue using the mumi CLI, first switch to it
mumi current 33535
Then, you may apply the latest patchset in this issue (with sign off)
mumi am -- -s
Or, compose a reply to this issue
mumi compose
Or, send patches to this issue
mumi send-email *.patch
You may also tag this issue. See list of standard tags. For example, to set the confirmed and easy tags
mumi command -t +confirmed -t +easy
Or, remove the moreinfo tag and set the help tag
mumi command -t -moreinfo -t +help