[PATCH] graph: Add '--max-depth'.

  • Done
  • quality assurance status badge
Details
2 participants
  • Ludovic Courtès
  • zimoun
Owner
unassigned
Submitted by
Ludovic Courtès
Severity
normal
L
L
Ludovic Courtès wrote on 17 Sep 2021 10:18
(address . guix-patches@gnu.org)(name . Ludovic Courtès)(address . ludovic.courtes@inria.fr)
20210917081848.14264-1-ludo@gnu.org
From: Ludovic Courtès <ludovic.courtes@inria.fr>

* guix/graph.scm (export-graph): Add #:max-depth and honor it, adding
'depths' argument to 'loop'.
* guix/scripts/graph.scm (%options, show-help): Add '--max-depth'.
(%default-options): Add 'max-depth'.
(guix-graph): Pass #:max-depth to 'export-graph'.
* tests/graph.scm ("package DAG, limited depth"): New test.
* doc/guix.texi (Invoking guix graph): Document it.
---
doc/guix.texi | 14 +++++++++++++
guix/graph.scm | 45 ++++++++++++++++++++++++++----------------
guix/scripts/graph.scm | 11 ++++++++++-
tests/graph.scm | 21 +++++++++++++++++++-
4 files changed, 72 insertions(+), 19 deletions(-)

Hello!

This patch adds a long-overdue ‘--max-depth’ option to ‘guix graph’,
which helps visualization somewhat.

Trimming of nodes beyond the max depth happens at export time. The
implementation is a bit naive (with a list containing the depth of
each node) but performance is mostly unchanged.

Feedback welcome!

Ludo’.

Toggle diff (186 lines)
diff --git a/doc/guix.texi b/doc/guix.texi
index 2fc9687910..6c0a581463 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -12598,6 +12598,20 @@ $ guix graph --path -t references emacs libunistring
/gnu/store/@dots{}-libunistring-0.9.10
@end example
+Sometimes you still want to visualize the graph but would like to trim
+it so it can actually be displayed. One way to do it is via the
+@option{--max-depth} (or @option{-M}) option, which lets you specify the
+maximum depth of the graph. In the example below, we visualize only
+@code{libreoffice} and the nodes whose distance to @code{libreoffice} is
+at most 2:
+
+@example
+guix graph -M 2 libreoffice | xdot -f fdp -
+@end example
+
+Mind you, that's still a big ball of spaghetti, but at least
+@command{dot} can render it quickly and it can be browsed somewhat.
+
The available options are the following:
@table @option
diff --git a/guix/graph.scm b/guix/graph.scm
index 0d4cd83667..3a1cab244b 100644
--- a/guix/graph.scm
+++ b/guix/graph.scm
@@ -337,11 +337,12 @@ nodeArray.push(nodes[\"~a\"]);~%"
(define* (export-graph sinks port
#:key
- reverse-edges? node-type
+ reverse-edges? node-type (max-depth +inf.0)
(backend %graphviz-backend))
"Write to PORT the representation of the DAG with the given SINKS, using the
given BACKEND. Use NODE-TYPE to traverse the DAG. When REVERSE-EDGES? is
-true, draw reverse arrows."
+true, draw reverse arrows. Do not represent nodes whose distance to one of
+the SINKS is greater than MAX-DEPTH."
(match backend
(($ <graph-backend> _ _ emit-prologue emit-epilogue emit-node emit-edge)
(emit-prologue (node-type-name node-type) port)
@@ -349,6 +350,7 @@ true, draw reverse arrows."
(match node-type
(($ <node-type> node-identifier node-label node-edges)
(let loop ((nodes sinks)
+ (depths (make-list (length sinks) 0))
(visited (set)))
(match nodes
(()
@@ -356,20 +358,29 @@ true, draw reverse arrows."
(emit-epilogue port)
(store-return #t)))
((head . tail)
- (mlet %store-monad ((id (node-identifier head)))
- (if (set-contains? visited id)
- (loop tail visited)
- (mlet* %store-monad ((dependencies (node-edges head))
- (ids (mapm %store-monad
- node-identifier
- dependencies)))
- (emit-node id (node-label head) port)
- (for-each (lambda (dependency dependency-id)
- (if reverse-edges?
- (emit-edge dependency-id id port)
- (emit-edge id dependency-id port)))
- dependencies ids)
- (loop (append dependencies tail)
- (set-insert id visited)))))))))))))
+ (match depths
+ ((depth . depths)
+ (mlet %store-monad ((id (node-identifier head)))
+ (if (set-contains? visited id)
+ (loop tail depths visited)
+ (mlet* %store-monad ((dependencies
+ (if (= depth max-depth)
+ (return '())
+ (node-edges head)))
+ (ids
+ (mapm %store-monad
+ node-identifier
+ dependencies)))
+ (emit-node id (node-label head) port)
+ (for-each (lambda (dependency dependency-id)
+ (if reverse-edges?
+ (emit-edge dependency-id id port)
+ (emit-edge id dependency-id port)))
+ dependencies ids)
+ (loop (append dependencies tail)
+ (append (make-list (length dependencies)
+ (+ 1 depth))
+ depths)
+ (set-insert id visited)))))))))))))))
;;; graph.scm ends here
diff --git a/guix/scripts/graph.scm b/guix/scripts/graph.scm
index 66de824ef4..439fae0b52 100644
--- a/guix/scripts/graph.scm
+++ b/guix/scripts/graph.scm
@@ -500,6 +500,10 @@ package modules, while attempting to retain user package modules."
(lambda (opt name arg result)
(alist-cons 'backend (lookup-backend arg)
result)))
+ (option '(#\M "max-depth") #t #f
+ (lambda (opt name arg result)
+ (alist-cons 'max-depth (string->number* arg)
+ result)))
(option '("list-backends") #f #f
(lambda (opt name arg result)
(list-backends)
@@ -537,6 +541,8 @@ Emit a representation of the dependency graph of PACKAGE...\n"))
-t, --type=TYPE represent nodes of the given TYPE"))
(display (G_ "
--list-types list the available graph types"))
+ (display (G_ "
+ --max-depth=DEPTH limit to nodes within distance DEPTH"))
(display (G_ "
--path display the shortest path between the given nodes"))
(display (G_ "
@@ -559,6 +565,7 @@ Emit a representation of the dependency graph of PACKAGE...\n"))
(define %default-options
`((node-type . ,%package-node-type)
(backend . ,%graphviz-backend)
+ (max-depth . +inf.0)
(system . ,(%current-system))))
@@ -582,6 +589,7 @@ Emit a representation of the dependency graph of PACKAGE...\n"))
(with-store store
(let* ((transform (options->transformation opts))
+ (max-depth (assoc-ref opts 'max-depth))
(items (filter-map (match-lambda
(('argument . (? store-path? item))
item)
@@ -613,7 +621,8 @@ nodes (given ~a)~%")
(export-graph (concatenate nodes)
(current-output-port)
#:node-type type
- #:backend backend)))
+ #:backend backend
+ #:max-depth max-depth)))
#:system (assq-ref opts 'system)))))
#t)
diff --git a/tests/graph.scm b/tests/graph.scm
index e374dad1a5..fadac265f9 100644
--- a/tests/graph.scm
+++ b/tests/graph.scm
@@ -1,5 +1,5 @@
;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020 Ludovic Courtès <ludo@gnu.org>
+;;; Copyright © 2015, 2016, 2017, 2018, 2019, 2020, 2021 Ludovic Courtès <ludo@gnu.org>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -94,6 +94,25 @@ edges."
(list p3 p3 p2)
(list p2 p1 p1))))))))
+(test-assert "package DAG, limited depth"
+ (let-values (((backend nodes+edges) (make-recording-backend)))
+ (let* ((p1 (dummy-package "p1"))
+ (p2 (dummy-package "p2" (inputs `(("p1" ,p1)))))
+ (p3 (dummy-package "p3" (inputs `(("p1" ,p1)))))
+ (p4 (dummy-package "p4" (inputs `(("p2" ,p2) ("p3" ,p3))))))
+ (run-with-store %store
+ (export-graph (list p4) 'port
+ #:max-depth 1
+ #:node-type %package-node-type
+ #:backend backend))
+ ;; We should see nothing more than these 3 packages.
+ (let-values (((nodes edges) (nodes+edges)))
+ (and (equal? nodes (map package->tuple (list p4 p2 p3)))
+ (equal? edges
+ (map edge->tuple
+ (list p4 p4)
+ (list p2 p3))))))))
+
(test-assert "reverse package DAG"
(let-values (((backend nodes+edges) (make-recording-backend)))
(run-with-store %store
--
2.33.0
Z
Z
zimoun wrote on 20 Sep 2021 17:00
(name . Ludovic Courtès)(address . ludo@gnu.org)
CAJ3okZ0Jq_2yResxcoeSShibNDgjhUyZ_+DX5R1UnxXaDx1oEA@mail.gmail.com
Hi,

On Fri, 17 Sept 2021 at 10:38, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (14 lines)
> * guix/graph.scm (export-graph): Add #:max-depth and honor it, adding
> 'depths' argument to 'loop'.
> * guix/scripts/graph.scm (%options, show-help): Add '--max-depth'.
> (%default-options): Add 'max-depth'.
> (guix-graph): Pass #:max-depth to 'export-graph'.
> * tests/graph.scm ("package DAG, limited depth"): New test.
> * doc/guix.texi (Invoking guix graph): Document it.
> ---
> doc/guix.texi | 14 +++++++++++++
> guix/graph.scm | 45 ++++++++++++++++++++++++++----------------
> guix/scripts/graph.scm | 11 ++++++++++-
> tests/graph.scm | 21 +++++++++++++++++++-
> 4 files changed, 72 insertions(+), 19 deletions(-)

LGTM!

Toggle quote (4 lines)
> Trimming of nodes beyond the max depth happens at export time. The
> implementation is a bit naive (with a list containing the depth of
> each node) but performance is mostly unchanged.

Well, I do not see how it could be better. :-)
And export time is also walk time, IIUC. :-)


Toggle quote (19 lines)
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 2fc9687910..6c0a581463 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -12598,6 +12598,20 @@ $ guix graph --path -t references emacs libunistring
> /gnu/store/@dots{}-libunistring-0.9.10
> @end example
>
> +Sometimes you still want to visualize the graph but would like to trim
> +it so it can actually be displayed. One way to do it is via the
> +@option{--max-depth} (or @option{-M}) option, which lets you specify the
> +maximum depth of the graph. In the example below, we visualize only
> +@code{libreoffice} and the nodes whose distance to @code{libreoffice} is
> +at most 2:
> +
> +@example
> +guix graph -M 2 libreoffice | xdot -f fdp -
> +@end example

I am not sure 'xdot' is part of the GraphViz toolsuite. Instead,

+@example
+guix graph -M 2 libreoffice | fdp -Tsvg > libreoffice.svg
+@end example


Cheers,
simon
L
L
Ludovic Courtès wrote on 21 Sep 2021 10:44
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 50632@debbugs.gnu.org)
87a6k6nwr1.fsf@gnu.org
Hi,

zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (2 lines)
> On Fri, 17 Sept 2021 at 10:38, Ludovic Courtès <ludo@gnu.org> wrote:

[...]

Toggle quote (6 lines)
>> +@example
>> +guix graph -M 2 libreoffice | xdot -f fdp -
>> +@end example
>
> I am not sure 'xdot' is part of the GraphViz toolsuite. Instead,

True, it’s a separate program, but it’s mentioned since
c2b2c19a7b8b75ef6dd153ca121dd8765cdcd746 because it’s more convenient
IMO.

Thanks for taking a look!

Ludo’.
Z
Z
zimoun wrote on 21 Sep 2021 11:19
(name . Ludovic Courtès)(address . ludo@gnu.org)(address . 50632@debbugs.gnu.org)
86v92ub80e.fsf@gmail.com
Hi Ludo,

On Tue, 21 Sep 2021 at 10:44, Ludovic Courtès <ludo@gnu.org> wrote:

Toggle quote (4 lines)
> True, it’s a separate program, but it’s mentioned since
> c2b2c19a7b8b75ef6dd153ca121dd8765cdcd746 because it’s more convenient
> IMO.

Ah, I should have missed this. However, it does not work out of the
box:

Toggle snippet (21 lines)
$ guix environment --ad-hoc xdot
$ guix graph coreutils | xdot -
Traceback (most recent call last):
File "/gnu/store/gm49bvwdgjpx23wlcfrm8mbf8n75a77n-xdot-1.1/bin/.xdot-real", line 11, in <module>
load_entry_point('xdot==1.1', 'gui_scripts', 'xdot')()
File "/gnu/store/gm49bvwdgjpx23wlcfrm8mbf8n75a77n-xdot-1.1/lib/python3.8/site-packages/xdot/__main__.py", line 70, in main
win = DotWindow(width=width, height=height)
File "/gnu/store/gm49bvwdgjpx23wlcfrm8mbf8n75a77n-xdot-1.1/lib/python3.8/site-packages/xdot/ui/window.py", line 546, in __init__
self.dotwidget = widget or DotWidget()
File "/gnu/store/gm49bvwdgjpx23wlcfrm8mbf8n75a77n-xdot-1.1/lib/python3.8/site-packages/xdot/ui/window.py", line 67, in __init__
self.connect("draw", self.on_draw)
TypeError: <window.DotWidget object at 0x7f465dd1f400 (xdot+ui+window+DotWidget at 0x17b50f0)>: unknown signal name: draw

(.xdot-real:5940): Gtk-WARNING **: 11:09:08.420: A floating object was finalized. This means that someone
called g_object_unref() on an object that had only a floating
reference; the initial floating reference is not owned by anyone
and must be removed with g_object_ref_sink().
guix graph: error: fport_write: Broken pipe
Segmentation fault

That’s why I suggest to keep examples in the manual as simple as
possible. From my point of view, this package should be mentioned but
should not be part of the example.

The core of the comment is when releasing. Examples involving a complex
stack are harder to fix. And from my point of view, release broken
examples in the manual is not acceptable*; for an instance of this, see

*not acceptable: well, it is not GNU high standard; even if we can live
with them. ;-)

Cheers,
simon
L
L
Ludovic Courtès wrote on 21 Sep 2021 15:45
Re: bug#50632: [PATCH] graph: Add '--max-depth'.
(address . 50632-done@debbugs.gnu.org)
87sfxyjb4y.fsf@gnu.org
Ludovic Courtès <ludo@gnu.org> skribis:

Toggle quote (8 lines)
> * guix/graph.scm (export-graph): Add #:max-depth and honor it, adding
> 'depths' argument to 'loop'.
> * guix/scripts/graph.scm (%options, show-help): Add '--max-depth'.
> (%default-options): Add 'max-depth'.
> (guix-graph): Pass #:max-depth to 'export-graph'.
> * tests/graph.scm ("package DAG, limited depth"): New test.
> * doc/guix.texi (Invoking guix graph): Document it.

Pushed as 5b32ad4f6f555d305659cee825879df075b06331 followed by a news
entry!

Ludo’.
Closed
L
L
Ludovic Courtès wrote on 21 Sep 2021 15:49
(name . zimoun)(address . zimon.toutoune@gmail.com)(address . 50632@debbugs.gnu.org)
87o88mjaxx.fsf_-_@gnu.org
zimoun <zimon.toutoune@gmail.com> skribis:

Toggle quote (29 lines)
> On Tue, 21 Sep 2021 at 10:44, Ludovic Courtès <ludo@gnu.org> wrote:
>
>> True, it’s a separate program, but it’s mentioned since
>> c2b2c19a7b8b75ef6dd153ca121dd8765cdcd746 because it’s more convenient
>> IMO.
>
> Ah, I should have missed this. However, it does not work out of the
> box:
>
> $ guix environment --ad-hoc xdot
> $ guix graph coreutils | xdot -
> Traceback (most recent call last):
> File "/gnu/store/gm49bvwdgjpx23wlcfrm8mbf8n75a77n-xdot-1.1/bin/.xdot-real", line 11, in <module>
> load_entry_point('xdot==1.1', 'gui_scripts', 'xdot')()
> File "/gnu/store/gm49bvwdgjpx23wlcfrm8mbf8n75a77n-xdot-1.1/lib/python3.8/site-packages/xdot/__main__.py", line 70, in main
> win = DotWindow(width=width, height=height)
> File "/gnu/store/gm49bvwdgjpx23wlcfrm8mbf8n75a77n-xdot-1.1/lib/python3.8/site-packages/xdot/ui/window.py", line 546, in __init__
> self.dotwidget = widget or DotWidget()
> File "/gnu/store/gm49bvwdgjpx23wlcfrm8mbf8n75a77n-xdot-1.1/lib/python3.8/site-packages/xdot/ui/window.py", line 67, in __init__
> self.connect("draw", self.on_draw)
> TypeError: <window.DotWidget object at 0x7f465dd1f400 (xdot+ui+window+DotWidget at 0x17b50f0)>: unknown signal name: draw
>
> (.xdot-real:5940): Gtk-WARNING **: 11:09:08.420: A floating object was finalized. This means that someone
> called g_object_unref() on an object that had only a floating
> reference; the initial floating reference is not owned by anyone
> and must be removed with g_object_ref_sink().
> guix graph: error: fport_write: Broken pipe
> Segmentation fault

Could you report a bug? This works for me:

Toggle snippet (9 lines)
$ guix describe
Generacio 189 Aug 30 2021 12:09:27 (nuna)
guix f91ae94
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: f91ae9425bb385b60396a544afe27933896b8fa3
$ guix graph coreutils | guix environment --pure -E ^DISPLAY -E ^XAUTH --ad-hoc xdot -- xdot -

Toggle quote (9 lines)
> That’s why I suggest to keep examples in the manual as simple as
> possible. From my point of view, this package should be mentioned but
> should not be part of the example.
>
> The core of the comment is when releasing. Examples involving a complex
> stack are harder to fix. And from my point of view, release broken
> examples in the manual is not acceptable*; for an instance of this, see
> <http://issues.guix.gnu.org/issue/47097>.

I sympathize with the general feeling. I think ‘xdot’ is not that bad
though, plus the first example in that section still uses ‘dot’.

Thanks,
Ludo’.
?