6:11 p.m.
Hello all,
There is no reason anybody should remember, but last summer I
mentioned 3 tools that I really wanted to help me deal with packaging
issues. The first was opam2rpm, which I started work on last fall.
It isn't done by a long shot, but it is useful enough that I started
working on tool #2.
Introducing "depict", a Python tool to visualize package dependencies:
https://pagure.io/depict
The name is supposed to suggest "DEpendency PICTure", unless somebody
can come up with a better expansion. :-) I wanted this tool to help
me see package dependency cycles, and find ways of breaking them.
Depict comes with the ability to detect cycles and suggest low-cost
ways of breaking them (for a user-defined notion of "cost"). It's
still alpha quality, but I think it has reached a point where some of
you may find it useful. I welcome suggestions for improvement, and
pull requests are even better.
Depict can currently read dependency information from a local RPM
database, from opam (a collection of OCaml packages), and from a
specialized JSON format. I would like to eventually add a "dnf
repoquery" style front end, as well as one for Maven Central.
Here are some excessively complex examples:
Example 1:
python3 -m depict \
--input=json examples/rpm.js \
--input=rpm --stop dbus-broker:dconf:glibc:gcc:libgcc:systemd-udev \
--exclude
alternatives:bash:ca-certificates:coreutils:crypto-policies:crypto-policies-scripts:device-mapper:dracut:fedora-release-common:fedora-release-workstation:fedora-repos:filesystem:fonts-filesystem:gawk:gdk-pixbuf2-modules:grep:grubby:gtk-update-icon-cache:hwdata:mkpasswd:nss-util:nss-sysinit:os-prober:pam:pcre2-syntax:python-unversioned-command:python3:python3-setuptools:rpm-libs:sed:setup:shadow-utils:shared-mime-info:systemd:tzdata
\
firefox --graph=dag --solver=ppl --output=dot firefox.dot
That says to:
1. read the JSON file examples/rpm.js, which contains some visual styling;
2. read from the RPM database:
a. If any of the packages dbus-broker, dconf, etc. are encountered, add them
to the dependency graph, but don't follow their dependencies any further.
b. If any of the packages alternatives, bash, etc. are encountered, do not
add them to the dependency graph.
c. Add the package firefox and its dependencies to the graph, subject to
(a) and (b).
3. compute a Directed Acyclic Graph (DAG) from the original dependency graph
a. use the PPL solver to find low-cost dependencies to remove
4. write the DAG in dot format to firefox.dot. The dot tool, from the graphviz
package, can then convert it into a variety of formats; e.g.,
dot -Tpdf firefox.dot > firefox.pdf
Example 2:
python3 -m depict --up --input=json examples/rpm.js --input=rpm flint
--output=dot flint.dot
That says to compute "up" dependencies instead of the default "down"
dependencies. "Up" means to find the packages that depend on those
listed on the command line. "Down" means to find packages that are
depended upon by the packages listed on the command line. So this
example finds every package installed on my machine that depends on
flint, directly or indirectly.
Example 3:
python3 -m depict --input=json examples/opam.js \
--input=opam --exclude mirage-xen:mirage-xen-ocaml --stop ocaml \
--omit-edges 'spawn,ppx_expect:bisect_ppx,ocamlformat:functoria-runtime,dune'\
--repository ~/src/opam-repository ocaml:4.12.0 \
async ANSITerminal apron atdgen base64 cmdliner core conf-autoconf conf-gmp \
conf-mpfr conf-sqlite3 conf-zlib ctypes dune elina javalib mlgmpidl mtime \
ocamlfind ocamlgraph ounit parmap ppx_deriving ppx_fields_conv ppx_let \
sawja sqlite3 utop xmlm yojson zarith \
--graph=dag --output=dot infer.dot --output=layered -
That says to:
1. read the JSON file examples/opam.js, which contains some visual styling;
2. read from opam:
a. If the packages mirage-xen or mirage-xen-ocaml are encountered, do not
add them to the dependency graph.
b. If the package ocaml is encountered, do not add any of its dependencies
to the graph.
c. If any opam file tries to add a dependency from spawn to ppx_expect,
from bisect_ppx to ocamlformat, etc. do not add those dependencies to the
graph. I omitted these because they represent dependencies on old
versions.
d. Use a local checkout of opam-repository rather than making HTTP requests
to github. You really, really do NOT want to make HTTP requests to
github to read opam information. That method is sloooooooooooow.
e. Add the packages ocaml, async, etc. and their dependencies to the graph,
subject to (a), (b), and (c). In addition, specify that we want version
4.12.0 of the ocaml package. These are the direct dependencies of
Facebook's Infer tool, in case you are wondering.
3. compute a Directed Acyclic Graph (DAG) from the original dependency graph
4. write the DAG in dot format to infer.dot.
5. write a "layered" description of the package dependencies to standard out.
Layer 0 consists of packages with no dependencies in the graph. Layer 1
contains packages whose dependencies all lie in layer 0, and so on.
The documentation is a horrible mess, and the tests are nearly
nonexistent, which is why I have marked this as alpha quality. The
RPM front end is actually in quite good shape. The opam front end is
coming along, but needs some more work.
Feel free to make pull requests against the TODO file if you see some
way depict can be improved that isn't already listed there.
Regards.
--
Jerry James
http://www.jamezone.org/