orbnet: GitHub star network
orbnet
is a collection of command line utilities that let you export your starred GitHub repositories as a network and serve it via a simple API.
There are a few command line utilties available in this repo:
dumper
: dumps your GitHub stars as JSON
blobs
grapher
: reads the dumped GitHub
star JSON
blobs and builds a simple Weighted Directed Graph
apisrv
: serves the graph over JSON API
Get started
If you have nix
installed just run
nix build
or
nix develop
You can also run individual commands which are provided as flake apps
nix run ".#apisrv"
nix run ".#dumper"
nix run ".#grapher"
Alternatively, if you have go toolchain installed globally, you can build the binaries using the project Makefile
:
make cmd
cmd
target fetches all dependencies, builds all three binaries and places them into _build
directory:
$ ls -1 ./_build
dumper
grapher
apisrv
HOWTO
Once the binaries have been built you can explore the available command line options using the familiar -help
switch. The below will demonstrate the basic usage of all project utilities.
Prerequisites
Before you proceed furhter you must obtain a GitHub
API token. Once you've got the token you can export it via an environment variable called GITHUB_TOKEN
which is then automatically read by the project utilities.
Optionally, I'd recommend installing GraphViz toolkit that helps exploring the results visually.
dumper: dump GitHub stars data
As described earlier, dumper
"scrapes" GitHub
API stars data and dumps them into JSON
blobs. The data are dumped into standard output by default, but you can also store them in a directory of your choice by passing the path to the output directory via -outdir
command line switch.
# dump data into standard output
./dumper -user milosgajdos
NOTE: foo
must exist before you run the command below!
# dump data into directory foo
./dumper -user milosgajdos -paging 100 -outdir foo/
grapher: build a graph of GitHub stars
grapher
builds the graph from the dumped data. You can "feed" grapher
either by passing the path to the directory that contains the JSON
blobs via -indir
command line option. Alternatively, you can also pipe
the data to the grapher
utility as by default it reads the data from standard input.
Building an in-memory graph is kinda fun, but there is no point of it if you can't visualise the results. grapher
provides -marshal
and -format
command line switches which let you export (marshal) the graph into various data formats:
Graphviz
(see here)
SigmaJS
(see here)
CytoscapeJS
(see here)
networkx
(see here)
gexf
(see here)
jsonapi
serializes the graph into orbnet
API model
# load graph data from dumps directory and output it in GEXF format
./grapher -marshal -indir foo/ -format gexf > repos.gexf
# pipe data from dumper to grapher and dump the graph into GEXF file
./dumper -user milosgajdos | ./grapher -marshal -format gexf > repos.gexf
NOTE: if your graph is large, sfdp
command might take a while to complete
# build the graph, dumpt it in dot format and export it to SVG
./dumper -user milosgajdos -paging 100 | ./grapher -marshal -format dot | sfdp -Tsvg > repos.svg
Alternatively you can run dot
with overlay options that should build a better overlay of data:
./grapher -marshal -indir foo/ -format dot | sfdp -x -Goverlap=scale -Tpng > repos.png
NOTE grapher
builds a Weighted Directed Graph that contains four types of nodes:
owner
: the repo owner
repo
: the name of the repo
topic
: the repo topic
lang
: the dominant programming language as returned by GitHub API
apisrv: serve the GitHub stars graph over a JSON API
apisrv
lets you serve the dumped graph over a JSON API. It even provides swagger
docs on /docs/
endpoint.
You can load the dumped graph via -dsn _path_to_graph.json
cli switch.