mongoeye

Console exploration tool for MongoDB written in Go.

Overview

This tool helps you get an overview of the data stored in the MongoDB database. Allows quick schema analysis and some other calculations. The analysis can run locally or in a database using the aggregation framework (require MongoDB 3.5.6+).

Key features

• Fast:  the fastest tool for analyzing the MongoDB scheme
• Single binary:  pre-built binaries for Windows, Linux, and MacOS (Darwin)
• Local analysis:  quick local analysis using a parallel algorithm (MongoDB 2.0+)
• Analysis in database:  using the aggregation framework (MongoDB 3.5.6+)
• Rich features:  histograms (value, length, weekday, hour), most frequent values, ...
• Integrable:  table, JSON and YAML output

Demo

Comparison

The speed of the analysis was compared with similar tools on the test collection under the same conditions.

• Studio3T
• MongoDB Compass
• Variety.js
• Schema.js

Installing

Mongoeye is one executable binary file.

You can download the archive from GitHub releases page and extract the binary file for your platform.

Or you can compile binary file from sources:

$ go get github.com/mongoeye/mongoeye
$ cd $GOPATH/src/github.com/mongoeye/mongoeye
$ make

Usage

mongoeye [host] database collection [flags]

The command mongoeye --help lists all available options.

Table output

Default output format is table that shows only documents schema.

It does not show histograms and other analysis. Example table output:

            KEY           │ COUNT  │   %    
────────────────────────────────────────────
  all documents           │ 2548   │        
  analyzed documents      │ 1000   │  39.2  
                          │        │        
  _id ➜ objectId          │ 1000   │ 100.0  
  address                 │ 1000   │ 100.0  
  │ ➜ int                 │    1   │   0.1  
  └╴➜ string              │  999   │  99.9  
  address line 2 ➜ string │ 1000   │ 100.0  
  name ➜ string           │ 1000   │ 100.0  
  outcode ➜ string        │ 1000   │ 100.0  
  postcode ➜ string       │ 1000   │ 100.0  
  rating                  │ 1000   │ 100.0  
  │ ➜ int                 │  523   │  52.3  
  │ ➜ double              │  451   │  45.1  
  └╴➜ string              │   26   │   2.6  
  type_of_food ➜ string   │ 1000   │ 100.0  
  URL ➜ string            │ 1000   │ 100.0  

OK  0.190s (local analysis)
    1000/2548 docs (39.2%)
    9 fields, depth 2

JSON and YAML output

Outputs in JSON and YAML format return the results of all analyzes.

Use --format json or --format yaml flags to set these formats.

For output to a file use the option -F /path/to/file.

All flags

Connection options

--host                    mongodb host (default "localhost:27017")
--connection-mode         connection mode (default "SecondaryPreferred")
--connection-timeout      connection timeout (default 5)
--socket-timeout          socket timeout (default 300)
--sync-timeout            sync timeout (default 300)

Authentication

-u, --user                username for authentication (default "admin")
-p, --password            password for authentication
    --auth-db             auth database (default "admin")
    --auth-mech           auth mechanism

Input options

    --db                  database for analysis
    --col                 collection for analysis
-q, --query               documents query (json)
-s, --scope               all, first:N, last:N, random:N (default "random:1000")
-d, --depth               max depth in nested documents (default 2)

Output options

    --full                all available analyzes
-v, --value               get min, max, avg value
-l, --length              get min, max, avg length
-V, --value-hist          get value histogram
    --value-hist-steps    max steps of value histogram >=3 (default 100)
-L, --length-hist         get length histogram
    --length-hist-steps   max steps of length histogram >=3 (default 100)
-W, --weekday-hist        get weekday histogram for dates
-H, --hour-hist           get hour histogram for dates
    --count-unique        get count of unique values
    --most-freq           get the N most frequent values
    --least-freq          get the N least frequent values
-f, --format              output format: table, json, yaml (default "table")
-F, --file                path to the output file

Other options

-t, --timezone            timezone, eg. UTC, Europe/Berlin (default "local")
    --use-aggregation     analyze with aggregation framework (mongodb 3.5.6+)
    --string-max-length   max string length (default 100)
    --array-max-length    analyze only first N array elements (default 20)
    --concurrency         number of local processes (default 0 = auto)
    --buffer              size of the buffer between local stages (default 5000)
    --batch               size of batch from database (default 500)
    --no-color            disable color output
    --version             show version
-h, --help                show this help

Environment variables

Environment variables can also be used for configuration.

The names of the environment variables have the MONGOEYE_ prefix and match the flags.

Instead of the --count-unique flag, for example, you can use export MONGOEYE_COUNT-UNIQUE=true.

TODO

• Create a shared library for integration into other languages.
• Optimize local and aggregation framework algorithms.
• Create a web interface.

Donation

If is this tool useful to you, so feel free to support its further development.

License

Mongoeye is under the GPL-3.0 license. See the LICENSE file for details.