mock-data

Mock-data

Here are my tables
Load them [with data] for me
I don't care how

Mock-data is the result of a Pivotal internal hackathon in July 2017. The idea behind it is to allow users to test database queries with sets of fake data in any pre-defined table.

With Mock-data users can have

• Their own tables defined with any particular (supported) data types. It's only needed to provide the target table(s) and the number of rows of randomly generated data to insert.
• Create a demo database
• Create n number of table with n number of column
• Custom fit data into the table

An ideal environment to make Mock-data work without any errors would be

• Tables with no constraints
• No custom data types

However, please DO MAKE SURE TO TAKE A BACKUP of your database before you mock data in it as it has not been tested extensively.

Check on the "Known Issues" section below for more information about current identified bugs.

Table of Contents

• Important information and disclaimer
• Supported database engines & data types
  • Database Engine
  • Data types
• How it works
• Usage
• Installation
• Examples
• Known Issues
• Developers / Collaboration
• License
• Authors

Important information and disclaimer

Mock-data idea is to generate fake data in new test cluster and it is NOT TO BE USED IN PRODUCTION ENVIRONMENTS. Please ensure you have a backup of your database before running Mock-data in an environment you can't afford losing.

Supported database engines & data types

Database Engine

• PostgresSQL
• Greenplum Database

Data types

• All datatypes that are listed on the postgres datatype website are supported
• As Greenplum are both base from postgres, the supported postgres datatype also apply in their case

How it works

• PARSES the CLI arguments
• CHECKS if the database connection can be established
• BASED on sub commands i.e either database , table or schema it pull / verifies the tables
• CREATES a backup of all constraints (PK, UK, CK, FK ) and unique indexes (due to cascade nature of the drop constraints)
• STORES this constraint/unique index information in memory and also saves it to the file under $HOME/mock
• REMOVES all the constraints on the table
• STARTS loading random data based on the columns datatype
• READS all the constraints information from memory
• FIXES PK and UK initially
• FIXES FK
• CHECK constraints are ignored (coming soon?)
• LOADS constraints that it had backed up (Mock-data can fail at this stage if its not able to fix the constraint violations)

Usage

$ mock --help
This program generates fake data into a postgres database cluster. 
PLEASE DO NOT run on a mission critical databases

Usage:
  mock [flags]
  mock [command]

Available Commands:
  custom      Controlled mocking of tables
  database    Mock at database level
  help        Help about any command
  schema      Mock at schema level
  tables      Mock at table level

Flags:
  -a, --address string    Hostname where the postgres database lives
  -d, --database string   Database to mock the data
  -q, --dont-prompt       Run without asking for confirmation
  -h, --help              help for mock
  -i, --ignore            Ignore checking and fixing constraints
  -w, --password string   Password for the user to connect to database
  -p, --port int          Port number of the postgres database
  -r, --rows int          Total rows to be faked or mocked (default 10)
  -u, --username string   Username to connect to the database
  -v, --verbose           Enable verbose or debug logging
      --version           version for mock

Use "mock [command] --help" for more information about a command.

Installation

Download the latest release for your OS & Architecture and you're ready to go!

Optional:

You can copy the mock program to the PATH folder, so that you can use the mock from anywhere in the terminal, for eg.s

cp mock-darwin-amd64-v2.0 /usr/local/bin/mock
chmod +x /usr/local/bin/mock

provided /usr/local/bin is part of the $PATH environment variable.

Examples

Here is a simple demo of how the tool works, provide us your table and we will load the data for you

For more examples how to use the tool, please check out the wiki page for categories like

• Look here on how the database connection works
• Read this section on how the subcommand custom works
• Read this section on how the subcommand database works
• Read this section on how the subcommand schema works
• Read this section on how the subcommand tables works

Known Issues

1. We do struggle when recreating constraints, even though we do try to fix the primary key , foreign key, unique key. So there is no guarantee that the tool will fix all the constraints and manual intervention is needed in some cases.
2. If you have a composite unique index where one column is part of foreign key column then there are chances the constraint creation would fail.
3. Fixing CHECK constraints isn't supported due to complexity, so recreating check constraints would fail, use custom subcommand to control the data being inserted
4. On Greenplum Database partition tables are not supported (due to check constraint issues defined above), so use the custom sub command to define the data to be inserted to the column with check constraints
5. Custom data types are not supported, use custom sub command to control the data for that custom data types

Developers / Collaboration

You can sumbit issues or pull request via github and we will try our best to fix them.

To customize this repository, follow the steps

1. Clone the git repository

2. Export the GOPATH

   export GOPATH=<path to the clone repository>
   

3. Install all the dependencies. If you don't have dep installed, follow the instruction from here

   dep ensure
   

4. Make sure you have a demo postgres database to connect.

5. You are all set, you can run it locally using

   go run *.go <commands> <flags.........>
   

6. To build the package use

   /bin/sh build.sh
   

License

The Project is licensed under MIT

Authors