README
¶
Mongoimport
CLI and go library for importing data from CSV, JSON or XML files into MongoDB.
go run github.com/romnnn/mongoimport/cmd/mongoimport --db-user=root --db-password=example csv <path-to-csv-files>
You can also download pre built binaries from the releases page.
For a list of options, run
go run github.com/romnnn/mongoimport/cmd/mongoimport csv --help
Usage as a library
Using the tool as a standalone CLI tool is great for quick loading of a few files. However, you might need more fine-grained control over what files are imported into which collection or perform additional pre/post processing (e.g. parsing timestamps). For this use case, we offer a very extensivle and modular API for configuring your imports.
import "github.com/romnnn/mongoimport"
// example t.b.a
For more examples, see examples/.
Development
All commits are automatically built and tested on travis-ci.com. In order to pass the required checks it is strongly recommended to install the repositories pre commit hooks (assuming you are in the repositories root):
pip install pre-commit invoke bump2version
pre-commit install
The pre commit hooks will run a number of go tools. Try to run pre-commit run --all-files and check for missing tools. You might need to install some of those:
go get -u golang.org/x/lint/golint
go get -u github.com/fzipp/gocyclo
In order to be found, make sure to include $GOPATH/bin in your $PATH ($GOPATH has to be set of course like export GOPATH="$HOME/go for example).
Before committing, pre-commit is run to make sure all checks pass!
Documentation
¶
Index ¶
Constants ¶
const Version = "0.1.11"
Version is incremented using bump2version
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Datasource ¶
type Datasource struct {
Options
Disabled bool
Description string
FileProvider files.FileProvider
// contains filtered or unexported fields
}
Datasource ...
type Import ¶
type Import struct {
Options
Connection *MongoConnection
Sources []*Datasource
MaxParallelism int
// contains filtered or unexported fields
}
Import ...
type ImportJob ¶ added in v0.1.9
type ImportJob struct {
Source *Datasource
Loader *loaders.Loader
File string
InsertionBatchSize int
IgnoreErrors bool
Collection *mongo.Collection
}
ImportJob ...
type ImportResult ¶
type ImportResult struct {
TotalFiles int
TotalSources int
Description string
Succeeded int
Failed int
Elapsed time.Duration
PartialResults []SourceResult
}
ImportResult ...
type LoggableResult ¶ added in v0.1.9
type LoggableResult interface {
Summary() string
}
LoggableResult ...
type MongoConnection ¶
type MongoConnection struct {
DatabaseName string
AuthDatabaseName string
User string
Password string
Host string
Port int
}
MongoConnection ...
type Options ¶ added in v0.1.9
type Options struct {
DatabaseName string
Collection string
Loader loaders.Loader
PostLoad PostLoadHook
PreDump PreDumpHook
UpdateFilter UpdateFilterHook
EmptyCollection *opt.Flag
Sanitize *opt.Flag
FailOnErrors *opt.Flag
CollectErrors *opt.Flag
IndividualProgress *opt.Flag
ShowCurrentFile *opt.Flag
InsertionBatchSize int
}
Options ...
type PartialResult ¶ added in v0.1.9
type PartialResult struct {
File string
Collection string
Source *Datasource
Succeeded int
Failed int
Elapsed time.Duration
Errors []error
}
PartialResult ...
type PostLoadHook ¶
PostLoadHook ...
type SourceResult ¶ added in v0.1.9
type SourceResult struct {
TotalFiles int
Collection string
Description string
Succeeded int
Failed int
Elapsed time.Duration
PartialResults []PartialResult
}
SourceResult ...
type UpdateFilterHook ¶
type UpdateFilterHook func(loaded interface{}) ([]interface{}, error)
UpdateFilterHook ...
Source Files
Directories