syncron

command module

v1.0.0 Latest Latest Go to latest Published: Feb 27, 2023 License: Apache-2.0 Imports: 2 Imported by: 0

Details

Valid go.mod file

The Go module system was introduced in Go 1.11 and is the official dependency management solution for Go.
Redistributable license

Redistributable licenses place minimal restrictions on how software can be used, modified, and redistributed.
Tagged version

Modules with tagged versions give importers more predictable builds.
Stable version

When a project reaches major version v1 it is considered stable.
Learn more about best practices

Repository

github.com/redhatcre/syncron

Links

Open Source Insights

README ¶

Syncron

Usage Patterns all in one cli based application written in Golang.

🔧 Installation

Download and install Go.

Follow the link for instructions 🔔 Please note: version 1.18 or higher required
Clone this repository.

git clone git@github.com:RedHatCRE/syncron.git
Navigate to local cloned repository folder and install Syncron with:

go install
Check that Syncron has been properly installed on your environment by running

syncron -v syncron version 1.0.0
- Make sure your gopath is added to your path
  
  export PATH=$PATH:$(go env GOPATH)/bin

📖 Setup

To use Syncron, two important steps must be taken.

Create configuration file.

Important information about config:

Yaml based
Path: ~/.config/
Naming: syncron.yaml

A minimal syncron configuration file is as follows:

s3:
  endpoint: "<endpoint>"
  region: "<region>"
  bucket: "<name of bucket to pull from>"
prefix: "<targeted keys starting path>"
downloadDir: "<path where files will be downloaded>"

For downloading files from a AWS buckets, proper credentials must be present on running machine. Two options:
- Credentials file at $HOME/.aws/credentials
  - The credentials file has the following format:
```
[default]
aws_access_key_id = "XXXXXXX"
aws_secret_access_key = "XXXXXXX"
```
- Environment variables must be set:
  - export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
  - export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY

⚙️ Commands & Options

The user is given several flags to pick which time frame to pull data from.

Download files

Option	Description	Type	Required?
`download`	Download files.	`string`	No

syncron download [option] [--flag] [number]

--flag	Description	Type	Default	Required?
`days`	Download files from the past x days.	`int`	`2`	No
`months`	Download files from the past x months.	`int`	`0`	No
`years`	Download files from the past x years.	`int`	`0`	No
`filter`	Filter files to download	`[]str`	[]str{}	No

Option	Description	Type	Default	Required?
`sosreports`	Download sosreports files.	`string`	`sosreports`	Yes

Read parquet files

syncron read-parquet [--flag] [option]

Option	Description	Type	Required?
`read-parquet`	Read local parquet files.	`string`	No

--flag	Description	Type	Default	Required?
`file`	What file to read.	`str`	`-`	Yes
`output`	Path to place unpacked file.	`str`		Yes

Query data from database

syncron queries

⚠️ License

Syncron is free and open-source software licensed under the Apache 2.0 License.

Documentation ¶

Overview ¶

Licensed under the Apache License, Version 2.0 (the "License"); you may
not use this file except in compliance with the License. You may obtain
a copy of the License at

     http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
License for the specific language governing permissions and limitations
under the License.

Source Files ¶

View all Source files

main.go

Directories ¶

Path	Synopsis
cmd
syncron
configuration
pkg
bucketaws
charts
cli
dumper
log
myquery
parquet_reader
pipelines
snappy_reader
test
mocks
utils
files
filter The purpose of the package filter is to reduce the download to the specified files by filtering through the configuration's files, downloading only the ones that match the supplied pattern given via --filter flag.	The purpose of the package filter is to reduce the download to the specified files by filtering through the configuration's files, downloading only the ones that match the supplied pattern given via --filter flag.
validators The purpose of the package filter is to reduce the download to the specified files by filtering through the configuration's files, downloading only the ones that match the supplied pattern given via --filter flag.	The purpose of the package filter is to reduce the download to the specified files by filtering through the configuration's files, downloading only the ones that match the supplied pattern given via --filter flag.

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL