README
¶
Vulnerability Database
Persistent store for vulnerabilities data obtained from Vulcan scans.
The Vulnerability Database acts as a worker which reads from a queue containing the checks status changes and data, it then processes this data, including checks report, in order to maintain a historic representation of vulnerabilities lifecycle, affected assets, executed checks, etc.
For running the component locally, clone and run at the root of the repo the following:
go install ./...
cd db && source postgres-start.sh && cd -
cd db && source flyway-migrate.sh && cd -
vulnerability-db-consumer -c _resources/config/local.toml
How to run the Vulnerability DB in development mode
You can test the Vulnerability DB Consumer locally in your machine. The commands bellow will launch the necessary components required by the application.
# Navigate to the local_deployment folder
cd local_deployment
# Start the dependencies
docker-compose up -d
# Build and run the vulnerability-db-consumer
./start.sh
You can test that everything works by sending a message to the mocked SNS topic, using the AWS cli:
AWS_ACCESS_KEY_ID=fake AWS_SECRET_ACCESS_KEY=fake aws sns publish \
--region local \
--endpoint-url http://localhost:4100 \
--topic-arn arn:aws:sns:local:012345678900:VulcanLocalhostChecks \
--message '
{
"status":"FINISHED",
"id":"old-model-happy-path-01",
"tag":"team:test",
"target":"api.example.com",
"checktype_name":"vulcan-http-headers",
"report":"http://localhost:8080/old-model-happy-path-01.json"
}' \
--message-attributes '{"status":{"DataType":"String","StringValue":"FINISHED"}}'
To stop the dependencies, run:
docker-compose down --remove-orphans
To purge local mocked SQS queue:
AWS_ACCESS_KEY_ID=fake AWS_SECRET_ACCESS_KEY=fake aws sqs purge-queue \
--region local \
--endpoint-url http://localhost:4100 \
--queue-url http://localhost:4100/012345678900/VulcanLocalhostVulnDBChecks
Docker execute
Those are the variables you have to use:
| Variable | Description | Sample |
|---|---|---|
| MAX_EVENT_AGE | Defines the max age for which check events are processed. Older events are discarded | 365 |
| LOG_LEVEL | error | |
| PG_HOST | Database host | localhost |
| PG_NAME | Database name | vulnerabilitydb |
| PG_USER | Database user | vulnerabilitydb |
| PG_PASSWORD | Database password | vulnerabilitydb |
| PG_PORT | Database port | 5432 |
| PG_SSLMODE | One of these (disable,allow,prefer,require,verify-ca,verify-full) | disable |
| PG_CA_B64 | A base64 encoded CA certificate | |
| SQS_NUMBER_OF_PROCESSORS | Number of concurrent SQS processors | Default: 10 |
| SQS_QUEUE_ARN | Checks queueu ARN | arn:aws:sqs:xxx:123456789012:yyy |
| SNS_TOPIC_ARN | ARN of topic to publish new vulnerabilities | arn:aws:sns:xxx:123456789012:yyy |
| RESULTS_URL | External vulcan-results URL | https://results.vulcan.com |
| RESULTS_INTERNAL_URL | Internal vulcan-results URL | http://vulcan-results |
| AWS_SQS_ENDPOINT | Endpoint for SQS creation queue (optional) | http://custom-aws-endpoint |
| AWS_SNS_ENDPOINT | Endpoint for SNS topic (optional) | http://custom-aws-endpoint |
docker build . -t vdb
# Use the default config.toml customized with env variables.
docker run --env-file ./local.env -e AWS_SECRET_ACCESS_KEY -e AWS_ACCESS_KEY_ID --name vdb --rm vdb
# Use custom config.toml
docker run -v `pwd`/custom.toml:/app/config.toml vdb
Directories