bbash

BBash

Tooling to enable a Bug Bash

Development

Setup

To get started with this project you will need:

• Make (on macOS, you could use brew via brew install make should suffice).

• Golang (see: Download and Install. This project started using Go 1.16.2, but likely anything above 1.14 is fine). Using 1.17.8 now.

• Docker (see: Get Docker)

• Npm (see: Node and npm)

  On Ubuntu, you could use these commands to get the latest stable node version setup:

  sudo apt install npm
  sudo npm cache clean -f
  sudo npm install -g n
  sudo n stable
  

  You may have to restart the terminal after running the above steps to see the latest node version.

• Yarn: sudo npm install --global yarn (see: Installation)

• Air

  To install air:

  • https://github.com/cosmtrek/air

  You can run:

  • go install github.com/cosmtrek/air@v1.29.0

  The air binary will be located in your ~/go/bin folder, which may need to added to your commands and/or path. The AIRCMD setting in the Makefile may need to be adjusted if a different location is used.

Running/Developing

Thanks to Air, there is some amount of "live-reload". To run the project, you can run air -c .air.toml in the project root. Once it is built, you should be able to access the site at http://localhost:7777/. The app pages live at: http://localhost:7777/index.html

Any code changes to golang files will cause a rebuild and restart, and will be accessible via the browser with a refresh!

Local Development Setup

For local development, a good first step is to copy the example .env.example file to .env and launch a local db and air like so:

cp .env.example .env
make run-air

Note that Datadog polling, which is used to calculate scores, is disabled by default in .env.example to decrease noise during local development. To re-enable remove this line.

Server Debugging

For some fun interactive debugging of the golang app with server.go, you could spin up the local docker db image, and manually run the server in debug mode. See the Makefile for the latest and greatest commands to cherry-pick.

$ docker run --name bug_bash_postgres -p 5432:5432 -e POSTGRES_PASSWORD=bug_bash -e POSTGRES_DB=db -d postgres
b6ac8769bab3b19b3e5818e726272bcee6957863b9a7af4261a0ae29ec5bc68e...

Then run server.go in debug mode in your favorite IDE, and enjoy break points activating when you connect to endpoints. Wee!

Frontend Development

For frontend work (with a previously manually launched database - see docker run ... above), this command is helpful for development:

make run-air-alone

Architecture

"Two apps in one" - This project contains two apps:

1. A golang application that provides REST endpoints for the UI and admin tasks, and polls Lift for scoring events.
2. A react application that provides a UI, and calls the REST endpoints served by the golang app.

Go

The go application specific files include:

• server.go
• internal
• go.mod

The go application communicates with the postgres database. The go application also periodically polls the Lift logs for scoring events.

React

The react application files include:

• src
• public
• package.json
• yarn.lock

Deployment

App environment configuration

Configuration of bbash is handled via a .env file in the repo (this is ignored by git by default, so you don't check in secrets):

A .example.env has been provided that looks similar to the following:

PG_USERNAME=postgres
PG_PASSWORD=bug_bash
PG_PORT=5432
PG_DB_NAME=db
PG_HOST=localhost
SSL_MODE=disable

Deploy Application to AWS

Thankfully, we've made this as simple as possible, we think? It'll get simpler with time, I'm sure :)

You will need:

• terraform
• aws cli
• aws-vault
• docker

• Sonatype employees see here for access request instructions and two factor authentication setup.

Terraform

• aws-vault exec <your_profile> terraform init
• aws-vault exec <your_profile> terraform apply

This should create all the nice lil AWS resources to manage this application, using ECS and ECR!

Docker

To create the docker image:

• make docker

Deployment

Some pre-requisite/one-time setup steps:

• setup aws cli configuration to verify working credentials. see: AWS CLI on mac

• install aws-vault

  $ brew install --cask aws-vault
  

• create AWS profile for "<your_profile>" below In AWS under Account -> "Security Credentials" -> “Access keys for CLI, SDK, & API access”

• add aws-vault profile ("<your_profile>" in steps below) for use in pushing images

  $ aws-vault add my-bbash-profile
  

  For sonatype employees: make sure to set up two factor auth (see link)

• (One-time) initialize terraform

  $ aws-vault exec <your_profile> terraform init
  

• View terraform actions to be taken:

  $ aws-vault exec <your_profile> terraform plan
  

An executable bash script (docker.sh?) similar to the following will make pushing images easier:

#!/bin/bash
aws-vault exec <your_profile> aws ecr get-login-password --region <aws_region> | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.<aws_region>.amazonaws.com
docker tag bug-bash:latest <aws_account_id>.dkr.ecr.<aws_region>.amazonaws.com/bug-bash-app:latest
docker push <aws_account_id>.dkr.ecr.<aws_region>.amazonaws.com/bug-bash-app:latest
aws-vault exec <your_profile> -- aws ecs update-service --cluster bug-bash-cluster --service bug-bash-service --force-new-deployment

Replace the stuff in the <> with your values (and remove the <> characters if that isn't immediately apparent), chmod +x docker.sh, and ./docker.sh

After you have done this, you SHOULD have a running service, somewhere in AWS :) - maybe someplace like this? : sandbox-dev or sandbox-dev/index.html

With all the above configured, here's the deployment command in full:

make && make docker && ./docker.sh

Please note that make docker will also increment the version number of this build and create a commit for this change.

Viewing log files in AWS (for newer users)

• For Sonatype employees make sure to Switch Roles to innovations-sandbox. Under main menu select "Switch Roles". Enter account number (12 digits) and role (ie admin). Please note that if using a Mac you may need to be on Safari browser for this to work.

In AWS console search for "CloudWatch".

From CloudWatch navigate to logs -> log groups -> bug-bash-cloudwatch-lergs.

Helpful Links:

• Echo web framework. repo
• How To Run A Campaign
• Locally runnable CI docs

The Fine Print

It is worth noting that this is NOT SUPPORTED by Sonatype, and is a contribution of ours to the open source community (read: you!)

Remember:

• Use this contribution at the risk tolerance that you have
• Do NOT file Sonatype support tickets related to bbash support in regard to this project
• DO file issues here on GitHub, so that the community can pitch in

Phew, that was easier than I thought. Last but not least of all:

Have fun creating and using bbash, we are glad to have you here!