GoProm

Promotions API

ADRs · CI

Overview

GoProm periodically load promotions into Redis and expose the data via an HTTP API. It's composed by two applications that run separately: api and loader.
The api simple expose an endpoint to query promotions by id.
The loader reads a csv file in a specific path, by default data, and sends all data to Redis. It's triggered by a cron. The loaderworkflow is the following:

• a crontab executes the loader every 1 minute.
• the loader checks if there's any new promotions.csv file in the specified directory.
• no promotions file, it will exit.
• found a promotions file, now it'll read the promotions and stream each entry to Redis, including a configurable expiration time.
• the promotions.csv file will be renamed, including a suffix DATETIME--imported.csv.

Getting Started

Prerequisites

• Go 1.18
• Docker
• Docker Compose

Configuration

This project uses environment variables for configuration. See .env.sample for more details.

Running

To execute a local environment with both the loader, the api and Redis, run:

make up

Processing Promotions

With the full docker compose environment up, in order to process new promotions, add the promotions.csv file to the directory data at the project root dir. After some time, the loader application will identify this file and start the process automatically.
If you just want to see the loader working, you can do:

• make sure a Redis instance is running. Use the command: make requirements.
• make sure to have a promotions file promotions.csv inside the folder data on the project root dir.
• run make loader to run the loader one time.

To query one promotion, use the following command:

curl http://localhost:8080/promotions/d018ef0b-dbd9-48f1-ac1a-eb4d90e57118

Development

Check the Makefile for more details.
Execute the following command to see the Makefile help:

make

Tools

Several tools, not only from the Go "world", are used to create a local development environment were we can easily keep the source code within the project/company standards, avoiding committing bad code.
These tools are not required to be installed in order to contribute to this project.
Check the list below with a brief explanation for each.

• Node.js - Husky and Commit Lint requires Node.js in order to run
• Air - Live reloads for Go apps
• Staticcheck - Additional linter for Go
• Husky - Git hooks management
• Commit Lint - Lint commit messages. Currently, using conventional commits.

Tests

Unit Tests

To execute all unit tests, run:

make test

End-to-End Tests

The end-to-end test available runs a docker-compose environment first, and then the tests scripts.
Run:

make test-e2e

Built With

Main libraries and tools used in this project:

• Docker
• Docker Compose
• Redis
• godotenv
• Zerolog
• go-env
• go-redis

Contributing

See CONTRIBUTING.md for more details.

Considerations and Thoughts

• To improve the CSV processing performance, a group of goroutines are used, instead of just one in the previous implementation. Each goroutine streams Redis commands to redis-cli in parallel now.
• Promotions will be removed by Redis using the Expiration feature. For every promotion imported, a configurable expiration time is set as well.
• For an ideal solution, the application should have:
  • centralized log solution
  • monitoring solution
  • metrics
  • deployment should be made using solutions like Kubernetes, which facilitates configuration and scaling when needed
  • ideally, the Redis should be operated as a cluster to horizontally scale it and increase availability

back to top