Automatically optimize your images & videos hosted on AWS S3 when an S3 event is detected
This project is inspired by image-optimizer & Lambda ECS Worker Pattern
Architecture
A tag is also applied to the optimized file so this file will be ignored in subsequent event
How it works ?
- S3
- S3 is configured to trigger a specific lambda on objectCreate event
- Lambda
- catch the S3 Event
- push the event to a SQS queue
- EC2
- constantly listens to messages from SQS queue
- when a message arrives :
- get the S3 Event from the message
- get information about S3 object (content-type)
- download the S3 object
- executes optimization chain on object (one or more command)
- upload the optimized object to S3
Run on AWS
In AWS dashboard, go to cloudformation & create a new stack from the template file cloudformation.yml
In the parameters, change the bucket name & the subnet value (a private subnet is fine)
When the stack is up, go to the created S3 bucket & upload some images. After some moment, you will notice they will be automatically optimized
You can check that the new optimized image has the tag optimizer_ignore
:
In CloudWatch you can get the logs of your Lambda & your docker container running the optimization program
Optimization configuration
The optimization configuration is located in optimizer.json file. In the actual version, the JSON file need to be at the same level as the media-optimizer executable
Current configuration
Currently using all the optimizers from image-optimizer project + ffmpeg for mp4 video :
- image/png
pngquant --output [output_file] -f [intput_file]
optipng -out [output_file] -i0 -o2 -clobber [intput_file]
- image/jpeg
jpegoptim -d [output_directory] -m85 --strip-all --all-progressive [intput_file]
- image/svg+xml
svgo -o [output_file] --disable={cleanupIDs,removeViewBox} [intput_file]
- image/gif
gifsicle -o [output_file] -b -O3 [intput_file]
- image/webp
cwebp -o [output_file] -m 6 -pass 10 -mt -q 80 [intput_file]
- video/mp4
ffmpeg -i [input_file] -vcodec libx264 -crf 24 -y [output_file]
You can add more format & more commands to the config file optimizer.json
{
"optimizers": [{
"contentType": "image/png",
"exec": [{
"binary": "pngquant",
"outputFile": "--output",
"params": ["-f"]
},{
"binary":"optipng",
"outputFile": "-out",
"params": ["-i0","-o2", "-clobber"]
}]
.....
}
binary
is the command to execute
params
are the command parameters
outputFile
is the parameter for the output file
outputDirectory
is the parameter for the output directory if outputFile
is not specified
inputFile
is the parameter for the input file if neither outputFile
nor outputDirectory
are specified. In case inputFile
is specified the outputFile value will be appended to the parameters (at the end)
Environment variables
Those variables configure S3 Object configuration & metadata :
name |
default value |
OBJECT_ACL |
private |
OBJECT_CACHE_CONTROL |
max-age=15552000 |
OBJECT_STORAGE_CLASS |
STANDARD |
OBJECT_SERVER_SIDE_ENCRYPTION |
none |
Note that if AWS::Region
is not specified in your ~/.aws
, the variable AWS_REGION
should be set (see cloudformation.yml)
Run locally
make install
make run
Run with docker (locally)
docker build . -t media-optimizer
docker run -v $HOME/.aws:/root/.aws -it media-optimizer
Run with docker (DockerHub)
docker pull bertrandmartel/media-optimizer:latest
docker run -v $HOME/.aws:/root/.aws -it bertrandmartel/media-optimizer:latest
Dependencies