photos

Photos

This repo defines two Google App Engine services:

1. photos
2. photoadd

The photos service serves most requests and accepts requests for the following url prefixes:

• /s/photos/
• /s/users/

The photoadd service serves requests to add/update photos and accepts requests for the following url prefixes:

• /s/photoadd/

The two services are separated as the resource requirements are quite different. Specifically,

• photo service runs on small instances (F1)
• photoadd service requires substantially more memory and runs on large instances (F4)

CI Variables

Environment-specific:

• MONGODB_URI_DEV
• DOMAIN_DEV
• DEPLOY_KEY_FILE_DEV
• PROJECT_ID_DEV

Other:

• AUTH_AUDIENCE
• PHOTO_NOTIFY_QUEUE_ID
• PHOTO_NOTIFY_QUEUE_LOC_ID

Google Cloud project setup

This section assumes that <PROJECT> is the name of the Google Cloud project that hosts the Google App Engine services.

Storage

• Photo master bucket. Can have any name but standard name would be <PROJECT>-photo-master
• Photo resize cache bucket: <PROJECT>-photo-resize-cache

Service account

• internal@<PROJECT>.iam.gserviceaccount.com
  • Service account with Service Account Token Creator role

Pub/Sub

Need a pub/sub subscription to events from the photo master bucket to the photos service.

• Create a topic e.g. photo-master-storage
• Create a subscription:
  • Push
  • Endpoint: <app/engine/domain>/s/photos/notify/master-storage-updates
  • Enable authentication
    • Use service account internal@<PROJECT>.iam.gserviceaccount.com
    • Audience: lookoutpoint.ca
  • Never expire
  • Acknowledgement deadline: 60 seconds
  • Retry policy: exponential backoff delay (optional)

Creating the pub/sub notifications

Run the following command in a Google Cloud shell: gsutil notification create -t <TOPIC_NAME> -f json gs://<PHOTO_MASTER_BUCKET>

Cloud Tasks

The following queue is required:

• photo-notify (defined by the PHOTO_NOTIFY_QUEUE_ID and PHOTO_NOTIFY_QUEUE_LOC_ID env vars)
  • Max concurrent: 8
  • Max attempts: unlimited
  • Min interval: 1s
  • Max interval: 300s
  • Max doublings: 8
  • Max retry duration: unlimited
  • Should be limited to a maximum of 4 concurrent dispatches (for rate-limiting photo updates)

License

MIT License. Full license