README
¶
shortly
url shortening service written in go
There is a hosted instance of this project managed by myself (aultimus) available at sh.foobarcat.com.
Design
- For the shortened URLs using base64 encoding, a 6 letter long key has been chosen granting 64^6 possible values (over 68 billion)
- This is a userless service so there will be no delete functionality and a second user requesting the same link will receive the same value as the first
- Our data consists of many small files, it is non-relational and read heavy. Dynamodb offers a low-effort managed solution which fits these criteria, thus it has been chosen as our datastore. We can always set a cache up in front of this if performance is insufficient.
- A URL, when MD5summed and base64 encoded results in a string of length 24 (144 bit), each character having 64 possible values. Thus there are 64^24 possible values for an md5sum hash. In truncating this string to six characters (32 bit) we are reducing the hash space to 64^6 possible values. Assuming an equal distribution of urls to hash buckets, when we get 30,084 entries we have a collision probability of 1 in 10 and when we have 77163 entries in our db we have a collision probability of 1 in 2. This collision factor is likely unworkable for large numbers of users, the alternative would be to use A) Longer shortened URLs or B) a Map Reduce job to iterate through and store all possible keys with a cache for each application providing a subset, that approach is considered as a potential extension to this project.
Test averaged 50 requests per collision level to instance running in aws. It looks like collisions are only going to be a problem for a large number of users so this can be a later optimisation after we have the front end and other features up.
Running
go run shortly/main.go
Runs a webserver listening on port 8080
Alternatively you can use build.sh and run.sh scripts to build and run the app in a docker image
Deployment
After building, run push.sh to run the docker image to ECR, and then upload Dockerrun.aws.json from the root directory of this project to the aws EB console.
API
A website is served at the domain root for use by humans. A JSON API exists for programatically interfacing with the service, that JSON API is detailed below
/v1/create endpoint
curl localhost:8080/v1/create -d '{"original_url": "http://foobarcat.blogspot.com"}'
{"shortened_url":"7RxfRd","error":""}
/v1/redirect/{url} endpoint
equivalent to /{url} endpoint but provides parsable json response.
curl http://localhost:8080/v1/redirect/foo
{"original_url":"","error":"could not find key foo"}
TODO
- add access via api key so that we are not vulnerable to malicious url creating attacks whereby our whole short url space is filled up
- add optional alias to create api
- Monitoring of popularity of URLs
- Inject dynamodb creds into docker container
- Rate limiting of clients
- Add handler for calls to favicon.ico
- Vendor deps
- Enable previews via a subdomain e.g. preview.sh.foobarcat.com which provides info on the redirect
Documentation
¶
Index ¶
- Constants
- func EnsurePrefix(originalURL string) (string, error)
- type App
- func (a *App) Create(req *CreateRequest, hasher Hasher) (string, error)
- func (a *App) CreateHandler(w http.ResponseWriter, r *http.Request)
- func (a *App) CreateJSONHandler(w http.ResponseWriter, r *http.Request)
- func (a *App) HealthHandler(w http.ResponseWriter, r *http.Request)
- func (a *App) Init(store db.DBer, portNum string) error
- func (a *App) RedirectHandler(w http.ResponseWriter, r *http.Request)
- func (a *App) RedirectJSONHandler(w http.ResponseWriter, r *http.Request)
- func (a *App) RootHandler(w http.ResponseWriter, r *http.Request)
- func (a *App) Run() error
- type CreateRequest
- type CreateResponse
- type Hasher
- type MD5Hash
- type RedirectResponse
- type ResultTemplateData
Constants ¶
const ( LenShortened = 6 JSONMimeType = "application/json" ContentType = "Content-Type" )
Variables ¶
This section is empty.
Functions ¶
func EnsurePrefix ¶
TODO: maybe we should pass around a net.url.URL object rather than a string
Types ¶
type App ¶
type App struct {
// contains filtered or unexported fields
}
func (*App) CreateHandler ¶
func (a *App) CreateHandler(w http.ResponseWriter, r *http.Request)
CreateHandler provides the create functionality for the website
func (*App) CreateJSONHandler ¶
func (a *App) CreateJSONHandler(w http.ResponseWriter, r *http.Request)
CreateJSONHandler handles the creation of new shortened URLS curl localhost:8080/v1/create -d '{"original_url": "http://foobarcat.blogspot.com"}' TODO: Add test case where mandatory field original_url is missing
func (*App) HealthHandler ¶
func (a *App) HealthHandler(w http.ResponseWriter, r *http.Request)
func (*App) RedirectHandler ¶
func (a *App) RedirectHandler(w http.ResponseWriter, r *http.Request)
TODO: refactor out common code amongst Redirect* handlers TODO: Test this handler
func (*App) RedirectJSONHandler ¶
func (a *App) RedirectJSONHandler(w http.ResponseWriter, r *http.Request)
RedirectJSONHandler handles GET access to shortened urls, this endpoint is publically available curl http://localhost:8080/v1/redirect/foo TODO: respond with correct status codes
func (*App) RootHandler ¶
func (a *App) RootHandler(w http.ResponseWriter, r *http.Request)
type CreateRequest ¶
type CreateRequest struct {
OriginalURL string `json:"original_url"`
}
Source Files
Directories