util

Overview ¶

Package shortid enables the generation of short, unique, non-sequential and by default URL friendly Ids. The package is heavily inspired by the node.js https://github.com/dylang/shortid library.

Id Length ¶

The standard Id length is 9 symbols when generated at a rate of 1 Id per millisecond, occasionally it reaches 11 (at the rate of a few thousand Ids per millisecond) and very-very rarely it can go beyond that during continuous generation at full throttle on high-performant hardware. A test generating 500k Ids at full throttle on conventional hardware generated the following Ids at the head and the tail (length > 9 is expected for this test):

-NDveu-9Q
iNove6iQ9J
NVDve6-9Q
VVDvc6i99J
NVovc6-QQy
VVoveui9QC
...
tFmGc6iQQs
KpTvcui99k
KFTGcuiQ9p
KFmGeu-Q9O
tFTvcu-QQt
tpTveu-99u

Life span ¶

The package guarantees the generation of unique Ids with zero collisions for 34 years (1/1/2016-1/1/2050) using the same worker Id within a single (although concurrent) application if application restarts take longer than 1 millisecond. The package supports up to 32 works, all providing unique sequences.

Implementation details ¶

Although heavily inspired by the node.js shortid library this is not a simple Go port. In addition it

• is safe to concurrency;
• does not require any yearly version/epoch resets;
• provides stable Id size over a long period at the rate of 1ms;
• guarantees no collisions (due to guaranteed fixed size of Ids between milliseconds and because multiple requests within the same ms lead to longer Ids with the prefix unique to the ms);
• supports 32 over 16 workers.

The algorithm uses less randomness than the original node.js implementation, which permits to extend the life span as well as reduce and guarantee the length. In general terms, each Id has the following 3 pieces of information encoded: the millisecond (first 8 symbols), the worker Id (9th symbol), running concurrent counter within the same millisecond, only if required, over all remaining symbols. The element of randomness per symbol is 1/2 for the worker and the millisecond and 0 for the counter. Here 0 means no randomness, i.e. every value is encoded using a 64-base alphabet; 1/2 means one of two matching symbols of the supplied alphabet, 1/4 one of four matching symbols. The original algorithm of the node.js module uses 1/4 throughout.

All methods accepting the parameters that govern the randomness are exported and can be used to directly implement an algorithm with e.g. more randomness, but with longer Ids and shorter life spans.