httphead: Index | Examples | Files

package httphead

import ""

Package httphead contains utils for parsing HTTP and HTTP-grammar compatible text protocols headers.

That is, this package first aim is to bring ability to easily parse constructions, described here



Package Files

cookie.go head.go httphead.go lexer.go octet.go option.go writer.go


var DefaultCookieScanner = CookieScanner{}

DefaultCookieScanner is a CookieScanner which is used by ScanCookie(). Note that it is intended to have the same behavior as http.Request.Cookies() has.

var OctetTypes [256]OctetType

OctetTypes is a table of octets.

func CanonicalizeHeaderKey Uses

func CanonicalizeHeaderKey(k []byte)

CanonicalizeHeaderKey is like standard textproto/CanonicalMIMEHeaderKey, except that it operates with slice of bytes and modifies it inplace without copying.

func IntFromASCII Uses

func IntFromASCII(bts []byte) (ret int, ok bool)

IntFromASCII converts ascii encoded decimal numeric value from HTTP entities to an integer.

func ParseHeaderLine Uses

func ParseHeaderLine(line []byte) (k, v []byte, ok bool)

ParseHeaderLine parses HTTP header as key-value pair. It returns parsed values and true if parse is ok.

func ParseVersion Uses

func ParseVersion(bts []byte) (major, minor int, ok bool)

ParseVersion parses major and minor version of HTTP protocol. It returns parsed values and true if parse is ok.

func ReadLine Uses

func ReadLine(br *bufio.Reader) ([]byte, error)

ReadLine reads line from br. It reads until '\n' and returns bytes without '\n' or '\r\n' at the end. It returns err if and only if line does not end in '\n'. Note that read bytes returned in any case of error.

It is much like the textproto/Reader.ReadLine() except the thing that it returns raw bytes, instead of string. That is, it avoids copying bytes read from br.

textproto/Reader.ReadLineBytes() is also makes copy of resulting bytes to be safe with future I/O operations on br.

We could control I/O operations on br and do not need to make additional copy for safety.

func RemoveByte Uses

func RemoveByte(data []byte, c byte) []byte

RemoveByte returns data without c. If c is not present in data it returns the same slice. If not, it copies data without c.

func ScanCookie Uses

func ScanCookie(data []byte, it func(key, value []byte) bool) bool

ScanCookie scans cookie pairs from data using DefaultCookieScanner.Scan() method.

func ScanOptions Uses

func ScanOptions(data []byte, it func(index int, option, attribute, value []byte) Control) bool

ScanOptions parses data in this form:

values = 1#value value = token *( ";" param ) param = token [ "=" (token | quoted-string) ]

It calls given callback with the index of the option, option itself and its parameter (attribute and its value, both could be nil). Index is useful when header contains multiple choises for the same named option.

Given callback should return one of the defined Control* values. ControlSkip means that passed key is not in caller's interest. That is, all parameters of that key will be skipped. ControlBreak means that no more keys and parameters should be parsed. That is, it must break parsing immediately. ControlContinue means that caller want to receive next parameter and its value or the next key.

It returns false if data is malformed.


foo := map[string]string{}

ScanOptions([]byte(`foo;bar=1;baz`), func(index int, key, param, value []byte) Control {
    foo[string(param)] = string(value)
    return ControlContinue

fmt.Printf("bar:%s baz:%s", foo["bar"], foo["baz"])


bar:1 baz:

func ScanPairGreedy Uses

func ScanPairGreedy(data []byte, open, close byte) (n int)

ScanPairGreedy scans for complete pair of opening and closing chars in greedy manner. Note that first opening byte must not be present in data.

func ScanToken Uses

func ScanToken(p []byte) (n int, t ItemType)

ScanToken scan for next token in p. It returns length of the token and its type. It do not trim p.

func ScanTokens Uses

func ScanTokens(data []byte, it func([]byte) bool) bool

ScanTokens parses data in this form:

list = 1#token

It returns false if data is malformed.


var values []string

ScanTokens([]byte(`a,b,c`), func(v []byte) bool {
    values = append(values, string(v))
    return v[0] != 'b'



[a b]

func ScanUntil Uses

func ScanUntil(data []byte, c byte) (n int)

ScanUntil scans for first non-escaped character c in given data. It returns index of matched c and -1 if c is not found.

func SkipSpace Uses

func SkipSpace(p []byte) (n int)

SkipSpace skips spaces and lws-sequences from p. It returns number ob bytes skipped.

func SplitRequestLine Uses

func SplitRequestLine(line []byte) (method, uri, version []byte)

SplitRequestLine splits given slice of bytes into three chunks without parsing.

func SplitResponseLine Uses

func SplitResponseLine(line []byte) (version, status, reason []byte)

SplitResponseLine splits given slice of bytes into three chunks without parsing.

func ValidCookieName Uses

func ValidCookieName(name []byte) bool

ValidCookieName reports wheter given bytes is a valid RFC2616 "token" bytes.

func ValidCookieValue Uses

func ValidCookieValue(value []byte, strict bool) bool

ValidCookieValue reports whether given value is a valid RFC6265 "cookie-octet" bytes.

cookie-octet = %x21 / %x23-2B / %x2D-3A / %x3C-5B / %x5D-7E

; US-ASCII characters excluding CTLs,
; whitespace DQUOTE, comma, semicolon,
; and backslash

Note that the false strict parameter disables errors on space 0x20 and comma 0x2c. This could be useful to bring some compatibility with non-compliant clients/servers in the real world. It acts the same as standard library cookie parser if strict is false.

func WriteOptions Uses

func WriteOptions(dest io.Writer, options []Option) (n int, err error)

WriteOptions write options list to the dest. It uses the same form as {Scan,Parse}Options functions: values = 1#value value = token *( ";" param ) param = token [ "=" (token | quoted-string) ]

It wraps valuse into the quoted-string sequence if it contains any non-token characters.


opts := []Option{
    NewOption("foo", map[string]string{
        "param": "hello, world!",
    NewOption("bar", nil),
    NewOption("b a z", nil),

buf := bytes.Buffer{}
bw := bufio.NewWriter(&buf)

WriteOptions(bw, opts)


foo;param="hello, world!",bar,"b a z"

type Control Uses

type Control byte

Control represents operation that scanner should perform.

const (
    // ControlContinue causes scanner to continue scan tokens.
    ControlContinue Control = iota
    // ControlBreak causes scanner to stop scan tokens.
    // ControlSkip causes scanner to skip current entity.

type CookieScanner Uses

type CookieScanner struct {
    // DisableNameValidation disables name validation of a cookie. If false,
    // only RFC2616 "tokens" are accepted.
    DisableNameValidation bool

    // DisableValueValidation disables value validation of a cookie. If false,
    // only RFC6265 "cookie-octet" characters are accepted.
    // Note that Strict option also affects validation of a value.
    // If Strict is false, then scanner begins to allow space and comma
    // characters inside the value for better compatibility with non standard
    // cookies implementations.
    DisableValueValidation bool

    // BreakOnPairError sets scanner to immediately return after first pair syntax
    // validation error.
    // If false, scanner will try to skip invalid pair bytes and go ahead.
    BreakOnPairError bool

    // Strict enables strict RFC6265 mode scanning. It affects name and value
    // validation, as also some other rules.
    // If false, it is intended to bring the same behavior as
    // http.Request.Cookies().
    Strict bool

CookieScanner contains options for scanning cookie pairs. See

func (CookieScanner) Scan Uses

func (c CookieScanner) Scan(data []byte, it func(name, value []byte) bool) bool

Scan maps data to name and value pairs. Usually data represents value of the Cookie header.

type ItemType Uses

type ItemType int

ItemType encodes type of the lexing token.

const (
    // ItemUndef reports that token is undefined.
    ItemUndef ItemType = iota
    // ItemToken reports that token is RFC2616 token.
    // ItemSeparator reports that token is RFC2616 separator.
    // ItemString reports that token is RFC2616 quouted string.
    // ItemComment reports that token is RFC2616 comment.
    // ItemOctet reports that token is octet slice.

type OctetType Uses

type OctetType byte

OctetType desribes character type.

From the "Basic Rules" chapter of RFC2616 See

OCTET = <any 8-bit sequence of data> CHAR = <any US-ASCII character (octets 0 - 127)> UPALPHA = <any US-ASCII uppercase letter "A".."Z"> LOALPHA = <any US-ASCII lowercase letter "a".."z"> ALPHA = UPALPHA | LOALPHA DIGIT = <any US-ASCII digit "0".."9"> CTL = <any US-ASCII control character (octets 0 - 31) and DEL (127)> CR = <US-ASCII CR, carriage return (13)> LF = <US-ASCII LF, linefeed (10)> SP = <US-ASCII SP, space (32)> HT = <US-ASCII HT, horizontal-tab (9)> <"> = <US-ASCII double-quote mark (34)> CRLF = CR LF LWS = [CRLF] 1*( SP | HT )

Many HTTP/1.1 header field values consist of words separated by LWS or special characters. These special characters MUST be in a quoted string to be used within a parameter value (as defined in section 3.6).

token = 1*<any CHAR except CTLs or separators> separators = "(" | ")" | "<" | ">" | "@" | "," | ";" | ":" | "\" | <"> | "/" | "[" | "]" | "?" | "=" | "{" | "}" | SP | HT

func (OctetType) IsChar Uses

func (t OctetType) IsChar() bool

IsChar reports whether octet is CHAR.

func (OctetType) IsControl Uses

func (t OctetType) IsControl() bool

IsControl reports whether octet is CTL.

func (OctetType) IsSeparator Uses

func (t OctetType) IsSeparator() bool

IsSeparator reports whether octet is separator.

func (OctetType) IsSpace Uses

func (t OctetType) IsSpace() bool

IsSpace reports whether octet is space (SP or HT).

func (OctetType) IsToken Uses

func (t OctetType) IsToken() bool

IsToken reports whether octet is token.

type Option Uses

type Option struct {
    Name       []byte
    Parameters Parameters

Option represents a header option.

func NewOption Uses

func NewOption(name string, params map[string]string) Option

NewOption creates named option with given parameters.

func ParseOptions Uses

func ParseOptions(data []byte, options []Option) ([]Option, bool)

ParseOptions parses all header options and appends it to given slice of Option. It returns flag of successful (wellformed input) parsing.

Note that appended options are all consist of subslices of data. That is, mutation of data will mutate appended options.


options, ok := ParseOptions([]byte(`foo;bar=1,baz`), nil)
fmt.Println(options, ok)


[{foo [bar:1]} {baz []}] true

func (Option) Copy Uses

func (opt Option) Copy(p []byte) Option

Copy copies all underlying []byte slices into p and returns new Option. Note that p must be at least of opt.Size() length.

func (Option) Equal Uses

func (opt Option) Equal(b Option) bool

Equal reports whether option is equal to b.

func (Option) Size Uses

func (opt Option) Size() int

Size returns number of bytes need to be allocated for use in opt.Copy.

func (Option) String Uses

func (opt Option) String() string

String represents option as a string.

type OptionSelector Uses

type OptionSelector struct {
    // Check is a filter function that applied to every Option that possibly
    // could be selected.
    // If Check is nil all options will be selected.
    Check func(Option) bool

    // Flags contains flags for options selection.
    Flags SelectFlag

    // Alloc used to allocate slice of bytes when selector is configured with
    // SelectCopy flag. It will be called with number of bytes needed for copy
    // of single Option.
    // If Alloc is nil make is used.
    Alloc func(n int) []byte

OptionSelector contains configuration for selecting Options from header value.

func (OptionSelector) Select Uses

func (s OptionSelector) Select(data []byte, options []Option) ([]Option, bool)

Select parses header data and appends it to given slice of Option. It also returns flag of successful (wellformed input) parsing.

type Parameters Uses

type Parameters struct {
    // contains filtered or unexported fields

Parameters represents option's parameters.

func (*Parameters) Copy Uses

func (p *Parameters) Copy(dst []byte) (Parameters, []byte)

Copy copies all underlying []byte slices into dst and returns new Parameters. Note that dst must be at least of p.Size() length.

func (Parameters) Equal Uses

func (p Parameters) Equal(b Parameters) bool

Equal reports whether a equal to b.

func (*Parameters) ForEach Uses

func (p *Parameters) ForEach(cb func(k, v []byte) bool)

ForEach iterates over parameters key-value pairs and calls cb for each one.

func (*Parameters) Get Uses

func (p *Parameters) Get(key string) (value []byte, ok bool)

Get returns value by key and flag about existence such value.

func (*Parameters) Set Uses

func (p *Parameters) Set(key, value []byte)

Set sets value by key.

func (*Parameters) Size Uses

func (p *Parameters) Size() int

Size returns number of bytes that needed to copy p.

func (*Parameters) String Uses

func (p *Parameters) String() (ret string)

String represents parameters as a string.

type RequestLine Uses

type RequestLine struct {
    Method  []byte
    URI     []byte
    Version Version

RequestLine contains parameters parsed from the first request line.

func ParseRequestLine Uses

func ParseRequestLine(line []byte) (r RequestLine, ok bool)

ParseRequestLine parses http request line like "GET / HTTP/1.0".

type ResponseLine Uses

type ResponseLine struct {
    Version Version
    Status  int
    Reason  []byte

ResponseLine contains parameters parsed from the first response line.

func ParseResponseLine Uses

func ParseResponseLine(line []byte) (r ResponseLine, ok bool)

ParseResponseLine parses first response line into ResponseLine struct.

type Scanner Uses

type Scanner struct {
    // contains filtered or unexported fields

Scanner represents header tokens scanner. See

func NewScanner Uses

func NewScanner(data []byte) *Scanner

NewScanner creates new RFC2616 data scanner.

func (*Scanner) Advance Uses

func (l *Scanner) Advance(n int) bool

Advance moves current position index at n bytes. It returns true on successful move.

func (*Scanner) Buffered Uses

func (l *Scanner) Buffered() int

Buffered reporst how many bytes there are left to scan.

func (*Scanner) Bytes Uses

func (l *Scanner) Bytes() []byte

Bytes returns current token bytes.

func (*Scanner) FetchUntil Uses

func (l *Scanner) FetchUntil(c byte) bool

FetchUntil fetches ItemOctet from current scanner position to first occurence of the c or to the end of the underlying data.

func (*Scanner) Next Uses

func (l *Scanner) Next() bool

Next scans for next token. It returns true on successful scanning, and false on error or EOF.

func (*Scanner) Peek Uses

func (l *Scanner) Peek() byte

Peek reads byte at current position without advancing it. On end of data it returns 0.

func (*Scanner) Peek2 Uses

func (l *Scanner) Peek2() (a, b byte)

Peek2 reads two first bytes at current position without advancing it. If there not enough data it returs 0.

func (*Scanner) Skip Uses

func (l *Scanner) Skip(c byte)

Skip skips all bytes until first occurence of c.

func (*Scanner) SkipEscaped Uses

func (l *Scanner) SkipEscaped(c byte)

SkipEscaped skips all bytes until first occurence of non-escaped c.

func (*Scanner) Type Uses

func (l *Scanner) Type() ItemType

Type reports current token type.

type SelectFlag Uses

type SelectFlag byte

SelectFlag encodes way of options selection.

const (
    // SelectCopy causes selector to copy selected option before appending it
    // to resulting slice.
    // If SelectCopy flag is not passed to selector, then appended options will
    // contain sub-slices of the initial data.
    SelectCopy SelectFlag = 1 << iota

    // SelectUnique causes selector to append only not yet existing option to
    // resulting slice. Unique is checked by comparing option names.

func (SelectFlag) String Uses

func (f SelectFlag) String() string

String represetns flag as string.

type Version Uses

type Version struct {
    Major int
    Minor int

Version contains protocol major and minor version.

Package httphead imports 5 packages (graph) and is imported by 5 packages. Updated 2018-02-07. Refresh now. Tools for package owners.