rd

package module
v0.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 15, 2023 License: Unlicense Imports: 15 Imported by: 0

README

Overview

Short for Request Decoding. Missing feature of the Go standard library: decoding arbitrary HTTP requests into structs. Features:

  • Transparent support for different content types / encoding formats:
    • URL query.
    • URL-encoded form.
    • Multipart form.
    • JSON.
  • Transparent support for different HTTP methods:
    • Read-only -> parse only URL query.
    • Non-read-only -> parse only request body.
  • Transparent support for various text-parsing interfaces.
  • Support for membership testing (was X present in request?), useful for PATCH semantics.
  • Tiny and dependency-free.

API docs: https://pkg.go.dev/github.com/mitranim/rd.

Example

1-call decoding. Works for any content type.

import "github.com/mitranim/rd"
import "github.com/mitranim/try"

var input struct {
  FieldOne string `json:"field_one"`
  FieldTwo int64  `json:"field_two"`
}
try.To(rd.Decode(req, &input))

Download once, decode many times. Works for any content type.

import "github.com/mitranim/rd"
import "github.com/mitranim/try"

dec := rd.TryDownload(req)

var input0 struct {
  FieldOne string `json:"field_one"`
}
try.To(dec.Decode(&input0))

var input1 struct {
  FieldTwo int64  `json:"field_two"`
}
try.To(dec.Decode(&input1))

// Membership testing.
haser := dec.Haser()
fmt.Println(haser.Has(`fieldTwo`))

Changelog

v0.2.3

Internal change: renamed Error.Append to Error.AppendTo for consistency with other libraries.

Minor change: added Type, TypeJsonUtf8, TypeFormUtf8, TypeMultiUtf8.

v0.2.2

Decoders now implement a new interface Setter that creates a Set of known keys.

v0.2.1

Fixed edge case bug where Form.Decode wouldn't invoke SliceParser for non-slices.

v0.2.0

Breaking revision:

  • Much more flexible.
  • Much faster.
  • Much more test coverage.
  • Renamed from reqdec to rd for brevity.
  • Dependency-free.

v0.1.7

Breaking: when decoding from JSON, {"<field>": null} zeroes the matching destination field, instead of being ignored. This is an intentional deviation. The json package makes no distinction between a missing field and a field whose value is null. However, in order to support PATCH semantics, we often want to decode into non-zero output structs, updating fields that are present in the input, while ignoring fields missing from the input. Using null to zero the output is essential for this use case. If null was ignored, clients would be unable to set empty values, able only to set new non-empty values.

Breaking: when decoding from formdata, "" is treated as a "null" or "zero value" for output fields that are not slices and don't implement SliceParser. This is extremely useful for standard DOM forms.

Removed Reqdec.DecodeAt. It wasn't compatible with the new logic for struct field decoding, and supporting it separately would require code duplication.

Renamed FromQuery → FromVals.

Added FromJson.

v0.1.6

Added FromQuery and FromReqQuery.

v0.1.5

Added SliceParser for parsing non-slices from lists.

v0.1.4

Changed the license to Unlicense.

v0.1.3

When decoding into a struct where some of the fields are embedded struct pointers, those nested structs are allocated only if some of their fields are present in the request.

Also moved some reflection-related utils to a tiny dependency.

v0.1.2

First tagged release.

License

https://unlicense.org

Misc

I'm receptive to suggestions. If this library almost satisfies you but needs changes, open an issue or chat me up. Contacts: https://mitranim.com/#contacts

Documentation

Index

Constants

View Source 
const (
	Type = `Content-Type`

	TypeJson     = `application/json`
	TypeJsonUtf8 = `application/json; charset=utf-8`

	TypeForm     = `application/x-www-form-urlencoded`
	TypeFormUtf8 = `application/x-www-form-urlencoded; charset=utf-8`

	TypeMulti     = `multipart/form-data`
	TypeMultiUtf8 = `multipart/form-data; charset=utf-8`

	// Used for `(*Request).ParseMultipartForm`.
	// 32 MB, same as the default in the "http" package.
	BufSize = 32 << 20
)

Variables

This section is empty.

Functions

func Decode 

func Decode(req *http.Request, out interface{}) error

Decodes an arbitrary request into an arbitrary Go structure. Transparently supports multiple content types: URL query, URL-encoded body, multipart body, and JSON. Read-only requests are decoded ONLY from the URL query, and other requests are decoded ONLY from the body. For non-JSON requests, the output must be a struct pointer. For JSON requests, the output may be a pointer to anything.

If the request is multipart, this also populates `req.MultipartForm` as a side effect. Downloaded files become available via `req.MultipartForm.File`.

Unlike `rd.Download`, this uses stream decoding for JSON, without buffering the entire body in RAM. For other content types, this should perform identically to `rd.Download`.

func Parse 

func Parse(input string, out r.Value) error

Missing feature of the standard library: parse arbitrary text into arbitrary Go value. Used internally by `rd.Form.Decode`. Exported for enterprising users. Adapted from "github.com/mitranim/untext". The output must be a settable non-pointer. Its original value is ignored/overwritten. If the output implements `rd.Parser` or `encoding.TextUnmarshaler`, the corresponding method is invoked automatically. Otherwise the output must be a "well-known" Go type: number, bool, string, or byte slice. Unlike "encoding/json", this doesn't support parsing into dynamically-typed `interface{}` values.

func ParseSlice 

func ParseSlice(inputs []string, out r.Value) error

Missing feature of the standard library: parse arbitrary strings into arbitrary Go values. Used internally by `rd.Form.Decode`. Exported for enterprising users. Adapted from "github.com/mitranim/untext". The output must be a settable non-pointer. Its original value is ignored/overwritten. If the output implements `rd.SliceParser`, the corresponding method is invoked automatically. Otherwise it must be a slice of some concrete type, where each element is parsed via `rd.Parse`. Unlike "encoding/json", this doesn't support parsing into dynamically-typed `interface{}` values.

func TryDecode 

func TryDecode(req *http.Request, out interface{})

Shortcut for `rd.Decode` that panics on errors.

Types

type Dec 

type Dec interface {
	Decoder
	Haserer
	Setter
}

Returned by `rd.Download`. Implemented by all decoder types in this package.

func Download 

func Download(req *http.Request) (Dec, error)

Downloads and partially decodes the request, returning a fully-buffered decoder, appropriately chosen for the request type. Transparently supports multiple content types: URL query, URL-encoded body, multipart body, and JSON. For read-only requests, returns `rd.Form` populated ONLY from the URL query. For JSON requests, returns `rd.Json` containing the fully-buffered response body, without any decoding or modification. For URL-encoded requests and multipart requests, returns `rd.Form` populated from the request body.

If the request is multipart, this also populates `req.MultipartForm` as a side effect. Downloaded files become available via `req.MultipartForm.File`.

func TryDownload 

func TryDownload(req *http.Request) Dec

Shortcut for `rd.Download` that panics on errors.

type Decoder 

type Decoder interface{ Decode(interface{}) error }

Short for "decoder". Represents a stored request body and decodes that body into Go structures. The exact representation and the supported output depends on the request:

  • GET request -> backed by `url.Values`, decodes into structs.
  • Form-encoded request -> backed by `url.Values`, decodes into structs.
  • Multipart request -> backed by `url.Values`, decodes into structs.
  • JSON request -> backed by `[]byte`, decodes into anything.

Once constructed, a decoder is considered immutable, concurrency-safe, and can decode into arbitrary outputs any amount of times. Also see `rd.Json` and `rd.Form` for type-specific semantics.

type Err 

type Err struct {
	Status int   `json:"status"`
	Cause  error `json:"cause"`
}

Wraps another error, adding an HTTP status code. Some errors returned by this package are wrapped with codes such as 400 and 500.

func (Err) AppendTo added in v0.2.3 

func (self Err) AppendTo(buf []byte) []byte

Appends the error representation. Used internally by `.Error`.

func (Err) Error 

func (self Err) Error() string

Implement the `error` interface.

func (Err) Format added in v0.2.4 

func (self Err) Format(out fmt.State, verb rune)

Implement `fmt.Formatter`.

func (Err) HttpStatusCode 

func (self Err) HttpStatusCode() int

Returns `.Status`. Implements a hidden interface supported by `github.com/mitranim/rout`.

func (Err) Unwrap 

func (self Err) Unwrap() error

Implement a hidden interface in "errors".

type Form 

type Form url.Values

Missing feature of the standard library: decodes URL-encoded and multipart requests into Go structs. Implements `rd.Decoder`. Transparently used by `rd.Decode` and `rd.Download` for appropriate request methods and content types.

Similarities with "encoding/json":

  • Uses reflection to decode into arbitrary outputs.

  • Uses the "json" field tag.

  • Supports embedded structs.

  • Supports arbitrary field types.

  • Supports customizable decoding via `encoding.TextUnmarshaler` and `rd.Parser` interfaces.

Differences from "encoding/json":

  • The top-level value must be a struct.

  • Doesn't support nested non-embedded structs.

  • Decodes only into fields with a "json" name, ignoring un-named fields.

  • For source fields which are "null", zeroes the corresponding fields of the output struct, instead of leaving them as-is. "null" is defined as:

  • []string(nil)

  • []string{}

  • []string{“}

  • Has better performance.

func (Form) Decode 

func (self Form) Decode(outVal interface{}) (err error)

Implement `rd.Decoder`, decoding into a struct. See `rd.Form` for the decoding semantics.

func (*Form) Download 

func (self *Form) Download(req *http.Request) error

Similar to the top-level function `rd.Download`, but without JSON support. For read-only requests, populates the receiver from the URL query. For non-read-only requests, populates the receiver from the request body, choosing the parsing mode from the request's content type header.

If the request is multipart, this also populates `req.MultipartForm` as a side effect. Downloaded files become available via `req.MultipartForm.File`.

func (*Form) DownloadBody 

func (self *Form) DownloadBody(req *http.Request, typ string) error

Downloads the request body and populates the receiver based on the given content type. Used by `(*rd.Form).Download`.

func (*Form) DownloadForm 

func (self *Form) DownloadForm(req *http.Request) error

Assumes that the request has a URL-encoded body, downloads that body as a side effect, and populates the receiver.

func (*Form) DownloadMultipart 

func (self *Form) DownloadMultipart(req *http.Request) error

Assumes that the request has a multipart body, downloads that body as a side effect, and populates the receiver. Uses the default buffer size of 32 megabytes.

func (*Form) DownloadMultipartWith 

func (self *Form) DownloadMultipartWith(req *http.Request, maxMem int64) error

Assumes that the request has a multipart body, downloads that body as a side effect, and populates the receiver. Passes the provided buffer size to `(*http.Request).ParseMultipartForm`.

func (Form) Has 

func (self Form) Has(key string) bool

Implement `rd.Haser`. Returns true if the key is present in the query map, regardless of its value.

func (Form) Haser 

func (self Form) Haser() Haser

Implement `rd.Haserer` by returning self..

func (*Form) Parse 

func (self *Form) Parse(src string) error

Implements `rd.Parser` via `url.ParseQuery`.

func (Form) Set added in v0.2.2 

func (self Form) Set() Set

Implement `rd.Setter` by creating an `rd.Set` composed of the keys present in the form decoder.

func (*Form) UnmarshalText 

func (self *Form) UnmarshalText(src []byte) error

Implements `encoding.TextUnmarshaler`.

func (*Form) Zero 

func (self *Form) Zero()

Deletes all key-values from the receiver.

type Haser 

type Haser interface{ Has(string) bool }

Represents an immutable string set, answering the question "is a particular key contained in this set". All decoders in this package implement either `rd.Haser` or `rd.Haserer`, answering the question "was the key present at the top level of the request".

type Haserer 

type Haserer interface{ Haser() Haser }

Converts to `rd.Haser`. Implemented by all decoder types in this package. For `rd.Form`, this is just a cast. For `rd.Json`, this involves reparsing the JSON to build a set of keys.

type Json 

type Json []byte

Implements `rd.Decoder` via `json.Unmarshal`. Unlike other decoders, supports arbitrary output types, not just structs. Unlike other decoders, this doesn't directly implement `rd.Haser`, but it does implement `rd.Haserer` which has a minor cost; see `rd.Json.Haser`.

func (Json) Decode 

func (self Json) Decode(out interface{}) error

Implement `rd.Decoder` by calling `json.Unmarshal`. The output must be a non-nil pointer to an arbitrary Go value.

func (*Json) Download 

func (self *Json) Download(req *http.Request) error

Fully downloads the request body and stores it as-is, without any modification or validation. Used by `rd.Download`.

func (Json) Haser 

func (self Json) Haser() Haser

Implement `rd.Haserer` by calling `rd.Json.Set`.

func (Json) Set added in v0.2.2 

func (self Json) Set() Set

Implement `rd.Setter`. Returns an instance of `rd.Set` with the keys of the top-level object in the JSON text. Assumes that JSON is either valid or completely empty (only whitespace). Panics on malformed JSON.

Unlike other decoders provided by this package, `rd.Json.Haser` is not a free cast; it has to re-parse the JSON to build the set of top-level object keys. It uses a custom JSON decoder optimized for this particular operation, which avoids reflection and performs much better than "encoding/json". The overhead should be barely measurable.

Caution: for efficiency, this assumes that `rd.Json` is immutable, and performs an unsafe cast from `[]byte` to `string`. Parts of the resulting string are used as map keys. Mutating the JSON slice after calling this method will result in undefined behavior. Mutating the resulting set is perfectly safe.

func (*Json) UnmarshalJSON 

func (self *Json) UnmarshalJSON(src []byte) error

Implement `json.Unmarshaler` by storing the input as-is, exactly like `json.RawMessage`. This allows to include `rd.Json` into other data structures, deferring the decoding until later.

func (*Json) Zero 

func (self *Json) Zero()

Clears the slice, preserving the capacity if any.

type Parser 

type Parser interface{ Parse(string) error }

Missing part of the "encoding" package. Commonly implemented by various types across various libraries. If implemented, this is automatically used when decoding strings from URL queries and form bodies. This is NOT used when decoding from JSON; instead, types are expected to implement either `json.Unmarshaler` or `encoding.TextUnmarshaler`.

type Set 

type Set map[string]struct{}

Simple string set backed by a Go map. Implements `rd.Haser`. Generated by `rd.Json.Haser`.

func (Set) Add 

func (self Set) Add(val string)

Adds the value to the set.

func (Set) Del 

func (self Set) Del(val string)

Deletes the value from the set.

func (Set) Has 

func (self Set) Has(val string) bool

Implement `rd.Haser`. Returns true if the value is among the map's keys.

func (Set) Haser 

func (self Set) Haser() Haser

Implements `rd.Haserer` by returning self.

type Setter added in v0.2.2 

type Setter interface{ Set() Set }

Converts to `rd.Set`. Implemented by all decoder types in this package.

type SliceParser 

type SliceParser interface{ ParseSlice([]string) error }

Interface for types that decode from `[]string`. Useful for parsing lists from form-encoded sources such as URL queries and form bodies. Should be implemented by any non-slice type that wants to be parsed from a list. Slice types don't need this; this package handles them automatically, by parsing items individually.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.