composer

package module
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jan 29, 2022 License: MIT Imports: 12 Imported by: 0

README

go-multipart-composer

PkgGoDev Test Status codecov

Prepares bodies of HTTP requests with MIME multipart messages according to RFC7578 without reading entire file contents to memory. Instead of writing files to a multipart writer right away, it collects readers for each part of the form and lets them stream to the network once the request has been sent. Avoids buffering of the request body simpler than with goroutines and pipes. See the documentation for more information.

Installation

Add this package to go.mod and go.sub in your Go project:

go get github.com/prantlf/go-multipart-composer

Usage

Upload a file with comment:

import (
  "net/http"
  composer "github.com/prantlf/go-multipart-composer"
)
// compose a multipart form-data content
comp := composer.NewComposer()
comp.AddField("comment", "a comment")
err := comp.AddFile("file", "test.txt")
// post a request with the generated content type and body
resp, err := http.DefaultClient.Post("http://host.com/upload",
  comp.FormDataContentType(), comp.DetachReader())

If the server does not support chunked encoding and requires Content-=Length in the header:

comp := composer.NewComposer()
comp.AddField("comment", "a comment")
err := comp.AddFile("file", "test.txt")
reqBody, contentLength, err := comp.DetachReaderWithSize()
if err != nil {
  comp.Close() // DetachReaderWithSize does not close the composer on failure
  log.Fatal(err)
}
// post a request with the generated body, content type and content length
req, err := http.NewRequest("POST", "http://host.com/upload", reqBody)
req.Header.Add("Content-Type", comp.FormDataContentType())
req.ContentLength = contentLength
resp, err := http.DefaultClient.Do(request)

See the documentation for the full interface.

Documentation

Overview

Package composer prepares bodies of HTTP requests with MIME multipart messages without reading entire file contents to memory. Instead of writing files to multipart Writer right away, it collects Readers for each part of the form and lets them stream to the network once the request has been sent. Avoids buffering of the request body simpler than with goroutines and pipes.

Text fields and files can be appended by convenience methods: 

comp := composer.NewComposer()
comp.AddField("comment", "a comment")
err := comp.AddFile("file", "test.txt")

The multipart form-data content type and a reader for the full request body can be passed directly the HTTP request methods. They close a closable writer even in case of failure: 

resp, err := http.DefaultClient.Post("http://host.com/upload",
  comp.FormDataContentType(), comp.DetachReader())
Example 
package main

import (
	"log"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	// Create a new multipart message composer with a random boundary.
	comp := composer.NewComposer()
	// Close added files or readers if a failure before DetachReader occurred.
	// Not needed if you add no file, or if you add or just one file and then
	// do not abandon the composer before you succeed to return the result of
	// DetachReader or DetachReaderWithSize.
	defer comp.Close()

	// Add a textual field.
	comp.AddField("comment", "a comment")
	// Add a file content. Fails if the file cannot be opened.
	if err := comp.AddFile("file", "demo/test.txt"); err != nil {
		log.Fatal(err)
	}

	// Get the content type of the composed multipart message.
	contentType := comp.FormDataContentType()
	// Collect the readers for added fields and files to a single compound
	// reader including the total size and empty the composer by detaching
	// the original readers from it.
	reqBody, contentLength, err := comp.DetachReaderWithSize()
	if err != nil {
		log.Fatal(err)
	}
	// Close added files or readers after the request body reader was used.
	// Not needed if the consumer of reqBody is called right away and will
	// guarantee to close the reader even in case of failure. Because this
	// is the case here, here it is for demonstration purposes only.
	defer reqBody.Close()

	// Make a network request with the composed content type and request body.
	demo.PrintRequestWithLength(contentLength, contentType, reqBody)
}

Output:

Content-Length: 383
Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="comment"

a comment
--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=utf-8

text file content
--1879bcd06ac39a4d8fa5--

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Composer 

type Composer struct {
	// CloseReaders, if set to false, prevents closing of added files
	// or readers when Close is called, or when the reader returned by
	// DetachReader is closed. The initial value set by NewComposer is true.
	CloseReaders bool
	// contains filtered or unexported fields
}

A Composer generates multipart messages with delayed content supplied by readers.

Example 
package main

import (
	"fmt"

	composer "github.com/prantlf/go-multipart-composer"
)

func main() {
	// Create an invalid composer for results returned in case of error.
	comp := composer.Composer{}

	fmt.Printf("Empty composer: %v", comp.Boundary() == "")
}

Output:

Empty composer: true

func NewComposer 

func NewComposer() *Composer

NewComposer returns a new multipart message Composer with a random boundary.

If you are going to add parts with readers that needs closing (files), defer a call to Close in case an error occurs, the best right after calling this method.

Example 
package main

import (
	"fmt"

	composer "github.com/prantlf/go-multipart-composer"
)

func main() {
	// Create a new multipart message composer with a random boundary.
	comp := composer.NewComposer()

	fmt.Printf("Close added files or readers: %v", comp.CloseReaders)
}

Output:

Close added files or readers: true

func (*Composer) AddField 

func (c *Composer) AddField(name, value string)

AddField creates a new multipart section with a field value. It inserts a header with the provided field name and value.

Example 
package main

import (
	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Add a textual field.
	comp.AddField("foo", "bar")

	demo.PrintRequestBody(comp.DetachReader())
}

Output:

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="foo"

bar
--1879bcd06ac39a4d8fa5--

func (*Composer) AddFieldReader 

func (c *Composer) AddFieldReader(name string, reader io.Reader)

AddFieldReader creates a new multipart section with a field value. It inserts a header using the given field name and then appends the value reader.

Example 
package main

import (
	"strings"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Add a textual field with a value supplied by a reader.
	comp.AddFieldReader("foo", strings.NewReader("bar"))

	demo.PrintRequestBody(comp.DetachReader())
}

Output:

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="foo"

bar
--1879bcd06ac39a4d8fa5--

func (*Composer) AddFile 

func (c *Composer) AddFile(fieldName, filePath string) error

AddFile is a convenience wrapper around AddFileReader. It opens the given file and uses its name, stats and content to create the new part.

The opened file wil be owned by the Composer. Do not forget to close the composer, once you do not need it, or defer the closure to perform it automatically in case of a failure.

Example 
package main

import (
	"log"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Add a file content. Fails if the file cannot be opened.
	if err := comp.AddFile("file", "demo/test.txt"); err != nil {
		log.Fatal(err)
	}

	demo.PrintRequestBody(comp.DetachReader())
}

Output:

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=utf-8

text file content
--1879bcd06ac39a4d8fa5--

func (*Composer) AddFileObject added in v1.1.0 

func (c *Composer) AddFileObject(fieldName string, file *os.File) error

AddFileObject is a convenience wrapper around AddFileReader. It uses the name, stats and content of the opened file to create the new part.

The opened file wil be owned by the Composer. Do not forget to close the composer, once you do not need it, or defer the closure to perform it automatically in case of a failure. However, do not close the source file. The reader taking part in the request body creation would fail.

func (*Composer) AddFileReader 

func (c *Composer) AddFileReader(fieldName, fileName string, reader io.Reader)

AddFileReader creates a new multipart section with a file content. It inserts a header using the given field name, file name and the content type inferred from the file extension, then appends the reader's content.

If the reader passed in is a ReaderCloser, it will be owned and eventually freed by the Composer. Do not forget to close the composer, once you do not need it, or defer the closure to perform it automatically in case of a failure. However, do not close the source file. The reader taking part in the request body creation would fail.

Example 
package main

import (
	"log"
	"os"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Add a file content supplied as a separate reader.
	file, err := os.Open("demo/test.txt")
	if err != nil {
		log.Fatal(err)
	}
	comp.AddFileReader("file", "test.txt", file)

	demo.PrintRequestBody(comp.DetachReader())
}

Output:

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=utf-8

text file content
--1879bcd06ac39a4d8fa5--

func (*Composer) AddPart added in v1.1.0 

func (c *Composer) AddPart(header textproto.MIMEHeader, reader io.Reader)

AddPart creates a new multipart section prepared earlier with CreatePart, CreateFieldPart or CreateFilePart. It inserts all headers prepared earlier and then appends the value reader.

func (*Composer) Boundary 

func (c *Composer) Boundary() string

Boundary returns the Composer's boundary.

Example 
package main

import (
	"fmt"

	composer "github.com/prantlf/go-multipart-composer"
)

func main() {
	comp := composer.NewComposer()

	// Get the initial randomly-genenrated boundary.
	boundary := comp.Boundary()

	fmt.Printf("Boundary set: %v", len(boundary) > 0)
}

Output:

Boundary set: true

func (*Composer) Clear 

func (c *Composer) Clear()

Clear closes all closable readers added by AddFileReader or AddFile and clears their collection, making the composer ready to start empty again.

Example 
package main

import (
	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()
	comp.AddField("foo", "bar")

	// Abandon the composed content and clear the added fields.
	comp.Clear()

	comp.AddField("foo", "bar")

	demo.PrintRequestBody(comp.DetachReader())
}

Output:

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="foo"

bar
--1879bcd06ac39a4d8fa5--

func (*Composer) Close 

func (c *Composer) Close() error

Close closes all closable readers added by AddFileReader or AddFile. If some of them fail, the first error will be returned.

Example 
package main

import (
	"log"
	"os"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Add a file reader which will be closed automatically.
	file, err := os.Open("demo/test.txt")
	if err != nil {
		log.Fatal(err)
	}
	comp.AddFileReader("file", "test.txt", file)

	// Close the added files and readers.
	comp.Close()
	if _, err := file.Stat(); err == nil {
		log.Fatal("open")
	}

	// Start again with disabled closing of files and readers.
	comp.Clear()
	comp.CloseReaders = false

	// Add a file reader which will not be closed automatically.
	file, err = os.Open("demo/test.txt")
	if err != nil {
		log.Fatal(err)
	}
	defer file.Close()
	comp.AddFileReader("file", "test.txt", file)

	// Adding a file by path is impossible if automatic closing is disabled.
	if err := comp.AddFile("file", "demo/test.txt"); err == nil {
		log.Fatal("added")
	}

	// Getting the final reader or closing the composer will not close the file.
	reqBody := comp.DetachReader()
	comp.Close()
	if _, err := file.Stat(); err != nil {
		log.Fatal(err)
	}

	demo.PrintRequest(comp.FormDataContentType(), reqBody)
}

Output:

Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5

--1879bcd06ac39a4d8fa5
Content-Disposition: form-data; name="file"; filename="test.txt"
Content-Type: text/plain; charset=utf-8

text file content
--1879bcd06ac39a4d8fa5--

func (*Composer) CreateFieldPart added in v1.1.0 

func (c *Composer) CreateFieldPart(name string) textproto.MIMEHeader

CreateFilePart creates a new multipart section for a field, but does not add it to the composer yet. Passing the returned header to AddPart will add it to the composer.

func (*Composer) CreateFilePart added in v1.1.0 

func (c *Composer) CreateFilePart(fieldName, fileName string) textproto.MIMEHeader

CreateFilePart creates a new multipart section for a file, but does not add it to the composer yet. Passing the returned header to AddPart will add it to the composer.

func (*Composer) CreatePart added in v1.1.0 

func (c *Composer) CreatePart(disposition map[string]string) textproto.MIMEHeader

CreateFilePart creates a new general multipart section, but does not add it to the composer yet. Passing the returned header to AddPart will add it to the composer.

func (*Composer) DetachReader 

func (c *Composer) DetachReader() io.ReadCloser

DetachReader finishes the multipart message by adding the trailing boundary end line to the output and moves the closable readers to be closed with the returned compound reader.

Example 
package main

import (
	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Get a multipart message with no parts.
	reqBody := comp.DetachReader()

	demo.PrintRequestBody(reqBody)
}

Output:

--1879bcd06ac39a4d8fa5--

func (*Composer) DetachReaderWithSize 

func (c *Composer) DetachReaderWithSize() (io.ReadCloser, int64, error)

DetachReaderWithSize finishes the multipart message by adding the trailing boundary end line to the output and moves the closable readers to be closed with the returned compound reader. It tries computing the total request body size, which will work if size was available for all readers.

If it fails, the composer instance will not be closed.

Example 
package main

import (
	"log"

	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Get a multipart message with no parts including its length.
	reqBody, contentLength, err := comp.DetachReaderWithSize()
	if err != nil {
		log.Fatal(err)
	}

	demo.PrintContentLength(contentLength)
	demo.PrintContentType(comp.FormDataContentType())
	demo.PrintRequestBody(reqBody)
}

Output:

Content-Length: 68
Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5

--1879bcd06ac39a4d8fa5--

func (*Composer) FormDataContentType 

func (c *Composer) FormDataContentType() string

FormDataContentType returns the value of Content-Type for an HTTP request with the body prepared by this Composer. It will include the constant "multipart/form-data" and this Composers's Boundary.

Example 
package main

import (
	composer "github.com/prantlf/go-multipart-composer"
	"github.com/prantlf/go-multipart-composer/demo"
)

func main() {
	comp := composer.NewComposer()

	// Get the content type for the composed multipart message.
	contentType := comp.FormDataContentType()

	demo.PrintContentType(contentType)
}

Output:

Content-Type: multipart/form-data; boundary=1879bcd06ac39a4d8fa5

func (*Composer) ResetBoundary 

func (c *Composer) ResetBoundary() error

ResetBoundary overrides the Composer's current boundary separator with a randomly generared one.

ResetBoundary must be called before any parts are added, or after all parts were detached by one of the DetachReader methods.

Example 
package main

import (
	"fmt"

	composer "github.com/prantlf/go-multipart-composer"
)

func main() {
	comp := composer.NewComposer()
	comp.SetBoundary("1")

	// Generate a new random boundary to separate the message parts.
	comp.ResetBoundary()

	fmt.Printf("Boundary reset: %v", len(comp.Boundary()) > 1)
}

Output:

Boundary reset: true

func (*Composer) SetBoundary 

func (c *Composer) SetBoundary(boundary string) error

SetBoundary overrides the Composer's initial boundary separator with an explicit value.

SetBoundary must be called before any parts are added, or after all parts were detached by one of the DetachReader methods. may only contain certain ASCII characters, and must be non-empty and at most 70 bytes long. (See RFC 2046, section 5.1.1.)

Example 
package main

import (
	"fmt"

	composer "github.com/prantlf/go-multipart-composer"
)

func main() {
	comp := composer.NewComposer()

	// Set an explicit boundary to separate the message parts.
	comp.SetBoundary("3a494cd3b73de6555202")

	fmt.Print(comp.Boundary())
}

Output:

3a494cd3b73de6555202

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.