hargo

package module
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 8, 2021 License: MIT Imports: 19 Imported by: 22

README

Hargo

Hargo Build Status GoDoc Go Report Card GitHub license GitHub issues Twitter

Hargo parses HAR files, can convert to curl format, and serve as a load test driver.

NAME:
   hargo - work with HTTP Archive (.har) files

USAGE:
   hargo <command> [arguments] <.har file>

VERSION:
   0.1.2-dev.57 (da53069)

AUTHOR:
   Mark A. Richman <mark@markrichman.com>

COMMANDS:
     fetch, f     Fetch URLs in .har
     curl, c      Convert .har to curl
     run, r       Run .har file
     validate, v  Validate .har file
     dump, d      Dump .har file
     load, l      Load test .har file
     help, h      Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug        Show debug output
   --help, -h     show help
   --version, -v  print the version

COPYRIGHT:
   (c) 2021 Mark A. Richman

Building and Running Hargo

git clone https://github.com/mrichman/hargo.git
cd hargo
make install
hargo validate test/golang.org.har

About HAR Files

If you use Google Chrome, you can record these files by following the steps below:

  1. Right-click anywhere on that page and click on Inspect Element to open Chrome's Developer Tools
  2. The Developer Tools will open as a panel at the bottom of the page. Click on the Network tab.
  3. Click the Record button, which is the solid black circle at the bottom of the Network tab, and you'll start recording activity in your browser.
  4. Refresh the page and start working normally
  5. Right-click within the Network tab and click Save as HAR with Content to save a copy of the activity that you recorded.
  6. Within the file window, save the HAR file.

Commands

Fetch

The fetch command downloads all resources references in .har file:

hargo fetch foo.har

This will produce a directory named hargo-fetch-yyyymmddhhmmss containing all assets references by the .har file. This is similar to what you'd see when invoking wget on a particular URL.

Curl

The curl command will output a curl command line for each entry in the .har file.

hargo curl foo.har

Run

The run command executes each HTTP request in .har file:

hargo run foo.har

This is similar to fetch but will not save any output.

Validate

The validate command will report any errors in the format of a .har file.

hargo validate foo.har

HAR file format is defined here: https://w3c.github.io/web-performance/specs/HAR/Overview.html

Dump

Dump prints information about all HTTP requests in .har file

hargo dump foo.har

Load

Hargo can act as a load test agent. Given a .har file, hargo can spawn a number of concurrent workers to repeat each HTTP request in order. By default, hargo will spawn 10 workers and run for a duration of 60 seconds.

Hargo will also save its results to InfluxDB, if available. Each HTTP response is stored as a point of time-series data, which can be graphed by Chronograf, Grafana, or similar visualization tool for analysis.

Docker

Build container
docker build -t hargo .
Run container
docker run --rm -v `pwd`/test:/test hargo hargo run /test/golang.org.har

Docker-compose

The example docker-compose file will start three containers:

  • hargo
  • influxdb
  • grafana

The hargo container will first needs to be built. See build. When the compose file is run it will start a hargo load process that will write the results to InfluxDB. This InfluxDB instance can be viewed using the grafana container. This contains an example dashboard showing the latency of the executed request. Username/password for all the containers is hargo/hargo.

commands
cd example/docker-compose
docker-compose up
docker-compose down -v

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Dump

func Dump(r *bufio.Reader)

Dump prints all HTTP requests in .har file

func EntryToRequest

func EntryToRequest(entry *Entry, ignoreHarCookies bool) (*http.Request, error)

EntryToRequest converts a HAR entry type to an http.Request

func Fetch

func Fetch(r *bufio.Reader) error

Fetch downloads all resources references in .har file

func LoadTest

func LoadTest(harfile string, file *os.File, workers int, timeout time.Duration, u url.URL, ignoreHarCookies bool, insecureSkipVerify bool) error

LoadTest executes all HTTP requests in order concurrently for a given number of workers.

func NewReader added in v0.1.2

func NewReader(r io.Reader) *bufio.Reader

NewReader returns a bufio.Reader that will skip over initial UTF-8 byte order marks. https://tools.ietf.org/html/rfc7159#section-8.1

func ReadStream added in v0.1.2

func ReadStream(file *os.File, entries chan Entry, stop chan bool)

ReadStream reads the har file as a stream and puts the entries on a chan for consumption. When the end of a file is reached it will start over until the stop signal is given. https://golang.org/pkg/encoding/json/#example_Decoder_Decode_stream

func Run

func Run(r *bufio.Reader, ignoreHarCookies bool, insecureSkipVerify bool) error

Run executes all entries in .har file

func ToCurl

func ToCurl(r *bufio.Reader) (string, error)

ToCurl converts a HAR Entry to a curl command line curl -X <method> -b "<name=value&name=value...>" -H <name: value> ... -d "<postData>" <url>

func Validate

func Validate(r *bufio.Reader) (bool, error)

Validate validates the format of a .har file

func WritePoint added in v0.1.2

func WritePoint(u url.URL, results chan TestResult)

WritePoint inserts data to InfluxDB

Types

type Browser

type Browser struct {
	// Required. The name of the browser that created the log.
	Name string `json:"name"`
	// Required. The version number of the browser that created the log.
	Version string `json:"version"`
	// Optional. A comment provided by the user or the browser.
	Comment string `json:"comment"`
}

Browser that created the log

type Cache

type Cache struct {
	// optional State of a cache entry before the request. Leave out this field
	// if the information is not available.
	BeforeRequest CacheObject `json:"beforeRequest,omitempty"`
	// optional State of a cache entry after the request. Leave out this field if
	// the information is not available.
	AfterRequest CacheObject `json:"afterRequest,omitempty"`
	// optional (new in 1.2) A comment provided by the user or the application.
	Comment string `json:"comment,omitempty"`
}

Cache contains info about a request coming from browser cache.

type CacheObject

type CacheObject struct {
	// optional - Expiration time of the cache entry.
	Expires string `json:"expires,omitempty"`
	// The last time the cache entry was opened.
	LastAccess string `json:"lastAccess"`
	// Etag
	ETag string `json:"eTag"`
	// The number of times the cache entry has been opened.
	HitCount int `json:"hitCount"`
	// optional (new in 1.2) A comment provided by the user or the application.
	Comment string `json:"comment,omitempty"`
}

CacheObject is used by both beforeRequest and afterRequest

type Content

type Content struct {
	// Length of the returned content in bytes. Should be equal to
	// response.bodySize if there is no compression and bigger when the content
	// has been compressed.
	Size int `json:"size"`
	// optional Number of bytes saved. Leave out this field if the information
	// is not available.
	Compression int `json:"compression,omitempty"`
	// MIME type of the response text (value of the Content-Type response
	// header). The charset attribute of the MIME type is included (if
	// available).
	MimeType string `json:"mimeType"`
	// optional Response body sent from the server or loaded from the browser
	// cache. This field is populated with textual content only. The text field
	// is either HTTP decoded text or a encoded (e.g. "base64") representation of
	// the response body. Leave out this field if the information is not
	// available.
	Text string `json:"text,omitempty"`
	// optional (new in 1.2) Encoding used for response text field e.g
	// "base64". Leave out this field if the text field is HTTP decoded
	// (decompressed & unchunked), than trans-coded from its original character
	// set into UTF-8.
	Encoding string `json:"encoding,omitempty"`
	// optional (new in 1.2) A comment provided by the user or the application.
	Comment string `json:"comment,omitempty"`
}

Content describes details about response content (embedded in <response> object).

type Cookie struct {
	// The name of the cookie.
	Name string `json:"name"`
	// The cookie value.
	Value string `json:"value"`
	// optional The path pertaining to the cookie.
	Path string `json:"path,omitempty"`
	// optional The host of the cookie.
	Domain string `json:"domain,omitempty"`
	// optional Cookie expiration time.
	// (ISO 8601 YYYY-MM-DDThh:mm:ss.sTZD, e.g. 2009-07-24T19:20:30.123+02:00).
	Expires string `json:"expires,omitempty"`
	// optional Set to true if the cookie is HTTP only, false otherwise.
	HTTPOnly bool `json:"httpOnly,omitempty"`
	// optional (new in 1.2) True if the cookie was transmitted over ssl, false
	// otherwise.
	Secure bool `json:"secure,omitempty"`
	// optional (new in 1.2) A comment provided by the user or the application.
	Comment bool `json:"comment,omitempty"`
}

Cookie contains list of all cookies (used in <request> and <response> objects).

type Creator

type Creator struct {
	// Required. The name of the application that created the log.
	Name string `json:"name"`
	// Required. The version number of the application that created the log.
	Version string `json:"version"`
	// Optional. A comment provided by the user or the application.
	Comment string `json:"comment,omitempty"`
}

Creator contains information about the log creator application

type Entry

type Entry struct {
	Pageref string `json:"pageref,omitempty"`
	// Date and time stamp of the request start
	// (ISO 8601 YYYY-MM-DDThh:mm:ss.sTZD).
	StartedDateTime string `json:"startedDateTime"`
	// Total elapsed time of the request in milliseconds. This is the sum of all
	// timings available in the timings object (i.e. not including -1 values) .
	Time float32 `json:"time"`
	// Detailed info about the request.
	Request Request `json:"request"`
	// Detailed info about the response.
	Response Response `json:"response"`
	// Info about cache usage.
	Cache Cache `json:"cache"`
	// Detailed timing info about request/response round trip.
	PageTimings PageTimings `json:"pageTimings"`
	// optional (new in 1.2) IP address of the server that was connected
	// (result of DNS resolution).
	ServerIPAddress string `json:"serverIPAddress,omitempty"`
	// optional (new in 1.2) Unique ID of the parent TCP/IP connection, can be
	// the client port number. Note that a port number doesn't have to be unique
	// identifier in cases where the port is shared for more connections. If the
	// port isn't available for the application, any other unique connection ID
	// can be used instead (e.g. connection index). Leave out this field if the
	// application doesn't support this info.
	Connection string `json:"connection,omitempty"`
	// (new in 1.2) A comment provided by the user or the application.
	Comment string `json:"comment,omitempty"`
}

Entry is a unique, optional Reference to the parent page. Leave out this field if the application does not support grouping by pages.

type Har

type Har struct {
	Log Log `json:"log"`
}

Har is a container type for deserialization

func Decode

func Decode(r *bufio.Reader) (Har, error)

Decode reads from a reader and returns Har object

type Log

type Log struct {

	// Required. Version number of the format.
	Version string `json:"version"`
	// Required. An object of type creator that contains the name and version
	// information of the log creator application.
	Creator Creator `json:"creator"`
	// Optional. An object of type browser that contains the name and version
	// information of the user agent.
	Browser Browser `json:"browser"`
	// Optional. An array of objects of type page, each representing one exported
	// (tracked) page. Leave out this field if the application does not support
	// grouping by pages.
	Pages []Page `json:"pages,omitempty"`
	// Required. An array of objects of type entry, each representing one
	// exported (tracked) HTTP request.
	Entries []Entry `json:"entries"`
	// Optional. A comment provided by the user or the application. Sorting
	// entries by startedDateTime (starting from the oldest) is preferred way how
	// to export data since it can make importing faster. However the reader
	// application should always make sure the array is sorted (if required for
	// the import).
	Comment string `json:"comment"`
}

Log represents the root of the exported data. This object MUST be present and its name MUST be "log".

type NVP

type NVP struct {
	Name    string `json:"name"`
	Value   string `json:"value"`
	Comment string `json:"comment,omitempty"`
}

NVP is simply a name/value pair with a comment

type Page

type Page struct {

	// Date and time stamp for the beginning of the page load
	// (ISO 8601 YYYY-MM-DDThh:mm:ss.sTZD, e.g. 2009-07-24T19:20:30.45+01:00).
	StartedDateTime string `json:"startedDateTime"`
	// Unique identifier of a page within the . Entries use it to refer the parent page.
	ID string `json:"id"`
	// Page title.
	Title string `json:"title"`
	// Detailed timing info about page load.
	PageTiming PageTiming `json:"pageTiming"`
	// (new in 1.2) A comment provided by the user or the application.
	Comment string `json:"comment,omitempty"`
}

Page object for every exported web page and one <entry> object for every HTTP request. In case when an HTTP trace tool isn't able to group requests by a page, the <pages> object is empty and individual requests doesn't have a parent page.

type PageTiming

type PageTiming struct {
	// Content of the page loaded. Number of milliseconds since page load started
	// (page.startedDateTime). Use -1 if the timing does not apply to the current
	// request.
	// Depeding on the browser, onContentLoad property represents DOMContentLoad
	// event or document.readyState == interactive.
	OnContentLoad int `json:"onContentLoad"`
	// Page is loaded (onLoad event fired). Number of milliseconds since page
	// load started (page.startedDateTime). Use -1 if the timing does not apply
	// to the current request.
	OnLoad int `json:"onLoad"`
	// (new in 1.2) A comment provided by the user or the application.
	Comment string `json:"comment"`
}

PageTiming describes timings for various events (states) fired during the page load. All times are specified in milliseconds. If a time info is not available appropriate field is set to -1.

type PageTimings

type PageTimings struct {
	Blocked int `json:"blocked,omitempty"`
	// optional - Time spent in a queue waiting for a network connection. Use -1
	// if the timing does not apply to the current request.
	DNS int `json:"dns,omitempty"`
	// optional - DNS resolution time. The time required to resolve a host name.
	// Use -1 if the timing does not apply to the current request.
	Connect int `json:"connect,omitempty"`
	// optional - Time required to create TCP connection. Use -1 if the timing
	// does not apply to the current request.
	Send int `json:"send"`
	// Time required to send HTTP request to the server.
	Wait int `json:"wait"`
	// Waiting for a response from the server.
	Receive int `json:"receive"`
	// Time required to read entire response from the server (or cache).
	Ssl int `json:"ssl,omitempty"`
	// optional (new in 1.2) - Time required for SSL/TLS negotiation. If this
	// field is defined then the time is also included in the connect field (to
	// ensure backward compatibility with HAR 1.1). Use -1 if the timing does not
	// apply to the current request.
	Comment string `json:"comment,omitempty"`
}

PageTimings describes various phases within request-response round trip. All times are specified in milliseconds.

type PostData

type PostData struct {
	//  Mime type of posted data.
	MimeType string `json:"mimeType"`
	//  List of posted parameters (in case of URL encoded parameters).
	Params []PostParam `json:"params"`
	//  Plain text posted data
	Text string `json:"text"`
	// optional (new in 1.2) A comment provided by the user or the
	// application.
	Comment string `json:"comment,omitempty"`
}

PostData describes posted data, if any (embedded in <request> object).

type PostParam

type PostParam struct {
	// name of a posted parameter.
	Name string `json:"name"`
	// optional value of a posted parameter or content of a posted file.
	Value string `json:"value,omitempty"`
	// optional name of a posted file.
	FileName string `json:"fileName,omitempty"`
	// optional content type of a posted file.
	ContentType string `json:"contentType,omitempty"`
	// optional (new in 1.2) A comment provided by the user or the application.
	Comment string `json:"comment,omitempty"`
}

PostParam is a list of posted parameters, if any (embedded in <postData> object).

type Request

type Request struct {
	// Request method (GET, POST, ...).
	Method string `json:"method"`
	// Absolute URL of the request (fragments are not included).
	URL string `json:"url"`
	// Request HTTP Version.
	HTTPVersion string `json:"httpVersion"`
	// List of cookie objects.
	Cookies []Cookie `json:"cookies"`
	// List of header objects.
	Headers []NVP `json:"headers"`
	// List of query parameter objects.
	QueryString []NVP `json:"queryString"`
	// Posted data.
	PostData PostData `json:"postData"`
	// Total number of bytes from the start of the HTTP request message until
	// (and including) the double CRLF before the body. Set to -1 if the info
	// is not available.
	HeaderSize int `json:"headerSize"`
	// Size of the request body (POST data payload) in bytes. Set to -1 if the
	// info is not available.
	BodySize int `json:"bodySize"`
	// (new in 1.2) A comment provided by the user or the application.
	Comment string `json:"comment"`
}

Request contains detailed info about performed request.

type Response

type Response struct {
	// Response status.
	Status int `json:"status"`
	// Response status description.
	StatusText string `json:"statusText"`
	// Response HTTP Version.
	HTTPVersion string `json:"httpVersion"`
	// List of cookie objects.
	Cookies []Cookie `json:"cookies"`
	// List of header objects.
	Headers []NVP `json:"headers"`
	// Details about the response body.
	Content Content `json:"content"`
	// Redirection target URL from the Location response header.
	RedirectURL string `json:"redirectURL"`
	// Total number of bytes from the start of the HTTP response message until
	// (and including) the double CRLF before the body. Set to -1 if the info is
	// not available.
	// The size of received response-headers is computed only from headers that
	// are really received from the server. Additional headers appended by the
	// browser are not included in this number, but they appear in the list of
	// header objects.
	HeadersSize int `json:"headersSize"`
	// Size of the received response body in bytes. Set to zero in case of
	// responses coming from the cache (304). Set to -1 if the info is not
	// available.
	BodySize int `json:"bodySize"`
	// optional (new in 1.2) A comment provided by the user or the application.
	Comment string `json:"comment,omitempty"`
}

Response contains detailed info about the response.

type TestResult

type TestResult struct {
	URL       string    `json:"url"`
	Status    int       `json:"status"` // 200, 500, etc.
	StartTime time.Time `json:"startTime"`
	EndTime   time.Time `json:"endTime"`
	Latency   int       `json:"latency"` // milliseconds
	Method    string    `json:"method"`
	HarFile   string    `json:"harfile"`
}

TestResult contains results for an individual HTTP request

Directories

Path Synopsis
cmd

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL