bozr

Minimalistic tool to perform API tests based on JSON description

Usage

bozr [OPTIONS] (DIR|FILE)

Options:
  -H, --host      Base URL prefix for test calls
  -w, --workers   Execute in parallel with specified number of workers
      --rewrite-response-location Rewrite response header (Location) before it get checked against expectations
      --header    Extra header to add to each request
      --throttle  Execute no more than specified number of requests per second (in suite)
  -h, --help      Print usage
  -i, --info      Enable info mode. Print request and response details.
  -d, --debug     Enable debug mode
      --junit     Enable junit xml reporter
  -v, --version   Print version information and quit

Examples:
  bozr ./examples/suite-file.suite.json
  bozr -w 2 ./examples
  bozr -H http://example.com ./examples
  bozr --header "X-Test-LaunchID: RDQ1341" ./examples

Usage demo

Installation

If you are using Linux, macOS or WSL you can use the following command

curl -fsSL https://raw.github.com/kajf/bozr/master/tools/install.sh | sudo bash

Otherwise, you need to download the latest binary release and unpack it manually.

Test Suite Format

Test suite (suite_name.suite.json)

├ Test A [test case]
|   ├ name
|   ├ ignore [ignore test due to a specified reason]
|   ├ args [value(s) for placeholders to use in request params, headers or body]
│   ├ Call one
|   |   ├ args 
│   │   ├ on [single http request]
│   │   ├ expect [http response asserts: code, headers, body, schema, etc.]
│   │   └ remember [optionally remember variable(s) for the next call to use in request params, headers or body]
│   └ Call two
|       ├ args
│       ├ on
│       ├ expect
│       └ remember
└ Test B
    ├ name
    └ Call one
        ├ args
        ├ on
        ├ expect
        └ remember

Suite file extension

All suites must have .suite.json extension.

If you want to temporary disable suite, change extension to .xsuite.json. Bozr does not execute ignored suites, but reports all test cases as skipped.

Section 'On'

Represents http request parameters

{
  "on": {
    "method": "POST",
    "url": "/api/company/users",
    "headers": {
        "Accept": "application/json",
        "Content-Type": "application/json"
    },
    "params": {
        "role": "admin"
    },
    "bodyFile" : "admins.json"
  }
}

Section 'Expect'

Represents assertions for http response of the test call.

Response:

{
  "errors": [{ "code": "FOO" }]
}

Passing Test:

{
  "expect": {
    "statusCode": 200,
    "contentType": "application/json",
    "bodyPath": {
        "errors.size()": 1
    }
  }
}

'Expect' body matchers

Response:

{
  "users": [
    {"name":"John", "surname":"Wayne", "age": 38},
    {"name":"John", "surname":"Doe", "age": 12}
  ],
  "errors": []
}

Could be used to partially match response body:

{
  "expect": {
    "body": {
      "users": [
        {"name":"John", "age": 38}
      ]
    }
  }
}

Exact match (no new properties in the response) can be checked using "exactBody".

'Expect' body path matchers

Response:

{
  "users": [
    {"name":"John", "surname":"Wayne", "age": 38},
    {"name":"John", "surname":"Doe", "age": 12}
  ],
  "errors": []
}

Passing Test expect fragment:

{
  "expect": {
    "bodyPath": {
        "users.1.surname" : "Doe",
        "users.name":"John",
        "users": {
          "name":"John",
          "age": 12
        },
        "errors.size()": 0
    }
  }
}

XML:

• To match attribute use - symbol before attribute name. E.g. users.0.-id
• Namespaces are ignored
• Only string matcher values are supported (since xml has no real data types, so everything is a string)

'Expect absent' body matchers

Represents paths not expected to be in response body. Mostly used for security checks (e.g. returned user object should not contain password or credit card number fields) Path fromat is the same as in expect.bodyPath section

{
  "expect": {
    "absent": ["user.cardNumber", "user.password"]
  }
}

Section 'Args'

Specifies placeholder values for future reference (within test scope)

Placeholder values could be used inside on.url, on.params, on.headers, on.body, on.bodyFile, expect.headers, expect.body, expect.bodyPath sections.

{
  "args": {
    "currencyCode" : "USD",
    "magicNumber" : "12f"
  }
}

Given args are defined like above, placeholders {currencyCode} and {magicNumber} may be used in correspondent test case.

example_bodyfile.json

{
  "bankAccount": {
    "currency": "{currencyCode}",
    "amount": 1000,
    "secret": "{magicNumber}"
  }
}

Resulting data will contain "USD" and "12f" values instead of placeholders.

{
  "bankAccount": {
    "currency": "USD",
    "amount": 1000,
    "secret": "12f"
  }
}

{
  "on": {
    "method": "GET",
    "url": "{hateoas_reference}",
    "headers": {
      "X-Secret-Key": "{secret_key}"
    }
  }
}

Duplicated or unused argements are reported as test failure

Functions and data generation

Hashes

.SHA1 calculates SHA-1 hash of it's argument

{
  "hash": "{{ .SHA1 `Username` }}"
}

.Base64 does Base64 transformation on provided string

{
  "encoded": "{{ .Base64 `some value` }}"
}

Date and time

.Now returns current date/time

{
  "currentDate": "{{ .Now | .FormatDateTime `2006-01-02` }}"
}

In example above 'currentDate' argument will have value of current date in yyyy-mm-dd format

It is also possible to specify IANA timezone

{
  "currentDateInNY": "{{ `America/New_York` | .Now }}"
}

.DaysFromNow returns date/time that is N days from now

{
  "yesterday": "{{-1 | .DaysFromNow | .FormatDateTime `2006-01-02` }}"
}

In example above 'yesterday' argument will have value of yesterday's date in yyyy-mm-dd format

.FormatDateTime returns string representation of date/time (useful in combination with .Now or .DaysFromNow)

Function uses example-based format (constant date '2006-01-02T15:04:05Z07:00' used as example instead of pattern)

.CurrentTimestampSec returns number representing current date/time in Unix format

SOAP

.WSSEPasswordDigest calculates password digest according to Web Service Security specification

{
  "digest": "{{ .WSSEPasswordDigest `{nonce}` `{created}` `{password}` }}"
}

Section 'Remember'

Similar to args section, specifies plaseholder values for future reference (within test case scope).

The difference is that values for placeholders are taken from response (syntax is similar to expect matchers).

There are two types of sources for values to remember: response body and headers.

{
  "remember": {
    "bodyPath": {
      "createdId": "path.to.id"
    },
    "headers": {
      "loc": "Location"
    }
  }
}

This section allows more complex test scenarios like:

• 'request login token, remember, then use remembered {token} to request some data and verify'
• 'create resource, remember resource id from response, then use remembered {id} to delete resource'

Rewrite response location

--rewrite-response-location flag allows to modify Location header of all response before they are passed to the expectations for verification. Value is a go template with the following variable available

For example to fix HATEOAS links generated by the app with HTTPS-redirect enabled while running on localhost

bozr \
      -H http://127.0.0.1:8080/api \
      --header "X-Forwarded-Proto:https" \
      --rewrite-response-location="{{index . \"ctx:base_url\"}}{{.response_header_location.Path}}" \
      examples/rewrite-response.suite.json

Using environment and context variables in tests

Similar to args and remember sections, OS environment variables could be used as placeholder values for future reference (within test case scope).

Given MY_FILTER environment variable exists in terminal session, the following syntax with env prefix enables its usage

{
  "on": {
    "url": "http://example.com",
    "method": "GET",
    "params": {
      "filter": "{env:MY_FILTER}"
    }
  }
}

Context variables are available with ctx prefix

List of context variables

{
  "expect" : {
    "bodyPath": {
      "_links.delete" : "{ctx:base_url}/my-resource/123"
    } 
  }
}

Editor integration

To make work with test files convenient, we suggest to configure you text editors to use this json schema. In this case editor will suggest what fields are available and highlight misspells.