seagate-exos-x-api-go

seagate-exos-x-api-go

MC API Client v2.0.0

This is a Go language library for interfacing with Seagate's Management Controller (MC) CLI/API. This library is used to generate an MC OpenAPI specification and then use that specification to generate a Go CLient API library. The OpenAPI specification is generated using an input yaml file and then gathering meta data from the MC API to create the OpenAPI specification. This library leverages an HTTP interface to communicate with the Management Controller of a live Seagate RAID Storage System. In addition, a higher-level API layer is used by clients such as the CSI Driver, and that higher layer relies on the auto-generated Go Client library.

The Seagate EXOS X API.

The HTTP API operations are used to execute provisioning and management commands.

MC API Design

This library was designed to generate an MC OpenAPI specification, as well as provide a library that clients can use to communicate with the MC in a Seagate storage array.

For most users of this library, you will only need to access the generated MC API Go library. The original library (v1) was written by hand and included support for a handful of commands used by the Kubernetes CSI Driver. The current library (v2) uses openapi tools to auto-generated a Go Language Client library. This library uses the MC OpenAPI Specification (api/mc-openapi.yaml) to auto-generate the Go client library. That same spec can be used to generate a Python, JavaScript, or other libraries.

This design uses a layered approach to the client API - where clients can call the pkg/client functions to communicate with an MC, or use the pkg/v1 or pkg/v2 client API functions. The pkg/v1 was created to support the original CSI driver and uses the hand-written code which parses XML responses from the MC. The pkg/v2 client API was also hand-written and leverages the auto-generated pkg/client functions. The v2 client parses JSON data returned from the MC and is 4-5 times faster than v1 based on regression test suite timings. Having both a v1 and v2 versions allowed for an easy transition in any client from importing github.com/Seagate/seagate-exos-x-api-go/pkg/v1 to github.com/Seagate/seagate-exos-x-api-go/pkg/v2 without having to change function calls.

This library also includes code written to auto-generate a new MC OpenAPI specification, but that step should not be required in most cases. Below are notes regarding current support, a road map, and regression testing notes.

OpenAPI Implementation

This section defines the commands, response status codes, and resource objects implemented in the OpenAPI specification. This is a superset of what was developed in API v1. Commands were added that are being used by other clients needing to interface with the MC Storage Controller API.

The following table provides a list of all possible MC API commands and which ones are supported currently in the MC OpenAPI spec and Go Client library. There are a total of 291 commands, with 29 supported, roughly 10%.

Generation Steps

Note: validator/cmd/main.go is used to validate a handful of MC client library commands, extend as desired.

MC OpenAPI Roadmap

This section describes proposed next steps for implementing remaining commands.

• Create a new MC API command that returns a list of all supported commands along with their parameters and other information needed to auto-generate mc-commands.yaml.
• Expand MC OpenAPI support to include all commands.
• Expand MC OpenAPI support to include all command parameters and variations.
• Work on a system of having multiple versions of the MC OpenAPI spec corresponding to various product releases.

Steps Needed to Update the MC OpenAPI Specification

In order to add a new command that is supported by the MC OpenAPI specification and also supported by the Go Client Library, follow these steps:

• Update internal/generator/mc-commands.yaml to include the new command and save your changes:
  • The main sections to include are: command URI, meta, include, nested, and options.
  • The command URI is essentially how you would run a command in a shell. For example, show volumes becomes /show/volumes.
  • The meta tag value is displayed when you run a command and have it return JSON data, it is one of the properties in the JSON dictionary.
  • The include tagn almost always contains status.
  • The nested tag describes all other JSON dictionaries that are nested, and this can be recursive. Review the JSON data returned for a command.
  • The options tag describes all possible command arguments, their types, required or not, keyword required or not, and an optional description.
• In a terminal, run make generator
• In an editor, copy internal/generator/generator-example.conf to internal/generator/generator.conf and modify as needed. This conf file specifies which storage array use for generation.
• In a terminal, run make rung to create a new api/mc-openapi.yaml
• Note: The next two commands require docker to be installed and working.
• In a terminal, run make validate to validate that api/mc-openapi.yaml contains no OpenAPI errors
• In a terminal, run make generate to generate a new Go client library in pkg/client
• Note: You may need to update a file or create a new file in pkg/v2 that leverages the new command generated. This isn't required, but the v2 layer was created to provide a simplified layer between the CSI Driver and the MC API.
• The last step is to run the regression test suite for v1 and/or v2.

Regression Testing Using A Live System

This option runs a API Regression test suite using Ginkgo expressive tests.

This option requires a configuration file, for example: api-regression.conf.

Each user should create their own copy of api-regression.conf and not check it in. Please see api-regression-example.conf as an example. Use the ./api-regression -config <filename> option to specify your configuration file.

#
# api-regression configuration
#
# Properties:
#   'ip' is required and is the IP Address of the storage controller used for testing
#   'protocol" specifies http or https
#   'username' is required and is the login credentials for the storage controller
#   'password' is required and is the login credentials for the storage controller
#
#   'initiator' is an array of host initiators, such as the iSCSI IQN value, or SAS, or FC
#   'pool' is the storage pool used for creating volumes
#
# Notes:
#    To find the iSCSI initiator:
#    sudo cat /etc/iscsi/initiatorname.iscsi | grep -v "##" | awk -F= '{print $2}'
api-regression: 1.0.0

# Example Controller
ip: "<ipaddress>"
protocol: "https"
username: "<username>"
password: "<password>"
initiator: ["iqn.2004-10.com.ubuntu:01:b6f76364a18"]
pool: "A"

The suggestion is to create your own configuration file and do not check it into the repo. For example: cp api-regression.conf myconfig.conf and then run ./api-regression -debug 4 -config myconfig.conf --ginkgo.v.

There are many options for running Ginkgo test cases, here are a few using an i1.conf configuration file:

• make regession to build the api-regression executable.
• ./api-regression -config i1.conf -debug 4 --ginkgo.v --ginkgo.fail-fast --ginkgo.focus "v1" to run all the v1 tests
• ./api-regression -config i1.conf -debug 4 --ginkgo.v --ginkgo.fail-fast --ginkgo.focus "v2" to run all the v2 tests
• ./api-regression -config i1.conf -debug 4 --ginkgo.v --ginkgo.fail-fast --ginkgo.focus "v2Login" to run only the v2 Login tests
• ./api-regression -config i1.conf -debug 4 --ginkgo.v --ginkgo.fail-fast --ginkgo.focus "v2System" to run only the v2 System tests
• ./api-regression -config i1.conf -debug 4 --ginkgo.v --ginkgo.fail-fast --ginkgo.focus "v2Volume" to run only v2 Volume tests
• ./api-regression -config i1.conf -debug 4 --ginkgo.v --ginkgo.fail-fast --ginkgo.focus "v2Snapshot" to run only v2 Snapshot tests
• ./api-regression -config i1.conf -debug 4 --ginkgo.v --ginkgo.fail-fast to run only all tests

There are also many command line flags:

• ./api-regression --help to see all options
• -debug level to set the debug level to 0, 1, 2, 3, or 4
• -config filename to use an alternate configuration file
• -version to display the api-regression version and exit
• --ginkgo.fail-fast to stop testing after the first failure
• --ginkgo.v to run testing in verbose mode