mantl-api

command module
v0.1.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 19, 2015 License: Apache-2.0 Imports: 14 Imported by: 0

README

Mantl API

Build Status

An API interface to Mantl.

Currently, Mantl API allows you to install and uninstall DCOS packages on Mantl. More capabilities are planned for the future.

Table of Contents

Comparison to Other Software

Mantl API leverages the Mesosphere DCOS package repository and provides capabilities similar to the package command in the DCOS CLI. The goal is to provide a simple, API-driven way to install and uninstall pre-built packages on Mantl clusters. In the future, mantl-api will contain additional functionality for maintaining and operating Mantl clusters.

Mantl API on Mantl Clusters

As of the 0.5 release, Mantl API is installed by default on Mantl clusters. It is available on control nodes at the /api endpoint and, by default, is behind SSL and basic authentication. For example, you could get a list of packages by running a command like the following on a default Mantl install:

curl -k -u admin:mantlpw https://mantl-control-01/api/1/packages

Building

docker build -t mantl-api .

Deploying Manually

You can run Mantl API on your cluster via Marathon. An example mantl-api.json is included below:

{
  "id": "/mantl-api",
  "container": {
    "type": "DOCKER",
    "docker": {
      "image": "ciscocloud/mantl-api:0.1.0",
      "network": "BRIDGE",
      "portMappings": [
        { "containerPort": 4001, "hostPort": 0 }
      ]
    }
  },
  "instances": 1,
  "cpus": 1.0,
  "mem": 512,
  "constraints": [["hostname", "UNIQUE"]],
  "env": {
    "CONSUL_HTTP_SSL_VERIFY": "false",
    "MANTL_API_LOG_LEVEL": "debug",
    "MANTL_API_CONSUL": "https://consul.service.consul:8500",
    "MANTL_API_MESOS_PRINCIPAL": "mantl-api",
    "MANTL_API_MESOS_SECRET": "secret"
  },
  "healthChecks": [
    {
      "protocol": "HTTP",
      "path": "/health",
      "gracePeriodSeconds": 3,
      "intervalSeconds": 10,
      "portIndex": 0,
      "timeoutSeconds": 10,
      "maxConsecutiveFailures": 3
    }
  ]
}

You will need to replace the MANTL_API_MESOS_PRINCIPAL and MANTL_API_MESOS_SECRET variables with valid Mesos credentials. These can be found in the security.yml file that was generated when you ran security-setup for your Mantl cluster.

All Mantl API configuration can be set with environment variables. See below for the additional configuration options that are available.

To install Mantl API, you can submit the above json to Marathon.

curl -k -u admin:mantlpw -X POST -d @mantl-api.json -H "Content-type: application/json" https://mantl-control-01/marathon/v2/apps

You will need to replace admin:mantlpw with valid Marathon credentials (see marathon_http_credentials in your security.yml). You will also replace mantl-control-01 with the host of one your Mantl control nodes.

After a few moments, Mantl API should be running on your cluster.

Options
Argument Default Description
log-level info one of debug, info, warn, error, or fatal
log-format text specify output (text or json)
consul http://localhost:8500 Consul API address
marathon Discovered via a marathon service registered in Consul Marathon API address
marathon-user None Marathon API user
marathon-password None Marathon API password
marathon-no-verify-ssl False When True, disables SSL verification for the Marathon API
mesos Discovered via a mesos service with a leader tag registered in Consul Mesos API address
mesos-principal None Mesos principal for framework authentication
mesos-secret None Mesos secret for framework authentication
mesos-no-verify-ssl False When True, disables SSL verification for the Mesos API
listen :4001 Listen for connections at this address
zookeeper Discovered via a marathon service registered in Consul Comma-delimited list of zookeeper servers
force-sync False Forces a synchronization of all repository sources at startup
config-file None Specifies an optional configuration file

Every option can be set via environment variables prefixed with MANTL_API. For example, you can use MANTL_API_LOG_LEVEL for log-level, MANTL_API_CONSUL for consul, and so on. You can also specify all configuration from a TOML configuration file using the config-file argument.

Package Repository

mantl-universe contains the list of packages that work out-of-the-box on Mantl today. You can install any of the DCOS packages but you will likely have to customize some of the configuration to work on Mantl. Most of the packages assume that service discovery is provided by Mesos-DNS and need to be converted to work with the Consul DNS interface. Contributions are welcome!

Package Repository Structure

Mantl API is compatible with the package repository structure created for Mesosphere DCOS packages. Each repository contains a directory structure that looks something like this:

repo/packages
├── C
│   ├── cassandra
│   │   ├── 0
│   │   │   ├── config.json
│   │   │   ├── marathon.json
│   │   │   └── package.json
│   │   ├── 1
│   │   │   ├── command.json
│   │   │   ├── config.json
│   │   │   ├── marathon.json
│   │   │   └── package.json
├── H
│   └── hdfs
│       ├── 0
│       │   ├── command.json
│       │   ├── config.json
│       │   ├── marathon.json
│       │   └── package.json
│       ├── 1
│       │   ├── command.json
│       │   ├── config.json
│       │   ├── marathon.json
│       │   └── package.json
|__ ...

For each named package (cassandra and hdfs in the example above), there is a directory of JSON files corresponding to each version.

Mantl API supports the ability to layer repositories. Mantl API is configured to use two repository sources by default:

The order of the repositories is important. By default, Mesosphere Universe is configured with index 0 and Mantl Universe is configured with index 1. In the Consul KV store, the repositories are stored as follows:

Name Consul KV Path
Mesosphere Universe mantl-install/repository/0
Mantl Universe mantl-install/repository/1

Repositories with higher indexes are prioritized. When Mantl API receives a request to install a particular package, it will use the package definition found in the repository with the highest index. If there are any required files missing, it will look for the corresponding package version in the repositories with the lower indexes until it is able to construct a valid, installable package. For example, if mantl-install/repository/1/repo/packages/H/hdfs/1 (Mantl Universe) exists but does not have a config.json file, Mantl API will merge in the config.json file from mantl-install/repository/0/repo/packages/H/hdfs/1 (Mesosphere universe). This enables us to use the existing packages in the Mesosphere Universe without having to recreate everything in the Mantl Universe. Packages in the Mantl Universe repository typically just contain only the changes that are needed to run the package on Mantl clusters.

This also enables users to create their own repository to customize packages for their specific needs.

Synchronizing Repository Sources

The package repositories are synchronized to the Consul K/V backend. If you want to refresh your repositories, you can run the following command:

mantl-api sync --consul http://consul.service.consul:8500
Configuring Sources

If you want to use different source repositories, you can specify a configuration file that contains your source definitions using the --config-file argument. Here is an example configuration file that uses 2 sources — one named "mesosphere" from a git repository and the other named "mantl-universe" from the local file system:

[sources]

[sources.mesosphere]
path = "https://github.com/mesosphere/universe.git"
type = "git"
index = 0

[sources.mantl]
path = "/home/ryan/Projects/mantl-universe"
index = 1

The following attributes can be specified for each source:

  • path
  • type — git or filesystem (default)
  • index
  • branch — only applicable for the git type

Usage

Installing a Package

It is a single API call to install a package. In the example below, we are going to run Cassandra on our Mantl cluster.

curl -X POST -d "{\"name\": \"cassandra\"}" http://mantl-worker-003:4001/1/packages

You will need to replace mantl-worker-003 and 4001 with the host and port where Mantl API is running. You can use the Marathon API or UI to find this. You'll also need to make sure that the port is accessible to the machine where you are calling the API from. Adjust the security groups or firewall rules for your platform accordingly.

After about 5 minutes, you should have Cassandra up and running on your Mantl cluster.

Uninstalling a Package

Uninstalling is just as easy. Run the command below to uninstall Cassandra:

curl -X DELETE http://mantl-worker-003.jossware.org:4001/1/packages/cassandra

After a moment, Cassandra will have been removed from your cluster. This will also remove the Zookeeper state for the Cassandra framework. In the future, we will add more flexibility in being able to control what is uninstalled.

Endpoints

Endpoint Method Description
/health GET health check - returns OK with an HTTP 200 status
/1/packages GET list available packages
/1/packages POST install a package
/1/packages/:name GET provides information about a specific package
/1/packages/:name DELETE uninstalls a specific package
GET /health

GET /health: returns OK

curl http://mantl-worker-003:4001/health
OK
GET /1/packages

GET /1/packages: returns a JSON representation of packages available to install.

curl http://mantl-control-01:4001/1/packages | jq .
[
  {
    "name": "cassandra",
    "description": "Apache Cassandra running on Apache Mesos",
    "framework": true,
    "currentVersion": "0.2.0-1",
    "supported": true,
    "tags": [
      "mesosphere",
      "framework",
      "data",
      "database"
    ],
    "versions": {
      "0.1.0-1": {
        "version": "0.1.0-1",
        "index": "0",
        "supported": true
      },
      "0.2.0-1": {
        "version": "0.2.0-1",
        "index": "1",
        "supported": true
      }
    }
  },
  ...
  {
    "name": "swarm",
    "description": "Swarm is a Docker-native clustering system.",
    "framework": true,
    "currentVersion": "0.4.0",
    "supported": false,
    "tags": [
      "mesosphere",
      "framework",
      "docker"
    ],
    "versions": {
      "0.4.0": {
        "version": "0.4.0",
        "index": "0",
        "supported": false
      }
    }
  }
]
GET /1/packages/

GET /1/packages/<package>: returns a JSON representation of a package.

curl http://mantl-worker-001:4001/1/packages/cassandra | jq .
{
  "name": "cassandra",
  "description": "Apache Cassandra running on Apache Mesos",
  "framework": true,
  "currentVersion": "0.2.0-1",
  "supported": true,
  "tags": [
    "mesosphere",
    "framework",
    "data",
    "database"
  ],
  "versions": {
    "0.1.0-1": {
      "version": "0.1.0-1",
      "index": "0",
      "supported": true
    },
    "0.2.0-1": {
      "version": "0.2.0-1",
      "index": "1",
      "supported": true
    }
  }
}
POST /1/packages

POST /1/packages: post a JSON representation of a package to install.

curl -X POST -d "{\"name\": \"cassandra\"}" http://mantl-worker-001:4001/1/packages | jq .
{
  "id": "/cassandra/dcos-test",
  "cmd": "$(pwd)/jre*/bin/java $JAVA_OPTS -classpath cassandra-mesos-framework.jar io.mesosphere.mesos.frameworks.cassandra.framework.Main",
  "args": null,
  "user": null,
  "env": {
    "CASSANDRA_ZK_TIMEOUT_MS": "10000",
    "JAVA_OPTS": "-Xms256m -Xmx256m",
    "MESOS_AUTHENTICATE": "true",
    "CASSANDRA_RESOURCE_CPU_CORES": "0.1",
    "CASSANDRA_FAILOVER_TIMEOUT_SECONDS": "604800",
    "DEFAULT_SECRET": "secret",
    "CASSANDRA_BOOTSTRAP_GRACE_TIME_SECONDS": "120",
    "CASSANDRA_DEFAULT_DC": "DC1",
    "CASSANDRA_RESOURCE_MEM_MB": "768",
    "MESOS_ZK": "zk://zookeeper.service.consul:2181/mesos",
    "CASSANDRA_DATA_DIRECTORY": ".",
    "DEFAULT_PRINCIPAL": "mantl-install",
    "CASSANDRA_FRAMEWORK_MESOS_ROLE": "*",
    "CASSANDRA_CLUSTER_NAME": "dcos-test",
    "CASSANDRA_RESOURCE_DISK_MB": "16",
    "CASSANDRA_SEED_COUNT": "2",
    "CASSANDRA_ZK": "zk://zookeeper.service.consul:2181/cassandra-mesos/dcos-test",
    "CASSANDRA_NODE_COUNT": "3",
    "CASSANDRA_DEFAULT_RACK": "RAC1",
    "CASSANDRA_HEALTH_CHECK_INTERVAL_SECONDS": "60"
  },
  "instances": 1,
  "cpus": 0.5,
  "mem": 512,
  "disk": 0,
  "executor": "",
  "constraints": [],
  "uris": [
    "https://downloads.mesosphere.io/cassandra-mesos/artifacts/0.2.0-1/cassandra-mesos-0.2.0-1.tar.gz",
    "https://downloads.mesosphere.io/java/jre-7u76-linux-x64.tar.gz"
  ],
  "storeUrls": [],
  "ports": [
    0
  ],
  "requirePorts": false,
  "backoffFactor": 0,
  "container": null,
  "healthChecks": [
    {
      "path": "/health/cluster",
      "protocol": "HTTP",
      "portIndex": 0,
      "command": null,
      "gracePeriodSeconds": 120,
      "intervalSeconds": 15,
      "timeoutSeconds": 5,
      "maxConsecutiveFailures": 0,
      "ignoreHttp1xx": false
    },
    {
      "path": "/health/process",
      "protocol": "HTTP",
      "portIndex": 0,
      "command": null,
      "gracePeriodSeconds": 120,
      "intervalSeconds": 30,
      "timeoutSeconds": 5,
      "maxConsecutiveFailures": 3,
      "ignoreHttp1xx": false
    }
  ],
  "dependencies": [],
  "upgradeStrategy": {
    "minimumHealthCapacity": 0,
    "maximumOverCapacity": 0
  },
  "labels": {
    "MANTL_PACKAGE_NAME": "cassandra",
    "MANTL_PACKAGE_IS_FRAMEWORK": "true",
    "DCOS_PACKAGE_FRAMEWORK_NAME": "cassandra.dcos-test",
    "MANTL_PACKAGE_FRAMEWORK_NAME": "cassandra.dcos-test",
    "MANTL_PACKAGE_INDEX": "1",
    "MANTL_PACKAGE_VERSION": "0.2.0-1"
  },
  "acceptedResourceRoles": null,
  "version": "2015-09-17T22:19:24.576Z",
  "tasks": [],
  "deployments": [
    {
      "id": "dd113528-388c-4438-a524-9eba45aa2908"
    }
  ],
  "tasksStaged": 0,
  "tasksRunning": 0,
  "tasksHealthy": 0,
  "tasksUnhealthy": 0,
  "backoffSeconds": 0,
  "maxLaunchDelaySeconds": 3600
}
DELETE /1/packages/

DELETE /1/packages/<package>: post a JSON representation of package specific uninstall options.

curl -X DELETE -d "{\"name\": \"cassandra\"}" http://mantl-worker-001:4001/1/packages/cassandra

License

mantl-api is released under the Apache 2.0 license (see LICENSE)

Documentation

The Go Gopher

There is no documentation for this package.

Directories

Path Synopsis
utils

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL