brigade-github-app

Brigade Github App: Advanced GitHub Gateway for Brigade

This is a Brigade gateway that provides a GitHub App with deep integration to GitHub's new Check API.

Installation

The installation for this gateway is multi-part, and not particularly easy at the moment.

Prerequisites:

• A Kubernetes cluster running Brigade
• kubectl and Helm

You will also need to pick out a domain name (referenced as YOUR_DOMAIN below) to send GitHub requests to. Example: gh-gateway.example.com. If you don't want to do this, see the notes in Step 3.

1. Create a GitHub App

A GitHub app is a special kind of trusted entity that is associated with either your account or an orgs account.

https://developer.github.com/apps/building-github-apps/creating-a-github-app/

• Set the Homepage URL to https://brigade.sh
• Set the User Authorization Callback URL to FIXME
• Set the Webhook URL to https://YOUR_DOMAIN/events/github
• Set the Webhook Secret to a randomly generated string. Make note of that string
• Subscribe to the following events:
  • Repository contents: read
  • Issues: read
  • Repository metadata: read
  • Pull requests: read
  • Repository webhooks: read
  • Commit Statuses: Read And Write
  • Checks: Read and Write
• Subscribe to the following webhooks:
  • Check suite
  • Check run
  • Issue comment (if intending to handle issue or general PR comments)
  • Pull request (if opting to create/rerequest Check suites from incoming PRs)
  • Push (if needing to handle push events, such as tag pushes)
• Choose "Only This Account" to connect to the app.

Once you have submitted you will be prompted to create a private key. Create one and save it locally. You will put this in your values.yaml file in the next step.

2. Install the Helm chart into your cluster

The [Brigade Github App Helm Chart][brigade-github-app-chart] is hosted at the [brigadecore/charts][charts] repository.

You must install this gateway into the same namespace in your cluster where Brigade is already running.

Make sure the gateway is accessible on a public IP address. You can do that either by setting the Service to be a load balancer, or setting up the Ingress. We STRONGLY recommend setting up an ingress to use Kube-LEGO or another SSL proxy.

$ helm repo add brigade https://brigadecore.github.io/charts
$ helm inspect values brigade/brigade-github-app > values.yaml
$ # Edit values.yaml
$ helm install -n gh-app brigade/brigade-github-app

The private key you created in Step 1 should be put in your values.yaml file:

# Other stuff...
github:
  key: |
    YOUR KEY DATA GOES HERE!
    AND EVERY LINE NEEDS TO BE INDENTED

On RBAC-enabled clusters, pass --set rbac.enabled=true to the helm install command.

3. (RECOMMENDED) Create a DNS entry for your app

In the prerequisites section, we suggested that you create a domain. At this point, you should be able to link that domain to the IP generated by your Service or Ingress created above.

For example, to map an Ingress IP on our cluster (where we are using the kube-lego chart with the nginx-ingress chart), we can run this to get our IP:

$ kubectl get svc -n kube-system nginx-nginx-ingress-controller

(helm status on the appropriate charts will also give you this information)

NOTE: If you do not want to use a domain name, go change your GitHub App configuration to use the IP address directly.

4. Test the App from GitHub

Go to the Advanced tab and check out the Recent Deliveries section. This should show a successful test run. If it is not successful, you will need to troubleshoot why GitHub could not successfully contact your app.

Likely reasons:

• Your app is not listening on a public IP
• SSL certificate is invalid
• The URL you entered is wrong (Go to the General tab and fix it)
• The Brigade Github App is returning incorrect data

5. Install the App

Go to the Install App tab and enable this app for your account.

Accept the permissions it asks for. You can choose between All repos and Only select repositories, and click Install

It is easy to change from All Repos to Only Selected, and vice versa, so we recommend starting with one repo, and adding the rest later.

6. Add Brigade projects for each GitHub project

For each GitHub project that you enabled the app for, you will now need to create a Project.

Remember that projects contain secret data, and should be handled with care.

$ helm inspect values brigade/brigade-project > values.yaml
$ # Edit values.yaml

You will want to make sure to set:

• project, repository, and cloneURL to point to your repo
• sharedSecret to use the shared secret you created when creating the app

7. (OPTIONAL): Forwarding pull_request to check_suite

This gateway can enable a feature that converts certain PR events to Check Suite requests. (Namely, PR events with an action that indicates code was affected and may be in need of checking.) Currently, this is enabled by default.

To disable this feature, set the environment variable CHECK_SUITE_ON_PR=false on the deployment for the server. This can also be done by setting github.checkSuiteOnPR to false in the chart's values.yaml.

To forward a pull request (pull_request) to a check suite run, you will need to provide the ID for your GitHub Brigade App instance. (Here also set at the chart-level via values.yaml):

github:
...
  appID: APP_ID

This value is provided after the GitHub App is created on GitHub (see 1. Create a GitHub App). To find this value after creation, visit https://github.com/settings/apps/your-app-name.

Using the application ID and the private key configured when deploying the Helm chart, this gateway creates a new GitHub token for each request, meaning that we don't have to create a per-repository token.

When these parameters are set, incoming pull requests will also trigger check_suite:created events.

Handling Events in brigade.js

This gateway behaves differently than the gateway that ships with Brigade. Because this is a GitHub App, an authentication token is generated for each request on-the-fly. And each token is only good for 60 minutes.

The token is generated for you on the gateway, and sent in the payload, which looks like this:

{
  "token": "some.really.long.string",
  "body": {
    "action": "requested",
    "check_suite": {},
    "...": "..."
  }
}

The above shows just the very top level of the object. The object you will really receive will be much more detailed.

Events Emitted by this Gateway

Select events received by this gateway from Github are, in turn, emitted into Brigade. In some cases, events received from Github contain an action field. For all such events, two events will be emitted into Brigade. One will be a coarse-grained event, unaqualified by action. The second will be more finely-grained and qualified by action. The latter permits Brigade users to to more easily subscribe to a relevant subset of events that are of interest to them. For instance, if a user is interested in subscribing only to events that indicate a new pull request was opened, they may subscribe to pull_request:opened instead of subscribing to the more broad pull_request event, which would have burdened the user with writing logic to select on the basis of action themselves.

The events emitted by this gateway into Brigade are:

• check_run: A check run event with any action. A second event qualified by action will also be emitted.
• check_run:completed: The status of a check run was updated to completed.
• check_run:created: A new check run was created.
• check_run:requested_action: Someone requested that an action be taken.
• check_run:rerequested: Someone requested to re-run your check run.
• check_suite:completed: The status of a check suite was updated to completed.
• check_suite:requested: A new check suite was created.
• check_suite:rerequested: Someone requested to re-run your check suite.
• commit_comment: A commit comment event with any action. A second event qualified by action will also be emitted.
• commit_comment:created: A commit comment was created.
• create: A branch or tag was created.
• deployment: A deployment was created.
• deployment_status: A deployment's sdtatus has changed.
• issue_comment: An issue comment event with any action. A second event qualified by action will also be emitted.
• issue_comment:created: An issue comment was created.
• issue_comment:edited: An issue comment was edited.
• issue_comment:deleted: An issue comment was deleted.
• pull_request: A pull request event with any action. A second event qualified by action will also be emitted.
• pull_request:assigned: A pull request was assigned.
• pull_request:closed: A pull request was closed.
• pull_request:edited: A pull request was edited (e.g. title or body is edited).
• pull_request:labeled: A new label was assigned to a pull request.
• pull_request:locked: A pull request was locked.
• pull_request:opened: A new pull request was opened.
• pull_request:ready_for_review: A pull request is ready for review.
• pull_request:reopened: A closed pulled request was re-opened.
• pull_request:review_request_removed: An existing request for pull request review was removed.
• pull_request:review_requested: A pull request review was re-requested.
• pull_request:unassigned: A pull request was unassigned.
• pull_request:unlabeled: A label was removed from a pull request.
• pull_request:unlocked: A pull request was unlocked.
• pull_request_review: A pull request review with any action. A second event qualified by action will also be emitted.
• pull_request_review:submitted: A pull request review was submitted.
• pull_request_review:edited: A pull request review was edited.
• pull_request_review:dismissed: A pull request review was dismissed.
• pull_request_review_comment: A pull request review comment with any action. A second event qualified by action will also be emitted.
• pull_request_review_comment:created: A new pull request review comment was created.
• pull_request_review_comment:deleted: An existing pull request review comment was deleted.
• pull_request_review_comment:edited: An existing pull request review comment was edited.
• push: A commit was pushed to a branch or a new tag was applied.
• release: A release event with any action. A second event qualified by action will also be emitted.
• release:created: A new release was created.
• release:deleted: An existing release was deleted.
• release:edited: An existing release was edited.
• release:prereleased: A release is pre-released.
• release:published: A release is published.
• release:unpublished: A release is unpublished.
• status: The status of a git commit was changed.

Each of these events is described in greater detail in Github's own API documentation.

A special note on an issue_comment event: Since GitHub considers Pull Requests as Issues with code, this event will also be produced for general comments on Pull Requests -- meaning, outside of a dedicated Pull Request review or a comment on a commit directly. (The latter events would be pull_request_review_comment and commit_comment, respectively.)

The check_suite events will let you start all of your tests at once, while the check_run events will let you work with individual tests. The example in the next section shows how to work with suites, while still supporting re-runs of the main test.

Running a new set of checks

Currently this gateway forwards all events on to the Brigade.js script, and does not create new check_run requests. The brigade.js must create a run, and then update GitHub as to the status of that run.

Here's an example that starts a new run, does a test, and then marks that run complete. On error, it marks the run failed.

const {events, Job, Group} = require("brigadier");
const checkRunImage = "brigadecore/brigade-github-check-run:latest"

events.on("check_suite:requested", checkRequested)
events.on("check_suite:rerequested", checkRequested)
events.on("check_run:rerequested", checkRequested)

function checkRequested(e, p) {
  console.log("check requested")
  // Common configuration
  const env = {
    CHECK_PAYLOAD: e.payload,
    CHECK_NAME: "MyService",
    CHECK_TITLE: "Echo Test",
  }

  // This will represent our build job. For us, it's just an empty thinger.
  const build = new Job("build", "alpine:3.7", ["sleep 60", "echo hello"])

  // For convenience, we'll create three jobs: one for each GitHub Check
  // stage.
  const start = new Job("start-run", checkRunImage)
  start.imageForcePull = true
  start.env = env
  start.env.CHECK_SUMMARY = "Beginning test run"

  const end = new Job("end-run", checkRunImage)
  end.imageForcePull = true
  end.env = env

  // Now we run the jobs in order:
  // - Notify GitHub of start
  // - Run the test
  // - Notify GitHub of completion
  //
  // On error, we catch the error and notify GitHub of a failure.
  start.run().then(() => {
    return build.run()
  }).then( (result) => {
    end.env.CHECK_CONCLUSION = "success"
    end.env.CHECK_SUMMARY = "Build completed"
    end.env.CHECK_TEXT = result.toString()
    return end.run()
  }).catch( (err) => {
    // In this case, we mark the ending failed.
    end.env.CHECK_CONCLUSION = "failure"
    end.env.CHECK_SUMMARY = "Build failed"
    end.env.CHECK_TEXT = `Error: ${ err }`
    return end.run()
  })
}

Further Examples

See docs/examples for further brigade.js examples exercising different event handling scenarios, including Issue/PR comment handling, (re-)running individual Checks and more.

Parameters available on the check-run container

The following parameters can be specified via environment variables:

• CHECK_PAYLOAD (REQUIRED): The contents of e.payload. Will be used to parse repo name, commit and branch (if not provided by corresponding env vars below), as well as auth token details
• CHECK_NAME (default: Brigade): The name of the check. You should set this unless you are only running a single check.
• CHECK_TITLE (default: "running check"): The title that will be displayed on GitHub
• CHECK_SUMMARY: A short summary of what the check did.
• CHECK_TEXT: A long message explaining the results.
• CHECK_CONCLUSION: One of: "succeeded", "failure", "neutral", "canceled", or "timed_out". The "action_required" conclusion can be set if CHECK_DETAILS_URL is also set.
• CHECK_STARTED_AT: The time that the check run started (timestamp in ISO 8601 format). This is used to calculate the running time of the check run.
• CHECK_DETAILS_URL: The URL of an external site that has more information. This is typically used with CHECK_CONCLUSION=action_required.
• CHECK_EXTERNAL_ID: An ID that correlates this run to another source. For example, it could be set to the Brigade build ID.
• CHECK_ACTIONS: Custom definition of further check run actions displayed as buttons. See the GitHub documentation on actions
• GITHUB_BASE_URL: The URL for GitHub Enterprise users.
• GITHUB_UPLOAD_URL: The upload URL for GitHub Enterprise users.

Annotations and Image attachments are not currently supported.

You can observe these in action on this screenshot:

Contributing

This Brigade project accepts contributions via GitHub pull requests. This document outlines the process to help get your contribution accepted.

Prerequisites

• Docker
• make

Containerized Development Environment

To ensure a consistent development environment for all contributors, this project relies heavily on Docker containers as sandboxes for all development activities including dependency resolution, executing tests, or running a development server.

make targets seamlessly handle the container orchestration.

If, for whatever reason, you must opt-out of executing development tasks within containers, set the SKIP_DOCKER environment variable to true, but be aware that by doing so, the success or failure of development-related tasks, tests, etc. will be dependent on the state of your system, with no guarantee of the same results in CI.

Developing on Windows

All development-related tasks should "just work" on Linux and Mac OS systems. When developing on Windows, the maintainers strongly recommend utilizing the Windows Subsystem for Linux.

This blog post provides excellent guidance on making the Windows Subsystem for Linux work seamlessly with Docker Desktop (Docker for Windows).

Working with Code

$ make lint          # to run linters
$ make test          # to run tests
$ make build         # to run multi-stage Docker build of binaries and images

Pushing Images

By default, built images are named using the following scheme: <component>:<version>. If you wish to push customized or experimental images you have built from source to a particular org on a particular Docker registry, this can be controlled with environment variables.

The following, for instance, will build images that can be pushed to the krancour org on Dockerhub (the registry that is implied when none is specified).

$ DOCKER_ORG=krancour make build

To build for the krancour org on a different registry, such as quay.io:

$ DOCKER_REGISTRY=quay.io DOCKER_ORG=krancour make build

Images built with names that specify registries and orgs for which you have write access can be pushed using make push. Note that the build target is a dependency for the push target, so the build and push processes can be accomplished together like so:

Note also that you must be logged into the registry in question before attempting this.

$ DOCKER_REGISTRY=quay.io DOCKER_ORG=krancour make push

Signed commits

A DCO sign-off is required for contributions to repos in the brigadecore org. See the documentation in Brigade's Contributing guide for how this is done.