Event-driven scripting on your K8s infrastructure with Brigade
What is Brigade?
Brigade is a tool from the team that brought you Helm and Draft which are designed to assist developers and operations teams to accomplish more work with event-driven scripting. The tool enables teams to build out any ordered workflow of containers in Kubernetes and trigger the workflow by listening for arbitrary events.
Using Kubernetes allows users to not only manage long-standing services, but also run quick short-term tasks that are commonly used for things like event-driven programming, code quality testing, and batch processing.
By leveraging Brigade, users are able to string multiple tasks together and execute them inside containers. It follows the same patterns found in other forms of serverless computing, where a Brigade pipeline is triggered by an event.
At first glance, Brigade:
- is Kubernetes-native, with one-line installation.
- supports projects and a customizable event model.
- enables arbitrarily complex (parallel or synchronous) pipelines.
- executes functions in containers.
- fully supports Kubernetes volumes, pods, role-based access control (RBAC), and secrets.
What can I build with Brigade?
- Batch processors
- Unit tests
- Slack notifications (Chat Ops)
- Cron Jobs
How it works
Brigade is an in-cluster runtime environment. It interprets scripts and executes them often by invoking resources inside of the cluster. Brigade is event-based scripting of Kubernetes pipelines.
- Event-based: A script execution is triggered by a Brigade event.
- Pipelines: A script expresses a series of related jobs that run-in parallel or serially.
Setting up a CI pipeline with Brigade
Throughout this tutorial, we’ll walk you through the creation of a basic web application with a Brigade CI pipeline for testing the application.
It’ll consist of two parts:
- A public site that lets people generate UUIDs.
- A brigade.js that tests the site
We’ll assume you have Brigade, Git (a version control system), and pip (a package management system for Python) installed already.
You can tell if Brigade is installed and its version by running the following command in a shell prompt (indicated by the $ prefix):
$ helm status brigade-server
Creating your first application
For this tutorial, we’ll be creating an example application written in Python which uses Flask to provide a very simple UUID generator web server. Flask is a microframework for Python based on Werkzeug, Jinja 2 and good intentions. (Note that this application is written for Python 3, specifically.)
Let’s write the web server. To create your app, type this command:
$ mkdir -p uuid-generator/app
Open the file
uuid-generator/app/__init__.py and put the following Python code into it:
from flask import Flask, Response app = Flask(__name__) @app.route("/") def hello(): return "Hello World!"
This is the simplest application possible in Flask. To call
hello(), we need to map it to a URL - in this case we want to map it to the root path (
Start the development server
Now that we’ve written the app, let’s run it!
$ echo "Flask" > requirements.txt $ pip install -r requirements.txt $ FLASK_APP=uuid-generator/app/__init__.py flask run * Serving Flask app "app" * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit)
Now, open a Web browser and go to “/” on your local domain – e.g., http://127.0.0.1:5000/. You should see “Hello World!”:
If you look back to the running flask app’s logs, you should see new logs pop up:
127.0.0.1 - - [18/Aug/2017 12:56:16] "GET / HTTP/1.1" 200 -
Let’s make the application generate a random UUID on every request. Edit the
uuid-generator/app/__init__.py file so that it looks like this:
from flask import Flask, Response import uuid app = Flask(__name__) @app.route("/") def hello(): return Response(str(uuid.uuid4()), status=200, mimetype='text/plain')
Re-run the web server and open the browser back to http://127.0.0.1:5000/. You should now see a random UUID:
Keep refreshing the page. You should see new UUIDs being generated every time you refresh the page.
Create a Github account
To use Github, you will need a Github account.
In your own browser:
- Open a new browser tab
- Navigate to https://github.com/
- Create an account
If you already have Github account, continue to the next exercise.
After you sign up, you will receive a verification e-mail. Be sure to verify your e-mail address to Github by following the instructions in that e-mail.
Initialize your repository
Now that you have a Github account and a working application, let’s create a git repository and push it up to Github!
Let’s first change working directories over to the Flask app we made in Tutorial 1.
$ cd uuid-generator
Now, we’ll create a git repository and push the project to Github.
$ git init $ echo -e "*.pyc\n*.egg-info" > .gitignore $ git commit -am "initial commit" $ git push origin master
We created a .gitignore file so all compiled python files and python eggs (noted with the .pyc and .egg-info extensions, respectively) are not added to source control.
We’ll now walk through the process for configuring your newly created Github repository with Brigade for testing new features. We’ll configure a new Brigade project, and have Github push events to trigger Brigade builds.
We want to build our project each time a new commit is pushed to master, and each time we get a new Pull Request.
To do this, follow the Brigade GitHub App documentation to set up a GitHub App. During configuration, copy the shared secret above (
mDXUDZyDsTUHw4KZIMPOQMN1) and set this as the “Webhook secret” value for the App.
Create a Brigade project
The Brigade server tracks separate configuration for each project you set up. To create and manage these configurations, we use the brig cli.
Here we create a project for our GitHub repo:
$ brig project create ? VCS or no-VCS project? VCS ? Project Name bacongobbler/uuid-generator ? Full repository name github.com/bacongobbler/uuid-generator ? Clone URL (https://github.com/your/repo.git) https://github.com/bacongobbler/uuid-generator.git ? Add secrets? No Auto-generated a Shared Secret: "mDXUDZyDsTUHw4KZIMPOQMN1" ? Configure GitHub Access? No ? Configure advanced options No Project ID: brigade-5ea0b3d7707afb5d04d55544485da6aff4f58006c1633f4ae0cb11
Note: To explore the advanced options, each prompt offers further info when
? is entered.
We’ll need to upgrade our Brigade server with our
brigade-github-app sub-chart configuration filled in:
$ helm inspect values brigade/brigade > brigade-values.yaml $ # Add configuration under the `brigade-github-app` section $ helm upgrade brigade brigade/brigade -f brigade-values.yaml
We can then get the IP needed to update the “Webhook URL” entry for our App. Run this command on your Kubernetes cluster, and look for the
$ kubectl get service NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE brigade-server-brigade-api ClusterIP 10.0.34.228 <none> 7745/TCP 1d brigade-server-brigade-github-app LoadBalancer 10.0.69.248 10.21.77.9 80:31980/TCP 1d
You will use the
EXTERNAL-IP address to form the full URL:
http://10.21.77.9/events/github. Update the App’s “Webhook URL” with this value. (Note: it is preferred that DNS be set up instead of a hard-coded IP. See the Brigade GitHub App docs for more.)
The next time you push to the repository, the webhook system should trigger a build.
Using Brigade can add a really useful tool for those trying to implement Kubernetes-native automation into their pipelines and processes. I hope this tutorial helped you learn something new in terms of automating your infrastructure, and I hope you have a chance to read more of my posts in the future. Thanks for reading!