Connect CI Pipelines

Learn how to connect CI pipelines to Humanitec.

Introduction

CI pipelines define the series of steps that must be performed in order to deliver a new version of software. Typical steps are "build", "test", "validate / compliance", "release", and maybe even "deploy". The "build" step produces an image that is then "pushed" into a registry.

Humanitec as your Internal Developer Platform needs to know about all available builds. This allows users of Humanitec to use these builds in their applications and it's the starting point for further automation.

In most setups, it's sufficient to notify Humanitec at the end of your CI pipeline about a newly available build after it has been pushed to your registry (see also Container Registries for more information).

Humanitec also offers a hosted registry in case you do not have your own registry. Using the hosted registry requires an additional step in your CI pipeline to build the image and to push it to the Humanitec registry. The following picture shows both approaches.

In most cases, the only change needed in existing CI pipelines to integrate with Humanitec is a simple curl or wget command to notify Humanitec about every new build available.

Why does Humanitec offer a hosted registry?

Almost all teams using Humanitec have their own registry either self-managed or managed by their cloud provider. Humanitec still offers a managed container registry that is compartmentalised for each organization in Humanitec. This registry is often used to test Humanitec since it is easy to just push an image with a shell script from a developer's local machine to make it available within Humanitec without already changing existing CI pipelines. The required shell script can be found further down.

Add new Image Sources

In the context of Humanitec, a CI pipeline is an image source. An image source provides Humanitec with container images which can then be deployed to the Environments of your Apps as Workloads.

Notify Humanitec

If you are already using your own container registry, the only thing you need to do is inform Humanitec about new builds. This can be done by adding a curl orwget request to the end of your existing CI pipeline:

UI
CLI
API
UI
  1. Start in the Images tab in Organization Settings.

  2. Click on Connect CI pipeline.

  3. Chose between using curl or wget for the post request.

  4. Copy and paste the post request at the end of your CI pipeline. Make sure you replace the placeholder variables with the appropriate values.

  5. As soon as new images are pushed they will be shown in the list below.

Step 1 & 2 of adding a new image source.
Step 3 & 4 of adding a new image source.
CLI

Our CLI is currently in closed beta. Please contact us if you want to learn more.

API

Image sources can be created via the Add a new Image Source endpoint.

Placeholder variables in the example above:

  • IMAGE_NAME is an ID identifying the image and all its versions. (Must be a valid Humanitec ID).

  • HUMANITEC_TOKEN contains a Humanitec Static Token.

  • CI_COMMIT_BRANCH is the branch that the image was built from.

  • CI_COMMIT_SHA is the git commit that the image was built from.

  • IMAGE_PATH is the full path to the image in your registry (incl. the image tag).

  • CI_IMAGE_TAG contains additional git tags associated with the commit. (optional).

Note that you can integrate Humanitec in any CI pipeline by adding the required steps yourself. You can find example bash scripts in Write your own integration.

GitHub Actions Workflow

Humanitec offers a out-of-the-box CI pipeline integration for Github Actions. The source code of the GitHub Action is available in this public GitHub repository: Build and Push to Humanitec.

If you do not have a GitHub Actions workflow set up, you will need to create one.

  1. Go to the GitHub repository you want to connect.

  2. Click the Actions tab.

  3. Click the Set up a workflow yourself button in the top right corner.

  4. Remove all of the lines after the line that says: - uses: actions/[email protected]. The resulting file should look like this:

# This is a basic workflow to help you get started with Actions.
name: CI
# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
# Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
- uses: actions/[email protected]

You need to add the HUMANITEC_TOKEN as a repository secret:

  1. Go to the repository you want to connect.

  2. Go to Settings, and then go to Secrets > New repository secret.

  3. Create a secret called HUMANITEC_TOKEN, and paste in the token.

Write your own Integration

You can integrate Humanitec with any CI pipeline or even use Humanitec from a terminal on your local machine in case you just want to give it a test without changing your existing CI pipelines. This section provides all required information and bash scripts to do that.

Full bash script for local usage

If you want to test Humanitec from your local machine (by pushing a local image to the registry provided by Humanitec), You can use the following shell script:

#!/bin/sh
print_usage ()
{
cat <<EOF
USAGE:
./local_build_push_notify.sh [options] IMAGE_NAME[:TAG]
OPTIONS:
-t, --token
The token to pass into the authorization header. Will override the
HUMANITEC_TOKEN environment variable if supplied.
-o, --org
The organization in Humanitec that the token belongs to.
-h, --help
Show this help text.
NOTES:
IMAGE_NAME is required and must consist only of lowercase letters and numbers
and -. It may not start or end with -.
If TAG is not provided, it will be set to the current commit SHA1.
By default, the token will be read from the HUMANITEC_TOKEN environment
variable.
EXAMPLE:
./local_build_push_notify.sh --org my-org my-image:0.3.2-rc5
EOF
}
key_from_json_obj ()
{
tr -d '\n' | sed 's/^[ \t\v\f]*{.*"'"${1}"'"[ \t\v\f]*:[ \t\v\f]*"\([^"]*\)"[ \t\v\f]*[,}].*$/\1/'
}
fetch_url ()
{
method="$1"
payload=""
if [ "$method" = "POST" ] || [ "$method" = "PUT" ] || [ "$method" = "PATCH" ]
then
payload="$2"
shift
fi
url="$2"
auth_header="Authorization: Bearer $HUMANITEC_TOKEN"
if command -v curl &> /dev/null
then
if [ "$payload" != "" ]
then
curl --fail -s \
-X "$method" \
-H "$auth_header" \
-H "Content-Type: application/json" \
-d "$payload" \
"$url"
else
curl --fail -s \
-X "$method" \
-H "$auth_header" \
"$url"
fi
elif command -v wget &> /dev/null
then
if [ "$payload" != "" ]
then
wget --quiet -O - \
--method="$method" \
--header="$auth_header" \
--header="Content-Type: application/json" \
--body-data="$payload" \
"$url"
else
wget --quiet -O - \
--method="$method" \
--header="$auth_header" \
"$url"
fi
else
echo "System does not have the commands wget or curl installed." >&2
exit 1
fi
}
api_prefix="https://api.humanitec.io"
while (( $# ))
do
case "$1" in
'-t'|'--token')
export HUMANITEC_TOKEN="$2"
shift
;;
'-o'|'--org')
export HUMANITEC_ORG="$2"
shift
;;
'--api-prefix')
api_prefix="$2"
shift
;;
'-h'|'--help')
print_usage
exit
;;
*)
if ! echo "$1" | grep -E '^[a-z0-9][a-z0-9-]+[a-z0-9]:[a-zA-Z0-9_.-]+$|^[a-z0-9][a-z0-9-]+[a-z0-9]$' > /dev/null
then
print_usage
exit 1
fi
image_with_tag="$1"
image_name=${image_with_tag%:*}
image_tag=${image_with_tag#*:}
if [ "$image_tag" = "$image_with_tag" ]
then
image_tag=""
fi
esac
shift
done
if [ -z "$HUMANITEC_TOKEN" ]
then
echo "No token specified as option or via HUMANITEC_TOKEN environment variable." >&2
exit 1
fi
if [ -z "$HUMANITEC_ORG" ]
then
echo "No Organization specified as option or via HUMANITEC_ORG environment variable." >&2
exit 1
fi
if [ -z "$image_with_tag" ]
then
echo "No IMAG:TAG provided." >&2
exit 1
fi
echo "Retrieving registry credentials"
registry_json="$(fetch_url GET "${api_prefix}/orgs/${HUMANITEC_ORG}/registries/humanitec/creds")"
if [ $? -ne 0 ]
then
echo "Unable to retrieve credentials for humanitec registry." >&2
exit 1
fi
username="$(echo "$registry_json" | key_from_json_obj "username")"
password="$(echo "$registry_json" | key_from_json_obj "password")"
server="$(echo "$registry_json" | key_from_json_obj "registry")"
branch="$(git rev-parse --abbrev-ref HEAD)"
commit="$(git rev-parse HEAD)"
if [ "$image_tag" = "" ]
then
image_tag="$commit"
fi
echo "Logging into docker registry"
echo "${password}" | docker login -u "${username}" --password-stdin "${server}"
if [ $? -ne 0 ]
then
echo "Unable to log into humanitec registry." >&2
exit 1
fi
echo "Performing docker build"
local_tag="${HUMANITEC_ORG}/${image_name}:${image_tag}"
if ! docker build -t "$local_tag" .
then
echo "Docker build failed." >&2
exit 1
fi
image_id="$(docker images -q "$local_tag")"
remote_tag="${server}/$local_tag"
if ! docker tag "$local_tag" "$remote_tag"
then
echo "Error pushing to remote registry: Cannot retag locally." >&2
exit 1
fi
echo "Pushing image to registry: $remote_tag"
if ! docker push "$remote_tag"
then
echo "Error pushing to remote registry: Push failed." >&2
exit 1
fi
echo "Notifying Humanitec"
payload="{\"commit\":\"${commit}\",\"branch\":\"${branch}\",\"tags\":"[]",\"image\":\"${remote_tag}\"}"
if ! fetch_url POST "$payload" "${api_prefix}/orgs/${HUMANITEC_ORG}/images/${image_name}/builds"
then
echo "Unable to notify Humanitec." >&2
exit 1
fi

The script will perform the following main actions:

  • Build a new image locally using docker.

  • Push the image to the Humanitec registry.

  • Notify Humanitec about the new image.

Obviously, you can tweak the individual steps if needed (e.g., in case you need to add additional BUILD_ARGS for your docker build). Feel free to reach out to us in case you need support.