Manage Kubernetes Clusters

Learn how to connect and manage Kubernetes clusters in Humanitec.

Introduction

Humanitec is designed to integrate with your existing Kubernetes clusters in the cloud provider of your choice. You can configure Humanitec to run your application in a single Kubernetes cluster or across different multi-cloud Kubernetes clusters while having an all-in-one solution for managing what is running where.

To connect your Kubernetes clusters to Humanitec you will have to use Static Resource Definitions. We currently support the major public cloud providers out-of-the-box.

If you are interested in integrating Humanitec with your on-prem/self-hosted Kubernetes cluster then please contact us and we are happy to support you with this.

Ingress Controllers

There are multiple ingress controllers available for Kubernetes. Some of them are officially supported by the Kubernetes project and some are not. In general, Humanitec can work with any ingress controller available. As of today, we are only supporting nginx ingress out of the box. If you are using a different ingress controller in your current project and don't want to switch controllers then please get in touch with us and we are happy to support you.

Google Cloud - GKE Cluster

You can easily connect Humanitec to Google Kubernetes Engine (GKE). The next paragraphs explain how.

Prerequisites

  • You must provide access to a service account with the Kubernetes Engine Admin role (roles/container.admin), or a role with the equivalent set of permissions.

    • You must create a key for this service account in JSON form. See Google's account keys documentation for more information. The access key must use the GCP Console/gcloud format (i.e., the first example in the linked documentation).

  • You must have set up a cluster in Google Kubernetes Engine (GKE).

    • Note: Humanitec has no resource requirements for the cluster where it will deploy your app. However, you must choose a machine type for the node pool that suits the needs of your app. See Google's machine types documentation for recommendations.

  • You must have the following APIs enabled for your project:

Prepare the Cluster

To prepare your cluster to support Humanitec app deployments, you need to install an NGINX Ingress Controller. Please follow the steps provided at kubernetes.github.io/ingress-nginx/deploy.

Connect as a Static Resource

Now that your cluster is ready, you need to connect your cluster as a Static Resource to Humanitec. You can do so in the Organization Settings.

  1. Click Static Resources. Here you find an overview of Static Resources.

  2. Click Kubernetes Cluster, which opens a dialog to define your cluster as a static resource.

  3. Select gke as a Driver and define an ID for your cluster resource.

  4. Next Resource data, Driver data, and Secrets need to be defined for your resource.

    • In Resource data you define the external IP of your Nginx Loadbalanacer, running in your cluster. You can find it by executing kubectl get services -n ingress-nginx while connected to your cluster.

    • In addition, you'll need to define your cluster name. Which you see in your Google Cloud Console​

    • In Driver Data you fill in your GCP Project ID as well as your GCP Zone. Both can be found through the Google Cloud Console.

    • Finally, you fill in the Secrets with the details of your service account as a JSON object.

Here is an example:

{
"auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"client_email": "[email protected]",
"client_id": "123456789123456789",
"client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/account%40gcp-project-id.iam.gserviceaccount.com",
"private_key": "-----BEGIN PRIVATE KEY-----\nYOUR-PRIVATE-KEY\n-----END PRIVATE KEY-----\n",
"private_key_id": "abcdefgh1234567890abcdefgh",
"project_id": "gcp-project-id",
"token_uri": "https://oauth2.googleapis.com/token",
"type": "service_account"
}
Add GKE cluster

Now you have a GKE cluster as a static resource registered in Humanitec.

Assign an Environment Type

Next, you need to define an environment type for your GKE cluster resource. This enables you to define a specific application environment to be deployed to this cluster.

Add environment type

Next to your GKE cluster in the list of static resources, you add the environment type by clicking on the plus icon. Either you define a new environment type or choose that all environments of the type development should use this resource.

Deploy to the Cluster

Navigate to your application and deploy the environment of the type that you associated with your GKE cluster. Once you deployed this environment your application will be running on your GKE cluster.

AWS - EKS Cluster

You can easily connect Humanitec to Amazon Elastic Kubernetes Service (Amazon EKS). The next paragraphs explain how.

Prerequisites

  • Be able to create IAM policies and attach them to an IAM user

  • Have an EKS cluster with a NodePool configured and kubectl access

Configuring an IAM User

You must set up a Policy and attach it to the IAM identity that Humanitec will use to interact with your cluster. Check out Policies and permissions in IAM in the AWS documentation for more information. The following permissions (actions) are needed:

  • eks:DescribeNodegroup

  • eks:ListNodegroups

  • eks:AccessKubernetesApi

  • eks:DescribeCluster

  • eks:ListClusters

A policy containing these permissions will look like this:

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": [
"eks:DescribeNodegroup",
"eks:ListNodegroups",
"eks:AccessKubernetesApi",
"eks:DescribeCluster",
"eks:ListClusters"
],
"Resource": "*"
}
]
}

See this AWS tutorial for step-by-step instructions: IAM Tutorial: Create and attach your first customer managed policy​

You must provide Humanitec with access to an IAM user.

  • This IAM user must have the above Policy attached.

  • You must provide an access key for the IAM user. Humanitec needs both the access key ID and the secret access key.

Mapping the IAM user into the Cluster

The IAM user must be mapped into your cluster. (This will be done automatically for the user who created the cluster.) The mapping is done by adding an entry into the aws-auth ConfigMap in the cluster. This AWS documentation provides more detailed information how to manage users and IAM roles for your cluster.

Editing the aws-auth ConfigMap can be done via kubectl

kubectl edit configmap aws-auth -n kube-system

The following configuration needs to be added for the IAM user:

mapUsers: |
- userarn: arn:aws:iam::XXXXXXXXXXXX:user/<username>
username: <username>
groups:
- system:masters

If the IAM User is assuming a role that has the policy described above, then the role is mapped as follows:

mapRoles: |
- rolearn: arn:aws:iam::XXXXXXXXXXXX:role/<rolename>
username: <rolename>
groups:
- system:masters

Prepare the Cluster

To prepare your cluster to support Humanitec app deployments, you need to install an NGINX Ingress Controller. Please follow the steps provided at kubernetes.github.io/ingress-nginx/deploy/#aws.

Connect as a Static Resource

Now that your cluster is ready, you need to connect your cluster as a static resource to Humanitec. You can do so in the Organization Settings.

  1. Click Static Resources. Here you find an overview of Static Resources.

  2. Click Kubernetes Cluster, which opens a dialog to define your cluster as a static resource.

  3. Select eks as a Driver and define an ID for your cluster resource.

  4. Next Resource data, Driver data, and Secrets need to be defined for your resource.

    • In Resource data you define the DNS name of the Amazon Load Balancer, running in your cluster. You can find it by executing kubectl get services -n ingress-nginx while connected to your cluster.

    • In addition, you'll need to define your cluster name. Which you see in your AWS Management Console.

    • In Driver Data you fill in the hosted zone of your load balancer as well as your AWS region. Both can be found through the AWS Management Console. In particular, load balancer data can be found under the section EC2 -> Load Balancing.

    • Finally, you fill in the Secrets with the details of your AWS account as a JSON object.

Here is an example:

{
"aws_access_key_id": "AAABBBCCCDDDEEEFFFGGG",
"aws_secret_access_key": "zZxXyY123456789aAbBcCdD"
}
Add EKS cluster

Now you have an EKS cluster as a static resource registered in Humanitec.

Assign an Environment Type

Next, you need to define an environment type for your EKS cluster resource. This enables you to define a specific application environment to be deployed to this cluster.

Add environment type

Next to your EKS cluster in the list of static resources, you add the environment type by clicking on the plus icon. Either you define a new environment type or choose that all environments of the type development should use this resource.

Deploy to the Cluster

Navigate to your application and deploy the environment of the type that you associated with your EKS cluster. Once you deployed this environment your application will be running on your EKS cluster.

Azure - AKS Cluster

You can easily connect Humanitec to Microsoft Azure Kubernetes Service (AKS). The next paragraphs explain how.

Prerequisites

  • You must have created a cluster in Microsoft Azure Kubernetes Engine (AKS)

    • Note: Humanitec has no resource requirements for the cluster where it will deploy your app. However, you must choose a machine type for the node pool that suits the needs of your app. See Microsoft Azure's sizes for cloud services documentation for more information.

  • You must provide access to the AKS cluster via service principal. Typically, a service principal is created automatically when you create a new AKS cluster.

    • You need to provide the service principal credentials according to the output from the az ad sp command (incl. appId, name, password, and tenant).

    • Please refer to the documentation for az ad sp credential for more information on how to list, create, and reset service principal credentials.

Prepare the Cluster

To prepare your cluster to support Humanitec app deployments, you need to install an NGINX Ingress Controller. Please follow the steps provided at kubernetes.github.io/ingress-nginx/deploy.

Connect as a Static Resource

Now that your cluster is ready, you need to connect your cluster as a static resource to Humanitec. You can do so in the Organization Settings.

  1. Click Static Resources. Here you find an overview of Static Resources

  2. Click Kubernetes Cluster, which opens a dialog to define your cluster as a static resource.

  3. Select aks as a Driver and define an ID for your cluster resource.

  4. Next Resource data, Driver data, and Secrets need to be defined for your resource.

    • In Resource data you define the external IP of your Nginx Loadbalanacer, running in your cluster. You can find it by executing kubectl get services -n ingress-nginx while connected to your cluster.

    • In addition, you'll need to define your cluster name. Which you see in your Azure Portal​

    • In Driver Data you fill in your Azure Resources Group as well as your Azure Subscription ID. Both can be obtained via the Azure CLI in the output of az aks list.

    • Finally, you fill in the Secrets with the details of your service principal as a JSON object.

Here is an example:

{
"appId": "559513bd-0c19-4c1a-87cd-851a26afd5fc",
"displayName": "myAKSClusterServicePrincipal",
"name": "http://myAKSClusterServicePrincipal",
"password": "e763725a-5eee-40e8-a466-dc88d980f415",
"tenant": "72f988bf-86f1-41af-91ab-2d7cd011db48"
}
Add AKS cluster

Now you have an AKS cluster as a static resource registered in Humanitec.

Assign an Environment Type

Next, you need to define an environment type for your AKS cluster resource. This enables you to define a specific application environment to be deployed to this cluster.

Add environment type

Next to your AKS cluster in the list of static resources, you add the environment type by clicking on the plus icon. Either you define a new environment type or choose that all environments of the type development should use this resource.

Deploy to the Cluster

Navigate to your application and deploy the environment of the type that you associated with your AKS cluster. Once you deployed this environment your application will be running on your AKS cluster.

Red Hat OpenShift

There is an experimental Driver to connect Humanitec to your OpenShift cluster. Please contact us to get access to the experimental Driver.