Introduction to Kubernetes Operations with Kops

Kubernetes has become the de-facto standard for container orchestration, with adoption rapidly growing year over year. According to CNCF, nearly 90% of organizations are using Kubernetes in production.

Content Navigation show

However, deploying and managing Kubernetes clusters brings operational overhead around provisioning infrastructure, installing software, configuring networking/storage as well as ongoing maintenance.

This is where a tool like Kops shines…

What is Kops?

Kops (short for Kubernetes Operations) is an open-source production grade tool for deploying and managing Kubernetes clusters in on-prem and cloud environments. It handles a lot of complexity around setting up highly available clusters and simplifies ongoing operations like upgrades, scaling, etc.

Some key capabilities:

  • Provisions cluster infrastructure on cloud providers (AWS, GCP etc.)
  • Installs latest Kubernetes version and sets up control plane
  • Manages node instance groups and auto-scaling
  • Simplifies cluster upgrades, scaling and node lifecycle management
  • Integrates with CI/CD pipelines and GitOps workflows
  • Supports high availability topologies and multi-AZ deployments
  • Easy teardown of clusters down to cloud infrastructure

Compared to hosted Kubernetes solutions like EKS, GKE and AKS, using a self-managed Kubernetes platform with Kops provides flexibility and minimizes vendor lock-in. And compared to manual deployment, Kops automates the heavy lifting so you avoid the toil.

Let‘s look at deploying Kubernetes on AWS with Kops…

Deploying Kubernetes on AWS

By default, Kops will deploy a single master cluster with one node which provides limited reliability. For production environments, we need to deploy a highly available topology across multiple AZs.

Here is an example production-grade architecture we will deploy using Kops on AWS:

Kops on AWS architecture

Some key components:

  • 3 Master nodes across 3 Availability Zones
  • 2 Worker node groups spread across 3 AZs
  • External facing network load balancer for the Kubernetes API server
  • Cluster level components like CoreDNS, Cluster Autoscaler etc.

Let‘s go through the step by step workflow for setting this up…

Provisioning Infrastructure with Terraform

First, we will use Terraform to provision the network infrastructure on AWS:

  • VPC with public and private subnets
  • Internet and NAT gateways for connectivity
  • Route tables and routes 
# Terraform for network infrastructure

resource "aws_vpc" "kops_vpc" {
  cidr_block = "172.20.0.0/16"

  tags = {
    Name = "kops-vpc"
  }
}

resource "aws_subnet" "private" {
  vpc_id     = aws_vpc.kops_vpc.id
  cidr_block = "172.20.1.0/24"

  tags = {
    "KubernetesCluster" = "kops.k8s.local"
    "Name" = "private-subnet"
  }
} 

resource "aws_internet_gateway" "gw" {
  vpc_id = aws_vpc.kops_vpc.id
}

# Route tables, routes, gateways etc.

We provide the VPC and subnet tags that will be used by Kops to associate cluster resources later.

Creating the Kubernetes Cluster

With the network foundation in place, we can now use Kops to deploy our Kubernetes cluster on top.

A basic 3 master and 2 worker node cluster configuration looks like:

# kops cluster configuration 

apiVersion: kops.k8s.io/v1alpha2
kind: Cluster
metadata:
  name: kops.k8s.local
spec:
  api:
    loadBalancer:
      type: Public  
  availabilityZones: 
  - us-east-1a  
  - us-east-1b 
  - us-east-1c
  etcdClusters: 
  - etcdMembers:
    - instanceGroup: master-us-east-1a
      name: a  
    - instanceGroup: master-us-east-1b
      name: b
    - instanceGroup: master-us-east-1c
      name: c
    name: main 
  networking:
    subnets:
    - region: us-east-1  
      name: private-subnet
  containerRuntime: containerd
  kubernetesVersion: 1.24.3
  masterPublicName: api.kops.k8s.local

# Instance groups, cluster addons etc.

We provide the AWS region/zones, reference the subnets created earlier and configure our desired Kubernetes version.

To create the cluster:

kops create -f kops-config.yaml --state=s3://my-state-store
kops update cluster kops.k8s.local --yes

Kops handles provisioning EC2 instances for master and worker nodes, attaching them to subnets, configuring security groups, installing Kubernetes binaries and add-ons etc.

Our HA Kubernetes cluster is now running on AWS!

Accessing and Managing the Cluster

To access the K8s API and work with the cluster: 

kubectl --kubeconfig=kubeconfig get nodes
  • Cluster upgrades use kops upgrade cluster
  • Node autoscaling based on demands automatically
  • Scaling up an instance group kops edit ig nodes-1 --name=kops.k8s.local
  • Deleting the cluster kops delete cluster --name kops.k8s.local --yes

And that‘s it! With Kops we were able to take advantage of AWS and deploy a production ready Kubernetes cluster with minimal effort.

Wrapping Up

We covered how Kops makes it easy to deploy and operate Kubernetes clusters on the infrastructure of your choice…

Some key takeaways:

  • Kops provisions the infrastructure and installs Kubernetes
  • Automates deployment of HA clusters across AZs
  • Manages upgrade, scaling and lifecycle of clusters
  • Integrates well with Terraform and GitOps workflows
  • Reduces vendor lock-in compared to managed Kubernetes
  • Handles multi-master multi-node cluster operations

Kops empowers you focus on developing applications rather than cloud infrastructure!

Hope you enjoyed this introduction to Kubernetes operations with Kops. Let me know if you have any other questions!

Tags: