For the past month I've been building a repository of manifests and configurations for standing up a kubernetes cluster using kOps. I wanted to put together basic structures for the functioning cluster (in AWS) that should suffice for most cases where you need to go beyond quick start and have vital pieces for running a sample application that does more than returning "Hello World", and includes real-world cases like databases, autoscaling and monitoring/alerting. This allowed me to go through a good chunk of the k8s functionality without spending too much time on installation as that's taken care of by kOps.
My criteria for the solution was:
- use existing VPC (versus letting kOps create it)
- install an IngressController that utilizes ELB
- leverage AWS IAM (authn, IRSA)
- create temporary static credentials
- deploy a sample tiered application (web, db, jobs)
- deploy a monitoring solution (Prometheus, Grafana)
- use spot instances
- use network policies for security/isolation
- use autoscaling
- create a Helm chart for the application
Before we begin, ensure you have following items:
- AWS account with admin access. We are going to go over the free tier with compute and storage, be sure you know the costs
- domain name that you can modify (e.g. add an NS record). We are going to create a sub-domain under it, in Route 53, and use that sub-domain to name our k8s cluster (and issue an ACM certificate for it)
- Unix shell (e.g. bash), with necessary binaries installed (
terraform,kops,kubectl,aws,helm) - an IDE (e.g. VSCodium) for viewing/editing files
Rather than putting everyting into a giant post, I'm going to divide it into separate one for focused content.