kubernetes - Change Public GKE to Private GKE cluster using terraform

Question

How to change the existing GKE cluster to GKE private cluster? Will I be able to connect to the Kubectl API from internet based on firewall rules or should I have a bastion host? I don't want to implement Cloud Nat or nat gateway. I have a squid proxy VM that can handle internet access for pods. I just need to be able to connect to Kubectl to apply or modify anything.

I'm unsure how to modify the existing module I wrote to make the nodes private and I'm not sure if the cluster will get deleted if I try and apply the new changes related to private gke cluster.

resource "google_container_cluster" "primary" {
  name     = "prod"
  network  = "prod"
  subnetwork = "private-subnet-a"
  location               = "us-west1-a"
  remove_default_node_pool = true
  initial_node_count = 1

  depends_on = [var.depends_on_vpc]
}

resource "google_container_node_pool" "primary_nodes" {
  depends_on = [var.depends_on_vpc]

  name       = "prod-node-pool"
  location   = "us-west1-a"
  cluster    = google_container_cluster.primary.name
  node_count = 2

  node_config {
    preemptible  = false
    machine_type = "n1-standard-2"

    metadata = {
      disable-legacy-endpoints = "true"
    }

    oauth_scopes = [
      "https://www.googleapis.com/auth/logging.write",
      "https://www.googleapis.com/auth/monitoring",
      "https://www.googleapis.com/auth/devstorage.read_only",
      "https://www.googleapis.com/auth/compute",
    ]
  }
}

question from:https://stackoverflow.com/questions/65916344/change-public-gke-to-private-gke-cluster-using-terraform

Answer 1

Answering the part of the question:

How to change the existing GKE cluster to GKE private cluster?

GKE setting: Private cluster is immutable. This setting can only be set during the GKE cluster provisioning.

To create your cluster as a private one you can either:

• Create a new GKE private cluster.
• Duplicate existing cluster and set it to private:
  • This setting is available in GCP Cloud Console -> Kubernetes Engine -> CLUSTER-NAME -> Duplicate
  • This setting will clone the configuration of your infrastructure of your previous cluster but not the workload (Pods, Deployments, etc.)

Will I be able to connect to the Kubectl API from internet based on firewall rules or should I have a bastion host?

Yes, you could but it will heavily depend on the configuration that you've chosen during the GKE cluster creation process.

As for ability to connect to your GKE private cluster, there is a dedicated documentation about it:

• Cloud.google.com: Kubernetes Engine: Docs: How to: Private clusters

---

As for how you can create a private cluster with Terraform, there is the dedicated site with configuration options specific to GKE. There are also parameters responsible for provisioning a private cluster:

• Registry.terraform.io: Providers: Hashicorp: Google: Latest: Docs: Resources: Container cluster

As for a basic example of creating a private GKE cluster with Terraform:

• main.tf

provider "google" {
  project = "INSERT_PROJECT_HERE" 
  region  = "europe-west3"
  zone    = "europe-west3-c"
}

• gke.tf

resource "google_container_cluster" "primary-cluster" {
  name               = "gke-private"
  location           = "europe-west3-c"
  initial_node_count = 1

  private_cluster_config {
    enable_private_nodes = "true"
    enable_private_endpoint = "false" # this option will make your cluster available through public endpoint 
    master_ipv4_cidr_block = "172.16.0.0/28"
  }

  ip_allocation_policy {
    cluster_secondary_range_name = "" 
    services_secondary_range_name = ""
  }

  
  node_config {
    machine_type = "e2-medium"
  }
}

A side note!

I've created a public GKE cluster, modified the .tf responsible for it's creation to support private cluster. After running: $ terraform plan Terraform responded with the information that the cluster will be recreated.