Kubernetes Architecture

flex.plane provides managed Kubernetes clusters on your own infrastructure. Instead of relying on upstream Cluster API providers that assume cloud-like APIs, flex.plane uses custom CAPI providers purpose-built for Proxmox and a hosted control plane model that keeps tenant worker nodes clean and focused on running workloads.

Understand the architecture

flex.plane's Kubernetes implementation consists of three main components:

1. Management cluster: the Kubernetes cluster that flex.plane itself runs on. It hosts the Cluster API controllers and the hosted control planes for all tenant clusters.
2. Custom CAPI providers: two purpose-built providers that handle infrastructure and control plane lifecycle on Proxmox.
3. Tenant clusters: the Kubernetes clusters that your workloads run on. Each tenant cluster has its control plane hosted on the management cluster and its worker nodes running as VMs on Proxmox.

The custom providers are:

• capi-infrastructure-provider: manages the Proxmox VMs that become Kubernetes nodes. It creates, configures, and deletes VMs based on the cluster spec, handling everything from disk provisioning to network configuration.
• capi-control-plane-provider: manages the control plane components (API server, etcd, controller-manager, scheduler) as pods on the management cluster instead of deploying them onto tenant VMs.

Explore hosted control planes

The most significant architectural decision is hosted control planes. In a traditional Kubernetes deployment, the control plane components run on dedicated nodes (or at least on some of the cluster's own nodes). In flex.plane, control planes run on the management cluster.

This means:

• Tenant worker nodes only run workloads. No etcd, no API server, no controller-manager consuming resources on your tenant VMs. Every CPU core and byte of memory on a worker node goes to your applications.
• Control plane availability is managed centrally. The management cluster handles high availability, monitoring, and upgrades for all tenant control planes. You do not need to provision or maintain separate control plane nodes.
• Faster cluster creation. Provisioning a new cluster does not require booting and configuring control plane VMs. The control plane components are deployed as pods, which start in seconds.
• Lower resource overhead. A traditional 3-node control plane requires 3 VMs worth of resources. Hosted control planes need a fraction of that, because they share the management cluster's resources.

The trade-off is that the management cluster becomes a critical dependency for tenant cluster API access. If the management cluster is down, you cannot reach the Kubernetes API of your tenant clusters. However, running workloads are unaffected. Pods on worker nodes continue to run even if the API server is temporarily unavailable.

Connect workers via Konnectivity

Since the control plane runs on the management cluster and worker nodes run as separate VMs (potentially on different networks), there needs to be a reliable communication channel between them. flex.plane uses Konnectivity for this.

Konnectivity establishes a tunnel between the hosted API server and the worker nodes:

1. A Konnectivity server runs alongside each tenant's API server on the management cluster.
2. A Konnectivity agent runs on each worker node and connects back to the server.
3. All API server-to-node communication (kubectl exec, logs, port-forward, metrics) is routed through this tunnel.

This solves the network topology challenge: the API server does not need direct network access to worker nodes, and worker nodes do not need to be on the same network as the management cluster. The Konnectivity agent initiates the connection outbound from the worker node, which works through firewalls and NAT.

Compare with traditional CAPI deployments

The hosted control plane model trades some operational simplicity (one management cluster to maintain) for significant resource and operational efficiency across all tenant clusters. For organizations running multiple Kubernetes clusters on shared infrastructure, this is a substantial win.