Leveraging AI for a Homelab: Declarative Proxmox VE Infrastructure with Terraform, Pulumi, and GitLab CI/CD

Leveraging AI for a Homelab: Declarative Proxmox VE Infrastructure with Terraform, Pulumi, and GitLab CI/CD

Modern homelabs have evolved far beyond a single hypervisor running a handful of manually clicked-together virtual machines. When your homelab mirrors production enterprise architecture—spanning a multi-node Proxmox VE cluster, Kubernetes workloads, VLAN-segmented MikroTik and UniFi switching, HashiCorp Vault secret management, and full observability—manual GUI provisioning (“ClickOps”) quickly leads to configuration drift.

Here is how we architected a 100% declarative, self-healing homelab by combining Terraform, Pulumi, GitLab CI/CD, and leveraging AI for a homelab (ai-ops) to manage the entire lifecycle from bare-metal hypervisors to application services.

1. Why Declarative IaC + Leveraging AI for a Homelab Matters

Traditional homelab setups fail in predictable ways: undocumented VM hardware tweaks, snowflake storage mounts, forgotten firewall rules, and stale DNS records. By treating the homelab as an immutable Infrastructure-as-Code (IaC) fleet and leveraging AI for a homelab operations plane, we achieve four core guarantees:

  • Zero ClickOps Drift: Every Proxmox VM, LXC container, VLAN bridge, and DNS record is defined in Git and reconciled via Terraform or Pulumi.
  • Single Source of Truth (SSOT): NetBox tracks every rack unit, hypervisor node, VLAN prefix (172.28.x.0/24), and IP assignment before a VM is ever instantiated.
  • Vault-Backed Ephemeral Secrets: API tokens, SSH keys, and Cloud-Init credentials are fetched dynamically from HashiCorp Vault at pipeline execution time—never hardcoded in repositories.
  • Agentic AI Topology Awareness: Our ai-ops agents query a pre-compiled knowledge graph (graphify) in <150 tokens to understand blast radius, author IaC Merge Requests, and validate post-deploy telemetry automatically.

2. Proxmox VE Cluster Foundation & Cloud-Init Golden Templates

Before Terraform or Pulumi touches the cluster, our Proxmox VE nodes (pve01–pve03) are standardized with ZFS/NVMe storage pools, VLAN-aware Linux bridges (vmbr0 trunking production, IoT, lab, and management VLANs), and minimal Cloud-Init golden templates.

Building a reproducible Ubuntu / Debian Cloud-Init template (VMID 9000) on Proxmox takes five CLI commands:

cli
qm create 9000 --name ubuntu-2404-cloudinit --memory 2048 --cores 2 --net0 virtio,bridge=vmbr0
qm importdisk 9000 noble-server-cloudimg-amd64.img local-zfs
qm set 9000 --scsihw virtio-scsi-single --scsi0 local-zfs:vm-9000-disk-0,discard=on,ssd=1
qm set 9000 --ide2 local-zfs:cloudinit --boot c --bootdisk scsi0 --serial0 socket --vga serial0
qm set 9000 --agent enabled=1 && qm template 9000

3. Declarative VM Provisioning with Terraform

For core stateful infrastructure—such as NetBox, GitLab runners, Wazuh SIEM, and DNS resolvers—we use Terraform with the bpg/proxmox provider and HashiCorp Vault integration. Cloning our Cloud-Init template into a production VM with static VLAN tagging and SSH key injection is completely declarative:

cli
terraform {
  required_providers {
    proxmox = {
      source  = "bpg/proxmox"
      version = "~> 0.66.0"
    }
    vault = {
      source  = "hashicorp/vault"
      version = "~> 4.4.0"
    }
  }
}

data "vault_kv_secret_v2" "proxmox_creds" {
  mount = "secret"
  name  = "ai-ops/proxmox"
}

provider "proxmox" {
  endpoint  = data.vault_kv_secret_v2.proxmox_creds.data["api_url"]
  api_token = data.vault_kv_secret_v2.proxmox_creds.data["api_token"]
  insecure  = false
}

resource "proxmox_virtual_environment_vm" "netctl_node" {
  name        = "netctl01"
  node_name   = "pve01"
  vm_id       = 25510
  description = "Managed by Terraform & GitLab CI/CD (ai-ops)"
  tags        = ["terraform", "homelab", "netdevops"]

  cpu {
    cores = 4
    type  = "host"
  }

  memory {
    dedicated = 8192
  }

  clone {
    vm_id = 9000
    full  = true
  }

  disk {
    datastore_id = "local-zfs"
    interface    = "scsi0"
    size         = 64
    discard      = "on"
    ssd          = true
  }

  network_device {
    bridge  = "vmbr0"
    model   = "virtio"
    vlan_id = 255
  }

  initialization {
    ip_config {
      ipv4 {
        address = "172.28.255.10/24"
        gateway = "172.28.255.1"
      }
    }
    user_account {
      username = "ops"
      keys     = [data.vault_kv_secret_v2.proxmox_creds.data["ssh_pubkey"]]
    }
  }
}

4. Dynamic Infrastructure & Kubernetes Fleets with Pulumi

While Terraform excels at static resource blocks, Pulumi allows us to use real Python and TypeScript to dynamically query our NetBox SSOT and generate entire fleets of Kubernetes (home1) worker nodes, GPU passthrough VMs, and monitoring probes without duplicating boilerplate HCL:

cli
import pulumi
import pulumi_proxmoxve as proxmox

# Dynamically provision Kubernetes worker nodes across Proxmox cluster hosts
k8s_workers = [
    {"name": "k8s-worker-01", "node": "pve01", "vmid": 25221, "ip": "172.28.252.21/24", "cores": 8, "mem": 16384},
    {"name": "k8s-worker-02", "node": "pve02", "vmid": 25222, "ip": "172.28.252.22/24", "cores": 8, "mem": 16384},
    {"name": "k8s-worker-03", "node": "pve03", "vmid": 25223, "ip": "172.28.252.23/24", "cores": 8, "mem": 16384},
]

for w in k8s_workers:
    proxmox.vm.VirtualMachine(
        w["name"],
        node_name=w["node"],
        vm_id=w["vmid"],
        name=w["name"],
        tags=["pulumi", "kubernetes", "home1"],
        cpu=proxmox.vm.VirtualMachineCpuArgs(cores=w["cores"], type="host"),
        memory=proxmox.vm.VirtualMachineMemoryArgs(dedicated=w["mem"]),
        clone=proxmox.vm.VirtualMachineCloneArgs(vm_id=9000, full=True),
        network_devices=[
            proxmox.vm.VirtualMachineNetworkDeviceArgs(bridge="vmbr0", model="virtio", vlan_id=252)
        ],
        initialization=proxmox.vm.VirtualMachineInitializationArgs(
            ip_configs=[
                proxmox.vm.VirtualMachineInitializationIpConfigArgs(
                    ipv4=proxmox.vm.VirtualMachineInitializationIpConfigIpv4Args(
                        address=w["ip"],
                        gateway="172.28.252.1",
                    )
                )
            ]
        ),
    )

5. Automated GitLab CI/CD Pipelines with Human-in-the-Loop Gating

Every infrastructure change flows through our self-hosted GitLab CI/CD instance. Merge Requests automatically trigger syntax validation, security scanning, terraform plan, and pulumi preview, posting the exact infrastructure diff back to the Merge Request before a human approves the production apply:

cli
stages:
  - validate
  - plan
  - apply

variables:
  TF_IN_AUTOMATION: "true"

terraform:validate:
  stage: validate
  image: hashicorp/terraform:1.9
  script:
    - terraform init -backend=false
    - terraform fmt -check
    - terraform validate

terraform:plan:
  stage: plan
  image: hashicorp/terraform:1.9
  script:
    - terraform init
    - terraform plan -out=tfplan.binary
    - terraform show -no-color tfplan.binary > plan.txt
  artifacts:
    paths:
      - tfplan.binary
      - plan.txt
    expire_in: 7 days

terraform:apply:
  stage: apply
  image: hashicorp/terraform:1.9
  script:
    - terraform init
    - terraform apply -auto-approve tfplan.binary
  dependencies:
    - terraform:plan
  rules:
    - if: $CI_COMMIT_BRANCH == "master"
      when: manual

6. Leveraging AI for a Homelab: The Agentic AI Control Plane

The true multiplier in our architecture is leveraging AI for a homelab across day-2 operations. Instead of giving an LLM unconstrained root access, we pair an Orchestrator + Scoped Sub-Agent architecture with deterministic guardrails:

  1. Knowledge Graph & Topology Context (<150 Tokens): Before proposing any Proxmox, Kubernetes, or network change, agents query our NetBox-backed topology CLI and Graphify knowledge graph to map dependencies across hypervisors, VLANs, and services.
  2. Automated Drift & CI Triage: When a GitLab CI/CD pipeline fails or Prometheus/Wazuh fires an alert, an autonomous remediation agent inspects the logs, correlates recent commits, and stages a targeted Merge Request.
  3. Maintenance Window Coordination: Before restarting Kubernetes workloads or migrating Proxmox VMs, agents pause auto-remediation watchdogs so health probes never fight an active deployment.
  4. Post-Change Telemetry Verification: After a Terraform or Pulumi apply completes, agents verify QEMU guest agent heartbeats, Prometheus node-exporter metrics, and Blackbox HTTPS probes to confirm 100% service health.

By combining declarative Proxmox IaC (Terraform + Pulumi + GitLab CI/CD) with Agentic AI operations, our homelab runs with the reliability, auditability, and velocity of a hyperscale cloud environment.

Related Engineering Notes

4 Comments

  1. Passing the SSH public key and static IPv4 CIDR directly through the Proxmox cloud-init initialization block eliminated all of our DHCP race conditions during automated k3s node bootstrap.

  2. Switching the Proxmox provider to `virtio-scsi-single` with `iothread=true` and `discard=on` made a noticeable difference in etcd write latency on our local NVMe ZFS pool.

  3. Running terraform plan -detailed-exitcode inside a self-hosted GitLab Runner before allowing any Proxmox API mutations is the exact drift gate we needed for our homelab cluster.

  4. The dual-stack IaC pattern here is spot on. Using Terraform (bpg/proxmox) for deterministic cloud-init VM provisioning and Pulumi Python for Kubernetes post-boot orchestration cleanly separates hypervisor state from workload config.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.