Enterprise networks rarely fail from a single bug—they fail from configuration entropy:
- Interface descriptions referencing retired peers (
TEMP_UPLINK_DO_NOT_TOUCH) - Drifted hostnames across datacenters, cloud interconnects, and branches
- Legacy ACLs & firewall policies with shadowed rules,
anyservices, and stale objects - Undocumented BGP neighbors, VTI tunnels, and LAG members
Manual audits take months. Here is how we built an Agentic AI (ai-ops) pipeline to enforce deterministic standards across 1,000+ devices—without letting an LLM wing a live commit:
1. Multi-Source Truth Correlation (~50ms)
Before proposing changes, agents correlate 4 live sources:
- Oxidized: Config history across firewalls, switches, and routers
- NetBox (SSOT): Authoritative racks, circuits, prefixes, and roles
- LibreNMS & Prometheus: Live LLDP/CDP, ARP/MAC, and BGP state
- Graphify Graph: Pre-compiled topology queried in
<150tokens for blast radius
2. Semantic Interface, Hostname & Tunnel Normalization
Cross-verifying LLDP/CDP and BGP state against IPAM enforces structured schemas:
- Interfaces:
<peer_host>:<peer_port> | <role/circuit> | <vrf/vlan> - Hostnames:
<site>-<role>-<id>aligned across CLI, DNS, IPAM, and monitoring - VTI Tunnels: Standardized metadata with remote ASN, loopbacks, and routing boundaries (cloud workload vs. management paths)
3. Autonomous ACL & Firewall Policy Hygiene
Agents parse vendor syntax into normalized models to:
- Replace inline IPs with canonical, role-scoped address objects
- Flag shadowed rules, expired exceptions, and zero-hit policies via hit counters
- Enforce least-privilege segmentation across corporate, cloud, and branch zones
4. Orchestrator + Disposable Sub-Agent Architecture
To prevent LLM context rot at scale, we use a two-tier design:
- Orchestrator (
<40ktokens): Plans site waves and synthesizes fleet diffs - Disposable Sub-Agents: Audit one site/rack, query the subgraph, and output compact diffs—cutting tokens ~80% with 100% precision
5. Candidate Staging & Human-in-the-Loop Approval
AI never autonomously commits to live gear:
- Stage in candidate config or dry-run mode
- Run syntax, policy, and routing pre-checks
- Attach an exact Before/After diff to the change ticket
- Require Human Approval before commit + post-change BGP/port verification
The Result
Multi-quarter cleanups are now a continuous pipeline—turning show interface status into a living diagram.
How are you tackling config drift and ACL hygiene at scale?
>_ Engineering Discussion 4 Comments
+ Add Field Note ↓Leave a Technical Comment
This site uses Akismet to reduce spam. Learn how your comment data is processed.
Using NetBox v4 as the programmatic Single Source of Truth before letting the AI agent generate any RouterOS or Junos remediation diffs is critical. Without an authoritative intent model, LLMs just guess VLAN tags.
The three-stage pipeline (Read-Only Telemetry -> Deterministic Diff Generation -> Pre-Change Oxidized Backup) solves the biggest fear network teams have around autonomous remediation.
Validating that every access port on the switch matches the NetBox interface role and flagging rogue MACs on trunk ports caught three misconfigured uplinks in our lab on day one.
I love how the agent pairs LibreNMS SNMP polling and Wazuh SIEM alerts with deterministic Python CLI tools instead of giving raw shell access to core routers.