Production LLMs and AI Agents on OCI: The Six-Plane Enterprise Architecture

TL;DR: Most GenAI initiatives stall because teams build the model layer and neglect the other five planes that make production possible. This blueprint maps enterprise architecture requirements to OCI services across six planes—from identity to observability—giving you the complete stack that survives compliance reviews and 3 AM incidents.

Why Architecture Matters More Than Models

Every enterprise I work with has built an LLM demo in a week.

Most are still running that same demo six months later.

The bottleneck isn't the model. It's everything around it:

The LLM is a dependency. The system is a product.

The Six-Plane Enterprise Architecture

This framework organizes production GenAI systems into six architectural planes. Each plane has:

• Clear responsibility boundaries
• Dedicated OCI services
• Specific governance controls
• Failure isolation

Plane 1: Experience Plane

Responsibility: User-facing interfaces that consume AI capabilities

Key Characteristics:

• Must handle streaming responses gracefully
• Show intermediate steps for agentic workflows
• Support human-in-the-loop approval flows
• Adapt to tool outputs (tables, charts, forms)

OCI Service Mapping:

Architecture Decision: APEX vs. Custom

Plane 2: Ingress & Policy Plane

Responsibility: Security boundary, traffic management, tenant governance

Why This Plane Fails First:

• Agents making tool calls can generate 10-100x the API traffic of a simple chat
• Token costs can explode without per-tenant quotas
• Unauthenticated endpoints become prompt injection vectors

OCI Service Mapping:

Reference Architecture: Multi-Tenant API Layer

Critical Controls:

Plane 3: Orchestration Plane

Responsibility: Agent runtime, workflow control, state management, tool routing

The Core Decision: Managed Agents vs. Framework Agents

OCI Service Mapping:

Orchestration Patterns on OCI (Detailed in Part 2):

Plane 4: Data & Retrieval Plane

Responsibility: Knowledge access, vector search, structured data, document processing

Oracle's Differentiator: Native Vector Search in Oracle Database 23ai

Unlike bolt-on vector stores, Oracle Database 23ai provides:

• Unified data model: Vectors alongside relational data
• ACID transactions: Vector operations in the same transaction as business data
• Enterprise security: Row-level security applies to vector content
• SQL integration: VECTOR_DISTANCE() in standard SQL queries

OCI Service Mapping:

The Select AI Pattern (Zero-Hallucination SQL):

-- Natural language query translated to SQL
SELECT AI 'Show me top 10 customers by revenue this quarter'
  FROM customers, orders
 WHERE order_date >= '2026-01-01';

-- Model generates SQL, executes against actual data
-- No hallucination possible: result is from real database

Plane 5: Model Plane

Responsibility: LLM endpoints, model selection, fine-tuning, inference optimization

OCI Generative AI: Model-Agnostic Enterprise AI

Enterprise Security Controls:

Plane 6: Operations & Governance Plane

Responsibility: Observability, evaluation, CI/CD, audit, lifecycle management

Why This Plane Is Non-Negotiable:

• AI systems behave non-deterministically
• Quality degrades silently without evaluation
• Compliance requires complete audit trails
• Debugging multi-agent systems requires distributed tracing

OCI Service Mapping:

Complete OCI Service Mapping

One-Page Reference: All Six Planes

Multi-Cloud Positioning: When to Choose OCI

What's Next

Part 2: Agent Orchestration Patterns — Six orchestration patterns mapped to OCI services, with decision criteria for each.

Part 3: The Operating Model — Evaluation pipelines, CI/CD for AI, incident response, and the maturity roadmap.

FAQ

Q: What's the minimum viable production architecture on OCI?

VCN with private subnets + OCI WAF + API Gateway + OCI AI Agent Platform (or OKE running LangGraph) + OCI Generative AI hosted models + Autonomous Database 23ai + Logging Analytics + APM.

Q: How does OCI compare to AWS Bedrock or Azure OpenAI?

OCI differentiates on: (1) native database integration with 23ai vector search, (2) model-agnostic multi-provider approach, (3) sovereign cloud regions for strict data residency, (4) dedicated AI clusters for isolation and predictable costs.

Q: Can I use open-source frameworks like LangGraph on OCI?

Yes. Deploy LangGraph on OKE, use OCI Generative AI as the model provider, and OCI services for all surrounding infrastructure. Part 2 details this pattern.

Q: What about cost management for unpredictable agent workloads?

Implement per-tenant token budgets at the API Gateway, use dedicated AI clusters for predictable costs, and set up cost alerts in OCI Cost Management. Part 3 covers operational cost controls.

This is Part 1 of a 3-part series on production LLM and agentic AI systems on OCI.

Sources:

• Microsoft Agent Framework Design Patterns
• Google Cloud Agent Design Patterns
• OCI Generative AI Documentation
• Oracle Database 23ai Vector Search

Related Articles

• Part 2: Agent Patterns — Implementation patterns for production agents
• Part 3: Operating Model — Day-2 operations and governance
• Oracle GenAI Agents vs LangGraph vs CrewAI — Framework comparison