Architecture

Oloyid sits between AI-consuming applications and LLM providers, intercepting every inference request to enforce policies, log audit events, and route to the optimal model.

System context

Developers integrate via the SDK or REST API. Security admins configure policies and guardrails through the admin portal. Human approvers review flagged requests via workflow queues. Client applications — chat bots, RAG systems, agentic workflows, and support bots — send inference requests through Oloyid, which proxies to OpenAI, Anthropic, Azure OpenAI, or local SLMs.

Actors

Developer — Integrates Oloyid SDK or API into applications
Security Admin — Configures policies, guardrails, and model routes
Human Approver — Reviews and resolves flagged requests
Client Apps — Chat, RAG, agents, support bots sending inference requests
LLM Providers — OpenAI, Anthropic, Azure OpenAI, local SLMs (Phi-4, Llama, Mistral)

Request lifecycle

Every inference request flows through a deterministic pipeline with short-circuit on violation:

Client App → Gateway (Auth) → Input Guardrails → Policy Engine → Model Router → LLM Proxy → Runtime Guardrails → Output Guardrails → Response to Client

Pipeline stages

Stage	Responsibility
Gateway	JWT authentication, tenant resolution, rate limiting
Input	Scan user prompts for PII, PHI, toxicity, jailbreak attempts
Policy	Evaluate YAML policies and symbolic rules against request context
Router	Select optimal model based on routing rules and cost/performance criteria
LLM Proxy	Forward request to selected provider, handle streaming responses
Runtime	Monitor tool calls, agent actions, and intermediate outputs during inference
Output	Scan LLM responses for sensitive data, policy violations, hallucination flags

Service architecture

Admin Portal (React, port 3000) → FastAPI API (port 8000) → PostgreSQL (5432) + Redis (6379)

Core services

FastAPI API — REST endpoints, guardrail pipeline orchestration, auth
Admin Portal — React/MUI dashboard for configuration and monitoring
PostgreSQL — Organizations, users, policies, guardrails, audit logs
Redis — Session cache, rate limiting, async job queues

Multi-tenancy

Every resource is scoped to an organization. Users belong to organizations with role-based access control. Policy chains, guardrail configurations, and audit logs are tenant-isolated. API requests resolve the tenant from the JWT token.

Industry packs

Oloyid ships with pre-built compliance packs that activate specialized guardrails and audit trails:

HealthShield — HIPAA compliance, PHI detection, patient consent tracking
FinanceShield — Financial data protection, PCI-aware scanning, audit trails

See Guardrails & Policies for configuration details and API Reference for HealthShield and FinanceShield endpoints.