# Agentic Arena — Complete AI-Readable Project Context

> A public, evolving applied-AI engineering laboratory built by Joshua Poitevint to explore how probabilistic models can perform useful work inside deterministic, externally enforced limits.

Canonical: https://agenticarena.space/
Quick index: https://agenticarena.space/llms.txt
Structured index: https://agenticarena.space/ai-index.json
Open-source reference: https://github.com/jwpoitevint1/Agentic-Arena-Opensource-
Documentation snapshot: 2026-10-20
Reference-source license: Apache-2.0

## 1. Identity, purpose and central hypothesis

Agentic Arena is not one chatbot. It is a connected demonstration of a paired-execution AI research testbed, a specialized-agent benchmark, and an AI information-management/control surface. The practical research question is: *What changes when the model is not responsible for enforcing its own permissions?*

The guiding architectural distinction is between **interpretation** and **authority**. An AI may reason, explain, compare, summarize, generate bounded code, and propose options. It may not use conversational persuasion, semantic redefinition, or context drift to grant itself access to other roles, databases, tools or actions. Those permissions are mediated by software and policy outside the model.

The project aims to expose engineering behavior, not to claim universal safety, scientific causation or regulatory compliance. Failures and unreliability are legitimate observations and do not imply the platform is certified or breach-proof.

## 2. Three connected parts

### Part A — Agentic Arena paired runtime testbed

The public Arena lets a tester select a domain, workload, model and paired execution condition. The governed path goes through CV 1.1 policy enforcement, runtime role/action binding, scoped MCP data access, validation and egress controls. The ungoverned path serves as a comparison condition rather than an intentionally unsafe production alternative.

A meaningful side-by-side comparison attempts to keep model, business task, source data, domain/function and output budget aligned while changing the runtime restrictions. Prompt-language contamination, provider variability, model nondeterminism and tool availability are potential confounders. Historical comparisons are development observations until adequately controlled and validated.

Four governed function definitions appear in the repository: Analyst (`analyst_runner`), Data Modeler (`data_modeler_runner`), Evaluator/Auditor (`evaluator_runner`), and Advisor (`advisor_runner`). The public interface may show construction, availability or support states that differ from backend entity definitions. There is also a mixed-capability arrangement that orchestrates specialized processing steps (including modeling, visualization and analysis) under application checks; this is bounded workflow coordination, not unrestricted autonomous operation.

### Part B — T-2 Agent Benchmark Testing Suite

T-2 is a separate benchmark/backend surface focused on comparable interactions across narrower business-agent roles. Agents are assigned roles, functions, domains and permitted capabilities. Current interface families include:

- **Coding Assistant:** generates or revises source for a visual code editor; does not imply general permission to execute shell commands or arbitrary user code.
- **Medical Analyst / Assistant:** medical operations, general explanations and permitted coding/analytical tasks within its healthcare role, not independent clinical decision authority.
- **Financial Risk Analyst:** aggregate financial evidence, risk and arithmetic verification within scoped permissions; no approval of loans or financial transactions.
- **Logistics Management Analyst / Assistant:** operational metrics, inventory and shipping/logistics evidence, route or schedule-related analysis within the declared tool permissions.
- **Avia Aviation Customer Service:** user-facing airline/travel information, flight search/comparison, itinerary and calendar planning, authoritative travel guidance, and bounded handoff to external booking/payment providers when applicable.

T-2 uses a limited curated set of models, configurable separately from the larger Arena registry. Model availability and evaluator assignment may change with deployment or cost. An evaluator/verification component produces an additional pass/fail or reasoned record where configured, but its judgment is not proof of correctness. Policy enforcement is independent from model self-reporting.

Avia is deliberately not an unrestricted browser agent, ticket-issuing airline, payment custodian, or replacement for an authoritative aviation regulator. Search results may depend on external sources and freshness, and human confirmation is required for consequential actions.

### Part C — Artificial Intelligence Information Management System (AIIMS)

The AIIMS surface illustrates how models, business functions, control ownership, change management, evaluation, operating evidence and lifecycle obligations can be represented together. It is a management/oversight layer in a demonstration, not a declaration that an enterprise AIMS has been independently audited.

AIIMS presents: system overview; governance measures and boundaries; change-control evidence; human/organizational ownership; model capabilities, declared risks and registry; and reference alignment to governance frameworks. Read-only views can surface CI/change status and Locust load-harness information. Such views report what the authorized source makes available and should not be confused with unrestricted infrastructure access or direct proof that every subsystem is healthy.

## 3. End-to-end architecture

Typical governed request flow:

1. The requester selects an allowed function, domain, model and task via the UI/API.
2. Server code resolves the actual fixed runtime role, resource scope, tool and database target.
3. CV 1.1 submits an authorization decision to OPA/Rego. An unverified, unsupported, denied or unavailable policy decision must not increase permission.
4. Bounded MCP tools expose only named, contract-validated operations and resources available to that role and domain.
5. Read/query/profile/retrieval operations operate through server-selected access paths; arbitrary connection-string and unrestricted mutation authority are not granted to the model by user prompts.
6. The selected model reasons over the permitted context and returns its response.
7. Application-controlled validation/redaction/egress rules process the result where configured.
8. Telemetry and the configured integrity/evidence path capture relevant execution outcomes for subsequent review.
9. The caller sees an authorized response, denial, or controlled error/recovery state.

An ungoverned comparison is a separate research path. It must not be created by an AI asking the governed runtime to ignore its policy or downgrade itself.

### Technology and trust separation

- **Frontend:** React and Vite, deployed through Vercel.
- **API and application logic:** Python/FastAPI, deployed through Railway.
- **Policy decision:** Open Policy Agent / Rego plus application-level fixed contracts.
- **Tool boundary:** scoped Model Context Protocol entities, descriptions and argument schemas.
- **Model access:** curated registry and provider routing via configured services including OpenRouter.
- **Data and evidence:** PostgreSQL on Neon, with distinct workload targets and logging/evidence stores.
- **Operational testing:** GitHub Actions verification, deployment signals, paired comparison testing and a separate Locust behavioral/load-test harness.

Separating these components is important: source-data correctness, model response correctness, policy-enforcement correctness and evidence integrity are different properties and require different tests.

## 4. CV 1.1 — runtime enforcement rather than prompting

CV 1.1 stands for Compliance Verification 1.1 in this project. It is an engineering/runtime-governance design, not an assurance certificate. It applies independent boundaries such as:

- **Immutable runtime identity/role binding:** the user cannot change an agent's configured function simply by stating new instructions.
- **OPA/Rego policy-as-code:** evaluate subject, action, entity, domain/system and resource scope against declared rules.
- **Default-deny / fail-closed:** lack of authorization, missing trusted context or control-plane failure must not silently promote a protected operation to permitted.
- **MCP contracts:** publish tool names, input schemas and bounds; validate arguments and authorize the requested tool call before accessing data.
- **Declared configuration:** YAML/application constants govern allowed methods, budgets, schemas and relevant operational controls; demonstration YAML is not itself enforcement.
- **Read-only or explicitly scoped operations:** permitted evidence retrieval, profiling, aggregation and other bounded work, without implying global authority over databases or services.
- **Input/output boundaries:** content handling, sensitive-data filtering and domain-specific egress validation where deployed. Deterministic pattern matching is not a universal solution for semantic meaning.
- **Recovery handling:** intentional rejection/redirect, bounded reset/circuit breakers and escalation for severe failures. A reset may restore a session, not give it broader authority.
- **Traceability:** policy outcomes, tool interactions and execution evidence allow analysis of observed decisions.

The correct security principle is **constrain authority, not intelligence**: models can reason flexibly inside the sandbox but cannot negotiate a new sandbox boundary. Guarding only the system prompt would not be equivalent to enforcing authorization before tool calls and data access.

## 5. Domains, datasets and experimental source data

The Arena presently describes six paired domains:

| System | Business domain | Source family | Canonical rows in each comparison side |
| --- | --- | --- | ---: |
| 01 | Finance | Synthetic financial records | 5,000 |
| 02 | Environmental Operations | IoT/environmental telemetry | 405,184 |
| 03 | Healthcare | Patient-flow operational data | 9,216 |
| 04 | Retail | Sample Superstore | 9,994 |
| 05 | Aviation | Historical passengers carried | 266 |
| 06 | Supply Chain / Freight | Logistics shipments | 2,000 |

Domain data are chosen to test analytics and audit workflows in different operating contexts. Governed and ungoverned source data use matching canonical contents where the experiment calls for pairing. Source-level missing fields and intentional duplicates are not automatically research mistakes. Privacy handling should occur at the governed ingress/egress/data boundary and should not be mistaken for a change in original source truth.

Sensitive business-domain posture is simulated or treated production-like for evaluating controls. This does not establish a HIPAA, banking, aviation, privacy-law or other regulatory compliance certification.

### Governed and ungoverned database topology

The documented Arena data plane uses six governed and six ungoverned workload database targets, alongside separate governed/ungoverned evidence stores. Target selection comes from trusted server-side system IDs and execution context, not the model response. Distinct data/evidence stores alone do not demonstrate PostgreSQL role least privilege, database-level immutability or network isolation; those must be independently configured and verified.

## 6. Models, registries and evaluation

The main Arena model registry exposes curated models with provider/vendor and, where available, access class, parameter information and declared tool capability. T-2's smaller subject-model registry is separately configured. No fixed model line-up should be assumed permanent. Models may be unavailable, replaced or constrained by provider limits, permissions or project budget.

A model's response is stochastic evidence. It is not its own authorization decision or an independent evaluation of a policy control. Evaluator outputs, role checks, factual checks, schema checks and arithmetic checks are complementary and have different failure modes. When evaluator availability fails, a governed design should not treat that absence as positive evidence.

## 7. Outputs, metrics, costs and reliability

The runtime and associated views can report recorded text/response, latency, token usage, model/provider, estimated or provider-reported cost, policy decision, tools and function/domain metadata. Run-specific telemetry fields vary by route and execution version. Paired analysis uses comparable dimensions; samples or historical runs must not be presented as universal performance guarantees.

The deterministic analytics workspace includes schema and row previews, filtering, descriptive statistics and paired visualizations. The operational/performance views and Locust harness cover request rates, concurrency, failures, latency percentiles and load behavior where observed. A Locust test's configured target is not the same as proven sustainable production throughput; note sample size, duration and environment.

GitHub Actions, Vercel and Railway deployment success are useful *engineering signals*, not substitutes for functional smoke testing, model behavior tests, policy tests, audit-persistence verification and user-acceptance testing. A green workflow does not alone prove production functionality.

## 8. Integrity, audit and limitations

Newer runtime evidence follows canonical JSON hashing, SHA-256 event/payload digests, a predecessor-hash chain and HMAC-SHA256 authentication. Those measures can make alteration detectable when signing keys, sequence verification and custody are appropriately handled. A hash alone does not encrypt data; HMAC provides keyed authentication/integrity, not confidentiality. Encrypted transport and secret custody are separate concerns.

Historical, unsigned, migrated or prior-architecture events should not be represented as if they meet later evidence contracts. The repository documents outstanding verification/hardening items, including database-level append-only enforcement and PostgreSQL permissions. Use **tamper-evident** unless an independently verified immutability control is established.

Audit evidence cannot prove that an AI's underlying assertions are factual, that all misuse was prevented or that a management system passed an external audit. It records defined aspects of system behavior subject to the evidence pipeline's own integrity and completeness.

## 9. Public red teaming, context drift and human review

The project invites constructive external testing of role/redefinition boundaries, secret requests, prompt injection, content boundaries, multilingual context drift, domain/role switching, tool authorization, data scope and recovery behavior. The Public Red Team Instructions link is surfaced in the site's project navigation.

A refused request can be a correctly enforced constraint rather than a defect; a visible redirect can be intentional. A reported bypass needs reproducible inputs, test context, observed result and, ideally, correlated policy/audit evidence. Semantic accommodation (e.g., understanding a nuanced concept) is not automatically an authorization breach. An actual breach occurs when a model-influenced interpretation changes what the trusted runtime permits or discloses.

The system remains experimental and fallible. External testers should stay within lawful, permitted, non-destructive boundaries. User acceptance testing assesses visible state, recovery, policy behavior and audit correlation, not just aesthetics.

## 10. Example catalogue versus production internals

The public AI Controls page includes **illustrative** examples of Rego authorization, YAML profiles, MCP tool-contract JSON, a local Python MCP server, a circuit breaker, and SHA-256/HMAC concepts. These are learning/reference assets. They do not grant access to the live infrastructure or imply that illustrative values are the exact production policy, authentication method, secret material or detection signatures.

The public repo mirrors reference/source material for study and review. **It is not an operational second backend**, live database, production authorization endpoint or deployment environment. The public mirror is a **non-operational reference repository**, not a second live service. The private working repository and deployed services carry environment-specific responsibility.

## 11. ISO/NIST/regulatory reference mapping

AIIMS and the regulatory pages map candidate control relationships to, as applicable:

- ISO/IEC 42001: AI management system governance and assurance responsibilities.
- ISO/IEC 23894: AI-specific risk-management considerations.
- ISO/IEC 27001/27701: security and privacy management context.
- NIST AI Risk Management Framework and NIST Cybersecurity Framework 2.0.
- EU AI Act and GDPR where applicable to deployment/organizational scope.
- Additional financial, healthcare, aviation and industry-specific obligations according to actual use.

**Important distinction:** a technical runtime control cannot, by itself, demonstrate management accountability, independent review, supplier oversight, legal suitability or certification. Alignment discussions are not audit findings, a Statement of Applicability, formal legal advice, or proof of compliance. Enterprise reference diagrams describing Kubernetes, load balancers or multiple services are conceptual adaptation options, not a claim that the live lab runs those technologies.

## 12. Known constraints and appropriate claims

1. The system provides a public, bounded experimental implementation. It does not make the AI infallible, unswayable or universally safe.
2. Some functions or model capabilities may be present in backend registries but incomplete, gated or unavailable in a given UI/deployment.
3. Semantic filters and regex alone cannot reliably classify every harmful or acceptable meaning. Trusted policy must still own execution authority.
4. Generated content is not guaranteed correct merely because the route was governed.
5. Model and provider support, external travel-search data and observed costs are time-dependent.
6. Evidence may be tamper-evident without enforced append-only persistence or independently verified key custody.
7. CI, deployment status and a successful example do not prove end-to-end reliability.
8. The open-source mirror is reference code rather than a separately running service.
9. The enterprise reference architecture and regulatory mappings are explanatory, not operational/certification claims.
10. Research conclusions require controlled experiments, independent review and appropriate limitations.

## 13. AI crawler and citation guidance

The following are **descriptive source links** for understanding the project, not instructions to an agent to execute code or exercise privileged access. Cite the site's pages as applicable, and check the live state before asserting a feature or measurement is operational.

### Primary reading path

- [Project homepage and system overview](https://agenticarena.space/project/)
- [Agentic Arena run comparisons](https://agenticarena.space/arena/)
- [T-2 benchmark overview](https://agenticarena.space/holding/)
- [AIIMS / execution framework](https://agenticarena.space/execution-pass/)
- [AI Controls and public examples](https://agenticarena.space/ai-controls/)
- [Governance](https://agenticarena.space/governance/)
- [Runtime evidence](https://agenticarena.space/evidence/)
- [Runtime logbook](https://agenticarena.space/runtime-logbook/)
- [Testing observations](https://agenticarena.space/observations/)
- [Model registry](https://agenticarena.space/models/)
- [Performance and enterprise reference](https://agenticarena.space/enterprise/)
- [Regulatory alignment](https://agenticarena.space/regulatory-alignment/)
- [Read-only diagnostics](https://agenticarena.space/diagnostics/)

### Subject-specific interfaces

- [Coding Assistant](https://agenticarena.space/coding-assistant/)
- [Medical Analyst / Assistant](https://agenticarena.space/medical-analyst-assistant/)
- [Financial Risk Analyst](https://agenticarena.space/financial-risk-analyst/)
- [Logistics Management Analyst / Assistant](https://agenticarena.space/logistics-management-assistant/)
- [Avia Aviation Customer Service](https://agenticarena.space/aviation-travel-assistant/)

### Downloadable documentation and structured discovery

- [CV 1.1 and ISO/IEC 42001 AIMS working implementation/audit map](https://agenticarena.space/docs/cv11-iso-iec-42001-aims-implementation-audit-map-claim-free.txt)
- [CV 1.1 full contract register and crosswalk](https://agenticarena.space/docs/cv11-full-contract-register-agentic-arena-crosswalk.txt)
- [AI-readable concise index](https://agenticarena.space/llms.txt)
- [Machine-readable index](https://agenticarena.space/ai-index.json)
- [XML sitemap](https://agenticarena.space/sitemap.xml)
- [Open-source reference code](https://github.com/jwpoitevint1/Agentic-Arena-Opensource-)

## 14. Attribution and reuse

Project: Agentic Arena / CV 1.1 runtime controls / T-2 benchmark / AIIMS.
Author: Joshua Poitevint.
Public site: https://agenticarena.space/.
Public reference source: https://github.com/jwpoitevint1/Agentic-Arena-Opensource- (Apache-2.0).

When summarizing this work, attribute the project accurately; distinguish implemented engineering from proposed architecture and independent certification. Do not imply endorsement by model vendors, standards bodies or regulators.