diff --git a/agentex/docs/docs/development_guides/architecture.md b/agentex/docs/docs/development_guides/architecture.md new file mode 100644 index 0000000..23d1829 --- /dev/null +++ b/agentex/docs/docs/development_guides/architecture.md @@ -0,0 +1,302 @@ +# Internal Architecture + +This document describes the internal code architecture of the Agentex backend. For a high-level conceptual overview, see [Agentex Overview](../getting_started/agentex_overview.md). + +## High-Level Architecture (Hexagonal/Clean Architecture) + +The codebase follows a **Hexagonal Architecture** (also known as Ports & Adapters) with clear separation between layers: + +```mermaid +graph TB + subgraph External["External Clients"] + UI[Agentex UI] + SDK[Python SDK] + Agents[Agent Containers] + end + + subgraph API["API Layer (src/api)"] + direction TB + HCI[HealthCheckInterceptor
ASGI Level] + CORS[CORSMiddleware] + Auth[AgentexAuthMiddleware] + ReqLog[RequestLoggingMiddleware] + Routes[Route Handlers
LoggedAPIRoute] + end + + subgraph Domain["Domain Layer (src/domain)"] + direction TB + UC[Use Cases] + SVC[Services] + ENT[Entities] + REPO[Repository Interfaces] + end + + subgraph Adapters["Adapters Layer (src/adapters)"] + direction TB + PG[PostgresCRUDRepository] + MONGO[MongoDBCRUDRepository] + REDIS[RedisStreamRepository] + TEMP[TemporalAdapter] + HTTP[HttpxGateway] + AUTHN[AuthenticationProxy] + AUTHZ[AuthorizationProxy] + end + + subgraph Infrastructure["Infrastructure"] + PostgreSQL[(PostgreSQL)] + MongoDB[(MongoDB)] + RedisDB[(Redis)] + Temporal[Temporal Server] + AuthSvc[Auth Service] + end + + UI --> HCI + SDK --> HCI + Agents --> HCI + + HCI --> CORS --> Auth --> ReqLog --> Routes + Routes --> UC + UC --> SVC + SVC --> REPO + + REPO -.-> PG + REPO -.-> MONGO + REPO -.-> REDIS + SVC -.-> TEMP + SVC -.-> HTTP + Auth -.-> AUTHN + UC -.-> AUTHZ + + PG --> PostgreSQL + MONGO --> MongoDB + REDIS --> RedisDB + TEMP --> Temporal + HTTP --> Agents + AUTHN --> AuthSvc + AUTHZ --> AuthSvc +``` + +--- + +## Request Flow + +Every HTTP request follows this path through the system: + +```mermaid +sequenceDiagram + participant C as Client + participant HCI as HealthCheckInterceptor + participant MW as Middleware Stack + participant R as Route Handler + participant UC as Use Case + participant SVC as Service + participant DB as Database + + C->>HCI: HTTP Request + + alt Health Check Path + HCI-->>C: 200 OK (fast path) + else Regular Request + HCI->>MW: Forward + MW->>MW: CORS → Auth → RequestLogging + MW->>R: Authenticated Request + R->>UC: Dependency Injection + UC->>SVC: Business Logic + SVC->>DB: Data Access + DB-->>SVC: Result + SVC-->>UC: Domain Entity + UC-->>R: Response Schema + R-->>C: JSON Response + end +``` + +--- + +## Domain Layer Structure + +The domain layer follows a layered pattern with clear responsibilities: + +- **Use Cases**: Application-level orchestration, convert between API schemas and domain entities +- **Services**: Business logic, coordinate between repositories +- **Repositories**: Data access interfaces (implemented by adapters) + +```mermaid +graph LR + subgraph UseCases["Use Cases"] + AgentsUC[AgentsUseCase] + TasksUC[TasksUseCase] + MessagesUC[MessagesUseCase] + SpansUC[SpanUseCase] + SchedulesUC[SchedulesUseCase] + ACPUC[AgentsACPUseCase] + end + + subgraph Services["Services"] + TaskSVC[AgentTaskService] + MsgSVC[TaskMessageService] + ACPSVC[AgentACPService] + AuthzSVC[AuthorizationService] + SchedSVC[ScheduleService] + end + + subgraph Repositories["Repositories"] + AgentRepo[AgentRepository] + TaskRepo[TaskRepository] + MsgRepo[TaskMessageRepository] + SpanRepo[SpanRepository] + EventRepo[EventRepository] + end + + AgentsUC --> AgentRepo + TasksUC --> TaskSVC + MessagesUC --> MsgSVC + SpansUC --> SpanRepo + SchedulesUC --> SchedSVC + ACPUC --> TaskSVC + ACPUC --> ACPSVC + ACPUC --> AuthzSVC + + TaskSVC --> TaskRepo + TaskSVC --> EventRepo + TaskSVC --> ACPSVC + MsgSVC --> MsgRepo + ACPSVC --> AgentRepo +``` + +--- + +## Data Storage Split + +Data is split across different storage backends based on access patterns: + +```mermaid +graph TB + subgraph PostgreSQL["PostgreSQL (Relational)"] + Agents[agents] + Tasks[tasks] + Events[events] + Spans[spans] + APIKeys[agent_api_keys] + Deployments[deployment_history] + Trackers[agent_task_trackers] + end + + subgraph MongoDB["MongoDB (Documents)"] + Messages[messages
Flexible schema] + States[task_states
Key-value pairs] + end + + subgraph Redis["Redis (Streams)"] + TaskEvents[task_events_*
Real-time streaming] + end + + subgraph Temporal["Temporal (Workflows)"] + Schedules[Recurring schedules] + Workflows[Long-running workflows] + end +``` + +| Storage | Use Case | Why | +|---------|----------|-----| +| **PostgreSQL** | Agents, Tasks, Spans, Events | Relational integrity, ACID transactions, complex queries | +| **MongoDB** | Messages, States | Flexible schema for varied content types, document-oriented | +| **Redis** | Task event streams | Real-time pub/sub, SSE streaming to clients | +| **Temporal** | Schedules, Workflows | Durable workflow orchestration, reliable scheduling | + +--- + +## API Routes Overview + +All routes are organized in `src/api/routes/`: + +```mermaid +graph LR + subgraph Routes["API Routes (10 files, 26 GET endpoints)"] + A["/agents"] --> A1[CRUD + Register + RPC] + T["/tasks"] --> T1[CRUD + Streaming] + M["/messages"] --> M1[CRUD + Pagination] + S["/spans"] --> S1[CRUD] + ST["/states"] --> ST1[CRUD] + E["/events"] --> E1[CRUD] + K["/agent_api_keys"] --> K1[CRUD] + D["/deployment-history"] --> D1[List + Get] + TR["/tracker"] --> TR1[Get + Update] + SC["/agents/{id}/schedules"] --> SC1[CRUD + Pause/Trigger] + end +``` + +--- + +## Middleware Stack + +Middleware executes in this order (first to last): + +```mermaid +graph TB + REQ[Incoming Request] --> HCI + + subgraph Stack["Middleware Stack"] + HCI[HealthCheckInterceptor - ASGI Fast Path] + CORS[CORSMiddleware - Handle CORS] + AUTH[AgentexAuthMiddleware - Verify Identity] + LOG[RequestLoggingMiddleware - Request ID] + ROUTE[LoggedAPIRoute - Log and Handle] + end + + HCI -->|regular paths| CORS + CORS --> AUTH + AUTH --> LOG + LOG --> ROUTE + ROUTE --> RESP[Response] + + HCI -->|health checks| FAST[200 OK - fast path] +``` + +| Middleware | Purpose | +|------------|---------| +| **HealthCheckInterceptor** | ASGI-level fast path for Kubernetes probes | +| **CORSMiddleware** | Handle cross-origin requests | +| **AgentexAuthMiddleware** | Verify agent identity or auth tokens | +| **RequestLoggingMiddleware** | Generate request ID for log correlation | +| **LoggedAPIRoute** | Log requests/responses, handle streaming | + +--- + +## Key Files Reference + +| Layer | Directory | Key Files | +|-------|-----------|-----------| +| **API** | `src/api/` | `app.py`, `logged_api_route.py`, `authentication_middleware.py` | +| **Routes** | `src/api/routes/` | `agents.py`, `tasks.py`, `messages.py`, `spans.py`, etc. | +| **Domain** | `src/domain/` | `entities/`, `use_cases/`, `services/`, `repositories/` | +| **Adapters** | `src/adapters/` | `crud_store/`, `temporal/`, `streams/`, `http/` | +| **Config** | `src/config/` | `dependencies.py`, `environment_variables.py` | + +--- + +## Dependency Injection + +Dependencies are managed via FastAPI's dependency injection with a singleton pattern: + +```python +# In src/config/dependencies.py +class GlobalDependencies(metaclass=Singleton): + database_async_read_write_engine: AsyncEngine + temporal_client: TemporalClient + mongodb_database: MongoDBDatabase + redis_pool: redis.ConnectionPool + +# In routes, use Annotated types +async def get_agent( + agent_id: str, + agents_use_case: DAgentsUseCase, # Auto-injected +) -> AgentResponse: + ... +``` + +This pattern ensures: + +- Single connection pools shared across requests +- Easy testing via dependency overrides +- Clear dependency graphs +- Proper lifecycle management (startup/shutdown) diff --git a/agentex/docs/mkdocs.yml b/agentex/docs/mkdocs.yml index 203b844..43a39d6 100644 --- a/agentex/docs/mkdocs.yml +++ b/agentex/docs/mkdocs.yml @@ -66,6 +66,7 @@ nav: - State: concepts/state.md - Streaming: concepts/streaming.md - Development Guides: + - Internal Architecture: development_guides/architecture.md - Tutorials: development_guides/tutorials.md - Jupyter Notebooks (dev.ipynb): development_guides/jupyter_notebooks.md - Streaming Patterns: development_guides/streaming_patterns.md