> ## Documentation Index
> Fetch the complete documentation index at: https://docs.profy.cn/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent runtime, sandboxes, and SSE streaming

> How Profy runs AI Experts on the Agent Runtime in declarative E2B and Docker sandboxes, with SSE streaming, projects, and credit-based billing.

# Agent Platform

The **Expert workforce** surface in Profy is the AI agent platform: **Experts** (AI expert products) run on the **Agent Runtime** (`services/agent-runtime`) inside **declarative sandboxes** (E2B/Docker), with **Profy** as the AI model brand, streaming (SSE), and credit-based billing. **Projects** group collaboration; the **Marketplace** connects creators to Experts and skills.

## Overview

```mermaid theme={null}
flowchart TB
    subgraph "Frontend (Next.js)"
        Chat[Chat UI]
        Mgmt[Expert / session management]
        Preview[App preview]
    end

    subgraph "Next.js API & tRPC proxy"
        AgentApi["/api/agent/* · /api/sessions/* · /api/projects/*"]
    end

    subgraph "Agent Runtime (FastAPI)"
        direction TB
        RT[Agent Runtime<br/>invoke · SSE · tools]
        SB1[Sandbox 1 · E2B/Docker]
        SB2[Sandbox 2 · E2B/Docker]
        SB3[Sandbox 3 · E2B/Docker]
    end

    subgraph "Core API"
        ExpertApi["tRPC · REST<br/>sessions · projects · billing"]
        OpenAPI["/openapi/credit/*"]
    end

    Chat --> AgentApi
    AgentApi -->|"invoke / stream"| RT
    RT --> SB1
    RT --> SB2
    RT --> SB3
    Mgmt --> ExpertApi
    AgentApi -->|"Credit billing"| OpenAPI
```

## Architecture

### Agent Runtime

The **Agent Runtime** is the engine in `services/agent-runtime/`. Each **Expert** conversation runs in an isolated **declarative sandbox** (for example **E2B** or **Docker/OpenSandbox**) with its own workspace, tools, and filesystem, exposed to the Web app via HTTP/SSE.

### Profy Models

**Profy** is the AI model product name. It is **not** a Kubernetes control plane in the current architecture: orchestration is **sandbox-provider driven** (E2B/Docker), configured via Agent Runtime environment variables.

| Concept                     | Description                                                                 |
| --------------------------- | --------------------------------------------------------------------------- |
| **Sandbox**                 | Isolated runtime backing an Expert session (provider-specific)              |
| **Expert / session config** | Model, skills, preferences — stored in Core and passed to the runtime       |
| **Secrets**                 | API keys and credentials — injected via env / secrets stores, not committed |
| **Agent Runtime service**   | Single logical entry point the Web app reaches via `BACKEND_URL`            |

### Expert & sandbox lifecycle

```mermaid theme={null}
stateDiagram-v2
    [*] --> provisioning: create session / prepare sandbox
    provisioning --> running: sandbox ready + skills loaded + workspace prefetched
    provisioning --> error: sandbox failed
    running --> stopped: user stop / idle timeout
    running --> error: sandbox crash
    stopped --> provisioning: user restart
    error --> running: auto-recovery (runtime reachable + healthy)
    error --> provisioning: manual restart
    stopped --> [*]: delete
```

**Idle timeout**: Long-lived resources may be reclaimed after extended inactivity; exact policy depends on deployment and provider.

### Stop vs delete

| Operation  | Sandbox behavior                                       | Recovery                   |
| ---------- | ------------------------------------------------------ | -------------------------- |
| **Stop**   | Sandbox paused or released; metadata often retained    | Fast resume when supported |
| **Delete** | Session / resources removed in Core; sandbox torn down | Unrecoverable              |

> **Design note**: Prefer explicit stop/resume over hard deletes for chat UX, so URLs and routing stay stable while sandboxes are elastic.

### Error state auto-recovery

The `error` state is not always terminal. Health checks against the **Agent Runtime** and sandbox readiness can move a session back to `running` when the runtime and workspace are healthy again; otherwise the user may **restart** to obtain a fresh sandbox.

### Chat entry readiness gate

Before entering the chat view, the frontend typically waits for real signals in sequence:

```
Sandbox allocated → Agent Runtime reachable → Skills loaded → Workspace files prefetched → Enter chat
```

| Step | Signal        | Meaning                                                 |
| ---- | ------------- | ------------------------------------------------------- |
| 1    | Runtime ready | Agent Runtime HTTP reachable for the session            |
| 2    | Skills ready  | Marketplace or URL skills materialized in the workspace |
| 3    | Files loaded  | Client-side tree / prefetch aligned with sandbox state  |

**Every entry can go through a loading state**: Even warm sessions may re-verify readiness after refresh.

### Frontend file state management

The platform keeps **session** trajectory separate from **workspace file listings** (sandbox-backed):

| Concept                 | Level            | Description                                          |
| ----------------------- | ---------------- | ---------------------------------------------------- |
| Workspace file list     | Session / Expert | Files visible in the IDE, tied to the active sandbox |
| Conversation trajectory | Session          | Tool calls, todos, streaming deltas                  |
| Persisted events        | Session          | Stored events for replay after refresh               |

Switching sessions should **not** silently wipe a freshly loaded workspace file list; that state is tied to the Expert/sandbox view, not only the chat transcript.

## APIs (Web → Core → Agent Runtime)

Most **session** and **project** operations go through **tRPC** on the Core API (`/api/trpc/*` from the browser), with Next.js route handlers under `/api/sessions/*` and `/api/projects/*` as thin proxies where needed.

### Representative Next.js routes (`apps/web`)

| Path                                       | Description                                                    |
| ------------------------------------------ | -------------------------------------------------------------- |
| `POST /api/agent/invoke`                   | Agent mode invoke (tools, planning) — proxies to Agent Runtime |
| `POST /api/agent/resume`                   | Resume / continue flows                                        |
| `GET/POST/PATCH/DELETE /api/sessions/[id]` | Session CRUD proxy                                             |
| `GET/POST /api/sessions/[id]/messages`     | Messages for a session                                         |
| `GET/POST /api/projects/...`               | Project-scoped collaboration                                   |
| `GET/POST /api/skills/*`                   | Skill install / metadata                                       |
| `GET /api/credits`                         | Credit balance                                                 |
| `GET/POST /api/finance/*`                  | Finance (recharge, subscription, giftcard)                     |

Exact surface evolves with the router tree; treat `apps/web/src/app/api/**` as the source of truth.

## SSE streaming chat

The chat interface uses **Server-Sent Events** between the user and the **Agent Runtime** (via Next.js).

### Request flow

```mermaid theme={null}
sequenceDiagram
    participant U as User
    participant W as Next.js
    participant R as Agent Runtime

    U->>W: POST /api/agent/invoke (or stream route)
    W->>W: Validate auth, resolve session / Expert
    W->>R: Forward to runtime (HTTP/SSE)
    R-->>W: SSE stream

    loop Stream events
        W->>W: Normalize event shape for the client
        W-->>U: SSE: text / tool_call / complete
    end

    Note over W: On completion
    W->>W: Extract token usage where applicable
    W->>W: POST /openapi/credit/consume (API Key)
    W-->>U: SSE: token_usage + complete
```

### SSE event types (illustrative)

| Event                           | Description                            |
| ------------------------------- | -------------------------------------- |
| `text`                          | Streaming model text                   |
| `tool_call_chunk` / `tool_call` | Tool invocation in progress / complete |
| `tool_call_result`              | Tool output                            |
| `token_usage`                   | Usage for billing                      |
| `complete`                      | Stream finished                        |
| `error`                         | Failure                                |

Sandbox and website preview readiness are **not** pushed over SSE. The Start Preview button is a synchronous `POST /api/preview/start` whose response carries the preview URL; sandbox status is pulled on demand from `GET /api/sandbox/status`. The agent SSE stream only carries "what the agent is doing right now".

### Routing to the runtime

The Web server uses **`BACKEND_URL`** (for example `http://agent-runtime:8000` in Docker Compose) to reach the Agent Runtime. Public ingress is handled by **Nginx** → **Next.js**; the browser does not call the runtime origin directly for authenticated chat.

## Credit billing

Expert usage is billed through the credit system. After a billable completion, the proxy layer reports token usage to the Core API.

```mermaid theme={null}
flowchart LR
    Chat[Chat complete] --> Extract[Extract token count]
    Extract --> Consume["POST /openapi/credit/consume"]
    Consume --> Core[Core API]
    Core --> Balance[Deduct from balance]
```

Consumption uses **API Key authentication** (`sk_*` prefix) where configured, so the Next.js layer can bill on behalf of the signed-in user without forwarding end-user JWTs to the OpenAPI billing surface.

## Service plans

Experts and Profy-backed features use tiered service plans (models, limits, entitlements). Example list query:

```
GET /api/finance/servicePlan/list?serviceCode=profy
```

(`profy` is the **service code** key in finance data.)

## Configuration & drift

Expert and session configuration (model, skills, system prompt, preferences) lives in **Core** and is applied on the next **restart** or **sandbox refresh** when drift is detected. Use the product **settings / drift** flows in the app to confirm when a restart is required.

## Health monitoring

Health combines:

1. **Agent Runtime** HTTP health (container / process up).
2. **Sandbox readiness** (skills, workspace) as reported by the runtime or Core aggregation.

Operational cleanup and admin routes are **environment-specific**; follow internal runbooks for production.

## Related Pages

<CardGroup cols={2}>
  <Card title="Finance & Payments" icon="credit-card" href="/en/documentation/finance">
    Credit billing and consumption
  </Card>

  <Card title="Authentication" icon="lock" href="/en/documentation/auth">
    API Keys bridge proxy and billing
  </Card>

  <Card title="Marketplace" icon="store" href="/en/documentation/marketplace">
    Skills and Experts from the Marketplace
  </Card>

  <Card title="Deployment" icon="rocket" href="/en/documentation/deployment">
    Docker Compose, Agent Runtime, and Nginx
  </Card>
</CardGroup>