# Aetheel — Process Flow How a Discord message becomes an AI response, step by step. ## The Big Picture ``` Discord User Aetheel Gateway Backend CLI │ │ │ │ @Aetheel what's 2+2? │ │ ├──────────────────────────────► │ │ │ │ 1. Extract prompt "what's 2+2?" │ │ │ 2. Check concurrency limit │ │ │ 3. Enqueue message event │ │ │ 4. Read config/*.md files │ │ │ 5. Assemble system prompt │ │ │ 6. Write prompt to temp file │ │ │ 7. Spawn CLI process │ │ │ │ │ │ claude -p "what's 2+2?" │ │ │ --output-format json │ │ │ --append-system-prompt-file ... │ │ │ --dangerously-skip-permissions │ │ ├──────────────────────────────────► │ │ │ │ │ │ ◄── JSON stream (init, result) │ │ │ ◄─────────────────────────────────┤ │ │ │ │ │ 8. Parse session_id from init │ │ │ 9. Parse result text │ │ │ 10. Split if > 2000 chars │ │ "2 + 2 = 4" │ │ │ ◄──────────────────────────────┤ │ │ │ 11. Save session for channel │ ``` ## Step-by-Step: Discord Message → Response ### Step 1: Message Arrives in Discord A user types `@Aetheel what's the weather like?` in a Discord channel. Discord delivers this to the bot as a `messageCreate` event via the WebSocket gateway. **File:** `src/discord-bot.ts` → `setupMessageHandler()` ``` Raw message content: "<@1473096872372600978> what's the weather like?" Author: tanmay11k6417 (bot: false) Channel: 1475008084022788312 ``` ### Step 2: Message Filtering & Prompt Extraction The bot checks: 1. Is the author a bot? → Skip (prevents feedback loops) 2. Does the message mention the bot? → Continue 3. Extract the prompt by stripping all mention tags **File:** `src/discord-bot.ts` → `extractPromptFromMention()` ``` Input: "<@1473096872372600978> what's the weather like?" Output: "what's the weather like?" ``` The regex `/<@[!&]?\d+>/g` strips user mentions (`<@ID>`), nickname mentions (`<@!ID>`), and role mentions (`<@&ID>`). ### Step 3: Prompt Handler (Gateway Core) The extracted prompt is wrapped in a `Prompt` object and passed to the gateway core's `onPrompt` handler. **File:** `src/gateway-core.ts` → `onPrompt` callback ```typescript { text: "what's the weather like?", channelId: "1475008084022788312", userId: "123456789", guildId: "987654321" } ``` The handler checks: - Is the gateway shutting down? → Reply "Gateway is shutting down" - Is `activeQueryCount >= maxConcurrentQueries` (default 5)? → Reply "System is busy" - Otherwise: increment counter, send typing indicator, enqueue event ### Step 4: Event Queue The prompt becomes a **message event** in the unified event queue. **File:** `src/event-queue.ts` ```typescript { id: 2, // Monotonically increasing type: "message", payload: { prompt: { text: "what's the weather like?", channelId: "1475008084022788312", userId: "123456789", guildId: "987654321" } }, timestamp: "2026-02-22T10:30:00.000Z", source: "discord" } ``` The queue processes events one at a time (FIFO). If a heartbeat or cron event is ahead in the queue, the message waits. ### Step 5: Agent Runtime — Read Config Files When the event reaches the front of the queue, the Agent Runtime reads ALL markdown config files fresh from disk. **File:** `src/markdown-config-loader.ts` → `loadAll()` ``` config/ ├── CLAUDE.md → Persona: identity, personality, user context, tools ├── agents.md → Operating rules, cron jobs, hooks ├── memory.md → Long-term memory (agent-writable) ├── heartbeat.md → Proactive check definitions └── sessions.json → Channel → session ID map (auto-generated) ``` Files are read fresh every time — edit them while the gateway is running and the next event picks up changes. If `memory.md` doesn't exist, it's auto-created with `# Memory\n`. ### Step 6: Assemble System Prompt The markdown file contents are concatenated into a single system prompt with section headers. **File:** `src/system-prompt-assembler.ts` → `assemble()` The assembled prompt looks like this: ``` You may update your long-term memory by writing to memory.md using the Write tool. Use this to persist important facts, lessons learned, and context across sessions. ## Identity # Identity - **Name:** Aetheel - **Vibe:** Helpful, sharp, slightly witty - **Emoji:** ⚡ ## Personality # Soul Be genuinely helpful. Have opinions. Be resourceful before asking. Keep responses concise for Discord. Use markdown formatting. ## Operating Rules # Operating Rules Be helpful and concise. Keep Discord messages short. ## Cron Jobs ... ## User Context # User Context - **Name:** Tanmay - **Timezone:** IST ... ## Long-Term Memory # Memory - Tanmay prefers short responses - Project aetheel-2 is the Discord gateway ... ## Tool Configuration # Tool Configuration (empty or tool-specific notes) ``` Sections with null or empty content are omitted entirely. ### Step 7: Write System Prompt to Temp File The assembled system prompt is written to a temporary file because it can be thousands of characters — too large for a CLI argument. **File:** `src/agent-runtime.ts` → `processMessage()` ``` /tmp/aetheel-prompt-1d6c77f1-4a4e-49f8-ae9b-cff6fb47b971.txt ``` This file is deleted after the CLI process completes (used by the Claude backend; other backends may prepend the system prompt directly to the user prompt). ### Step 8: Spawn Backend CLI The gateway delegates to the configured **Backend Adapter**, which spawns the corresponding CLI as a child process. **File:** `src/backends/{claude,codex,gemini,opencode,pi}-backend.ts` → `spawnCli()` The backend is selected via the `AGENT_BACKEND` environment variable (default: `claude`). Each backend adapter translates the common interface into the CLI-specific flags. #### Example: Claude Code CLI (default) ```bash claude \ -p "what's the weather like?" \ --output-format json \ --dangerously-skip-permissions \ --append-system-prompt-file /tmp/aetheel-prompt-xxx.txt \ --allowedTools Read \ --allowedTools Write \ --allowedTools Edit \ --allowedTools Glob \ --allowedTools Grep \ --allowedTools WebSearch \ --allowedTools WebFetch \ --max-turns 25 ``` #### Example: Pi CLI ```bash pi \ -p "what's the weather like?" \ --mode json \ --append-system-prompt "" \ --no-session \ --no-extensions --no-skills --no-themes ``` #### Example: Codex CLI ```bash codex exec \ "" \ --json \ --dangerously-bypass-approvals-and-sandbox \ --cd /path/to/config ``` #### Example: OpenCode CLI ```bash opencode run \ "" \ --format json ``` #### Example: Gemini CLI ```bash gemini \ "" \ --output-format json \ --approval-mode yolo ``` Key flags (Claude): - `-p` — Print mode (non-interactive, exits after response) - `--output-format json` — Returns JSON array of message objects - `--dangerously-skip-permissions` — No interactive permission prompts - `--append-system-prompt-file` — Appends our persona/memory to Claude's default prompt - `--allowedTools` — Which tools Claude can use (one flag per tool; Claude-only feature) - `--max-turns` — Prevents runaway agent loops - `--resume SESSION_ID` — Added when resuming an existing conversation Other backends use equivalent flags for their CLIs (e.g., Pi uses `--mode json` and `--append-system-prompt`, Codex uses `--json` and `--dangerously-bypass-approvals-and-sandbox`). The process runs with `cwd` set to the `config/` directory, so Claude can read/write files there (like `memory.md`). `stdin` is set to `"ignore"` to prevent the CLI from waiting for interactive input. ### Step 9: Session Resumption If this channel has chatted before, the session manager has a stored session ID. **File:** `src/session-manager.ts` ``` config/sessions.json: { "1475008084022788312": "37336c32-73cb-4cf5-9771-1c8f694398ff" } ``` When a session ID exists, `--resume 37336c32-73cb-4cf5-9771-1c8f694398ff` is added to the CLI args. Claude loads the full conversation history from `~/.claude/` and continues the conversation. ### Step 10: Parse CLI Output (Streaming) The CLI returns a JSON array on stdout. The gateway parses it as chunks arrive. **File:** `src/backends/{backend}-backend.ts` → `spawnCli()` stdout handler Example CLI output (Claude): ```json [ { "type": "system", "subtype": "init", "session_id": "37336c32-73cb-4cf5-9771-1c8f694398ff", "tools": ["Read", "Write", "Edit", "Bash", "Glob", "Grep", "WebSearch", "WebFetch"] }, { "type": "assistant", "message": { "content": [{ "type": "text", "text": "Let me check..." }] } }, { "type": "result", "subtype": "success", "result": "I don't have access to real-time weather data, but I can help you check! Try asking me to search the web for current weather in your area.", "session_id": "37336c32-73cb-4cf5-9771-1c8f694398ff", "is_error": false, "cost_usd": 0.003 } ] ``` The parser extracts: 1. `session_id` from the `init` message → stored for future resumption 2. `result` from the `result` message → sent to Discord When streaming is active, result text is sent to Discord immediately as it's parsed, before the CLI process exits. ### Step 11: Response Formatting & Delivery The result text is split into Discord-safe chunks (max 2000 characters each). **File:** `src/response-formatter.ts` → `splitMessage()` If the response contains code blocks that span a split boundary, the formatter closes the code block with ` ``` ` at the end of one chunk and reopens it with ` ``` ` at the start of the next. The chunks are sent sequentially to the Discord channel via the bot. ### Step 12: Session Persistence The session ID is saved to `config/sessions.json` so it survives gateway restarts. **File:** `src/session-manager.ts` → `saveToDisk()` Next time the user sends a message in the same channel, the conversation continues from where it left off. --- ## Other Event Types ### Heartbeat Flow ``` Timer fires (every N seconds) → HeartbeatScheduler creates heartbeat event → Event enters queue → AgentRuntime reads config files, assembles prompt → CLI runs with heartbeat instruction as prompt → Response sent to OUTPUT_CHANNEL_ID ``` Example heartbeat.md: ```markdown ## check-email Interval: 1800 Instruction: Check my inbox for anything urgent. If nothing, reply HEARTBEAT_OK. ``` The instruction becomes the `-p` argument to the CLI. ### Cron Job Flow ``` Cron expression matches (e.g., "0 9 * * *" = 9am daily) → CronScheduler creates cron event → Event enters queue → AgentRuntime reads config files, assembles prompt → CLI runs with cron instruction as prompt → Response sent to OUTPUT_CHANNEL_ID ``` Cron jobs are defined in `config/agents.md`: ```markdown ## Cron Jobs ### morning-briefing Cron: 0 9 * * * Instruction: Good morning! Check email and give me a brief summary. ``` ### Hook Flow ``` Lifecycle event occurs (startup, shutdown) → HookManager creates hook event → Event enters queue → AgentRuntime reads config files, assembles prompt → CLI runs with hook instruction as prompt → Response sent to OUTPUT_CHANNEL_ID ``` Hooks are defined in `config/agents.md`: ```markdown ## Hooks ### startup Instruction: Say hello, you just came online. ### shutdown Instruction: Save important context to memory.md before shutting down. ``` `agent_begin` and `agent_stop` hooks fire inline (not through the queue) before and after every non-hook event. --- ## What Gets Sent to the Backend For every event, the backend CLI receives: 1. **Default system prompt** (built-in from the CLI — varies by backend) 2. **Appended system prompt** (from our assembled markdown files): - Identity (who the agent is) - Personality (how it behaves) - Operating rules (safety, workflows) - User context (who it's helping) - Long-term memory (persistent facts) - Tool configuration (API notes) - Preamble about writing to memory.md 3. **The prompt text** (user message, heartbeat instruction, or cron instruction) 4. **Session history** (if resuming via backend-specific session flags) 5. **Allowed tools** (Claude only: Read, Write, Edit, Glob, Grep, WebSearch, WebFetch) > **Note:** How the system prompt is delivered varies by backend. Claude Code uses `--append-system-prompt-file`, Pi uses `--append-system-prompt`, and other backends prepend it to the user prompt. The backend CLI runs in the `config/` directory, so the agent can read and write files there — including updating `memory.md` with new facts. --- ## File Map ``` src/ ├── index.ts ← Entry point: creates GatewayCore, registers shutdown handler ├── gateway-core.ts ← Orchestrator: wires everything, manages lifecycle ├── config.ts ← Reads env vars (DISCORD_BOT_TOKEN, etc.) ├── logger.ts ← Pino structured logger with pretty-printing ├── discord-bot.ts ← Discord.js wrapper: messages, slash commands, typing ├── event-queue.ts ← FIFO queue: all events (message, heartbeat, cron, hook) ├── agent-runtime.ts ← Core engine: reads configs, delegates to backend adapter ├── backends/ ← Pluggable CLI backend adapters │ ├── types.ts ← BackendAdapter interface & BackendName union type │ ├── registry.ts ← resolveBackendName() & createBackend() factory │ ├── index.ts ← Barrel exports │ ├── claude-backend.ts ← Claude Code CLI adapter │ ├── codex-backend.ts ← OpenAI Codex CLI adapter │ ├── gemini-backend.ts ← Google Gemini CLI adapter │ ├── opencode-backend.ts ← OpenCode CLI adapter │ └── pi-backend.ts ← Pi Coding Agent CLI adapter ├── markdown-config-loader.ts ← Reads config/*.md files fresh each event ├── system-prompt-assembler.ts ← Concatenates markdown into system prompt with headers ├── skills-loader.ts ← Loads skills from config/skills/*/SKILL.md ├── session-manager.ts ← Channel → session ID mapping (persisted to JSON) ├── message-history.ts ← Per-channel message storage ├── conversation-archiver.ts ← Markdown conversation logs ├── ipc-watcher.ts ← Polls ipc/outbound/ for proactive messages ├── response-formatter.ts ← Splits long text for Discord's 2000 char limit ├── error-formatter.ts ← Sanitizes errors (strips keys, paths, stacks) ├── heartbeat-scheduler.ts ← setInterval timers from heartbeat.md ├── cron-scheduler.ts ← node-cron jobs from agents.md ├── hook-manager.ts ← Lifecycle hooks from agents.md ├── bootstrap-manager.ts ← First-run: validates/creates config files ├── channel-queue.ts ← Per-channel sequential processing ├── shutdown-handler.ts ← SIGTERM/SIGINT → graceful shutdown ├── dashboard-server.ts ← Embedded HTTP server for Mission Control dashboard ├── dashboard-dev.ts ← Standalone dashboard dev server (mock data) ├── activity-log.ts ← In-memory + file-persisted activity event log └── local-store.ts ← JSON file persistence layer (brain, tasks, content) dashboard/ ← Mission Control frontend (served by dashboard-server) ├── index.html ← SPA shell with sidebar navigation ├── styles.css ← Dark-theme design system └── app.js ← Client-side routing, API calls, rendering config/ ├── CLAUDE.md ← Persona: identity, personality, user context, tools ├── agents.md ← Rules, cron jobs, hooks (parsed at startup) ├── heartbeat.md ← Heartbeat checks (parsed at startup) ├── memory.md ← Long-term memory (agent-writable, auto-created) ├── sessions.json ← Channel → session ID map (auto-generated) ├── brain.json ← Second Brain facts (auto, managed by dashboard) ├── tasks.json ← Productivity tasks (auto, managed by dashboard) ├── content-items.json ← Content Intel items (auto, managed by dashboard) ├── activity-log.json ← Persisted activity log (auto) ├── bot-config.json ← Dashboard bot config (auto) ├── messages/ ← Message history (auto) ├── conversations/ ← Conversation archives (auto) ├── ipc/outbound/ ← Proactive message queue (auto) └── skills/ ← Skill definitions ``` --- ## Dashboard Flow The Mission Control dashboard runs as an embedded HTTP server inside the gateway process. ### Real-Time Activity (SSE) ``` Agent Runtime processes event → ActivityLog.record() called → Entry added to in-memory buffer (capped at 200) → Entry persisted to config/activity-log.json (capped at 2000) → SSE broadcast to all connected dashboard clients → Dashboard UI updates live activity feed ``` ### Data Persistence Flow ``` Dashboard UI (browser) → POST /api/brain { content, type, category, tags } → DashboardServer routes to LocalStores → JsonStore.append() adds entry with generated ID → Debounced write to config/brain.json (300ms) → Response with updated data ``` All persistent stores use the same pattern: - **Read:** Load from JSON file on startup, serve from memory - **Write:** Append/update in memory, debounced flush to disk - **Shutdown:** `flushAll()` called to ensure all pending writes complete