feat: openclaw-style secrets (env.vars + \) and per-task model routing

- Replace python-dotenv with config.json env.vars block + \ substitution - Add models section for per-task model routing (heartbeat, subagent, default) - Heartbeat/subagent tasks can use different models/providers than main chat - Remove python-dotenv from dependencies - Update all docs to reflect new config approach - Reorganize docs into project/ and research/ subdirectories
2026-02-20 23:49:05 -05:00
parent 55c6767e69
commit 82c2640481
35 changed files with 2904 additions and 422 deletions
--- a/docs/project/spec-phase3-features.md
+++ b/docs/project/spec-phase3-features.md
@@ -0,0 +1,560 @@
+# Aetheel Phase 3 Feature Spec
+
+> Specification for implementing heartbeat, CLI, webchat, self-modification, agent-to-agent communication, and tool/MCP/web search integration.
+
+---
+
+## Table of Contents
+
+1. [Runtime Capabilities Audit](#1-runtime-capabilities-audit)
+2. [Tool System, MCP & Web Search](#2-tool-system-mcp--web-search)
+3. [Heartbeat / Proactive System](#3-heartbeat--proactive-system)
+4. [CLI Interface](#4-cli-interface)
+5. [WebChat Interface](#5-webchat-interface)
+6. [Self-Modification](#6-self-modification)
+7. [Agent-to-Agent Communication](#7-agent-to-agent-communication)
+8. [Implementation Order](#8-implementation-order)
+
+---
+
+## 1. Runtime Capabilities Audit
+
+Before building anything, here's what OpenCode and Claude Code already provide natively that Aetheel can leverage without custom implementation.
+
+### OpenCode Built-in Tools
+
+OpenCode ships with these tools enabled by default (no config needed):
+
+| Tool | Description | Aetheel Status |
+|------|-------------|----------------|
+| `bash` | Execute shell commands | ✅ Available via runtime |
+| `read` | Read file contents | ✅ Available via runtime |
+| `write` | Create/overwrite files | ✅ Available via runtime |
+| `edit` | Modify files via string replacement | ✅ Available via runtime |
+| `grep` | Regex search across files | ✅ Available via runtime |
+| `glob` | Find files by pattern | ✅ Available via runtime |
+| `list` | List directory contents | ✅ Available via runtime |
+| `websearch` | Web search via Exa AI (no API key needed) | ✅ Available — no setup required |
+| `webfetch` | Fetch and read web pages | ✅ Available via runtime |
+| `skill` | Load SKILL.md files | ✅ Available via runtime |
+| `todowrite` / `todoread` | Task tracking | ✅ Available via runtime |
+| `lsp` | Code intelligence (experimental) | ✅ Available via runtime |
+| `patch` | Apply diffs | ✅ Available via runtime |
+| Custom tools | User-defined JS/TS tools | 🟡 Available but not wired |
+| MCP servers | External tool servers | 🟡 Available but not configured |
+
+### Claude Code Built-in Tools
+
+Claude Code provides these tools natively:
+
+| Tool | Description | Aetheel Status |
+|------|-------------|----------------|
+| `Bash` | Shell execution | ✅ Available via runtime |
+| `Read`, `Write`, `Edit` | File operations | ✅ Available via runtime |
+| `Glob`, `Grep` | File search | ✅ Available via runtime |
+| `WebSearch`, `WebFetch` | Web access | ✅ Available via runtime |
+| `Task`, `TaskOutput`, `TaskStop` | Subagent spawning | ✅ Available via runtime |
+| `TeamCreate`, `TeamDelete`, `SendMessage` | Agent teams | ✅ Available via runtime |
+| `Skill` | Load skills | ✅ Available via runtime |
+| MCP servers | Via `.mcp.json` config | 🟡 Available but not configured |
+
+### Key Insight
+
+Both runtimes already have web search, file ops, bash, and subagent tools built in. Aetheel doesn't need to implement these — it just needs to stop blocking them. Currently Aetheel's system prompt doesn't tell the agent about these capabilities, and Claude Code's `--allowedTools ""` (no_tools=true) actively disables them.
+
+---
+
+## 2. Tool System, MCP & Web Search
+
+### What Needs to Change
+
+**Problem**: Aetheel currently runs both runtimes in "pure chat" mode — tools are disabled or not mentioned in the system prompt.
+
+**Solution**: Enable tool access and configure MCP servers.
+
+### 2.1 Enable Runtime Tools
+
+For OpenCode (CLI mode), tools are enabled by default. No change needed.
+
+For OpenCode (SDK mode), tools are enabled by default on the server. No change needed.
+
+For Claude Code, change `no_tools` default from `true` to `false` in config, and set a sensible `allowedTools` list:
+
+```python
+# config.py — ClaudeConfig
+@dataclass
+class ClaudeConfig:
+    no_tools: bool = False  # Changed from True
+    allowed_tools: list[str] = field(default_factory=lambda: [
+        "Bash", "Read", "Write", "Edit", "Glob", "Grep",
+        "WebSearch", "WebFetch",
+        "Task", "Skill",
+    ])
+```
+
+### 2.2 Update System Prompt
+
+Add tool awareness to `build_aetheel_system_prompt()`:
+
+```
+# Your Tools
+- You have access to shell commands, file operations, and web search
+- Use web search to look up current information when needed
+- You can read and write files in the workspace (~/.aetheel/workspace/)
+- You can execute shell commands for system tasks
+```
+
+### 2.3 MCP Server Configuration
+
+Create an `opencode.json` in the workspace for OpenCode MCP config, or a `.mcp.json` for Claude Code. This lets users add external tool servers.
+
+Add to `~/.aetheel/config.json`:
+
+```json
+{
+  "mcp": {
+    "servers": {
+      "example": {
+        "command": "uvx",
+        "args": ["some-mcp-server"],
+        "env": {}
+      }
+    }
+  }
+}
+```
+
+Aetheel writes this to the appropriate config file (`opencode.json` or `.mcp.json`) in the workspace directory before launching the runtime.
+
+### 2.4 Implementation Tasks
+
+1. Change `ClaudeConfig.no_tools` default to `False`
+2. Add `allowed_tools` to `ClaudeConfig` with sensible defaults
+3. Update `_build_cli_args()` in `claude_runtime.py` to pass `--allowedTools` from config
+4. Update `build_aetheel_system_prompt()` to mention available tools
+5. Add `mcp` section to `config.py` and `config.json`
+6. Write MCP config files to workspace before runtime launch
+7. Add `TOOLS.md` to workspace identity files (like PicoClaw/OpenClaw)
+
+### Inspiration
+
+- **NanoClaw**: Passes `allowedTools` list to `query()` call, runs custom MCP server for IPC
+- **PicoClaw**: Built-in tool registry in `pkg/tools/`, `TOOLS.md` describes tools to agent
+- **OpenClaw**: Tool registry with streaming, `TOOLS.md` in workspace
+
+---
+
+## 3. Heartbeat / Proactive System
+
+### How Inspiration Repos Do It
+
+**PicoClaw**: Reads `HEARTBEAT.md` every 30 minutes. Quick tasks run directly, long tasks spawn subagents. The heartbeat file contains prompts like "Check if any cron jobs need attention" or "Review MEMORY.md for stale entries."
+
+**Nanobot**: Has a `heartbeat/` module that triggers periodic agent runs.
+
+**NanoClaw**: Uses scheduled tasks via MCP tools — the agent itself schedules recurring tasks using `schedule_task` with cron expressions. No separate heartbeat file.
+
+**OpenClaw**: Cron system + wakeups. Heartbeat runs are suppressed from WebChat broadcast when `showOk: false`.
+
+### Design for Aetheel
+
+Combine PicoClaw's `HEARTBEAT.md` approach (simple, file-based) with Aetheel's existing scheduler:
+
+### 3.1 HEARTBEAT.md
+
+Create `~/.aetheel/workspace/HEARTBEAT.md` as a user-editable file:
+
+```markdown
+# Heartbeat Tasks
+
+Tasks that run periodically. Each section is a task prompt.
+
+## Every 30 minutes
+- Check if any scheduled reminders need attention
+- Review recent session logs for anything worth remembering
+
+## Every morning (9:00 AM)
+- Summarize yesterday's conversations
+- Check for any pending follow-ups in MEMORY.md
+
+## Every evening (6:00 PM)
+- Update MEMORY.md with today's key learnings
+```
+
+### 3.2 Heartbeat Runner
+
+New module: `heartbeat/heartbeat.py`
+
+```python
+class HeartbeatRunner:
+    def __init__(self, runtime, send_fn, workspace_dir, scheduler):
+        ...
+
+    def start(self):
+        """Register heartbeat tasks with the scheduler."""
+        # Parse HEARTBEAT.md
+        # Register cron jobs for each section
+        # Jobs route through ai_handler with synthetic messages
+
+    def _parse_heartbeat_md(self) -> list[HeartbeatTask]:
+        """Parse HEARTBEAT.md into structured tasks."""
+        ...
+
+    def _on_heartbeat(self, task: HeartbeatTask):
+        """Called when a heartbeat fires. Routes to AI handler."""
+        ...
+```
+
+### 3.3 Integration
+
+- Parse `HEARTBEAT.md` at startup, register tasks with existing `Scheduler`
+- Heartbeat tasks create synthetic `IncomingMessage` with `source="heartbeat"`
+- Results can be sent to a configured channel or logged silently
+- Add `heartbeat` section to config:
+
+```json
+{
+  "heartbeat": {
+    "enabled": true,
+    "default_channel": "slack",
+    "default_channel_id": "C123456",
+    "silent": false
+  }
+}
+```
+
+### 3.4 Implementation Tasks
+
+1. Create `heartbeat/` module with `HeartbeatRunner`
+2. Create default `HEARTBEAT.md` in workspace (like SOUL.md bootstrap)
+3. Parse markdown sections into cron expressions + prompts
+4. Register with existing `Scheduler` at startup
+5. Add heartbeat config section
+6. Wire into `main.py` initialization
+
+---
+
+## 4. CLI Interface
+
+### How Inspiration Repos Do It
+
+**Nanobot**: `nanobot agent -m "..."`, `nanobot gateway`, `nanobot status`, `nanobot cron add/list/remove`
+
+**OpenClaw**: `openclaw gateway`, `openclaw agent --message "..."`, `openclaw doctor`, `openclaw pairing approve`
+
+**PicoClaw**: Binary with subcommands via Go's `cobra` library
+
+### Design for Aetheel
+
+Use Python's `click` library for a clean CLI with subcommands.
+
+### 4.1 CLI Structure
+
+```
+aetheel                     # Start with default adapters (same as current main.py)
+aetheel start               # Start with all configured adapters
+aetheel start --discord     # Start with specific adapters
+aetheel chat "message"      # One-shot chat (no adapter, just AI)
+aetheel status              # Show runtime status
+aetheel cron list           # List scheduled jobs
+aetheel cron remove <id>    # Remove a job
+aetheel sessions list       # List active sessions
+aetheel sessions clear      # Clear stale sessions
+aetheel config show         # Print current config
+aetheel config edit         # Open config in $EDITOR
+aetheel config init         # Reset to defaults
+aetheel memory search "q"   # Search memory
+aetheel memory sync         # Force memory re-index
+aetheel doctor              # Diagnostics (check runtime, tokens, etc.)
+```
+
+### 4.2 Implementation
+
+New file: `cli.py`
+
+```python
+import click
+
+@click.group()
+def cli():
+    """Aetheel — AI-Powered Personal Assistant"""
+    pass
+
+@cli.command()
+@click.option("--discord", is_flag=True)
+@click.option("--telegram", is_flag=True)
+@click.option("--claude", is_flag=True)
+@click.option("--model", default=None)
+@click.option("--test", is_flag=True)
+@click.option("--log", default="INFO")
+def start(discord, telegram, claude, model, test, log):
+    """Start Aetheel with configured adapters."""
+    # Current main() logic moves here
+    ...
+
+@cli.command()
+@click.argument("message")
+def chat(message):
+    """One-shot chat with the AI (no adapter needed)."""
+    ...
+
+@cli.group()
+def cron():
+    """Manage scheduled jobs."""
+    pass
+
+@cron.command("list")
+def cron_list():
+    ...
+
+@cron.command("remove")
+@click.argument("job_id")
+def cron_remove(job_id):
+    ...
+
+# ... etc
+```
+
+### 4.3 Entry Point
+
+Add to `pyproject.toml`:
+
+```toml
+[project.scripts]
+aetheel = "cli:cli"
+```
+
+### 4.4 Implementation Tasks
+
+1. Add `click` dependency to `pyproject.toml`
+2. Create `cli.py` with command groups
+3. Move `main()` logic into `start` command
+4. Add `chat`, `status`, `cron`, `sessions`, `config`, `memory`, `doctor` commands
+5. Add entry point to `pyproject.toml`
+6. Keep `main.py` as backward-compatible wrapper
+
+---
+
+## 5. WebChat Interface
+
+### How Inspiration Repos Do It
+
+**OpenClaw**: Full WebSocket-based WebChat as an internal channel. Messages routed through the gateway with `messageChannel: "webchat"`. Has a web UI built with React.
+
+**Nanobot/NanoClaw/PicoClaw**: No webchat.
+
+### Design for Aetheel
+
+A lightweight HTTP server with WebSocket support, served as a new adapter.
+
+### 5.1 Architecture
+
+```
+Browser ←→ WebSocket ←→ WebChatAdapter ←→ ai_handler ←→ Runtime
+                ↕
+         Static HTML/JS
+```
+
+### 5.2 Implementation
+
+New adapter: `adapters/webchat_adapter.py`
+
+Use `aiohttp` for both the static file server and WebSocket handler:
+
+- `GET /` — Serves the chat UI (single HTML file with embedded JS/CSS)
+- `WS /ws` — WebSocket for real-time chat
+- Each WebSocket connection = one conversation (session isolation)
+- Messages flow through the same `BaseAdapter._dispatch` → `ai_handler` path
+
+### 5.3 Chat UI
+
+Single self-contained HTML file at `static/chat.html`:
+- Minimal chat interface (message list + input box)
+- WebSocket connection to the local server
+- Markdown rendering for AI responses
+- No build step, no npm, no framework — just vanilla HTML/JS/CSS
+
+### 5.4 Config
+
+```json
+{
+  "webchat": {
+    "enabled": false,
+    "port": 8080,
+    "host": "127.0.0.1"
+  }
+}
+```
+
+### 5.5 Implementation Tasks
+
+1. Add `aiohttp` dependency
+2. Create `adapters/webchat_adapter.py` extending `BaseAdapter`
+3. Create `static/chat.html` — self-contained chat UI
+4. Add webchat config section
+5. Wire into `main.py` / CLI `start` command with `--webchat` flag
+6. Add to `_adapters` dict like other adapters
+
+---
+
+## 6. Self-Modification
+
+### Can the Runtime Do This?
+
+**Yes.** Both OpenCode and Claude Code have `Write`, `Edit`, and `Bash` tools that can modify any file the process has access to. The agent can already:
+
+- Edit `~/.aetheel/config.json` (via file write tools)
+- Create new skill files in `~/.aetheel/workspace/skills/<name>/SKILL.md`
+- Update `SOUL.md`, `USER.md`, `MEMORY.md`
+- Modify `HEARTBEAT.md` to add/remove periodic tasks
+
+### What Aetheel Needs to Do
+
+The agent just needs to be told it can do this. Update the system prompt:
+
+```
+# Self-Modification
+- You can edit your own config at ~/.aetheel/config.json
+- You can create new skills by writing SKILL.md files to ~/.aetheel/workspace/skills/<name>/SKILL.md
+- You can update your identity files (SOUL.md, USER.md, MEMORY.md)
+- You can modify HEARTBEAT.md to change your periodic tasks
+- After editing config, tell the user to restart for changes to take effect
+- After adding a skill, it will be available on next restart (or use /reload if implemented)
+```
+
+### 6.1 Hot Reload (Optional Enhancement)
+
+Add a `/reload` command that re-reads config and skills without restart:
+
+```python
+if text_lower in ("reload", "/reload"):
+    cfg = load_config()
+    _skills.reload()
+    return "🔄 Config and skills reloaded."
+```
+
+### 6.2 Implementation Tasks
+
+1. Update system prompt to mention self-modification capabilities
+2. Ensure tools are enabled (see section 2)
+3. Add `/reload` command to `ai_handler`
+4. Add workspace path to system prompt so agent knows where files are
+
+---
+
+## 7. Agent-to-Agent Communication
+
+### Can the Runtime Do This?
+
+**OpenCode**: Has `Task` tool for spawning subagents. Subagents run in child sessions. No direct inter-session messaging.
+
+**Claude Code**: Has `Task`, `TaskOutput`, `TaskStop` for subagent management, plus `TeamCreate`, `TeamDelete`, `SendMessage` for agent teams. `SendMessage` allows agents to communicate within a team.
+
+### How Inspiration Repos Do It
+
+**NanoClaw**: Uses Claude Code's `TeamCreate` + `SendMessage` tools. The `allowedTools` list includes both. Agent teams allow multiple agents to work together with message passing.
+
+**OpenClaw**: `sessions_send` tool allows one session to send a message to another session. Supports fire-and-forget or wait-for-reply modes. Visibility config controls which sessions can see each other.
+
+### Design for Aetheel
+
+Aetheel already has `SubagentManager` for spawning background tasks. What's missing is the ability for subagents to communicate back to the main agent or to each other.
+
+### 7.1 Approach: Leverage Runtime's Native Tools
+
+Since both OpenCode and Claude Code have subagent/team tools built in, the simplest approach is to enable them:
+
+For Claude Code, add to `allowed_tools`:
+```python
+["Task", "TaskOutput", "TaskStop", "TeamCreate", "TeamDelete", "SendMessage"]
+```
+
+For OpenCode, these are enabled by default.
+
+### 7.2 Aetheel-Level Inter-Subagent Messaging
+
+For Aetheel's own `SubagentManager`, add a message bus:
+
+```python
+class SubagentBus:
+    """Simple pub/sub for subagent communication."""
+
+    def __init__(self):
+        self._channels: dict[str, list[Callable]] = {}
+
+    def subscribe(self, channel: str, callback: Callable):
+        self._channels.setdefault(channel, []).append(callback)
+
+    def publish(self, channel: str, message: str, sender: str):
+        for cb in self._channels.get(channel, []):
+            cb(message, sender)
+```
+
+Wire into `SubagentManager` so subagents can:
+- Publish results to a channel
+- Subscribe to messages from other subagents
+- Send messages to the originating user via `_send_fn`
+
+### 7.3 Implementation Tasks
+
+1. Enable `TeamCreate`, `SendMessage`, `Task` tools in Claude Code config
+2. Update system prompt to mention subagent capabilities
+3. Add `SubagentBus` to `agent/subagent.py`
+4. Wire bus into `SubagentManager.spawn()` so subagents can communicate
+5. Add `/subagents` command to list active subagents and their status
+
+---
+
+## 8. Implementation Order
+
+Ordered by dependency and impact:
+
+### Phase 3A: Enable What Already Exists (1-2 days)
+1. **Tool enablement** — Change `no_tools` default, update allowed tools, update system prompt
+2. **Self-modification** — Just system prompt changes + `/reload` command
+3. **Agent-to-agent** — Enable runtime tools + system prompt
+
+### Phase 3B: New Modules (3-5 days)
+4. **CLI interface** — `cli.py` with click, move main() logic
+5. **Heartbeat system** — `heartbeat/` module, `HEARTBEAT.md`, scheduler integration
+
+### Phase 3C: WebChat (3-5 days)
+6. **WebChat adapter** — aiohttp server, WebSocket handler, static HTML UI
+
+### Dependencies
+
+```
+Tool enablement ──→ Self-modification (needs tools enabled)
+                ──→ Agent-to-agent (needs tools enabled)
+                ──→ Heartbeat (agent needs tools for heartbeat tasks)
+
+CLI ──→ WebChat (webchat flag in CLI)
+
+Heartbeat ──→ (standalone, uses existing scheduler)
+```
+
+### New Dependencies to Add
+
+```toml
+[project.dependencies]
+# ... existing ...
+click = ">=8.1.0"
+aiohttp = ">=3.9.0"
+```
+
+### New Files
+
+```
+Aetheel/
+├── cli.py                          # CLI entry point
+├── heartbeat/
+│   ├── __init__.py
+│   └── heartbeat.py                # HeartbeatRunner
+├── adapters/
+│   └── webchat_adapter.py          # WebChat adapter
+├── static/
+│   └── chat.html                   # WebChat UI
+└── ~/.aetheel/workspace/
+    ├── HEARTBEAT.md                # Periodic task prompts
+    └── TOOLS.md                    # Tool descriptions for agent
+```