A native macOS client for OpenClaw — a Jarvis-style AI agent with push-to-talk voice, a live "cognition" dashboard, a geo-mapped news monitor, and scheduled automation.
The agent itself (memory, tools, heartbeat, cron) runs on an OpenClaw Gateway — on this Mac or on an always-on machine like a Mac Mini reached over Tailscale. Athena is a thin, stateful-free client: close the lid, move networks, reopen — the agent never stopped.
Inspired by the Bailongma interface, rebuilt natively in SwiftUI, in English.
Chat — streaming responses, image/video/audio/file attachments, full session history synced from the gateway.
Voice — hold SPACE anywhere to talk, release to send, spoken reply. Typed messages get typed replies; only voice turns are spoken back. Pause as long as you like mid-sentence — the whole recording is transcribed on release.
Speech-to-text (choose in Settings → Voice):
| Engine | Runs where | Notes |
|---|---|---|
| Parakeet (default) | Local, embedded | Parakeet-TDT on the Apple Neural Engine — accurate on long dictation, tolerant of pauses. ~600 MB, one-time download. Nothing leaves your Mac. |
| Apple | Local | SFSpeechRecognizer, no download. Tuned for short commands; less accurate on long-form speech. |
Text-to-speech:
| Engine | Runs where | Notes |
|---|---|---|
| System | Local | macOS voices; download a Premium voice for a big quality jump |
| Kokoro | Local, embedded | Kokoro-82M on Apple Silicon — offline, no server, no Docker |
| CosyVoice 3 | Local, embedded | Emotional TTS with voice cloning from a recorded sample (macOS 15+) |
| Server | Remote | Any OpenAI-compatible /v1/audio/speech (kokoro-fastapi, CosyVoice) |
Embedded models download once into ~/Library/Application Support, persist across launches, and load automatically on startup — download progress and a retry are shown in Settings and during first-run setup.
Live cognition dashboard — a particle sphere that reacts to real microphone and speech amplitude, a heartbeat ECG that spikes on each agent wake, an action log, a thinking/tool feed, and a stats strip (status, tokens/sec, tokens, messages, ticks).
News monitor — a rotating dotted globe with pulsing targets where your stories are happening. Headlines are geo-tagged locally by keyword (no API, no tokens); click a target to see that region's stories. Topic columns flank the globe; RSS/Atom feeds are fetched natively. Add, rename, and delete topics and sources behind the gear. A daily AI brief can be scheduled on the gateway.
Jobs — full cron manager for gateway automations: create from templates or scratch with a plain-English schedule builder, edit, pause/resume, run-now, delete. Each job card shows its schedule, next/last run, and an expandable run history assembled from live gateway events. Jobs run on the gateway, so they fire while this Mac sleeps.
┌─ MacBook — Athena.app ──────────┐ ┌─ Mac Mini (24/7) ───────────┐
│ SwiftUI UI │ │ OpenClaw Gateway (launchd) │
│ • Chat, News, Jobs │ WS │ • Sessions + memory │
│ • Voice: local STT/TTS │◄──────►│ • Tools, skills │
│ • Globe, orb, dashboards │ │ • Heartbeat (self-wake) │
│ • No agent state │ │ • Cron (daily brief) │
└─────────────────────────────────┘ └─────────────────────────────┘
└── Tailscale (wss://mini.tailnet.ts.net) ──┘
All state lives on the gateway. Athena reconnects with exponential backoff, re-syncs history, and self-heals stale device tokens.
Athena/
├── App/ AthenaApp, AppState, MainView, Theme
├── Gateway/ WebSocket client, protocol v4, device identity, RPC wrappers
├── Chat/ ChatStore (streaming + history + turn watchdog), ChatView
├── Voice/ VoiceManager (capture + TTS), ParakeetSTT + AudioSampleBuffer
│ (embedded STT), KokoroEngine, CosyVoiceEngine, ModelDownloadRow,
│ ParticleSphereView, VoiceHUD, FileDownloader
├── News/ NewsStore, RSSFeed parser, GeoTagger, WorldGlobe, NewsView
├── Jobs/ JobsStore (cron + run history), JobsView, JobEditor, Schedule
├── Panels/ ActivityStore, dashboard panels
├── Setup/ First-run wizard, OpenClaw installer, ModelSetupStep
└── Settings/ Connection + voice/model settings
- macOS 15+, Apple Silicon (M1+) — embedded Parakeet/Kokoro/CosyVoice run on the Neural Engine / MLX
- Xcode 16+ (FluidAudio requires a Swift 6 toolchain to compile; the app targets Swift 5.9)
- XcodeGen —
brew install xcodegen - A running OpenClaw gateway (Athena can install one for you)
git clone <your-repo-url> Athena && cd Athena
./Scripts/fetch-kokoro.sh # optional: vendors the embedded TTS package
xcodegen
open Athena.xcodeproj # set your signing team, then ⌘ROr build and launch without Xcode:
./Scripts/run.sh # build + run
./Scripts/run.sh --watch # rebuild + relaunch on save (needs fswatch)The setup wizard offers two paths:
- Connect to existing OpenClaw — enter the gateway URL and token.
Get the token on the gateway machine:
openclaw config get gateway.auth.token - Install OpenClaw on this Mac — runs the official installer, starts the gateway, then drives OpenClaw's onboarding wizard natively over RPC (no terminal).
The first remote connection needs one-time device approval on the gateway
(openclaw devices, or the Control UI). Athena shows the instructions and
connects automatically once approved.
Run the agent on a machine that's always on; connect from anywhere over Tailscale.
On the gateway machine:
# Install OpenClaw + set a token
openclaw config set gateway.auth.token "$(openssl rand -hex 32)"
# Option A — Tailscale Serve (HTTPS, also gives you the web dashboard)
openclaw config set gateway.bind loopback
tailscale serve --bg --https=443 http://127.0.0.1:18789
# → Athena URL: wss://<machine>.<tailnet>.ts.net (no port)
# Option B — direct tailnet bind (simplest, no proxy)
openclaw config set gateway.bind tailnet
# → Athena URL: ws://100.x.y.z:18789Then disable sleep on that machine (System Settings → Energy / caffeinate).
Never port-forward 18789 to the public internet, and don't use Tailscale Funnel for the gateway — it exposes the agent (which has shell access) to the world. Serve is tailnet-only; that's what you want.
Scheme and port must match:
| Setup | URL |
|---|---|
| Tailscale Serve | wss://host.tailnet.ts.net — no port |
| Direct tailnet bind | ws://100.x.y.z:18789 |
| Local | ws://127.0.0.1:18789 |
wss://host:18789 fails — 18789 speaks plaintext.
Hard-won notes from getting this talking to a real gateway.
-1022 App Transport Security / "requires a secure connection"
macOS blocks plaintext ws:// to non-localhost. Info.plist sets
NSAllowsArbitraryLoads (safe here — Tailscale already encrypts end-to-end).
Note a ts.net exception domain does not work: ts.net is on the Public
Suffix List, so ATS ignores it. After changing the plist, clean the build
folder (⇧⌘K) — Xcode caches it.
-1004 Could not connect — nothing is listening on that address. The
gateway binds to loopback by default: lsof -iTCP:18789 -sTCP:LISTEN. Use
gateway.bind tailnet or Tailscale Serve.
-1011 bad response / HTTP 502 through Serve — the proxy mapping is
wrong. tailscale serve status should show https:443 → http://127.0.0.1:18789.
A stale Funnel config is a common culprit: tailscale funnel reset && tailscale serve reset,
then re-add. Note OpenClaw can manage Serve itself (gateway.tailscale.mode) —
pick one owner, not both.
Handshake stalls after connect.challenge — the client must not block its
receive loop while awaiting the connect response (this bit us; see
GatewayClient.handleFrame).
INVALID_REQUEST: invalid connect params — schema mismatch. Notably:
client.id/client.mode are enums (cli, gateway-client, openclaw-macos…),
signedAt must serialize as an integer (not 1.7e12), and this gateway
speaks protocol 4.
DEVICE_AUTH_SIGNATURE_INVALID — the v2 signature payload is
v2|deviceId|clientId|clientMode|role|scopes,csv|signedAtMs|token|nonce.
Verified against the Control UI's own client (dist/control-ui/assets/gateway-*.js);
re-check there if a future release changes it.
PAIRING_REQUIRED — approve the device on the gateway. Athena keeps
retrying and connects itself once approved. Requesting broader scopes triggers a
fresh approval.
Token works in the browser but not the probe — openclaw gateway probe --url
never uses config credentials; pass --token explicitly.
Kokoro download fails — it resumes and retries automatically; if it still fails, install manually (below).
"No audio is reaching the microphone" — the mic tap must be installed at
the device's hardware sample rate (AVAudioInputNode.inputFormat), not
outputFormat, which can report a stale 48 kHz while a Bluetooth headset
delivers 24 kHz. A mismatch makes AVAudioEngine silently fail to create the
tap. Athena reads the hardware format and, if the device is momentarily busy
(HAL error 35), stops and re-arms the tap once. The console logs
[voice] input format (hardware): … and [voice] first buffer: … — if the
second line never appears, another app is holding the mic.
Parakeet says "not downloaded" after it clearly downloaded — detection
scans both cache locations FluidAudio may use
(~/Library/Application Support/FluidAudio/Models and ~/.cache/fluidaudio)
at any nesting depth. Use Settings → Voice → Clear download to force a
clean re-fetch if a download was interrupted.
mkdir -p ~/Library/Application\ Support/Athena/Kokoro
cd ~/Library/Application\ Support/Athena/Kokoro
BASE=https://huggingface.co/mweinbach/Kokoro-82M-Swift/resolve/main/MLX_GPU
curl -L -C - -o config.json "$BASE/config.json"
curl -L -C - -o kokoro-v1_0.safetensors "$BASE/kokoro-v1_0.safetensors" # ~330MB-C - resumes; rerun if it drops. Then Settings → Voice → Kokoro → Load model.
Athena is a display layer. Anything that looks like intelligence belongs on the gateway, not in Swift. On first connect (and via Settings → Agent → Sync) the app provisions the agent so the backend understands the contract:
AGENTS.md— an operating manual: what Athena is, how replies are rendered, the[voice]convention and speakable-reply rules, thenews/YYYY-MM-DD.mdarchive format, and the instruction to summarize from the archive rather than re-fetching.HEARTBEAT.md— atasks:block so hourly wakes sweep the feeds.- Cron jobs — hourly collector, daily brief.
Managed content is fenced between <!-- ATHENA:BEGIN --> / <!-- ATHENA:END -->
markers, so hand-written additions to those files survive re-syncs.
On connect (once per launch) Athena reads athena/manifest.json from the
workspace. If the contract version and topic fingerprint match, nothing is
sent — the agent already knows its job and normal messages carry no overhead.
Only a missing/stale manifest triggers the setup write. Changing topics or
bumping AgentProvisioner.contractVersion invalidates it automatically.
The agent writes news/YYYY-MM-DD.md; the UI reads those same files back via
agents.files.get and parses them into the dashboard. That's how stories
collected while your Mac was asleep appear when you open the app. Because the
UI parses it, the manual states plainly that the format is a contract.
Because the rules live server-side, voice turns only need a [voice] marker
instead of restating the whole directive on every message. If provisioning
hasn't run, the app falls back to sending the full inline directive.
Re-run Sync after changing topics, sources, or the brief time — the manual embeds them.
The UI must never block. Anything that can take more than a few milliseconds runs off the main actor:
| Work | Where it runs |
|---|---|
| Kokoro model load + inference + WAV encoding | KokoroRunner (actor) |
| Reading attachment files | ChatAttachment.load (detached task) |
| Pasted-image → PNG conversion | ChatAttachment.fromPasteboardImage |
Base64 + JSON encoding of chat.send |
detached task in GatewayRPC |
| RSS fetch + XML parsing | task group, non-isolated parser |
| Model weight download | URLSessionDownloadTask delegate queue |
The main actor keeps only: SwiftUI state, NSPasteboard reads (AppKit
requirement — but just a buffer copy), and WebSocket frame dispatch.
When adding a feature, ask what it does with a 50MB video or a 2000-word reply.
If the answer involves a loop over bytes or a synchronous file read, wrap it in
Task.detached or move it into an actor, and surface progress in the UI.
- Code-only changes: just ⌘R.
xcodegenis only needed when files are added/removed orproject.ymlchanges. - Skip xcodegen entirely (Xcode 16+): replace the
Athenagroup with a synchronized/buildable folder, commitAthena.xcodeproj, dropproject.yml. - UI iteration: SwiftUI Previews render without launching the app or reconnecting to the gateway.
- Gateway debugging: every frame is logged as
[gateway] ←/→in the Xcode console.
- Menu-bar mini mode + global hotkey
- Wake word ("Hey Athena") via
voicewake.*RPC - Native dialogs for exec-approval requests
- Real token accounting from the gateway's
usage.*RPCs - Notifications when the daily brief lands
- Streaming token-by-token rendering
- Richer globe: arcs between related stories, day/night terminator
- OpenClaw — agent runtime and gateway (the actual brains)
- Bailongma — interface inspiration
- FluidAudio — embedded Parakeet-TDT speech-to-text on the Neural Engine
- Kokoro-82M by hexgrad, via kokoro-swift — embedded TTS
- CosyVoice — emotional TTS with voice cloning
- Apple: SwiftUI, Speech, AVFoundation, MLX
MIT — see LICENSE.