Skip to content

plan.md

Latest commit

 

History

History
306 lines (230 loc) · 45.1 KB

plan.md

File metadata and controls

306 lines (230 loc) · 45.1 KB
Raw

mac-control — remote control for classic Mac OS (System 7.0-7.5)

Context

Drive a classic Mac (System 7.0-7.5) from a modern host — both real 68k/PPC hardware and emulators, initially targeting your existing MAME setup — for screen viewing, file transfer, scripting classic apps via Apple Events, and (later) reboot / power management. Workspace at /Users/nick/code/mac-control is empty; greenfield.

Surface the whole thing as an MCP server so an AI agent can drive it, with classic-Mac reference docs (Inside Macintosh, Apple Event registries) delivered through a separate, existing MCP docs server you already run — treated here as an assumed dependency, not something we build.

Load-bearing constraint: Remote Apple Events over TCP (eppc://, port 3031) was introduced in Mac OS 9, not System 7. In 7.x, remote Apple Events ride PPC Toolbox over AppleTalk; no mature Linux/macOS library speaks that. Scripting from the host therefore requires a small on-guest bridge that listens on TCP and dispatches Apple Events locally.

Approach

Two pieces:

  • mc-agent — a custom native classic-Mac application we build ourselves, cross-compiled from a modern host using Retro68 (GCC-based, targets 68k and PowerPC classic Mac OS). Background-only faceless app. Listens on a TCP port, speaks a small framed-JSON protocol, dispatches Apple Events locally via the Apple Event Manager, manages files (data + resource forks), captures screen contents via QuickDraw, synthesizes input via posted events plus interrupt-time cursor/button-state updates, runs jobs. Shipped as two separate binaries — mc-agent-68k and mc-agent-ppc — not a fat binary; a Mac Plus with 4 MB RAM and a full hard disk shouldn't pay for PowerPC code it can't run. No MiniVNC, no RFB server, no second binary at all on the guest — everything the controller needs is in one place.
  • mac-controld — modern-host daemon in Rust, single static binary. Speaks framed JSON over TCP to mc-agent for everything on the guest — script, files, screen capture, input. Also handles subprocess IPC for emulators and (later) smart-plug HTTP for power. Exposes MCP tools; relies on your existing docs MCP server for reference material rather than embedding its own.

Everything rides one TCP link. No AFP, no AppleShare, no FTP, no Netatalk sidecar. Files are chunked over the mc-agent JSON protocol in either direction — small transfers inline, large transfers as a stream of framed chunks. This means the guest needs exactly one thing: a TCP stack (MacTCP or Open Transport). AppleTalk-only guests are out of scope.

┌─ modern host ────────────────────────────────────────┐
│  ┌─ MCP: mac-control ──────┐  ┌─ MCP: docs (yours) ┐ │
│  │  screen.*   fs.*        │  │  inside-mac / aete │ │
│  │  applescript.*  app.*   │  │  reference         │ │
│  │  dialog.*   clipboard.* │  │  (assumed dep)     │ │
│  │  menu.*     process.*   │  └────────────────────┘ │
│  │  volume.*   log.*       │                         │
│  │  power.status  ...      │                         │
│  └───────────┬─────────────┘                         │
│              │                                       │
│  ┌─ mac-controld (Rust) ────────────────────────┐    │
│  │  target registry → transport → drivers       │    │
│  │  ┌─mc-agent-client─┐ ┌─emu─┐ ┌─power─┐       │    │
│  │  │  screen  input  │ │MAME │ │ later │       │    │
│  │  │  script  fs     │ │IPC  │ │       │       │    │
│  │  └────────┬────────┘ └──┬──┘ └───┬───┘       │    │
│  └───────────┼─────────────┼────────┼───────────┘    │
└──────────────┼─────────────┼────────┼────────────────┘
               │ mc-agent    │ emu    │
               │ TCP  4780   │ IPC    │ (plug: later)
               ▼             ▼        ▼
┌─ classic Mac (System 7.0-7.5) ───────────────────────┐
│                     ┌─ mc-agent ──────────┐          │
│                     │  native app         │          │
│                     │  (Retro68, C)       │          │
│                     │  ─ TCP + JSON       │          │
│                     │  ─ Apple Events     │          │
│                     │  ─ QuickDraw capture│          │
│                     │  ─ Input synth      │          │
│                     │  ─ Files (chunked)  │          │
│                     └──────────┬──────────┘          │
│                                │                     │
│                                ▼                     │
│      Finder + [app list TBD after evaluation]        │
└──────────────────────────────────────────────────────┘

The MCP tool list above is the aggregate v1 surface. power is power.status only; reboot / hard power off are deferred to M5.

On-guest agent (mc-agent)

What it is. A background-only classic Mac app (SIZE resource with onlyBackground set; never calls InitWindows, displays no windows or menu bar, not listed in the Application menu), built with Retro68 on the host as two separate binaries — mc-agent-68k (68k, tested down to 68000 / Mac Plus) and mc-agent-ppc (PowerPC, tested on 601). One INIT-style install: the installer applet detects the CPU via Gestalt(gestaltSysArchitecture) (68k vs PPC), falling back to gestaltProcessorType / SysEnvirons on 7.0/7.1 where newer selectors are absent (gestaltNativeCPUtype is not guaranteed pre-7.5), drops the matching binary into System Folder:Startup Items:, and deletes the other. No MacHTTP, no CGIs, no scripting-additions dance.

Keep it small. No hard byte ceiling, but hold the line on bloat — the target audience includes 4 MB Mac Pluses. Concretely: written in C, not C++ (no libstdc++, no RTTI, no exceptions, no iostreams); no dynamic linking beyond the System; hand-rolled minimal JSON parser rather than a full serialization library; no debug symbols in the shipped .bin. Track binary sizes in CI as a reported metric so regressions are visible, but don't gate merges on a specific number. New methods have to earn their footprint — features that only help niche workflows can be optional or omitted.

Requirements on the guest: System 7.0 or later with any TCP stack — MacTCP or Open Transport. Not OT-only. The agent uses a small internal socket shim (BSD-ish surface over MacTCP's Device-Manager API on one side, OT's endpoint API on the other; a couple hundred lines each) so the same source builds work on 7.0/7.1 (MacTCP) through 7.5.3+ (OT). AppleScript is optional — apple_event.send works without it; applescript.eval needs AppleScript installed (ships in 7.5, add-on for 7.0/7.1).

Transport. Persistent TCP connection, length-prefixed frames. Every frame has a 5-byte header: 1 byte type + 4 bytes big-endian length, then payload.

  • Type 0x01 = JSON. Request {id, method, params}, response {id, ok, result | error}, or server-initiated {event, payload} notification. UTF-8.
  • Type 0x02 = Binary attachment. First 4 bytes of the payload are the request/response id (big-endian), remainder is raw bytes. Correlates to a JSON response that has {"$binary": true, "size": N} in the value's place — the caller reads the binary frame that follows on the same connection and substitutes it in.

This means a PICT screenshot ships as (a) small JSON envelope + (b) one raw binary frame. No base64. mc-agent writes OpenPicture/CopyBits/ClosePicture output straight to the socket after the header — the classic Mac never touches every byte.

Bearer token in the first hello message. Single-client only in v1 — a second inbound connection is refused with a busy error until the first disconnects. Classic Mac cooperative scheduling makes concurrent-client semantics a footgun (two clients racing on focus, input synthesis, jobs); revisit if a real use case appears.

Notifications vs responses. Response frames carry the request id. Server-initiated notifications carry event instead of id — that's how the client distinguishes them. Binary frames (type 0x02) always carry the correlating response id in their header; there are no unsolicited binary frames.

Jobs. Methods that can exceed the per-handler time budget return {job_id, status:"running"} synchronously; results arrive later as job.done notifications (payload = the eventual result envelope) or, if the caller subscribed, job.progress events during execution. jobs.status {job_id} polls; jobs.cancel {job_id} requests a clean stop (the job checks a cancel flag at each yield). One job per method call; no job dependencies or fan-out in v1.

Methods (minimum useful set — grows with milestones):

Method Purpose
hello {token, client_id, protocol_version} → {server_version, sys_version, machine, cpu, ram, update_state:"clean"|"staged", last_crash_breadcrumb?}
health cheap poll: {uptime, free_mem, last_activity_at} — for heartbeat / power.status. Deeper introspection is on system.info and process.list
applescript.eval {source, timeout_ms} → {ok, result, error:{number, message, line}}; uses OSA to compile+run
apple_event.send raw AE dispatch: `{target:{sig
fs.list fs.stat fs.get fs.put fs.delete fs.move files with resource-fork awareness; fs.get can return data fork, resource fork, or MacBinary III
screen.capture {region?:{x,y,w,h}, depth?} → PICT of the full screen (equivalent to Cmd-Shift-3) or a sub-rectangle, via CopyBits from the main screen device (raw bytes framed on the wire). PICT only — cheap to produce with OpenPicture/CopyBits/ClosePicture, no encoder library in the agent. The agent ships pixels only: window, dialog, and menu semantics live on the controller (OCR + vision). Capture always shows what's on screen; to see an occluded window's true contents, bring its app forward with process.front first and yield for the redraw
screen.input {events:[{type:"click"|"drag"|"key"|"type", x?, y?, path?, keys?, modifiers?, ...}]} in screen coordinates — compiled in the main loop into a pre-built low-level script, then executed step-by-step by an interrupt-time task: cursor moves via the Cursor Device Manager (CrsrDevMoveTo, trap $AADB) where present, else the low-mem globals RawMouse/MTemp/CrsrNew ($82C/$828/$8CE); button state via MBState ($172); keystrokes via PostEvent. Interrupt-time execution is what lets gestures continue while the front app runs a tight tracking loop (menus, drags). Fire-and-report — the controller verifies effects via screen.capture
clipboard.get {flavors:["TEXT","PICT","styl",...]} → {flavor, bytes} for the first available. TEXT for plain text, PICT for images, styl for styled text runs. Reads via Scrap Manager. Staleness caveat: apps with a private scrap only publish to the global scrap on suspend — data copied in the front app may not be visible until that app is suspended; the controller can force this by bouncing focus via process.front
clipboard.set {flavor, bytes} → puts data on the scrap; multiple flavors settable in one call. Same coherency caveat in reverse: the front app only reads the new scrap on a resume-with-convert, so set the scrap while the target is in the background, then bring it forward
process.list running processes: [{psn, name, creator, type, launched_at, mem_size, mem_used, front:bool, background_only:bool}] via GetNextProcess + GetProcessInformation
process.launch {path?, creator?, args?} → PSN of launched app. Path via LaunchApplication; creator resolves through Desktop Database
process.front {psn} → SetFrontProcess
process.quit {psn} → send Quit Apple Event. If the app raises a save dialog, the controller resolves it through its dialog tools (capture + OCR + screen.input) — the agent doesn't parse dialogs
volume.list mounted volumes: [{vref, name, fs, total_bytes, free_bytes, ejectable, startup:bool}]
cursor.get {x, y, busy} where busy = true when a SetCursor(*GetCursor(watchCursor)) or SpinCursor-style busy indicator is showing; strong signal that Apple Events won't answer
system.time current guest date/time + timezone offset. Optional {set: iso8601} param to update the clock
log.tail {n?:100, level?:"info", categories?:[...]} → recent log lines from the in-memory ring buffer
log.download full log file as a binary frame (uses type-0x02)
log.set_level {level:"error"|"warn"|"info"|"debug"|"trace"} → live-change without restart
system.info richer system query: OS version, machine, CPU, physical RAM, free RAM, current user, timezone
update.stage {size, crc32, version} + chunked type-0x02 frames (MacBinary III, both forks) → writes a staged copy of the new agent binary next to the Preferences file; CRC-verified after landing. Never touches the running binary
update.apply launch the staged binary via LaunchApplication, then close the listener and quit. The new instance binds the port with the existing retry-with-backoff (same path as the MacTCP rebind window); its hello reports the new server_version and update_state:"staged"
update.commit the now-running staged instance replaces the Startup Items binary with itself: delete the old file (its owner already quit — deleting open files is what fBsyErr forbids), then CatMove + Rename its own open file into place (renaming open files is explicitly legal — "access paths currently in use aren't affected"). If CatMove balks on the open file, fall back to copying own forks to the destination and deleting the staged copy on next launch. update_state goes clean
update.abort delete the staged file
jobs.status jobs.cancel manage long-running operations
subscribe unsubscribe opt into event stream for a channel (e.g. jobs.*, log.line)

Concurrency model. Cooperative — classic Mac has no preemptive threads for our purposes. The agent uses the standard event loop with WaitNextEvent, MacTCP/OT async completion, and Apple Event dispatch. Long-running work (big file transfers, deep folder walks, raise-and-redraw yields before capture) is chunked across the loop and reported as a job.

Hang-avoidance discipline (mc-agent must not take the system down). The agent runs cooperatively alongside every other app, so a stuck handler stalls the whole machine. Rules the code must follow:

  • No blocking Toolbox calls without an explicit timeout. In particular AESend always passes a short timeout (default 5 s, caller-overridable via method timeout_ms), never kAEDefaultTimeout (which is ~60 s). File I/O uses async PBRead/PBWrite variants for anything over ~4 KB. MacTCP/OT reads and writes are always async with completion.
  • Per-handler time budget. No single request handler runs more than ~50 ms of straight-line work before either completing or yielding via WaitNextEvent. Anything longer becomes a job — the handler returns {job_id} synchronously and the work continues chunked in subsequent event-loop turns.
  • Chunk everything that could grow. fs.list on a 10,000-entry folder, PICT capture of a 1024×768 screen — all iterate in bounded chunks with WaitNextEvent between chunks. Better a stream of 20 ms slices than one 500 ms handler.
  • The input-executor task must return in microseconds. It runs at interrupt time (Time Manager / VBL); each firing only reads the next step from the pre-built script, writes the cursor/button globals (or calls CrsrDevMoveTo), posts at most one event, and returns. All parsing and gesture planning happens in the main loop before execution starts. No memory allocation, no QuickDraw, no file I/O at interrupt time.
  • Watchdog via Time Manager task. A Time Manager task (runs at interrupt time, immune to main-loop stalls) fires every second and checks a heartbeat counter incremented by the main event loop. If the counter hasn't moved in 3 ticks: append "STALLED in handler X for N ticks" to an in-memory ring buffer that's safe to write from interrupt context (no file I/O allowed at interrupt time). The main loop's next iteration — if it ever gets one — flushes the ring buffer to the log file. If configured, the watchdog also writes the line to the modem serial port at interrupt time (SCC async control call is interrupt-safe) so an external observer (host, MAME serial pipe) captures it even when nothing on the guest can. It can't recover the stall — but recovery isn't the goal; observability is.
  • AppleScript timeouts. applescript.eval wraps the script in a with timeout of N seconds ... end timeout block so long-running tell blocks against a hung target don't hang the OSA runtime.
  • No unbounded loops in the agent, ever. All loops have a max-iteration guard tied to elapsed time. If we hit it, log it and return an error rather than continuing.

Logging. Everything the agent does is logged — this is the primary tool for debugging and for building out functionality. The user shouldn't need a debugger to figure out why a method returned wrong.

  • Log file at System Folder:Preferences:mc-agent Log, plain text, MacRoman + CR line endings (native format), append-only, rotated at 256 KB (keeps mc-agent Log.1, .2; deletes older). Small enough to survive on a 4 MB Plus.
  • In-memory ring buffer of the most recent ~200 log lines, so log.tail returns instantly without disk I/O.
  • Format: HH:MM:SS.mmm LEVEL category: message. Categories: transport, dispatch, applescript, apple_event, screen, input, clipboard, process, fs, update, job, watchdog.
  • Levels: error, warn, info (default), debug, trace. Configurable in the Preferences file; also log.set_level method for live changes.
  • What we log at info: every request received (method, id, param summary), every response sent (id, ok/error, duration, result-size or first-error-line), every subscribed event emitted, watchdog stalls, startup/shutdown/reconfigure.
  • What we log at debug: raw JSON payloads (truncated), Apple Event descriptors before send + after reply, input-script compilation and per-step execution timestamps, chunk boundaries for long jobs.
  • log.tail {n?, level?, categories?} returns recent log lines; log.subscribe streams new lines as log.line events; log.download returns the full log file via a binary frame. All three exist so the controller can surface logs in an MCP resource for the agent to read without setting foot on the guest.
  • Crash breadcrumb via the log file (not low memory). Writing to random low-mem addresses on classic Mac is unsafe — only ToolScratch/ApplScratch (8-12 bytes) are documented as app-usable and other apps may use them too, and a hard crash / reboot wipes RAM anyway. Instead: before any risky Toolbox call, write a single short "BEGIN op=X id=N" line to the log file with an explicit PBFlushFileSync so it's on disk before the call runs. On successful return, write "END op=X id=N" (batched, no flush). On the next mc-agent startup, if the log file's last BEGIN has no matching END, the previous run crashed inside that operation — surface as last_crash_breadcrumb in the hello response. Cost is one disk flush per risky call (milliseconds on an HD, tolerable); benefit is a durable record with no low-mem hacks.

Cross-process realities. Classic Mac OS has no memory protection, no preemptive scheduling, and no accessibility tree. Several capabilities look natural on paper but are constrained by the environment; the plan reflects these honestly:

  • Toolbox UI state is per-process — the agent cannot see other apps' windows, dialogs, or menus. Under the Process Manager each process has its own window list and menu bar; FrontWindow, SelectWindow, GetMenuBar, MenuKey, and the Dialog Manager all operate on the calling process's state, and mc-agent (background-only, never calls InitWindows) has none. Enumerating another app's windows, parsing its DITLs, or reading its menus would mean walking its heap through undocumented structures — rejected as flaky. Instead the agent ships pixels (screen.capture) and the controller derives windows/dialogs/menus via OCR and vision. The documented cross-app "raise" is SetFrontProcess (whole app layer, exposed as process.front); raising a specific window within an app is a synthesized click on it.
  • Silent capture of obscured windows is not possible cross-process. The offscreen-GWorld trick only works inside the drawing app's own process. Capture always shows the screen as-is; for an occluded window's true contents, process.front the owning app, yield for the redraw, then capture. Disturbs z-order — there is no free lunch.
  • Synthesized input has a focus race. Posted events go to whichever process is frontmost at dispatch time, not to a specific window. "Click in window X" really means "raise X's app, yield, then inject." If focus shifts between raise and inject, events land in the wrong place. Best-effort; the controller verifies via screen.capture when it matters.
  • Clipboard coherency is suspend/resume-based. Apps with a private scrap publish to the global scrap only when suspended, and read it back only on resume-with-convert. clipboard.get right after a foreground Copy may be stale; clipboard.set isn't seen until the target resumes. The controller bounces focus via process.front to force conversion on both paths.
  • Standard File Package (Open/Save) is not scriptable in System 7. Neither Apple Events nor the Dialog Manager reach into it. When possible, drive apps via their own open / save Apple Events (which take an FSSpec and bypass the picker) rather than trying to automate the file dialog itself. Where the dialog is unavoidable, OCR + screen.input is the escape hatch.
  • Cooperative scheduling means one hung app hangs everyone, including mc-agent. If a target app enters a tight loop without calling WaitNextEvent, our TCP socket goes silent along with everything else. No user-mode mitigation — recovery escalates to reboot control (M5).
  • No memory protection. A crash in any process can take mc-agent with it. Same mitigation as above.
  • MacTCP port rebind window. After the agent quits, its listen port may sit in a MacTCP TIME_WAIT-equivalent state for a while; a fast restart may fail to bind. Retry with backoff on startup.
  • Screensaver / idle sleep return black frames. Bootstrap installer disables both by default (Preferences:General Controls and Preferences:Energy Saver settings); mc-agent hello reports back if either is still enabled so the controller can warn.

Dealing with blocking modal dialogs. A modal dialog in a target app (e.g. Disk Copy's "cannot read this disk" alert) will block that app's Apple Event dispatch until dismissed — applescript.eval against it will time out. mc-agent itself keeps running as a background app, so screen.capture and screen.input still work. Standard recovery loop for the controller: applescript.eval times out → screen.capture + OCR (the controller's dialog.list) detects a dialog on screen → read its text, pick a button, click it via screen.input (dialog.dismiss) → retry the original call. Detection is capture-polling by the controller — the agent has no way to observe another app's dialogs directly. System-modal alerts (bomb dialog, out-of-memory) can't be dismissed from user code — v1 escalates to manual intervention; M5 will add automated reboot.

Errors. Always a well-typed {error:{code, message, details?}}. Never a success-shaped empty response on failure — this echoes your global rule about network-call error handling.

Binary output. All binary payloads (PICT screenshots, file contents, MacBinary blobs) use the type-0x02 binary frame described above — never base64 inside JSON. mc-agent can therefore stream a large PICT or file directly from its source (Picture handle, file fork, GWorld PixMap) to the socket without ever allocating a copy or CPU-encoding it — critical on 4 MB machines where a 200 KB base64-of-a-PICT would eat significant RAM and seconds of 68030 time. Big transfers can be chunked by emitting several type-0x02 frames for the same id and letting the receiver concatenate — the response envelope's size field is the total.

Self-update over the wire. The first agent feature (M1): after the one-time bootstrap install per guest, every subsequent build ships over the existing TCP link — no more disk images, no manual setup loop. The one hard rule: never overwrite the running binary in place. On 68k the Segment Loader keeps CODE segments purgeable and re-reads them from the app file at runtime; on PPC with virtual memory enabled the application's data fork is the paging file for its code (file-mapped, assumed read-only by the OS). Overwriting either while running corrupts a live process. So updates are staged and two-phase:

  1. update.stage streams the new binary (chunked, CRC-checked) to a staging file — running binary untouched.
  2. update.apply launches the staged copy and quits; the controller reconnects and smoke-tests the new instance (hello, health, a trivial applescript.eval).
  3. Only then does update.commit promote the staged file into Startup Items, replacing the old binary.

Rollback for free: until commit, Startup Items still holds the old version — if the staged agent is broken or unreachable, a reboot (controller-driven for emulators; manual or M5 power control for real hardware) comes back up on the old agent, and the controller re-stages a fixed build. apply and commit are wrapped in the crash-breadcrumb BEGIN/END markers so a mid-swap crash is diagnosable from the next hello.

No TLS. Treat the link as trusted LAN. For remote access, tunnel through SSH or WireGuard on the host side.

Non-goals for v1. No RFB / live framebuffer streaming (screen is poll-based via screen.capture). No built-in AFP server. No shell / MPW dispatch (we cross-compile on the host with Retro68; nothing needs a build environment on the guest). No reboot / restart / shutdown control (surface power.status for detection only; recovery from hangs is out of scope until we have hardware or emulator control paths in a later milestone). No multi-client.

Host controller (mac-controld)

Rust, single static binary. Runs equally well on a laptop, NAS, or Raspberry Pi next to a real IIci.

Repo layout:

mac-control/
├── crates/                        # Rust host side
│   ├── mc-core/                   # Target, Transport, Capability, Driver traits
│   ├── mc-transport/              # tcp-direct, emu-ipc (MAME first)
│   ├── mc-agent-client/           # framed-JSON client; AppleScript + AE builders; screen, input, fs helpers
│   ├── mc-ocr/                    # OCR of window/screen images; engine trait + Tesseract impl
│   ├── mc-power/                  # emu-subproc now; smart-plug drivers deferred
│   ├── mc-discovery/              # ping, ARP, heartbeat poller
│   └── mc-mcp/                    # MCP server; tool registration
├── bin/mac-controld/              # daemon; config in ~/.config/mac-control/targets.toml
├── agent/                         # Retro68 C source for mc-agent
│   ├── src/                       # shared C source
│   ├── rsrc/                      # resource templates (SIZE, vers, BNDL, etc.)
│   ├── build-68k.sh               # -> dist/mc-agent-68k.bin
│   └── build-ppc.sh               # -> dist/mc-agent-ppc.bin
└── shared/protocol/               # JSON schema for the mc-agent wire protocol
                                   # generates Rust types + C headers

The wire protocol lives in shared/protocol/ and drives codegen on both sides so the Rust client and the C agent can't drift.

Every Target picks a transport; drivers check transport.supports(capability) before registering their MCP tools, so tools 404 cleanly on capability-missing targets.

MCP surface

Tools (minimum useful set):

  • targets.list, targets.describe
  • screen.capture (full screen or region), screen.click, screen.key, screen.type (backed by mc-agent screen.capture / screen.input)
  • screen.ocr {image | region?} → [{text, bbox, confidence}]. Runs on the host (mc-ocr crate), not on the guest — the classic Mac has no business running Tesseract. Includes bounding boxes so the agent can locate text spatially.
  • screen.click_text {text, match:"exact"|"contains"|"regex", occurrence?, region?} → capture, OCR, locate the matching text, click its center via screen.input. Ergonomic single call for the AI agent — no round-trip to compute pixel coordinates.
  • dialog.list, dialog.dismiss — controller-implemented on capture + OCR + input (the agent can't see other apps' dialogs). Buttons are located by OCR; the default button is identifiable visually by its bold rounded outline. First-class tools so the agent recovers from modal blockers without raw click math.
  • clipboard.get, clipboard.set — Scrap Manager access; TEXT / PICT / styled text. The controller handles the suspend/resume scrap-coherency dance (focus bounce via process.front).
  • menu.select — controller-implemented: Cmd-key keystroke when the equivalent is known (from the app's aete via the docs server, or prior OCR), else a blind press-drag-release gesture over the menu using standard menu metrics, verified by a follow-up capture. There is deliberately no menu.list — menu contents can't be enumerated cross-app in System 7, and an open menu can't be captured (the front app monopolizes the CPU while tracking); consult the docs server for an app's menus and cmd-keys.
  • process.list, process.launch, process.front, process.quit — process management
  • volume.list — mounted volumes with sizes
  • cursor.get — position + busy flag
  • system.info, system.time
  • fs.list, fs.get, fs.put (with optional type/creator codes), fs.delete
  • applescript.eval, app.tell (sugar with a param builder)
  • agent.update {target, dist_dir?} — orchestrates the full over-the-wire update: picks mc-agent-68k or -ppc from the target's cpu (reported in hello), then stage → apply → reconnect → smoke-test → commit. Any failure before commit leaves the guest on the old version and reports the phase that failed.
  • power.status → up|down|unknown + last_heartbeat. Reboot / hard power off deferred to M5.
  • discovery.scan — v1 is config-driven (targets listed in targets.toml); live subnet scan is a later milestone.
  • log.tail, log.download, log.set_level — surfaces the guest's own log so the agent can debug from the outside

No embedded docs:// resources. The agent consults your existing MCP docs server for Inside Macintosh, aete registries, etc. mac-controld links to your docs server in its own README and points agents at it in tool descriptions.

OCR on the host (mc-ocr)

Classic Mac has no accessibility tree — text on screen only exists as pixels. The controller needs OCR so the AI agent can read dialog text, list contents, status messages, and document text in apps without an aete.

Engine trait. mc-ocr::Engine with a default Tesseract implementation (via the tesseract Rust crate or subprocess to the tesseract CLI). A pluggable second implementation for Apple Vision (macOS hosts only) as an optional feature — significantly better quality on small bitmap fonts, but ties the controller to macOS. Vendor-neutral by default; users on a Mac can opt in.

Preprocessing. Classic Mac fonts (Chicago, Geneva, Monaco) at 9-12pt are unkind to generic OCR. Before invoking the engine: upscale 2-4x (nearest-neighbor to preserve pixel edges), binarize if not already 1-bit, and optionally invert for dark-mode themes (rare on System 7 but possible). Cache preprocessed images by source image hash + region so a re-OCR of the same window doesn't repeat the work.

Custom Tesseract training for classic Mac bitmap fonts is a known technique in the retro-computing community — worth investigating in M2 if generic Tesseract accuracy is poor. Ship a .traineddata file trained on Chicago/Geneva/Monaco 9/10/12pt bitmaps.

Transport modes

Each Target in targets.toml declares a transport; mac-controld picks the driver stack from that.

  1. emu-mame — MAME driving the classic Mac. TCP networking confirmed working on the user's local setup, so mc-agent inside the guest is reachable over TCP the same way it is on real hardware. Controller owns lifecycle (power.* = process control).
  2. tcp — real hardware with any TCP stack (MacTCP on 7.0/7.1, MacTCP or Open Transport on 7.5+). Full capability set through mc-agent.

Same driver stack for both. AppleTalk-only guests are out of scope; if a user has one, they need to add MacTCP or upgrade to a system that has one.

Bootstrap on a fresh 7.5

mac-controld bootstrap-image produces mc-boot.dsk (raw HFS for emulators) or .img / .sit set (real hardware) containing:

  • mc-agent-68k and mc-agent-ppc — the two binaries plus a Preferences template (listen port, token, log level).
  • A tiny Install mc-agent applet (ours, Retro68-built) that reads gestaltSysArchitecture (falling back to gestaltProcessorType / SysEnvirons on 7.0/7.1), copies the matching mc-agent-* into System Folder:Startup Items: and deletes the other from disk, writes the preferences file with a controller-supplied token, then quits.

Optional per-CPU images (mc-boot-68k.dsk / mc-boot-ppc.dsk) for the smallest possible download.

Delivery paths to get the disk image onto a fresh guest:

  • Emulator: attach the .dsk as a second disk.
  • Real Mac with an FTP-capable browser (Fetch, Anarchie, MacTCP + any FTP client): mac-controld serve-bootstrap runs a temporary HTTP/FTP server on the modern host; user fetches from the guest.
  • Real Mac with no network yet: floppy set (image splits into 1.4 MB chunks) or Zip cartridge / SCSI2SD / BlueSCSI image.

First-contact handshake: controller opens a TCP connection to mc-agent, sends hello with the pre-shared token, verifies the returned server_version and sys_version, and writes the target's fingerprint into targets.toml. Any hello failure aborts the handshake and reports the specific error — never silent success.

Milestones

  1. M1 — Toolchain + mc-agent v0.1 + self-update. Retro68 installed and reproducibly producing both mc-agent-68k.bin and mc-agent-ppc.bin from agent/; CI reports binary sizes as a tracked metric. Establish shared/protocol/ codegen. Agent implements hello, health, and the full update.* cycle — self-update is deliberately the first feature, because it is the dev loop: one manual bootstrap install per guest, then every subsequent build ships over the wire with agent.update. That pulls the chunked type-0x02 binary framing into M1 (update.stage needs it; screen and fs reuse it later). Once the update loop is proven, add applescript.eval + apple_event.send and wire the Rust mc-agent-client and the MCP tools applescript.eval + app.tell — shipped to the guest via the update loop itself. Run against MAME. This is the "AI drives Finder" demo.

  2. M2 — Screen + input. screen.capture (full screen + region) and screen.input (posted keys + interrupt-time cursor/button script) in mc-agent; corresponding MCP tools wired. Host-side mc-ocr, screen.click_text, and the controller-implemented dialog.list / dialog.dismiss / menu.select land here too — the whole window/dialog/menu semantic layer is OCR on the controller, so screen without OCR isn't just less useful, it's incomplete.

  3. M3 — Files. mc-agent's fs.* methods, reusing the chunked binary framing built for self-update in M1. Resource-fork aware.

  4. M4 — Real hardware. Real IIci/Quadra (or your box of choice) with MacTCP or OT; bootstrap disk image + delivery path validated end-to-end; installer applet exercised. Run the M1-M3 flow against it.

  5. M5 (deferred) — Reboot / power control. Soft restart via Finder AE, hard reboot via smart-plug (real hardware) or subprocess bounce (emulator). Deferred until we hit a real hang and need it.

Risks / unknowns

  • MAME screen/input scripting. MAME Lua can inject input and read the framebuffer, but for mc-agent's in-guest capture path this doesn't matter — screen/input flow through mc-agent over TCP, not through MAME. The MAME Lua path is only relevant as an out-of-band fallback (e.g. to unstick a hung guest); keep in mind but not on the critical path.
  • Retro68 toolchain. Cross-compiler is well-maintained but not zero-setup — need Docker or a documented native install path. Two separate binaries (68k, PPC) means two independent Retro68 pipeline runs — simpler than fat, but doubles CI matrix rows. Pin the Retro68 image version and reproduce builds in CI.
  • MacTCP vs OT socket abstraction inside mc-agent. The two APIs are genuinely different (MacTCP is Device-Manager PBControl calls, OT is BSD-ish endpoints). Our shim is a couple hundred lines each side but the abstractions leak — async completion semantics, notifier callbacks, and out-of-band data all differ. Budget time to get this right or the agent will hang on one stack while working on the other.
  • AppleScript missing on bare System 7.0/7.1. Ships standard from 7.5; earlier versions need AppleScript 1.1 from Scripting Additions. mc-agent should detect and expose applescript.eval capability accordingly (raw apple_event.send still works).
  • Raise-then-capture depends on well-behaved apps. Bringing an app forward via process.front and yielding for the redraw before capturing works cleanly for any standard Toolbox app that redraws on update events (nearly all do). A game or custom-draw utility that lazily draws may return a partially-blank frame — capture again after a short delay, or accept the occluded on-screen state.
  • Input synthesis rests on de-facto-stable low-memory globals. The Event Manager journaling mechanism is not an option — it's documented only in the pre-System 7 Inside Mac volumes and was dropped without replacement from the System 7-era docs (built for the single-app world; doesn't function under the Process Manager). The working technique is the one remote-control software of the era used: CrsrDevMoveTo (Cursor Device Manager, trap $AADB) where present, else RawMouse/MTemp/CrsrNew, plus MBState for the button and PostEvent for keys. These globals are stable across System 7 in practice but documented only in third-party references — validate against Finder (plus whatever else is on the disk image) in MAME during M2, and beware races with real user input at the console.
  • Menus are press-and-hold in System 7 (sticky menus arrive in Mac OS 8), and MenuSelect tracking is a tight loop in the front app — the agent gets no CPU mid-gesture, so menu drags must execute entirely at interrupt time and an open menu can never be captured or OCR'd. Menu items are selected blind (standard metrics or docs-server knowledge) and verified by capture afterward.
  • MacRoman↔UTF-8 and CR↔LF normalization. All strings crossing the wire need normalization at the mc-agent-client transport layer. Common footgun even without MPW — file names, dialog text, and AppleScript source all come through in MacRoman.
  • eppc:// temptation. Not available pre-OS 9. Document prominently in the README.
  • Wire-protocol port assignment. mc-agent needs a stable non-clashing port. Suggest TCP/4780 (unassigned, memorable — "MAC" if you squint). Confirm before we ship the first bootstrap image.

Verification

  • M1 acceptance: agent/build-68k.sh and agent/build-ppc.sh each produce a .bin; CI reports sizes as a tracked metric. mac-controld start launches MAME with a bundled disk image, opens a TCP connection to mc-agent, sends hello, receives a valid server_version. Update loop: agent.update upgrades the running agent to a build with a bumped server_version end-to-end in MAME, and a deliberately broken staged build (bad CRC, then a binary that crashes at startup) leaves the guest recoverable on the old version via reboot — this is the gate before any further agent features, since they all ship through it. Then: an MCP client calls applescript.eval with tell application "Finder" to make new folder at desktop with properties {name:"hello"} and the folder appears (verified visually in the emulator before M2; via screen.capture after M2; via fs.list after M3). Exercise both the 68k and PPC binaries against appropriate MAME configurations.
  • M2 acceptance: screen.capture returns a decodable PICT of the full screen and of a sub-region (host decodes to PNG for verification); screen.input selects "About This Macintosh" from the Apple menu via a blind interrupt-time press-drag-release gesture, verified by a follow-up capture; the controller's OCR-based dialog.list detects a Finder "Really empty Trash?" alert and dialog.dismiss {button:"cancel"} dismisses it; screen.click_text {text:"OK"} succeeds against a live dialog end-to-end.
  • M3 acceptance: round-trip a multi-megabyte file both directions with resource fork intact (compare MacBinary hashes host-side).
  • M4 acceptance: run the M1-M3 flow against a real Mac on the LAN with mc-agent installed via the bootstrap image, including one over-the-wire agent update. Exercise both a MacTCP-only guest (proves the socket shim) and an OT guest.
  • Automated CI: headless MAME in a container, exercised through the MCP client end-to-end. mc-agent unit-tested via a host-run build with a mock Toolbox layer where feasible; wire protocol contract-tested against golden fixtures shared with the Rust client.

Open inputs

  • Existing docs MCP server — resolved: the doc-search server already configured in this workspace's .mcp.json (nas.local:8080), with a classic-mac category of 308 docs (~182k chunks) including the Inside Macintosh volumes, Toolbox Essentials, Files, Processes, Imaging With QuickDraw, Interapplication Communication, PPC System Software, and era programming books. Gap: Apple Event Registry: Standard Suites itself is not indexed (IAC summarizes it); per-app aetes are read from the apps themselves via apple_event.send when needed. Worth adding the Registry PDF to the index if scripting work leans on standard-suite details.
  • mc-agent listen port — confirm TCP/4780 or specify a preference.

Closed decisions (from this planning session)

  • Floor: System 7.0, 68000 hardware included.
  • Guest TCP stack: MacTCP or Open Transport, via a small internal socket shim in mc-agent. Not building our own IP stack.
  • Screen: in-guest QuickDraw capture (full screen / region only — no per-window capture; window/dialog/menu semantics are controller-side OCR), input via posted events + interrupt-time cursor/button script. No MiniVNC. Cross-process memory traversal of other apps' windows/DITLs/menus rejected as flaky; OCR on the controller is the fallback of record.
  • Files: chunked over the mc-agent TCP link, both directions. No AFP, no AppleShare, no FTP, no Netatalk sidecar.
  • AppleTalk-only guests: out of scope. Users must add MacTCP or upgrade.
  • MAME networking: already proven working on the user's local setup; no spike needed.
  • Binary strategy: two separate binaries (68k, PPC), installer picks by Gestalt. Not fat.
  • Size discipline: C not C++, no libstdc++ / RTTI / exceptions, hand-rolled JSON, no debug symbols shipped. Tracked in CI as metric, not gated.
  • Docs: external MCP docs server (yours), not embedded.
  • Power: power.status (detection) only in v1. Reboot / restart / hard power off deferred to M5.
  • MPW: dropped. We cross-compile with Retro68 on the host; no need for a guest build environment.
  • Multi-client: single-client v1; second connection gets busy.
  • Self-update: over-the-wire, staged two-phase (stage → apply → smoke-test → commit); never overwrite the running binary (68k Segment Loader re-reads CODE segments; PPC+VM file-maps the data fork). Old binary stays in Startup Items until commit — reboot is the rollback.