lux

A visual output surface for AI agents.

Lux gives agents and apps a shared visual surface. The intended architecture is a hub/display split: clients send UI descriptions to luxd, the Hub owns authoritative element state and behavior, and the Display renders a replica of the current scene while forwarding user interactions back to the Hub.

The design draws on X11's client/server split and Smalltalk-style live introspection. MCP is one gateway into Lux, not the whole architecture. If you want the short version of the rewrite target, start with docs/architecture/target/target.md. If you need help navigating the docs, use docs/README.md. For the product direction, positioning, and risk assessment — the Working Backwards PR/FAQ — see prfaq.pdf.

Platforms: macOS, Linux

Stage: alpha --- protocol is stable, published on PyPI as punt-lux

A Claude Code plugin displaying a project issue board --- the agent fetches live data from DoltDB via bd list --json, builds a filterable table with detail panel, and renders it in a single tool call. Filters and row selection run at 60fps with zero MCP round-trips.

The same list/detail pattern generalizes to any tabular data. Search, combo filters, pagination, and a detail panel --- all driven by a single show_table() call.

Dashboards compose metric cards, charts, and tables. show_dashboard() builds the layout from structured data --- no manual element positioning needed.

Quick Start

curl -fsSL https://raw.githubusercontent.com/punt-labs/lux/92d9172/install.sh | sh

Restart Claude Code twice. The Lux display window opens automatically when agents send visual output.

Manual install (if you already have uv)

uv tool install 'punt-lux[display]'

Then install the plugin via the marketplace:

claude plugin marketplace add punt-labs/claude-plugins
claude plugin install lux@punt-labs

Lightweight install (library use only)

If you only need DisplayClient to send scenes from Python (no display server):

uv add punt-lux

This pulls ~2 MB of lightweight deps. The 66 MB display stack (imgui-bundle, numpy, Pillow, PyOpenGL) is only needed for lux display and is available via punt-lux[display].

Verify before running

curl -fsSL https://raw.githubusercontent.com/punt-labs/lux/92d9172/install.sh -o install.sh
shasum -a 256 install.sh
cat install.sh
sh install.sh

Run a demo

lux display &
uv run python demos/dashboard.py

Demos are in demos/ --- each connects as a client and drives the display:

Demo	What it shows
interactive.py	Sliders, checkboxes, combos, text inputs, color pickers
containers.py	Windows, tab bars, collapsing headers, groups
dashboard.py	Multi-window layout with draw canvases and live controls
data_viz.py	Tables, plots, progress bars, spinners, markdown
menu_bar.py	Custom menus, event handling, periodic refresh

Features

• 25 element kinds --- text, buttons (arrow, small), images, sliders, checkboxes, combos, inputs (text, number), radios, color pickers (alpha, full picker), selectables, trees, tables, plots, progress bars, spinners, markdown, draw canvases, modals, dialogs, groups, tab bars, collapsing headers, windows, separators
• Frames --- scenes target named frames (inner windows) via frame_id. Frames persist after disconnect, can be adopted by new clients, and support initial sizing (frame_size) and ImGui window flags (frame_flags)
• Layout nesting --- windows contain tab bars contain groups contain any element, arbitrarily deep
• Incremental updates --- update patches individual elements by ID without replacing the scene
• World menu --- per-client namespaced menus. Each connected MCP server gets its own submenu. Items registered via register_tool are routed only to the owning client
• Interaction handling --- button clicks, slider changes, and menu clicks fire their handlers on the Hub (D21 remote dispatch); the raw event log is readable via list_recent_events. Hub handlers can publish app events that the agent reads via recv
• Frame auto-focus --- frames automatically focus (brought to front) when they receive a scene update
• Persistent tabs --- each show() call opens a dismissable tab; same scene_id replaces content in-place. Users can close individual tabs
• Themes --- 11 themes via set_theme: imgui_colors_dark, imgui_colors_light, imgui_colors_classic, darcula, darcula_darker, material_flat, photoshop_style, grey_flat, cherry, light_rounded, microsoft_style
• Auto-spawn --- DisplayClient starts the display server on first connection if it isn't running
• Unix socket IPC --- length-prefixed JSON frames, no HTTP overhead, no threads

MCP Tools

Agents interact with Lux through 27 MCP tools exposed by lux serve:

Tool	What it does
Scene management
show(scene_id, elements)	Replace the display with a new element tree. Supports frame_id, frame_size, frame_flags for windowed frames
show_table(scene_id, columns, rows)	Display a filterable data table with optional detail panel
show_dashboard(scene_id, ...)	Display a dashboard with metric cards, charts, and a table
update(scene_id, patches)	Patch elements by ID (set fields or remove)
clear()	Remove all content from the display
Communication
ping()	Round-trip latency check
recv(timeout)	Block for the next published app event for this session (pub/sub); returns event:<topic>:<payload> or none. UI interactions are handled Hub-side, not delivered here
set_menu(menus)	Add custom menus to the menu bar
register_tool(id, label)	Register a World menu item routed only to the calling server via recv()
set_theme(theme)	Switch display theme
Configuration
display_mode(repo)	Read current display mode (y/n) for the caller's project --- pass the absolute project path
set_display_mode(mode, repo)	Set display mode for the caller's project --- pass the absolute project path
set_window_settings(...)	Configure opacity, font scale, decoration, idle FPS
set_frame_state(frame_id, ...)	Minimize or restore a frame
Introspection
inspect_scene(scene_id)	Return element tree for a scene
list_scenes()	List all active scenes with metadata
screenshot()	Capture display as base64 PNG
get_display_info()	Display dimensions, frame count, client count
get_window_settings()	Current window configuration
get_theme()	Current theme name
list_clients()	Connected clients with names and scene counts
list_menus()	Registered menu items
list_recent_events(count)	Recent interaction events
list_errors(count)	Recent error log entries
Pub/Sub (Agent Subscribe)
subscribe(topic)	Subscribe to a Hub-scoped app topic; delivered via recv
unsubscribe(topic)	Stop receiving a topic
publish(topic, payload)	Publish an app event to a Hub topic (separate from the UI observer mechanism)

What It Looks Like

Show text and a button

{"tool": "show", "input": {
  "scene_id": "hello",
  "elements": [
    {"kind": "text", "id": "t1", "content": "Hello from the agent"},
    {"kind": "button", "id": "b1", "label": "Click me"}
  ]
}}

Returns "ack:hello". A button click fires its handler on the Hub (the agent does not poll for it). To observe interactions, read the introspection log:

{"tool": "list_recent_events", "input": {"count": 5}}

A Hub-side handler can publish an app event that the agent then reads with recv (see the Pub/Sub tools above).

Multi-window dashboard

{"tool": "show", "input": {
  "scene_id": "dash",
  "elements": [
    {"kind": "window", "id": "w1", "title": "Controls", "x": 10, "y": 10,
     "children": [
       {"kind": "slider", "id": "vol", "label": "Volume", "value": 50}
     ]},
    {"kind": "window", "id": "w2", "title": "Chart", "x": 320, "y": 10,
     "children": [
       {"kind": "plot", "id": "p1", "title": "Trend",
        "series": [{"label": "y", "type": "line",
          "x": [1,2,3,4], "y": [10,20,15,25]}]}
     ]}
  ]
}}

Update a single element

{"tool": "update", "input": {
  "scene_id": "dash",
  "patches": [
    {"id": "vol", "set": {"value": 75}}
  ]
}}

Element Kinds

Category	Kinds
Display	text, button (arrow, small variants), image, separator
Interactive	slider, checkbox, combo, input_text, input_number, radio, color_picker (alpha, picker modes)
Lists	selectable, tree
Data	table, plot, progress, spinner, markdown
Canvas	draw (line, rect, circle, triangle, polyline, text, bezier)
Layout	group, tab_bar, collapsing_header, window, modal, dialog (modal confirm dialog with Hub-side handler dispatch)

All elements with an id support an optional tooltip field (string shown on hover).

CLI Commands

Command	What it does
lux display	Start the display server (ImGui window)
lux serve	Start the MCP server (stdio transport)
lux enable	Enable visual output for this project
lux disable	Disable visual output for this project
lux status	Check if the display server is running
lux doctor	Check installation health (Python, fonts, plugin)
lux install	Install the Claude Code plugin via the marketplace
lux uninstall	Uninstall the Claude Code plugin
lux show beads	Display the beads issue board (no LLM needed)
lux ping	Ping the display server; print round-trip time
lux hub-install	Register the luxd session hub as a launchd/systemd service
lux hub-uninstall	Remove the luxd service
lux ensure-hub	Ensure luxd is running (--restart to restart)
lux hub-status	Report luxd service status
lux setup-proxy	Write the mcp-proxy config for the hub
lux version	Print version

Architecture

Agent or app
  │ MCP or direct Hub API
  ▼
luxd (Hub)
  │ authoritative state + introspection
  │ scene replicas + remote invocations
  ▼
lux display (ImGui + OpenGL)
  │ renders at 60fps
  ▼
Window on screen

The Hub is the single source of truth for element state, ownership, and handler dispatch. The Display is a rendering replica: it paints the current scene and forwards interactions back to the Hub, which runs the real handler and re-pushes updated state. MCP is one entry point, not the only one.

Documentation

Docs Guide | Target Architecture | Target Topology | Target UI Model | Target Introspection | Current Architecture | Design Log | Changelog

Development

uv sync --extra display       # Install dependencies (dev group installs by default)
uv run ruff check .            # Lint
uv run ruff format --check .   # Check formatting
uv run mypy src/ tests/        # Type check (mypy)
uv run pyright                 # Type check (pyright)
uv run pytest                  # Test

Acknowledgements

Lux is a thin orchestration layer. The rendering is done by Dear ImGui, Omar Cornut's immediate-mode GUI library. ImGui handles all the hard problems --- text layout, widget state, input handling, GPU rendering --- and does so in a single-pass retained-mode-free architecture that maps naturally to Lux's "send JSON, render this frame" model. The 60fps render loop, the composable widget tree, and the ability to drive a full UI from a socket with no threading are all consequences of ImGui's design.

Python bindings come from imgui-bundle by Pascal Thomet, which packages ImGui, ImPlot, and several other ImGui extensions into a single pip-installable wheel with complete type stubs. imgui-bundle is what makes "install one Python package, get a GPU-accelerated UI" possible.

FastMCP provides the MCP server layer.

License

MIT