Skip to content

Commit 0193cc6

Browse files
jeff-zuckerclaude
andcommitted
test: video-playback probe + mac-smoke video phase (macOS video reports)
Users report videos do not play on macOS; nothing mac-specific is wrong in the app layer and the Linux packaged pipeline is fully green, so give the mac-smoke workflow a real playback test: - tools/video-playback-probe.mjs: CDP probe — codec matrix (canPlayType), synthetic archive.org h.264 stream (retry: /download/ 500s transiently), and a DRIVE_MOVIES=1 drive of the real Movies room (re-clicks the tab; a cold-boot click can land before the menu tree wires). In tools/, not claude/smoke-tests/, because claude/ is gitignored and CI needs it. - packaged-smoke: SMOKE_VIDEO=1 boots with CDP (and WITHOUT --disable-gpu on darwin — users run with the GPU and hardware decode is the prime suspect) then runs the probe. Teardown fixes found while testing: fail() called process.exit(), which SKIPS finally — any post-boot fail left a zombie app whose orphaned servers strand the next boot; and the happy path's unref'd SIGKILL/cleanup timers never fired. Both replaced with graceful SIGTERM → wait → SIGKILL. ELECTRON_RUN_AS_NODE is scrubbed from the app env (local runs use electron-as-node for the probe's WebSocket; node 18 lacks it). - mac-smoke.yml: second "Video playback probe" step, gated by a ports-free wait so it cannot adopt the first boot's dying servers. Verified: npm test 149 pass; full SMOKE_VIDEO=1 packaged-smoke green twice on linux (film plays from a cold packaged boot, frames decoded). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent fee1d03 commit 0193cc6

3 files changed

Lines changed: 309 additions & 13 deletions

File tree

.github/workflows/mac-smoke.yml

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,3 +53,21 @@ jobs:
5353
SMOKE_EXPECT_VERSION="${TAG#v}" \
5454
SMOKE_MAC_APP="$RUNNER_TEMP/dk/unzipped/Solid Data Kitchen.app" \
5555
node tools/packaged-smoke.mjs
56+
57+
# Video-playback probe (the 2026-07-16 "videos do not play on macOS"
58+
# reports): a second boot WITH the GPU (users run with it; the plain
59+
# smoke boot disables it) + CDP, then codec matrix + a real archive.org
60+
# h.264 stream + a drive of the actual Movies room.
61+
- name: Video playback probe
62+
run: |
63+
# the first boot's server children can outlive their SIGKILLed app —
64+
# wait for the smoke ports to free so this boot can't adopt dying
65+
# servers (the stale-server relaunch gotcha)
66+
for i in $(seq 1 30); do
67+
lsof -nP -iTCP:18400 -iTCP:18410 -iTCP:18401 -sTCP:LISTEN >/dev/null 2>&1 || break
68+
sleep 2
69+
done
70+
SMOKE_EXPECT_VERSION="${TAG#v}" \
71+
SMOKE_MAC_APP="$RUNNER_TEMP/dk/unzipped/Solid Data Kitchen.app" \
72+
SMOKE_VIDEO=1 DRIVE_MOVIES=1 \
73+
node tools/packaged-smoke.mjs

tools/packaged-smoke.mjs

Lines changed: 83 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -29,8 +29,18 @@
2929
// against THAT .app and BOOT its binary. SMOKE_EXPECT_VERSION
3030
// overrides the Info.plist version to assert (defaults to
3131
// package.json's — pass it when testing an older release).
32+
// SMOKE_VIDEO=1 additionally boots with a CDP port and runs the
33+
// video-playback probe (tools/video-playback-probe.mjs)
34+
// against the booted app — codec matrix + a real
35+
// archive.org h.264 stream (+ DRIVE_MOVIES=1 drives the Movies
36+
// room). Added for the "videos do not play on macOS" reports
37+
// (2026-07-16). Needs node ≥22 (global WebSocket) — the
38+
// mac-smoke workflow's node 24 qualifies. On darwin this mode
39+
// boots WITHOUT --disable-gpu so the decode path matches what
40+
// users run (GPU/VideoToolbox is the prime mac suspect).
41+
// SMOKE_CDP_PORT overrides the debug port (default 9333).
3242
// Run automatically by tools/prepare-release.mjs while linux-unpacked exists.
33-
import { spawn } from 'node:child_process';
43+
import { spawn, spawnSync } from 'node:child_process';
3444
import { existsSync, lstatSync, mkdtempSync, readFileSync, rmSync, statSync } from 'node:fs';
3545
import { tmpdir } from 'node:os';
3646
import { fileURLToPath } from 'node:url';
@@ -46,7 +56,22 @@ const PORT = 18400; // spare ports — never the live 8000
4656
const CSS_PORT = 18410;
4757
const PROXY_PORT = 18401;
4858

49-
function fail(msg) { console.error(`[smoke] FAIL: ${msg}`); process.exit(1); }
59+
// process.exit() SKIPS finally blocks — a fail() after the boot spawn must
60+
// kill the app itself or it survives as a zombie holding the smoke ports
61+
// (bit 2026-07-16: a failed video probe left the app + servers running, and
62+
// the next boot adopted the dying servers).
63+
let bootedChild = null;
64+
function fail(msg) {
65+
console.error(`[smoke] FAIL: ${msg}`);
66+
if (bootedChild && bootedChild.exitCode === null) {
67+
try { bootedChild.kill('SIGTERM'); } catch {}
68+
// synchronous 2s grace so the app can take its server children with it,
69+
// then make sure it is gone
70+
try { Atomics.wait(new Int32Array(new SharedArrayBuffer(4)), 0, 0, 2000); } catch {}
71+
try { bootedChild.kill('SIGKILL'); } catch {}
72+
}
73+
process.exit(1);
74+
}
5075
function ok(msg) { console.log(`[smoke] ok — ${msg}`); }
5176

5277
if (!MAC_APP && !existsSync(appDir)) fail(`no ${appDir} — run \`npm run dist:cross\` first`);
@@ -144,16 +169,29 @@ const tmp = mkdtempSync(join(tmpdir(), 'dk-smoke-'));
144169
const podHome = join(tmp, 'pod');
145170
const userData = join(tmp, 'userData');
146171

172+
const VIDEO = process.env.SMOKE_VIDEO === '1';
173+
const CDP_PORT = process.env.SMOKE_CDP_PORT || '9333';
174+
const bootArgs = ['--no-sandbox', `--user-data-dir=${userData}`];
175+
// --disable-gpu stays for the plain boot test (headless-CI friendly), but the
176+
// video phase must exercise the same GPU decode path users get — on darwin
177+
// especially, where hardware H.264 decode is the prime failure suspect.
178+
if (!(VIDEO && process.platform === 'darwin')) bootArgs.push('--disable-gpu');
179+
if (VIDEO) bootArgs.push(`--remote-debugging-port=${CDP_PORT}`);
180+
147181
console.log(`[smoke] booting ${bin}\n[smoke] pod home ${podHome}, ports ${PORT}/${CSS_PORT}/${PROXY_PORT}`);
148-
const child = spawn(bin, ['--no-sandbox', '--disable-gpu', `--user-data-dir=${userData}`], {
149-
env: {
150-
...process.env,
151-
DK_POD_ROOT: podHome,
152-
DK_PUBLIC_PORT: String(PORT),
153-
DK_CSS_INTERNAL_PORT: String(CSS_PORT),
154-
DK_PROXY_PORT: String(PROXY_PORT),
155-
},
156-
});
182+
const childEnv = {
183+
...process.env,
184+
DK_POD_ROOT: podHome,
185+
DK_PUBLIC_PORT: String(PORT),
186+
DK_CSS_INTERNAL_PORT: String(CSS_PORT),
187+
DK_PROXY_PORT: String(PROXY_PORT),
188+
};
189+
// When packaged-smoke itself runs under `ELECTRON_RUN_AS_NODE=1 electron`
190+
// (the local node is too old for the probe's WebSocket), that variable must
191+
// not reach the app binary — it would boot as a bare node process.
192+
delete childEnv.ELECTRON_RUN_AS_NODE;
193+
const child = spawn(bin, bootArgs, { env: childEnv });
194+
bootedChild = child;
157195

158196
let log = '';
159197
const onData = (d) => { log += d.toString(); };
@@ -192,10 +230,42 @@ try {
192230
if (!res) fail(`router on :${PORT} did not answer`);
193231
ok(`router answers on :${PORT} (HTTP ${res.status})`);
194232

233+
if (VIDEO) {
234+
// CDP must be answering before the probe connects.
235+
let cdpUp = false;
236+
for (let i = 0; i < 20 && !cdpUp; i++) {
237+
cdpUp = await fetch(`http://localhost:${CDP_PORT}/json`, { signal: AbortSignal.timeout(2000) })
238+
.then((r) => r.ok).catch(() => false);
239+
if (!cdpUp) await new Promise((r) => setTimeout(r, 1000));
240+
}
241+
if (!cdpUp) fail(`CDP endpoint on :${CDP_PORT} never answered`);
242+
ok(`CDP endpoint answers on :${CDP_PORT}`);
243+
// give the shell a moment to finish rendering before the probe drives it
244+
await new Promise((r) => setTimeout(r, 8000));
245+
// lives in tools/ (not claude/smoke-tests/) because claude/ is gitignored
246+
// and the mac-smoke workflow needs the probe in the checkout
247+
const probe = join(root, 'tools', 'video-playback-probe.mjs');
248+
console.log('[smoke] running video-playback probe…');
249+
const pr = spawnSync(process.execPath, [probe], {
250+
env: { ...process.env, CDP_PORT },
251+
stdio: 'inherit',
252+
timeout: 300_000,
253+
});
254+
if (pr.status !== 0) fail('video-playback probe failed (see output above)');
255+
ok('video playback verified');
256+
}
257+
195258
console.log('[smoke] PASS — packaged app boots, seeds, and serves');
196259
} finally {
260+
// graceful first (lets the app take its CSS/proxy children down), then
261+
// hard — and only THEN delete the throwaway home. The old unref'd timers
262+
// never fired: process.exit(0) below ran before them.
197263
child.kill('SIGTERM');
198-
setTimeout(() => { try { child.kill('SIGKILL'); } catch {} }, 3000).unref();
199-
setTimeout(() => { try { rmSync(tmp, { recursive: true, force: true }); } catch {} }, 4000).unref();
264+
const dead = await new Promise((res) => {
265+
const to = setTimeout(() => res(false), 3000);
266+
child.once('exit', () => { clearTimeout(to); res(true); });
267+
});
268+
if (!dead) { try { child.kill('SIGKILL'); } catch {} }
269+
try { rmSync(tmp, { recursive: true, force: true }); } catch {}
200270
}
201271
process.exit(0);

tools/video-playback-probe.mjs

Lines changed: 208 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,208 @@
1+
// Video playback diagnostic — written for the "videos do not play on macOS"
2+
// reports (2026-07-16). Connects to a running dk instance over CDP and:
3+
// A. reports the renderer's codec support matrix (canPlayType) — tells a
4+
// missing-ffmpeg / proprietary-codec problem apart from a render one,
5+
// B. plays a known-good archive.org h.264 mp4 in a synthetic muted <video>
6+
// and asserts currentTime advances AND videoWidth > 0 (frames decoded),
7+
// C. (DRIVE_MOVIES=1) drives the real Movies room: clicks a film row and
8+
// measures the app's own .ia-video element the same way.
9+
// Platform-neutral: run on Linux for a baseline, on the mac-smoke runner for
10+
// the real answer. Exit 0 = video pipeline works, 1 = something failed.
11+
//
12+
// Usage: node tools/video-playback-probe.mjs
13+
// env CDP_PORT=9222 (default) DRIVE_MOVIES=1 (optional section C)
14+
// The app must be running with --remote-debugging-port=$CDP_PORT.
15+
// Run standalone against any live instance, or via packaged-smoke.mjs
16+
// SMOKE_VIDEO=1 (how the mac-smoke workflow runs it). Lives in tools/
17+
// because claude/ is gitignored and CI needs this file in the checkout.
18+
// Needs node ≥22 (global WebSocket/fetch) — locally use
19+
// `ELECTRON_RUN_AS_NODE=1 npx electron tools/video-playback-probe.mjs`.
20+
21+
const PORT = process.env.CDP_PORT || 9222;
22+
const DRIVE = process.env.DRIVE_MOVIES === '1';
23+
// h.264 + AAC, public item, streams via range requests (we need ~3s of it).
24+
const TEST_MP4 = 'https://archive.org/download/BigBuckBunny_124/Content/big_buck_bunny_720p_surround.mp4';
25+
26+
const targets = await (await fetch(`http://localhost:${PORT}/json`)).json();
27+
const page = targets.find(t => t.type === 'page' && /index\.html/.test(t.url)) || targets.find(t => t.type === 'page');
28+
if (!page) { console.error('no page target on CDP port', PORT); process.exit(1); }
29+
const ws = new WebSocket(page.webSocketDebuggerUrl);
30+
let id = 0; const pending = new Map();
31+
const send = (method, params = {}) => new Promise((res, rej) => {
32+
const i = ++id; pending.set(i, { res, rej });
33+
ws.send(JSON.stringify({ id: i, method, params }));
34+
});
35+
ws.onmessage = m => { const d = JSON.parse(m.data);
36+
if (d.id && pending.has(d.id)) { const { res, rej } = pending.get(d.id); pending.delete(d.id);
37+
d.error ? rej(new Error(d.error.message)) : res(d.result); } };
38+
await new Promise(r => ws.onopen = r);
39+
await send('Runtime.enable');
40+
const sleep = ms => new Promise(r => setTimeout(r, ms));
41+
async function evalJS(expr) {
42+
const r = await send('Runtime.evaluate', { expression: `(async()=>{${expr}})()`, awaitPromise: true, returnByValue: true });
43+
if (r.exceptionDetails) throw new Error(r.exceptionDetails.exception?.description || r.exceptionDetails.text);
44+
return r.result?.value;
45+
}
46+
let fails = 0;
47+
const check = (label, ok, detail = '') => { console.log(`${ok ? '✔' : '✘'} ${label}${detail ? ' — ' + detail : ''}`); if (!ok) fails++; };
48+
49+
const ver = await send('Browser.getVersion').catch(() => null);
50+
console.log('BROWSER:', ver ? `${ver.product} on ${ver.userAgent.match(/\((.*?)\)/)?.[1]}` : 'unknown');
51+
52+
// ---- A. codec support matrix -------------------------------------------
53+
const codecs = await evalJS(`
54+
const v = document.createElement('video');
55+
const t = s => v.canPlayType(s) || '(no)';
56+
return {
57+
'mp4 h.264+aac': t('video/mp4; codecs="avc1.42E01E, mp4a.40.2"'),
58+
'mp4 h.264 high': t('video/mp4; codecs="avc1.64001F"'),
59+
'mp4 mpeg4-part2': t('video/mp4; codecs="mp4v.20.240"'),
60+
'ogg theora': t('video/ogg; codecs="theora"'),
61+
'webm vp8': t('video/webm; codecs="vp8, vorbis"'),
62+
'webm vp9': t('video/webm; codecs="vp9"'),
63+
'audio mp3': t('audio/mpeg'),
64+
'audio aac': t('audio/mp4; codecs="mp4a.40.2"'),
65+
};
66+
`);
67+
console.log('CODECS:', JSON.stringify(codecs, null, 1));
68+
check('h.264 mp4 decodable (proprietary codecs present)', /probably|maybe/.test(codecs['mp4 h.264+aac']), codecs['mp4 h.264+aac']);
69+
check('mp3 decodable', /probably|maybe/.test(codecs['audio mp3']), codecs['audio mp3']);
70+
71+
// ---- B. synthetic playback of a known-good h.264 mp4 --------------------
72+
// archive.org's /download/ endpoint 500s transiently — retry once before
73+
// calling the pipeline broken.
74+
console.log('SYNTHETIC PLAY:', TEST_MP4);
75+
const playOnce = () => evalJS(`
76+
const v = document.createElement('video');
77+
v.muted = true; v.playsInline = true; v.preload = 'auto';
78+
v.style.cssText = 'position:fixed;left:0;top:0;width:320px;height:180px;opacity:0.01;pointer-events:none;z-index:-1';
79+
v.src = ${JSON.stringify(TEST_MP4)};
80+
document.body.appendChild(v);
81+
const errInfo = () => v.error ? { code: v.error.code, message: v.error.message } : null;
82+
try {
83+
// wait for enough data or an error (network on CI can be slow)
84+
await new Promise((res, rej) => {
85+
const to = setTimeout(() => rej(new Error('timeout waiting for canplay (45s)')), 45000);
86+
v.addEventListener('canplay', () => { clearTimeout(to); res(); }, { once: true });
87+
v.addEventListener('error', () => { clearTimeout(to); rej(new Error('media error')); }, { once: true });
88+
});
89+
await v.play();
90+
await new Promise(r => setTimeout(r, 3000));
91+
const t1 = v.currentTime;
92+
await new Promise(r => setTimeout(r, 2000));
93+
const t2 = v.currentTime;
94+
return { ok: true, t1, t2, advanced: t2 > t1, videoWidth: v.videoWidth, videoHeight: v.videoHeight,
95+
readyState: v.readyState, decodedFrames: (v.getVideoPlaybackQuality?.() || {}).totalVideoFrames ?? null,
96+
droppedFrames: (v.getVideoPlaybackQuality?.() || {}).droppedVideoFrames ?? null, error: errInfo() };
97+
} catch (e) {
98+
return { ok: false, why: String(e && e.message || e), readyState: v.readyState,
99+
networkState: v.networkState, error: errInfo() };
100+
} finally { v.pause(); v.removeAttribute('src'); v.load(); v.remove(); }
101+
`);
102+
let play = await playOnce();
103+
if (!play.ok) {
104+
console.log(' first attempt failed, retrying in 5s…', JSON.stringify(play));
105+
await sleep(5000);
106+
play = await playOnce();
107+
}
108+
console.log('RESULT:', JSON.stringify(play, null, 1));
109+
check('media loaded + play() resolved', play.ok === true, play.ok ? '' : `${play.why} mediaError=${JSON.stringify(play.error)} networkState=${play.networkState}`);
110+
if (play.ok) {
111+
check('currentTime advances', play.advanced, `t1=${play.t1?.toFixed(2)} t2=${play.t2?.toFixed(2)}`);
112+
check('frames decoded (videoWidth > 0)', play.videoWidth > 0, `${play.videoWidth}x${play.videoHeight}`);
113+
check('decoder produced frames', play.decodedFrames === null || play.decodedFrames > 0, `decoded=${play.decodedFrames} dropped=${play.droppedFrames}`);
114+
}
115+
116+
// ---- C. optional: drive the real Movies room ----------------------------
117+
if (DRIVE) {
118+
console.log('DRIVING the Movies room…');
119+
const nav = await evalJS(`
120+
function deepQueryAll(sel, root = document) {
121+
const hits = [...root.querySelectorAll(sel)];
122+
for (const el of root.querySelectorAll('*')) if (el.shadowRoot) hits.push(...deepQueryAll(sel, el.shadowRoot));
123+
return hits;
124+
}
125+
// desktop tab strip lives in sol-tabs' shadow root
126+
const tabs = deepQueryAll('[role="tab"], .sol-tab, button');
127+
const movies = tabs.find(t => /movies/i.test(t.textContent || ''));
128+
if (!movies) return 'no-movies-tab';
129+
movies.click();
130+
return 'clicked:' + movies.textContent.trim().slice(0, 30);
131+
`);
132+
console.log(' nav:', nav);
133+
// lazy ui:module mount + cold-cache library load — poll up to 90s for a
134+
// visible room, re-clicking the Movies trigger every 20s (a click that
135+
// landed before the menu tree finished wiring selects nothing).
136+
let mounted = false;
137+
const t0 = Date.now();
138+
for (let i = 0; i < 45 && !mounted; i++) {
139+
await sleep(2000);
140+
const st = await evalJS(`
141+
function deepQueryAll(sel, root = document) {
142+
const hits = [...root.querySelectorAll(sel)];
143+
for (const el of root.querySelectorAll('*')) if (el.shadowRoot) hits.push(...deepQueryAll(sel, el.shadowRoot));
144+
return hits;
145+
}
146+
const apps = deepQueryAll('.ia-player-app.media-video');
147+
return { visible: apps.some(a => a.offsetParent !== null), present: apps.length };
148+
`);
149+
mounted = st.visible;
150+
if (!mounted && i > 0 && i % 10 === 0) {
151+
const re = await evalJS(`
152+
function deepQueryAll(sel, root = document) {
153+
const hits = [...root.querySelectorAll(sel)];
154+
for (const el of root.querySelectorAll('*')) if (el.shadowRoot) hits.push(...deepQueryAll(sel, el.shadowRoot));
155+
return hits;
156+
}
157+
const tabs = deepQueryAll('[role="tab"], .sol-tab, button');
158+
const movies = tabs.find(t => /movies/i.test(t.textContent || ''));
159+
if (movies) movies.click();
160+
return movies ? 're-clicked' : 'no-trigger';
161+
`);
162+
console.log(` …not mounted after ${Math.round((Date.now() - t0) / 1000)}s (present=${st.present}) — ${re}`);
163+
}
164+
}
165+
check('movies room mounted', mounted, `after ${Math.round((Date.now() - t0) / 1000)}s`);
166+
if (mounted) {
167+
// Drive the browse cascade: (All film types) → first collection → first
168+
// film. In movies each film (album row) plays on click — loaded paused
169+
// behind the film-intro overlay; we then call play() directly.
170+
const room = await evalJS(`
171+
function deepQueryAll(sel, root = document) {
172+
const hits = [...root.querySelectorAll(sel)];
173+
for (const el of root.querySelectorAll('*')) if (el.shadowRoot) hits.push(...deepQueryAll(sel, el.shadowRoot));
174+
return hits;
175+
}
176+
const app = deepQueryAll('.ia-player-app.media-video').find(a => a.offsetParent !== null);
177+
const rowsIn = col => [...app.querySelectorAll('[data-column="' + col + '"] .ia-listbox-item')]
178+
.filter(li => !li.classList.contains('ia-listbox-all') && li.offsetParent !== null);
179+
const firstArtist = rowsIn('artist')[0];
180+
if (firstArtist) { firstArtist.click(); await new Promise(r => setTimeout(r, 8000)); }
181+
const films = rowsIn('album');
182+
if (!films.length) return { films: 0, artistClicked: firstArtist?.textContent.trim().slice(0, 50) || null };
183+
films[0].click();
184+
// wait for the app to fetch item metadata and set the video src
185+
const v = app.querySelector('.ia-video');
186+
for (let i = 0; i < 20 && !(v && v.src); i++) await new Promise(r => setTimeout(r, 1000));
187+
if (!v || !v.src) return { films: films.length, clicked: films[0].textContent.trim().slice(0, 60), loaded: false };
188+
v.muted = true;
189+
try { await v.play(); } catch (e) { return { loaded: true, src: v.src.slice(0, 110), playRejected: String(e) }; }
190+
await new Promise(r => setTimeout(r, 5000));
191+
return { films: films.length, clicked: films[0].textContent.trim().slice(0, 60), loaded: true,
192+
src: v.src.slice(0, 110), t: v.currentTime, videoWidth: v.videoWidth, readyState: v.readyState,
193+
error: v.error ? { code: v.error.code, message: v.error.message } : null };
194+
`);
195+
console.log(' room:', JSON.stringify(room, null, 1));
196+
if (room.loaded) {
197+
check('film plays (currentTime > 0)', room.t > 0 && !room.error && !room.playRejected,
198+
`t=${room.t?.toFixed?.(2)} err=${JSON.stringify(room.error)} rejected=${room.playRejected || 'no'}`);
199+
check('film frames decoded', room.videoWidth > 0, `videoWidth=${room.videoWidth}`);
200+
} else {
201+
check('film loaded a src', false, JSON.stringify(room));
202+
}
203+
}
204+
}
205+
206+
console.log(fails ? `FAILED: ${fails} check(s)` : 'ALL CHECKS PASSED');
207+
ws.close();
208+
process.exit(fails ? 1 : 0);

0 commit comments

Comments
 (0)