Paste a public share link (ChatGPT, Claude, Gemini, Grok, Meta AI, Perplexity) and get the full conversation back as clean text and also you can summarize.
These share pages are client-rendered SPAs — the raw HTML is just an empty
app shell, and the actual conversation is loaded into the page by
JavaScript after it mounts. A plain fetch() never sees it, no matter how
good the parsing logic is. So instead, the server renders the page with a
real headless browser (Puppeteer, via puppeteer-extra + the stealth
plugin — see below) and then:
- Navigates to the URL and waits for the network to go idle so client-side data loading finishes.
- Waits (best-effort) for a message-shaped element to appear, then auto-scrolls in case anything is lazily rendered.
- Attempt 1 — in-page state extraction: runs a script inside the
rendered page that walks known client-side state containers
(
window.__reactRouterDataRouter,__NEXT_DATA__,__remixContext,__APOLLO_STATE__,__INITIAL_STATE__, etc.) looking for anything shaped like a list of chat turns. - Attempt 2 — structured HTML extraction: falls back to any JSON a platform still embeds directly in the rendered HTML.
- Attempt 3 — DOM fallback: scans the rendered HTML for elements that
look like chat turns (
data-message-author-role, class names containing "message"/"turn", etc.). - Formats whatever it found into a labeled transcript (
--- YOU ---/--- CLAUDE ---/ etc.) and returns it to the browser.
Sites like claude.ai sit behind bot-management (Cloudflare, etc.) that can return a 403 on the initial document response to a headless browser, even for a genuinely public share link. Two things address this:
puppeteer-extra-plugin-stealthpatches over the common headless tells (missingnavigator.webdriverspoofing, nochromeruntime object, etc.) that trigger bot detection in the first place.- A 401/403 status no longer causes an instant hard failure. The server keeps trying to extract content regardless, and only reports "may be private" if extraction still comes up empty afterward — so a false-positive block page doesn't kill a link that's actually public.
Once a conversation is extracted, an optional Summarize button sends the transcript to a configured AI provider and returns a short, plain-language summary.
- A Summary / Full text toggle appears once a summary is generated — it controls both what's displayed and what the Copy button copies.
- Multiple API keys can be configured (comma-separated) in
.env. If one key is rate-limited, invalid, or over quota, the server automatically falls back to the next one, so a single exhausted free-tier key doesn't take the feature down. - If no keys are configured, extraction still works normally — only clicking Summarize will return an error until at least one key is added.
- See
.envfor the exact environment variable names expected byserver.js.
cd chat-extractor
npm install
npm startThen open http://localhost:3000.
npm install will download a bundled Chromium build the first time (a few
hundred MB) — that's normal. On a bare Linux server/container you may also
need to install some system libraries Chromium depends on (things like
libnss3, libatk-bridge2.0-0, libgtk-3-0, etc.). If npm start throws
an error about a missing shared library, install Google's documented
Chrome dependency list for your distro, or just install google-chrome /
chromium and point Puppeteer at it with the PUPPETEER_EXECUTABLE_PATH
env var. If you're deploying to a serverless platform (Vercel, Netlify
functions, etc.) rather than a normal long-running Node server, you'll want
to swap puppeteer for puppeteer-core + @sparticuz/chromium, since
full Puppeteer doesn't run well in those environments.
Because each request spins up a real browser page, extraction is noticeably slower than a plain-fetch approach (a few seconds instead of under a second) — that's the trade-off for actually being able to see the content.
| Platform | Domain(s) matched |
|---|---|
| ChatGPT | chatgpt.com, chat.openai.com |
| Claude | claude.ai |
| Gemini | gemini.google.com, gemini.google, g.co |
| Grok | grok.com, x.ai |
| Meta AI | meta.ai |
| Perplexity | perplexity.ai |
Note: Google's Gemini share links moved from
gemini.google.com/share/…to the standaloneshare.gemini.googledomain — both are recognized. The domain list is duplicated in two places (server.jsfor the actual fetch, andpublic/index.htmlfor the instant client-side badge), so if a platform ever changes its share domain again, update both.
- Only works on links the platform has made publicly shareable — it can't access private conversations or anything requiring login.
- No data is stored server-side; each request is rendered and processed in memory and discarded after the response is sent.
- Bot-detection countermeasures (stealth plugin, non-fatal 401/403 handling) reduce false "may be private" errors but can't guarantee every platform's protections are bypassed — if a link is genuinely gated behind a login wall, extraction will still fail as expected.
- Open the share link in your own browser with DevTools open.
- In the Console, try the same global-state probes the server checks:
window.__reactRouterDataRouter?.state?.loaderData,window.__NEXT_DATA__,window.__remixContext, etc. — see which one (if any) holds the conversation. - If none do, check the Elements panel for a repeating pattern around each
message (a
data-*attribute or consistent class name), and add/adjust a selector intryDomExtractinserver.js. - If the response comes back 401/403 but the link is definitely public, it's likely bot-management flagging the request — try again, or check whether the site changed something the stealth plugin doesn't cover yet.
- Restart the server (
npm start) and retest.
chat-extractor/
├── package.json
├── server.js # Express server: headless-render + parse + API endpoints
├── public/
│ ├── index.html # Main app UI
│ ├── privacy.html # Privacy policy page
│ ├── terms.html # Terms & conditions page
│ └── styles.css # Shared stylesheet used by all three pages above
└── README.md