Skip to content

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

12 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

ChatPluck

Live Demo

Paste a public share link (ChatGPT, Claude, Gemini, Grok, Meta AI, Perplexity) and get the full conversation back as clean text and also you can summarize.

How it works

These share pages are client-rendered SPAs — the raw HTML is just an empty app shell, and the actual conversation is loaded into the page by JavaScript after it mounts. A plain fetch() never sees it, no matter how good the parsing logic is. So instead, the server renders the page with a real headless browser (Puppeteer, via puppeteer-extra + the stealth plugin — see below) and then:

  1. Navigates to the URL and waits for the network to go idle so client-side data loading finishes.
  2. Waits (best-effort) for a message-shaped element to appear, then auto-scrolls in case anything is lazily rendered.
  3. Attempt 1 — in-page state extraction: runs a script inside the rendered page that walks known client-side state containers (window.__reactRouterDataRouter, __NEXT_DATA__, __remixContext, __APOLLO_STATE__, __INITIAL_STATE__, etc.) looking for anything shaped like a list of chat turns.
  4. Attempt 2 — structured HTML extraction: falls back to any JSON a platform still embeds directly in the rendered HTML.
  5. Attempt 3 — DOM fallback: scans the rendered HTML for elements that look like chat turns (data-message-author-role, class names containing "message"/"turn", etc.).
  6. Formats whatever it found into a labeled transcript (--- YOU --- / --- CLAUDE --- / etc.) and returns it to the browser.

Bot-detection handling

Sites like claude.ai sit behind bot-management (Cloudflare, etc.) that can return a 403 on the initial document response to a headless browser, even for a genuinely public share link. Two things address this:

  • puppeteer-extra-plugin-stealth patches over the common headless tells (missing navigator.webdriver spoofing, no chrome runtime object, etc.) that trigger bot detection in the first place.
  • A 401/403 status no longer causes an instant hard failure. The server keeps trying to extract content regardless, and only reports "may be private" if extraction still comes up empty afterward — so a false-positive block page doesn't kill a link that's actually public.

Summarize feature

Once a conversation is extracted, an optional Summarize button sends the transcript to a configured AI provider and returns a short, plain-language summary.

  • A Summary / Full text toggle appears once a summary is generated — it controls both what's displayed and what the Copy button copies.
  • Multiple API keys can be configured (comma-separated) in .env. If one key is rate-limited, invalid, or over quota, the server automatically falls back to the next one, so a single exhausted free-tier key doesn't take the feature down.
  • If no keys are configured, extraction still works normally — only clicking Summarize will return an error until at least one key is added.
  • See .env for the exact environment variable names expected by server.js.

Setup

cd chat-extractor
npm install
npm start

Then open http://localhost:3000.

A note on Puppeteer

npm install will download a bundled Chromium build the first time (a few hundred MB) — that's normal. On a bare Linux server/container you may also need to install some system libraries Chromium depends on (things like libnss3, libatk-bridge2.0-0, libgtk-3-0, etc.). If npm start throws an error about a missing shared library, install Google's documented Chrome dependency list for your distro, or just install google-chrome / chromium and point Puppeteer at it with the PUPPETEER_EXECUTABLE_PATH env var. If you're deploying to a serverless platform (Vercel, Netlify functions, etc.) rather than a normal long-running Node server, you'll want to swap puppeteer for puppeteer-core + @sparticuz/chromium, since full Puppeteer doesn't run well in those environments.

Because each request spins up a real browser page, extraction is noticeably slower than a plain-fetch approach (a few seconds instead of under a second) — that's the trade-off for actually being able to see the content.

Supported platforms & share-link domains

Platform Domain(s) matched
ChatGPT chatgpt.com, chat.openai.com
Claude claude.ai
Gemini gemini.google.com, gemini.google, g.co
Grok grok.com, x.ai
Meta AI meta.ai
Perplexity perplexity.ai

Note: Google's Gemini share links moved from gemini.google.com/share/… to the standalone share.gemini.google domain — both are recognized. The domain list is duplicated in two places (server.js for the actual fetch, and public/index.html for the instant client-side badge), so if a platform ever changes its share domain again, update both.

Important limitations

  • Only works on links the platform has made publicly shareable — it can't access private conversations or anything requiring login.
  • No data is stored server-side; each request is rendered and processed in memory and discarded after the response is sent.
  • Bot-detection countermeasures (stealth plugin, non-fatal 401/403 handling) reduce false "may be private" errors but can't guarantee every platform's protections are bypassed — if a link is genuinely gated behind a login wall, extraction will still fail as expected.

If a specific platform still doesn't work

  1. Open the share link in your own browser with DevTools open.
  2. In the Console, try the same global-state probes the server checks: window.__reactRouterDataRouter?.state?.loaderData, window.__NEXT_DATA__, window.__remixContext, etc. — see which one (if any) holds the conversation.
  3. If none do, check the Elements panel for a repeating pattern around each message (a data-* attribute or consistent class name), and add/adjust a selector in tryDomExtract in server.js.
  4. If the response comes back 401/403 but the link is definitely public, it's likely bot-management flagging the request — try again, or check whether the site changed something the stealth plugin doesn't cover yet.
  5. Restart the server (npm start) and retest.

Project structure

chat-extractor/
├── package.json
├── server.js         # Express server: headless-render + parse + API endpoints
├── public/
│   ├── index.html     # Main app UI
│   ├── privacy.html    # Privacy policy page
│   ├── terms.html      # Terms & conditions page
│   └── styles.css      # Shared stylesheet used by all three pages above
└── README.md

About

Paste a public share link (ChatGPT, Claude, Gemini, Grok, Meta AI, Perplexity) and get the full conversation back as clean text and also you can summarize.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Packages

Contributors

Languages