Skip to content

Add Internet Archive search-query bulk import - #96

Open
dbvisel wants to merge 5 commits into
asteasolutions:mainfrom
dbvisel:feature/ia-search-import
Open

Add Internet Archive search-query bulk import#96
dbvisel wants to merge 5 commits into
asteasolutions:mainfrom
dbvisel:feature/ia-search-import

Conversation

@dbvisel

@dbvisel dbvisel commented Aug 18, 2026

Copy link
Copy Markdown

Summary

  • Recognizes Internet Archive search URLs (e.g. https://archive.org/search?query=subject%3A%22Gondavalekar%22) and opens the existing IA import picker so the user can choose which results to import, up to the shared MAX_SELECTION cap.
  • Reuses the existing HandleIAImportDialog/IAImport/import-items-list picker mechanism (originally built for IACollection/IAPlaylist), adding a new IASearchCollection member and a sibling list component that queries the raw search string instead of collection:<id>.
  • Imports by reference (embeds the archive.org item), consistent with how IACollection/IAPlaylist already behave.

Test plan

  • tsc --build passes in core and client
  • eslint passes in core and client
  • prettier --check passes in core and client
  • Manually paste an archive.org/search?query=... URL into a tapestry and confirm the picker opens, lists results, "select all" respects the 50-item cap, and confirmed items import correctly

Recognizes archive.org search URLs (e.g. ?query=subject%3A%22X%22) and
opens the existing IA import picker to let users choose which search
results to import, reusing the IACollection/IAPlaylist picker
infrastructure and by-reference item creation.
@zmarinov-astea
zmarinov-astea self-requested a review August 19, 2026 13:22
Comment thread client/src/stage/item-factories.ts Outdated
Comment thread client/src/stage/item-factories.ts Outdated
Comment thread core/src/internet-archive.ts Outdated
- Delete IACollectionList; consolidate into IASearchList, parameterized by a
  raw query string. IACollection now passes collection:<id> through the same
  component instead of a near-duplicate sibling.
- Always exclude collection-type results (AND NOT mediatype:collection) via a
  shared excludeIACollections() helper, applied to both the list's query and
  the count probe - previously missing from the search path entirely.
- Merge iaCollectionFactory and iaSearchCollectionFactory into one iaFactory
  handling every IA URL shape.
- parseIASearchURL -> parseIASearchURLQuery, returning string | null instead
  of a single-field wrapper object.

Addresses all actionable comments from
asteasolutions#96 (review)
Comment thread core/src/internet-archive.ts Outdated
Comment thread client/src/stage/item-factories.ts Outdated
Comment thread core/src/internet-archive.ts Outdated
Comment thread core/src/internet-archive.ts
Dan Visel and others added 3 commits August 21, 2026 13:58
- fetchIASearchCount no longer calls excludeIACollections internally;
  the one call site (iaFactory) now passes the already-filtered query,
  so the count function has no special knowledge of collection
  exclusion.
- Removed three comments describing straightforward code rather than
  a non-obvious constraint (the iaFactory summary, the emptyPlaceholder
  field doc, the excludeIACollections rationale).
- Added tab= support to parseIASearchURLQuery: archive.org/search's
  media-type tabs (Books/Documents, Video, Audio, Software, Images,
  Live Music, Data) map directly onto the existing IAMediaType values
  and now constrain the query with an added `mediatype:` clause -
  verified against the real site (which tab produces which tab= value)
  and the real advancedsearch.php API (the composed query returns the
  correctly filtered result). Deliberately not honoring tab=collection
  (would always return zero results against our own collection
  exclusion) or tab=radio/tv/fulltext (no corresponding mediatype
  value - these aren't part of the standard media-type facet at all).
We don't support image types or full-text search filtering yet, so
honoring the tab= parameter would only be a partial implementation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants