Human review of the auto-drafted subject decisions.csv (650/651)

[rybesh]

Summary

sources/subjects/decisions.csv is the hand-curation surface for the new subject-terms concept scheme (topical/geographic subject headings, MARC 650/651) added in #107 (issue #105) — the exact parallel of sources/construction/decisions.csv and its review issue #104.

It was auto-seeded as a draft: mined once from the per-record zip MARC into candidates.csv/occurrences.csv, then every one of the 6,406 heading clusters was written out at include=y with a suggested conceptId/prefLabel, an auto-assigned category, and the note auto-seeded draft; review. None of these calls are human-reviewed yet — that's what this issue tracks.

Draft as committed (all include=y):

• subject-terms.ttl: 4,936 concepts (valid Turtle)
• ab:hasSubject: 3,137 books, 9,741 edges
• categories: 5,778 topical (650) · 628 geographic (651)

Why the seed differs from construction (#104)

Construction biased ambiguous → include=n (a wrong y adds junk to the scheme). Here the seed is the opposite — everything starts y — because 650/651 headings genuinely are subjects by definition, so the majority call is y and the review is about trimming and reconciling, not gating. Concretely, the reviewer:

1. Flips non-subjects to n. A construction method that leaked into 650 (some carry AAT/RBMS/LCGFT $2) → include=n, note → construction. Also noise/duplicates → n.
2. Merges twins. Give two rows the same conceptId to collapse them into one concept — folding a bare heading into its authority twin and unifying FAST/LCSH/local/foreign-language variants of one idea. The seed already merges clusters with an identical prefLabel+category into one conceptId (e.g. the FAST/LCSH/CCT variants of "Toy and movable books"); merging the remaining differently-spelled/worded twins is manual.
3. Reconciles bare headings. 3,159 of the 6,406 clusters carry no $0 in any record. Where one names a real authority, put its URI in the new exactMatch column (3,247 clusters already carry a $0 and need nothing here).
4. Fixes preferred labels. The suggested prefLabel is just the first raw heading string (e.g. Art -- 20e siècle -- Expositions); clean it up for y rows, especially new concepts.

The columns you're reviewing

cluster,labels,include,conceptId,category,prefLabel,exactMatch,note

• cluster — the reconciliation key (a $0 authority URI, or <scheme>|<normalized-heading>). Do not edit. It's the join key to occurrences.csv.
• labels — read-only: the distinct raw MARC heading strings in this cluster. This is what you're judging.
• include — y (a subject) or n (a construction method → note → construction, or noise/dup). The gate.
• conceptId — for y rows: the SKOS concept slug. Reuse an existing id to merge variants; keep/clean the unique slug for a genuinely distinct concept.
• category — for y rows: exactly topical (from 650) or geographic (from 651). Auto-seeded from the tag; correct if wrong.
• prefLabel — for y rows: the human-facing label. Ignored when conceptId reuses an existing concept (that concept's prefLabel wins).
• exactMatch — for y rows with no $0 in any record: a hand-assigned authority URI. Blank when the occurrences already supply one.
• note — the auto-rationale (auto-seeded draft; review). Delete it once you've reviewed the row so remaining notes = remaining work.

To exclude a row: set include=n and blank out conceptId, category, prefLabel, exactMatch (the build only reads y rows).

How to review

There's no reliable tiering to grep by (unlike #104) — every row is an equal-confidence draft. Suggested order:

1. Geographic first (628 rows, category=geographic) — smaller and cleaner; mostly place names. Confirm the category, merge obvious twins (New York (State) vs New York (N.Y.) are different places — don't over-merge), reconcile bare ones.
2. Topical by record count — candidates.csv is sorted by recs (how many books use the heading), so the highest-impact concepts are at the top. Work down until the long tail is clearly noise.
3. Watch for construction leakage (headings with an AAT/RBMS/LCGFT vocabulary that describe making, not aboutness) → n + → construction.

Rebuild & sanity-check after editing

make -C sources subject-terms.ttl
make -B graph/artists-books.ttl

tools/jena/bin/arq --data=graph/artists-books.ttl -q <<'SPARQL'
PREFIX ab: <https://dkglab.github.io/ns/artists-books/>
SELECT (COUNT(DISTINCT ?b) AS ?books) (COUNT(*) AS ?edges) WHERE { ?b ab:hasSubject ?c }
SPARQL

tools/jena/bin/riot --validate sources/subject-terms.ttl

Integrity rules the build assumes: every y row has a non-empty conceptId, prefLabel, and a category of topical/geographic; every n row leaves the other columns blank. Two y rows sharing a conceptId should also share a category (they merge into one concept).

Definition of done

Every auto-seeded draft; review note has been removed, and decisions.csv reflects deliberate human calls for all 6,406 clusters.

Follow-up to #107 (issue #105); models the construction review issue #104.