Skip to content

Repository files navigation

fromwhere

fromwhere provides artifact provenance tracking, including support for composite artifacts like figure PNGs, publication PDFs, static notebook and website HTML, machine learning models, and more, which can be traced all the way back to their primary inputs, e.g., scripts or images generated with AI tools. This helps authors and readers know exactly how an output was created, so they can assess if it's trustworthy.

fromwhere outputs are signed and contain Content Credentials metadata. All of a project's artifact provenance information lives inside a provenance.toml file, which includes a comment at the top to prevent AI agents from tampering with the provenance information, which may indicate falsified evidence in scientific projects.

Installation

uv tool install fromwhere

The distribution, the import name, and the command it installs are all fromwhere.

Usage

If you have a script that generates a figure, run it through the fromwhere CLI:

fromwhere run -i data/raw.csv -o figures/plot.png -- uv run plot.py

The command (what comes after --) must be run with an environment manager that uses lock files so those can be tracked as part of the provenance, since they are important input information. Practically this means that the command must start with one of the following:

  • uv run (as long as a uv.lock file will be created)
  • pixi run
  • bun run
  • cargo run
  • nix develop --command (as long as a flake.nix file is present)
  • calkit xenv (use for Docker, Conda, Julia, renv environments)
  • calkit nb exec
  • calkit latex build

Declaring primary artifacts

Every chain ends somewhere. Follow a figure back far enough and you reach a file fromwhere did not make: measurements typed into a CSV, a dataset someone downloaded, a plotting script. Left alone those sit at the bottom of the chain unexplained, and fromwhere status warns about them, because every automated check above them passes — which is exactly what makes the gap easy to miss.

Declare them with:

fromwhere declare <path> --mine

This records your "attestation". AI agents should similarly declare when they've created primary artifacts so readers can assess if that is an acceptable use of generative AI — but an agent cannot be the one answering for the file, so the tool is disclosed beside a person rather than in place of one:

fromwhere declare <path> --mine --with-ai 'Claude Opus 5'
fromwhere declare <path> --author 'A Researcher' --with-ai 'Claude Opus 5'

Using --with-ai without naming a person is refused.

Both flags repeat, because an artifact rarely has exactly one author and code almost never does — a script grows through several hands and, increasingly, several models:

fromwhere declare <path> --author 'A Researcher' --author 'A Colleague' \
    --with-ai 'Claude Opus 5' --with-ai 'GitHub Copilot'

Git already knows most of this, so it can be read out instead of retyped. Every commit touching the file names an author, and Co-authored-by: trailers name everyone else — which is exactly where an agent's own signature lands:

fromwhere declare <path> --from-git-history

Whether an author is a person or a tool is a guess when it comes from git, so fromwhere declare prints what it decided and --author/--with-ai override it.

Hand-authored artifacts have authors too. A .drawio canvas is assembled from recorded panels but arranged by people and agents, so it carries a derivation chain and an author list; declaring it adds the authors without disturbing its inputs.

An attestation is the weakest thing in the record: nothing verifies it, and fromwhere says so wherever it is displayed. When the file came from somewhere a reader could actually fetch, say that instead:

fromwhere declare <path> --doi 10.5281/zenodo.1234567
fromwhere declare <path> --git github.com/myuser/myproject/path/to/data.csv@{git_rev}
fromwhere declare <path> --calkit calkit.io/myuser/myproject/path/to/data.csv@{git_rev}

The revision is required. github.com/myuser/myproject/data.csv names whatever is at that path today — a mutable claim wearing the costume of a citation. The --calkit form exists because Calkit tracks large files with DVC, so a path in a Calkit project can name data that is not in the git tree at all.

Scripts count too. A figure resting on a script nobody will claim is as unplaceable as one resting on data nobody collected, and a script a model wrote is exactly what a reader needs told.

A declaration is about people, not bytes, so editing a declared file does not invalidate it and never needs redoing. fromwhere declare records a hash too, but only as a note of what was seen — nothing checks it. fromwhere run keeps it current whenever it uses the file, so the record does not drift, and says so rather than doing it quietly:

recorded figures/cp_curve.png (signed)
   in   data/performance.csv
   in   scripts/plot_cp.py
   refreshed hash for scripts/plot_cp.py (declared; authorship unchanged)

Only declarations are eligible. A hash fromwhere wrote itself is evidence, and rewriting that is the tampering the record exists to catch — so an artifact with a command keeps reporting modified no matter how many later runs consume it.

Downloads

Some data just lives at a URL, and --url is the one origin fromwhere checks while making it:

fromwhere declare data/raw.csv --url https://example.org/datasets/raw.csv

fromwhere fetches the address. If the file is not there yet it is downloaded; if it is, fromwhere downloads it anyway and compares — and refuses if the bytes differ, naming both hashes, because either the file was edited after it was downloaded or the address has moved on and both are things you want told:

https://example.org/datasets/raw.csv does not serve what is in raw.csv.
   on disk:    sha256:5179d9e0…
   downloaded: sha256:4fd12f8c…

What lands in the record is the address and the moment:

origin_kind = "url"
origin = "https://example.org/datasets/raw.csv"
origin_fetched = "2026-08-06T02:49:24+00:00"

That pair is the claim: this address served exactly these bytes, at this time. It sits between the other two kinds and is honest about being there — a DOI resolves to a deposit that will not change, an attestation cannot be checked by anyone, and this was genuinely fetched and genuinely verified but names something mutable. So the panel shows the date beside the link every time, and says the URL can serve something else later. Being told a file came from a URL is an attestation; going and looking is evidence, and the timestamp is what keeps that distinction meaningful a year on.

Downloading uses nothing but the standard library. A provenance tool that pulled in an HTTP stack to fetch a CSV would have made itself harder to trust than the claim it records.

Where AI disclosure lives

A file that carries Content Credentials already says whether it is machine-generated, signed by whoever made it — the example's AI-generated figure really does carry Google's own "Created by Google Generative AI". There is no need to restate that in provenance.toml, and fromwhere reads it from the file instead.

But a manifest is fragile: any tool that re-encodes the bytes silently discards it, which is the whole reason fromwhere drawio import exists. And most formats cannot carry one at all — a .py, a .csv, a PDF, or a notebook's HTML have nowhere to put it. So the record holds the disclosure whenever the file cannot, and where both exist fromwhere compares them and reports the disagreement rather than quietly picking a winner. A disclosure that evaporates the first time somebody opens the image in an editor is exactly the failure worth seeing.

What can be signed

Reading credentials and writing them are different questions, and the second set is smaller. Verified against c2pa-python 0.37.4 (c2pa-rs 0.90.4), fromwhere can embed a manifest in PNG, JPEG, GIF, SVG, TIFF, WebP and WAV.

PDF is the case worth knowing about. The C2PA specification covers PDF, and other tools do sign them — but c2pa-rs cannot yet write one, though it reads them perfectly well. So a PDF signed elsewhere will have its credentials read and displayed by fromwhere; a PDF fromwhere produces is recorded but unsigned. That is a limitation of the library, not of the format, and it should lift on its own.

HTML is not applicable: c2pa does not recognize the type at all, in either direction. There is nowhere in an HTML page for a manifest to live.

This matters for whole-document outputs — a built paper is exactly the artifact you would want to hand someone with its provenance attached. So for a MyST PDF, for calkit latex build, and for the HTML a notebook renders to, fromwhere records the artifact in provenance.toml, skips signing rather than failing the build, and says which it did:

recorded paper.pdf (not signed: .pdf cannot carry Content Credentials;
                    the record in provenance.toml is its only provenance)

For those outputs the record is the only provenance there is, which is why the warning at the top of provenance.toml is not decoration.

Freshness checking

To see if a given output's inputs (including environment lock files) have changed, rendering it stale, use the status command.

fromwhere status <path>

The states are kept apart, because they have different fixes:

  • stale — an input changed; regenerate it.
  • modified — a produced output changed without going through fromwhere, so the record no longer describes the file it names. This is how tampering with an output after generation is caught.
  • upstream — sound in every direct link, but resting on one that is not.

A declared artifact is never checked against its own hash, and this is the rule worth stating plainly, because everything else follows from it:

A declaration says who is answerable for a file, not what it contained on some particular afternoon. What matters about an edit is whether it reached an output — and that is already recorded, because every output carries the hash of each input as it was when the output was made, recomputed whenever fromwhere run regenerates it.

So editing a script, a dataset, or a document source is not a finding and never requires re-declaring anything. The consequence shows up where it can be acted on: on the outputs built from the old bytes. The honest cost is that a declared file nothing consumes — no output to go stale — is not watched at all; if it is swapped, nothing notices.

  • upstream — this artifact is sound in every direct link but rests on one that is not. Staleness does not stop at the first link: a composite whose .drawio is untouched passes every direct check even when the data three steps back was edited, because nothing regenerated the panel in between.

None of them is repaired by editing provenance.toml, which is the one thing a reader in a hurry might try.

Rebuilding

Every artifact records the command that made it and the inputs it was made from, so the repair sequence is a property of the record rather than something you have to reconstruct:

fromwhere rebuild             # everything that is out of date
fromwhere rebuild <path>      # that artifact, and everything behind it
fromwhere rebuild --dry-run   # say what would happen, in order

Dependencies first, because rebuilding a document before the figure it embeds accomplishes nothing and you would end up running everything twice. Each command is repeated in-process rather than through a shell, so nothing depends on quoting the record was written to avoid.

Nothing is rebuilt that does not need it — an artifact whose inputs still hash to what the record says keeps its bytes and its signature. Anything that cannot be rebuilt is named rather than passed over: nobody can regenerate raw data, and a hand-arranged diagram is refreshed by the export that consumes it.

Signing certificates

The fromwhere run command includes the --cert option to provide a signing certificate for the Content Credentials metadata.

draw.io

fromwhere enables composite figure provenance while retaining interactive editing with draw.io. To import a PNG into a draw.io diagram, run:

fromwhere drawio import my-figure.png my-composite-figure.drawio

It's important to use the fromwhere CLI since this will embed metadata into the .drawio file.

import is an authoring step: it puts a new panel on the canvas. You should not need to run it again when a figure is redrawn, and you don't — each shape records the src it came from and the hash it had, so the export below re-embeds any panel that has moved on, keeping the position and size you gave it, and says which ones it touched. Re-running import on a panel already present is still safe: it replaces rather than adding a second copy.

To export an SVG containing the provenance information, run:

fromwhere drawio export my-diagram.drawio my-diagram.svg

Export is the build step. It refreshes stale panels first — draw.io renders the copy inside the diagram, so refreshing afterwards would publish the old pictures — then renders, signs the result with every panel as an ingredient, and re-records the diagram itself.

A panel that went in through the GUI

Sometimes a figure is already on the canvas: dragged in, pasted, or inserted with draw.io's own Insert → Image before anybody had heard of fromwhere. fromwhere cannot tell where such a shape came from, so the diagram is recorded without it and the composite rests on a picture nothing accounts for.

You can say where it came from by hand. In draw.io, select the shape and use Edit → Edit Style… — or right-click → Edit Style — and check that the shape is wrapped in an <object>; if the style dialog shows a bare mxCell, add a label first (F2, then anything), which is what makes draw.io wrap it. Then use Extras → Edit Diagram… and add one attribute to that <object>:

<object label="" src="figures/cp_curve.png" id="7">

That is the whole of it. src is a path relative to the project root, and it is the only thing fromwhere needs:

  • The hash is optional. fromwhere computes one from the bytes already embedded in the shape, so a person who can type a path but cannot work out a SHA256 is not stuck. Requiring both would silently drop the panel from the record — the diagram would look complete while resting on a figure nothing accounted for, which is the exact failure the record exists to make visible.
  • The next export fills it in. fromwhere drawio export writes the hash onto the shape once it has verified it, so the gap closes itself and the diagram becomes checkable by anything that reads it without also reading provenance.toml.

One caveat that is the reason fromwhere drawio import exists at all: draw.io re-encodes anything over 1200 px through a canvas when it embeds an image, which destroys any Content Credentials the file carried, including an AI-generation disclosure. Adding src afterwards restores the link, not the manifest. For a panel that carries credentials worth keeping, re-import it.

GIMP

To export a PNG from GIMP with provenance tracking, run:

fromwhere gimp export my-input.xcf my-output.png

MyST

fromwhere includes a MyST plugin for inspecting and checking embedded figure provenance and freshness. It also enables inspecting document-level provenance, since that's a composite artifact itself.

There is one directive, because there was only ever one question: what is this, what was it made from, and is it still true. The scope is the argument. Name a file and the panel is about that file:

:::{fromwhere} figures/composite.svg
:name: fig-performance
The composite figure.
:::

The directive is not only for figures. Point it at a .csv or .tsv and it renders the numbers as a table — numbered and cross-referenceable like any other, with the same provenance panel underneath:

:::{fromwhere} data/performance.csv
:name: tbl-performance
:rows: 25
The measurements underlying [](#fig-performance).
:::

That is where the record earns the most, because a CSV cannot carry Content Credentials at all: the line in provenance.toml is its only provenance. Long tables are truncated at :rows: (25 by default) with a note saying so — a table is for reading, and a thousand rows of it is a scroll bar.

Point it at an .html file — an interactive Plotly or Altair chart, a rendered notebook — and it is embedded in a frame, numbered and cross-referenced like any other figure:

:::{fromwhere} figures/cp_curve_interactive.html
:name: fig-cp
:height: 400px
Hover for the numbers; drag to zoom.
:::

That is the case where the record earns the most of all. HTML cannot carry Content Credentials in either direction — c2pa does not recognize the type — so for an interactive figure the line in provenance.toml is the only provenance there is. It also has to be self-contained: an artifact that fetches half of itself from a CDN at read time is not an artifact anyone can hash, because what a reader sees depends on what that URL served them.

The panel has a state between green and red, and it is the one worth knowing about: incomplete provenance. Every hash matches, nothing is stale, nothing was tampered with — and something in the chain is a file nobody has claimed:

⚠️ Figure has incomplete provenance — nothing accounts for scripts/plot_cp.py
   To fix: fromwhere declare scripts/plot_cp.py --mine [--with-ai <tool>]

A warning rather than an error, because nothing is broken and no output needs regenerating; what is missing is a person's statement, and only a person can supply it. It gets its own state because every automated check passes, which is exactly what makes the gap easy to miss — and a panel that printed a green tick with the finding folded away inside it would be the thing doing the hiding. The document-level panel says the same, so the two cannot appear to disagree.

Anything that is none of those renders as its filename with the panel attached, and the panel names what it is looking at: Figure, Table, or Artifact. The extension decides which, and :kind: overrides it for the .dat that is really delimited, or the .svg that is a diagram of the method and has no business being numbered as a figure.

Name no file and the panel is about the document you are reading — a composite artifact in its own right, and the same question one scope wider. :artifact: names the document's own output(s):

:::{fromwhere}
:artifact: _build/html/index.html, _build/exports/paper.pdf
:table: true
:graph: true
:::

(:kind: document says it outright, for a block that has a path and wants the document anyway.)

Naming them does two things. It shows the command that rebuilds the document, read from the record so it cannot drift from what actually produced the file. And it leaves those outputs out of the panel's own freshness tally — a document cannot honestly report on itself from the inside, because while the page is being written its recorded hash still describes the previous build. Without this the panel reads "out of date" on every single build and stops meaning anything. fromwhere status checks them from outside, where the answer is settled.

The table lists outputs — anything the project made, including intermediates like a .drawio — and for each one names the input responsible when it is behind:

Output Built from State
figures/cp_curve.png 3 ⚠️ scripts/plot_cp.py changed
figures/composite.svg 1 ⚠️ waiting on figures/cp_curve.png

Sources are what those answers point at, not rows of their own. Asking whether a plotting script is "up to date" has no answer — nothing produces it — and listing it green above the figure it just broke is the confusion this avoids.

One caveat about live preview. myst start re-renders when one of its own sources changes: markdown, myst.yml, a linked image. Editing a script or a dataset is invisible to it, so the panels keep showing the previous render and a stale figure looks current for as long as the tab is open. The example's make serve works around this by watching everything fromwhere records as an input and touching the document when any of it moves.

Quarto

The same panels, in Quarto. Install the extension into a project and wire it into _quarto.yml:

fromwhere quarto install
filters:
  - at: pre-ast
    path: fromwhere

It comes from the package rather than from quarto add because the Lua filter and the fromwhere-quarto executable it spawns speak a protocol private to fromwhere, and installing them together is what keeps the two the same version.

at: pre-ast is not decoration. Quarto builds callouts, figure numbers, and cross-references in its own filters, so a panel emitted after them is a grey box sitting under an unnumbered picture — it renders, and everything about it is subtly wrong.

A fenced div does what MyST's directive does, because it is the one Quarto construct that takes a path, options, and a caption that is parsed — so citations and cross-references inside a caption keep working:

::: {.fromwhere src="figures/composite.svg" #fig-performance width="95%"}
Power coefficient against tip speed ratio.
:::

One Pandoc habit to know about, because scientific captions walk into it constantly: a caption beginning (a) is parsed as an ordered list, not as a panel label. Pandoc's fancy_lists treats (a), (1) and a. at the start of a block as list markers, and the result is a caption turned into an <ol> with Quarto's own "Figure 1:" prefix chopped into list items. Nothing warns you. Bold the label — **(a)** Power coefficient… — or escape it as \(a\). This is plain Quarto behavior rather than anything fromwhere does, and it applies to every caption in the document.

Point it at a .csv or .tsv and it renders the numbers as a table, numbered and cross-referenceable like any other, with the same provenance panel underneath. Anything that is neither a picture nor a table renders as its filename with the panel attached:

::: {.fromwhere src="data/performance.csv" #tbl-performance rows="25"}
The measurements underlying [@fig-performance].
:::

Name no src and the panel is about the document, exactly as in MyST. It takes the same artifact option, naming the document's own output(s) so it can show the rebuild command and leave them out of its own freshness tally:

::: {.fromwhere artifact="_site/index.html" table="true" graph="true"}
:::

A .fromwhere-provenance div is no longer a thing, and rather than rendering as an anonymous grey box it says what replaced it — an unknown class is silence, and silence is the failure this panel exists to prevent.

Everything that decides what to say — the chain, the AI disclosure, the headline, the truncation note — is the same Python that answers fromwhere status and renders the MyST panels. The Lua half only turns those nodes into Pandoc's AST, so a document built either way says the same thing.

Two things are Quarto-specific and worth knowing. The derivation graph is drawn with Quarto's own bundled Mermaid, asked for as an HTML dependency: Quarto renders Mermaid in its engine, before pandoc runs, so a filter cannot produce a diagram the ordinary way. That works in HTML; in a format with no Mermaid runtime — PDF, docx — the graph falls back to its source rather than disappearing.

The second is live preview, and it has a sharp edge worth knowing before you hit it. quarto preview, like myst start, re-renders only when one of its own sources changes: the .qmd, _quarto.yml, a linked image. Editing a plotting script is invisible to it, so the panels keep showing the previous render and a stale figure looks current for as long as the tab is open. The example's make preview closes that by watching everything fromwhere records as an input and touching the document when any of it moves.

That workaround does not work under VS Code's Quarto extension, which launches quarto preview --no-watch-inputs and re-renders only when you save the document it is previewing. With input watching off, nothing on the filesystem can trigger a render — not the script, and not a touched .qmd. Use make preview in a terminal, or save the .qmd after editing a script.

Examples

Five worked projects, each runnable, in examples/:

Example What it shows
myst The MyST plugin, a draw.io composite, and an AI-generated panel whose disclosure travels in its own Content Credentials
quarto The Quarto extension: Plotly panels, an interactive figure, a composite, a CSV as a cross-referenceable table — under pixi, so the lock pins Quarto too
calkit fromwhere wrapping a Calkit stage command, with provenance.toml beside a dvc.lock — and why signing belongs at the boundary
snakemake The same arrangement under Snakemake, and what a build cache is for versus what a record is for
astra ASTRA declaring the decision space while fromwhere records which options actually produced the bytes

The last three share a shape worth naming. fromwhere wraps the stage command, never the workflow manager: wrapping the manager would make fromwhere the entrypoint, record one enormous artifact, and lose the per-figure chain that is the whole point. And in each of them the repair for a stale artifact is the manager's own command, not fromwhere rebuild — two components that each claim to know how to rebuild a project, from different graphs, will drift.

What the lock file does not cover

fromwhere records the lock of the environment the command ran in. Anything that wraps that command from the outside is not in it — fromwhere itself, the workflow manager, and any tool they shell out to. That is a real limit on what the record means, and it is worth knowing where it bites:

  • MyST needs Node. mystmd on PyPI is a wrapper around a JavaScript CLI. The JS ships in the wheel so the MyST version is pinned, but the runtime is taken from PATH — and with no Node installed it stops and asks interactively whether to install one, which hangs a CI build rather than failing it. MYSTMD_ALLOW_NODEENV=1 answers that up front.
  • draw.io is a desktop app. No conda or PyPI package exists, so nothing can pin it, and an exported composite rests on a version the record cannot name.
  • Quarto can be pinned, and should be. The Quarto example uses pixi for exactly this reason: rendering the document is a recorded step, so a lock that did not include the renderer would be asserting an environment it does not control.

The general remedy is to choose an environment manager that can hold the whole toolchain — pixi reaches conda-forge, which has Quarto and Node — rather than one that can only hold the Python part. Each example's README says which of these applies to it.

Releases

Packages

Contributors

Languages