Problem
UK Chat should eventually be able to answer questions such as:
- what does
income_tax calculate?
- which entity does Universal Credit live on?
- what variables are available for a programme?
- what formula/source definition is associated with a variable?
However, the chat backend currently uses policyengine_uk_compiled as its calculation source of truth, and the compiled package does not yet expose a variable metadata/formula catalog. Pulling variable metadata and formulas from the separate policyengine_uk package would risk showing users metadata that diverges from the compiled engine actually used for calculations.
Dependency
Blocked on PolicyEngine/policyengine-uk-v2#106 exposing machine-readable variable metadata from policyengine_uk_compiled, especially:
- canonical variable name
- label/title
- entity
- description/definition
- input/output/intermediate status where available
- formula/source reference where feasible
Suggested scope after the dependency lands
- Add a
lookup_variable model-facing tool backed by policyengine_uk_compiled metadata.
- Support exact variable names and deterministic natural-language matching.
- Return
needs_confirmation with ranked options when the string match is ambiguous.
- Clearly label formula/source fields according to the compiled package contract.
- Add unit tests for exact, natural-language, ambiguous, and unknown variable queries.
- Add eval coverage across
tool_contract, trajectory, and answer so model responses do not over-claim source/formula certainty.
Out of scope
Do not source variable formulas from policyengine_uk in UK Chat unless the product explicitly accepts that divergence risk.
Problem
UK Chat should eventually be able to answer questions such as:
income_taxcalculate?However, the chat backend currently uses
policyengine_uk_compiledas its calculation source of truth, and the compiled package does not yet expose a variable metadata/formula catalog. Pulling variable metadata and formulas from the separatepolicyengine_ukpackage would risk showing users metadata that diverges from the compiled engine actually used for calculations.Dependency
Blocked on PolicyEngine/policyengine-uk-v2#106 exposing machine-readable variable metadata from
policyengine_uk_compiled, especially:Suggested scope after the dependency lands
lookup_variablemodel-facing tool backed bypolicyengine_uk_compiledmetadata.needs_confirmationwith ranked options when the string match is ambiguous.tool_contract,trajectory, andanswerso model responses do not over-claim source/formula certainty.Out of scope
Do not source variable formulas from
policyengine_ukin UK Chat unless the product explicitly accepts that divergence risk.