Add Talos runtime evaluation contract

[Pro777]

Summary

• add structured Talos models for policy, snapshots, state, and evaluations
• split runtime persistence/evaluation helpers out of the pure scorer module
• add tests for runtime state, history, policy, and evaluation behavior

Testing

• . .venv/bin/activate && pytest --cov=talos --cov-report=term-missing --cov-report=xml

Summary by CodeRabbit

• New Features

  • Customizable policy settings to adjust enforcement thresholds
  • Snapshot evaluation now returns structured results with computed level and human-readable reasons
  • Persistent save/load for policy, state, status, and history

• Refactor

  • State persistence and evaluation logic consolidated into a dedicated runtime area

• Bug Fixes

  • More robust persistence with safe writes and graceful fallbacks/logging on failure

• Tests

  • Expanded unit tests covering models, runtime IO, evaluation, and policy effects

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 451f44f4-fc14-4dc3-910f-2b873c47f0df

📥 Commits

Reviewing files that changed from the base of the PR and between 5d1d01b and ff26001.

📒 Files selected for processing (4)

• talos/models.py
• tests/test_models.py
• tests/test_runtime.py
• tests/test_scorer.py

---

Walkthrough

Adds typed Talos models, makes compute_talos_level policy-aware, introduces talos.runtime for centralized atomic persistence and evaluation (explain_level/evaluate_snapshot/IO helpers), updates package exports to re-export models and runtime helpers, and adjusts tests to exercise the new modules and behaviors.

Changes

Talos Architecture Refactoring

Layer / File(s)	Summary
Data Models Contract talos/models.py	Adds dataclass models: TalosPolicy, TalosSnapshot, TalosState, TalosEvaluation with from_dict()/to_dict() and validation for list fields.
Scorer Refactor (policy-aware) talos/scorer.py	Removes persistence; adds DEFAULT_POLICY and updates compute_talos_level(..., policy: Optional[TalosPolicy]) to use policy-driven thresholds.
Runtime: imports & file constants talos/runtime.py	Adds runtime module, logger, and environment-configurable file path constants for cycles/status/history/policy.
Runtime: atomic IO & helpers talos/runtime.py	Implements atomic JSON write/read helpers and load/save/append functions for state, policy, status, history, and snapshot parsing with broad error handling.
Runtime: explain level talos/runtime.py	Implements explain_level(snapshot, level, policy) to produce human-readable reason strings including global_max special-case.
Runtime: evaluate snapshot & cycles talos/runtime.py	Implements evaluate_snapshot to compute level via compute_talos_level, track cycles/streaks, populate TalosEvaluation, and return reasons.
Runtime: state mapping & compatibility talos/runtime.py	Adds next_state_from_evaluation and backward-compatible load_cycles/save_cycles wrappers.
Package API Surface Expansion talos/__init__.py	Re-exports model types and runtime helpers and updates __all__.
Cycles tests migrated to runtime tests/test_cycles.py	Updates tests to exercise talos.runtime for path resolution, load/save round-trips, atomic writes, and updated failure logging.
Model serialization tests tests/test_models.py	Adds round-trip and validation tests for models, including rejection of non-list fields.
Runtime behavior tests tests/test_runtime.py	Adds tests for evaluation output, state persistence, history appends, policy loading, status saving, snapshot reading, explain_level edge case, and failure logging.
Scorer policy tests tests/test_scorer.py	Adds tests asserting compute_talos_level respects TalosPolicy friction and write-block thresholds.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

• Spitfire-Cowboy/talos#1: Introduced the original scorer and cycles persistence that this PR refactors and relocates.

🐰 I hopped through models, neat and bright,
Policies, snapshots, states in sight.
Runtime hums, evaluations sing,
Exports shine — I prance and spring! 🥕✨

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 6.25% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the primary change: introducing structured data models and runtime evaluation functions that formalize the Talos evaluation contract.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feature/runtime-evaluation-contract

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (2)

tests/test_scorer.py (1)

13-15: 💤 Low value

Consider adding a test for write_block_cycles policy override.

This test validates friction_ratio from the policy but doesn't exercise write_block_cycles. Adding a test that passes a policy with a different write_block_cycles value (e.g., write_block_cycles=3) would confirm both policy fields affect the calculation as intended.

📝 Example test for `write_block_cycles`

def test_policy_can_raise_write_block_threshold() -> None:
    # With write_block_cycles=3, cycles=2 should yield level 2 instead of level 3
    assert compute_talos_level(10, 10, [], 0, 2, policy=TalosPolicy(write_block_cycles=3)) == 2

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_scorer.py` around lines 13 - 15, Add a unit test to exercise the
TalosPolicy.write_block_cycles override: create a test (e.g.,
test_policy_can_raise_write_block_threshold) that calls compute_talos_level with
cycles set lower than the default threshold but below the policy's
write_block_cycles (for example, cycles=2 and
policy=TalosPolicy(write_block_cycles=3)) and assert the returned level reflects
the policy override (expected level 2 per the example). Locate usages around
test_policy_can_raise_friction_threshold and add this new test so both
friction_ratio and write_block_cycles fields are validated.

talos/models.py (1)

45-48: 💤 Low value

Type coercion with list() could mask invalid input types.

Lines 45 and 48 wrap list() around the result of .get(). While this defensively creates a new list (avoiding mutation), it will also silently convert non-list iterables:

• list("abc") → ["a", "b", "c"]
• list((1, 2)) → [1, 2]

If stricter type checking is desired, consider explicit validation:

at_cap_val = payload.get("at_cap_projects", [])
if not isinstance(at_cap_val, list):
    raise TypeError(f"at_cap_projects must be a list, got {type(at_cap_val)}")
at_cap_projects = at_cap_val.copy()

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@talos/models.py` around lines 45 - 48, The current assignments use
list(payload.get(...)) which will silently convert non-list iterables; instead,
fetch the raw values from payload (e.g., at_cap_val =
payload.get("at_cap_projects", []) and proj_val = payload.get("projects", [])),
validate with isinstance(..., list) and raise a TypeError if not a list, then
assign copies (e.g., at_cap_projects = at_cap_val.copy(), projects =
proj_val.copy()) so callers cannot mutate internal state; update the code paths
that set at_cap_projects and projects in talos/models.py to follow this pattern.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/test_models.py`:
- Around line 9-22: The test_snapshot_round_trip currently only asserts source
== "rowan", leaving serialization unchecked; update test_snapshot_round_trip to
assert the full round-trip by comparing the original dict input to
TalosSnapshot.from_dict(...) with snapshot.to_dict(), ensuring fields like
wip_total, global_max, at_cap_projects, backlog_total, backlog_delta, projects,
source, and timestamp match; use TalosSnapshot.from_dict and snapshot.to_dict to
perform the equality check (or assert each key individually) so the
serialization/deserialization contract is fully validated.
- Around line 30-42: The test_evaluation_to_dict currently only asserts the
"reasons" field; update test_evaluation_to_dict to validate the full
serialization by comparing the output of TalosEvaluation.to_dict() against the
expected dict built from the same constructor inputs (level, previous_level,
cycles_at_level, wip_total, global_max, backlog_total, backlog_delta,
at_cap_projects, reasons). Locate the test function test_evaluation_to_dict and
the TalosEvaluation.to_dict usage and add assertions that each key/value pair in
the produced dict equals the corresponding input value (or perform a single
equality check against the complete expected dict) to ensure all fields are
covered.

---

Nitpick comments:
In `@talos/models.py`:
- Around line 45-48: The current assignments use list(payload.get(...)) which
will silently convert non-list iterables; instead, fetch the raw values from
payload (e.g., at_cap_val = payload.get("at_cap_projects", []) and proj_val =
payload.get("projects", [])), validate with isinstance(..., list) and raise a
TypeError if not a list, then assign copies (e.g., at_cap_projects =
at_cap_val.copy(), projects = proj_val.copy()) so callers cannot mutate internal
state; update the code paths that set at_cap_projects and projects in
talos/models.py to follow this pattern.

In `@tests/test_scorer.py`:
- Around line 13-15: Add a unit test to exercise the
TalosPolicy.write_block_cycles override: create a test (e.g.,
test_policy_can_raise_write_block_threshold) that calls compute_talos_level with
cycles set lower than the default threshold but below the policy's
write_block_cycles (for example, cycles=2 and
policy=TalosPolicy(write_block_cycles=3)) and assert the returned level reflects
the policy override (expected level 2 per the example). Locate usages around
test_policy_can_raise_friction_threshold and add this new test so both
friction_ratio and write_block_cycles fields are validated.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: c52d9dbb-b60d-4763-b52a-217932f1c5b7

📥 Commits

Reviewing files that changed from the base of the PR and between c4e3bab and 5d1d01b.

📒 Files selected for processing (8)

• talos/__init__.py
• talos/models.py
• talos/runtime.py
• talos/scorer.py
• tests/test_cycles.py
• tests/test_models.py
• tests/test_runtime.py
• tests/test_scorer.py