chore(evaluation): type strict - #1121
Conversation
|
Important Review skippedAuto reviews are limited based on label configuration. 🏷️ Required labels (at least one) (1)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
OpenAPI changes ⚪ No API surface changesNote This PR does not modify the API contract.
|
Codecov Report❌ Patch coverage is 📢 Thoughts on this report? Let us know! |
Issue
Part of #1083
Summary
TypedDictpayloads passed asdict[str, Any],Optionalvalues used without narrowing.Nonepaths the checker exposed are handled.Types:
col()inorder_by/is_not; typed score payloads (EvaluationScore,SummaryScore,TraceData) threaded through merge/summary/processing/fast; read-only args widened toSequence/Mapping;TraceData.category→NotRequired;failure_response→APIResponse[T].Behaviour (only runtime-affecting bits): skip traces with
Nonequestion_id/ missingitem_id; skip + warn on missingcosine_similarityinstead of sendingNoneto Langfuse; readstring_valuefor Langfuse text scores; fail the run explicitly when config/provider is unresolvable; raisetrace_download_failedwhenscore_trace_urlis missing.Checklist
Before submitting a pull request, please ensure that you mark these task.
fastapi run --reload app/main.pyordocker compose upin the repository root and test.Notes
Evaluation modules only — TTS/STT from #1083 not covered here.