Problem
Ask ALYF Settings currently exposes a single model field. In practice the workload is bimodal: many simple lookups/lists, plus some compute-/grounding-heavy or ambiguous questions. A single model forces a trade-off between speed and reliability.
Proposal
Optional model routing: a cheap, fast model for simple lookups/lists, with escalation to a stronger model for complex/ambiguous questions. First-pass classification could be a lightweight heuristic or a small model call.
Note on priority
Lower priority than curated query tools (see the related issue). If the hard queries become deterministic tool calls, even a fast/cheap model answers them correctly, which largely removes the need for routing.
Context
Feedback from production use with ERPNext v15 + Aakvatech PropMS.
Problem
Ask ALYF Settingscurrently exposes a singlemodelfield. In practice the workload is bimodal: many simple lookups/lists, plus some compute-/grounding-heavy or ambiguous questions. A single model forces a trade-off between speed and reliability.Proposal
Optional model routing: a cheap, fast model for simple lookups/lists, with escalation to a stronger model for complex/ambiguous questions. First-pass classification could be a lightweight heuristic or a small model call.
Note on priority
Lower priority than curated query tools (see the related issue). If the hard queries become deterministic tool calls, even a fast/cheap model answers them correctly, which largely removes the need for routing.
Context
Feedback from production use with ERPNext v15 + Aakvatech PropMS.