How much of a frontier model's tool-calling ability can you hand to a model that costs 1/60th as much? Context engineering -> prompt compilation -> inference scaffolds -> distillation, each measured as % of the frontier gap closed, on a hidden split, for under $20.

Cascaded Framework for Cost-Aware LLM Serving

Sample code for two-tier LLM inference (cascading) on Amazon Bedrock, using a single Bedrock Converse API surface. NVIDIA Nemotron Nano handles routine support-ticket classification on every request; Anthropic Claude Sonnet handles cases the routing logic flags as harder. Includes a bake-off harness, sample data, and tests. Next.js + TypeScript.

A self-distilling neuro-symbolic cascade that amortises LLM cost across knowledge-graph QA and regulatory-compliance checking, with auditable Datalog proof trees.

In-browser LLM router over 12 models from 6 providers, with a cheapest-first cascade that grades and escalates

🤖 Production-grade Hybrid AI Assistant for Avito Messenger (Dental & Medical Clinics) with Zero-Hallucination Veto System and Telegram Admin Panel

A leakage-controlled, paired-bootstrap-rigorous study of whether an LLM's internal TopK Sparse Autoencoder (SAE) feature spaces encode a difficulty-predictive routing signal. Benchmarked on HellaSwag (rigorous predictive null) and SQuAD continuous perplexity (positive Pareto cascade), with Platt recalibration and causal ablatings.

Authoring checklist for deliberate model downshifting: a strong model writes a next-session prompt a cheaper model can execute, escalating only at pre-named forks.