Add a pluggable, trend-driven DSE search agent alongside GridSearchAgent

[shreyaskommuri]

Context

@podkidyshev — separate thread from #985 and #919, filing this on its own since it's a different part of the codebase and I don't want to conflate it with either of those still-open discussions.

While building a small local companion tool for tuning CloudAI benchmark configs (CloudAI-Autotune, public, linked below), I ran into the same problem CloudAI's own DSE stack has: sweeping is exhaustive grid search only, no cheaper option exists for problems where a smarter search could get close to the best config in a fraction of the trials. I ended up building and shipping a working, tested search strategy there, and since configurator/base_agent.py already defines a generic BaseAgent interface for exactly this kind of pluggability, porting the working piece back seemed like a natural, scoped contribution rather than a from-scratch design.

Flagging up front: I used an AI coding assistant (Claude Code) both for the code archaeology behind this write-up (tracing BaseAgent/GridSearchAgent/define_action_space here, and my own recommender.py/optimizer.py over in Autotune) and for drafting this issue text. Happy to go deeper on any part of it.

Problem

src/cloudai/configurator/base_agent.py defines a generic BaseAgent interface (select_action / update_policy / configure), but grid_search.py's GridSearchAgent — exhaustive combinatorial search over the action space — is the only implementation that ships. Every DSE sweep today pays for the full cartesian product of define_action_space(), with nothing in between "try everything" and "guess."

Proposal

A small, first-step BaseAgent implementation — deliberately not the more powerful version, see "Out of scope" below.

A new agent (naming TBD, e.g. TrendSearchAgent) that, instead of enumerating every combination:

1. Starts from a baseline point (same start_action convention BaseAgent already has).
2. After each step, looks at one knob at a time: is the metric trend (via update_policy's reward) still improving as that knob moves in its current direction, holding the others fixed at the best point seen so far? If yes, keep moving that knob the same way (doubling/halving-style step); if the trend reverses, back off to the best point observed and try a different knob.
3. Picks the knob to move next by preferring whichever one's own trend still looks favorable, falling back to any knob with a valid untried step otherwise.

This is the same logic already shipped and tested in CloudAI-Autotune's recommend_next / suggest_untried_combo (single-knob and multi-knob-one-at-a-time versions):
https://github.com/shreyaskommuri/CloudAI-Autotune/blob/6f07b65/autotune/recommender.py#L114
https://github.com/shreyaskommuri/CloudAI-Autotune/blob/6f07b65/autotune/recommender.py#L331

Because BaseAgent.run() already drives a single continuous select_action → env.step() → update_policy() loop, the agent can just keep tried (action, reward) pairs in memory as it runs — none of the cross-invocation state-tracking my CLI version needed (each Autotune command is a separate process, so I had to persist pending suggestions in a DB table; a live BaseAgent doesn't have that problem). No new dependency: it's plain arithmetic, same as GridSearchAgent.

Needs a stopping criterion, unlike GridSearchAgent (which naturally stops when the grid is exhausted) — proposing a max_trials field on a new TrendSearchAgentConfig(BaseAgentConfig).

Open questions

• Action space value types. CloudAIGymEnv.define_action_space() returns Dict[str, list[Any]] (cloudai_gym.py). Autotune's version assumes numeric, orderable knob values (doubling/halving needs a number line). Is that a safe assumption for every workload's param_space today, or can sweepable values be non-numeric (e.g. categorical string choices) for some CmdArgs fields? If the latter, the agent would need to fall back to something else (or simply skip) non-numeric knobs.
• Stopping criterion shape. Fixed max_trials budget, or terminate early once no knob has a favorable-trend direction left to try? Could support both, but want to confirm which matters more before building it.
• Naming/placement — configurator/ alongside grid_search.py, matching its structure?

Out of scope (this issue)

• A real fitted response-surface search (regression/least-squares over observed points, replacing the trend heuristic with an actual objective function) — I also have a working, tested version of this in Autotune (recommend --joint --optimize):
  https://github.com/shreyaskommuri/CloudAI-Autotune/blob/6f07b65/autotune/optimizer.py
  It's a meaningfully bigger diff (needs cloudai.util.lazy_imports.lazy.np per the banned-module-level-imports lint rule, a fit-or-fallback path, its own tests) and a stronger capability, so I'd rather propose it as an explicit follow-up once this simpler agent is in and reviewed, than bundle both into one PR.
• Reporting-side changes (e.g. a Pareto frontier in DSEReport instead of a single scalar "best" step) — separate, unrelated proposal, not bundled here.
• Not touching GridSearchAgent, handle_dse_job, or anything from the Add machine-readable scenario summary report #919 / Support relative-path test references and lazy-load only referenced test/hook tomls #985 threads — this is purely additive.

No PR yet per CONTRIBUTING.md — filing this first to confirm direction before writing code.

[shreyaskommuri]

One thing I want to flag proactively rather than have you point out after review: while checking for duplicates before filing this, I came across #956, which references a BO agent's _detect_log_scale and says encoding logic should be "reused by GA / MAB / RL agents." I don't see any of those (BO/GA/MAB/RL) in configurator/ on main today — only GridSearchAgent. Given #919 got parked partly because of an internal initiative covering similar ground, this smells like the same thing: if there's already internal work adding non-grid-search agents, this proposal may be redundant, or at least worth coordinating with rather than duplicating. Wanted to ask directly up front instead of building something that already exists on your side.

[podkidyshev]

Yeah I must say it's a hard decision of CloudAI:

• we have internal agents that truly provide optimization of searches
• we deliberately don't include any real agents in this repo (even simple ones)

product decision ¯_(ツ)_/¯

[shreyaskommuri]

Appreciate the straight answer. Makes sense, no interest in fighting a deliberate product decision. Closing this out. If that internal agent work ever does get open-sourced, happy to help wire it into BaseAgent at that point.