Skip to content

Fix perf test build against pre-6.0 MDS baselines - #4509

Open
cheenamalhotra wants to merge 4 commits into
mainfrom
dev/cheena/fix-perf-baseline-api-gating
Open

Fix perf test build against pre-6.0 MDS baselines#4509
cheenamalhotra wants to merge 4 commits into
mainfrom
dev/cheena/fix-perf-baseline-api-gating

Conversation

@cheenamalhotra

Copy link
Copy Markdown
Member

Summary

The perf pipeline's baseline pass compiles the PerformanceTests project against a released MDS package (-p:ReferenceType=Package -p:MdsPackageVersion=<version>). JsonVsVarcharReadRunner calls SqlDataReader.GetSqlJson, which was introduced in MDS 6.0, so any pre-6.0 baseline broke that pass:

JsonVsVarcharReadRunner.cs(115,32): error CS1061: 'SqlDataReader' does not contain a
definition for 'GetSqlJson' ... [::TargetFramework=net9.0]

Reproduced locally with -p:MdsPackageVersion=5.2.3; 6.0.2 / 7.0.2 / 7.1.0-preview build fine.

Changes

Baseline API gating (Microsoft.Data.SqlClient.PerformanceTests.csproj)

  • Derives MdsBaselineMajorVersion from MdsPackageVersion and defines cumulative MDS_GE_6 / MDS_GE_7 constants, so any future benchmark can guard a post-baseline API without new one-off plumbing.
  • Tolerates whitespace and a v prefix. Project mode, unpinned Package mode, and unparsable versions all fail toward "current" (sentinel 99999), since those builds reference a current MDS.

JsonVsVarcharReadRunner

  • JSON reads are guarded with #if MDS_GE_6, falling back to GetString / GetFieldValueAsync<string>. The async fallback keeps an async read path so the async benchmark stays meaningful.
  • Warns loudly from GlobalSetup when the fallback is compiled in — the baseline then measures a different code path, so that benchmark's delta is not comparable.
  • Fails with an actionable message when the server lacks native JSON column support, instead of a raw SqlException.
  • Validates RowCount and the configured connection string up front.
  • Drops tables created by a partially-failed GlobalSetup (BenchmarkDotNet does not guarantee GlobalCleanup runs when setup throws, and table names are unique per run, so leaks accumulate on the perf server).
  • GlobalCleanup attempts both drops even if one fails, skips tables never created, aggregates failures, and still clears pools via finally.
  • Minor: non-static _rowCount, trailing whitespace.

Docs (eng/pipelines/perf/README.md)

  • New "Benchmarks must compile against the oldest baseline" section with the guard pattern.
  • Troubleshooting row mapping CS1061 in the baseline pass to the fix.

Validation

Emitted constants:

MdsPackageVersion constants
5.2.3, v5.2.3, V5.2.3, 0.1.0 (none) → fallback
6.0.2 MDS_GE_6
7.0.2, 7.1.0-preview1.26124.5 MDS_GE_6;MDS_GE_7
latest (unparsable), unpinned, Project mode all → current

Builds clean in Package mode for 5.2.3 / 6.0.2 / 7.0.2 / 7.1.0-preview1.26124.5 and in Project mode across net8.0/net9.0/net10.0.

  • Tests added or updated (perf benchmark project only; no product code changed)
  • Public API changes documented (none — no API changes)
  • Verified against customer repro (build break reproduced and fixed locally)
  • Ensure no breaking changes introduced

No product code is touched, so no release note is needed.

cheenamalhotra and others added 4 commits August 6, 2026 22:27
JsonVsVarcharReadRunner used SqlDataReader.GetSqlJson, which was
introduced in MDS 6.0. When the perf pipeline builds the test project in
Package mode against an older pinned baseline (e.g.
-p:MdsPackageVersion=5.2.3), compilation failed with CS1061.

Define MDS_SQLJSON_SUPPORTED in the perf test project unless a pre-6.0
baseline package is pinned, and fall back to the string accessor in the
JSON benchmarks when it is not defined.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 773aff6a-7a03-477e-8229-705eb9bc2f13
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 773aff6a-7a03-477e-8229-705eb9bc2f13
Follow-up to the GetSqlJson build break:

* Replace the one-off MDS_SQLJSON_SUPPORTED constant with cumulative
  MDS_GE_<major> constants derived from MdsPackageVersion, so future
  benchmarks can guard any post-baseline API without new plumbing.
  Unparsable/absent versions fail toward "current" (all constants set).
* Warn loudly from JsonVsVarcharReadRunner's GlobalSetup when the JSON
  fallback is compiled in, since the baseline pass then measures a
  different code path and its delta is not comparable.
* Fail with an actionable message when the server has no native JSON
  column type instead of a raw SqlException.
* Document the "benchmarks must compile against the oldest baseline"
  constraint and add a CS1061 troubleshooting row in the perf README.
* Minor: make the row count a non-static instance field and drop
  trailing whitespace.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 773aff6a-7a03-477e-8229-705eb9bc2f13
Edge cases surfaced while reviewing the baseline API gating:

* Tolerate leading whitespace and a "v" prefix in MdsPackageVersion when
  deriving the baseline major version.
* Validate RowCount and the configured connection string in GlobalSetup
  instead of failing later with an opaque error.
* Drop any tables created by a partially-failed GlobalSetup, since
  GlobalCleanup is not guaranteed to run when setup throws.
* Attempt both drops in GlobalCleanup even if one fails, skip tables that
  were never created, still clear pools on failure, and aggregate the
  failures rather than leaking a table silently.
* Only emit the pre-6.0 fallback warning for the JSON parameterization.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 773aff6a-7a03-477e-8229-705eb9bc2f13
Copilot AI review requested due to automatic review settings August 7, 2026 05:48
@cheenamalhotra
cheenamalhotra requested a review from a team as a code owner August 7, 2026 05:48
@github-project-automation github-project-automation Bot moved this to To triage in SqlClient Board Aug 7, 2026
@cheenamalhotra cheenamalhotra added this to the 7.1.0-preview3 milestone Aug 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes the perf pipeline’s baseline build (Package mode with a pinned, released Microsoft.Data.SqlClient version) by ensuring the PerformanceTests project can compile against pre-6.0 baselines that don’t expose newer APIs like SqlDataReader.GetSqlJson. It does this via MSBuild-defined compile-time constants and runner-side API gating, while also improving benchmark setup/cleanup robustness to avoid leaking tables on failure.

Changes:

  • Add MSBuild logic in PerformanceTests to derive a baseline major version from MdsPackageVersion and define cumulative MDS_GE_<major> constants (e.g., MDS_GE_6, MDS_GE_7).
  • Gate JsonVsVarcharReadRunner’s JSON accessor usage behind #if MDS_GE_6, with meaningful fallbacks and clearer/setup-time warnings when the fallback path is used.
  • Document the “oldest baseline must compile” rule and troubleshooting guidance in perf pipeline docs.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File Description
src/Microsoft.Data.SqlClient/tests/PerformanceTests/Microsoft.Data.SqlClient.PerformanceTests.csproj Derives baseline major from MdsPackageVersion and defines MDS_GE_6 / MDS_GE_7 compile-time constants to enable API gating.
src/Microsoft.Data.SqlClient/tests/PerformanceTests/BenchmarkRunners/JsonVsVarcharReadRunner.cs Guards GetSqlJson behind MDS_GE_6 with fallbacks; improves setup validation, failure cleanup, and drop robustness to reduce perf-server table leaks.
eng/pipelines/perf/README.md Adds guidance for guarding post-baseline APIs with MDS_GE_<major> and links troubleshooting for baseline CS1061 build failures.

@cheenamalhotra cheenamalhotra moved this from To triage to In review in SqlClient Board Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: In review

Development

Successfully merging this pull request may close these issues.

4 participants