Skip to content

Clean up a few test designs to reduce testing times - #3219

Open
Alan Zimmer (alzimmermsft) wants to merge 5 commits into
microsoft:mainfrom
alzimmermsft:SpeedupCoreTesting
Open

Clean up a few test designs to reduce testing times#3219
Alan Zimmer (alzimmermsft) wants to merge 5 commits into
microsoft:mainfrom
alzimmermsft:SpeedupCoreTesting

Conversation

@alzimmermsft

@alzimmermsft Alan Zimmer (alzimmermsft) commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Merges together tests starting an MCP server instance to share that instance when start logic is the same by adding additional assertions into the shared test.

Force usage of PlaybackCredential in test that doesn't care if real authentication is called.

Improves CI times by a few minutes per job when running all tests (modifications to shared code / infrastructure, which is quite common).

GitHub issue number?

[Link to the GitHub issue this PR addresses]

Pre-merge Checklist

  • Required for All PRs
    • Read contribution guidelines
    • PR title clearly describes the change
    • Commit history is clean with descriptive messages (cleanup guide)
    • Added comprehensive tests for new/modified functionality
    • Created a changelog entry if the change falls among the following: new feature, bug fix, UI/UX update, breaking change, or updated dependencies. Follow the changelog entry guide
  • For MCP tool changes:
    • One tool per PR: This PR adds or modifies only one MCP tool for faster review cycles
    • Updated servers/Azure.Mcp.Server/README.md and/or servers/Fabric.Mcp.Server/README.md documentation
    • Validate README.md changes running the script ./eng/scripts/Process-PackageReadMe.ps1. See Package README
    • For new or modified tool descriptions, ran ToolDescriptionEvaluator and obtained a score of 0.4 or more and a top 3 ranking for all related test prompts
    • For tools with new names, including new tools or renamed tools, update consolidated-tools.json
    • For renamed tools, follow the Tool Rename Checklist and tag the PR with the breaking-change label
    • For new tools associated with Azure services or publicly available tools/APIs/products, add URL to documentation in the PR description
  • Extra steps for Azure MCP Server tool changes:
    • Updated command list in servers/Azure.Mcp.Server/docs/azmcp-commands.md
    • Ran ./eng/scripts/Update-AzCommandsMetadata.ps1 to update tool metadata in azmcp-commands.md (required for CI)
    • Updated test prompts in servers/Azure.Mcp.Server/docs/e2eTestPrompts.md
    • 👉 For Community (non-Microsoft team member) PRs:
      • Security review: Reviewed code for security vulnerabilities, malicious code, or suspicious activities before running tests (crypto mining, spam, data exfiltration, etc.)
      • Manual tests run: added comment /azp run mcp - pullrequest - live to run Live Test Pipeline

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI lite review requested due to automatic review settings August 5, 2026 19:46

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces overall test runtime by consolidating MCP server-starting live tests (so they can share a single server instance when startup logic is identical) and by tightening authentication behavior in a concurrency-focused credential test.

Changes:

  • Enabled xUnit parallel execution settings for Microsoft.Mcp.Tests via runner config updates.
  • Refactored CustomChainedCredential to expose its lazy-initialized inner credential for test validation and performed small cleanup/refactors.
  • Merged/adjusted Azure MCP server start live tests and added additional assertions (e.g., uniqueness checks in all-mode).

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
core/Microsoft.Mcp.Core/tests/Microsoft.Mcp.Tests/xunit.runner.json Updates local xUnit runner settings to enable parallelization.
core/Microsoft.Mcp.Core/tests/Microsoft.Mcp.Tests/xunit.runner.ci.json Updates CI xUnit runner settings to enable parallelization.
core/Microsoft.Mcp.Core/src/Services/Azure/Authentication/CustomChainedCredential.cs Exposes the lazy credential for tests and renames token cache field; minor refactors.
core/Azure.Mcp.Core/tests/Azure.Mcp.Core.Tests/Services/Azure/Authentication/CustomChainedCredentialTests.cs Simplifies tests by removing reflection and forcing playback mode for the concurrency test.
core/Azure.Mcp.Core/tests/Azure.Mcp.Core.Tests/Areas/Server/ServerStartCommandTests.cs Merges server-starting live tests and adds stronger assertions (including uniqueness checks in all-mode).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants