This project shows how a security questionnaire response workflow can be built with Azure AI Foundry agents, Azure AI Search, and synthetic evidence documents.
The walkthrough follows the newer Microsoft Foundry / Azure portal experience as of June 2026.
The idea is to use two agents:
- A RAG answer agent searches indexed evidence documents and drafts an answer.
- A validation agent reviews that answer and checks whether it appears supported by the available evidence.
Instead of answering security questionnaire questions from memory, the workflow retrieves from indexed policy/evidence documents first and then produces a grounded response.
The walkthrough uses fictional company data for AcmeCloud Analytics. All evidence documents and questionnaire rows in this repository are synthetic.
The project builds a RAG workflow for security questionnaire answering:
- Create an Azure AI Foundry project.
- Deploy a chat model and embedding model.
- Upload synthetic evidence documents to Azure Blob Storage.
- Build a RAG index using Azure AI Search.
- Connect to the Foundry project from a local Python notebook.
- Create a temporary RAG agent that answers questions from the indexed documents.
- Create a second temporary validation agent that reviews whether an answer is supported by the indexed evidence.
Start with the walkthrough files in order.
Sections 01–05 are documented as general Foundry/Azure setup steps that can be reused for similar Azure AI Foundry + Azure AI Search RAG projects.
Sections 06–07 are specific to this security questionnaire demo.
Complete Sections 01–04 before running the notebooks. Section 04 covers the local environment, dependencies, .env file, and Jupyter setup.
After that, run the notebooks in order:
walkthrough/05_foundry_connection_test.ipynb
walkthrough/06_rag_agent_demo.ipynb
walkthrough/07_multi_agent_validation_demo.ipynb
| Section | File | Purpose |
|---|---|---|
| 01 | Foundry project and models | Create the Foundry project and deploy the chat/embedding models |
| 02 | Storage, Search, and permissions | Create Blob Storage, Azure AI Search, and managed identity permissions |
| 03 | RAG index creation | Upload synthetic Markdown evidence and create the Azure AI Search RAG index |
| 04 | Local environment setup | Set up the Python environment and .env file |
| 05 | Foundry connection test | Confirm the local notebook can connect to the Foundry project |
| 06 | RAG agent demo | Create a temporary RAG agent and answer a question from evidence |
| 07 | Multi-agent validation demo | Use one agent to answer and another to review support from evidence |
The repository includes synthetic test data under synthetic_data/.
The documents are fictional policies for AcmeCloud Analytics and cover areas such as:
- access control
- encryption
- incident response
- data retention
- vendor management
- business continuity
The sample CSV contains security questionnaire questions that can be used to test the RAG agent.
To run it yourself:
- Follow Sections 01–03 to create the Azure resources and RAG index.
- Follow Section 04 to set up the local Python environment and
.envfile. - Run Notebooks 05–07 in order.
One useful nuance came up during portal verification.
The Azure AI Search index showed successful ingestion, a successful indexer run, a nonzero document count, and vector index usage. However, Search Explorer did not display visible results for simple manual queries during portal testing.
Because the source files were Markdown documents, this may have been related to how the Markdown content was ingested, chunked, or exposed through Search Explorer queries. The later agent notebooks were still able to retrieve answers through the Azure AI Search tool, so this was treated as a Search Explorer / Markdown ingestion verification nuance rather than a failed RAG index.
Azure resources can continue to create cost after testing.
If you created a dedicated resource group for this project, the simplest cleanup is to delete that resource group after you are done.
Only delete the resource group if it contains resources for this project only. Deleting it will remove the Foundry resource, Azure AI Search service, Storage Account, uploaded documents, indexes, and related resources created during the walkthrough.