feat(azure)!: Azure AI Search retrieval without On Your Data (3.0.0)

[owndev]

Problem

Microsoft retires Azure OpenAI On Your Data on October 14, 2026. The Azure AI Search integration of azure_ai_foundry.py (up to 2.8.1) is built entirely on it: the pipeline adds data_sources to the chat request and Azure runs the search. After that date, chats configured with AZURE_AI_DATA_SOURCES are expected to fail or to be answered without the index.

On Your Data also had other limits:

• it only supports GPT-4o / GPT-4o-mini;
• it ignores the data source when tools are present;
• it rejects stream_options.

This is step 2 of #187. Step 1, the notice in 2.8.1, was merged in #190.

Change (Azure AI Foundry 2.8.1 → 3.0.0)

The pipeline queries Azure AI Search itself and adds the hits to the prompt as [doc1]..[docN]. This works for every chat endpoint and model: Azure OpenAI deployments, Foundry /models, serverless and non-OpenAI models.

• Configuration: the existing AZURE_AI_DATA_SOURCES JSON (azure_search) is read as the search configuration: endpoint, index, fields_mapping, query_type, semantic_configuration, filter, top_n_documents, strictness, in_scope, role_information, embedding_dependency, authentication.
• Search:
  • Search REST 2026-04-01 for simple, semantic, vector, vector_simple_hybrid and vector_semantic_hybrid;
  • query vectors from embedding_dependency (/openai/v1/embeddings or an endpoint), or from the index vectorizer;
  • strictness is approximated per query, hits are merged and de-duplicated, then cut to top_n_documents and a token budget;
  • document text is sanitised against prompt injection.
• Query generation, as On Your Data's "intent" step: on follow-up turns, the same deployment turns the conversation into search queries. Any failure falls back to the user's text, and after 3 timeouts in a row it pauses for that model. Valve AZURE_AI_SEARCH_QUERY_GENERATION: auto (default) | always | off.
• Citations as before. The 2.x citation, [docX] link, relevance-score and history-unlink code runs on an On Your Data-shaped context built from the hits. API clients still receive context, in the first SSE event or in message.context. In tool rounds each source is emitted once.
• Auth:
  • the new encrypted valve AZURE_AI_SEARCH_KEY, which wins over a key in the JSON;
  • access_token;
  • managed identity of the Open WebUI host, through the bundled azure-identity (system- or user-assigned).
• Fail closed: search, embeddings or configuration errors end with Error: Azure AI Search: … and a terminal status. Nobody gets an answer without the documents.
• Safety:
  • The retrieval cache is keyed per user, chat and message, and only reused by tool rounds of the same message.
  • Search and embeddings responses are size-limited at 16 MiB.
  • No key and no document text appears in the logs.
• Now allowed in RAG chats: tools, tool_choice and stream_options.
• New valves: AZURE_AI_SEARCH_KEY, AZURE_AI_SEARCH_API_VERSION, AZURE_AI_SEARCH_QUERY_GENERATION, AZURE_AI_SEARCH_MAX_CONTEXT_TOKENS. No released valve is renamed or removed.

BREAKING (hence 3.0.0)

• data_sources sent by a client or an inlet filter now end the request with a clear error and are never forwarded.
• With a managed identity, the Open WebUI host's identity calls Azure AI Search and needs the Search Index Data Reader role. The Open WebUI host must also reach the search service over the network.
• Vector query types need embedding_dependency or an index vectorizer.
• strictness, in_scope and role_information are applied by the pipeline and are approximations of On Your Data.
• The 2.8.1 retirement warning is gone.

docs/azure-ai-integration.md has a "Migrating from On Your Data (2.x)" section with a checklist. Users who need On Your Data until Microsoft turns it off can stay on 2.8.1 (permalink).

E2E

• New:
  • mocks/mock_search.py: Search and token endpoints with request recording;
  • embeddings and query-generation routes in mock_azure.py;
  • the rag group in suites/_azure_rag.py.
• Removed:
  • the oyd group: every check whose behaviour still exists was ported to pipeline mode, the OYD-only checks were removed (the PR's commit history lists each one);
  • the On Your Data emulation of the mock;
  • all temporary markers.
• Coverage: the azure suite grows from 75 to 198 checks. Among them is an in-process DEBUG-log check, because the server runs at INFO.

Verification

Run	Result
All suites, v0.11.4-slim, normal and --strict-known	418 PASS, 0 FAIL, 1 KNOWN (infomaniak-name-prefix, no fix yet)
azure suite, v0.11.3-slim, normal and --strict-known	198 PASS, 0 FAIL, 0 KNOWN
Meta-test (E2E_MOCK_FAULT=500)	0 KNOWN
Helper and flow tests in a container (with -X dev)	112/112 and 83/83, no ResourceWarning
Mutation testing, 2.9.0 round (4 reviews)	32 findings fixed, among them nested-tag prompt injection, the per-user cache key, user-assigned MI parameter names (mi_res_id/msi_res_id), and size limits. All surviving mutants killed
Mutation testing, 3.0.0 round (3 reviews)	12 findings fixed, among them a stream-only data_sources bypass and a held-back link after [DONE]. All surviving mutants killed

The spec was built from Microsoft Learn: Search REST, embeddings, auth, and the On Your Data semantics. A critic reviewed it, and the maintainer decided its open points: pipeline only (no legacy mode), query generation auto, fail closed.

Not verified against real Azure

• the managed-identity token requests on App Service, Container Apps, AKS and VMs;
• whether models cite [docN] reliably with the prompt format;
• the strictness thresholds against real scores;
• the /openai/v1/embeddings route on every resource type behind API Management.

Please test with a real search index before release.

Closes #187

🤖 Generated with Claude Code