Skip to content

Complete resilient Data Management migrations - #1100

Closed
paullizer wants to merge 20 commits into
microsoft:Developmentfrom
paullizer:feature/1043-data-management-migration-resilience
Closed

Complete resilient Data Management migrations#1100
paullizer wants to merge 20 commits into
microsoft:Developmentfrom
paullizer:feature/1043-data-management-migration-resilience

Conversation

@paullizer

Copy link
Copy Markdown
Collaborator

Summary

  • Completes resilient Data Management migrations with durable checkpoints, incremental modes, server-owned preview/reconciliation, keyset Search paging, manifests, retry/cancel recovery, and target capacity restoration.
  • Adds target-side coordination and Azure AI Search writer fencing across independent SimpleChat sources, including response-loss quarantine, external-writer confirmation, and safe mirror deletes.
  • Keeps authorization changes fail closed: personal unshare waits for acknowledged Search ACL projection and returns retryable 503 while a target Search fence is active.
  • Updates the admin migration workflow, feature documentation, release notes, and application version to 0.250.078.

Validation

  • 85 passed, 1 deselected: focused Make Data Management migrations resilient for large blob transfers with cancel, resume, and incremental sync #1043 migration suite. The deselected case is the local env-gated browser test command.
  • 20 passed: final ACL, route-policy, and shared migration script tests.
  • Changed Python compilation, JavaScript syntax checks, whitespace checks, and VS Code diagnostics passed.
  • azd deploy succeeded to https://simplechat-20260506-app.azurewebsites.net/.
  • Authenticated live admin validation passed: Data Management Migration pane loaded on v0.250.078; Delta baseline, Mirror confirmation, and external Search-writer freeze controls rendered correctly.

Closes #1043

paullizer added 20 commits July 22, 2026 07:39
…zed-source-manifest-pr

Phase 1: add authorized mixed-source manifest contracts
…arch-mixed-source-consistency

Phase 2: honor explicit mixed-source Chat selections
…-document-csv-artifacts

Fix CSV artifacts for non-tabular documents
…ource-analyze

Phase 3: mixed-source combined Analyze
…-cross-format-compare

Phase 4: cross-format Compare orchestration
…ation-continuity

Phase 5: preserve mixed-source context across follow-up turns
Implement Phase 6 terminal coverage, cancellation, finalization, bounded Analyze All, telemetry, and compatibility hardening.\n\nImprove durable CSV export throughput and timeout recovery.\n\nRefs microsoft#1061\nRefs microsoft#1071
…ource-hardening

Harden mixed-source orchestration and CSV exports
Unify CSV artifact finalization across chat, document actions, and workflows; add durable schema clarification and authorized background export handling. Remove the mixed-source Analyze availability gate and update validation, documentation, and release notes.

Refs microsoft#1071

Refs microsoft#1058
…sv-generation

Add universal CSV generation across response paths
Promote current-turn structured function results into a format-neutral CSV, DOCX, and PDF artifact pipeline across chat, agents, document actions, and workflows. Preserve durable CSV handling and coverage-aware tabular exports.\n\nRefs microsoft#1071
Increase bounded search, metadata, directory, workflow, assigned-knowledge, and mixed-source candidate retrieval limits while preserving authorization and existing maximum constraints. Update affected functional coverage and release notes.
Copilot AI review requested due to automatic review settings July 30, 2026 00:52

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot wasn't able to review this pull request because it exceeds the maximum number of files (300). Try reducing the number of changed files and requesting a review from Copilot again.

@paullizer
paullizer changed the base branch from main to Development July 30, 2026 01:04
@paullizer paullizer closed this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make Data Management migrations resilient for large blob transfers with cancel, resume, and incremental sync

2 participants