fix(migration): preserve task creator/comments/reactions/resolution and replay incident state on 2.0 upgrade

[sonika-shah]

Problem

On a 1.12.1 / 1.13 → 2.0 upgrade, the thread→task migration preserves the task rows and their type/status mapping, but drops several fields and resets incident state. On migrated (pre-existing) data:

• Created By is blank — the migration never carries the original requester onto the task.
• Completed tasks lose their resolver — resolution.resolvedBy is empty and the comment is a generic "Migrated from thread-based task system".
• Task comments are dropped — the thread's posts (replies + resolve/close records) don't reach task_entity.comments.
• Task reactions are dropped.
• Migrated incidents reset to New / No Owners — the incident workflow restarts at NewStage and never replays the recorded test_case_resolution_status lifecycle, so a pre-upgrade Ack/Assigned incident loses its stage and assignee (and the stale state then makes the "Update Status" resolve fail).

New tasks/incidents created in 2.0 are unaffected — this is migration-only.

Fix

v200/MigrationUtil.java — in the thread→task migration:

• set createdBy (entityReference) in addition to createdById;
• set resolution.resolvedBy from the legacy closedBy (the original close note now survives as a migrated comment, so the placeholder is dropped);
• migrate the thread's posts[] into task comments[] (+ commentCount);
• carry thread-level reactions[] onto the task.

For incidents, after backfillOpenTasksToWorkflowInstances starts the workflow, replay the test case's latest resolution status onto the instance, reusing the existing runtime bridge (resolveLegacyTransitionId / buildLegacyResolvedPayload) so migration and runtime stay consistent. New public entry point: TestCaseResolutionStatusRepository.backfillMigratedIncidentTaskStage. Idempotent — no-ops when the task is already at the recorded stage.

Complementary to #29978 (which restores the ASSIGNED_TO / MENTIONED_IN relationships on Postgres); this PR restores the field- and incident-state fidelity that #29978 does not cover.

Testing

• openmetadata-service compiles; spotless applied.
• Reviewers: please verify the incident replay end-to-end on a migrated instance — a test case that was Ack/Assigned before upgrade should come back at the same stage with its assignee, and "Update Status" should transition without error. (This path touches Flowable workflow state and warrants a live migration check.)

---

Additional fix: task resolve/close change events (2.0 regression)

Symptom (verified on 1.12.1→2.0, Postgres, in the change_event table): in 1.12.1, resolving/closing a task emitted change events (taskResolved ×77, taskClosed ×9 under entityType=THREAD), which drove webhook/email/Slack alerts. After the 2.0 task refactor, resolving/closing a migrated task produces zero change events — so task-resolution alerts silently stop working after upgrade.

Origin — PR #25894 (Task redesign): the pre-redesign feed endpoints (/v1/feed/tasks/{id}/resolve and /close in FeedResource) returned via RestUtil's .toResponse() wrapper, which sets the X-OpenMetadata-Change header automatically — so ChangeEventHandler recorded a taskResolved/taskClosed event. PR #25894 introduced TaskResource with new resolveTask/closeTask/suggestion-apply endpoints that return a bare Response.ok(task).build() — the header behavior the old feed path got for free via .toResponse() was not carried over, so no change event is recorded (this is a behavior-not-ported gap in the redesign, not a deleted line). The add-comment endpoint (header added in #29023) proves the mechanism is intended for tasks; resolve/close were simply missed.

How it's recorded: ChangeEventHandler.getEventTypeFromResponse emits an event only when the response carries the X-OpenMetadata-Change header (or is HTTP 201); a bare 200 produces nothing.

Fix: set RestUtil.CHANGE_CUSTOM_HEADER = ENTITY_UPDATED on the resolve/close/suggestion-apply responses, restoring task lifecycle events (and thus alerting). create already fires entityCreated via the 201 path. Bulk operations return a BulkTaskOperationResult (not an entity) and would need per-item emission — noted, out of scope here.

Verification: confirmed at runtime on the same task — create→entityCreated ✓, add-comment→entityUpdated ✓, and (pre-fix) resolve/close→nothing; the fix makes resolve/close emit entityUpdated.

Greptile Summary

This PR fixes several data-fidelity gaps in the 1.x→2.0 thread-to-task migration and a change-event regression in TaskResource. The migration now carries createdBy, resolvedBy, and thread posts (as comments) onto each migrated task, and replays pre-upgrade incident TCRS state (Ack/Assigned + assignee) onto freshly created workflow instances. The TaskResource resolve/close/suggestion-apply endpoints now emit X-OpenMetadata-Change: entityUpdated, restoring task-lifecycle alerts lost in the PR #25894 redesign.

• Migration field preservation (MigrationUtil.java): createdBy entity reference and createdById are now set from the legacy thread's requester; resolution.resolvedBy is populated from closedBy; thread posts[] are mapped to task comments[] with per-post reactions; and the resolveUserReference helper is promoted to a shared static method.
• Incident state replay (TestCaseResolutionStatusRepository.java): New backfillMigratedIncidentTaskStage method replays the latest TCRS record onto a just-started workflow instance via the existing applyLegacyStatusToIncidentTask bridge; idempotent by design.
• Task change events (TaskResource.java): Adds the X-OpenMetadata-Change header to resolveTask, closeTask, and applySuggestion responses so task-resolution webhooks/alerts fire correctly after upgrade.

Confidence Score: 4/5

Safe to merge with awareness that thread-level reactions are silently dropped during migration and the Flowable workflow timing concern flagged in prior review remains unaddressed.

The migration field-preservation and change-event fixes are correct and well-scoped. The one notable gap is that the PR description claims thread-level reactions are carried over, but the Task schema has no top-level reactions field so they are dropped; only per-post reactions survive via comments. The Flowable async-startup timing issue (replay fires immediately after triggerByKey) was flagged in a prior review round and is still present — if the workflow job executor hasn't advanced to NewStage before resolveTaskWithWorkflow runs, the transition silently fails and the incident stays at New.

MigrationUtil.java — the replayMigratedIncidentState call sequence after workflowHandler.triggerByKey warrants a live migration test to confirm the Flowable instance is in a stable state before the transition fires.

Important Files Changed

Filename	Overview
openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java	Thread-to-task migration now populates createdBy, resolvedBy, and migrates post comments with per-comment reactions. Thread-level reactions are intentionally dropped (Task schema has no top-level reactions field). The timing question around Flowable workflow initialization before replayMigratedIncidentState is a carry-over concern noted in prior review.
openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/TestCaseResolutionStatusRepository.java	New public backfillMigratedIncidentTaskStage entry point reuses applyLegacyStatusToIncidentTask to replay legacy TCRS state onto a freshly created workflow instance; correctly idempotent via resolveLegacyTransitionId's null-on-already-at-stage contract.
openmetadata-service/src/main/java/org/openmetadata/service/resources/tasks/TaskResource.java	Adds X-OpenMetadata-Change: entityUpdated header to resolve, close, and suggestion-apply responses, restoring change-event emission for task lifecycle alerts. Pattern is consistent with the existing add-comment endpoint.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant M as MigrationUtil (v200)
    participant TR as TaskRepository
    participant WH as WorkflowHandler
    participant TCRS as TestCaseResolutionStatusRepository

    Note over M: Thread-to-Task migration
    M->>M: buildUserRef(createdByName) → createdBy + createdById
    M->>M: migrateThreadPostsToComments(threadJson, taskJson)
    M->>M: buildUserRef(closedBy) → resolution.resolvedBy
    M->>TR: insertTask(handle, threadId, taskJson)

    Note over M: backfillOpenTasksToWorkflowInstances
    M->>TR: listAll(open tasks without workflowInstanceId)
    loop Each open task
        M->>WH: triggerByKey(workflowDefinitionFQN, taskId, variables)
        alt "task.category == Incident"
            M->>TCRS: backfillMigratedIncidentTaskStage(task)
            TCRS->>TCRS: getLatestRecord(testCase FQN)
            TCRS->>TCRS: applyLegacyStatusToIncidentTask(latest, FQN)
            TCRS->>TR: resolveTaskWithWorkflow(task, transitionId, ...)
        end
    end

    Note over M: TaskResource resolve/close
    participant TR2 as TaskResource
    participant CEH as ChangeEventHandler
    TR2->>TR2: repository.resolveTaskWithWorkflow(...)
    TR2-->>CEH: Response + X-OpenMetadata-Change: entityUpdated
    CEH->>CEH: emit taskResolved / taskClosed change event

Loading

%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant M as MigrationUtil (v200)
    participant TR as TaskRepository
    participant WH as WorkflowHandler
    participant TCRS as TestCaseResolutionStatusRepository

    Note over M: Thread-to-Task migration
    M->>M: buildUserRef(createdByName) → createdBy + createdById
    M->>M: migrateThreadPostsToComments(threadJson, taskJson)
    M->>M: buildUserRef(closedBy) → resolution.resolvedBy
    M->>TR: insertTask(handle, threadId, taskJson)

    Note over M: backfillOpenTasksToWorkflowInstances
    M->>TR: listAll(open tasks without workflowInstanceId)
    loop Each open task
        M->>WH: triggerByKey(workflowDefinitionFQN, taskId, variables)
        alt "task.category == Incident"
            M->>TCRS: backfillMigratedIncidentTaskStage(task)
            TCRS->>TCRS: getLatestRecord(testCase FQN)
            TCRS->>TCRS: applyLegacyStatusToIncidentTask(latest, FQN)
            TCRS->>TR: resolveTaskWithWorkflow(task, transitionId, ...)
        end
    end

    Note over M: TaskResource resolve/close
    participant TR2 as TaskResource
    participant CEH as ChangeEventHandler
    TR2->>TR2: repository.resolveTaskWithWorkflow(...)
    TR2-->>CEH: Response + X-OpenMetadata-Change: entityUpdated
    CEH->>CEH: emit taskResolved / taskClosed change event

Loading

Comments Outside Diff (1)

1. openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java, line 1959-1970 (link)

   Replay fires before workflow instance is stable

   replayMigratedIncidentState is called synchronously right after workflowHandler.triggerByKey. If the Flowable job executor processes the "start" event asynchronously, the workflow process may not yet have transitioned into NewStage when resolveTaskWithWorkflow runs, causing the transition call to fail. The failure is caught and downgraded to a WARN, so the incident stays at New with no visible error — exactly the regression this PR is meant to fix. A brief Flowable-state check or retry before the transition (or moving the replay into an afterCommit hook where the workflow is guaranteed to be in a stable state) would close this gap.

_{Reviews (9): Last reviewed commit: "chore: re-trigger CI after merge with ma..." | Re-trigger Greptile}

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

• No GitHub issue is linked. Link an issue in the Development section of the PR (or add Fixes #12345 to the description). For a same-org cross-repo issue, add Fixes open-metadata/<repo>#123 to the description.

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[github-actions]

🔴 Playwright Results — 4 pipeline/setup failure(s)

✅ 0 passed · ❌ 0 failed · 🟡 0 flaky · ⏭️ 0 skipped

Pipeline and setup failures

• The build job finished with status failure.
• Shard detection finished with status failure.
• The Playwright shard matrix was unexpectedly skipped.
• No expected Playwright shards were declared.

Shard	Passed	Failed	Flaky	Skipped

📦 Download artifacts

How to debug locally

# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip    # view trace

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[sonarqubecloud]

Quality Gate passed for 'open-metadata-ingestion'

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[sonika-shah]

Superseded by #30248 — re-raised from an internal branch because fork PRs are currently blocked by GitHub's new pull_request_target fork-checkout guard (CI refuses to check out fork code, so build/checkstyle/integration all fail at checkout). #30248 carries the identical backend diff, rebased on current main (so it also includes #29978), scoped to the 3 backend files. Closing this one to avoid duplicates.

[gitar-bot]

Code Review ⚠️ Changes requested 2 resolved / 4 findings

Restores field and incident state fidelity during task migration and fixes alert regression by adding missing headers to TaskResource endpoints. However, the migration logic introduces an unbounded IN-list lookup and retains dead code that must be addressed.

⚠️ Edge Case: Unbounded IN-list in umbrella workflow lookup can silently fail

📄 openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java:626-640 📄 openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java:1541-1555

In migrateThreadTasksToTaskEntity, all task threads are loaded unpaged (line 626) and the full set is passed to resolveUmbrellaWorkflowInstanceIdBatch -> lookupLegacyUmbrellaWorkflowInstanceIds, which builds a single ... IN (<threadIds>) query with one bind parameter per thread id. On large catalogs (tens of thousands of pre-2.0 tasks) this exceeds the JDBC/DB bind-parameter limit (~65535 on Postgres), and the query throws. Because the catch block only does LOG.debug, the failure is silent: every workflowInstanceId resolves to null, so migrated open tasks don't get their umbrella instance id and postCreate spawns duplicate workflow subprocesses — the exact defect this batch resolution is meant to prevent. Note the javadoc claims 'one SQL per input page' assuming 200-item pages, but this caller does not page. Chunk the thread ids (e.g. partition into batches of a few hundred) before calling the resolver, matching the paged TaskWorkflow.migrateLegacyThreadTasks path.

💡 Quality: Dead code: unused single-thread umbrella lookup method

📄 openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java:1530-1533

lookupLegacyUmbrellaWorkflowInstanceId(Handle, String) (singular) is never invoked — it is referenced only from a javadoc {@link} on the batch variant. Remove it to avoid dead code, or use it where a single-id lookup is actually needed.

✅ 2 resolved

✅ Quality: Comment fallback fabricates a non-existent "system" user reference

📄 openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java:1526-1533
When a post's author cannot be resolved, the comment's author entityReference is set to a fabricated id UUID.nameUUIDFromBytes("system") with name system. No such row exists in user_entity, so this is a dangling reference — the same class of blank/unresolvable actor this PR sets out to fix; the UI will fail to resolve the author. Consider falling back to a real actor (e.g. resolve the thread's createdBy, or the ingestionBot/admin user id) or omitting the author rather than writing a synthetic user id.

✅ Performance: Redundant user_entity lookup for createdBy reference

📄 openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java:748-755 📄 openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java:1495-1505
createdByUserId is already resolved via lookupUserId, then buildUserRef immediately re-runs the identical lookupUserId query for the same createdByName. lookupUserId is uncached and hits user_entity each call, so every migrated task issues one redundant query. Build the reference from the id already in hand rather than querying again.

🤖 Prompt for agents

Code Review: Restores field and incident state fidelity during task migration and fixes alert regression by adding missing headers to TaskResource endpoints. However, the migration logic introduces an unbounded IN-list lookup and retains dead code that must be addressed.

1. ⚠️ Edge Case: Unbounded IN-list in umbrella workflow lookup can silently fail
   Files: openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java:626-640, openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java:1541-1555

   In `migrateThreadTasksToTaskEntity`, all task threads are loaded unpaged (line 626) and the full set is passed to `resolveUmbrellaWorkflowInstanceIdBatch` -> `lookupLegacyUmbrellaWorkflowInstanceIds`, which builds a single `... IN (<threadIds>)` query with one bind parameter per thread id. On large catalogs (tens of thousands of pre-2.0 tasks) this exceeds the JDBC/DB bind-parameter limit (~65535 on Postgres), and the query throws. Because the catch block only does `LOG.debug`, the failure is silent: every `workflowInstanceId` resolves to null, so migrated open tasks don't get their umbrella instance id and `postCreate` spawns duplicate workflow subprocesses — the exact defect this batch resolution is meant to prevent. Note the javadoc claims 'one SQL per input page' assuming 200-item pages, but this caller does not page. Chunk the thread ids (e.g. partition into batches of a few hundred) before calling the resolver, matching the paged `TaskWorkflow.migrateLegacyThreadTasks` path.

2. 💡 Quality: Dead code: unused single-thread umbrella lookup method
   Files: openmetadata-service/src/main/java/org/openmetadata/service/migration/utils/v200/MigrationUtil.java:1530-1533

   `lookupLegacyUmbrellaWorkflowInstanceId(Handle, String)` (singular) is never invoked — it is referenced only from a javadoc `{@link}` on the batch variant. Remove it to avoid dead code, or use it where a single-id lookup is actually needed.

Options

Display: compact → Showing less information.

Comment with these commands to change the behavior for this request:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}