Skip to content

[Bug] PD task REST: balanceLeaders throws a bare HTTP 500 during the balance-shard window, and task responses cannot be told apart from a no-op or a follower answer #3231

Description

@bitflicker64

Bug Type (问题类型)

rest-api (结果不合预期)

Before submit

  • I have confirmed and searched that there are no similar problems in the historical issue and documents

Environment (环境信息)

  • Server Version: master 83ef9f3 (pd image sha256:69dc1d4e8625)
  • Backend: HStore, 3 PD + 3 Store, 12 shard groups
  • OS: kind v0.33 (Kubernetes 1.37.0)

Expected & Actual behavior (期望与实际表现)

Measured 2026-09-22 while executing the documented recovery sequence (patrolPartitions, balanceLeaders, balancePartitions) on the PD leader with a follower as control:

  1. GET /v1/task/balanceLeaders within 180 s of a balancePartitions call (which sets the balance-shard key even when it moves nothing) answers {"timestamp":...,"status":500,"error":"Internal Server Error","path":"/v1/task/balanceLeaders"} with no reason. The reason (PDException: balance shard is processing, please try later!, TaskScheduleService.java:478) appears only in the PD log. Cause: TaskAPI.balanceLeaders (TaskAPI.java:92-95) declares throws PDException with no handler, while its siblings catch and return toJSON(e). One-line fix.
  2. GET /v1/task/patrolPartitions returns the byte-identical body {"status": 0,"partitions": [ ]} on the leader and on a follower, whether or not it repaired anything; a follower does no work and reports success. balancePartitions differs by two bytes ({} from the leader, empty body from a follower). Operators driving recovery over REST cannot tell "ran and repaired", "ran, nothing to do" and "hit a follower, did nothing" apart without reading PD logs.

Ask: catch the PDException in balanceLeaders like the sibling endpoints; and have the task endpoints either redirect to the leader (as the read APIs do via getMembers) or answer an explicit error naming the leader, plus include what was done (groups reallocated, leaders moved, partitions moved) in the body.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions