Skip to content

fix(helper): don't let the accept deadline bound the whole transfer (#44) - #45

Merged
talmacschen-arch merged 2 commits into
cloudberry-contrib:mainfrom
talmacschen-arch:fix/issue44-serve-numconn
Jul 14, 2026
Merged

fix(helper): don't let the accept deadline bound the whole transfer (#44)#45
talmacschen-arch merged 2 commits into
cloudberry-contrib:mainfrom
talmacschen-arch:fix/issue44-serve-numconn

Conversation

@talmacschen-arch

@talmacschen-arch talmacschen-arch commented Jul 14, 2026

Copy link
Copy Markdown
Collaborator

fix #44


Change logs

ConcurrentServer.serve() runs an unbounded Accept() loop that lives for the whole transfer. The listener's absolute 600 s deadline (set in ServerBase.Start()) therefore bounds the total transfer time of a single table, not just the accept phase: once all expected clients (--client-numbers) have connected, the loop blocks on Accept() again, and at start + 600 s that still-blocked Accept() returns i/o timeoutsetError → the transfer is torn down. This is confined to ExtDestLtCopy + push (destination has fewer segments than source), i.e. any table whose data takes longer than ~10 minutes to move. See #44 for the full root-cause analysis.

Fix: make serve() accept-count aware — stop accepting once numConn connections are in, then Close() the listener so the in-flight data connections stream to completion with no listener deadline over them. A timeout/error before all expected clients have connected is still fatal (a client genuinely failed to connect), and the error path now returns instead of continue-ing, so a tripped deadline can no longer busy-spin flooding the log.

Scope: only ConcurrentServer (ExtDestLtCopy + push) is affected. OneTimeServer (CopyOnMaster / CopyOnSegment / ExtDestGeCopy and all pull-mode senders) closes its listener after a single Accept() and is unchanged — so --connection-mode=pull was, and remains, unaffected.

Validation

On a 16 → 8 segment setup (source 16 primaries, destination 8) copying a single large unpartitioned table (TPC-DS store_sales, 1 TB scale) in default push mode:

The helper package currently has no unit tests; happy to add a focused serve() test (accept numConn → listener closed, Err()==nil, no timeout; fewer than numConn before the deadline → Err()!=nil, no busy-spin) if maintainers would like one in this PR.

Summary by CodeRabbit

  • Bug Fixes
    • Server listeners now stop accepting connections after the expected number of clients connect.
    • Accept errors now terminate the connection-acceptance process promptly instead of being ignored.
    • Listeners close automatically once all expected connections are established, while existing connections remain active.

…loudberry-contrib#44)

ConcurrentServer.serve() ran an unbounded Accept() loop that lives for the
entire transfer. After all expected clients (--client-numbers) connect, the
loop blocks on Accept() again; the listener's absolute 600s deadline set in
ServerBase.Start() then fires mid-transfer, turning "no more inbound
connections" (a normal state) into a fatal error. This caps the total
transfer time of a single table for ExtDestLtCopy + push, so any table whose
data takes longer than ~10 minutes to move is torn down -- surfacing as
SQLSTATE 22P04 / "could not write to COPY program: Broken pipe", or,
depending on the accept/stream race, an indefinite hang with no error
surfaced to either side.

Make serve() accept-count aware: stop accepting once numConn connections are
in, then close the listener so the in-flight data connections stream to
completion with no deadline over them. A timeout/error before all expected
clients connect is still treated as fatal, and the error path now returns
instead of continuing so a tripped deadline can no longer busy-spin.

Only affects ConcurrentServer (ExtDestLtCopy + push). OneTimeServer
(CopyOnMaster/CopyOnSegment/ExtDestGeCopy and all pull-mode senders) closes
its listener after a single accept and is unchanged.
@coderabbitai

coderabbitai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

ServerBase.serve() now accepts only the expected number of client connections, exits immediately on accept errors, and closes the listener after the final expected connection is accepted.

Changes

Server acceptance lifecycle

Layer / File(s) Summary
Bounded accept and listener shutdown
helper/server.go
serve() tracks accepted connections, stops accepting at t.numConn, returns on accept errors, and closes the listener after all expected clients connect.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main fix: preventing the accept deadline from capping the whole transfer.
Description check ✅ Passed The description follows the required issue reference and change-log structure and provides substantial implementation and validation detail.
Linked Issues check ✅ Passed The changes match #44 by stopping acceptance after numConn, closing the listener, and preserving fatal pre-connection errors.
Out of Scope Changes check ✅ Passed The PR appears scoped to the listener/accept-loop fix in helper/server.go with no unrelated code changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@helper/server.go`:
- Line 159: Update the cleanup path containing t.listener.Close() to handle its
returned error instead of discarding it. Match the established approach in
helper/one_time_server.go by logging and surfacing listener shutdown failures
while preserving the existing cleanup flow.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 58f05fb0-c7d5-47c5-be1d-d81274c7de4a

📥 Commits

Reviewing files that changed from the base of the PR and between 22d64a7 and 105c8c5.

📒 Files selected for processing (1)
  • helper/server.go

Comment thread helper/server.go Outdated
…y-contrib#44)

Address review feedback: surface a listener Close() failure instead of
silently discarding it. Log-only on purpose -- all expected clients are
already connected and streaming by this point, so failing the transfer here
(as one_time_server.go does on its pre-stream close) would reintroduce the
spurious-teardown class of bug that cloudberry-contrib#44 fixes.
leaocx
leaocx approved these changes Jul 14, 2026

@leaocx leaocx left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Precise fix with solid validation.

  • Bounded accept loop correctly prevents the listener deadline from capping the entire transfer
  • continuereturn on error path eliminates busy-spin
  • Closing the listener does not affect already-established data connections
  • Log-only (no setError) on listener-close failure is the right call — avoids reintroducing spurious teardown

@talmacschen-arch
talmacschen-arch merged commit 90fc6a2 into cloudberry-contrib:main Jul 14, 2026
2 checks passed
@talmacschen-arch
talmacschen-arch deleted the fix/issue44-serve-numconn branch July 14, 2026 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Large-table copy aborts with SQLSTATE 22P04 after exactly 600s (ExtDestLtCopy + push): listener deadline caps total transfer time

2 participants