Skip to content

SimulatePubnetMixedLoad: a node that loses sync can never rejoin (local-only archive), failing the mission #418

Description

@sisuresh

Observed on a SimulatePubnetMixedLoad run (stellar-core 27.1.1 133f5bcf0, protocol 27). Pod ssc-0942z-2aa52f-sts-node-139-0 ended the run stuck in catchup, so the mission's final EnsureAllNodesInSync fullCoreSet failed.

What the node's log shows, in order:

  1. It's a non-tier1 leaf with a single peer for the whole run — TARGET_PEER_CONNECTIONS: 1, PREFERRED_PEERS_ONLY. It spends the first ~2.5 min unable to resolve its preferred peer and rejecting all others, then authenticates one connection:
Unable to resolve peer 'ssc-...-sts-satoshipay-2...': Host not found (authoritative)
Non preferred inbound authenticated peer 172.22.177.113:11625 rejected because all available slots are taken.
Authenticated to 172.22.177.113:11625
  1. Under load (~1150 tx/ledger, network delay enabled) tx-set fetches from that one peer degrade until it drops out of sync. Apply time stays at 100–400 ms/ledger throughout, so it's starved for data, not slow to apply:
'fetch-e65310' took 13.515000 s
Ledger took 25.794006148 seconds        (ledger 116)
'fetch-3cd058' took 40.518002 s
Herder WARNING Lost track of consensus
Lost sync, local LCL is 123, network closed ledger 126
  1. Catchup then can't proceed. FullPubnetCoreSets sets historyNodes = Some([]) (StellarNetworkData.fs:580), so the only archive is the node's own local at /data/history. It last published checkpoint 63 before losing sync, so the checkpoint it now needs was never written, and it retries indefinitely:
Catching up to ledger 127: Downloading state file history/00/00/00/history-0000007f.json
cp: cannot stat '/data/history/history/00/00/00/history-0000007f.json': No such file or directory
Missing HAS for ledger 127: maybe stale archive local

That message repeats with growing backoff for the remaining ~11 minutes of the run. Meanwhile it keeps buffering externalized ledgers (mSyncingLedgers reaches 66) without ever applying them, and never returns to Synced!.

So in these missions a single loss of sync is terminal for that node, and the mission fails at the end regardless of how the rest of the network behaved.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions