fix(autoclaim): EthTxManager defaults and start-block defaulting - #1875
Conversation
TOML array-of-tables elements like [[AutoClaim.Claimers]] get no per-element
defaults from the config loader, and config/default.go's AutoClaim section
stops at BridgeServiceFinder, so every claimer's EthTxManager that an operator
does not fully spell out starts from Go zero values.
Two zero values are actively dangerous in
github.com/0xPolygon/zkevm-ethtx-manager@v0.2.18/ethtxmanager:
- FrequencyToMonitorTxs=0 makes time.After(0) fire immediately, spinning the
tx-monitor loop with no delay (ethtxmanager.go:485).
- GasPriceMarginFactor=0 multiplies the suggested gas price by zero, so every
tx is rejected as "transaction underpriced" (ethtxmanager.go:989-991).
Add Config.ApplyDefaults (autoclaim/config/config.go), called once in
config/config.go right after unmarshalling and before Validate, which fills
each claimer's EthTxManager zero-valued fields with aggkit's own
AggOracle.EVMSender.EthTxManager opinion (config/default.go): 1s monitor
frequency, 2s wait-to-be-mined, 250ms/1s receipt polling, a
GasPriceMarginFactor of 1 (<=0 coerced to 1, since a zero factor can never be
a legitimate request), and the network 5/10 safe/finalized block overrides
plus 1 max gas-estimate retry. It never touches a value the operator set.
Also fix config/default.go: AggOracle.EVMSender.EthTxManager rendered the TOML
keys GetReceiptMaxTime/GetReceiptWaitInterval, but ethtxmanager.Config's
mapstructure tags are WaitReceiptMaxTime/WaitReceiptCheckInterval
(zkevm-ethtx-manager@v0.2.18/ethtxmanager/config.go:19,22), so those two
AggOracle defaults were silently never binding.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DJuYNKE9jRLRrCWTrrEJ2T
|
Claude finished @arnaubennassar's task in 4m 26s —— View job Review:
|
| Location | Impact |
|---|---|
tools/force_ger_update/example-config.toml:47-48 |
live config for a shipped tool — same footgun, unfixed |
tools/force_ger_update/README.md:93-94, docs/ethtxmanager.md:12-13 |
docs teach the non-binding key |
config/config_test.go:391-392, tools/force_ger_update/config_test.go:109-110 |
fixtures assert nothing about these, so they pass either way |
test/e2e/forcegerupdate_test.go:251-252 |
e2e config |
test/config/*.template, test/e2e/kurtosis/*.template.toml, test/e2e/envs/**/aggkit-config.toml |
commented-out, so harmless, but they document the wrong key |
tools/force_ger_update is the interesting one: it embeds ethtxmanager.Config the same way (tools/force_ger_update/config.go:68), has no ApplyDefaults equivalent, and its example config uses the non-binding spelling. That's the identical defect this PR exists to fix. Fixing the spelling repo-wide in this PR is cheap; if you'd rather not, please say so explicitly in the description so it doesn't get lost. Fix this →
🟡 Suggested
3. No test covers the config/default.go fix, which is a real behaviour change for AggOracle. Grepping the repo, nothing asserts on AggOracle.EVMSender.EthTxManager.GetReceiptMaxTime after a default load — which is exactly why the mismatch survived. Add an assertion that loading the default config yields 250ms / 1s there. Also: the PR body frames this as "silently never binding", but the effect of fixing it is that AggOracle's receipt timings move from 0 to 250ms/1s in every existing deployment. That belongs in the release notes, not just in a parenthetical.
4. WaitReceiptMaxTime = "250ms" with WaitReceiptCheckInterval = "1s" looks swapped. A 250 ms total budget with a 1 s poll interval means the budget expires before the first check. These values never bound before, so nobody would have noticed. Worth confirming against the module's receipt loop before making them live — and the same pair is copied verbatim into autoclaim/config/config.go:28-29, so a mistake propagates to claimers too.
5. Constant names don't match the fields they're assigned to. autoclaim/config/config.go:161-166 assigns DefaultEthTxManagerWaitReceiptMaxTime to cfg.GetReceiptMaxTime. Two names for one thing on adjacent lines. Since these constants are consumed by Go code, naming them after the Go fields (DefaultEthTxManagerGetReceiptMaxTime) reads better; leave the TOML-tag spelling to default.go and the docs.
6. docs/autoclaim.md's claimer table (lines 615-625) should record the new defaults. Right now EthTxManager is documented as Yes (required) with no mention that eight of its fields are now filled in for you. That table is where an operator will look.
🟢 Optional
7. All eight Default* constants are exported but used only within the package and its tests — unexport unless the public surface is intentional.
8. Defaults are applied to disabled claimers too. Harmless, and arguably right for consistency, just noting it's unconditional.
9. require.InDelta(t, ..., 0) for float equality (config_test.go:270, :326) works but require.Equal(t, float64(DefaultEthTxManagerGasPriceMarginFactor), ...) states the intent more directly.
10. Config.Checksum() / the saved merged config now include the defaulted values, so InternalConfigChecksum will shift for existing deployments on upgrade. Expected, but someone watching that field will see a change with no config edit behind it.
11. Adjacent gap, not this PR: ClaimerConfig.Validate requires EthTxManager.StoragePath but not EthTxManager.Etherman.URL, which is equally fatal downstream at ethtxmanager.New (autoclaim/runtime/runtime.go:210).
⚠️ Verification caveat
I could not verify the central claim of change #2 in this environment: the Go module cache is outside my accessible paths and network fetches (WebFetch, gh api to the upstream repo) were not permitted, so I could not read zkevm-ethtx-manager@v0.2.18/ethtxmanager/config.go to confirm the mapstructure tags. go build and go test were likewise unavailable, so I did not independently run the build or the test suite — I'm taking the PR's reported results at face value. What I can say is that the claim is corroborated from inside the repo: docs/autoclaim.md:549-550 and all six test/e2e/autoclaim_test.go claimer fixtures already use the WaitReceiptMaxTime spelling, while the older force_ger_update and AggOracle configs use the other one. A reviewer with module access should confirm the tags directly before merging — findings #2 and #4 both hinge on it.
🚀 What's New
autoclaim/config.Config.ApplyDefaults() fills each [[AutoClaim.Claimers]] entry's EthTxManager fields that were left at their Go zero value with aggkit's established AggOracle.EVMSender.EthTxManager values. It's called once from loadString (config/config.go:500), after viper unmarshalling and before AutoClaim.Validate(), so the defaults are also reflected in the saved merged config. TOML array-of-tables elements get no per-element defaults from the default-TOML merge, so before this every unspecified claimer field started from its Go zero value.
🐛 Bug Fixes
- A claimer configured without an explicit
EthTxManagerblock no longer starts withFrequencyToMonitorTxs = 0, which makes the tx-monitor loop'stime.After(0)fire immediately and busy-spin (observed in production at ~1,500 log lines/second for ~4 days, on a pod that reported healthy throughout). - A claimer no longer starts with
GasPriceMarginFactor = 0, which multiplies the suggested gas price by zero and gets every transaction rejected astransaction underpriced. Any value<= 0is coerced to1. config/default.go's[AggOracle.EVMSender.EthTxManager]block used the TOML keysGetReceiptMaxTime/GetReceiptWaitInterval, which don't match the vendoredethtxmanager.Configmapstructure tags, so those two AggOracle defaults never bound. Renamed toWaitReceiptMaxTime/WaitReceiptCheckInterval. Not a no-op: AggOracle's receipt timings move from0to250ms/1s. The same misspelling remains intools/force_ger_updateand indocs/ethtxmanager.md(see 🔴 Split L1 / L2 responsibilities for lastgersync #2).- The module-level deadlock in 0xPolygon/zkevm-ethtx-manager#139 is not fixed — an underpriced-at-creation transaction still can never be re-priced, because re-pricing is gated on
MonitoredTxStatusSent. A claimer already wedged before this ships still needs manual recovery.
📋 Config Updates
No new config keys. Eight existing [[AutoClaim.Claimers]].EthTxManager fields are now defaulted when omitted:
| Field (Go / TOML) | Default |
|---|---|
FrequencyToMonitorTxs |
1s |
WaitTxToBeMined |
2s |
GetReceiptMaxTime / WaitReceiptMaxTime |
250ms |
GetReceiptWaitInterval / WaitReceiptCheckInterval |
1s |
GasPriceMarginFactor |
1 (any value <= 0 is coerced) |
SafeStatusL1NumberOfBlocks |
5 |
FinalizedStatusL1NumberOfBlocks |
10 |
EstimateGasMaxRetries |
1 |
Two TOML keys renamed under [AggOracle.EVMSender.EthTxManager]: GetReceiptMaxTime → WaitReceiptMaxTime, GetReceiptWaitInterval → WaitReceiptCheckInterval. EthTxManager.StoragePath remains required per claimer and is untouched by the defaulting.
⚠️ Breaking Changes
No API or schema breakage, but three silent behavioural changes on upgrade:
- A claimer that explicitly sets
SafeStatusL1NumberOfBlocks = 0,FinalizedStatusL1NumberOfBlocks = 0, orEstimateGasMaxRetries = 0— which the repo's own example atdocs/autoclaim.md:559-560does for the first two — gets5/10/1instead, unlogged. See 🔴 Unify L1 syncers #1. - AggOracle's receipt timings change from
0to250ms/1snow that the keys bind. InternalConfigChecksumshifts for existing deployments, since the defaults land in the config before it's hashed.
Review only — no code changes were made or pushed. The build and test suite were not run in this environment (see verification caveat).
• branch fix/autoclaim-claimer-ethtxmanager-defaults
…back, not genesis Both bridge detectors' start-block fields (L1ToL2BridgeDetector.StartBlock, L2ToLxBridgeDetector.StartL1Block) were plain uint64, so 0 was indistinguishable from "unset" and meant "scan from genesis". A production deployment that omitted them scanned ~6.9M Sepolia blocks (~19h) twice, and the L2-to-Lx side silently fell back to full history whenever it could not resolve a baseline LER -- the same zero-value-means-something-dangerous class as the GasPriceMarginFactor fix already in this PR. Both fields become *uint64 (nil = unset, explicit value incl. 0 used verbatim), with a new StartLookback duration (default 24h) used to resolve an unset value by binary-searching L1 block timestamps for "now - lookback" and clamping to [minBlock, head] -- never below the L1 bridge sync's/RollupManager's own genesis, never above head. This avoids estimating a block number from an average block-time constant, which has been observed elsewhere in this codebase to be wrong enough to skew a multi-hour ETA by over ten hours. The L2-to-Lx detector's silent fall-back to full history when it cannot derive an initial LER (predates the first L1 info tree leaf, or resolves to a zero LER) is replaced with a loud, per-source retry: it no longer fails the whole poll (that would let one bad source block every other source), just skips and retries that one source every poll with a WARN, until the operator fixes the config or explicitly opts into full history via StartL1Block=0. Also updates config/default.go, which previously baked StartBlock=0/ StartL1Block=0 into every rendered config -- leaving those lines in place would have made every existing deployment "explicit genesis" forever and defeated this fix entirely. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DJuYNKE9jRLRrCWTrrEJ2T
0332cac to
9b48c33
Compare
…-lx block cursor (#1882) ## 🔄 Changes Summary - **Fix**: a source network permanently excluded from bridge service resolution no longer pins the `L2ToLx` detector's shared L1 block-window cursor, which previously stopped **all** L2-origin claim discovery. `autoclaim/bridgedetector/l2_to_lx.go` keeps **one** block-window cursor (`l2-to-lx`) shared by every source network. `processSource` returned `sourceRetryLater` for *any* `fetcher.GetURL` error, and `PollOnce` then held that shared cursor just before the earliest skipped source's verify row — or, when that row sat at the very start of the window, `return result, nil` **without saving the cursor at all**, re-scanning the identical window on the next poll. `bridgeservicefinder.GetURL` returns `ErrNetworkDisabled` for every network listed in `Config.IgnoreNetworkIDs`. That error is a deliberate, permanent exclusion — it can never clear. So a single ignored network with a `VerifyBatches` row anywhere in the scanned range wedged the shared cursor **forever**, and no L2-origin claim request could ever be created for **any** source. Observed in a deployment whose ignore list covers most registered rollups: over 8 hours the only `skip source` line emitted named an ignored network, and the request store contained L1-origin requests exclusively — not one L2-origin request. **The fix**: a permanently excluded source is `sourceUpToDate` ("nothing to do") instead of `sourceRetryLater`. It no longer contributes to `retryBlock` and no longer holds the window back. Its own `(source, destination)` LER cursors are still left untouched, so when the network is later un-ignored, its **next** verify row is fetched from the old pair cursor and covers everything bridged in between. That recovery does depend on the source publishing another verify row: the shared window has already moved past the row seen while the source was excluded and only rewinds by `overlapBlocks` (default 1), so it never returns to it. To un-ignore a source that has **stopped** producing verified batches, reset the `l2-to-lx` block-window cursor so its old verify row falls inside the scanned range again. The trade is deliberate — the old behaviour bought that re-observability at the cost of deadlocking every source. A genuinely retryable `GetURL` failure (service not yet announced, or unhealthy) keeps its existing hold-the-window behaviour, unchanged. Two smaller points: - The permanence distinction is expressed in the fetcher contract rather than read out of `bridgeservicefinder` by the detector. `ClaimCandidatesFetcher` gains a detector-local `ErrSourceDisabled`, the permanent counterpart of `ErrURLNotFound`; `ServiceFetcher.GetURL` — the designated translation point — maps the finder's disabled error to it, checked **before** the not-found branch that error wraps, and still wrapping `ErrURLNotFound` since an excluded source is also a source with no URL. `processSource` branches on the local sentinel and no longer imports `bridgeservicefinder`, and both sentinels are documented on the `GetURL` contract: the whole bug was a permanence distinction the contract never expressed. A test pins the mapping, so a future refactor cannot silently reintroduce the deadlock, and any non-finder-backed fetcher can now express the same thing. - An excluded source is now reported once per source rather than on every poll. The exclusion never clears and the detector polls for the lifetime of the process, so the repetition carried no information. The doc comments on `sourceOutcome`, `PollOnce` and `processSource` said "transient" where the code accepted any error; they are updated to match the new semantics. ##⚠️ Breaking Changes - None. No config, API, CLI or storage change. ## 📋 Config Updates - None. ## 🔌 API Updates - None. ## ✅ Testing - 🤖 **Automatic**: 6 new tests in `autoclaim/bridgedetector`: - `TestL2ToLxDisabledSourceDoesNotHoldBlockCursor` — an excluded source does not hold the cursor back; the window advances past its verify row. - `TestL2ToLxTransientURLErrorStillHoldsBlockCursor` — a retryable URL miss **still** holds the cursor (guards against regressing the existing behaviour). - `TestL2ToLxDisabledSourceDoesNotStarveHealthySource` — mixed window: one excluded source plus one healthy source at a later block; the healthy source processes and the cursor advances. - `TestL2ToLxDisabledSourceAtWindowStartDoesNotPinCursor` — the production regression: an excluded source's verify row at the very start of the window must not take the "return without saving the cursor" path. - `TestL2ToLxDisabledSourceIsReportedOncePerSource` — the report is emitted once, not per poll. - `TestServiceFetcherGetURL/network_disabled_maps_to_the_permanent_sentinel` — pins that the finder's disabled error arrives at the detector as `ErrSourceDisabled`, never as a plain retryable `ErrURLNotFound`, including the branch ordering that makes it so. - **Confirmed to be genuine regression tests.** With the test files in place and `l2_to_lx.go` reverted to its pre-fix state, 4 of the 5 behavioural tests fail: ``` --- FAIL: TestL2ToLxDisabledSourceDoesNotHoldBlockCursor (cursor 0x13 = 19, want 0x31 = 49) --- FAIL: TestL2ToLxDisabledSourceDoesNotStarveHealthySource (cursor 0x13 = 19, want 0x31 = 49) --- FAIL: TestL2ToLxDisabledSourceAtWindowStartDoesNotPinCursor (CursorAdvanced: should be true) --- FAIL: TestL2ToLxDisabledSourceIsReportedOncePerSource (mock: Unexpected Method Call) --- PASS: TestL2ToLxTransientURLErrorStillHoldsBlockCursor (behaviour intentionally unchanged) --- PASS: TestServiceFetcherGetURL (pins existing behaviour by design) ``` All pass with the fix applied, alongside the pre-existing retry tests (`TestL2ToLxNotSyncedSkipsSourceWithoutAdvancingCursor`, `TestL2ToLxRetriesSkippedSourceOnLaterPollWithoutNewLER`, `TestL2ToLxRetrySkipAtWindowStartKeepsStoredCursor`). - `go test ./autoclaim/... ./bridgeservicefinder/...` — all packages `ok`. - `golangci-lint run --timeout 5m` — `0 issues.` - 🖱️ **Manual**: none required. ## 🐞 Issues - No tracking issue. ## 🔗 Related PRs - #1875 also touches `autoclaim/bridgedetector/l2_to_lx.go` (and its test file) in the same `processSource` `GetURL` error branch, so the two will conflict textually on merge. The changes are logically independent: #1875 only lowers the log level of the skip line, this PR changes the control flow so the shared cursor is not held. This PR is based directly on `develop` and contains nothing from #1875. ## 📝 Notes - Design decision: the excluded source's per-pair LER cursors are deliberately **not** advanced. The block window is only the trigger for re-examining a source; the per-`(source, destination)` LER cursor is what bounds the actual candidate fetch. Leaving it untouched means that if the operator later un-ignores the network, its next verify row is fetched from the old pair cursor and still covers every bridge exit made while it was excluded — even though the block window has long since moved past. The converse is the operational caveat above: a re-enabled source that never publishes another verify row is never re-examined, because the window does not rewind to the row it already passed. Resetting the block-window cursor is the recovery for that case. - The once-per-source reporting is a plain `map[uint32]struct{}` on the detector, touched only from the single poll goroutine — deliberately not a general log-dedup mechanism. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01DJuYNKE9jRLRrCWTrrEJ2T --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
…mer-ethtxmanager-defaults # Conflicts: # autoclaim/bridgedetector/l2_to_lx.go
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…p (CodeQL) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
…o int64 (CodeQL) Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
|
@claude review |
|
Claude finished @joanestebanr's task in 1m 19s —— View job Review of #1875: EthTxManager defaults and start-block defaultingThe diagnosis is solid and the PR is well documented. The latest commits also resolved the CodeQL I read the diff only. 🔴 Required1. Explicit
2.
🟡 Suggested3. The
4. The resolved start block is not persisted, so it moves forward on every restart.
5. Validate
6.
7. Constant names don't match the Go fields they are assigned to.
8. The
Fix them in this PR, or list them as an explicit follow-up in the description. 9. Add a test that loading the default config yields
10. Document the new EthTxManager defaults in the claimer table in 🟢 Optional
🚀 What's New
🐛 Bug Fixes
📋 Config UpdatesNew key (both detectors), defaulted in [AutoClaim.L1ToL2BridgeDetector]
StartLookback = "24h" # used only when StartBlock is absent
[AutoClaim.L2ToLxBridgeDetector]
StartLookback = "24h" # used only when StartL1Block is absent
|
…r, key renames) - stop defaulting Safe/Finalized/EstimateGasMaxRetries (explicit 0 is meaningful) - treat ErrBlockNotProcessed as retry-later per source; throttle retry-later logs - reject negative StartLookback; unexport default constants and name after Go fields - rename GetReceipt* keys to WaitReceipt* in remaining docs/templates/tool config - document EthTxManager defaults and restart-relative start window Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This PR collects a small set of operational fixes found while running
autoclaimin a productiondeployment. It started as a single fix for
EthTxManagerdefaults and has grown a second,unrelated fix (start-block defaulting) found during the same operating window — each small enough
on its own not to warrant a separate PR, but included here explicitly so reviewers know why this
is multi-part.
1. Apply EthTxManager defaults to claimers
Config.ApplyDefaults()inautoclaim/config/config.go, called once inconfig/config.goright after config unmarshalling and beforeAutoClaim.Validate(). It fills each[[AutoClaim.Claimers]]entry'sEthTxManagerfields that were left at their Go zero value with aggkit's own establishedAggOracle.EVMSender.EthTxManageropinion (config/default.go), and never touches a value the operator explicitly set.config/default.go's[AggOracle.EVMSender.EthTxManager]block: it rendered TOML keysGetReceiptMaxTime/GetReceiptWaitInterval, butethtxmanager.Config's mapstructure tags areWaitReceiptMaxTime/WaitReceiptCheckInterval(zkevm-ethtx-manager@v0.2.18/ethtxmanager/config.go:19,22), so those two AggOracle defaults were silently never binding. Verified against the vendored module source before including it.Why
TOML array-of-tables elements (like
[[AutoClaim.Claimers]]) get no per-element defaults from the config loader, andconfig/default.go's[AutoClaim]section stops atBridgeServiceFinder— it defines no[[AutoClaim.Claimers]]at all. So any claimer field an operator does not fully spell out starts from its Go zero value.Two of those zero values are actively dangerous in
github.com/0xPolygon/zkevm-ethtx-manager@v0.2.18/ethtxmanager:FrequencyToMonitorTxs=0→time.After(0)fires immediately, spinning the tx-monitor loop with no delay (ethtxmanager.go:485).GasPriceMarginFactor=0→ the suggested gas price is multiplied by zero, so every transaction is rejected astransaction underpriced(ethtxmanager.go:989-991).Both were hit back-to-back in a production deployment: a claimer without an explicit
EthTxManagerblock first submitted underpriced transactions, then — once that surfaced andGasPriceMarginFactoralone was patched at the deploy-config layer — the same already-wedged monitored transaction kept retrying in the busy-loop, because re-pricing an existing transaction is only ever applied to a transaction that reachedSentstatus, and an underpriced-at-creation transaction never gets there. The result was ~1,500-1,600 log lines/second sustained for roughly 4 days, on the order of 100 million log lines, from a pod Kubernetes reported as perfectly healthy the entire time (no crash, no restart, probes green) — nothing in standard pod-health monitoring would have caught it.What is defaulted, and why those values
Config.ApplyDefaults()mirrors aggkit's ownAggOracle.EVMSender.EthTxManagervalues (config/default.go), so everyEthTxManagerinstance in the binary behaves the same way absent explicit operator config:FrequencyToMonitorTxs1sWaitTxToBeMined2sGetReceiptMaxTime(WaitReceiptMaxTime)250msGetReceiptWaitInterval(WaitReceiptCheckInterval)1sGasPriceMarginFactor1(any value<= 0is coerced to1)1is a neutral no-margin multiplierSafeStatusL1NumberOfBlocks5FinalizedStatusL1NumberOfBlocks10EstimateGasMaxRetries1A note on the last three: the vendored module documents
0as a meaningful sentinel forSafeStatusL1NumberOfBlocks/FinalizedStatusL1NumberOfBlocks("use the network's own safe/finalized tag") and forEstimateGasMaxRetries("retry forever"). These aren't zero-values that are unconditionally broken the way the durations and gas margin are. But aggkit's ownAggOraclepath already overrides all three of those module sentinels with a fixed opinion, so this PR mirrors that same override for claimers, for consistency across everyEthTxManagerinstance in the binary — rather than leaving claimers as the only place that silently falls through to the vendored zero-value behavior.I considered a validation error instead of silent defaulting for some of these fields, but rejected it: unlike
StoragePath(already required byClaimerConfig.Validate), none of these fields have an "obviously correct" value an operator must supply — they're tuning knobs with the project's own established defaults, and requiring every deployment to spell out all eight of them just to avoid a footgun would only reproduce the same gap the next time a field is added to the vendored config. Defaulting to the project's own already-established opinion is safer than either silently zero-valuing or hard-failing.Testing (fix 1)
go build ./...— passes.go test ./autoclaim/... ./config/...— passes (existing suite unaffected; addedTestApplyDefaultsFillsUnsetEthTxManagerFields,TestApplyDefaultsPreservesExplicitlySetEthTxManagerFields,TestApplyDefaultsCoercesNonPositiveGasPriceMarginFactorinautoclaim/config/config_test.go).2. Default bridge-detector start blocks to a recent lookback, not genesis
Evidence
Both bridge detectors' start-block config fields —
L1ToL2BridgeDetector.StartBlockandL2ToLxBridgeDetector.StartL1Block(autoclaim/config/config.go, ~:67 and ~:81) — were plainuint64, so0was indistinguishable from "unset", and0means scan from L1 genesis. This is thesame zero-value-means-something-dangerous class as the
GasPriceMarginFactorfix in section 1.In the same production deployment: a rollout that omitted these scanned ~6.9M Sepolia blocks,
taking ~19h, twice (once after a datadir wipe), and would have mass-enqueued years of stale
deposits the moment
DryRunwas lifted. The workaround was hand-pinning a wall-clock-derived magicnumber (head minus 7 days) with a "re-pin if the deploy slips past date X" comment, which goes stale
by construction. On the L2-to-Lx side, if the resolved LER was zero or the configured block predated
the first L1 info tree leaf,
initialFromLER(autoclaim/bridgedetector/l2_to_lx.go, ~:592) silentlyfell back to full history — invisible, and with no in-product recovery for bridges that fall below
a detector's start block (they are permanently unclaimable; the admin API only acts on
already-detected requests).
Fix
Both fields become
*uint64:nil(the field absent from config) means "resolve automatically",an explicit value — including
0— is used verbatim and never adjusted, which is what makes thischange backwards compatible for every deployment that already pins a start block. A new
StartLookback cfgtypes.Durationfield (default24h, applied inApplyDefaults()) controls howfar back an unset value resolves from.
mapstructure/viper in this codebase decode*uint64correctly out of the box — verified directlyagainst this repo's own
viper/mapstructureversions before committing to the pointer approach(absent key →
nil;= 0→ pointer to0;= 42→ pointer to42) — so no sentinel value orcompanion
isSetfield was needed.Resolution (
autoclaim/bridgedetector/startblock.go,ResolveStartBlock): when unset, itbinary-searches L1 block headers (
EthClienter.CustomHeaderByNumber) for the highest block at orbefore
now - StartLookback— aboutlog2(head-minBlock)header lookups, not a linear scan — thenclamps the result to
[minBlock, head].minBlockis the L1 bridge sync's own configured genesisblock for
L1ToL2BridgeDetector(BridgeL1Sync.InitialBlockNum) and the RollupManager contract'screation block for
L2ToLxBridgeDetector(L1NetworkConfig.RollupManagerCreationBlock, whichconfig validation already guarantees is
> 0) — a detector can never usefully start earlier thanthe data source it reads from or the contract whose events it needs. It deliberately does not
estimate the block number from an average block-time constant: an orchestration tool used elsewhere
in this operating window assumed 12.0s/block on Sepolia when the real figure was 12.512s, which
skewed a multi-hour ETA by 14 hours — reading real timestamps back from the chain avoids that class
of error entirely.
StartLookbackdefaults to24h: a bridge only becomes claimable after certificate settlementand destination GER injection, which has been observed in production to take over an hour, so
a short window risks leaving valid bridges permanently unclaimable, while 24h of Sepolia is only
~7,000 blocks versus the ~6.9M actually backfilled by the genesis default.
The resolution runs once at startup per enabled detector and logs an
INFOnaming the lookback, theresolved block and its timestamp, plus a
WARNthat bridges originating before that block willnever be autoclaimed and need manual claiming.
config/default.gopreviously bakedStartBlock = 0/StartL1Block = 0into the renderedconfig template for every deployment. Those lines are removed (not set to a placeholder — simply
absent), since leaving them in would have made every existing deployment "explicit genesis" forever
and defeated this fix entirely; the Go-side
ApplyDefaults()is now the only placeStartLookbackgets a default, matching how the EthTxManager defaults in section 1 are applied.
Silent-fallback decision: WARN + per-source retry, not a hard error
initialFromLER's two silent-fallback branches (leaf-not-found, zero LER) now return a newErrCannotResolveInitialLERsentinel instead ofnil, nil— but only when the configured startblock is non-zero;
StartL1Block = 0remains an explicit, honored request for full history, andsince automatic resolution is always clamped to a strictly-positive
minBlock, a literal0canonly ever be deliberate operator intent, never a resolution artifact.
I initially made this a hard error, but
initialFromLER's caller (resolveFromLER→processSource) propagates any error all the way up throughPollOnce, which aborts the entirepoll — meaning one source with a bad
StartL1Blockrelative to its history would silently stallevery other source's discovery too, until an operator intervened. That is too large a blast radius
for a per-source config problem, and risks the same "wedged and silently retrying" pattern the
EthTxManagerfix in section 1 exists to prevent. Instead,processSourcenow catchesErrCannotResolveInitialLERspecifically and treats it like the existing finder-miss/not-synced-yetcases: it logs a prominent
WARNnaming the source and remediation ("lowerStartL1Block, or set itto
0"), and returnssourceRetryLater— that source alone is retried every poll until fixed;every other source in the same poll is unaffected.
Testing (fix 2)
go build ./...— passes.go test ./autoclaim/... ./config/... ./cmd/...— passes. Addedautoclaim/bridgedetector/startblock_test.go(TestResolveStartBlock*: explicit value/explicit-zero used verbatim with zero RPC calls, unset resolves from lookback, clamping at both the head and the configured floor, L1-client error propagation); updated the two existingl2_to_lx_test.gosilent-fallback tests to cover the explicit-genesis path, and added their non-explicit-genesis counterparts asserting the new retry-later behavior (no candidates fetched, poll still succeeds,SkippedSourceCountincrements); updatedconfig/config_test.goandautoclaim/runtime/runtime_test.gofixtures for the pointer field.gofmt -l— clean.go vet ./...(whole repo, including test files) — clean.golangci-lint run ./autoclaim/... ./config/... ./cmd/...(v2.4.0) — 0 issues (onemndfinding on the binary-search midpoint calculation addressed with//nolint:mnd, matching the existing convention intypes/block_finality.go).Also updates
docs/autoclaim.mdThe default-value callouts and config-reference table rows for
StartBlock/StartL1Blockdescribed the old "default
0" behavior directly; left as-is they would have actively misledoperators about the very fields this fix changes, so they are updated in place alongside two new
StartLookbackrows. No other content in that file changed.Out of scope (follow-ups, not implemented here)
window (the datadir-wipe gap hazard from section 1's incident — a real problem, but too invasive
for this PR).
🐞 Issues
Cross-references agglayer/aggkit#1874 and
0xPolygon/zkevm-ethtx-manager#139 (does
not close either).
This PR removes the cause (an unconfigured
EthTxManagerreaching zero values), but the deadlockfiled as zkevm-ethtx-manager#139 remains a separate, module-level defect: an underpriced-at-creation
transaction can never be re-priced, because re-pricing is gated on
MonitoredTxStatusSent, a statusthat transaction never reaches. A wedged claimer created before this fix ships still needs the
operational recovery described there; this PR only prevents new claimers from getting into that state.
📝 Notes
zkevm-ethtx-managerchanges.L1ToL2BridgeDetector.StartBlock/L2ToLxBridgeDetector.StartL1Blockwill now resolve a recentblock via
StartLookback(default 24h) instead of scanning from genesis. A deployment that alreadysets either field explicitly, to any value including
0, is unaffected.resolution algorithm and a config-template default, not just a defaulting pass. The owner chose to
include it here as a second small operational fix from the same deployment window, but it is the
one most defensible as its own PR if reviewers would rather split it out.
IgnoreNetworkIDsskip-source logging change that was previously part of this PR has beendropped from here. It is now handled in
#1882, together with the underlying
l2-to-lxcursor fix that log line is the visible symptom of — changing the log level alone would have hidden
the evidence while leaving the bug in place.
🤖 Generated with Claude Code
https://claude.ai/code/session_01DJuYNKE9jRLRrCWTrrEJ2T