Abandon schedules far larger than the fragment they would replace - #2
Merged
Merged
Conversation
A constant ROLL or PICK index in the tens of thousands makes a fragment need that many inputs. When the operation moves the deepest of them, finish() arranges every one back into place, scanning the stack for each: quadratic. OP_DUP OP_DROP 0x0159e3 OP_ROLL OP_SWAP OP_DROP took 75 s to optimize, and the wide fuzz test in #1 hit a script like it in CI (410 s on Node 24). A schedule is only used when it is smaller than the fragment, so the machine now stops once it has emitted more than twice the fragment's size plus 16 bytes, and that variant is skipped. The generic arrangement of a short tail is exempt, because the search that may beat it runs afterwards. That script now takes 3.8 s with the same 8-byte result. The Miller loop and the final exponentiation come out byte for byte as before, as do the 19 module scripts checked, field-300.hex, sha256.block and g1.inSubgroup.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
A constant
ROLL/PICKindex in the tens of thousands makes a fragment need that many inputs. When the operation moves the deepest one,Machine.finish()→arrange()puts every input back in place and scans the stack for each one, which is quadratic:The wide fuzz test in #1 generated a 53-byte script like this, which is why its Node 24 job took 7m21s (the test itself: 410 s) while Node 20 and 22 took under a minute.
Fix
A reschedule is only used when it is smaller than the fragment. The scheduler now counts the bytes it emits and gives up on a variant once it passes
2 × fragment + 16bytes. The limit is set above the fragment size because peephole runs afterwards. Hitting it only skips that variant; the output is never wrong. The generic arrangement of a short tail (≤ 6 items) is exempt, because the stack search that may beat it runs afterwards. Beam candidates finish under the same limit, one at a time.Unchanged output
Optimized at medium effort before and after, compared by SHA-256 of the output:
pairing.miller(63)pairing.finalExpsha256.blockg1.inSubgroupexamples/scripts/field-300.hex19 script-high-level-modules scripts (int, sha256, hmac, ec, fp2/fp6/fp12, g2 steps) also have identical sizes (74,362 → 57,462 total), with every honest case passing and every refusal case refused.
Tests
New unit test: the script above optimizes, verifies and stays ≤ 8 bytes, with a 40 s test timeout (it took 75 s before).
npm test: 24/24.