tx-firehose: yet another load generator#6613
Open
ch1bo wants to merge 12 commits into
Open
Conversation
ch1bo
added
the
leios
ch1bo
force-pushed
the
ch1bo/tx-firehose
branch
2 times, most recently
from
5f81a15 to
5ef395c
Compare
A minimal single-node load generator driven over node-to-client. Discovers its own UTxO via QueryUTxOByAddress, submits transactions with a persistent LocalTxSubmission session, and reacts to accept/reject on every tx. Contrast with tx-centrifuge: N2C means stop/start on demand works (rejects surface directly), rejection reasons are logged, and there is no "optimistic recycle" divergence between the local fund set and ledger truth — a periodic UTxO requery replaces the local set to bound drift. Currently Conway-only.
The Leios devnet runs in DijkstraEra; ensureConway would refuse to start against it. Switch the hardcoded era from Conway to Dijkstra. The tx body content is otherwise the same shape — no plutus, explicit fee, no metadata — so this is a straight rename of era witnesses. If/when this needs to run against a Conway node again, generalise the tx builder over a ShelleyBasedEra evidence rather than adding a runtime branch.
The stable Cardano.Api.createTransactionBody path errors out on Dijkstra
inside caseShelleyToBabbageOrConwayEraOnwards — the branch that would
extract vote / proposal script witnesses hasn't been ported to
Dijkstra's cert model yet. See:
Cardano.Api.Era.Internal.Case: "TODO Dijkstra: caseShelleyToBabbage
OrConwayEraOnwards: Dijkstra requires a separate cert path"
makeUnsignedTx from Cardano.Api.Experimental bypasses that path and is
already Dijkstra-complete. The rewritten Tx.hs uses it end-to-end:
Exp.defaultTxBodyContent
|> setTxIns (with AnyKeyWitnessPlaceholder for each key input)
|> setTxOuts (Exp.TxOut . toShelleyTxOutAny sbe . Api.TxOut ...)
|> setTxFee
|> Exp.makeUnsignedTx Exp.DijkstraEra
|> Exp.makeKeyWitness + Exp.signTx
|> Api.ShelleyTx sbe ledgerTx -- back to old-API Tx era
The last step lets Main.hs keep using TxInMode / LocalTxSubmission
without any further changes.
Under LANG=C (the default in the nix sandbox / process-compose) GHC
gives stderr a locale encoder that can't encode em-dashes ('\8212').
The first non-ASCII byte throws 'hPutChar: invalid argument' mid-write,
which garbles the log line and drops the message.
Two belt-and-braces fixes:
* Replace em-dashes with '-' in runtime strings so the ASCII-only
stderr encoder is happy regardless of locale.
* Force stderr to UTF-8 + LineBuffering at the top of main so any
future non-ASCII messages don't repeat the problem, and each log
line flushes on its own.
Shelley initialFunds with a delegator setup live at base addresses (payment credential + stake credential). Deriving an enterprise address from just the payment key looks up a different, empty address. Add an optional stakingKeyFile config field: when set, we load the stake signing key, derive its verification key hash, and construct a base address. When omitted, behaviour is unchanged (enterprise address), matching the genesis-utxo-key case. Note that only the vkey hash is used — the stake key does not have to sign anything, so we don't need to keep the signing key around after boot.
Reconciling the local fund set against on-chain UTxO during a run was buying almost nothing: the local set diverges by design (we recycle mempool outputs optimistically), and a periodic replace-reconcile introduced its own race — replacing while descendants were still in mempool caused a wave of AllInputsAreSpent rejects. Simpler: keep the local set append-only during a run, count consecutive rejects, and exit with a message when the count crosses a configurable threshold (default 50). A supervisor restart then requeries UTxO from a clean slate. Configuration: - Removed: reconcileEvery - Added: maxConsecutiveErrors (default 50) The reconcileLoop function and its Async.race_ wiring are gone; main now just links the persistent submission client and runs the submit loop in the foreground.
Most of the config was static per-run and the JSON round-trip added more code than it saved. Replace it with an optparse-applicative parser whose flag names mirror cardano-cli conventions: --socket-path (was JSON socketPath) --testnet-magic (was JSON networkMagic) --signing-key-file (was JSON signingKeyFile) --staking-key-file (was JSON stakingKeyFile) --tps --inputs-per-tx --outputs-per-tx --fee --max-consecutive-errors Net: deletes the Config module (~60 lines of Aeson boilerplate) in exchange for a ~30-line parser inline in Main. `--help` and error messages come for free. New dep: optparse-applicative (transitive in this repo already).
The proto-devnet Alloy pipeline expects log lines in the cardano-node
trace schema (@{at, sev, host, thread, ns, data}@) so it can extract
namespaces and drive Grafana panels off Loki queries filtered by
@ns=...@. Plain-text stderr got dropped by the JSON-parse stage in
extract_cardano_node_logs.
Add a small @trace@ helper that emits one JSON line per event with
that schema, and switch every hPutStrLn site over. Namespaces are
hierarchical so dashboards can filter with exact-match or a prefix:
TxFirehose.Startup.Query -- one at boot; data: address
TxFirehose.Startup.Seeded -- one at boot; data: utxos, totalLovelace
TxFirehose.Submit.Success -- one per accepted tx; data: tx
TxFirehose.Submit.Reject -- one per rejected tx; data: tx, reason
TxFirehose.Build.Fail -- buildTx errors
TxFirehose.Exit.MaxErrors -- consecutive-rejects kill switch fired
Throughput = rate(Submit.Success), reject rate = rate(Submit.Reject).
Two small trace payload fixes on the Submit.Success / Submit.Reject
events:
- "tx" -> "txId", serialised via serialiseToRawBytesHexText. Was
T.pack (show txId), which round-tripped through Show and gave
"\"aba32...\"" - a JSON string with embedded quotes.
- Add "size": the wire size of the tx (BS.length of serialiseToCBOR).
Lets downstream dashboards compute byte-rate directly instead of
multiplying tx-count by an assumed constant.
Sample after:
{"data":{"txId":"adebeacd...","size":228},"ns":"TxFirehose.Submit.Reject",...}
Api.getTxBody is deprecated in favour of Cardano.Api.Experimental's UnsignedTx, and with -Werror on the deprecation warning the Nix build now fails. Rather than round-trip through a new experimental function, compute both the txId and the wire size directly on the ledger tx we already have in hand. Pattern taken from Test.ThreadNet.Leios in ouroboros-consensus: txIdTx :: EraTx era => Tx l era -> TxId -- Cardano.Ledger.Core sizeTxF :: SimpleGetter (Tx l era) Word32 -- Cardano.Ledger.Core buildTx now returns a BuiltTx record with the signed api-Tx, ledger txId, wire size, and output funds. Main.hs uses the fields directly, so it no longer needs Api.getTxBody, Api.getTxId, Api.serialiseToCBOR, or the manual serialiseToRawBytesHexText step — TxId has a hex-string ToJSON instance from cardano-ledger.
Sweeping refactor covering four things at once: 1. Structure Main.hs top-down. Options + parser, then main, then runFirehoseInEra, then mkFirehoseClient, then trace, then startup helpers, then queries. Reader can follow from the entry point downward without hopping. 2. No separate submission thread, no queue, no TVar. The LocalTxSubmission client itself carries the loop's state - fund set (`Map TxIn Integer`) and consecutive-error counter (`!Int`) - as strict recursive parameters through its continuations. No Async.withAsync, no TQueue, no IORef. 3. Query the node's current era at boot and dispatch the whole pipeline on the resulting existential (via caseByronOrShelleyBasedEra). deriveAddress / queryFundsInEra / mkFirehoseClient all take a `ShelleyBasedEra era` witness rather than hardcoding Dijkstra. Byron gets a clean rejection. 4. Tx builder is era-generic. Tx.buildTx uses the ledger type classes (EraTx / EraTxBody / EraTxOut / EraTxWits, via ShelleyBasedEraConstraints) with mkBasicTxBody + lens edits. Signing goes through Api.makeShelleyKeyWitness' which is era-generic and Dijkstra-safe (unlike createTransactionBody). The internal Fund set is ledger-typed (L.TxIn) now; api ↔ ledger conversion happens only at the query and submit boundaries. Fund set updates: - success: remove consumed inputs, add fresh outputs - reject: keep inputs (never touched) - build fail: keep inputs, bump the error counter
Sweep the tx-firehose sources to use where clauses instead of let ... in wherever it's in scope. Only the let bindings that close over IO-bound values or lambda-captured variables (where a where clause can't reach) are kept as let. Also: switch feeLovelace's local pattern-let to a where-level pattern binding on the Coin newtype for the same reason.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Vibe coded this last night when I needed a "restartable" load generator for a local Leios testnet.
This one uses N2C to query the utxo owned by a key and starts infinite re-spending at a given rate and input/output tx structure. Stops if txs are rejected by the node (> 50 errors) -> can be restarted by orchestration to pick up load.
Note, any txs still in the mempool would not be seen by the initial query and thus have the firehose not be able to start spending -> need to wait for the firehose txs to be cleared from mempool.