Skip to content

Rust Iterator parity: dedup_with_count, scan, is_sorted, eq - #22

Open
virgesmith wants to merge 3 commits into
mainfrom
feat/rust-iterator-parity
Open

Rust Iterator parity: dedup_with_count, scan, is_sorted, eq#22
virgesmith wants to merge 3 commits into
mainfrom
feat/rust-iterator-parity

Conversation

@virgesmith

Copy link
Copy Markdown
Owner

Supersedes #21, which GitHub closed when its head branch was renamed (feat/dedup-with-countfeat/rust-iterator-parity) to match the widened scope. Same branch, same commits, plus two more.

Four methods bringing Itr closer to Rust's Iterator trait, each addressing a gap with no clean Python equivalent.

dedup_with_count() — run-length encoding

Getting the length of each consecutive run previously meant chunk_by(lambda x: x).map(lambda x: (x[0], len(x[1]))) — clunky, and it materialises every run just to measure it.

Itr([4, 4, 2, 3, 3, 1]).dedup_with_count().collect()
# ((4, 2), (2, 1), (3, 2), (1, 1))
  • Extends the existing dedup family, as Rust's itertools crate pairs them, rather than inventing a run_lengths/rle concept.
  • Yields (item, count) — deliberately the reverse of Rust's dedup_with_count, to match value_counts's existing Itr[tuple[T, int]] shape. Consistency within the library beats consistency with Rust here; flagged in a comment, the docstring and the relnotes.
  • The lazy, positional counterpart to value_counts: adjacent runs in source order vs occurrences overall. A test pins the divergence.
  • Counts with sum(1 for _ in g) rather than len(tuple(g)), so a run is consumed without being materialised.

scan(init, func) — stateful map with early exit

The one genuine capability gap against the trait: accumulate yields intermediates but forces the state to share the items' type and cannot stop early.

Itr([1, 2, 3, 4]).scan(0, lambda total, x: None if total + x > 5 else (total + x, total + x)).collect()
# (1, 3)
  • func(state, item) returns (new_state, output), or None to stop.
  • Rust signals termination with Option; Python has no such type, but the halt signal here is the entire return value, not the output value — so (new_state, None) still yields None as an output and the two are never ambiguous. This is why scan is safe to add where filter_map would not be (its None-as-drop sentinel is genuinely ambiguous when None is a legitimate mapped value).

is_sorted(key=None, *, reverse=False)

No Python builtin does this; list(it) == sorted(it) materialises twice and cannot short-circuit.

  • Non-strict, so runs of equal items are sorted, and empty/single-item iterators are sorted — matching Rust.
  • key covers Rust's is_sorted_by_key and matches the max/min signature convention; reverse is an addition for symmetry with sorted_by, since checking descending order otherwise has no workaround short of materialising.
  • Short-circuits at the first inversion.

eq(other)

Element-wise comparison that short-circuits at the first difference, where tuple(a) == tuple(b) materialises both sides. Documented explicitly: Itr defines no __eq__, so == remains an identity check and .eq() is the content comparison. A test confirms it terminates against an infinite iterator.

Not included, deliberately

filter_map and find_map (the None sentinel is ambiguous in a language without Option.map(f).filter(lambda x: x is not None) is explicit about that assumption at the call site), the comparison family beyond eq (cmp/lt/ge/partial_cmp — lexicographic ordering nobody reaches for in Python), fuse, advance_by, rposition, is_partitioned, intersperse_with, and everything untranslatable (by_ref, copied/cloned, collect_into, partition_in_place, size_hint, the try_* family).

Notes

  • is_sorted widens the keyed sequence to Iterable[Any] via cast, following the existing sorted_by/groupby idiom: T is unbounded, so it is not known to be orderable, and ty correctly rejects <= on it.
  • src/itrx/skill/SKILL.md (added in Add installable agent skill (itrx-skill) #20) is a third place a new public method must be recorded, alongside README.md and doc/apidoc.md. All four methods are added to its lists, with gotcha entries for each. AGENTS.md's reviewer checklist item 8 still names only the README and apidoc — it should name SKILL.md too, or the shipped agent reference drifts silently, with nothing in the test suite to catch it. Left for a separate PR.

Verification

  • ruff check, ruff format --check, ty check src clean; pytest 222 passed, coverage 100%.
  • 29 new tests across test_aggregation.py and test_transform_filter.py, covering empty/single-element/unhashable inputs, laziness on infinite sources, short-circuiting, and agreement with the neighbouring methods (dedup, sorted_by, value_counts).
  • doc/apidoc.md regenerated via uv run python src/scripts/introspect.py; README lazy/eager lists updated; relnotes entries under ## Unreleased.

🤖 Generated with Claude Code

virgesmith and others added 3 commits September 1, 2026 09:27
Getting run lengths previously meant
`chunk_by(lambda x: x).map(lambda x: (x[0], len(x[1])))`, which is clunky
for what is a common operation, and materialises each run.

`dedup_with_count` extends the existing `dedup` family (Rust's itertools
crate pairs them the same way) and is the lazy, positional counterpart to
`value_counts`: same `Itr[tuple[T, int]]` output shape, but counting
adjacent runs rather than occurrences overall, preserving order, comparing
by equality rather than requiring hashability, and staying lazy on infinite
sources (provided no individual run is infinite).

The (item, count) ordering deliberately matches `value_counts` rather than
Rust's dedup_with_count, which yields (count, item); noted in a comment and
the relnotes.

Counts via `sum(1 for _ in g)` rather than `len(tuple(g))` so a run is
consumed without being materialised.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant