feat(sdk): v0.2.0 — feature parity with Arc server + hardening#2
Open
xe-nvdk wants to merge 1 commit into
Open
feat(sdk): v0.2.0 — feature parity with Arc server + hardening#2xe-nvdk wants to merge 1 commit into
xe-nvdk wants to merge 1 commit into
Conversation
Bring the Python SDK up to date with the Arc server and fix several confirmed bugs. Sync and async paths are both covered throughout. New capabilities: - query_msgpack(): experimental columnar MessagePack query response with native timestamp decoding; raises ArcNotSupportedError on 501 (server built without the duckdb_arrow tag). - Bulk import client (client.import_): CSV, Parquet, line-protocol file, and TLE; accepts paths, file-like objects, or bytes. - TLE streaming write (write.write_tle) + tle_stats. - Databases management (client.databases): list/get/list_measurements/ create/delete. - Full enterprise surface (sync + async): rbac, governance, audit, queries (query management), cluster, compaction, tiering, backup, mqtt. - zstd write compression (now the default; gzip/none still supported) via a Compression enum; no Content-Encoding header (server auto-detects from magic bytes). - Automatic retries with exponential backoff + jitter for connection errors, 429 (honoring Retry-After), and 503. - New exceptions: ArcNotSupportedError (501), ArcLicenseError (403 for a missing enterprise-license feature). Fixes: - Polars datetime->columnar used a fragile private attribute that raised at runtime; now casts via the public polars API and normalizes time_unit. - Pandas datetime timestamp scaling respected only nanoseconds; now honors the column resolution, including tz-naive numpy datetime64 columns whose dtype has no .unit (read via numpy.datetime_data) — these previously wrote timestamps ~1000x too small. - Buffered writer now runs a background flush timer (thread for sync, task for async) so idle buffers still flush on flush_interval. - show_tables validates the database identifier before SQL interpolation. - User-Agent derived from the package version (single source of truth); removed the dead dataframe package stub. Testing: - First real HTTP round-trip coverage via pytest-httpx (headers, error mapping, on-the-wire compression, retries, msgpack decode, imports, databases, enterprise wrappers) for both sync and async. - Opt-in live integration tests under tests/integration (gated on ARC_TEST_URL). 167 unit tests passing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings the Python SDK (
arc-tsdb-client) up to date with the Arc server after a long gap, fixes several confirmed bugs, and adds the first real HTTP-layer test coverage. Ships as v0.2.0. Sync and async paths are both covered throughout.New capabilities
query_msgpack()— experimental columnar MessagePack query response with native timestamp decoding; returns the sameQueryResponseshape asquery()(plus per-column Arrowtypes). RaisesArcNotSupportedErroron 501 (server built without theduckdb_arrowtag).client.import_) — CSV, Parquet, line-protocol file, and TLE. Accepts paths, file-like objects, or bytes.write.write_tle) +tle_stats.client.databases) — list / get / list_measurements / create / delete.rbac,governance,audit,queries(query management),cluster,compaction,tiering,backup,mqtt.Compressionenum. NoContent-Encodingheader (server auto-detects from magic bytes).Retry-After), and 503.ArcNotSupportedError(501),ArcLicenseError(403 for a missing enterprise-license feature).Fixes
time_unit.datetime64columns whose dtype has no.unit(read vianumpy.datetime_data). These previously wrote timestamps ~1000× too small (landing in ~1970). Caught by adversarial review after an initial incomplete fix.flush_interval.show_tablesvalidates the database identifier before SQL interpolation.dataframepackage stub.Testing
pytest-httpx: header assembly, error mapping (incl. 501/license), on-the-wire gzip/zstd magic bytes, retry/backoff, msgpack columnar decode, imports, databases, and enterprise wrappers — sync and async.tests/integration(gated onARC_TEST_URL).Review
Four independent adversarial reviewers ran over the diff (async/sync twin parity, wire-protocol fidelity vs the Go server, concurrency/retry correctness, algorithmic edge-cases). Three came back clean; the algorithmic pass surfaced the tz-naive timestamp corruption bug above, which is now fixed and covered by a
4 resolutions × {tz-naive, tz-aware}regression matrix.🤖 Generated with Claude Code