Skip to content

feat(sdk): v0.2.0 — feature parity with Arc server + hardening#2

Open
xe-nvdk wants to merge 1 commit into
mainfrom
feat/sdk-0.2.0-feature-parity
Open

feat(sdk): v0.2.0 — feature parity with Arc server + hardening#2
xe-nvdk wants to merge 1 commit into
mainfrom
feat/sdk-0.2.0-feature-parity

Conversation

@xe-nvdk

@xe-nvdk xe-nvdk commented Jul 3, 2026

Copy link
Copy Markdown
Member

Summary

Brings the Python SDK (arc-tsdb-client) up to date with the Arc server after a long gap, fixes several confirmed bugs, and adds the first real HTTP-layer test coverage. Ships as v0.2.0. Sync and async paths are both covered throughout.

New capabilities

  • query_msgpack() — experimental columnar MessagePack query response with native timestamp decoding; returns the same QueryResponse shape as query() (plus per-column Arrow types). Raises ArcNotSupportedError on 501 (server built without the duckdb_arrow tag).
  • Bulk import (client.import_) — CSV, Parquet, line-protocol file, and TLE. Accepts paths, file-like objects, or bytes.
  • TLE streaming write (write.write_tle) + tle_stats.
  • Databases management (client.databases) — list / get / list_measurements / create / delete.
  • Full enterprise surface (sync + async) — rbac, governance, audit, queries (query management), cluster, compaction, tiering, backup, mqtt.
  • zstd write compression — now the default (gzip/none still supported) via a Compression enum. No Content-Encoding header (server auto-detects from magic bytes).
  • Automatic retries — exponential backoff + jitter for connection errors, 429 (honoring Retry-After), and 503.
  • New exceptions: ArcNotSupportedError (501), ArcLicenseError (403 for a missing enterprise-license feature).

Fixes

  • Polars datetime→columnar used a fragile private attribute that raised at runtime; now casts via the public polars API and normalizes time_unit.
  • Pandas timestamp scaling honored only nanoseconds; now respects the column resolution — including tz-naive datetime64 columns whose dtype has no .unit (read via numpy.datetime_data). These previously wrote timestamps ~1000× too small (landing in ~1970). Caught by adversarial review after an initial incomplete fix.
  • Buffered writer now runs a background flush timer (thread for sync, task for async) so idle buffers still flush on flush_interval.
  • show_tables validates the database identifier before SQL interpolation.
  • User-Agent derived from the package version (single source of truth); removed the dead dataframe package stub.

Testing

  • First real HTTP round-trip coverage via pytest-httpx: header assembly, error mapping (incl. 501/license), on-the-wire gzip/zstd magic bytes, retry/backoff, msgpack columnar decode, imports, databases, and enterprise wrappers — sync and async.
  • Opt-in live integration tests under tests/integration (gated on ARC_TEST_URL).
  • 167 unit tests passing; ruff + ruff-format + mypy clean.

Review

Four independent adversarial reviewers ran over the diff (async/sync twin parity, wire-protocol fidelity vs the Go server, concurrency/retry correctness, algorithmic edge-cases). Three came back clean; the algorithmic pass surfaced the tz-naive timestamp corruption bug above, which is now fixed and covered by a 4 resolutions × {tz-naive, tz-aware} regression matrix.

🤖 Generated with Claude Code

Bring the Python SDK up to date with the Arc server and fix several
confirmed bugs. Sync and async paths are both covered throughout.

New capabilities:
- query_msgpack(): experimental columnar MessagePack query response with
  native timestamp decoding; raises ArcNotSupportedError on 501 (server
  built without the duckdb_arrow tag).
- Bulk import client (client.import_): CSV, Parquet, line-protocol file,
  and TLE; accepts paths, file-like objects, or bytes.
- TLE streaming write (write.write_tle) + tle_stats.
- Databases management (client.databases): list/get/list_measurements/
  create/delete.
- Full enterprise surface (sync + async): rbac, governance, audit,
  queries (query management), cluster, compaction, tiering, backup, mqtt.
- zstd write compression (now the default; gzip/none still supported)
  via a Compression enum; no Content-Encoding header (server auto-detects
  from magic bytes).
- Automatic retries with exponential backoff + jitter for connection
  errors, 429 (honoring Retry-After), and 503.
- New exceptions: ArcNotSupportedError (501), ArcLicenseError (403 for a
  missing enterprise-license feature).

Fixes:
- Polars datetime->columnar used a fragile private attribute that raised
  at runtime; now casts via the public polars API and normalizes time_unit.
- Pandas datetime timestamp scaling respected only nanoseconds; now honors
  the column resolution, including tz-naive numpy datetime64 columns whose
  dtype has no .unit (read via numpy.datetime_data) — these previously
  wrote timestamps ~1000x too small.
- Buffered writer now runs a background flush timer (thread for sync, task
  for async) so idle buffers still flush on flush_interval.
- show_tables validates the database identifier before SQL interpolation.
- User-Agent derived from the package version (single source of truth);
  removed the dead dataframe package stub.

Testing:
- First real HTTP round-trip coverage via pytest-httpx (headers, error
  mapping, on-the-wire compression, retries, msgpack decode, imports,
  databases, enterprise wrappers) for both sync and async.
- Opt-in live integration tests under tests/integration (gated on
  ARC_TEST_URL). 167 unit tests passing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant