feat(CLI): add tg batches commands for the batch API by blainekasten · Pull Request #525 · togethercomputer/together-py

blainekasten · 2026-08-13T13:03:48Z

Fixes DX-918.

Adds CLI commands for the Batch API:

tg batches submit FILE_ID_OR_PATH API_TYPE MODEL
tg batches ls
tg batches get|retrieve BATCH_ID
tg batches download BATCH_ID [--output PATH]
tg batches cancel BATCH_ID

Submit

FILE_ID_OR_PATH behaves like ft create: local paths are uploaded with purpose=batch-api, otherwise treated as a file ID
API_TYPE is chat.completions | audio.transcriptions | audio.translations, positional or --api
MODEL is positional or --model/-M
Optional --completion-window and --priority

Retrieve

Human output is a curated summary (created/completed time, API, model, output/error file IDs)
In-progress jobs show a progress bar; completed/failed jobs print a download hint
tg batches <batch-id> implicit retrieve

Download

Writes output (and error) files to --output (file or directory)
Omitting --output prints the output file to stdout
Refactors file-content download into a shared helper used by tg files download

Also supports --json and ls/get aliases.

Linear Issue: DX-918

Add submit, list/ls, get/retrieve, and cancel under `tg batches`, including local-file upload (like `ft create`), positional + `--api`/`--model`/`-M` flags, and JSON output. Co-authored-by: Blaine Kasten <blainekasten@gmail.com>

broly-code-security-scanner · 2026-08-13T13:03:57Z

Broly Security Scan

Note

✅ Clean scan
No vulnerabilities detected in this PR.

Note

Re-scan this PR anytime with /broly scan — useful after /broly undismiss, or to refresh findings without a new push.

Broly — SAST (zai-org/GLM-5.2) · Secrets · SCA · IaC · GH Actions · Base Images · Supply Chain Threats · Exploit Chains · Adversarial Verification

We're continuously improving Broly's accuracy and finding quality — your feedback is valuable. False positives, missed findings, bugs, and feature requests all welcome.

blainekasten · 2026-08-13T19:39:42Z

@cursoragent fix tests and update the PR description for changes I made

cursor · 2026-08-13T19:39:50Z

Fixed the three CI failures and updated the PR description for your download/output changes.

tg batches <id> was broken: a duplicate "batches": UUID key overwrote the batch* matcher. Restored implicit retrieve.
Retrieve tests now match the curated output (no raw COMPLETED dump; in-progress shows In_progress + bar, not Progress:).

PR: #525 — 43 CLI tests passed.

The UUID identifier overwrote the `batch*` prefix matcher, so `tg batches <batch-id>` stopped resolving to retrieve. Align retrieve assertions with the curated human output (no raw COMPLETED dump, progress bar uses In_progress). Co-authored-by: Blaine Kasten <blainekasten@gmail.com>

zainhas · 2026-08-20T22:26:11Z

+    if output is not None:
+        saved: list[dict[str, str]] = []
+        directory_output = _is_directory_output(output)
+        error_file_id = job.error_file_id
+
+        if job.output_file_id:
+            out_path = await download_file_content(
+                config.client,
+                job.output_file_id,
+                output=output,
+                loading_message="Downloading batch output...",
+            )
+            assert isinstance(out_path, Path)
+            saved.append({"kind": "output", "id": job.output_file_id, "path": str(out_path)})
+        elif error_file_id and not directory_output:
+            # No output file — write the error file to the exact path the user asked for.
+            err_path = await download_file_content(
+                config.client,
+                error_file_id,
+                output=output,
+                loading_message="Downloading batch errors...",
+            )
+            assert isinstance(err_path, Path)
+            saved.append({"kind": "error", "id": error_file_id, "path": str(err_path)})
+            error_file_id = None
+
+        if error_file_id:
+            err_dest = output if directory_output else _error_output_path(output)
+            err_path = await download_file_content(
+                config.client,
+                error_file_id,
+                output=err_dest,
+                loading_message="Downloading batch errors...",
+            )
+            assert isinstance(err_path, Path)
+            saved.append({"kind": "error", "id": error_file_id, "path": str(err_path)})


When --output is a directory and the job has both an output and an error file, both downloads go through download_file_content(output=
), which names each file from whatever the Files API returns for it. If those two filenames match, the second write clobbers the first — and we still print both "Output saved to …" and "Errors saved to …" as if two files landed. I mocked both files returning filename: "batch.jsonl" and ended up with a single batch.jsonl containing only the error content.

_error_output_path() already solves this for the concrete-file case; the directory case needs the same guarantee (suffix on collision, or force .errors into the error filename).

zainhas · 2026-08-20T22:27:08Z

+    console.print(raw.decode("utf-8"))
+    if job.error_file_id:
+        console.print(f"\n[dim]Error file also available: tg batches download {id} --output ./out[/dim]")


The default (no --output) path is console.print(raw.decode("utf-8")), so Rich does two things to the payload: it parses markup, and it hard-wraps at the console width. With a completion containing [bold]…[/bold] and a line over 80 chars, the tags get eaten and newlines get injected mid-JSON — > results.jsonl produces invalid JSONL. Model output containing an unbalanced tag like [/close] raises MarkupError and kills the command outright.

Since this is the default mode of the command, I think it has to be sys.stdout.buffer.write(raw) (or at minimum a markup=False, soft_wrap=True console). Two related things in the same block:

No UnicodeDecodeError guard here, even though the --json branch right above has one — a non-UTF-8 byte traces back instead of falling back.

The "Error file also available: …" hint on line 139 goes to stdout, so it becomes the last line of a redirected results.jsonl. Hints belong on stderr.

zainhas · 2026-08-20T22:28:06Z

+    if job.status in _INCOMPLETE_STATUSES and job.progress is not None:
+        console.print(f"{format_progress(job.progress)} {format_status(job.status)}")


print_batch_detail only prints a status line when status in _INCOMPLETE_STATUSES and progress is not None. So on a CANCELLED job you get created-at, the API, the model, and nothing else — no indication it was cancelled. Same for EXPIRED, for FAILED with no error field, and for VALIDATING before progress is populated. STATUS_COLORS and format_status() already cover all six states, so the four terminal ones are effectively dead code. Can we just always print the status line?

zainhas · 2026-08-20T22:28:44Z

+    if job is None or not job.id:
+        console.print("[red]x[/red] Batch job was not created")
+        if response.warning:
+            console.print(response.warning)
+        return


If the API comes back with {"job": null, "warning": …} we print x Batch job was not created and then return, so the shell sees success. tg batches submit … && next-step will happily keep going against a batch that doesn't exist. Needs sys.exit(1).

zainhas · 2026-08-20T22:29:31Z

+def _is_directory_output(path: Path) -> bool:
+    return path.is_dir() or path.suffix == ""


--output ./results crashes when ./results already exists as a file: _is_directory_output() treats any suffix-less path as a directory, and since the validator here was loosened to file_okay=True (files download uses file_okay=False), that path now reaches output.mkdir(parents=True, exist_ok=True) inside download_file_content — and exist_ok does not tolerate an existing non-directory:

touch ./results && tg batches download --output ./results
→ Error: [Errno 17] File exists: '/…/results'

An explicit output.exists() and not output.is_dir() check before the mkdir would sort it, or restore file_okay=False.

zainhas · 2026-08-20T22:35:36Z

+    if not job.output_file_id:
+        console.print(
+            "[red]Batch job has no output file[/red]. "
+            "Use [primary]--output[/primary] to download the error file instead."
+        )
+        sys.exit(1)


All three early exits print Rich prose to stdout regardless of config.json, so tg batches download --json | jq fails on non-JSON input. I noticed download is the one command excluded from the new test_json_mode_pipeable_to_jq case (tests/cli/test_json_mode_pipeable_to_jq.py:100) — if that's why, I'd rather fix the output than skip the check.

zainhas · 2026-08-20T22:39:33Z

+        )
+        sys.exit(1)
+
+    raw = await download_file_content(


response.read() pulls the entire batch output into RAM, and in --json mode we then build a second full copy as a string (a third, base64, on decode failure). Batch outputs can be very large by design, and download_file_content already has a streaming write_to_file path for --output. tg batches download | head shouldn't need the whole body resident.

zainhas · 2026-08-20T22:41:25Z

+    if job.error_file_id:
+        console.print(f" - [red]An error occurred[/red]")
+        console.print(f" - Error file ID {job.error_file_id}")
+
+    if job.error:
+        console.print(f" - [red]An error occurred[/red]")


nit: A job with both error_file_id and error set renders the header line once per block:

An error occurred

Error file ID file-err

An error occurred
boom

zainhas · 2026-08-20T22:42:00Z

+    console.print(f"[green]√ Batch job submitted.[/green] [dim]({job.id})[/dim]")
+    if response.warning:
+        console.print(f"[yellow]{response.warning}[/yellow]")
+    print_model_dump(job, show_nulls=False)


BatchJob declares x_model_id = FieldInfo(alias="model_id"), and print_model_dump renders model_dump() (field names, not aliases), so submit shows a row literally labelled "X Model Id:". cancel is worse — the top-level parse keeps model_id as an extra field alongside x_model_id, so it prints both rows with the same value (I confirmed this against the real construct_type path, not just a plain model_validate). retrieve sidesteps the whole thing with its curated view; submit/cancel want either the same treatment or a by_alias dump.

zainhas · 2026-08-20T22:42:11Z

+        return
+
+    console.print("[green]√[/green] Cancelled batch job")
+    print_model_dump(response, show_nulls=False)


BatchJob declares x_model_id = FieldInfo(alias="model_id"), and print_model_dump renders model_dump() (field names, not aliases), so submit shows a row literally labelled "X Model Id:". cancel is worse — the top-level parse keeps model_id as an extra field alongside x_model_id, so it prints both rows with the same value (I confirmed this against the real construct_type path, not just a plain model_validate). retrieve sidesteps the whole thing with its curated view; submit/cancel want either the same treatment or a by_alias dump.

feat(CLI): add tg batches commands for the batch API

a28b45c

Add submit, list/ls, get/retrieve, and cancel under `tg batches`, including local-file upload (like `ft create`), positional + `--api`/`--model`/`-M` flags, and JSON output. Co-authored-by: Blaine Kasten <blainekasten@gmail.com>

blainekasten marked this pull request as ready for review August 13, 2026 13:03

add download and improve outputs

29b51ee

blainekasten requested review from Nutlope and zainhas August 13, 2026 19:41

zainhas requested changes Aug 20, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat(CLI): add tg batches commands for the batch API - #525

feat(CLI): add tg batches commands for the batch API#525
blainekasten wants to merge 3 commits into
mainfrom
cursor/cli-batches-commands-c96a

blainekasten commented Aug 13, 2026 •

edited by cursor Bot

Loading

Uh oh!

broly-code-security-scanner Bot commented Aug 13, 2026

Uh oh!

blainekasten commented Aug 13, 2026

Uh oh!

cursor Bot commented Aug 13, 2026 •

edited

Loading

Uh oh!

zainhas Aug 20, 2026

Uh oh!

zainhas Aug 20, 2026

Uh oh!

zainhas Aug 20, 2026

Uh oh!

zainhas Aug 20, 2026

Uh oh!

zainhas Aug 20, 2026

Uh oh!

zainhas Aug 20, 2026

Uh oh!

zainhas Aug 20, 2026

Uh oh!

zainhas Aug 20, 2026

Uh oh!

zainhas Aug 20, 2026

Uh oh!

zainhas Aug 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		if job.status in _INCOMPLETE_STATUSES and job.progress is not None:
		console.print(f"{format_progress(job.progress)} {format_status(job.status)}")

		def _is_directory_output(path: Path) -> bool:
		return path.is_dir() or path.suffix == ""

Uh oh!

Conversation

blainekasten commented Aug 13, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

broly-code-security-scanner Bot commented Aug 13, 2026

Broly Security Scan

Uh oh!

blainekasten commented Aug 13, 2026

Uh oh!

cursor Bot commented Aug 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

blainekasten commented Aug 13, 2026 •

edited by cursor Bot

Loading

cursor Bot commented Aug 13, 2026 •

edited

Loading