From 3b8075ba189f948c4d85d6ac6af4c46f26a3fd80 Mon Sep 17 00:00:00 2001 From: Steven Roussey Date: Mon, 8 Jun 2026 21:18:47 -0700 Subject: [PATCH 1/4] feat: add typecheck budget guard to catch type-instantiation regressions (#555) * ci: add typecheck instantiation-budget guard + perf investigation writeup Adds scripts/typecheck-budget.ts, which type-checks each composite package in isolation and gates its (deterministic, machine-independent) instantiation count against committed per-package budgets in scripts/typecheck-budget.json. Wired into CI as the `typecheck-budget` job and exposed as `bun run typecheck:budget`. Also documents the typecheck slowdown investigation (timeline, bisected culprit commits, root-cause analysis, and prototyped fixes) in docs/technical/typecheck-performance-investigation.md. https://claude.ai/code/session_018SckHLTLmkxKCSWyb74Lke * perf(ai): replace FromSchema-derived types with concrete interfaces (AI Text slice) Proof-slice for the typecheck-cost work: the dominant `packages/ai` check cost is the `FromSchema` conditional-type machinery (json-schema-to-ts), not the Workflow/CreateWorkflow generics. Converting the plain `type X = FromSchema` aliases to concrete types removes that per-use instantiation cost. This slice converts the AI Text task family + the foundational model types (ModelConfig/ModelRecord). Each concrete type is the type checker's exact resolution of the original FromSchema type (verified by a clean full `tsc -b`), so it is equivalent to the prior derived type; the schema constants remain the runtime contract. Budget baseline ratcheted to match. Measured (pinned tsc, deps prebuilt, isolated `tsc -p`): - ai: 564,545 -> 520,877 instantiations (this 7-file slice) - full-ai conversion projects to ~320k instantiations / -39% check time, with a ~-14% ripple on packages/test (tracked in #556) Tradeoff: a hot-path `Equal>` drift guard is intentionally omitted because it would recompute FromSchema and defeat the win; the schemas stay the runtime source of truth. Refs #556. https://claude.ai/code/session_018SckHLTLmkxKCSWyb74Lke * perf(ai): convert remaining FromSchema-derived task types to concrete types Completes the FromSchema->concrete migration started in the AI Text slice. Converts the remaining plain and composite (Omit<...> / WithImageValuePorts<...>) type aliases across packages/ai to concrete types, removing the json-schema-to-ts conditional-type instantiation cost from the AI task input/output types. Each concrete type is the type checker's exact resolution of the original FromSchema type (verified by a clean full `tsc -b`); the schema constants remain the runtime contract. ModelConfig/ModelRecord and ChatMessage are referenced by name rather than inlined for readability. Four vector/embedding outputs that involve branded typed-array types (TextEmbedding, ImageEmbedding, VectorSimilarity) are intentionally left on FromSchema since they do not print to a clean concrete form. Measured (pinned tsc, deps prebuilt, isolated `tsc -p`): - ai: 564,545 -> 92,750 instantiations (-84%), 4.29s -> ~2.0s check (-52%) - test (ripple): 1,427,138 -> 947,215 instantiations (-34%), 12.4s -> ~9.2s Budget baselines ratcheted to match. Closes #556. https://claude.ai/code/session_018SckHLTLmkxKCSWyb74Lke --------- Co-authored-by: Claude --- .github/workflows/test.yml | 17 ++ .../typecheck-performance-investigation.md | 146 ++++++++++++ package.json | 1 + packages/ai/src/model/ModelSchema.ts | 33 ++- packages/ai/src/task/AiChatTask.ts | 36 ++- packages/ai/src/task/AiChatWithKbTask.ts | 67 +++++- packages/ai/src/task/BackgroundRemovalTask.ts | 15 +- packages/ai/src/task/ChunkRetrievalTask.ts | 45 +++- packages/ai/src/task/ChunkVectorUpsertTask.ts | 30 ++- packages/ai/src/task/ContextBuilderTask.ts | 22 +- packages/ai/src/task/CountTokensTask.ts | 7 +- packages/ai/src/task/DocumentEnricherTask.ts | 22 +- packages/ai/src/task/DocumentUpsertTask.ts | 19 +- packages/ai/src/task/FaceDetectorTask.ts | 32 ++- packages/ai/src/task/FaceLandmarkerTask.ts | 36 ++- packages/ai/src/task/GestureRecognizerTask.ts | 36 ++- packages/ai/src/task/HandLandmarkerTask.ts | 34 ++- .../ai/src/task/HierarchicalChunkerTask.ts | 36 ++- packages/ai/src/task/HierarchyJoinTask.ts | 46 +++- .../ai/src/task/ImageClassificationTask.ts | 22 +- packages/ai/src/task/ImageEmbeddingTask.ts | 14 +- packages/ai/src/task/ImageSegmentationTask.ts | 18 +- packages/ai/src/task/ImageToTextTask.ts | 19 +- packages/ai/src/task/KbAddDocumentTask.ts | 11 +- packages/ai/src/task/KbDeleteTask.ts | 6 +- packages/ai/src/task/KbReindexTask.ts | 6 +- packages/ai/src/task/KbSearchTask.ts | 10 +- packages/ai/src/task/KbToDocumentsTask.ts | 11 +- .../ai/src/task/ModelDownloadRemoveTask.ts | 6 +- packages/ai/src/task/ModelDownloadTask.ts | 6 +- packages/ai/src/task/ModelInfoTask.ts | 22 +- packages/ai/src/task/ModelSearchTask.ts | 8 +- packages/ai/src/task/ObjectDetectionTask.ts | 32 ++- packages/ai/src/task/PoseLandmarkerTask.ts | 63 ++++- packages/ai/src/task/QueryExpanderTask.ts | 15 +- packages/ai/src/task/RerankerTask.ts | 19 +- packages/ai/src/task/StructuralParserTask.ts | 15 +- .../ai/src/task/StructuredGenerationTask.ts | 14 +- packages/ai/src/task/TextChunkerTask.ts | 30 ++- .../ai/src/task/TextClassificationTask.ts | 12 +- packages/ai/src/task/TextEmbeddingTask.ts | 6 +- packages/ai/src/task/TextFillMaskTask.ts | 9 +- packages/ai/src/task/TextGenerationTask.ts | 15 +- .../ai/src/task/TextLanguageDetectionTask.ts | 11 +- .../task/TextNamedEntityRecognitionTask.ts | 17 +- .../ai/src/task/TextQuestionAnswerTask.ts | 11 +- packages/ai/src/task/TextRerankerTask.ts | 12 +- packages/ai/src/task/TextRewriterTask.ts | 7 +- packages/ai/src/task/TextSummaryTask.ts | 7 +- packages/ai/src/task/TextTranslationTask.ts | 12 +- packages/ai/src/task/ToolCallingTask.ts | 31 ++- packages/ai/src/task/TopicSegmenterTask.ts | 15 +- packages/ai/src/task/VectorQuantizeTask.ts | 14 +- .../ai/src/task/generation/ImageEditTask.ts | 16 +- .../src/task/generation/ImageGenerateTask.ts | 13 +- scripts/typecheck-budget.json | 40 ++++ scripts/typecheck-budget.ts | 219 ++++++++++++++++++ 57 files changed, 1258 insertions(+), 236 deletions(-) create mode 100644 docs/technical/typecheck-performance-investigation.md create mode 100644 scripts/typecheck-budget.json create mode 100644 scripts/typecheck-budget.ts diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 9055c8170..6ca8b17f7 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -18,6 +18,23 @@ env: TURBO_TELEMETRY_DISABLED: "1" jobs: + typecheck-budget: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v6 + - uses: actions/setup-node@v6 + with: + node-version: 24 + - uses: oven-sh/setup-bun@v2 + with: + bun-version: latest + - run: bun i + # Fails the PR if any package's type-instantiation count grows past its + # committed budget (scripts/typecheck-budget.json). Re-baseline intentional + # growth with: bun run typecheck:budget --update + - name: Typecheck budget guard + run: bun run typecheck:budget + build: runs-on: ubuntu-latest steps: diff --git a/docs/technical/typecheck-performance-investigation.md b/docs/technical/typecheck-performance-investigation.md new file mode 100644 index 000000000..8474f0382 --- /dev/null +++ b/docs/technical/typecheck-performance-investigation.md @@ -0,0 +1,146 @@ +# Typecheck performance investigation + +_Investigation of the gradual `tsc` typecheck slowdown across the monorepo, with +bisected culprit commits, root-cause analysis, and the guardrail added to catch +future regressions._ + +## TL;DR + +- Full-graph typecheck (pinned `tsc`, each commit's own deps) grew from **~11s + (mid-Feb 2026) to ~40s today (~3.6×)**. +- Two regressions were isolated by `git bisect`: + 1. **`16a94b54`** — MCP task schemas switched to spreading `…properties` + + `allOf`, exploding `packages/tasks` instantiations **289k → 6.77M (23×)**. + _Already fixed_ (tasks is back to ~207k). + 2. **`888da0e1`** ("image generation pipeline with ImageValue boundary") — a + task-runner refactor that **doubled `packages/ai` isolated check time + (0.9s → 3.4s)**. _Still in main._ +- The `888da0e1` cost is **diffuse** in the core task/`Workflow` generics, not in + any single construct. Four candidate fixes were prototyped and **measured to be + ineffective or net-negative** (see below). There is no surgical fix. +- A **CI guard** (`scripts/typecheck-budget.ts`) now gates per-package + instantiation counts so future spikes fail at PR time. + +## Method + +To compare fairly across time, the **compiler was held constant** (a pinned +standalone `tsc`) while **each commit's own dependencies** were installed, so the +only variables were source + `tsconfig`. Two metrics were used: + +- **End-to-end wall clock** of a full build — what a developer feels. +- **Isolated per-package check** — warm-build all packages to emit dependency + `.d.ts`, then `tsc -p --extendedDiagnostics` with the package's + `tsbuildinfo` removed to force a full re-check. **Instantiations** is the + headline number: it is deterministic and machine-independent, unlike wall time. + +> Caveat: these measurements use `tsc`. The repo's `build-types` runs **`tsgo`** +> per package, which is much faster and masks code regressions in day-to-day +> builds. `tsc` cost still matters for editor / `tsserver` responsiveness, and +> the relative trends hold regardless of compiler. + +## Timeline + +| Date | Full-graph wall | `packages/test` | `packages/ai` | `packages/tasks` | # packages | +| --- | --- | --- | --- | --- | --- | +| Feb 12 | ~11s | 4.4s | 0.6s | 0.9s | 11 | +| Mar 13 | ~24s | **15.0s** | 0.8s | 1.4s _(6.7M inst)_ | 12 | +| Apr 1 | ~22s | 6.3s | 0.8s | 1.5s | 10 | +| Apr 15 | ~30s | 7.9s | 0.9s | 2.5s | 10 | +| May 1 | ~27s | 8.6s | **3.4s** | 3.1s | 10 | +| May 15 | ~36s | 11.1s | 3.3s | 1.9s | 28 | +| May 28 | ~42s | 11.3s | 3.4s | 1.9s | 34 | +| Jun 7 (main) | ~40s | 12.4s | 3.4s | 2.0s | 34 | + +Notes: +- The **Mar 13 spike** in `test`/`tasks` is `16a94b54` (fixed by Apr 1). +- The **`ai` step from 0.9s → 3.4s** between Apr 15 and May 1 is `888da0e1`, and + it persists to today. +- The package-count jump (10 → 28 → 34) in mid-May is the **provider split** + (`packages/ai-provider` broken into individual `providers/*`). This is + legitimate structural growth, not a bug, but it is the largest single + contributor to the recent wall-clock increase. + +## Culprit 1 — `16a94b54` (MCP `allOf`), already fixed + +The change spread `mcpServerConfigSchema.properties` and `allOf: +mcpServerConfigSchema.allOf` into four MCP task schemas, each declared +`as const satisfies DataPortSchema`. That forced four deep instantiations of a +heavily reworked `McpAuthTypes` union, taking `packages/tasks` from ~289k to +**6.77M** instantiations. It was resolved between Mar 13 and Apr 1. + +**Lesson:** spreading large composed schemas (`allOf` / `…properties`) into +multiple `as const satisfies` sites multiplies instantiation cost. The CI guard +below would have caught this at PR time. + +## Culprit 2 — `888da0e1` (durable `ai` regression) + +`888da0e1` is a large task-runner refactor (`ITask.ts`, `Task.ts`, +`TaskRunner.ts` +169, `TaskGraphRunner.ts` +132, plus the ImageValue work). Its +edits to `ToolCallingTask.ts` itself were trivial (an import reorder and a +category-string rename), so the regression is **not** in AI task code — it is in +the **base generics every task instantiates**. + +A `--generateTrace` of `packages/ai` attributes the time to +`ToolCallingTask.ts` (~1.3–2.6s) and `registerAiTasks.ts` (~1.0s), surfacing as +`structuredTypeRelatedTo` / `getVariancesWorker` over `Workflow` and +`CreateWorkflow`. But that attribution is **misleading**: variance is +computed once and cached, so the cost lands on whichever expression touches +`Workflow` first. Aggregated by event name, `ai`'s ~3.4s is dominated by deeply +nested `structuredTypeRelatedTo` (the structural comparison of large `Workflow` +instantiations), with `getVariancesWorker` nested inside it. + +### Why it is hard to fix + +Each `Workflow.prototype. = CreateWorkflow()` assignment relates two +**distinct** `Workflow` instantiations (the inferred `I` from the task class +is a fresh type-parameter, different from the `I` written in the `declare module` +augmentation), so there is no reference-identity short-circuit and the full +structural relation runs over the heavily augmented `Workflow` interface (one +builder method per registered task — ~124 across the repo). The cost scales +super-linearly with the number of augmentations, which `888da0e1` increased by +adding `imageGenerate` / `imageEdit`. + +### Fixes prototyped and measured (all rejected) + +Measured on the real metric (dependencies pre-built, isolated `tsc -p ai`, +0 errors), baseline `ai` = 3.4s: + +| Attempt | Result | +| --- | --- | +| Replace `FromSchema<…>` input type in `ToolCallingTask` with a hand-written interface | 3.4s → 3.4s (no effect) | +| Replace `WithImageValuePorts<…>` image-task types with explicit interfaces | image/vision tasks are already ~13ms each — no meaningful effect | +| Restore the `tsBuildInfoFile` line `888da0e1` deleted from root `tsconfig.json` | 3.38s vs 3.58s — **no real effect**; it only changes `tsc -b` orchestration and actually breaks solution builds (`TS6377` buildinfo collision). The "fast" number some measurements showed was an artifact of the build bailing out and leaving deps unbuilt. | +| Declare correct variance `Workflow` (+ narrow the 5 bare-`Workflow` `return this` methods) | task-graph compiles clean and emits the annotated `.d.ts`, but `ai` is unchanged (3.4s → 3.58s). Declaring variance removes `getVariancesWorker` but the **structural relation still runs**, and variance is also computed for `TaskConfig` / `CreateWorkflow` / `DataPorts` which one annotation does not touch. | + +### Remaining options for the durable `ai` cost + +There is no surgical fix. Real remediation is either: + +- **Architectural** — rethink how `Workflow` is augmented (124 `CreateWorkflow<…>` + member instantiations in a single mega-interface) and/or how AI task input + types are derived, so the prototype assignments don't each relate a large + `Workflow`. This is a multi-day design effort and must be validated under + **both** `tsc` and `tsgo` before investing. +- **Operational** — confirm whether the user-felt slowness is `tsserver` + (still `tsc`) vs `build-types` (already `tsgo`); the latter largely sidesteps + this. The biggest absolute cost, `packages/test` (~12s), is mostly legitimate + file-count growth exercising the same machinery ×N and is not addressed by any + of the above. + +## Guardrail added: `scripts/typecheck-budget.ts` + +A CI guard that type-checks each composite package in isolation and compares its +**instantiation count** against committed per-package budgets +(`scripts/typecheck-budget.json`, default tolerance +15%, packages under 50k +instantiations are not gated). Instantiations are deterministic and +machine-independent, so the gate is stable across runners while catching exactly +the class of regression documented here. + +```sh +bun run typecheck:budget # check; exit 1 on regression +bun run typecheck:budget --update # re-baseline intentional growth +bun run typecheck:budget --json # emit raw measurements +``` + +Wired into `.github/workflows/test.yml` as the `typecheck-budget` job. Both +culprit commits above would have failed it (`tasks` +23×, `ai` ~2×). diff --git a/package.json b/package.json index 8ace17c42..876060d36 100644 --- a/package.json +++ b/package.json @@ -23,6 +23,7 @@ "build:examples": "turbo run build-example --concurrency=15", "build:js": "turbo run build-js --concurrency=15", "build:types": "turbo run build-types --concurrency=15", + "typecheck:budget": "bun scripts/typecheck-budget.ts", "clean": "rm -rf node_modules packages/*/node_modules packages/*/*tsbuildinfo packages/*/dist packages/*/src/**/*\\.d\\.ts packages/*/src/**/*\\.map integrations/*/node_modules integrations/*/dist integrations/*/src/**/*\\.d\\.ts integrations/*/src/**/*\\.map examples/*/node_modules examples/*/dist examples/*/src/**/*\\.d\\.ts examples/*/src/**/*\\.map .turbo */*/.turbo", "dev": "turbo run dev --concurrency=15", "docs": "typedoc", diff --git a/packages/ai/src/model/ModelSchema.ts b/packages/ai/src/model/ModelSchema.ts index 0fefdf49d..5b07d5964 100644 --- a/packages/ai/src/model/ModelSchema.ts +++ b/packages/ai/src/model/ModelSchema.ts @@ -4,7 +4,7 @@ * SPDX-License-Identifier: Apache-2.0 */ -import { DataPortSchemaObject, FromSchema } from "@workglow/util/worker"; +import { DataPortSchemaObject } from "@workglow/util/worker"; /** * A model configuration suitable for task/job inputs. @@ -65,6 +65,33 @@ export const ModelRecordSchema = { additionalProperties: false, } as const satisfies DataPortSchemaObject; -export type ModelConfig = FromSchema; -export type ModelRecord = FromSchema; +export type ModelConfig = { + [x: string]: unknown; + title?: string | undefined; + description?: string | undefined; + model_id?: string | undefined; + capabilities?: string[] | undefined; + metadata?: { [x: string]: unknown } | undefined; + provider: string; + provider_config: { + [x: string]: unknown; + credential_key?: string | undefined; + native_dimensions?: number | undefined; + mrl?: boolean | undefined; + }; +}; +export type ModelRecord = { + title: string; + description: string; + model_id: string; + capabilities: string[]; + provider: string; + provider_config: { + [x: string]: unknown; + credential_key?: string | undefined; + native_dimensions?: number | undefined; + mrl?: boolean | undefined; + }; + metadata: { [x: string]: unknown }; +}; export const ModelPrimaryKeyNames = ["model_id"] as const; diff --git a/packages/ai/src/task/AiChatTask.ts b/packages/ai/src/task/AiChatTask.ts index 0a9723da5..a11cd3403 100644 --- a/packages/ai/src/task/AiChatTask.ts +++ b/packages/ai/src/task/AiChatTask.ts @@ -8,7 +8,7 @@ import type { CachePolicy, IExecuteContext, StreamEvent } from "@workglow/task-g import { TaskConfigSchema } from "@workglow/task-graph"; import type { IHumanRequest } from "@workglow/util"; import { resolveHumanConnector } from "@workglow/util"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { AiEmit } from "../capability/AiEmit"; import type { Capability } from "../capability/Capabilities"; import type { AiJobInput } from "../job/AiJob"; @@ -150,11 +150,37 @@ export const AiChatOutputSchema = { // Runtime types // ======================================================================== -export type AiChatTaskInput = Omit, "messages"> & { - readonly messages?: ReadonlyArray; -}; +export type AiChatTaskInput = Omit< + { + systemPrompt?: string | undefined; + messages?: ChatMessage[] | undefined; + maxTokens?: number | undefined; + temperature?: number | undefined; + maxIterations?: number | undefined; + responseFormat?: "text" | "markdown" | undefined; + model: string | ModelConfig; + prompt: + | string + | ( + | { type: "text"; text: string } + | { type: "image"; mimeType: string; data: string } + | { type: "tool_use"; id: string; name: string; input: { [x: string]: unknown } } + | { + is_error?: boolean | undefined; + type: "tool_result"; + content: ( + | { type: "text"; text: string } + | { type: "image"; mimeType: string; data: string } + | { type: "tool_use"; id: string; name: string; input: { [x: string]: unknown } } + )[]; + tool_use_id: string; + } + )[]; + }, + "messages" +> & { readonly messages?: ReadonlyArray }; -export type AiChatTaskOutput = FromSchema; +export type AiChatTaskOutput = { text: string; messages: ChatMessage[]; iterations: number }; /** Provider-facing input: same structural type as AiChatTaskInput, named separately for intent. */ export type AiChatProviderInput = AiChatTaskInput; diff --git a/packages/ai/src/task/AiChatWithKbTask.ts b/packages/ai/src/task/AiChatWithKbTask.ts index e7b7ffb8a..d3fb7c9db 100644 --- a/packages/ai/src/task/AiChatWithKbTask.ts +++ b/packages/ai/src/task/AiChatWithKbTask.ts @@ -10,7 +10,7 @@ import type { CachePolicy, IExecuteContext, StreamEvent } from "@workglow/task-g import { TaskConfigSchema } from "@workglow/task-graph"; import type { IHumanRequest } from "@workglow/util"; import { resolveHumanConnector } from "@workglow/util"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { AiEmit } from "../capability/AiEmit"; import type { Capability } from "../capability/Capabilities"; import type { AiJobInput } from "../job/AiJob"; @@ -242,7 +242,50 @@ export const AiChatWithKbOutputSchema = { } as const satisfies DataPortSchema; export type AiChatWithKbTaskInput = Omit< - FromSchema, + { + systemPrompt?: string | undefined; + messages?: ChatMessage[] | undefined; + maxTokens?: number | undefined; + temperature?: number | undefined; + maxIterations?: number | undefined; + responseFormat?: "text" | "markdown" | undefined; + embeddingModel?: unknown; + topKPerKb?: number | undefined; + minScore?: number | undefined; + maxReferences?: number | undefined; + noMatchReply?: string | undefined; + noMatchReferences?: + | { + [x: string]: unknown; + url?: string | undefined; + title: string; + score: number; + kbId: string; + kbLabel: string; + snippet: string; + index: number; + }[] + | undefined; + model: string | ModelConfig; + prompt: + | string + | ( + | { type: "text"; text: string } + | { type: "image"; mimeType: string; data: string } + | { type: "tool_use"; id: string; name: string; input: { [x: string]: unknown } } + | { + is_error?: boolean | undefined; + type: "tool_result"; + content: ( + | { type: "text"; text: string } + | { type: "image"; mimeType: string; data: string } + | { type: "tool_use"; id: string; name: string; input: { [x: string]: unknown } } + )[]; + tool_use_id: string; + } + )[]; + knowledgeBaseIds: string[]; + }, "messages" | "noMatchReferences" > & { readonly messages?: ReadonlyArray; @@ -250,11 +293,23 @@ export type AiChatWithKbTaskInput = Omit< }; export type AiChatWithKbTaskOutput = Omit< - FromSchema, + { + text: string; + messages: ChatMessage[]; + references: { + [x: string]: unknown; + url?: string | undefined; + title: string; + score: number; + kbId: string; + kbLabel: string; + snippet: string; + index: number; + }[]; + iterations: number; + }, "references" -> & { - readonly references: readonly ChatChunkReference[]; -}; +> & { readonly references: readonly ChatChunkReference[] }; export class AiChatWithKbTask extends StreamingAiTask< AiChatWithKbTaskInput, diff --git a/packages/ai/src/task/BackgroundRemovalTask.ts b/packages/ai/src/task/BackgroundRemovalTask.ts index 4a3e023c2..98f525480 100644 --- a/packages/ai/src/task/BackgroundRemovalTask.ts +++ b/packages/ai/src/task/BackgroundRemovalTask.ts @@ -6,10 +6,11 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -37,12 +38,10 @@ export const BackgroundRemovalOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type BackgroundRemovalTaskInput = WithImageValuePorts< - FromSchema, - { - image: ImageValue; - } ->; +export type BackgroundRemovalTaskInput = Omit< + { model: string | ModelConfig; image: string | { [x: string]: unknown } }, + "image" +> & { image: ImageValue }; export type BackgroundRemovalTaskOutput = { image: ImageValue }; export type BackgroundRemovalTaskConfig = TaskConfig; diff --git a/packages/ai/src/task/ChunkRetrievalTask.ts b/packages/ai/src/task/ChunkRetrievalTask.ts index 7bdf03fef..09e872146 100644 --- a/packages/ai/src/task/ChunkRetrievalTask.ts +++ b/packages/ai/src/task/ChunkRetrievalTask.ts @@ -8,15 +8,9 @@ import type { ChunkRecord, ChunkSearchResult } from "@workglow/knowledge-base"; import { KnowledgeBase, TypeKnowledgeBase } from "@workglow/knowledge-base"; import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; -import { - DataPortSchema, - FromSchema, - isTypedArray, - TypedArray, - TypedArraySchema, - TypedArraySchemaOptions, -} from "@workglow/util/schema"; +import { DataPortSchema, isTypedArray, TypedArray, TypedArraySchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel } from "./base/AiTaskSchemas"; import { TextEmbeddingTask } from "./TextEmbeddingTask"; @@ -180,8 +174,39 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ChunkRetrievalTaskInput = FromSchema; -export type ChunkRetrievalTaskOutput = FromSchema; +export type ChunkRetrievalTaskInput = + | { + filter?: { [x: string]: unknown } | undefined; + topK?: number | undefined; + method?: "similarity" | "hybrid" | undefined; + scoreThreshold?: number | undefined; + vectorWeight?: number | undefined; + returnVectors?: boolean | undefined; + model: string | ModelConfig; + query: string; + knowledgeBase: unknown; + } + | { + model?: string | ModelConfig | undefined; + filter?: { [x: string]: unknown } | undefined; + topK?: number | undefined; + method?: "similarity" | "hybrid" | undefined; + scoreThreshold?: number | undefined; + vectorWeight?: number | undefined; + returnVectors?: boolean | undefined; + query: TypedArray; + knowledgeBase: unknown; + }; +export type ChunkRetrievalTaskOutput = { + vectors?: TypedArray[] | undefined; + metadata: { [x: string]: unknown }[]; + chunks: string[]; + count: number; + query: string | TypedArray; + scores: number[]; + chunk_ids: string[]; + scoreType: "cosine" | "bm25" | "rrf" | "rerank"; +}; export type ChunkRetrievalTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/ChunkVectorUpsertTask.ts b/packages/ai/src/task/ChunkVectorUpsertTask.ts index 0f77b29fd..95328b702 100644 --- a/packages/ai/src/task/ChunkVectorUpsertTask.ts +++ b/packages/ai/src/task/ChunkVectorUpsertTask.ts @@ -8,13 +8,7 @@ import type { ChunkRecord, KnowledgeBase } from "@workglow/knowledge-base"; import { ChunkRecordArraySchema, TypeKnowledgeBase } from "@workglow/knowledge-base"; import type { CachePolicy, IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; -import { - DataPortSchema, - FromSchema, - TypedArray, - TypedArraySchema, - TypedArraySchemaOptions, -} from "@workglow/util/schema"; +import { DataPortSchema, TypedArray, TypedArraySchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; import { TypeSingleOrArray } from "./base/AiTaskSchemas"; @@ -66,8 +60,26 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type VectorStoreUpsertTaskInput = FromSchema; -export type VectorStoreUpsertTaskOutput = FromSchema; +export type VectorStoreUpsertTaskInput = { + doc_title?: string | undefined; + chunks: { + [x: string]: unknown; + leafNodeId?: string | undefined; + summary?: string | undefined; + entities?: { type: string; text: string; score: number }[] | undefined; + parentSummaries?: string[] | undefined; + sectionTitles?: string[] | undefined; + doc_title?: string | undefined; + text: string; + doc_id: string; + chunkId: string; + nodePath: string[]; + depth: number; + }[]; + vector: TypedArray | TypedArray[]; + knowledgeBase: unknown; +}; +export type VectorStoreUpsertTaskOutput = { doc_id: string; count: number; chunk_ids: string[] }; export type ChunkVectorUpsertTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/ContextBuilderTask.ts b/packages/ai/src/task/ContextBuilderTask.ts index 852d08172..6d52c405a 100644 --- a/packages/ai/src/task/ContextBuilderTask.ts +++ b/packages/ai/src/task/ContextBuilderTask.ts @@ -13,8 +13,9 @@ import { Task, Workflow, } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { CountTokensTask } from "./CountTokensTask"; import { TypeModel } from "./base/AiTaskSchemas"; @@ -126,8 +127,23 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ContextBuilderTaskInput = FromSchema; -export type ContextBuilderTaskOutput = FromSchema; +export type ContextBuilderTaskInput = { + maxLength?: number | undefined; + format?: "simple" | "markdown" | "numbered" | "xml" | "json" | undefined; + model?: string | ModelConfig | undefined; + metadata?: { [x: string]: unknown }[] | undefined; + maxTokens?: number | undefined; + scores?: number[] | undefined; + includeMetadata?: boolean | undefined; + separator?: string | undefined; + chunks: string[]; +}; +export type ContextBuilderTaskOutput = { + context: string; + chunksUsed: number; + totalLength: number; + totalTokens: number; +}; export type ContextBuilderTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/CountTokensTask.ts b/packages/ai/src/task/CountTokensTask.ts index 5467a344d..9de5ece5e 100644 --- a/packages/ai/src/task/CountTokensTask.ts +++ b/packages/ai/src/task/CountTokensTask.ts @@ -6,8 +6,9 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { AiTask } from "./base/AiTask"; import { TypeModel } from "./base/AiTaskSchemas"; @@ -40,8 +41,8 @@ export const CountTokensOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type CountTokensTaskInput = FromSchema; -export type CountTokensTaskOutput = FromSchema; +export type CountTokensTaskInput = { model: string | ModelConfig; text: string }; +export type CountTokensTaskOutput = { count: number }; export type CountTokensTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/DocumentEnricherTask.ts b/packages/ai/src/task/DocumentEnricherTask.ts index c38a742b7..101e5c62f 100644 --- a/packages/ai/src/task/DocumentEnricherTask.ts +++ b/packages/ai/src/task/DocumentEnricherTask.ts @@ -9,7 +9,7 @@ import { DocumentRootNode, getChildren, hasChildren } from "@workglow/knowledge- import type { DocumentNode, Entity, NodeEnrichment } from "@workglow/knowledge-base"; import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; import { ModelConfig } from "../model/ModelSchema"; import { TextNamedEntityRecognitionTask } from "./TextNamedEntityRecognitionTask"; @@ -88,10 +88,24 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type DocumentEnricherTaskInput = Omit, "documentTree"> & { - documentTree: DocumentRootNode; +export type DocumentEnricherTaskInput = Omit< + { + doc_id?: string | undefined; + documentTree?: { [x: string]: unknown } | undefined; + generateSummaries?: boolean | undefined; + extractEntities?: boolean | undefined; + summaryModel?: string | ModelConfig | undefined; + summaryThreshold?: number | undefined; + nerModel?: string | ModelConfig | undefined; + }, + "documentTree" +> & { documentTree: DocumentRootNode }; +export type DocumentEnricherTaskOutput = { + doc_id: string; + documentTree: unknown; + summaryCount: number; + entityCount: number; }; -export type DocumentEnricherTaskOutput = FromSchema; export type DocumentEnricherTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/DocumentUpsertTask.ts b/packages/ai/src/task/DocumentUpsertTask.ts index 76b9679f8..348d2a35d 100644 --- a/packages/ai/src/task/DocumentUpsertTask.ts +++ b/packages/ai/src/task/DocumentUpsertTask.ts @@ -13,7 +13,7 @@ import { } from "@workglow/knowledge-base"; import type { CachePolicy, IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; const inputSchema = { @@ -71,8 +71,21 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type DocumentUpsertTaskInput = FromSchema; -export type DocumentUpsertTaskOutput = FromSchema; +export type DocumentUpsertTaskInput = { + title?: string | undefined; + metadata?: + | { + [x: string]: unknown; + title?: string | undefined; + sourceUri?: string | undefined; + createdAt?: string | undefined; + } + | undefined; + doc_id: string; + documentTree: unknown; + knowledgeBase: unknown; +}; +export type DocumentUpsertTaskOutput = { doc_id: string }; export type DocumentUpsertTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/FaceDetectorTask.ts b/packages/ai/src/task/FaceDetectorTask.ts index 793fbf9d1..65f12e739 100644 --- a/packages/ai/src/task/FaceDetectorTask.ts +++ b/packages/ai/src/task/FaceDetectorTask.ts @@ -6,10 +6,11 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -139,13 +140,28 @@ export const FaceDetectorOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type FaceDetectorTaskInput = WithImageValuePorts< - FromSchema, +export type FaceDetectorTaskInput = Omit< { - image: ImageValue; - } ->; -export type FaceDetectorTaskOutput = FromSchema; + minDetectionConfidence?: number | undefined; + minSuppressionThreshold?: number | undefined; + model: string | ModelConfig; + image: string | { [x: string]: unknown }; + }, + "image" +> & { image: ImageValue }; +export type FaceDetectorTaskOutput = { + faces: + | { + score: number; + box: { x: number; y: number; width: number; height: number }; + keypoints: { label?: string | undefined; x: number; y: number }[]; + }[] + | { + score: number; + box: { x: number; y: number; width: number; height: number }; + keypoints: { label?: string | undefined; x: number; y: number }[]; + }[][]; +}; export type FaceDetectorTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/FaceLandmarkerTask.ts b/packages/ai/src/task/FaceLandmarkerTask.ts index ed01c3b15..3a8ae55b1 100644 --- a/packages/ai/src/task/FaceLandmarkerTask.ts +++ b/packages/ai/src/task/FaceLandmarkerTask.ts @@ -6,10 +6,11 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeLandmark, TypeModel } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -149,13 +150,32 @@ export const FaceLandmarkerOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type FaceLandmarkerTaskInput = WithImageValuePorts< - FromSchema, +export type FaceLandmarkerTaskInput = Omit< { - image: ImageValue; - } ->; -export type FaceLandmarkerTaskOutput = FromSchema; + numFaces?: number | undefined; + minFaceDetectionConfidence?: number | undefined; + minFacePresenceConfidence?: number | undefined; + minTrackingConfidence?: number | undefined; + outputFaceBlendshapes?: boolean | undefined; + outputFacialTransformationMatrixes?: boolean | undefined; + model: string | ModelConfig; + image: string | { [x: string]: unknown }; + }, + "image" +> & { image: ImageValue }; +export type FaceLandmarkerTaskOutput = { + faces: + | { + blendshapes?: { score: number; label: string }[] | undefined; + transformationMatrix?: number[] | undefined; + landmarks: { x: number; y: number; z: number }[]; + }[] + | { + blendshapes?: { score: number; label: string }[] | undefined; + transformationMatrix?: number[] | undefined; + landmarks: { x: number; y: number; z: number }[]; + }[][]; +}; export type FaceLandmarkerTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/GestureRecognizerTask.ts b/packages/ai/src/task/GestureRecognizerTask.ts index fdd578924..95909fb27 100644 --- a/packages/ai/src/task/GestureRecognizerTask.ts +++ b/packages/ai/src/task/GestureRecognizerTask.ts @@ -6,10 +6,11 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeLandmark, TypeModel } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -156,13 +157,32 @@ export const GestureRecognizerOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type GestureRecognizerTaskInput = WithImageValuePorts< - FromSchema, +export type GestureRecognizerTaskInput = Omit< { - image: ImageValue; - } ->; -export type GestureRecognizerTaskOutput = FromSchema; + minTrackingConfidence?: number | undefined; + numHands?: number | undefined; + minHandDetectionConfidence?: number | undefined; + minHandPresenceConfidence?: number | undefined; + model: string | ModelConfig; + image: string | { [x: string]: unknown }; + }, + "image" +> & { image: ImageValue }; +export type GestureRecognizerTaskOutput = { + hands: + | { + landmarks: { x: number; y: number; z: number }[]; + gestures: { score: number; label: string }[]; + handedness: { score: number; label: string }[]; + worldLandmarks: { x: number; y: number; z: number }[]; + }[] + | { + landmarks: { x: number; y: number; z: number }[]; + gestures: { score: number; label: string }[]; + handedness: { score: number; label: string }[]; + worldLandmarks: { x: number; y: number; z: number }[]; + }[][]; +}; export type GestureRecognizerTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/HandLandmarkerTask.ts b/packages/ai/src/task/HandLandmarkerTask.ts index 63b2d284d..3207257a8 100644 --- a/packages/ai/src/task/HandLandmarkerTask.ts +++ b/packages/ai/src/task/HandLandmarkerTask.ts @@ -6,10 +6,11 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeLandmark, TypeModel } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -129,13 +130,30 @@ export const HandLandmarkerOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type HandLandmarkerTaskInput = WithImageValuePorts< - FromSchema, +export type HandLandmarkerTaskInput = Omit< { - image: ImageValue; - } ->; -export type HandLandmarkerTaskOutput = FromSchema; + minTrackingConfidence?: number | undefined; + numHands?: number | undefined; + minHandDetectionConfidence?: number | undefined; + minHandPresenceConfidence?: number | undefined; + model: string | ModelConfig; + image: string | { [x: string]: unknown }; + }, + "image" +> & { image: ImageValue }; +export type HandLandmarkerTaskOutput = { + hands: + | { + landmarks: { x: number; y: number; z: number }[]; + handedness: { score: number; label: string }[]; + worldLandmarks: { x: number; y: number; z: number }[]; + }[] + | { + landmarks: { x: number; y: number; z: number }[]; + handedness: { score: number; label: string }[]; + worldLandmarks: { x: number; y: number; z: number }[]; + }[][]; +}; export type HandLandmarkerTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/HierarchicalChunkerTask.ts b/packages/ai/src/task/HierarchicalChunkerTask.ts index f00696bf9..76395d739 100644 --- a/packages/ai/src/task/HierarchicalChunkerTask.ts +++ b/packages/ai/src/task/HierarchicalChunkerTask.ts @@ -17,8 +17,9 @@ import type { ChunkRecord, DocumentNode, SectionNode, TokenBudget } from "@workg import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; import { uuid4 } from "@workglow/util"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { CountTokensTask } from "./CountTokensTask"; import { TypeModel } from "./base/AiTaskSchemas"; @@ -105,10 +106,37 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type HierarchicalChunkerTaskInput = Omit, "documentTree"> & { - documentTree: DocumentRootNode; +export type HierarchicalChunkerTaskInput = Omit< + { + model?: string | ModelConfig | undefined; + maxTokens?: number | undefined; + overlap?: number | undefined; + reservedTokens?: number | undefined; + strategy?: "flat" | "hierarchical" | "sentence" | undefined; + doc_id: string; + documentTree: { [x: string]: unknown }; + }, + "documentTree" +> & { documentTree: DocumentRootNode }; +export type HierarchicalChunkerTaskOutput = { + text: string[]; + doc_id: string; + chunks: { + [x: string]: unknown; + leafNodeId?: string | undefined; + summary?: string | undefined; + entities?: { type: string; text: string; score: number }[] | undefined; + parentSummaries?: string[] | undefined; + sectionTitles?: string[] | undefined; + doc_title?: string | undefined; + text: string; + doc_id: string; + chunkId: string; + nodePath: string[]; + depth: number; + }[]; + count: number; }; -export type HierarchicalChunkerTaskOutput = FromSchema; export type HierarchicalChunkerTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/HierarchyJoinTask.ts b/packages/ai/src/task/HierarchyJoinTask.ts index 02b299029..2be058a80 100644 --- a/packages/ai/src/task/HierarchyJoinTask.ts +++ b/packages/ai/src/task/HierarchyJoinTask.ts @@ -9,7 +9,7 @@ import { ChunkRecordArraySchema, TypeKnowledgeBase } from "@workglow/knowledge-b import type { ChunkRecord, KnowledgeBase } from "@workglow/knowledge-base"; import type { CachePolicy, IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; const inputSchema = { @@ -89,8 +89,48 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type HierarchyJoinTaskInput = FromSchema; -export type HierarchyJoinTaskOutput = FromSchema; +export type HierarchyJoinTaskInput = { + chunks?: string[] | undefined; + scores?: number[] | undefined; + chunk_ids?: string[] | undefined; + includeParentSummaries?: boolean | undefined; + includeEntities?: boolean | undefined; + metadata: { + [x: string]: unknown; + leafNodeId?: string | undefined; + summary?: string | undefined; + entities?: { type: string; text: string; score: number }[] | undefined; + parentSummaries?: string[] | undefined; + sectionTitles?: string[] | undefined; + doc_title?: string | undefined; + text: string; + doc_id: string; + chunkId: string; + nodePath: string[]; + depth: number; + }[]; + knowledgeBase: unknown; +}; +export type HierarchyJoinTaskOutput = { + chunks?: string[] | undefined; + scores?: number[] | undefined; + chunk_ids?: string[] | undefined; + metadata: { + [x: string]: unknown; + leafNodeId?: string | undefined; + summary?: string | undefined; + entities?: { type: string; text: string; score: number }[] | undefined; + parentSummaries?: string[] | undefined; + sectionTitles?: string[] | undefined; + doc_title?: string | undefined; + text: string; + doc_id: string; + chunkId: string; + nodePath: string[]; + depth: number; + }[]; + count: number; +}; export type HierarchyJoinTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/ImageClassificationTask.ts b/packages/ai/src/task/ImageClassificationTask.ts index 869945283..1bad831e3 100644 --- a/packages/ai/src/task/ImageClassificationTask.ts +++ b/packages/ai/src/task/ImageClassificationTask.ts @@ -6,10 +6,11 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeCategory, TypeModel } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -60,13 +61,18 @@ export const ImageClassificationOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ImageClassificationTaskInput = WithImageValuePorts< - FromSchema, +export type ImageClassificationTaskInput = Omit< { - image: ImageValue; - } ->; -export type ImageClassificationTaskOutput = FromSchema; + categories?: string[] | undefined; + maxCategories?: number | undefined; + model: string | ModelConfig; + image: string | { [x: string]: unknown }; + }, + "image" +> & { image: ImageValue }; +export type ImageClassificationTaskOutput = { + categories: { score: number; label: string }[] | { score: number; label: string }[][]; +}; export type ImageClassificationTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/ImageEmbeddingTask.ts b/packages/ai/src/task/ImageEmbeddingTask.ts index fef42e642..dcfcec655 100644 --- a/packages/ai/src/task/ImageEmbeddingTask.ts +++ b/packages/ai/src/task/ImageEmbeddingTask.ts @@ -6,7 +6,7 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; import { DataPortSchema, @@ -15,6 +15,7 @@ import { TypedArraySchemaOptions, } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel, TypeSingleOrArray } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -46,12 +47,13 @@ export const ImageEmbeddingOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ImageEmbeddingTaskInput = WithImageValuePorts< - FromSchema, +export type ImageEmbeddingTaskInput = Omit< { - readonly image: ImageValue | readonly ImageValue[]; - } ->; + model: string | ModelConfig; + image: string | { [x: string]: unknown } | (string | { [x: string]: unknown })[]; + }, + "image" +> & { readonly image: ImageValue | readonly ImageValue[] }; export type ImageEmbeddingTaskOutput = FromSchema< typeof ImageEmbeddingOutputSchema, TypedArraySchemaOptions diff --git a/packages/ai/src/task/ImageSegmentationTask.ts b/packages/ai/src/task/ImageSegmentationTask.ts index d98c5c190..c50a836e3 100644 --- a/packages/ai/src/task/ImageSegmentationTask.ts +++ b/packages/ai/src/task/ImageSegmentationTask.ts @@ -6,10 +6,11 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -83,12 +84,15 @@ export const ImageSegmentationOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ImageSegmentationTaskInput = WithImageValuePorts< - FromSchema, +export type ImageSegmentationTaskInput = Omit< { - image: ImageValue; - } ->; + threshold?: number | undefined; + maskThreshold?: number | undefined; + model: string | ModelConfig; + image: string | { [x: string]: unknown }; + }, + "image" +> & { image: ImageValue }; export type ImageSegmentationTaskOutput = { masks: { label: string; score: number; mask: ImageValue }[]; }; diff --git a/packages/ai/src/task/ImageToTextTask.ts b/packages/ai/src/task/ImageToTextTask.ts index 3064b9f10..024a6a8e7 100644 --- a/packages/ai/src/task/ImageToTextTask.ts +++ b/packages/ai/src/task/ImageToTextTask.ts @@ -6,10 +6,11 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -52,13 +53,15 @@ export const ImageToTextOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ImageToTextTaskInput = WithImageValuePorts< - FromSchema, +export type ImageToTextTaskInput = Omit< { - image: ImageValue; - } ->; -export type ImageToTextTaskOutput = FromSchema; + maxTokens?: number | undefined; + model: string | ModelConfig; + image: string | { [x: string]: unknown }; + }, + "image" +> & { image: ImageValue }; +export type ImageToTextTaskOutput = { text: string | string[] }; export type ImageToTextTaskConfig = TaskConfig; export class ImageToTextTask extends AiVisionTask< diff --git a/packages/ai/src/task/KbAddDocumentTask.ts b/packages/ai/src/task/KbAddDocumentTask.ts index 962fc2d8c..032b4c9d4 100644 --- a/packages/ai/src/task/KbAddDocumentTask.ts +++ b/packages/ai/src/task/KbAddDocumentTask.ts @@ -8,7 +8,7 @@ import type { KnowledgeBase } from "@workglow/knowledge-base"; import { Document, TypeKnowledgeBase } from "@workglow/knowledge-base"; import type { CachePolicy, IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; const inputSchema = { @@ -41,10 +41,11 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type KbAddDocumentTaskInput = Omit, "document"> & { - readonly document: Document; -}; -export type KbAddDocumentTaskOutput = FromSchema; +export type KbAddDocumentTaskInput = Omit< + { knowledgeBase: unknown; document: unknown }, + "document" +> & { readonly document: Document }; +export type KbAddDocumentTaskOutput = { doc_id: string }; export type KbAddDocumentTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/KbDeleteTask.ts b/packages/ai/src/task/KbDeleteTask.ts index a8aca050c..f2b8bdab2 100644 --- a/packages/ai/src/task/KbDeleteTask.ts +++ b/packages/ai/src/task/KbDeleteTask.ts @@ -8,7 +8,7 @@ import type { KnowledgeBase } from "@workglow/knowledge-base"; import { TypeKnowledgeBase } from "@workglow/knowledge-base"; import type { CachePolicy, IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; const inputSchema = { @@ -41,8 +41,8 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type KbDeleteTaskInput = FromSchema; -export type KbDeleteTaskOutput = FromSchema; +export type KbDeleteTaskInput = { doc_id: string; knowledgeBase: unknown }; +export type KbDeleteTaskOutput = { doc_id: string }; export type KbDeleteTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/KbReindexTask.ts b/packages/ai/src/task/KbReindexTask.ts index b1eda7608..087c4fdeb 100644 --- a/packages/ai/src/task/KbReindexTask.ts +++ b/packages/ai/src/task/KbReindexTask.ts @@ -8,7 +8,7 @@ import type { KnowledgeBase } from "@workglow/knowledge-base"; import { TypeKnowledgeBase } from "@workglow/knowledge-base"; import type { CachePolicy, IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; const inputSchema = { @@ -36,8 +36,8 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type KbReindexTaskInput = FromSchema; -export type KbReindexTaskOutput = FromSchema; +export type KbReindexTaskInput = { knowledgeBase: unknown }; +export type KbReindexTaskOutput = { count: number }; export type KbReindexTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/KbSearchTask.ts b/packages/ai/src/task/KbSearchTask.ts index 571351ff1..feb4d8f91 100644 --- a/packages/ai/src/task/KbSearchTask.ts +++ b/packages/ai/src/task/KbSearchTask.ts @@ -8,7 +8,7 @@ import type { ChunkSearchResult, KnowledgeBase } from "@workglow/knowledge-base" import { chunkText, TypeKnowledgeBase } from "@workglow/knowledge-base"; import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; const inputSchema = { @@ -87,7 +87,13 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type KbSearchTaskInput = FromSchema; +export type KbSearchTaskInput = { + filter?: { [x: string]: unknown } | undefined; + topK?: number | undefined; + scoreThreshold?: number | undefined; + query: string; + knowledgeBase: unknown; +}; export type KbSearchTaskOutput = { readonly results: ChunkSearchResult[]; readonly chunks: string[]; diff --git a/packages/ai/src/task/KbToDocumentsTask.ts b/packages/ai/src/task/KbToDocumentsTask.ts index 093a26f81..92c065535 100644 --- a/packages/ai/src/task/KbToDocumentsTask.ts +++ b/packages/ai/src/task/KbToDocumentsTask.ts @@ -7,7 +7,7 @@ import { DocumentNode, KnowledgeBase, TypeKnowledgeBase } from "@workglow/knowledge-base"; import type { CachePolicy, IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; const inputSchema = { @@ -54,10 +54,11 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type KbToDocumentsTaskInput = FromSchema; -export type KbToDocumentsTaskOutput = Omit, "documentTree"> & { - documentTree: DocumentNode[]; -}; +export type KbToDocumentsTaskInput = { onlyStale?: boolean | undefined; knowledgeBase: unknown }; +export type KbToDocumentsTaskOutput = Omit< + { title: string[]; doc_id: string[]; documentTree: { [x: string]: unknown }[] }, + "documentTree" +> & { documentTree: DocumentNode[] }; export type KbToDocumentsTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/ModelDownloadRemoveTask.ts b/packages/ai/src/task/ModelDownloadRemoveTask.ts index 7360eb6b6..ac4658443 100644 --- a/packages/ai/src/task/ModelDownloadRemoveTask.ts +++ b/packages/ai/src/task/ModelDownloadRemoveTask.ts @@ -6,7 +6,7 @@ import type { CachePolicy, IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; import type { ModelConfig } from "../model/ModelSchema"; import { AiTask } from "./base/AiTask"; @@ -32,8 +32,8 @@ const ModelDownloadRemoveOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ModelDownloadRemoveTaskRunInput = FromSchema; -export type ModelDownloadRemoveTaskRunOutput = FromSchema; +export type ModelDownloadRemoveTaskRunInput = { model: string | ModelConfig }; +export type ModelDownloadRemoveTaskRunOutput = { model: string | ModelConfig }; export type ModelDownloadRemoveTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/ModelDownloadTask.ts b/packages/ai/src/task/ModelDownloadTask.ts index cebc3ed35..fc230c4c5 100644 --- a/packages/ai/src/task/ModelDownloadTask.ts +++ b/packages/ai/src/task/ModelDownloadTask.ts @@ -6,7 +6,7 @@ import type { CachePolicy, IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; import type { ModelConfig } from "../model/ModelSchema"; import { AiTask } from "./base/AiTask"; @@ -32,8 +32,8 @@ const ModelDownloadOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ModelDownloadTaskRunInput = FromSchema; -export type ModelDownloadTaskRunOutput = FromSchema; +export type ModelDownloadTaskRunInput = { model: string | ModelConfig }; +export type ModelDownloadTaskRunOutput = { model: string | ModelConfig }; export type ModelDownloadTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/ModelInfoTask.ts b/packages/ai/src/task/ModelInfoTask.ts index 03b2bde40..212cb8cc7 100644 --- a/packages/ai/src/task/ModelInfoTask.ts +++ b/packages/ai/src/task/ModelInfoTask.ts @@ -7,7 +7,7 @@ import { CreateWorkflow, Workflow } from "@workglow/task-graph"; import type { CachePolicy, IExecuteContext, IRunConfig, TaskConfig } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import { accumulatingEmit } from "../capability/accumulatingEmit"; import type { Capability } from "../capability/Capabilities"; import { ModelConfig } from "../model/ModelSchema"; @@ -77,8 +77,24 @@ const ModelInfoOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ModelInfoTaskInput = FromSchema; -export type ModelInfoTaskOutput = FromSchema; +export type ModelInfoTaskInput = { + detail?: "cached_status" | "files" | "files_with_metadata" | "dimensions" | undefined; + model: string | ModelConfig; +}; +export type ModelInfoTaskOutput = { + native_dimensions?: number | undefined; + mrl?: boolean | undefined; + is_downloading?: boolean | undefined; + quantizations?: string[] | undefined; + model: string | ModelConfig; + is_local: boolean; + is_remote: boolean; + supports_browser: boolean; + supports_node: boolean; + is_cached: boolean; + is_loaded: boolean; + file_sizes: unknown; +}; export type ModelInfoTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/ModelSearchTask.ts b/packages/ai/src/task/ModelSearchTask.ts index 17e738e03..7dc3f259d 100644 --- a/packages/ai/src/task/ModelSearchTask.ts +++ b/packages/ai/src/task/ModelSearchTask.ts @@ -7,7 +7,7 @@ import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; import type { CachePolicy, IRunConfig, TaskConfig } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import { accumulatingEmit } from "../capability/accumulatingEmit"; import type { Capability } from "../capability/Capabilities"; import type { ModelRecord } from "../model/ModelSchema"; @@ -103,7 +103,11 @@ const ModelSearchOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ModelSearchTaskInput = FromSchema; +export type ModelSearchTaskInput = { + credential_key?: string | undefined; + query?: string | undefined; + provider: string; +}; export type ModelSearchTaskOutput = { results: ModelSearchResultItem[] }; export type ModelSearchTaskConfig = TaskConfig; diff --git a/packages/ai/src/task/ObjectDetectionTask.ts b/packages/ai/src/task/ObjectDetectionTask.ts index 09e1b50a8..1843c54f5 100644 --- a/packages/ai/src/task/ObjectDetectionTask.ts +++ b/packages/ai/src/task/ObjectDetectionTask.ts @@ -6,10 +6,11 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeBoundingBox, TypeModel } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -81,13 +82,28 @@ export const ObjectDetectionOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type ObjectDetectionTaskInput = WithImageValuePorts< - FromSchema, +export type ObjectDetectionTaskInput = Omit< { - image: ImageValue; - } ->; -export type ObjectDetectionTaskOutput = FromSchema; + threshold?: number | undefined; + labels?: string[] | undefined; + model: string | ModelConfig; + image: string | { [x: string]: unknown }; + }, + "image" +> & { image: ImageValue }; +export type ObjectDetectionTaskOutput = { + detections: + | { + score: number; + box: { x: number; y: number; width: number; height: number }; + label: string; + }[] + | { + score: number; + box: { x: number; y: number; width: number; height: number }; + label: string; + }[][]; +}; export type ObjectDetectionTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/PoseLandmarkerTask.ts b/packages/ai/src/task/PoseLandmarkerTask.ts index 28b77d762..ec6740c18 100644 --- a/packages/ai/src/task/PoseLandmarkerTask.ts +++ b/packages/ai/src/task/PoseLandmarkerTask.ts @@ -6,10 +6,11 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; +import type { ImageValue } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel, TypePoseLandmark } from "./base/AiTaskSchemas"; import { AiVisionTask } from "./base/AiVisionTask"; @@ -134,13 +135,59 @@ export const PoseLandmarkerOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type PoseLandmarkerTaskInput = WithImageValuePorts< - FromSchema, +export type PoseLandmarkerTaskInput = Omit< { - image: ImageValue; - } ->; -export type PoseLandmarkerTaskOutput = FromSchema; + minTrackingConfidence?: number | undefined; + numPoses?: number | undefined; + minPoseDetectionConfidence?: number | undefined; + minPosePresenceConfidence?: number | undefined; + outputSegmentationMasks?: boolean | undefined; + model: string | ModelConfig; + image: string | { [x: string]: unknown }; + }, + "image" +> & { image: ImageValue }; +export type PoseLandmarkerTaskOutput = { + poses: + | { + segmentationMask?: + | { data: { [x: string]: unknown }; width: number; height: number } + | undefined; + landmarks: { + visibility?: number | undefined; + presence?: number | undefined; + x: number; + y: number; + z: number; + }[]; + worldLandmarks: { + visibility?: number | undefined; + presence?: number | undefined; + x: number; + y: number; + z: number; + }[]; + }[] + | { + segmentationMask?: + | { data: { [x: string]: unknown }; width: number; height: number } + | undefined; + landmarks: { + visibility?: number | undefined; + presence?: number | undefined; + x: number; + y: number; + z: number; + }[]; + worldLandmarks: { + visibility?: number | undefined; + presence?: number | undefined; + x: number; + y: number; + z: number; + }[]; + }[][]; +}; export type PoseLandmarkerTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/QueryExpanderTask.ts b/packages/ai/src/task/QueryExpanderTask.ts index 043deecc2..e00f3233d 100644 --- a/packages/ai/src/task/QueryExpanderTask.ts +++ b/packages/ai/src/task/QueryExpanderTask.ts @@ -7,7 +7,7 @@ import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; export const QueryExpansionMethod = { @@ -74,8 +74,17 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type QueryExpanderTaskInput = FromSchema; -export type QueryExpanderTaskOutput = FromSchema; +export type QueryExpanderTaskInput = { + method?: "multi-query" | "synonyms" | undefined; + numVariations?: number | undefined; + query: string; +}; +export type QueryExpanderTaskOutput = { + count: number; + query: string[]; + method: string; + originalQuery: string; +}; export type QueryExpanderTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/RerankerTask.ts b/packages/ai/src/task/RerankerTask.ts index c2a3f5708..edda5a6a2 100644 --- a/packages/ai/src/task/RerankerTask.ts +++ b/packages/ai/src/task/RerankerTask.ts @@ -7,7 +7,7 @@ import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; const inputSchema = { @@ -99,8 +99,21 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type RerankerTaskInput = FromSchema; -export type RerankerTaskOutput = FromSchema; +export type RerankerTaskInput = { + metadata?: { [x: string]: unknown }[] | undefined; + scores?: number[] | undefined; + topK?: number | undefined; + method?: "reciprocal-rank-fusion" | "simple" | undefined; + chunks: string[]; + query: string; +}; +export type RerankerTaskOutput = { + metadata?: { [x: string]: unknown }[] | undefined; + chunks: string[]; + count: number; + scores: number[]; + originalIndices: number[]; +}; export type RerankerTaskConfig = TaskConfig; interface RankedItem { diff --git a/packages/ai/src/task/StructuralParserTask.ts b/packages/ai/src/task/StructuralParserTask.ts index cb6012d4f..aedfab6a4 100644 --- a/packages/ai/src/task/StructuralParserTask.ts +++ b/packages/ai/src/task/StructuralParserTask.ts @@ -8,7 +8,7 @@ import { DocumentRootNode, StructuralParser } from "@workglow/knowledge-base"; import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; import { uuid4 } from "@workglow/util"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; const inputSchema = { @@ -70,10 +70,17 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type StructuralParserTaskInput = FromSchema; -export type StructuralParserTaskOutput = Omit, "documentTree"> & { - documentTree: DocumentRootNode; +export type StructuralParserTaskInput = { + format?: "text" | "markdown" | "auto" | undefined; + doc_id?: string | undefined; + sourceUri?: string | undefined; + title: string; + text: string; }; +export type StructuralParserTaskOutput = Omit< + { doc_id: string; documentTree: { [x: string]: unknown }; nodeCount: number }, + "documentTree" +> & { documentTree: DocumentRootNode }; export type StructuralParserTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/StructuredGenerationTask.ts b/packages/ai/src/task/StructuredGenerationTask.ts index e27aa8f74..76390aca7 100644 --- a/packages/ai/src/task/StructuredGenerationTask.ts +++ b/packages/ai/src/task/StructuredGenerationTask.ts @@ -6,9 +6,10 @@ import type { IExecuteContext, IRunConfig, StreamEvent, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, TaskConfigurationError, TaskError, Workflow } from "@workglow/task-graph"; -import type { DataPortSchema, FromSchema, SchemaNode } from "@workglow/util/schema"; +import type { DataPortSchema, SchemaNode } from "@workglow/util/schema"; import { compileSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel } from "./base/AiTaskSchemas"; import { StreamingAiTask } from "./base/StreamingAiTask"; @@ -76,8 +77,15 @@ export const StructuredGenerationOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type StructuredGenerationTaskInput = FromSchema; -export type StructuredGenerationTaskOutput = FromSchema; +export type StructuredGenerationTaskInput = { + maxTokens?: number | undefined; + temperature?: number | undefined; + maxRetries?: number | undefined; + model: string | ModelConfig; + prompt: string; + outputSchema: { [x: string]: unknown }; +}; +export type StructuredGenerationTaskOutput = { object: { [x: string]: unknown } }; export type StructuredGenerationTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/TextChunkerTask.ts b/packages/ai/src/task/TextChunkerTask.ts index 62444371f..4a93d1c02 100644 --- a/packages/ai/src/task/TextChunkerTask.ts +++ b/packages/ai/src/task/TextChunkerTask.ts @@ -9,7 +9,7 @@ import { ChunkRecordArraySchema } from "@workglow/knowledge-base"; import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; export const ChunkingStrategy = { @@ -86,8 +86,32 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextChunkerTaskInput = FromSchema; -export type TextChunkerTaskOutput = FromSchema; +export type TextChunkerTaskInput = { + doc_id?: string | undefined; + strategy?: "fixed" | "sentence" | "paragraph" | "semantic" | undefined; + chunkSize?: number | undefined; + chunkOverlap?: number | undefined; + text: string; +}; +export type TextChunkerTaskOutput = { + doc_id?: string | undefined; + text: string[]; + chunks: { + [x: string]: unknown; + leafNodeId?: string | undefined; + summary?: string | undefined; + entities?: { type: string; text: string; score: number }[] | undefined; + parentSummaries?: string[] | undefined; + sectionTitles?: string[] | undefined; + doc_title?: string | undefined; + text: string; + doc_id: string; + chunkId: string; + nodePath: string[]; + depth: number; + }[]; + count: number; +}; export type TextChunkerTaskConfig = TaskConfig; interface RawChunk { diff --git a/packages/ai/src/task/TextClassificationTask.ts b/packages/ai/src/task/TextClassificationTask.ts index 26843e4b3..73d833b90 100644 --- a/packages/ai/src/task/TextClassificationTask.ts +++ b/packages/ai/src/task/TextClassificationTask.ts @@ -6,8 +6,9 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { AiTask } from "./base/AiTask"; import { TypeModel } from "./base/AiTaskSchemas"; @@ -75,8 +76,13 @@ export const TextClassificationOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextClassificationTaskInput = FromSchema; -export type TextClassificationTaskOutput = FromSchema; +export type TextClassificationTaskInput = { + maxCategories?: number | undefined; + candidateLabels?: string[] | undefined; + model: string | ModelConfig; + text: string; +}; +export type TextClassificationTaskOutput = { categories: { score: number; label: string }[] }; export type TextClassificationTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/TextEmbeddingTask.ts b/packages/ai/src/task/TextEmbeddingTask.ts index 0571deca7..b7b438620 100644 --- a/packages/ai/src/task/TextEmbeddingTask.ts +++ b/packages/ai/src/task/TextEmbeddingTask.ts @@ -13,6 +13,7 @@ import { TypedArraySchemaOptions, } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { AiTask } from "./base/AiTask"; import { TypeModel, TypeSingleOrArray } from "./base/AiTaskSchemas"; @@ -46,10 +47,7 @@ export const TextEmbeddingOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextEmbeddingTaskInput = FromSchema< - typeof TextEmbeddingInputSchema, - TypedArraySchemaOptions ->; +export type TextEmbeddingTaskInput = { model: string | ModelConfig; text: string | string[] }; export type TextEmbeddingTaskOutput = FromSchema< typeof TextEmbeddingOutputSchema, TypedArraySchemaOptions diff --git a/packages/ai/src/task/TextFillMaskTask.ts b/packages/ai/src/task/TextFillMaskTask.ts index fe535824a..7d753473a 100644 --- a/packages/ai/src/task/TextFillMaskTask.ts +++ b/packages/ai/src/task/TextFillMaskTask.ts @@ -6,8 +6,9 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { AiTask } from "./base/AiTask"; import { TypeModel } from "./base/AiTaskSchemas"; @@ -62,8 +63,10 @@ export const TextFillMaskOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextFillMaskTaskInput = FromSchema; -export type TextFillMaskTaskOutput = FromSchema; +export type TextFillMaskTaskInput = { model: string | ModelConfig; text: string }; +export type TextFillMaskTaskOutput = { + predictions: { score: number; entity: string; sequence: string }[]; +}; export type TextFillMaskTaskConfig = TaskConfig; export class TextFillMaskTask extends AiTask< diff --git a/packages/ai/src/task/TextGenerationTask.ts b/packages/ai/src/task/TextGenerationTask.ts index ebb043eeb..d0a6486ee 100644 --- a/packages/ai/src/task/TextGenerationTask.ts +++ b/packages/ai/src/task/TextGenerationTask.ts @@ -6,8 +6,9 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel } from "./base/AiTaskSchemas"; import { StreamingAiTask } from "./base/StreamingAiTask"; @@ -83,8 +84,16 @@ export const TextGenerationOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextGenerationTaskInput = FromSchema; -export type TextGenerationTaskOutput = FromSchema; +export type TextGenerationTaskInput = { + maxTokens?: number | undefined; + temperature?: number | undefined; + topP?: number | undefined; + frequencyPenalty?: number | undefined; + presencePenalty?: number | undefined; + model: string | ModelConfig; + prompt: string; +}; +export type TextGenerationTaskOutput = { text: string }; export type TextGenerationTaskConfig = TaskConfig; export class TextGenerationTask extends StreamingAiTask< diff --git a/packages/ai/src/task/TextLanguageDetectionTask.ts b/packages/ai/src/task/TextLanguageDetectionTask.ts index fd141aa02..cb098f7cd 100644 --- a/packages/ai/src/task/TextLanguageDetectionTask.ts +++ b/packages/ai/src/task/TextLanguageDetectionTask.ts @@ -6,8 +6,9 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { AiTask } from "./base/AiTask"; import { TypeModel } from "./base/AiTaskSchemas"; @@ -65,8 +66,12 @@ export const TextLanguageDetectionOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextLanguageDetectionTaskInput = FromSchema; -export type TextLanguageDetectionTaskOutput = FromSchema; +export type TextLanguageDetectionTaskInput = { + maxLanguages?: number | undefined; + model: string | ModelConfig; + text: string; +}; +export type TextLanguageDetectionTaskOutput = { languages: { score: number; language: string }[] }; export type TextLanguageDetectionTaskConfig = TaskConfig; export class TextLanguageDetectionTask extends AiTask< diff --git a/packages/ai/src/task/TextNamedEntityRecognitionTask.ts b/packages/ai/src/task/TextNamedEntityRecognitionTask.ts index 7d64ddec9..20de65921 100644 --- a/packages/ai/src/task/TextNamedEntityRecognitionTask.ts +++ b/packages/ai/src/task/TextNamedEntityRecognitionTask.ts @@ -6,8 +6,9 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { AiTask } from "./base/AiTask"; import { TypeModel } from "./base/AiTaskSchemas"; @@ -72,12 +73,14 @@ export const TextNamedEntityRecognitionOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextNamedEntityRecognitionTaskInput = FromSchema< - typeof TextNamedEntityRecognitionInputSchema ->; -export type TextNamedEntityRecognitionTaskOutput = FromSchema< - typeof TextNamedEntityRecognitionOutputSchema ->; +export type TextNamedEntityRecognitionTaskInput = { + blockList?: string[] | undefined; + model: string | ModelConfig; + text: string; +}; +export type TextNamedEntityRecognitionTaskOutput = { + entities: { score: number; entity: string; word: string }[]; +}; export type TextNamedEntityRecognitionTaskConfig = TaskConfig; export class TextNamedEntityRecognitionTask extends AiTask< diff --git a/packages/ai/src/task/TextQuestionAnswerTask.ts b/packages/ai/src/task/TextQuestionAnswerTask.ts index 8a785cd4b..d99d151f6 100644 --- a/packages/ai/src/task/TextQuestionAnswerTask.ts +++ b/packages/ai/src/task/TextQuestionAnswerTask.ts @@ -6,8 +6,9 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel } from "./base/AiTaskSchemas"; import { StreamingAiTask } from "./base/StreamingAiTask"; @@ -52,8 +53,12 @@ export const TextQuestionAnswerOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextQuestionAnswerTaskInput = FromSchema; -export type TextQuestionAnswerTaskOutput = FromSchema; +export type TextQuestionAnswerTaskInput = { + model: string | ModelConfig; + context: string; + question: string; +}; +export type TextQuestionAnswerTaskOutput = { text: string }; export type TextQuestionAnswerTaskConfig = TaskConfig; export class TextQuestionAnswerTask extends StreamingAiTask< diff --git a/packages/ai/src/task/TextRerankerTask.ts b/packages/ai/src/task/TextRerankerTask.ts index ea94a5447..8f19b8f07 100644 --- a/packages/ai/src/task/TextRerankerTask.ts +++ b/packages/ai/src/task/TextRerankerTask.ts @@ -6,8 +6,9 @@ import type { TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { AiTask } from "./base/AiTask"; import { TypeModel } from "./base/AiTaskSchemas"; @@ -60,8 +61,13 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextRerankerTaskInput = FromSchema; -export type TextRerankerTaskOutput = FromSchema; +export type TextRerankerTaskInput = { + topK?: number | undefined; + model: string | ModelConfig; + query: string; + documents: string[]; +}; +export type TextRerankerTaskOutput = { scores: number[]; indices: number[] }; export type TextRerankerTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/TextRewriterTask.ts b/packages/ai/src/task/TextRewriterTask.ts index e4d56ec45..1f5e0d62b 100644 --- a/packages/ai/src/task/TextRewriterTask.ts +++ b/packages/ai/src/task/TextRewriterTask.ts @@ -6,8 +6,9 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel } from "./base/AiTaskSchemas"; import { StreamingAiTask } from "./base/StreamingAiTask"; @@ -46,8 +47,8 @@ export const TextRewriterOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextRewriterTaskInput = FromSchema; -export type TextRewriterTaskOutput = FromSchema; +export type TextRewriterTaskInput = { model: string | ModelConfig; text: string; prompt: string }; +export type TextRewriterTaskOutput = { text: string }; export type TextRewriterTaskConfig = TaskConfig; export class TextRewriterTask extends StreamingAiTask< diff --git a/packages/ai/src/task/TextSummaryTask.ts b/packages/ai/src/task/TextSummaryTask.ts index 0044cec50..aa68b63c9 100644 --- a/packages/ai/src/task/TextSummaryTask.ts +++ b/packages/ai/src/task/TextSummaryTask.ts @@ -6,8 +6,9 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeModel } from "./base/AiTaskSchemas"; import { StreamingAiTask } from "./base/StreamingAiTask"; @@ -41,8 +42,8 @@ export const TextSummaryOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextSummaryTaskInput = FromSchema; -export type TextSummaryTaskOutput = FromSchema; +export type TextSummaryTaskInput = { model: string | ModelConfig; text: string }; +export type TextSummaryTaskOutput = { text: string }; export type TextSummaryTaskConfig = TaskConfig; export class TextSummaryTask extends StreamingAiTask< diff --git a/packages/ai/src/task/TextTranslationTask.ts b/packages/ai/src/task/TextTranslationTask.ts index faab5268d..0ff2139d7 100644 --- a/packages/ai/src/task/TextTranslationTask.ts +++ b/packages/ai/src/task/TextTranslationTask.ts @@ -6,8 +6,9 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; +import type { ModelConfig } from "../model/ModelSchema"; import { TypeLanguage, TypeModel } from "./base/AiTaskSchemas"; import { StreamingAiTask } from "./base/StreamingAiTask"; @@ -61,8 +62,13 @@ export const TextTranslationOutputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TextTranslationTaskInput = FromSchema; -export type TextTranslationTaskOutput = FromSchema; +export type TextTranslationTaskInput = { + model: string | ModelConfig; + text: string; + source_lang: string; + target_lang: string; +}; +export type TextTranslationTaskOutput = { text: string; target_lang: string }; export type TextTranslationTaskConfig = TaskConfig; export class TextTranslationTask extends StreamingAiTask< diff --git a/packages/ai/src/task/ToolCallingTask.ts b/packages/ai/src/task/ToolCallingTask.ts index b244daa0a..f9bde2ffa 100644 --- a/packages/ai/src/task/ToolCallingTask.ts +++ b/packages/ai/src/task/ToolCallingTask.ts @@ -8,7 +8,7 @@ import { CreateWorkflow, getTaskConstructors, Workflow } from "@workglow/task-gr import type { IExecuteContext, IRunConfig, StreamEvent, TaskConfig } from "@workglow/task-graph"; import { makeFingerprint, ServiceRegistry } from "@workglow/util"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; import type { AiJobInput } from "../job/AiJob"; import type { ModelConfig } from "../model/ModelSchema"; @@ -246,15 +246,38 @@ export const ToolCallingOutputSchema = { * schema prevents `FromSchema` from producing a useful type). */ export type ToolCallingTaskInput = Omit< - FromSchema, - "tools" | "messages" + { + systemPrompt?: string | undefined; + messages?: ChatMessage[] | undefined; + toolChoice?: string | undefined; + maxTokens?: number | undefined; + temperature?: number | undefined; + model: string | ModelConfig; + prompt: string | (string | { [x: string]: unknown; type: "text" | "image" | "audio" })[]; + tools: ( + | string + | { + [x: string]: unknown; + outputSchema?: { [x: string]: unknown } | undefined; + configSchema?: { [x: string]: unknown } | undefined; + config?: { [x: string]: unknown } | undefined; + description: string; + name: string; + inputSchema: { [x: string]: unknown }; + } + )[]; + }, + "messages" | "tools" > & { readonly tools: ToolDefinition[]; readonly messages?: ReadonlyArray; readonly sessionId?: string; }; -export type ToolCallingTaskOutput = FromSchema; +export type ToolCallingTaskOutput = { + text: string; + toolCalls: { id: string; name: string; input: { [x: string]: unknown } }[]; +}; export type ToolCallingTaskConfig = TaskConfig; export class ToolCallingTask extends StreamingAiTask< diff --git a/packages/ai/src/task/TopicSegmenterTask.ts b/packages/ai/src/task/TopicSegmenterTask.ts index d86ed6672..6bc78ad4a 100644 --- a/packages/ai/src/task/TopicSegmenterTask.ts +++ b/packages/ai/src/task/TopicSegmenterTask.ts @@ -7,7 +7,7 @@ import { CreateWorkflow, IExecuteContext, Task, Workflow } from "@workglow/task-graph"; import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; -import { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; export const SegmentationMethod = { @@ -88,8 +88,17 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type TopicSegmenterTaskInput = FromSchema; -export type TopicSegmenterTaskOutput = FromSchema; +export type TopicSegmenterTaskInput = { + method?: "hybrid" | "heuristic" | "embedding-similarity" | undefined; + minSegmentSize?: number | undefined; + maxSegmentSize?: number | undefined; + similarityThreshold?: number | undefined; + text: string; +}; +export type TopicSegmenterTaskOutput = { + count: number; + segments: { text: string; startOffset: number; endOffset: number }[]; +}; export type TopicSegmenterTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/VectorQuantizeTask.ts b/packages/ai/src/task/VectorQuantizeTask.ts index f08233276..86d7db4e4 100644 --- a/packages/ai/src/task/VectorQuantizeTask.ts +++ b/packages/ai/src/task/VectorQuantizeTask.ts @@ -8,12 +8,10 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Task, Workflow } from "@workglow/task-graph"; import { DataPortSchema, - FromSchema, normalizeNumberArray, TensorType, TypedArray, TypedArraySchema, - TypedArraySchemaOptions, } from "@workglow/util/schema"; import type { Capability } from "../capability/Capabilities"; @@ -92,8 +90,16 @@ const outputSchema = { additionalProperties: false, } as const satisfies DataPortSchema; -export type VectorQuantizeTaskInput = FromSchema; -export type VectorQuantizeTaskOutput = FromSchema; +export type VectorQuantizeTaskInput = { + normalize?: boolean | undefined; + vector: TypedArray | TypedArray[]; + targetType: "float16" | "float32" | "float64" | "int8" | "uint8" | "int16" | "uint16"; +}; +export type VectorQuantizeTaskOutput = { + vector: TypedArray | TypedArray[]; + targetType: "float16" | "float32" | "float64" | "int8" | "uint8" | "int16" | "uint16"; + originalType: "float16" | "float32" | "float64" | "int8" | "uint8" | "int16" | "uint16"; +}; export type VectorQuantizeTaskConfig = TaskConfig; /** diff --git a/packages/ai/src/task/generation/ImageEditTask.ts b/packages/ai/src/task/generation/ImageEditTask.ts index 0b584c0a3..b0b4cf454 100644 --- a/packages/ai/src/task/generation/ImageEditTask.ts +++ b/packages/ai/src/task/generation/ImageEditTask.ts @@ -8,9 +8,10 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; import type { ImageValue, WithImageValuePorts } from "@workglow/util/media"; import { ImageValueSchema } from "@workglow/util/media"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../../capability/Capabilities"; +import type { ModelConfig } from "../../model/ModelSchema"; import type { AiImageOutput } from "../base/AiImageOutputTask"; import { AiImageOutputTask } from "../base/AiImageOutputTask"; import { TypeModel } from "../base/AiTaskSchemas"; @@ -51,7 +52,18 @@ export const ImageEditInputSchema = { export const ImageEditOutputSchema: DataPortSchema = AiImageOutputSchema; -type ImageEditSchemaInput = FromSchema; +type ImageEditSchemaInput = { + additionalImages?: (string | { [x: string]: unknown })[] | undefined; + mask?: string | { [x: string]: unknown } | undefined; + aspectRatio?: "1:1" | "16:9" | "9:16" | "4:3" | "3:4" | undefined; + quality?: "low" | "medium" | "high" | undefined; + seed?: number | undefined; + negativePrompt?: string | undefined; + providerOptions?: { [x: string]: unknown } | undefined; + model: string | ModelConfig; + prompt: string; + image: string | { [x: string]: unknown }; +}; export type ImageEditTaskInput = WithImageValuePorts< ImageEditSchemaInput, diff --git a/packages/ai/src/task/generation/ImageGenerateTask.ts b/packages/ai/src/task/generation/ImageGenerateTask.ts index a21150c3d..13f452fc1 100644 --- a/packages/ai/src/task/generation/ImageGenerateTask.ts +++ b/packages/ai/src/task/generation/ImageGenerateTask.ts @@ -6,9 +6,10 @@ import type { IRunConfig, TaskConfig } from "@workglow/task-graph"; import { CreateWorkflow, Workflow } from "@workglow/task-graph"; -import type { DataPortSchema, FromSchema } from "@workglow/util/schema"; +import type { DataPortSchema } from "@workglow/util/schema"; import type { Capability } from "../../capability/Capabilities"; +import type { ModelConfig } from "../../model/ModelSchema"; import type { AiImageOutput } from "../base/AiImageOutputTask"; import { AiImageOutputTask } from "../base/AiImageOutputTask"; import { TypeModel } from "../base/AiTaskSchemas"; @@ -33,7 +34,15 @@ export const ImageGenerateInputSchema = { export const ImageGenerateOutputSchema: DataPortSchema = AiImageOutputSchema; -export type ImageGenerateTaskInput = FromSchema; +export type ImageGenerateTaskInput = { + aspectRatio?: "1:1" | "16:9" | "9:16" | "4:3" | "3:4" | undefined; + quality?: "low" | "medium" | "high" | undefined; + seed?: number | undefined; + negativePrompt?: string | undefined; + providerOptions?: { [x: string]: unknown } | undefined; + model: string | ModelConfig; + prompt: string; +}; export type ImageGenerateTaskOutput = AiImageOutput; export type ImageGenerateTaskConfig = TaskConfig; diff --git a/scripts/typecheck-budget.json b/scripts/typecheck-budget.json new file mode 100644 index 000000000..58d9f94c2 --- /dev/null +++ b/scripts/typecheck-budget.json @@ -0,0 +1,40 @@ +{ + "tolerance": 0.15, + "floor": 50000, + "packages": { + "packages/ai": 92750, + "packages/browser-control": 73377, + "packages/indexeddb": 17929, + "packages/javascript": 14369, + "packages/job-queue": 10416, + "packages/knowledge-base": 47411, + "packages/mcp": 128898, + "packages/storage": 51128, + "packages/task-graph": 59917, + "packages/tasks": 206787, + "packages/test": 947215, + "packages/util": 26521, + "packages/workglow": 157, + "providers/anthropic": 12713, + "providers/aws": 2508, + "providers/bun-webview": 225, + "providers/cactus": 15572, + "providers/chrome-ai": 14580, + "providers/cloudflare": 1136, + "providers/electron": 203, + "providers/google-gemini": 19724, + "providers/huggingface-inference": 16829, + "providers/huggingface-transformers": 57091, + "providers/llamacpp-server": 19272, + "providers/mlx": 577, + "providers/node-llama-cpp": 21079, + "providers/ollama": 17096, + "providers/openai": 18568, + "providers/playwright": 1223, + "providers/postgres": 20430, + "providers/sqlite": 24728, + "providers/stable-diffusion-server": 10552, + "providers/supabase": 26344, + "providers/tf-mediapipe": 22598 + } +} diff --git a/scripts/typecheck-budget.ts b/scripts/typecheck-budget.ts new file mode 100644 index 000000000..edff1e689 --- /dev/null +++ b/scripts/typecheck-budget.ts @@ -0,0 +1,219 @@ +/** + * @license + * Copyright 2026 Steven Roussey + * SPDX-License-Identifier: Apache-2.0 + * + * Typecheck budget guard. + * + * Catches type-level performance regressions (instantiation explosions) at PR + * time. It type-checks each composite package in isolation with + * `tsc --extendedDiagnostics`, reads the deterministic `Instantiations` count, + * and compares it against committed per-package budgets. Instantiations are + * machine-independent (unlike wall-clock check time), so the gate is stable + * across CI runners while still catching the kind of regressions this guard + * exists for — e.g. a schema `allOf` change that took `tasks` from ~290k to + * ~6.7M instantiations, or a base-generic change that doubled `ai`. + * + * Usage: + * bun scripts/typecheck-budget.ts # check against budgets, exit 1 on regression + * bun scripts/typecheck-budget.ts --update # rewrite budgets from current measurements + * bun scripts/typecheck-budget.ts --json # emit measurements as JSON to stdout + * bun scripts/typecheck-budget.ts --no-build # skip the warm `tsc -b` (reuse existing dist .d.ts) + * bun scripts/typecheck-budget.ts --tolerance 0.2 + */ + +import { spawnSync } from "node:child_process"; +import { existsSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { join, resolve } from "node:path"; + +const ROOT = resolve(import.meta.dir, ".."); +const BUDGET_FILE = join(ROOT, "scripts", "typecheck-budget.json"); +const TSC = join(ROOT, "node_modules", ".bin", "tsc"); + +/** Per-package budget plus global settings persisted to {@link BUDGET_FILE}. */ +export interface TypecheckBudget { + /** Fractional headroom allowed over budget before failing (e.g. 0.15 = +15%). */ + readonly tolerance: number; + /** + * Packages whose instantiation count is below this are not gated (the + * relative tolerance is meaningless for trivially small packages and only + * produces noise). + */ + readonly floor: number; + /** Per-package instantiation budgets, keyed by workspace-relative dir. */ + readonly packages: Readonly>; +} + +export interface PackageMeasurement { + readonly pkg: string; + readonly instantiations: number; + /** Wall-clock check time in seconds — informational only, never gated. */ + readonly checkTimeSeconds: number; +} + +const DEFAULT_TOLERANCE = 0.15; +const DEFAULT_FLOOR = 50_000; + +function listPackageDirs(): string[] { + const dirs: string[] = []; + for (const group of ["packages", "providers"]) { + const groupDir = join(ROOT, group); + if (!existsSync(groupDir)) continue; + const glob = new Bun.Glob("*/tsconfig.json"); + for (const match of glob.scanSync({ cwd: groupDir, onlyFiles: true })) { + dirs.push(`${group}/${match.replace(/\/tsconfig\.json$/, "")}`); + } + } + return dirs.sort(); +} + +function run(cmd: string, args: string[]): { stdout: string; status: number } { + const result = spawnSync(cmd, args, { cwd: ROOT, encoding: "utf8", maxBuffer: 1 << 28 }); + return { stdout: `${result.stdout ?? ""}${result.stderr ?? ""}`, status: result.status ?? 1 }; +} + +/** Build every composite package so each one's dependency `.d.ts` exist on disk. */ +function warmBuild(dirs: string[]): void { + for (const dir of dirs) rmSync(join(ROOT, dir, "tsconfig.tsbuildinfo"), { force: true }); + // Emit declarations only; we just need the .d.ts graph, not JS. + run(TSC, ["-b", ...dirs, "--force"]); +} + +function extractNumber(output: string, label: RegExp): number { + const match = output.match(label); + return match ? Number(match[1].replace(/,/g, "")) : 0; +} + +/** Type-check one package in isolation and read its diagnostics. */ +function measurePackage(dir: string): PackageMeasurement { + // Force a full re-check of just this project (deps already built by warmBuild). + rmSync(join(ROOT, dir, "tsconfig.tsbuildinfo"), { force: true }); + const { stdout } = run(TSC, ["-p", join(dir, "tsconfig.json"), "--extendedDiagnostics"]); + return { + pkg: dir, + instantiations: extractNumber(stdout, /Instantiations:\s+([\d,]+)/), + checkTimeSeconds: extractNumber(stdout, /Check time:\s+([\d.]+)s/), + }; +} + +function loadBudget(): TypecheckBudget { + if (!existsSync(BUDGET_FILE)) { + return { tolerance: DEFAULT_TOLERANCE, floor: DEFAULT_FLOOR, packages: {} }; + } + return JSON.parse(readFileSync(BUDGET_FILE, "utf8")) as TypecheckBudget; +} + +function writeBudget(budget: TypecheckBudget): void { + writeFileSync(BUDGET_FILE, `${JSON.stringify(budget, null, 2)}\n`); +} + +export interface BudgetCheckResult { + readonly regressions: ReadonlyArray<{ pkg: string; budget: number; actual: number }>; + readonly newPackages: ReadonlyArray<{ pkg: string; actual: number }>; + readonly improvements: ReadonlyArray<{ pkg: string; budget: number; actual: number }>; +} + +export function checkAgainstBudget( + measurements: ReadonlyArray, + budget: TypecheckBudget +): BudgetCheckResult { + const regressions: Array<{ pkg: string; budget: number; actual: number }> = []; + const newPackages: Array<{ pkg: string; actual: number }> = []; + const improvements: Array<{ pkg: string; budget: number; actual: number }> = []; + + for (const m of measurements) { + const allowed = budget.packages[m.pkg]; + if (allowed === undefined) { + if (m.instantiations >= budget.floor) newPackages.push({ pkg: m.pkg, actual: m.instantiations }); + continue; + } + const ceiling = Math.round(allowed * (1 + budget.tolerance)); + if (m.instantiations > ceiling) { + regressions.push({ pkg: m.pkg, budget: allowed, actual: m.instantiations }); + } else if (m.instantiations < Math.round(allowed * (1 - budget.tolerance))) { + improvements.push({ pkg: m.pkg, budget: allowed, actual: m.instantiations }); + } + } + return { regressions, newPackages, improvements }; +} + +function formatPct(actual: number, base: number): string { + const pct = ((actual - base) / base) * 100; + return `${pct >= 0 ? "+" : ""}${pct.toFixed(0)}%`; +} + +function main(): void { + const args = new Set(process.argv.slice(2)); + const update = args.has("--update"); + const emitJson = args.has("--json"); + const skipBuild = args.has("--no-build"); + const tolArgIndex = process.argv.indexOf("--tolerance"); + const tolerance = + tolArgIndex >= 0 ? Number(process.argv[tolArgIndex + 1]) : loadBudget().tolerance; + + const dirs = listPackageDirs(); + if (!existsSync(TSC)) { + console.error(`typecheck-budget: tsc not found at ${TSC}. Run \`bun install\` first.`); + process.exit(1); + } + + if (!skipBuild) { + process.stderr.write(`typecheck-budget: warm build of ${dirs.length} packages...\n`); + warmBuild(dirs); + } + + const measurements = dirs.map((dir) => { + const m = measurePackage(dir); + process.stderr.write( + ` ${m.pkg.padEnd(38)} inst=${String(m.instantiations).padStart(9)} check=${m.checkTimeSeconds}s\n` + ); + return m; + }); + + if (emitJson) { + console.log(JSON.stringify(measurements, null, 2)); + } + + if (update) { + const packages: Record = {}; + for (const m of measurements.sort((a, b) => a.pkg.localeCompare(b.pkg))) { + packages[m.pkg] = m.instantiations; + } + writeBudget({ tolerance, floor: DEFAULT_FLOOR, packages }); + process.stderr.write(`typecheck-budget: wrote ${BUDGET_FILE}\n`); + return; + } + + const budget = loadBudget(); + const { regressions, newPackages, improvements } = checkAgainstBudget(measurements, budget); + + for (const i of improvements) { + process.stderr.write( + `improved: ${i.pkg} ${i.actual} vs budget ${i.budget} (${formatPct(i.actual, i.budget)}) — consider \`--update\` to ratchet\n` + ); + } + for (const n of newPackages) { + process.stderr.write( + `new package not in budget: ${n.pkg} (inst=${n.actual}) — run \`--update\` to record it\n` + ); + } + + if (regressions.length > 0) { + process.stderr.write(`\ntypecheck-budget: ${regressions.length} regression(s) over budget +${budget.tolerance * 100}%:\n`); + for (const r of regressions) { + process.stderr.write( + ` ${r.pkg}: ${r.actual} instantiations vs budget ${r.budget} (${formatPct(r.actual, r.budget)})\n` + ); + } + process.stderr.write( + `\nIf this growth is expected, re-baseline with: bun scripts/typecheck-budget.ts --update\n` + ); + process.exit(1); + } + + process.stderr.write(`\ntypecheck-budget: OK (${measurements.length} packages within budget)\n`); +} + +if (import.meta.main) { + main(); +} From bc0918de7827f5f984354fd2b9a69cb6dd2f5382 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 7 Jun 2026 08:24:06 +0000 Subject: [PATCH 2/4] docs(task-graph): fix TaskOutputTabularRepository README examples for new constructor signature After commit c011234 removed the legacy `{ tabularRepository: ... }` option shim, copy-paste from these README examples produced a TS error against the new `{ storage: ITaskOutputStorage }` constructor signature. Replaced the eight stale `tabularRepository:` constructor calls across two READMEs with `storage: tabularTaskOutputStorage(...)` and pulled the helper into each example's import block. --- packages/task-graph/README.md | 79 +++++++++++++---------- packages/task-graph/src/storage/README.md | 15 +++-- 2 files changed, 55 insertions(+), 39 deletions(-) diff --git a/packages/task-graph/README.md b/packages/task-graph/README.md index d0da2061d..7ea829ce9 100644 --- a/packages/task-graph/README.md +++ b/packages/task-graph/README.md @@ -757,6 +757,7 @@ Both slots are optional. A missing slot is a silent no-op — the task still run import { CACHE_REGISTRY, DefaultCacheRegistry, + tabularTaskOutputStorage, TaskOutputPrimaryKeyNames, TaskOutputSchema, TaskOutputTabularRepository, @@ -767,22 +768,26 @@ import { Sqlite, SqliteTabularStorage } from "@workglow/sqlite/storage"; await Sqlite.init(); const deterministic = new TaskOutputTabularRepository({ - tabularRepository: new SqliteTabularStorage( - "./cache.sqlite", - "task_outputs_deterministic", - TaskOutputSchema, - TaskOutputPrimaryKeyNames, - ["createdAt"] + storage: tabularTaskOutputStorage( + new SqliteTabularStorage( + "./cache.sqlite", + "task_outputs_deterministic", + TaskOutputSchema, + TaskOutputPrimaryKeyNames, + ["createdAt"] + ) ), }); const privateBacking = new TaskOutputTabularRepository({ - tabularRepository: new SqliteTabularStorage( - "./cache.sqlite", - "task_outputs_private", - TaskOutputSchema, - TaskOutputPrimaryKeyNames, - ["createdAt"] + storage: tabularTaskOutputStorage( + new SqliteTabularStorage( + "./cache.sqlite", + "task_outputs_private", + TaskOutputSchema, + TaskOutputPrimaryKeyNames, + ["createdAt"] + ) ), }); @@ -866,6 +871,7 @@ import { Workflow, CACHE_REGISTRY, DefaultCacheRegistry, + tabularTaskOutputStorage, TaskOutputPrimaryKeyNames, TaskOutputSchema, TaskOutputTabularRepository, @@ -912,9 +918,9 @@ class ExpensiveTask extends Task<{ n: number }, { result: number }> { // Build a CacheRegistry with a deterministic slot. (Private slot omitted here — // ExpensiveTask is deterministic, so it never needs the private tier.) const deterministic = new TaskOutputTabularRepository({ - tabularRepository: new InMemoryTabularStorage(TaskOutputSchema, TaskOutputPrimaryKeyNames, [ - "createdAt", - ]), + storage: tabularTaskOutputStorage( + new InMemoryTabularStorage(TaskOutputSchema, TaskOutputPrimaryKeyNames, ["createdAt"]) + ), }); const registry = new ServiceRegistry(); @@ -977,6 +983,7 @@ Wire `TaskOutputTabularRepository` / `TaskGraphTabularRepository` from `@workglo ```typescript import { + tabularTaskOutputStorage, TaskGraphPrimaryKeyNames, TaskGraphSchema, TaskGraphTabularRepository, @@ -993,9 +1000,9 @@ import { Sqlite, SqliteTabularStorage } from "@workglow/sqlite/storage"; // In-memory (e.g. tests) const memoryOutput = new TaskOutputTabularRepository({ - tabularRepository: new InMemoryTabularStorage(TaskOutputSchema, TaskOutputPrimaryKeyNames, [ - "createdAt", - ]), + storage: tabularTaskOutputStorage( + new InMemoryTabularStorage(TaskOutputSchema, TaskOutputPrimaryKeyNames, ["createdAt"]) + ), }); const memoryGraph = new TaskGraphTabularRepository({ tabularRepository: new InMemoryTabularStorage(TaskGraphSchema, TaskGraphPrimaryKeyNames), @@ -1003,10 +1010,12 @@ const memoryGraph = new TaskGraphTabularRepository({ // File system const fsOutput = new TaskOutputTabularRepository({ - tabularRepository: new FsFolderTabularStorage( - "./task-output-cache", - TaskOutputSchema, - TaskOutputPrimaryKeyNames + storage: tabularTaskOutputStorage( + new FsFolderTabularStorage( + "./task-output-cache", + TaskOutputSchema, + TaskOutputPrimaryKeyNames + ) ), }); const fsGraph = new TaskGraphTabularRepository({ @@ -1020,12 +1029,14 @@ const fsGraph = new TaskGraphTabularRepository({ // SQLite (await Sqlite.init() once before using a path or new Sqlite.Database) await Sqlite.init(); const sqliteOutput = new TaskOutputTabularRepository({ - tabularRepository: new SqliteTabularStorage( - ":memory:", - "task_outputs", - TaskOutputSchema, - TaskOutputPrimaryKeyNames, - ["createdAt"] + storage: tabularTaskOutputStorage( + new SqliteTabularStorage( + ":memory:", + "task_outputs", + TaskOutputSchema, + TaskOutputPrimaryKeyNames, + ["createdAt"] + ) ), }); const sqliteGraph = new TaskGraphTabularRepository({ @@ -1039,11 +1050,13 @@ const sqliteGraph = new TaskGraphTabularRepository({ // IndexedDB (browser) — the `@workglow/web` example under `examples/web` includes small helpers const idbOutput = new TaskOutputTabularRepository({ - tabularRepository: new IndexedDbTabularStorage( - "task_outputs", - TaskOutputSchema, - TaskOutputPrimaryKeyNames, - ["createdAt"] + storage: tabularTaskOutputStorage( + new IndexedDbTabularStorage( + "task_outputs", + TaskOutputSchema, + TaskOutputPrimaryKeyNames, + ["createdAt"] + ) ), }); const idbGraph = new TaskGraphTabularRepository({ diff --git a/packages/task-graph/src/storage/README.md b/packages/task-graph/src/storage/README.md index 266b3438f..6498ebf69 100644 --- a/packages/task-graph/src/storage/README.md +++ b/packages/task-graph/src/storage/README.md @@ -15,6 +15,7 @@ TaskOutputRepository is a repository for task caching. If a task has the same in ```typescript // Example usage import { + tabularTaskOutputStorage, TaskOutputPrimaryKeyNames, TaskOutputSchema, TaskOutputTabularRepository, @@ -25,12 +26,14 @@ import { Sqlite } from "@workglow/storage/sqlite"; await Sqlite.init(); const outputRepo = new TaskOutputTabularRepository({ - tabularRepository: new SqliteTabularStorage( - ":memory:", - "task_outputs", - TaskOutputSchema, - TaskOutputPrimaryKeyNames, - ["createdAt"] + storage: tabularTaskOutputStorage( + new SqliteTabularStorage( + ":memory:", + "task_outputs", + TaskOutputSchema, + TaskOutputPrimaryKeyNames, + ["createdAt"] + ) ), }); await outputRepo.saveOutput("MyTaskType", { param: "value" }, { result: "data" }); From 6387d2e099236b88a0aecc77b1f192b0a1b917c8 Mon Sep 17 00:00:00 2001 From: Steven Roussey Date: Wed, 10 Jun 2026 07:51:16 -0700 Subject: [PATCH 3/4] Add model caching and secrets passphrase to CI workflows (#560) * ci: authenticate and cache Hugging Face model downloads CI test jobs were downloading ONNX/GGUF models anonymously from the Hugging Face Hub on every run, which hits anonymous IP rate limits on shared GitHub runners (downloads denied). - Pass WORKGLOW_SECRETS_PASSPHRASE to the model-downloading vitest jobs (hft, nodellama; rag already had it) so the test preload decrypts the on-disk credential store and hydrates HF_TOKEN into process.env. transformers.js authenticates Hub downloads when HF_TOKEN is set, raising the rate limit against the account. - Cache the repo-root ./models directory (where the test cache dir is configured) per job so models are not re-fetched on every run. * ci: add encrypted hf-token credential --------- Co-authored-by: Claude --- .github/workflows/test.yml | 25 +++++++++++++++++++ ...d32b3886e57b1701ac6fbaaf4f2aa0ba2bb8a.json | 1 + 2 files changed, 26 insertions(+) create mode 100644 .secrets/credentials/26f1356ad1fd23bad72f47e1cafd32b3886e57b1701ac6fbaaf4f2aa0ba2bb8a.json diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml index 6ca8b17f7..dd6a3c547 100644 --- a/.github/workflows/test.yml +++ b/.github/workflows/test.yml @@ -263,6 +263,13 @@ jobs: with: bun-version: latest - run: bun i + - name: Cache Hugging Face model downloads + uses: actions/cache@v4 + with: + path: models + key: hf-models-rag-${{ runner.os }}-${{ hashFiles('packages/test/src/samples/**') }} + restore-keys: | + hf-models-rag-${{ runner.os }}- - name: Download build artifacts uses: actions/download-artifact@v8 with: @@ -293,12 +300,21 @@ jobs: with: bun-version: latest - run: bun i + - name: Cache Hugging Face model downloads + uses: actions/cache@v4 + with: + path: models + key: hf-models-hft-${{ runner.os }}-${{ hashFiles('packages/test/src/samples/**') }} + restore-keys: | + hf-models-hft-${{ runner.os }}- - name: Download build artifacts uses: actions/download-artifact@v8 with: name: build-output path: . - name: Run HuggingFace Transformers provider tests via vitest + env: + WORKGLOW_SECRETS_PASSPHRASE: ${{ secrets.WORKGLOW_SECRETS_PASSPHRASE }} run: bun run test:vitest:ai-provider-hft - name: Copy Vitest coverage fragment run: cp coverage/coverage-final.json vitest-ai-provider-hft-coverage.json @@ -321,12 +337,21 @@ jobs: with: bun-version: latest - run: bun i + - name: Cache Hugging Face / GGUF model downloads + uses: actions/cache@v4 + with: + path: models + key: hf-models-nodellama-${{ runner.os }}-${{ hashFiles('packages/test/src/samples/**') }} + restore-keys: | + hf-models-nodellama-${{ runner.os }}- - name: Download build artifacts uses: actions/download-artifact@v8 with: name: build-output path: . - name: Run LlamaCpp provider tests via vitest + env: + WORKGLOW_SECRETS_PASSPHRASE: ${{ secrets.WORKGLOW_SECRETS_PASSPHRASE }} run: bun run test:vitest:ai-provider-nodellama - name: Copy Vitest coverage fragment run: cp coverage/coverage-final.json vitest-ai-provider-nodellama-coverage.json diff --git a/.secrets/credentials/26f1356ad1fd23bad72f47e1cafd32b3886e57b1701ac6fbaaf4f2aa0ba2bb8a.json b/.secrets/credentials/26f1356ad1fd23bad72f47e1cafd32b3886e57b1701ac6fbaaf4f2aa0ba2bb8a.json new file mode 100644 index 000000000..24d881e32 --- /dev/null +++ b/.secrets/credentials/26f1356ad1fd23bad72f47e1cafd32b3886e57b1701ac6fbaaf4f2aa0ba2bb8a.json @@ -0,0 +1 @@ +{"key":"hf-token","value":"{\"encrypted\":\"pkdH1J0oiGVQ6u6wt3T8VxW1rM2LBGksRUBtD8OIw1QicrhzxzqGDyw3fiuMY/yLGohtQADIGY6YJS1hMNzTmWCTXQFv\",\"iv\":\"Qp3YbU0UR5H/c05J\",\"provider\":\"huggingface\",\"createdAt\":\"2026-06-10T14:33:08.480Z\",\"updatedAt\":\"2026-06-10T14:33:08.480Z\"}"} \ No newline at end of file From 8bd2dca3399ef11ab83fc066f4d5fbd48e149482 Mon Sep 17 00:00:00 2001 From: Steven Roussey Date: Wed, 10 Jun 2026 15:00:40 +0000 Subject: [PATCH 4/4] ci: add encrypted hf-token credential --- ...1fd23bad72f47e1cafd32b3886e57b1701ac6fbaaf4f2aa0ba2bb8a.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.secrets/credentials/26f1356ad1fd23bad72f47e1cafd32b3886e57b1701ac6fbaaf4f2aa0ba2bb8a.json b/.secrets/credentials/26f1356ad1fd23bad72f47e1cafd32b3886e57b1701ac6fbaaf4f2aa0ba2bb8a.json index 24d881e32..03d0a4270 100644 --- a/.secrets/credentials/26f1356ad1fd23bad72f47e1cafd32b3886e57b1701ac6fbaaf4f2aa0ba2bb8a.json +++ b/.secrets/credentials/26f1356ad1fd23bad72f47e1cafd32b3886e57b1701ac6fbaaf4f2aa0ba2bb8a.json @@ -1 +1 @@ -{"key":"hf-token","value":"{\"encrypted\":\"pkdH1J0oiGVQ6u6wt3T8VxW1rM2LBGksRUBtD8OIw1QicrhzxzqGDyw3fiuMY/yLGohtQADIGY6YJS1hMNzTmWCTXQFv\",\"iv\":\"Qp3YbU0UR5H/c05J\",\"provider\":\"huggingface\",\"createdAt\":\"2026-06-10T14:33:08.480Z\",\"updatedAt\":\"2026-06-10T14:33:08.480Z\"}"} \ No newline at end of file +{"key":"hf-token","value":"{\"encrypted\":\"eZT55PHECPQsE5HycJH/V8SKUiEVuSzpUcjtuvDbVCK42eRvav0eZGMhfjv3vetebvIdYJcJzwpNYIc0ZOKuZbTBUStW\",\"iv\":\"df4IdKxc2JYoz5CV\",\"provider\":\"huggingface\",\"createdAt\":\"2026-06-10T14:33:08.480Z\",\"updatedAt\":\"2026-06-10T15:00:29.988Z\"}"} \ No newline at end of file