Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 22 additions & 21 deletions influxdata/nori_regression/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,8 +60,8 @@ The Nori gateway API key is a secret and is **never** read from trigger argument
body (both are logged). It is resolved, in order:

1. a non-empty `X-Nori-Api-Key: <key>` request header (HTTP trigger only), then
2. the `NORI_API_KEY` environment variable set on the InfluxDB host (required for the scheduled
trigger).
2. the `SYNTHEFY_NORI_API_KEY` environment variable set on the InfluxDB host (required for the
scheduled trigger).

The key is intentionally **not** accepted in the `Authorization` header: InfluxDB parses
`Authorization` for its own request authorization, so a key placed there never reaches the plugin.
Expand All @@ -84,6 +84,7 @@ InfluxDB host. It must be an `https://` URL; plain `http://` is accepted only fo
| `measurement` | string | required | Source measurement (table) to read from. |
| `field` | string | required | The numeric field to predict. The plugin trains on the rows where it is present and predicts the rows where it is null. |
| `feature_fields` | string | required | Numeric feature columns (X) used to predict `field`, **space-separated** (for example `temp humidity`). Use spaces, not commas (`--trigger-arguments` splits argument pairs on commas) and not dots (a field name may contain a `.`). |
| `model` | string | required | The Nori gateway slug to call. There is no default: the slug selects a priced model, so the plugin will not choose one for you. See [Supported models](#supported-models). Trigger argument only. |

A column name that contains a space cannot be expressed in `feature_fields` as a trigger argument,
because every string form splits on whitespace. Name such a column from a TOML array
Expand All @@ -97,7 +98,6 @@ because every string form splits on whitespace. Name such a column from a TOML a
| `start_time` | string | *(none)* | ISO 8601 start of a fixed window. Given alone, the window ends now. |
| `end_time` | string | *(none)* | ISO 8601 end of a fixed window. Given alone, the window starts one `window` earlier. |
| `tags` | string | *(none)* | Filter to a single series. Format: `key:val key2:val2` (space-separated pairs, one value per key). A token without a `:` is rejected. Required when the window holds more than one series. |
| `model` | string | `synthefy/nori-30m` | The Nori gateway slug to call. See [Supported models](#supported-models). |
| `output_measurement` | string | `<measurement>_regressed` | Measurement to write predictions to. Must differ from `measurement`. |
| `target_database` | string | *(trigger db)* | Write predictions to a different database. |
| `dry_run` | boolean | `false` | Log the first few predictions and return them all, without writing anything. |
Expand All @@ -107,7 +107,7 @@ because every string form splits on whitespace. Name such a column from a TOML a
| `max_predict_rows` | integer | `5000` | Cap on rows predicted per run; the most recent rows are kept and the rest wait for a later run. |
| `max_read_rows` | integer | `50000` | Ceiling on rows read from InfluxDB in one run, applied as a `LIMIT` on the query. The most recent rows are read, and a truncated read is logged with a warning. This bounds the plugin's memory: a row costs roughly 0.7 KB while it is held, so the default is about 35 MB. |
| `predict_batch_size` | integer | `1000` | Rows per gateway call. Each batch re-sends the training context and is billed separately, so a larger value costs less. |
| `request_timeout` | string | `300s` | Timeout for one gateway call. A cold start has been measured at 60-130 seconds, so keep this well above that. |
| `request_timeout` | string | `300s` | Timeout for one gateway call. A model that has scaled to zero cold-starts on the first request, measured between roughly one and four minutes depending on the variant, so keep this well above the warm response time. |
| `max_retries` | integer | `3` | Maximum attempts per gateway call and per write. `1` disables retry. |
| `config_file_path` | string | *(none)* | Path to a TOML file supplying every parameter, relative to `PLUGIN_DIR`. Cannot be combined with other inline arguments or a request body. |

Expand Down Expand Up @@ -185,7 +185,7 @@ influxdb3 create trigger \
3. Set the Nori gateway key on the InfluxDB host, so the scheduled trigger can read it:

```bash
export NORI_API_KEY="<your Nori API key>"
export SYNTHEFY_NORI_API_KEY="<your Nori API key>"
```

### Data requirements
Expand Down Expand Up @@ -273,7 +273,7 @@ influxdb3 create trigger \
--database mydb \
--path "gh:influxdata/nori_regression/nori_regression.py" \
--trigger-spec "every:15m" \
--trigger-arguments measurement=sensors,field=pressure,feature_fields="temp humidity",tags=site:A,min_history=3 \
--trigger-arguments measurement=sensors,field=pressure,feature_fields="temp humidity",tags=site:A,model=synthefy/nori-30m,min_history=3 \
nori_example
```

Expand Down Expand Up @@ -305,7 +305,7 @@ Call the HTTP endpoint (exposed at `/api/v3/engine/<path>`), passing the Nori ke

```bash
curl -X POST http://localhost:8181/api/v3/engine/nori_regress \
-H "X-Nori-Api-Key: $NORI_API_KEY" \
-H "X-Nori-Api-Key: $SYNTHEFY_NORI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"measurement":"sensors","field":"pressure","feature_fields":["temp","humidity"],"tags":{"site":"A"}}'
```
Expand Down Expand Up @@ -335,7 +335,7 @@ The top-level `status` is one of `success`, `partial`, `skipped`, `dry_run` or `

```bash
curl -X POST http://localhost:8181/api/v3/engine/nori_regress \
-H "X-Nori-Api-Key: $NORI_API_KEY" \
-H "X-Nori-Api-Key: $SYNTHEFY_NORI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"measurement":"sensors","field":"pressure","feature_fields":["temp","humidity"],"tags":{"site":"A"},"start_time":"2026-01-01T00:00:00Z","end_time":"2026-02-01T00:00:00Z"}'
```
Expand All @@ -347,7 +347,7 @@ reads the `window` before it.

```bash
curl -X POST http://localhost:8181/api/v3/engine/nori_regress \
-H "X-Nori-Api-Key: $NORI_API_KEY" \
-H "X-Nori-Api-Key: $SYNTHEFY_NORI_API_KEY" \
-H "Content-Type: application/json" \
-d '{"measurement":"sensors","field":"pressure","feature_fields":["temp","humidity"],"tags":{"site":"A"},"dry_run":true}'
```
Expand Down Expand Up @@ -414,15 +414,16 @@ ORDER BY 1 DESC

## Supported models

The `model` argument is the Nori gateway slug your API key is granted:
The `model` argument is the Nori gateway slug your API key is granted. It is **required**: the
slug selects a priced model, so the plugin will not choose one on your behalf. Synthefy's own
client and local package take the same position.

| Slug | Parameters | Notes |
|---|---|---|
| `synthefy/nori-30m` | ~29M | The default, and the variant Synthefy's own documentation recommends. Priced higher and slower to cold-start (measured at ~125s). |
| `synthefy/nori-6m` | ~6M | Cheaper per request and faster to cold-start (measured at ~69s). |
Synthefy publishes the current models, their sizes and their slugs at
[docs.synthefy.com/nori/quickstart#models](https://docs.synthefy.com/nori/quickstart#models). That list is the authoritative one:
it changes when Synthefy releases a variant, and a slug not on it will not route.

Which one predicts better depends on your data; try both with `dry_run=true` before committing a
schedule to one.
Which model predicts better depends on your data, and the larger ones cost more per request and
take longer to cold-start. Try a couple with `dry_run=true` before committing a schedule to one.

The bare `synthefy/nori` slug has been retired and no longer routes; the plugin rejects it with a
pointed message rather than letting the gateway answer `404`. One API key from the
Expand Down Expand Up @@ -469,8 +470,8 @@ caller-facing message with the full detail in `processing_engine_logs`.

The plugin cannot find a Nori gateway key.

**Solution:** set `NORI_API_KEY` on the InfluxDB host, or pass an `X-Nori-Api-Key: <key>` header
when calling the HTTP trigger (see
**Solution:** set `SYNTHEFY_NORI_API_KEY` on the InfluxDB host, or pass an
`X-Nori-Api-Key: <key>` header when calling the HTTP trigger (see
[Authentication](#authentication-for-the-nori-gateway)). An empty header value is ignored and the
environment variable is used instead.

Expand Down Expand Up @@ -526,9 +527,9 @@ names `time`/`y`.

#### Cold-start latency and timeouts

The models scale to zero, so the first request after an idle period is slow: about 69 seconds for
`synthefy/nori-6m` and 125 seconds for `synthefy/nori-30m` in measurement, and it can return a
`503` or a non-JSON body from the fronting proxy once.
The models scale to zero, so the first request after an idle period is slow: measurements have
ranged from roughly one minute to nearly four, with the larger variants slower, and it can return
a `503` or a non-JSON body from the fronting proxy once.

**Solution:** the default `request_timeout` of `300s` and `max_retries` of `3` are set to absorb
this; a `503`, a `429` and a connection error are retried with backoff. A read timeout is **not**
Expand Down
50 changes: 35 additions & 15 deletions influxdata/nori_regression/nori_regression.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
{"name": "start_time", "example": "2026-01-01T00:00:00Z", "description": "ISO start of a fixed window instead of a trailing one. With skip_existing left on, a schedule over a fixed window backfills and then stops calling the gateway.", "required": false},
{"name": "end_time", "example": "2026-02-01T00:00:00Z", "description": "ISO end of a fixed window. Given alone, the window starts one `window` earlier.", "required": false},
{"name": "tags", "example": "site:A", "description": "Filter to a single series. Format: key:val key2:val2 (space-separated pairs, single value per key). A token without ':' is rejected. Required if the window holds more than one series.", "required": false},
{"name": "model", "example": "synthefy/nori-30m", "description": "The Nori gateway slug to call. Current slugs: synthefy/nori-30m (default, ~29M params) and synthefy/nori-6m (cheaper and faster to cold-start). The bare 'synthefy/nori' slug is retired. Your API key must be granted the slug.", "required": false},
{"name": "model", "example": "synthefy/nori-30m", "description": "The Nori gateway slug to call. Required: there is no default, because the slug selects a priced model. The current list of models and their slugs is at https://docs.synthefy.com/nori/quickstart#models. The bare 'synthefy/nori' slug is retired. Your API key must be granted the slug.", "required": true},
{"name": "output_measurement", "example": "sensors_regressed", "description": "Where to write predictions. Default: <measurement>_regressed.", "required": false},
{"name": "target_database", "example": "predictions", "description": "Write predictions to this database instead of the trigger's own.", "required": false},
{"name": "dry_run", "example": "false", "description": "If true, log predictions but do not write them.", "required": false},
Expand All @@ -32,7 +32,7 @@
{"name": "start_time", "example": "2026-01-01T00:00:00Z", "description": "ISO start of the window. May also be in the request body. Given alone, the window ends now.", "required": false},
{"name": "end_time", "example": "2026-02-01T00:00:00Z", "description": "ISO end of the window. May also be in the request body. Given alone, the window starts one `window` earlier.", "required": false},
{"name": "dry_run", "example": "false", "description": "If true, log predictions but do not write them. May also be in the request body.", "required": false},
{"name": "model", "example": "synthefy/nori-30m", "description": "The Nori gateway slug to call. Trigger argument only: it selects a billed model, so the request body cannot override it.", "required": false},
{"name": "model", "example": "synthefy/nori-30m", "description": "The Nori gateway slug to call. Required, and a trigger argument only: it selects a billed model, so there is no default and the request body cannot override it.", "required": true},
{"name": "output_measurement", "example": "sensors_regressed", "description": "Where to write predictions. Trigger argument only: the request body cannot override a write target.", "required": false},
{"name": "target_database", "example": "predictions", "description": "Write predictions to this database instead of the trigger's own. Trigger argument only: the request body cannot override a write target.", "required": false},
{"name": "min_history", "example": "50", "description": "Minimum labeled rows required to train. Trigger argument only.", "required": false},
Expand Down Expand Up @@ -80,18 +80,31 @@
DEFAULT_GATEWAY_URL = "https://inference.baseten.co/predict"
GATEWAY_URL_ENV_VAR = "NORI_GATEWAY_URL"

# The key is a SECRET: it is read from the NORI_API_KEY environment variable on the InfluxDB host,
# or (HTTP trigger only) from an incoming X-Nori-Api-Key request header. It is NEVER read from
# trigger args or the request body (both are logged), and never from the incoming `Authorization`
# header (InfluxDB consumes that header for its own request authorization).
API_KEY_ENV_VAR = "NORI_API_KEY"
# The key is a SECRET: it is read from the SYNTHEFY_NORI_API_KEY environment variable on the
# InfluxDB host, or (HTTP trigger only) from an incoming X-Nori-Api-Key request header. It is NEVER
# read from trigger args or the request body (both are logged), and never from the incoming
# `Authorization` header (InfluxDB consumes that header for its own request authorization).
#
# The name carries the vendor prefix because an InfluxDB host runs plugins from several authors:
# an unprefixed NORI_API_KEY says nothing about whose service it authenticates against.
API_KEY_ENV_VAR = "SYNTHEFY_NORI_API_KEY"
API_KEY_HEADER = "X-Nori-Api-Key"

# Slugs name their parameter count so the model behind a slug never silently changes. The bare
# `synthefy/nori` slug was retired and now returns 404, so it is rejected with a pointed message.
DEFAULT_MODEL_SLUG = "synthefy/nori-30m"
# There is no default model. Synthefy's own client and local package both require an explicit
# size and raise rather than pick one, because a variant is a priced choice: selecting one on the
# operator's behalf spends their money and pins them to a slug they never named. This plugin
# follows that. The slug below is only the example shown in help text and suggested when a retired
# slug is used.
EXAMPLE_MODEL_SLUG = "synthefy/nori-30m"
RETIRED_MODEL_SLUGS = {"synthefy/nori"}

# Which models exist, and their sizes, is Synthefy's to publish and changes when they release one.
# This plugin names only its default and points at the vendor's list, so a new variant does not
# make the plugin's documentation wrong.
MODEL_LIST_URL = "https://docs.synthefy.com/nori/quickstart#models"

# Gateway faults worth another attempt: 408/409/425 (transient conflicts), 429 (per-key rate limit,
# 50/min) and every 5xx (a cold start can 503, or 500 once). A 4xx outside that set is a permanent
# input, key or slug problem, so retrying it only wastes the caller's time and the gateway's quota.
Expand Down Expand Up @@ -120,8 +133,9 @@
# that STARTS WITH "@": @format and @jinja interpolate `env`, @read_file reads the host filesystem,
# @get reads other settings. Every one of dynaconf's ~30 tokens begins with this single character,
# and nothing else triggers them (a leading space or a doubled @ is inert), so refusing a leading
# "@" is a complete guard. Without it a request body of {"measurement": "@format {env[NORI_API_KEY]}"}
# comes back resolved in the error message - the plugin's own secret, handed to the caller.
# "@" is a complete guard. Without it a request body of
# {"measurement": "@format {env[SYNTHEFY_NORI_API_KEY]}"} comes back resolved in the error
# message - the plugin's own secret, handed to the caller.
DYNACONF_TOKEN_PREFIX = "@"

# Parameters an earlier revision accepted. They are rejected by name rather than ignored, so an
Expand Down Expand Up @@ -186,7 +200,7 @@ class GatewayError(PublicError):
Validator("field", default="", cast=str),
Validator("feature_fields", default="", cast=parse_delimited_list),
Validator("window", default="30d", cast=parse_timedelta),
Validator("model", default=DEFAULT_MODEL_SLUG, cast=str),
Validator("model", default="", cast=str),
Validator("dry_run", default=False, cast=parse_bool),
Validator("skip_existing", default=True, cast=parse_bool),
Validator("min_history", default=50, gte=1, cast=int),
Expand Down Expand Up @@ -320,11 +334,17 @@ def _normalize_config(cfg) -> dict:
f"feature_fields cannot include the target field or 'time': {reserved}"
)

model = str(cfg.get("model") or DEFAULT_MODEL_SLUG).strip()
model = str(cfg.get("model") or "").strip()
if not model:
raise ConfigError(
"`model` is required: the Nori gateway slug to call, for example "
f"{EXAMPLE_MODEL_SLUG!r}. There is no default, because the slug selects a priced "
f"model. The current slugs are listed at {MODEL_LIST_URL}."
)
if model in RETIRED_MODEL_SLUGS:
raise ConfigError(
f"model slug {model!r} is retired and no longer routes. Use 'synthefy/nori-30m' "
f"(~29M parameters) or 'synthefy/nori-6m' (cheaper, faster cold start)."
f"model slug {model!r} is retired and no longer routes. Use {EXAMPLE_MODEL_SLUG!r} "
f"or another current slug, listed at {MODEL_LIST_URL}."
)

try:
Expand Down Expand Up @@ -648,7 +668,7 @@ def _gateway_url() -> str:

def _get_api_key(request_headers=None) -> str:
"""Resolve the gateway key: a non-empty X-Nori-Api-Key header wins (HTTP trigger), else the
NORI_API_KEY environment variable.
SYNTHEFY_NORI_API_KEY environment variable.

An empty or non-string header value falls through to the environment rather than winning it: a
header sent with no value used to suppress the fallback and send `Api-Key ` with no key at all.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,11 @@ feature_fields = ["temp", "humidity"]
# Filter to a single series. A run must resolve to one series; the plugin fails if the window
# holds more. Default: no filter.
#tags = { site = "A" }
# Nori gateway slug. "synthefy/nori-30m" (default, ~29M parameters) or "synthefy/nori-6m"
# (cheaper, faster cold start). The bare "synthefy/nori" slug is retired.
#model = "synthefy/nori-30m"
# Nori gateway slug. REQUIRED: there is no default, because the slug selects a priced model.
# The current models and their slugs are
# listed at https://docs.synthefy.com/nori/quickstart#models
# The bare "synthefy/nori" slug is retired.
model = "synthefy/nori-30m"
# Measurement to write predictions to. Default: "<measurement>_regressed".
#output_measurement = "sensors_regressed"
# Write predictions to another database. Default: the trigger's database.
Expand Down
Loading