From d09da6ded6b640bddc9bce056f1903413fd38a17 Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Thu, 21 May 2026 09:09:34 +0200 Subject: [PATCH 01/15] tmp --- AGENTS.md | 1 + 1 file changed, 1 insertion(+) create mode 120000 AGENTS.md diff --git a/AGENTS.md b/AGENTS.md new file mode 120000 index 0000000000..ac55cbdc9c --- /dev/null +++ b/AGENTS.md @@ -0,0 +1 @@ +.claude/CLAUDE.md \ No newline at end of file From 715d7fc8701ff4960c89a1012b9b8bc53ed24d7f Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Mon, 14 Sep 2026 06:56:48 +0200 Subject: [PATCH 02/15] [FSTORE-2116] Batch inference for future prediction timestamps: get_batch_data(entries, prediction_times) https://hopsworks.atlassian.net/browse/FSTORE-2116 A feature view is anchored on its root feature group, so get_batch_data(start_time, end_time) can only return rows for times that feature group has already observed. Inference over the future has no such rows: a view joining air quality observations to weather observations and forecasts returns nothing for tomorrow, even though the weather feature group holds the forecast. Users work around it by reading the forecast feature group directly, which bypasses the view's joins, its feature selection and its transformations. Batch inference now accepts the rows to predict for instead of deriving them from the root feature group. get_batch_data(entries, prediction_times) takes a dataframe of entities and a set of future timestamps, forms their cross product as an inference spine, and anchors the query on that spine. Every other feature group is looked up ASOF each prediction time, so a forecast row dated in the future is returned when it is the most recent row at or before that time. The client contributes the rows and the backend authors the SQL, which DuckDB renders as an ASOF LEFT JOIN and Spark as a ranked window. Offline feature groups only; online serving is unchanged. The guide explains why a time range returns nothing for the future, then covers the three ways to build a set of prediction times, what `entries` may carry, how an as-of lookup resolves and what it returns when it matches nothing, and why `max_feature_age` is what makes a stale carried-forward value visible. It states the parts that surprise people: the returned event time is the prediction time asked for rather than the event time of the matched row, keys and event time default to included for this call where they default to excluded for the others, and ambiguous key columns come back fully qualified. Limits, engine coverage and the refused query shapes are listed rather than left to be discovered. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01F22onm6M6KvU686d5imQw5 --- .../user_guides/fs/feature_view/batch-data.md | 6 + .../fs/feature_view/future-batch-data.md | 162 ++++++++++++++++++ mkdocs.yml | 1 + 3 files changed, 169 insertions(+) create mode 100644 docs/user_guides/fs/feature_view/future-batch-data.md diff --git a/docs/user_guides/fs/feature_view/batch-data.md b/docs/user_guides/fs/feature_view/batch-data.md index 2f6db323b4..bb58ad3986 100644 --- a/docs/user_guides/fs/feature_view/batch-data.md +++ b/docs/user_guides/fs/feature_view/batch-data.md @@ -21,6 +21,12 @@ The resultant DataFrame (or batch-scoring DataFrame) can then be fed to models t Dataset ds = featureView.getBatchData("20220620", "20220627") ``` +## Batch data for timestamps in the future + +A time range can only return rows the root feature group has already observed, so it returns nothing for the future. +To score timestamps that have not happened yet, pass the entities and the prediction times instead of a range. +See [Batch data for future timestamps][batch-data-for-future-timestamps]. + ## Retrieve batch data with primary keys and event time For certain use cases, e.g., time series models, the input data needs to be sorted according to the primary key(s) and event time combination. diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md new file mode 100644 index 0000000000..8ba3a63a1e --- /dev/null +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -0,0 +1,162 @@ +# Batch data for future timestamps + +## Why a time range is not enough + +`get_batch_data(start_time, end_time)` filters the feature view's root feature group, the left-most feature group of its query, and joins the rest onto the rows it finds. +That works whenever the root feature group has an observation for every entity and time you want to score. +It returns nothing for the future, because the future has no observations yet. + +An air quality feature view is the common shape. +It joins `air_quality` observations to a `weather` feature group that holds both observations and a forecast. +`get_batch_data(start_time=tomorrow, end_time=tomorrow + 7 days)` returns zero rows, even though `weather` holds all seven forecast days, because `air_quality` holds none of them. +The usual workaround is to read the forecast feature group directly, which loses the feature view's joins, its feature selection and its transformations. + +Instead, tell the feature view which rows you want to predict for. +`entries` is the set of entities and `prediction_times` is the set of timestamps. +Their cross product replaces the root feature group as the anchor of the query, and every feature group is looked up as of each prediction time. + +## Retrieving batch data for future timestamps + +```python +import datetime + +import pandas as pd +from hsfs.constructor.prediction_times import PredictionTimes + +tomorrow = datetime.date.today() + datetime.timedelta(days=1) + +batch_data = feature_view.get_batch_data( + entries=pd.DataFrame( + [{"country": "sweden", "city": "stockholm", "street": "sveavagen"}] + ), + prediction_times=PredictionTimes.every( + "daily", offset="00:00", start=tomorrow, count=7 + ), +) +``` + +The result has one row per entity per prediction time, so the example returns seven rows. +Rows come back in the order of `entries`, and within an entity in prediction-time order, so the frame can be handed to a model and its predictions joined back positionally. + +Every feature is taken from the most recent row at or before its prediction time. +For a forecast row dated in the future, that is the forecast for that day. +A feature group with no matching row contributes `NULL` rather than removing the row, the same as any left join, so an entity the feature store has never seen still comes back with the features that do resolve. + +## Choosing the entities + +`entries` accepts a pandas or polars DataFrame, or a list of dictionaries. +Its columns may be: + +- the feature view's required serving keys, which identify the entity; +- any column of the root feature group, which is then used as supplied instead of being looked up. + +The second kind is the same idea as a passed feature in an online deployment. +If the root feature group holds a `pm25` column and `entries` carries one, the value you passed is returned and no lookup is made for it. + +Omitting some serving keys is allowed and warns, because a model may be able to infer what is missing. +Omitting all of them is an error: nothing in the feature view can then be looked up, which is a mistake rather than an empty result. +A column that matches neither a serving key nor a root feature is an error naming the columns that are accepted. + +## Choosing the timestamps + +`PredictionTimes` builds the set of timestamps three ways. + +```python +from hsfs.constructor.prediction_times import PredictionTimes + +# A named interval with an offset. Intervals: hourly, daily, weekly, monthly. +PredictionTimes.every("daily", offset="08:00", start=tomorrow, count=7) +PredictionTimes.every("weekly", offset="mon:09:30", start=tomorrow, count=4) + +# A cron expression, in the five-field Vixie dialect, where 0 and 7 are both Sunday. +PredictionTimes.cron("0 8 * * 1-5", start=tomorrow, count=10) + +# An explicit list, for a schedule no rule describes. +PredictionTimes.of([datetime.datetime(2026, 3, 1, 8, 0)]) +``` + +A plain list of timestamps is accepted wherever `PredictionTimes` is, so `prediction_times=[t1, t2]` is shorthand for `PredictionTimes.of([t1, t2])`. + +Times are local, and daylight saving is resolved the way a scheduler resolves it. +A local time that does not exist on the day the clocks go forward is skipped. +A local time that occurs twice on the day they go back is taken at its first occurrence. + +!!! note + The prediction time in the returned frame is the time you asked for, not the event time of the row that matched it. + A prediction time of 08:00 matching a forecast row written at 00:00 comes back as 08:00. + +## Bounding how stale a feature may be + +An as-of lookup carries the last value forward for ever. +If a forecast is missing for one day, that day silently inherits the previous day's weather, and the frame gives no sign of it. + +`max_feature_age` bounds how old a matched row may be, relative to the prediction time. +A row older than the bound is returned as `NULL`, so the gap is visible to you and to the model. + +```python +batch_data = feature_view.get_batch_data( + entries=entries, + prediction_times=PredictionTimes.every( + "daily", offset="00:00", start=tomorrow, count=7 + ), + # Per feature group, by name. + max_feature_age={"weather": datetime.timedelta(days=1)}, +) +``` + +A single `timedelta` bounds every feature group instead of one. +A name that is not a feature group of the feature view is an error rather than a bound that applies to nothing. + +## Keys and event time in the result + +For a normal `get_batch_data` call, `primary_key` and `event_time` default to `False`. +For a call with `entries` and `prediction_times` they default to `True`, because without the keys and the prediction time the frame does not say which row belongs to which entity or day. +Pass `False` explicitly to leave them out, which is what you want when the model consumes the frame directly. + +```python +batch_data = feature_view.get_batch_data( + entries=entries, + prediction_times=prediction_times, + primary_key=False, + event_time=False, +) +``` + +!!! note + When feature groups in the view share a column name, key columns come back fully qualified as `___`. + This is how `get_batch_data` has always named ambiguous key columns; it is not specific to this call. + +## Limits and performance + +Offline feature groups only. +Online serving through `get_feature_vector` is unchanged and does not take prediction times. + +Both engines are supported. +The Hopsworks Query Service renders each lookup as a DuckDB `ASOF LEFT JOIN`, and Spark renders it as a ranked window over the same rows. +Both return the same frame. + +The size of the cross product of `entries` and `prediction_times` is bounded by cluster limits, which an administrator sets: + +| Variable | Bounds | +| --- | --- | +| `featurestore_asof_spine_max_rows` | Rows, meaning entities multiplied by prediction times | +| `featurestore_asof_spine_max_bytes` | The serialized size of those rows | +| `featurestore_asof_spine_max_columns` | Columns in `entries` | +| `featurestore_asof_spine_max_horizon_days` | How far ahead a schedule may expand | + +A request over any of them is refused before it runs, with the limit named. + +Without a `lookback`, each feature group is scanned from its first row up to the last prediction time. +The upper bound excludes forecast rows beyond your horizon, but it does not bound history. +On a large feature group, `lookback` is what bounds the work, and `max_feature_age` bounds how many candidate rows each lookup considers. + +## Restrictions + +- The feature view's joins must be `LEFT` or `INNER`. + A `RIGHT` or `FULL` join keeps source rows that have no prediction time to align to. +- Feature groups joined through another feature group are not supported. +- Every feature group in the view needs an event time, because an as-of lookup has nothing to order on without one. +- `entries` and `prediction_times` cannot be combined with `start_time` and `end_time`. + The prediction times define the time axis. +- A feature view created with a spine group uses `spine=` instead; the two cannot be combined. +- A filter on a column the entities supply is refused, because it would drop rows you asked to predict for. diff --git a/mkdocs.yml b/mkdocs.yml index 68e3f7aec6..d1c3fb824e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -102,6 +102,7 @@ nav: - Overview: user_guides/fs/feature_view/overview.md - Training data: user_guides/fs/feature_view/training-data.md - Batch data: user_guides/fs/feature_view/batch-data.md + - Batch data for future timestamps: user_guides/fs/feature_view/future-batch-data.md - Feature vectors: user_guides/fs/feature_view/feature-vectors.md - Feature server: user_guides/fs/feature_view/feature-server.md - Query: user_guides/fs/feature_view/query.md From 530cf9bc43016c9d5d6f52a61fa1b895a291db61 Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Tue, 15 Sep 2026 17:47:58 +0200 Subject: [PATCH 03/15] [FSTORE-2116] Batch inference for future prediction timestamps: get_batch_data(entries, prediction_times) https://hopsworks.atlassian.net/browse/FSTORE-2116 Rename the new batch-inference parameter from entries to serving_keys. The feature store already calls these values the view's serving keys, and the intent is for serving_keys to be the name across the API rather than entry or entries. The parameter is new and unreleased, so this is a plain rename with no alias and nothing to deprecate. The frame may still carry columns that are not serving keys: any column of the root feature group is taken as supplied instead of being looked up, which is the batch analogue of a passed feature. The documentation and the parameter's own docstring say so, so the narrower name does not hide it. The user guide names the parameter in prose, in every example and in the limits table, so all of them follow the client. The section on what the frame may carry already said that a root feature group column is taken as supplied, which is what keeps the narrower name honest. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01F22onm6M6KvU686d5imQw5 --- .../fs/feature_view/future-batch-data.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index 8ba3a63a1e..9b54498b27 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -12,7 +12,7 @@ It joins `air_quality` observations to a `weather` feature group that holds both The usual workaround is to read the forecast feature group directly, which loses the feature view's joins, its feature selection and its transformations. Instead, tell the feature view which rows you want to predict for. -`entries` is the set of entities and `prediction_times` is the set of timestamps. +`serving_keys` is the set of entities and `prediction_times` is the set of timestamps. Their cross product replaces the root feature group as the anchor of the query, and every feature group is looked up as of each prediction time. ## Retrieving batch data for future timestamps @@ -26,7 +26,7 @@ from hsfs.constructor.prediction_times import PredictionTimes tomorrow = datetime.date.today() + datetime.timedelta(days=1) batch_data = feature_view.get_batch_data( - entries=pd.DataFrame( + serving_keys=pd.DataFrame( [{"country": "sweden", "city": "stockholm", "street": "sveavagen"}] ), prediction_times=PredictionTimes.every( @@ -36,7 +36,7 @@ batch_data = feature_view.get_batch_data( ``` The result has one row per entity per prediction time, so the example returns seven rows. -Rows come back in the order of `entries`, and within an entity in prediction-time order, so the frame can be handed to a model and its predictions joined back positionally. +Rows come back in the order of `serving_keys`, and within an entity in prediction-time order, so the frame can be handed to a model and its predictions joined back positionally. Every feature is taken from the most recent row at or before its prediction time. For a forecast row dated in the future, that is the forecast for that day. @@ -44,14 +44,14 @@ A feature group with no matching row contributes `NULL` rather than removing the ## Choosing the entities -`entries` accepts a pandas or polars DataFrame, or a list of dictionaries. +`serving_keys` accepts a pandas or polars DataFrame, or a list of dictionaries. Its columns may be: - the feature view's required serving keys, which identify the entity; - any column of the root feature group, which is then used as supplied instead of being looked up. The second kind is the same idea as a passed feature in an online deployment. -If the root feature group holds a `pm25` column and `entries` carries one, the value you passed is returned and no lookup is made for it. +If the root feature group holds a `pm25` column and `serving_keys` carries one, the value you passed is returned and no lookup is made for it. Omitting some serving keys is allowed and warns, because a model may be able to infer what is missing. Omitting all of them is an error: nothing in the feature view can then be looked up, which is a mistake rather than an empty result. @@ -95,7 +95,7 @@ A row older than the bound is returned as `NULL`, so the gap is visible to you a ```python batch_data = feature_view.get_batch_data( - entries=entries, + serving_keys=serving_keys, prediction_times=PredictionTimes.every( "daily", offset="00:00", start=tomorrow, count=7 ), @@ -110,12 +110,12 @@ A name that is not a feature group of the feature view is an error rather than a ## Keys and event time in the result For a normal `get_batch_data` call, `primary_key` and `event_time` default to `False`. -For a call with `entries` and `prediction_times` they default to `True`, because without the keys and the prediction time the frame does not say which row belongs to which entity or day. +For a call with `serving_keys` and `prediction_times` they default to `True`, because without the keys and the prediction time the frame does not say which row belongs to which entity or day. Pass `False` explicitly to leave them out, which is what you want when the model consumes the frame directly. ```python batch_data = feature_view.get_batch_data( - entries=entries, + serving_keys=serving_keys, prediction_times=prediction_times, primary_key=False, event_time=False, @@ -135,13 +135,13 @@ Both engines are supported. The Hopsworks Query Service renders each lookup as a DuckDB `ASOF LEFT JOIN`, and Spark renders it as a ranked window over the same rows. Both return the same frame. -The size of the cross product of `entries` and `prediction_times` is bounded by cluster limits, which an administrator sets: +The size of the cross product of `serving_keys` and `prediction_times` is bounded by cluster limits, which an administrator sets: | Variable | Bounds | | --- | --- | | `featurestore_asof_spine_max_rows` | Rows, meaning entities multiplied by prediction times | | `featurestore_asof_spine_max_bytes` | The serialized size of those rows | -| `featurestore_asof_spine_max_columns` | Columns in `entries` | +| `featurestore_asof_spine_max_columns` | Columns in `serving_keys` | | `featurestore_asof_spine_max_horizon_days` | How far ahead a schedule may expand | A request over any of them is refused before it runs, with the limit named. @@ -156,7 +156,7 @@ On a large feature group, `lookback` is what bounds the work, and `max_feature_a A `RIGHT` or `FULL` join keeps source rows that have no prediction time to align to. - Feature groups joined through another feature group are not supported. - Every feature group in the view needs an event time, because an as-of lookup has nothing to order on without one. -- `entries` and `prediction_times` cannot be combined with `start_time` and `end_time`. +- `serving_keys` and `prediction_times` cannot be combined with `start_time` and `end_time`. The prediction times define the time axis. - A feature view created with a spine group uses `spine=` instead; the two cannot be combined. - A filter on a column the entities supply is refused, because it would drop rows you asked to predict for. From b14bd295c72794e156dd2b17e137685768c2ad7d Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Tue, 15 Sep 2026 19:20:51 +0200 Subject: [PATCH 04/15] [FSTORE-2116] Batch inference for future prediction timestamps: get_batch_data(serving_keys, prediction_times) https://hopsworks.atlassian.net/browse/FSTORE-2116 Accept serving_keys on the online retrieval methods too, and keep entry working as a deprecated alias. The values that identify which rows to serve are called serving keys everywhere else in the product, including the feature view's own serving_keys property and the batch inference parameter this branch adds, so the online methods were the last place naming them something different. Filed as FSTORE-2118 and folded in here rather than shipped separately, so the API lands consistent in one release instead of disagreeing with itself for one. Every feature vector example moves to the new name so readers learn the preferred spelling, and a note records that entry still works, warns, and will be removed. The agent deployment guide keeps its own unrelated entry argument. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01F22onm6M6KvU686d5imQw5 --- .../on_demand_transformations.md | 16 +++--- .../fs/feature_view/feature-vectors.md | 50 +++++++++++-------- .../model-dependent-transformations.md | 6 +-- .../fs/transformation_functions.md | 2 +- 4 files changed, 41 insertions(+), 33 deletions(-) diff --git a/docs/user_guides/fs/feature_group/on_demand_transformations.md b/docs/user_guides/fs/feature_group/on_demand_transformations.md index 63dde49cc7..3300234323 100644 --- a/docs/user_guides/fs/feature_group/on_demand_transformations.md +++ b/docs/user_guides/fs/feature_group/on_demand_transformations.md @@ -128,7 +128,7 @@ The on-demand features in the feature vector can be computed using real-time dat ```python feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, + serving_keys={"id": 1}, request_parameter={ "transaction_time": datetime(2022, 12, 28, 23, 55, 59), "current_time": datetime.now(), @@ -147,7 +147,7 @@ The `request_parameter` in this case, can be a list of dictionaries that specifi ```python # Specify unique request parameters for each serving key. feature_vector = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], + serving_keys=[{"id": 1}, {"id": 2}], request_parameter=[ { "transaction_time": datetime(2022, 12, 28, 23, 55, 59), @@ -162,7 +162,7 @@ The `request_parameter` in this case, can be a list of dictionaries that specifi # Specify common request parameters for all serving key. feature_vector = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], + serving_keys=[{"id": 1}, {"id": 2}], request_parameter={ "transaction_time": datetime(2022, 12, 28, 23, 55, 59), "current_time": datetime.now(), @@ -180,10 +180,10 @@ To achieve this, set the parameters `transform` and `on_demand_features` to `Fa ```python untransformed_feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, transform=False, on_demand_features=False + serving_keys={"id": 1}, transform=False, on_demand_features=False ) untransformed_feature_vectors = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False + serving_keys=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False ) ``` @@ -201,7 +201,7 @@ The `request_parameter` in this case, can be a list of dictionaries that specifi ```python # Specify request parameters for each serving key. untransformed_feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, transform=False, on_demand_features=False + serving_keys={"id": 1}, transform=False, on_demand_features=False ) # re-compute and add on-demand features to the feature vector @@ -218,7 +218,7 @@ The `request_parameter` in this case, can be a list of dictionaries that specifi # Specify request parameters for each serving key. untransformed_feature_vectors = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False + serving_keys=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False ) # re-compute and add on-demand features to the feature vectors - Specify unique request parameter for each feature vector @@ -259,7 +259,7 @@ On-demand transformation functions can also be accessed and executed as normal f ```python # Specify request parameters for each serving key. feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, + serving_keys={"id": 1}, transform=False, on_demand_features=False, return_type="pandas", diff --git a/docs/user_guides/fs/feature_view/feature-vectors.md b/docs/user_guides/fs/feature_view/feature-vectors.md index 125e3615b5..3a2fff6606 100644 --- a/docs/user_guides/fs/feature_view/feature-vectors.md +++ b/docs/user_guides/fs/feature_view/feature-vectors.md @@ -15,19 +15,27 @@ If you need to get more familiar with the concept of feature vectors, you can re You can get back feature vectors from either python or java client by providing the primary key value(s) for the feature view. Note that filters defined in feature view and training data will not be applied when feature vectors are returned. -If you need to retrieve a complete value of feature vectors without missing values, the required `entry` are [FeatureView.primary_keys][hsfs.feature_view.FeatureView.primary_keys]. -Alternative, you can provide the primary key of the feature groups as the key of the entry. -It is also possible to provide a subset of the entry, which will be discussed [below](#partial-feature-retrieval). +If you need to retrieve a complete value of feature vectors without missing values, the required `serving_keys` are [FeatureView.primary_keys][hsfs.feature_view.FeatureView.primary_keys]. +Alternative, you can provide the primary key of the feature groups as the key of the serving keys. +It is also possible to provide a subset of the serving keys, which will be discussed [below](#partial-feature-retrieval). + +!!! note "`entry` was renamed to `serving_keys`" + The argument used to be called `entry`, on `get_feature_vector`, `get_feature_vectors`, + `get_inference_helper` and `get_inference_helpers`. + `entry` still works and takes the same value, but it emits a `DeprecationWarning` and will + be removed in a future release. + Passing both names in one call is an error. + Positional calls such as `get_feature_vector({"pk1": 1})` are unaffected. === "Python" ```python # get a single vector - feature_view.get_feature_vector(entry={"pk1": 1, "pk2": 2}) + feature_view.get_feature_vector(serving_keys={"pk1": 1, "pk2": 2}) # get multiple vectors feature_view.get_feature_vectors( - entry=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}] + serving_keys=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}] ) ``` @@ -47,7 +55,7 @@ It is also possible to provide a subset of the entry, which will be discussed [b featureView.getFeatureVectors(Lists.newArrayList(entry1, entry2)); ``` -### Required entry +### Required serving keys Starting from python client v3.4, you can specify different values for the primary key of the same name which exists in multiple feature groups but are not joint by the same name. The table below summarises the value of `primary_keys` in different settings. @@ -89,7 +97,7 @@ Take the above example assuming the feature view consists of two joined feature ```python # get a single vector - feature_view.get_feature_vector(entry={"pk1": 1, "pk2": 2}) + feature_view.get_feature_vector(serving_keys={"pk1": 1, "pk2": 2}) ``` === "Java" @@ -111,7 +119,7 @@ When retrieving a batch of vectors, the behaviour is slightly different. ```python # get multiple vectors feature_view.get_feature_vectors( - entry=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}] + serving_keys=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}] ) ``` @@ -137,18 +145,18 @@ If you are aware of missing features, you can use the [*passed features*](#passe ### Partial feature retrieval If your model can handle missing value or if you want to impute the missing value, you can get back feature vectors with partial values using python client starting from version 3.4 (Note that this does not apply to java client.). -In the example below, let's say you join 2 feature groups by `fg1.join(fg2, left_on=["pk1"], right_on=["pk2"])`, required keys of the `entry` are `pk1` and `pk2`. +In the example below, let's say you join 2 feature groups by `fg1.join(fg2, left_on=["pk1"], right_on=["pk2"])`, required keys of `serving_keys` are `pk1` and `pk2`. If `pk2` is not provided, this returns feature values from the first feature group and null values from the second feature group when using the option `allow_missing=True`, otherwise it raises exception. === "Python" ```python # get a single vector with - feature_view.get_feature_vector(entry={"pk1": 1}, allow_missing=True) + feature_view.get_feature_vector(serving_keys={"pk1": 1}, allow_missing=True) # get multiple vectors feature_view.get_feature_vectors( - entry=[ + serving_keys=[ {"pk1": 1}, {"pk1": 3}, ], @@ -184,12 +192,12 @@ Please note that passed features is only available in the python client but not ```python # get a single vector feature_view.get_feature_vector( - entry={"pk1": 1, "pk2": 2}, passed_features={"feature_a": "value_a"} + serving_keys={"pk1": 1, "pk2": 2}, passed_features={"feature_a": "value_a"} ) # get multiple vectors feature_view.get_feature_vectors( - entry=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}], + serving_keys=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}], passed_features=[ {"feature_a": "value_a1"}, {"feature_a": "value_a2"}, @@ -210,7 +218,7 @@ In this second case, you do not have to provide the primary key value for that f # in this case feature_b and feature_c feature_view.get_feature_vector( - entry={"pk1": 1}, + serving_keys={"pk1": 1}, passed_features={ "feature_a": "value_a", "feature_b": "value_b", @@ -231,12 +239,12 @@ However, you can retrieve the untransformed feature vectors without applying mod ```python # Fetching untransformed feature vector. untransformed_feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, transform=False + serving_keys={"id": 1}, transform=False ) # Fetching untransformed feature vectors. untransformed_feature_vectors = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], transform=False + serving_keys=[{"id": 1}, {"id": 2}], transform=False ) ``` @@ -250,10 +258,10 @@ To achieve this, set the parameters `transform` and `on_demand_features` to `Fa ```python untransformed_feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, transform=False, on_demand_features=False + serving_keys={"id": 1}, transform=False, on_demand_features=False ) untransformed_feature_vectors = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False + serving_keys=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False ) ``` @@ -267,7 +275,7 @@ After [defining a transformation function using a context variable](../transform ```python # Passing context variable to IN-MEMORY Training Dataset. batch_data = feature_view.get_feature_vectors( - entry=[{"pk1": 1}], transformation_context={"context_parameter": 10} + serving_keys=[{"pk1": 1}], transformation_context={"context_parameter": 10} ) ``` @@ -320,12 +328,12 @@ my_feature_view.init_serving( # this will fetch a feature vector via REST try: my_feature_view.get_feature_vector( - entry={"pk1": 1, "pk2": 2}, + serving_keys={"pk1": 1, "pk2": 2}, ) except TimeoutException: # if the REST client times out, the SQL client will be used my_feature_view.get_feature_vector( - entry={"pk1": 1, "pk2": 2}, force_sql=True + serving_keys={"pk1": 1, "pk2": 2}, force_sql=True ) ``` diff --git a/docs/user_guides/fs/feature_view/model-dependent-transformations.md b/docs/user_guides/fs/feature_view/model-dependent-transformations.md index 61d980c660..6e7e73352d 100644 --- a/docs/user_guides/fs/feature_view/model-dependent-transformations.md +++ b/docs/user_guides/fs/feature_view/model-dependent-transformations.md @@ -146,7 +146,7 @@ Model-dependent transformation functions can also be manually applied to a featu # Get untransformed feature Vector feature_vector = fv.get_feature_vector( - entry={"index": 10}, transform=False, return_type="pandas" + serving_keys={"index": 10}, transform=False, return_type="pandas" ) # Apply Model Dependent transformations @@ -164,12 +164,12 @@ To achieve this, set the `transform` parameter to False. ```python # Fetching untransformed feature vector. untransformed_feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, transform=False + serving_keys={"id": 1}, transform=False ) # Fetching untransformed feature vectors. untransformed_feature_vectors = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], transform=False + serving_keys=[{"id": 1}, {"id": 2}], transform=False ) # Fetching untransformed batch data. diff --git a/docs/user_guides/fs/transformation_functions.md b/docs/user_guides/fs/transformation_functions.md index 25047b227b..73113409b4 100644 --- a/docs/user_guides/fs/transformation_functions.md +++ b/docs/user_guides/fs/transformation_functions.md @@ -424,7 +424,7 @@ For online serving, spawning the worker pool during the first request would add fv.init_serving(training_dataset_version=1, n_processes=2) # Served using the pool of two workers spawned at init time. - vector = fv.get_feature_vector(entry={"id": 1}) + vector = fv.get_feature_vector(serving_keys={"id": 1}) ``` The worker pool start method defaults to `fork` on Linux and `spawn` on macOS and Windows. Set the `HOPSWORKS_TF_POOL_START_METHOD` environment variable to `fork`, `forkserver`, or `spawn` to override it. From 516ad18a503e29efb7eddd238349ecfd7fee2b7d Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Tue, 15 Sep 2026 20:56:22 +0200 Subject: [PATCH 05/15] [FSTORE-2116] Batch inference for future prediction timestamps: get_batch_data(serving_keys, prediction_times) https://hopsworks.atlassian.net/browse/FSTORE-2116 Let the spine carry columns the feature view does not define, so the same mechanism that anchors batch inference on caller-supplied rows can anchor training data on a labels dataframe. A spine group already allows this, but only for a view created with one as its left side; the inference spine is the late-bound form, available to any view at read time. The guide gains a section on building training data from the same rows, covering the two differences from a batch read, the relationship to spine groups, and the note that a materialized dataset built this way cannot be regenerated from its metadata. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01F22onm6M6KvU686d5imQw5 --- .../fs/feature_view/future-batch-data.md | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index 9b54498b27..aa9c98b1fb 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -126,6 +126,33 @@ batch_data = feature_view.get_batch_data( When feature groups in the view share a column name, key columns come back fully qualified as `___`. This is how `get_batch_data` has always named ambiguous key columns; it is not specific to this call. +## Training data from the same rows + +The same mechanism builds training data. Pass `serving_keys` to `training_data`, +`train_test_split`, `train_validation_test_split` or any of the `create_*` methods, and the +query is anchored on your rows instead of on the root feature group. + +```python +train_x, test_x, train_y, test_y = feature_view.train_test_split( + test_size=0.2, + serving_keys=labels, # keys, an event time per row, and the label +) +``` + +Two differences from a batch read. The frame supplies the times itself, one per row under the +event time column, so there is no `prediction_times`: a training row is one entity at one +moment, not an entity scored repeatedly. And columns the feature view does not define are +carried through to the output untouched, which is how the label rides along. A batch read stays +strict about unknown columns, because inference has no labels and a mistyped column there is +worth catching. + +This is what a spine group does, without having had to create the feature view with one. +`serving_keys` and `spine` both replace the left side of the query, so passing both is an error. + +!!! note "Materialized training datasets built this way are not reproducible" + A `create_*` call records the query, not your dataframe, so the dataset cannot be rebuilt + from its metadata alone. Keep the frame if you need to regenerate it. + ## Limits and performance Offline feature groups only. From 3b8259bd1da65c7ab67846d031c587491ed1fc8c Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Tue, 15 Sep 2026 21:47:18 +0200 Subject: [PATCH 06/15] [FSTORE-2116] Batch inference for future prediction timestamps: get_batch_data(spine_df, prediction_times) https://hopsworks.atlassian.net/browse/FSTORE-2116 Rename the dataframe parameter from serving_keys to spine_df, on get_batch_data and on all six training-data methods. The frame is a spine: it replaces the left side of the query and may carry passed features and labels as well as keys, so naming it for the keys described only part of what it holds. The methods whose argument is a dict of keys keep serving_keys, because there the name is exact. get_feature_vector and get_feature_vectors take a dict and a list of dicts, and the two inference helper methods take the same, so none of them change. Follows the client. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01F22onm6M6KvU686d5imQw5 --- .../fs/feature_view/future-batch-data.md | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index aa9c98b1fb..2855c9a491 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -12,7 +12,7 @@ It joins `air_quality` observations to a `weather` feature group that holds both The usual workaround is to read the forecast feature group directly, which loses the feature view's joins, its feature selection and its transformations. Instead, tell the feature view which rows you want to predict for. -`serving_keys` is the set of entities and `prediction_times` is the set of timestamps. +`spine_df` is the set of entities and `prediction_times` is the set of timestamps. Their cross product replaces the root feature group as the anchor of the query, and every feature group is looked up as of each prediction time. ## Retrieving batch data for future timestamps @@ -26,7 +26,7 @@ from hsfs.constructor.prediction_times import PredictionTimes tomorrow = datetime.date.today() + datetime.timedelta(days=1) batch_data = feature_view.get_batch_data( - serving_keys=pd.DataFrame( + spine_df=pd.DataFrame( [{"country": "sweden", "city": "stockholm", "street": "sveavagen"}] ), prediction_times=PredictionTimes.every( @@ -36,7 +36,7 @@ batch_data = feature_view.get_batch_data( ``` The result has one row per entity per prediction time, so the example returns seven rows. -Rows come back in the order of `serving_keys`, and within an entity in prediction-time order, so the frame can be handed to a model and its predictions joined back positionally. +Rows come back in the order of `spine_df`, and within an entity in prediction-time order, so the frame can be handed to a model and its predictions joined back positionally. Every feature is taken from the most recent row at or before its prediction time. For a forecast row dated in the future, that is the forecast for that day. @@ -44,14 +44,14 @@ A feature group with no matching row contributes `NULL` rather than removing the ## Choosing the entities -`serving_keys` accepts a pandas or polars DataFrame, or a list of dictionaries. +`spine_df` accepts a pandas or polars DataFrame, or a list of dictionaries. Its columns may be: - the feature view's required serving keys, which identify the entity; - any column of the root feature group, which is then used as supplied instead of being looked up. The second kind is the same idea as a passed feature in an online deployment. -If the root feature group holds a `pm25` column and `serving_keys` carries one, the value you passed is returned and no lookup is made for it. +If the root feature group holds a `pm25` column and `spine_df` carries one, the value you passed is returned and no lookup is made for it. Omitting some serving keys is allowed and warns, because a model may be able to infer what is missing. Omitting all of them is an error: nothing in the feature view can then be looked up, which is a mistake rather than an empty result. @@ -95,7 +95,7 @@ A row older than the bound is returned as `NULL`, so the gap is visible to you a ```python batch_data = feature_view.get_batch_data( - serving_keys=serving_keys, + spine_df=spine_df, prediction_times=PredictionTimes.every( "daily", offset="00:00", start=tomorrow, count=7 ), @@ -110,12 +110,12 @@ A name that is not a feature group of the feature view is an error rather than a ## Keys and event time in the result For a normal `get_batch_data` call, `primary_key` and `event_time` default to `False`. -For a call with `serving_keys` and `prediction_times` they default to `True`, because without the keys and the prediction time the frame does not say which row belongs to which entity or day. +For a call with `spine_df` and `prediction_times` they default to `True`, because without the keys and the prediction time the frame does not say which row belongs to which entity or day. Pass `False` explicitly to leave them out, which is what you want when the model consumes the frame directly. ```python batch_data = feature_view.get_batch_data( - serving_keys=serving_keys, + spine_df=spine_df, prediction_times=prediction_times, primary_key=False, event_time=False, @@ -128,14 +128,14 @@ batch_data = feature_view.get_batch_data( ## Training data from the same rows -The same mechanism builds training data. Pass `serving_keys` to `training_data`, +The same mechanism builds training data. Pass `spine_df` to `training_data`, `train_test_split`, `train_validation_test_split` or any of the `create_*` methods, and the query is anchored on your rows instead of on the root feature group. ```python train_x, test_x, train_y, test_y = feature_view.train_test_split( test_size=0.2, - serving_keys=labels, # keys, an event time per row, and the label + spine_df=labels, # keys, an event time per row, and the label ) ``` @@ -147,7 +147,7 @@ strict about unknown columns, because inference has no labels and a mistyped col worth catching. This is what a spine group does, without having had to create the feature view with one. -`serving_keys` and `spine` both replace the left side of the query, so passing both is an error. +`spine_df` and `spine` both replace the left side of the query, so passing both is an error. !!! note "Materialized training datasets built this way are not reproducible" A `create_*` call records the query, not your dataframe, so the dataset cannot be rebuilt @@ -162,13 +162,13 @@ Both engines are supported. The Hopsworks Query Service renders each lookup as a DuckDB `ASOF LEFT JOIN`, and Spark renders it as a ranked window over the same rows. Both return the same frame. -The size of the cross product of `serving_keys` and `prediction_times` is bounded by cluster limits, which an administrator sets: +The size of the cross product of `spine_df` and `prediction_times` is bounded by cluster limits, which an administrator sets: | Variable | Bounds | | --- | --- | | `featurestore_asof_spine_max_rows` | Rows, meaning entities multiplied by prediction times | | `featurestore_asof_spine_max_bytes` | The serialized size of those rows | -| `featurestore_asof_spine_max_columns` | Columns in `serving_keys` | +| `featurestore_asof_spine_max_columns` | Columns in `spine_df` | | `featurestore_asof_spine_max_horizon_days` | How far ahead a schedule may expand | A request over any of them is refused before it runs, with the limit named. @@ -183,7 +183,7 @@ On a large feature group, `lookback` is what bounds the work, and `max_feature_a A `RIGHT` or `FULL` join keeps source rows that have no prediction time to align to. - Feature groups joined through another feature group are not supported. - Every feature group in the view needs an event time, because an as-of lookup has nothing to order on without one. -- `serving_keys` and `prediction_times` cannot be combined with `start_time` and `end_time`. +- `spine_df` and `prediction_times` cannot be combined with `start_time` and `end_time`. The prediction times define the time axis. - A feature view created with a spine group uses `spine=` instead; the two cannot be combined. - A filter on a column the entities supply is refused, because it would drop rows you asked to predict for. From ca5d14907660aac2b70e269ed47e69b6c5168eb0 Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Tue, 15 Sep 2026 23:10:15 +0200 Subject: [PATCH 07/15] [FSTORE-2116] Batch inference for future prediction timestamps: get_batch_data(spine_df, prediction_times) https://hopsworks.atlassian.net/browse/FSTORE-2116 Leave spine_df as the only spine-related argument of get_batch_data. Two parameters came off it, both because the API was saying the same thing in more than one way. prediction_times is gone, losslessly: the times were always expressible as a column of the frame, and the training-data methods already required them there, so the batch path was the odd one out. PredictionTimes remains, as the thing that builds a schedule from a cron expression, a named interval or a list, and it gains cross(). That matters, because the row order is a documented contract, entities as given and ascending in time within each; leaving callers to hand-roll the cross product would have made that contract theirs to get right. max_feature_age moved onto the feature view. It could not follow the times into the frame: it bounds how stale a lookup may be per feature group, which is a property of the lookup and not of a row. Deleting it would have removed the only defence against an as-of join carrying a stale value forward silently. As a property of the view it now applies to every read anchored on a spine, batch inference and training data alike, which is what it should always have done. The guide loses the cross-product parameter, gains cross() and the row order it guarantees, and says that the bound is a property of the view that applies to training reads too, with the reason: a training example built from a stale feature is worse than an inference row built from one, because the model learns from it. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01F22onm6M6KvU686d5imQw5 --- .../fs/feature_view/future-batch-data.md | 67 +++++++++++-------- 1 file changed, 40 insertions(+), 27 deletions(-) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index 2855c9a491..b67a6f3218 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -12,8 +12,8 @@ It joins `air_quality` observations to a `weather` feature group that holds both The usual workaround is to read the forecast feature group directly, which loses the feature view's joins, its feature selection and its transformations. Instead, tell the feature view which rows you want to predict for. -`spine_df` is the set of entities and `prediction_times` is the set of timestamps. -Their cross product replaces the root feature group as the anchor of the query, and every feature group is looked up as of each prediction time. +`spine_df` is the set of rows to predict for: one row per entity and moment, carrying the serving keys and the prediction time. +It replaces the root feature group as the anchor of the query, and every feature group is looked up as of each row's own time. ## Retrieving batch data for future timestamps @@ -25,13 +25,13 @@ from hsfs.constructor.prediction_times import PredictionTimes tomorrow = datetime.date.today() + datetime.timedelta(days=1) +entities = pd.DataFrame( + [{"country": "sweden", "city": "stockholm", "street": "sveavagen"}] +) +schedule = PredictionTimes.every("daily", offset="00:00", start=tomorrow, count=7) + batch_data = feature_view.get_batch_data( - spine_df=pd.DataFrame( - [{"country": "sweden", "city": "stockholm", "street": "sveavagen"}] - ), - prediction_times=PredictionTimes.every( - "daily", offset="00:00", start=tomorrow, count=7 - ), + spine_df=schedule.cross(entities, event_time="date"), ) ``` @@ -59,6 +59,10 @@ A column that matches neither a serving key nor a root feature is an error namin ## Choosing the timestamps +The frame carries one timestamp per row, under the root feature group's event time column. +Build it yourself if you already have the rows, or let `PredictionTimes` build a schedule and cross it with your entities. +`cross` is worth preferring over rolling your own: it fixes the row order, entities as given and ascending in time within each, which is the order the result comes back in. + `PredictionTimes` builds the set of timestamps three ways. ```python @@ -75,7 +79,6 @@ PredictionTimes.cron("0 8 * * 1-5", start=tomorrow, count=10) PredictionTimes.of([datetime.datetime(2026, 3, 1, 8, 0)]) ``` -A plain list of timestamps is accepted wherever `PredictionTimes` is, so `prediction_times=[t1, t2]` is shorthand for `PredictionTimes.of([t1, t2])`. Times are local, and daylight saving is resolved the way a scheduler resolves it. A local time that does not exist on the day the clocks go forward is skipped. @@ -94,29 +97,26 @@ If a forecast is missing for one day, that day silently inherits the previous da A row older than the bound is returned as `NULL`, so the gap is visible to you and to the model. ```python -batch_data = feature_view.get_batch_data( - spine_df=spine_df, - prediction_times=PredictionTimes.every( - "daily", offset="00:00", start=tomorrow, count=7 - ), - # Per feature group, by name. - max_feature_age={"weather": datetime.timedelta(days=1)}, -) +# Per feature group, by name. Set on the view, so it applies to every read anchored on a +# spine_df, batch inference and training data alike. +feature_view.max_feature_age = {"weather": datetime.timedelta(days=1)} + +batch_data = feature_view.get_batch_data(spine_df=spine) ``` -A single `timedelta` bounds every feature group instead of one. +A single `timedelta` bounds every feature group instead of one, and `"*"` is the catch-all key. +It is set on the view object rather than persisted with it, so set it again after `get_feature_view`. A name that is not a feature group of the feature view is an error rather than a bound that applies to nothing. ## Keys and event time in the result For a normal `get_batch_data` call, `primary_key` and `event_time` default to `False`. -For a call with `spine_df` and `prediction_times` they default to `True`, because without the keys and the prediction time the frame does not say which row belongs to which entity or day. +For a call with `spine_df` they default to `True`, because without the keys and the prediction time the frame does not say which row belongs to which entity or day. Pass `False` explicitly to leave them out, which is what you want when the model consumes the frame directly. ```python batch_data = feature_view.get_batch_data( - spine_df=spine_df, - prediction_times=prediction_times, + spine_df=spine, primary_key=False, event_time=False, ) @@ -139,13 +139,26 @@ train_x, test_x, train_y, test_y = feature_view.train_test_split( ) ``` -Two differences from a batch read. The frame supplies the times itself, one per row under the -event time column, so there is no `prediction_times`: a training row is one entity at one -moment, not an entity scored repeatedly. And columns the feature view does not define are +One difference from a batch read: columns the feature view does not define are carried through to the output untouched, which is how the label rides along. A batch read stays strict about unknown columns, because inference has no labels and a mistyped column there is worth catching. +`max_feature_age` applies here too, because it is set on the view rather than on the call. +A training example built from a feature that stopped being produced is the same silent +staleness as an inference row built from one, and it is worse: the model learns from it. + +```python +feature_view.max_feature_age = {"weather": datetime.timedelta(days=1)} + +# a row whose weather is older than a day now carries NULL rather than a stale value +train_x, test_x, train_y, test_y = feature_view.train_test_split( + test_size=0.2, spine_df=labels +) +``` + +The label is the caller's own column and is never nulled by the bound. + This is what a spine group does, without having had to create the feature view with one. `spine_df` and `spine` both replace the left side of the query, so passing both is an error. @@ -162,7 +175,7 @@ Both engines are supported. The Hopsworks Query Service renders each lookup as a DuckDB `ASOF LEFT JOIN`, and Spark renders it as a ranked window over the same rows. Both return the same frame. -The size of the cross product of `spine_df` and `prediction_times` is bounded by cluster limits, which an administrator sets: +The size of `spine_df` is bounded by cluster limits, which an administrator sets: | Variable | Bounds | | --- | --- | @@ -183,7 +196,7 @@ On a large feature group, `lookback` is what bounds the work, and `max_feature_a A `RIGHT` or `FULL` join keeps source rows that have no prediction time to align to. - Feature groups joined through another feature group are not supported. - Every feature group in the view needs an event time, because an as-of lookup has nothing to order on without one. -- `spine_df` and `prediction_times` cannot be combined with `start_time` and `end_time`. - The prediction times define the time axis. +- `spine_df` cannot be combined with `start_time` and `end_time`. + The frame's own timestamps define the time axis. - A feature view created with a spine group uses `spine=` instead; the two cannot be combined. - A filter on a column the entities supply is refused, because it would drop rows you asked to predict for. From 967cb42f1889f7bb4b44eacc52245b607b41f62a Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Wed, 16 Sep 2026 09:03:42 +0200 Subject: [PATCH 08/15] [FSTORE-2116] Batch inference and training data anchored on a caller-supplied spine_df https://hopsworks.atlassian.net/browse/FSTORE-2116 Fix max_feature_age at feature view creation and make it read-only after. Allowing it per call let a training set and an inference read be built with different bounds, which is training/serving skew of exactly the kind a feature view exists to prevent. A read-only setting is only honest if it survives a round trip, so it is persisted with the view rather than living on the object. The guide and the batch inference skill set the bound at creation and say why it cannot be changed later. The skill also gains the spine_df sections: a schedule crossed with entities for future prediction times, one row per entity at a single instant for latest values, the staleness bound, training data from the same rows, and what is refused. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01F22onm6M6KvU686d5imQw5 --- .../fs/feature_view/future-batch-data.md | 40 +++++++++++++++---- 1 file changed, 32 insertions(+), 8 deletions(-) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index b67a6f3218..97f555f13b 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -88,6 +88,26 @@ A local time that occurs twice on the day they go back is taken at its first occ The prediction time in the returned frame is the time you asked for, not the event time of the row that matched it. A prediction time of 08:00 matching a forecast row written at 00:00 comes back as 08:00. +## Latest feature values for every entity + +One row per entity, all at the same instant, is the offline equivalent of `get_feature_vectors`. +Capture the timestamp once so every entity is read at the same moment rather than each drifting. + +```python +import datetime + +now = datetime.datetime.now(datetime.timezone.utc) + +spine = pd.DataFrame({"entity_id": [1, 2, 3]}) +spine["event_time"] = now # named after the root feature group's event time column + +latest = feature_view.get_batch_data(spine_df=spine) +``` + +There is no implicit "as of now": the time is always in the frame. +A wall-clock default would make the same call return different rows on a re-run, and a training dataset materialized that way could never be reproduced. +Event times are kept to the millisecond, so sub-millisecond precision in the timestamp you pass is dropped rather than rejected. + ## Bounding how stale a feature may be An as-of lookup carries the last value forward for ever. @@ -97,15 +117,20 @@ If a forecast is missing for one day, that day silently inherits the previous da A row older than the bound is returned as `NULL`, so the gap is visible to you and to the model. ```python -# Per feature group, by name. Set on the view, so it applies to every read anchored on a -# spine_df, batch inference and training data alike. -feature_view.max_feature_age = {"weather": datetime.timedelta(days=1)} +# Set when the view is created, so it applies to every read anchored on a spine_df, +# batch inference and training data alike. +feature_view = fs.create_feature_view( + name="air_quality_fv", + query=query, + max_feature_age={"weather": datetime.timedelta(days=1)}, +) batch_data = feature_view.get_batch_data(spine_df=spine) ``` A single `timedelta` bounds every feature group instead of one, and `"*"` is the catch-all key. -It is set on the view object rather than persisted with it, so set it again after `get_feature_view`. +It is read-only after creation and stored with the view. +That is deliberate: if it could be changed per call, a training set and an inference read could be built with different bounds, which is the training/serving skew a feature view exists to prevent. A name that is not a feature group of the feature view is an error rather than a bound that applies to nothing. ## Keys and event time in the result @@ -144,14 +169,13 @@ carried through to the output untouched, which is how the label rides along. A b strict about unknown columns, because inference has no labels and a mistyped column there is worth catching. -`max_feature_age` applies here too, because it is set on the view rather than on the call. +`max_feature_age` applies here too, because it belongs to the view rather than to the call. A training example built from a feature that stopped being produced is the same silent staleness as an inference row built from one, and it is worse: the model learns from it. ```python -feature_view.max_feature_age = {"weather": datetime.timedelta(days=1)} - -# a row whose weather is older than a day now carries NULL rather than a stale value +# the view was created with max_feature_age={"weather": timedelta(days=1)}, so a row whose +# weather is older than a day carries NULL rather than a stale value train_x, test_x, train_y, test_y = feature_view.train_test_split( test_size=0.2, spine_df=labels ) From 2002691ec56e06ebc3ce9d873432555479a8e1b7 Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Wed, 16 Sep 2026 10:12:27 +0200 Subject: [PATCH 09/15] [FSTORE-2116] Batch inference and training data anchored on a caller-supplied spine_df https://hopsworks.atlassian.net/browse/FSTORE-2116 Document the feature-age bound as one value per feature view. It is a timedelta at creation and reads back as one from feature_view.max_feature_age, so the guide no longer describes a map keyed by feature group, a catch-all key, or the error for a name that matches no feature group. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01F22onm6M6KvU686d5imQw5 --- docs/user_guides/fs/feature_view/future-batch-data.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index 97f555f13b..6d237b482c 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -122,16 +122,16 @@ A row older than the bound is returned as `NULL`, so the gap is visible to you a feature_view = fs.create_feature_view( name="air_quality_fv", query=query, - max_feature_age={"weather": datetime.timedelta(days=1)}, + max_feature_age=datetime.timedelta(days=1), ) batch_data = feature_view.get_batch_data(spine_df=spine) ``` -A single `timedelta` bounds every feature group instead of one, and `"*"` is the catch-all key. +One bound covers the whole view: every feature group it reads is held to the same limit. It is read-only after creation and stored with the view. That is deliberate: if it could be changed per call, a training set and an inference read could be built with different bounds, which is the training/serving skew a feature view exists to prevent. -A name that is not a feature group of the feature view is an error rather than a bound that applies to nothing. +Read it back with `feature_view.max_feature_age`, which returns a `timedelta` or `None` when the view is unbounded. ## Keys and event time in the result @@ -174,8 +174,8 @@ A training example built from a feature that stopped being produced is the same staleness as an inference row built from one, and it is worse: the model learns from it. ```python -# the view was created with max_feature_age={"weather": timedelta(days=1)}, so a row whose -# weather is older than a day carries NULL rather than a stale value +# the view was created with max_feature_age=timedelta(days=1), so any feature whose newest +# row is older than a day carries NULL rather than a stale value train_x, test_x, train_y, test_y = feature_view.train_test_split( test_size=0.2, spine_df=labels ) From 2a14f9840b5e34d528d5c38d2eef9614130c2b17 Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Wed, 16 Sep 2026 11:42:01 +0200 Subject: [PATCH 10/15] [FSTORE-2116] Batch inference and training data anchored on a caller-supplied spine_df https://hopsworks.atlassian.net/browse/FSTORE-2116 Document reading the entity set off the feature view's root feature group with get_root_fg().read_primary_keys(), as the alternative to maintaining a list of entity ids by hand. Say plainly that the result is not a spine_df on its own, since it carries no time and is refused, and what it costs: the whole feature group's key columns are read to take the distinct rows, and only the root's keys are covered. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01F22onm6M6KvU686d5imQw5 --- .../fs/feature_view/future-batch-data.md | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index 6d237b482c..784702e230 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -104,6 +104,25 @@ spine["event_time"] = now # named after the root feature group's event time co latest = feature_view.get_batch_data(spine_df=spine) ``` +To cover every entity the feature store knows about rather than a list you maintain, read them +off the feature view's root feature group. +`get_root_fg()` returns the feature group the view is anchored on, and `read_primary_keys()` returns its distinct primary key values, one row per entity. + +```python +import datetime +from hsfs.constructor.prediction_times import PredictionTimes + +fg = feature_view.get_root_fg() +now = datetime.datetime.now(datetime.timezone.utc) + +spine = PredictionTimes.of([now]).cross(fg.read_primary_keys(), event_time=fg.event_time) +latest = feature_view.get_batch_data(spine_df=spine) +``` + +`read_primary_keys()` returns entities and no time, so it is not a `spine_df` on its own and passing it directly is refused. +Crossing it with one instant is what makes it one. +It reads the key columns of the whole feature group to take the distinct rows, so the cost scales with the feature group rather than with the number of entities, and it returns the root's keys only: a joined feature group keyed on something the root does not carry is not covered by it and comes back `NULL`. + There is no implicit "as of now": the time is always in the frame. A wall-clock default would make the same call return different rows on a re-run, and a training dataset materialized that way could never be reproduced. Event times are kept to the millisecond, so sub-millisecond precision in the timestamp you pass is dropped rather than rejected. From d72b2d9788d1f53bbb79a355cbb1502008364abb Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Wed, 16 Sep 2026 12:24:02 +0200 Subject: [PATCH 11/15] [FSTORE-2116] Batch inference and training data anchored on a caller-supplied spine_df https://hopsworks.atlassian.net/browse/FSTORE-2116 Say which frames spine_df accepts, now that a Spark DataFrame is one of them under the Spark engine, and what that costs: the rows are collected to the driver, and a Spark DataFrame has no row order, so the positional zip-back does not apply to one. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01F22onm6M6KvU686d5imQw5 --- docs/user_guides/fs/feature_view/future-batch-data.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index 784702e230..99a7b6e2a8 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -44,7 +44,10 @@ A feature group with no matching row contributes `NULL` rather than removing the ## Choosing the entities -`spine_df` accepts a pandas or polars DataFrame, or a list of dictionaries. +`spine_df` accepts a pandas or polars DataFrame, a list of dictionaries, or, under the Spark engine, a Spark DataFrame. +A Spark DataFrame is refused under the Python engine, which has no session to evaluate it. +A Spark spine is collected to the driver to be registered as a session temporary view, which is what the Spark spine path has always done, so size the frame to the entities you are scoring. +A Spark DataFrame also has no row order, so the positional zip-back below does not apply to one: join predictions back on the serving keys instead. Its columns may be: - the feature view's required serving keys, which identify the entity; From 749e76647200c30e9fb2fa3337e37641c53d3899 Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Wed, 16 Sep 2026 16:20:23 +0200 Subject: [PATCH 12/15] [FSTORE-2116] Batch inference and training data anchored on a caller-supplied spine_df https://hopsworks.atlassian.net/browse/FSTORE-2116 Apply a review of this change to itself. Thirteen findings, the two that matter being an unvalidated column name reaching SQL and a staleness bound the backend took from the request rather than from the feature view it is stored on. The spine's column names are now held to an identifier grammar in InferenceSpineResolver, alongside the table name, the Parquet basename and the types that were already checked there. A passthrough column has no feature in the store behind it, so its name arrived exactly as the caller wrote it and was rendered into a backtick identifier by both SQL renderings. Only flyingduck's own check stood between that and a scalar subquery on the signed path, which is sanitisation in the service that trusts the backend rather than in the one that builds the statement. max_feature_age is no longer sent with the read. The spine names its feature view instead, by featurestore id, name and version, and the resolver reads the bound off that view's row; the featurestore is resolved through the caller's project, shared ones included, so naming a view the caller cannot reach is refused with the new code 332. The column exists to stop a training set and the inference reads scored against it being held to different limits, and a bound the request carries is a bound the caller picked, so the guarantee only held for callers who used the SDK. It now holds however the request was built. The sweeper globs for spine files once across the cluster rather than issuing an exists and a listStatus per project every hour, which on a cluster of thousands of projects was thousands of NameNode round trips to find nothing. DistributedFileSystemOps gains globStatus for it. Two annotations in the client were wrong about their own behaviour. get_root_fg returned FeatureGroup while its docstring promised an ExternalFeatureGroup or a SpineGroup for views built on those, which are siblings under FeatureGroupBase, so the documented cases raised under HOPSWORKS_RUN_WITH_TYPECHECK. max_feature_age still advertised the withdrawn per-feature-group map, which always raised; it now advertises what it accepts and a dict is refused with a message naming the replacement. recreate_training_dataset takes spine_df, which its own parameter docs already told callers to use. Without it a training dataset built from a spine was silently rebuilt anchored on the root feature group, since the frame is not recorded with the dataset; the docstring now says so. The row and column ceilings are checked client-side before the frame is typed, written and uploaded, and on the product of entities and prediction times before cross builds it at all. They mirror the shipped defaults and the backend stays authoritative. Smaller: the passthrough dtype map covers the pandas nullable extension types, which is what a label column with missing values is; the bindable column check no longer counts passthrough columns, which let a frame of an event time and a label past a check meant to catch exactly that; the misspelled SplineDataFrameTypes alias becomes DeprecatedSpineTypes, one character from SpineDataFrameTypes being too close to read; a duplicated polars class leaves an isinstance tuple; PASSTHROUGH_TYPES becomes the set it always was. Both CI jobs on the client PR were already failing and are fixed here: ruff format on two files, and the PEP-8 check on InferenceSpine.arrow_table and write_parquet, public-named methods that are internal plumbing and are now _arrow_table and _write_parquet. The user guide and the batch inference skill record what changed for a reader: the bound is applied from the stored value, the client refuses an oversized frame before uploading it, and the limits are per request with no per-user or per-project concurrency cap. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01YMTv9mFtW88Cy2rcf6Gvzm --- docs/user_guides/fs/feature_view/future-batch-data.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index 99a7b6e2a8..2e14123465 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -153,6 +153,7 @@ batch_data = feature_view.get_batch_data(spine_df=spine) One bound covers the whole view: every feature group it reads is held to the same limit. It is read-only after creation and stored with the view. That is deliberate: if it could be changed per call, a training set and an inference read could be built with different bounds, which is the training/serving skew a feature view exists to prevent. +The bound is applied from the stored value, not from anything the read sends, so a read cannot opt out of it. Read it back with `feature_view.max_feature_age`, which returns a `timedelta` or `None` when the view is unbounded. ## Keys and event time in the result @@ -231,6 +232,12 @@ The size of `spine_df` is bounded by cluster limits, which an administrator sets | `featurestore_asof_spine_max_horizon_days` | How far ahead a schedule may expand | A request over any of them is refused before it runs, with the limit named. +The client checks the row and column ceilings itself, so a frame that is too large is refused before it is written and uploaded rather than after the round trip. +Those client-side ceilings are the shipped defaults; raising the cluster variables above them means raising the client's too. + +The limits are per request. +There is no cap on how many spine reads a user or a project may have in flight at once, so on a shared cluster a single caller can occupy the query service with repeated large reads. +Size the variables for the concurrency you expect, rather than for one request in isolation. Without a `lookback`, each feature group is scanned from its first row up to the last prediction time. The upper bound excludes forecast rows beyond your horizon, but it does not bound history. From 20c1ce5a5c26cbe3e660dae6c8de3608d834f6c4 Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Thu, 17 Sep 2026 08:18:54 +0200 Subject: [PATCH 13/15] [FSTORE-2116] Batch inference and training data anchored on a caller-supplied spine_df https://hopsworks.atlassian.net/browse/FSTORE-2116 Record what changed after an external review: a training dataset built from spine_df needs the frame again and refuses to run without it; the prediction time is written in the root's event time type and joined groups may differ; a lookback composes with a spine; a filter on a joined feature restricts its candidates and one spanning two groups is refused; max_feature_age applies to spine-anchored reads only; and what bounds Spark's candidate cardinality. Signed-off-by: Jim Dowling Co-Authored-By: Claude Fable 5.1 Claude-Session: https://claude.ai/code/session_01YMTv9mFtW88Cy2rcf6Gvzm --- .../fs/feature_view/future-batch-data.md | 31 +++++++++++++++++-- 1 file changed, 28 insertions(+), 3 deletions(-) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index 2e14123465..7f7f635717 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -192,7 +192,14 @@ carried through to the output untouched, which is how the label rides along. A b strict about unknown columns, because inference has no labels and a mistyped column there is worth catching. +A column named like a feature the view looks up from a joined feature group is refused rather than +carried through. The view reads that feature from the feature store, and carrying the frame's copy +as well would put two columns of one name in the result. Features of the root feature group are +different: a root feature in the frame is a passed feature and takes the place of the lookup. + `max_feature_age` applies here too, because it belongs to the view rather than to the call. +It applies to every read anchored on a `spine_df` and to nothing else: an ordinary `get_batch_data` +window and online serving through `get_feature_vector` are unchanged by it. A training example built from a feature that stopped being produced is the same silent staleness as an inference row built from one, and it is worse: the model learns from it. @@ -209,9 +216,19 @@ The label is the caller's own column and is never nulled by the bound. This is what a spine group does, without having had to create the feature view with one. `spine_df` and `spine` both replace the left side of the query, so passing both is an error. -!!! note "Materialized training datasets built this way are not reproducible" - A `create_*` call records the query, not your dataframe, so the dataset cannot be rebuilt - from its metadata alone. Keep the frame if you need to regenerate it. +!!! note "Materialized training datasets built this way need the frame again" + A `create_*` call records the query and the fact that a `spine_df` anchored it, not the + frame itself. `training_data(training_dataset_version=n)`, `recreate_training_dataset` and the + materialization job refuse to run for such a version without a `spine_df`, so the dataset is + never quietly rebuilt from the feature view's own rows under the same version number. Keep the + frame if you need to regenerate it, and pass it again. The reverse holds too: a version built + from the feature view's rows is not recreated on a frame. + +The prediction time column is written in the root feature group's own event time type. A +`timestamp` root takes timestamps, a `date` root takes dates, and a `bigint` root takes epoch +milliseconds, which is also how an integer column in `spine_df` is read whatever the root's type. +Feature groups joined into the view may keep their event time in a different type from the root; +the lookup converts theirs to the root's before comparing. ## Limits and performance @@ -242,6 +259,11 @@ Size the variables for the concurrency you expect, rather than for one request i Without a `lookback`, each feature group is scanned from its first row up to the last prediction time. The upper bound excludes forecast rows beyond your horizon, but it does not bound history. On a large feature group, `lookback` is what bounds the work, and `max_feature_age` bounds how many candidate rows each lookup considers. +A `lookback` passed together with `spine_df`, or recorded on the training dataset, restricts which rows of each feature group are candidates. +It never removes a row of `spine_df`: a spine row whose lookup finds nothing inside the window comes back with `NULL` features. + +Spark ranks every candidate row per spine row before keeping the newest, so its intermediate size is the number of eligible history rows across the spine, not the number of spine rows. +A hot key with deep history multiplied by many prediction times is where that grows; `lookback` and `max_feature_age` are the two bounds on it. ## Restrictions @@ -253,3 +275,6 @@ On a large feature group, `lookback` is what bounds the work, and `max_feature_a The frame's own timestamps define the time axis. - A feature view created with a spine group uses `spine=` instead; the two cannot be combined. - A filter on a column the entities supply is refused, because it would drop rows you asked to predict for. + A filter on the root feature group's event time is the exception: it bounds candidate rows, the same way a `lookback` does. +- A filter on a feature of a joined feature group restricts that feature group's candidate rows, wherever in the query it was added. + A single predicate that references two feature groups, such as an `OR` across them, is refused, because there is no one lookup it can restrict without changing its meaning. From 0bb4ce28a87bd131b61ad69e4d6c8790e5f3d558 Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Fri, 18 Sep 2026 14:08:05 +0200 Subject: [PATCH 14/15] [FSTORE-2116] Batch inference and training data anchored on a caller-supplied spine_df https://hopsworks.atlassian.net/browse/FSTORE-2116 Apply a review round from manu-sj on the eight pull requests. The limits table lists the variables the chart actually seeds. It named a horizon variable that does not exist, and omitted the file age, which is the fourth. How far ahead a schedule expands is the client's own max_horizon_days argument, and the file age bounds how long a staged spine file survives rather than bounding a request, so both are described as what they are. The sections this ticket added are one sentence per line, as the repo asks. They also record that reading a spine-anchored training dataset back takes the frame again, and that a root column the view does not select is refused by the client. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) --- .../fs/feature_view/future-batch-data.md | 53 ++++++++----------- 1 file changed, 23 insertions(+), 30 deletions(-) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index 7f7f635717..b8bd56b329 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -107,8 +107,7 @@ spine["event_time"] = now # named after the root feature group's event time co latest = feature_view.get_batch_data(spine_df=spine) ``` -To cover every entity the feature store knows about rather than a list you maintain, read them -off the feature view's root feature group. +To cover every entity the feature store knows about rather than a list you maintain, read them off the feature view's root feature group. `get_root_fg()` returns the feature group the view is anchored on, and `read_primary_keys()` returns its distinct primary key values, one row per entity. ```python @@ -176,9 +175,8 @@ batch_data = feature_view.get_batch_data( ## Training data from the same rows -The same mechanism builds training data. Pass `spine_df` to `training_data`, -`train_test_split`, `train_validation_test_split` or any of the `create_*` methods, and the -query is anchored on your rows instead of on the root feature group. +The same mechanism builds training data. +Pass `spine_df` to `training_data`, `train_test_split`, `train_validation_test_split` or any of the `create_*` methods, and the query is anchored on your rows instead of on the root feature group. ```python train_x, test_x, train_y, test_y = feature_view.train_test_split( @@ -187,21 +185,17 @@ train_x, test_x, train_y, test_y = feature_view.train_test_split( ) ``` -One difference from a batch read: columns the feature view does not define are -carried through to the output untouched, which is how the label rides along. A batch read stays -strict about unknown columns, because inference has no labels and a mistyped column there is -worth catching. +One difference from a batch read: columns the feature view does not define are carried through to the output untouched, which is how the label rides along. +A batch read stays strict about unknown columns, because inference has no labels and a mistyped column there is worth catching. -A column named like a feature the view looks up from a joined feature group is refused rather than -carried through. The view reads that feature from the feature store, and carrying the frame's copy -as well would put two columns of one name in the result. Features of the root feature group are -different: a root feature in the frame is a passed feature and takes the place of the lookup. +A column named like a feature the view looks up from a joined feature group is refused rather than carried through. +The view reads that feature from the feature store, and carrying the frame's copy as well would put two columns of one name in the result. +Features of the root feature group are different: a root feature in the frame is a passed feature and takes the place of the lookup. +A root column the view does not select is refused as well, by the client rather than after the round trip, since the view neither looks it up nor returns it. `max_feature_age` applies here too, because it belongs to the view rather than to the call. -It applies to every read anchored on a `spine_df` and to nothing else: an ordinary `get_batch_data` -window and online serving through `get_feature_vector` are unchanged by it. -A training example built from a feature that stopped being produced is the same silent -staleness as an inference row built from one, and it is worse: the model learns from it. +It applies to every read anchored on a `spine_df` and to nothing else: an ordinary `get_batch_data` window and online serving through `get_feature_vector` are unchanged by it. +A training example built from a feature that stopped being produced is the same silent staleness as an inference row built from one, and it is worse: the model learns from it. ```python # the view was created with max_feature_age=timedelta(days=1), so any feature whose newest @@ -217,18 +211,15 @@ This is what a spine group does, without having had to create the feature view w `spine_df` and `spine` both replace the left side of the query, so passing both is an error. !!! note "Materialized training datasets built this way need the frame again" - A `create_*` call records the query and the fact that a `spine_df` anchored it, not the - frame itself. `training_data(training_dataset_version=n)`, `recreate_training_dataset` and the - materialization job refuse to run for such a version without a `spine_df`, so the dataset is - never quietly rebuilt from the feature view's own rows under the same version number. Keep the - frame if you need to regenerate it, and pass it again. The reverse holds too: a version built - from the feature view's rows is not recreated on a frame. - -The prediction time column is written in the root feature group's own event time type. A -`timestamp` root takes timestamps, a `date` root takes dates, and a `bigint` root takes epoch -milliseconds, which is also how an integer column in `spine_df` is read whatever the root's type. -Feature groups joined into the view may keep their event time in a different type from the root; -the lookup converts theirs to the root's before comparing. + A `create_*` call records the query and the fact that a `spine_df` anchored it, not the frame itself. + `get_training_data`, `get_train_test_split`, `get_train_validation_test_split`, `recreate_training_dataset` and the materialization job all refuse to run for such a version without a `spine_df`, so the dataset is never quietly rebuilt from the feature view's own rows under the same version number. + Keep the frame if you need to regenerate it, and pass it to those methods as well. + The reverse holds too: a version built from the feature view's rows is not recreated on a frame. + +The prediction time column is written in the root feature group's own event time type. +A `timestamp` root takes timestamps, a `date` root takes dates, and a `bigint` root takes epoch milliseconds, which is also how an integer column in `spine_df` is read whatever the root's type. +`PredictionTimes` reads an integer the same way. +Feature groups joined into the view may keep their event time in a different type from the root; the lookup converts theirs to the root's before comparing. ## Limits and performance @@ -246,9 +237,11 @@ The size of `spine_df` is bounded by cluster limits, which an administrator sets | `featurestore_asof_spine_max_rows` | Rows, meaning entities multiplied by prediction times | | `featurestore_asof_spine_max_bytes` | The serialized size of those rows | | `featurestore_asof_spine_max_columns` | Columns in `spine_df` | -| `featurestore_asof_spine_max_horizon_days` | How far ahead a schedule may expand | +| `featurestore_asof_spine_max_file_age_ms` | How long a staged spine file survives before the backend reclaims it | A request over any of them is refused before it runs, with the limit named. +The file age is not a limit on a request: it is how long the file a read stages stays readable, and it has to outlive the longest materialization job, which reads the file after the call that wrote it has returned. +How far ahead a schedule may expand is not a cluster variable; it is the `max_horizon_days` argument of [`PredictionTimes`][hsfs.constructor.prediction_times.PredictionTimes], which defaults to ten years. The client checks the row and column ceilings itself, so a frame that is too large is refused before it is written and uploaded rather than after the round trip. Those client-side ceilings are the shipped defaults; raising the cluster variables above them means raising the client's too. From 4c3ee8ab2a917f5148ac33de5d60548c5724e850 Mon Sep 17 00:00:00 2001 From: Jim Dowling Date: Fri, 18 Sep 2026 14:18:16 +0200 Subject: [PATCH 15/15] [FSTORE-2116] Batch inference and training data anchored on a caller-supplied spine_df https://hopsworks.atlassian.net/browse/FSTORE-2116 Name PredictionTimes in prose rather than as a cross-reference. The docs build resolves API references against the client checkout it is given, which does not carry this module until the client release lands, so the reference aborted the strict build. It passed locally only because the local checkout is this ticket's branch. Signed-off-by: Jim Dowling Co-Authored-By: Claude Opus 5 (1M context) --- docs/user_guides/fs/feature_view/future-batch-data.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md index b8bd56b329..7c13aa88e3 100644 --- a/docs/user_guides/fs/feature_view/future-batch-data.md +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -241,7 +241,7 @@ The size of `spine_df` is bounded by cluster limits, which an administrator sets A request over any of them is refused before it runs, with the limit named. The file age is not a limit on a request: it is how long the file a read stages stays readable, and it has to outlive the longest materialization job, which reads the file after the call that wrote it has returned. -How far ahead a schedule may expand is not a cluster variable; it is the `max_horizon_days` argument of [`PredictionTimes`][hsfs.constructor.prediction_times.PredictionTimes], which defaults to ten years. +How far ahead a schedule may expand is not a cluster variable; it is the `max_horizon_days` argument of `PredictionTimes`, which defaults to ten years. The client checks the row and column ceilings itself, so a frame that is too large is refused before it is written and uploaded rather than after the round trip. Those client-side ceilings are the shipped defaults; raising the cluster variables above them means raising the client's too.