diff --git a/docs/user_guides/fs/feature_group/on_demand_transformations.md b/docs/user_guides/fs/feature_group/on_demand_transformations.md index 63dde49cc7..3300234323 100644 --- a/docs/user_guides/fs/feature_group/on_demand_transformations.md +++ b/docs/user_guides/fs/feature_group/on_demand_transformations.md @@ -128,7 +128,7 @@ The on-demand features in the feature vector can be computed using real-time dat ```python feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, + serving_keys={"id": 1}, request_parameter={ "transaction_time": datetime(2022, 12, 28, 23, 55, 59), "current_time": datetime.now(), @@ -147,7 +147,7 @@ The `request_parameter` in this case, can be a list of dictionaries that specifi ```python # Specify unique request parameters for each serving key. feature_vector = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], + serving_keys=[{"id": 1}, {"id": 2}], request_parameter=[ { "transaction_time": datetime(2022, 12, 28, 23, 55, 59), @@ -162,7 +162,7 @@ The `request_parameter` in this case, can be a list of dictionaries that specifi # Specify common request parameters for all serving key. feature_vector = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], + serving_keys=[{"id": 1}, {"id": 2}], request_parameter={ "transaction_time": datetime(2022, 12, 28, 23, 55, 59), "current_time": datetime.now(), @@ -180,10 +180,10 @@ To achieve this, set the parameters `transform` and `on_demand_features` to `Fa ```python untransformed_feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, transform=False, on_demand_features=False + serving_keys={"id": 1}, transform=False, on_demand_features=False ) untransformed_feature_vectors = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False + serving_keys=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False ) ``` @@ -201,7 +201,7 @@ The `request_parameter` in this case, can be a list of dictionaries that specifi ```python # Specify request parameters for each serving key. untransformed_feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, transform=False, on_demand_features=False + serving_keys={"id": 1}, transform=False, on_demand_features=False ) # re-compute and add on-demand features to the feature vector @@ -218,7 +218,7 @@ The `request_parameter` in this case, can be a list of dictionaries that specifi # Specify request parameters for each serving key. untransformed_feature_vectors = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False + serving_keys=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False ) # re-compute and add on-demand features to the feature vectors - Specify unique request parameter for each feature vector @@ -259,7 +259,7 @@ On-demand transformation functions can also be accessed and executed as normal f ```python # Specify request parameters for each serving key. feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, + serving_keys={"id": 1}, transform=False, on_demand_features=False, return_type="pandas", diff --git a/docs/user_guides/fs/feature_view/batch-data.md b/docs/user_guides/fs/feature_view/batch-data.md index 2f6db323b4..bb58ad3986 100644 --- a/docs/user_guides/fs/feature_view/batch-data.md +++ b/docs/user_guides/fs/feature_view/batch-data.md @@ -21,6 +21,12 @@ The resultant DataFrame (or batch-scoring DataFrame) can then be fed to models t Dataset ds = featureView.getBatchData("20220620", "20220627") ``` +## Batch data for timestamps in the future + +A time range can only return rows the root feature group has already observed, so it returns nothing for the future. +To score timestamps that have not happened yet, pass the entities and the prediction times instead of a range. +See [Batch data for future timestamps][batch-data-for-future-timestamps]. + ## Retrieve batch data with primary keys and event time For certain use cases, e.g., time series models, the input data needs to be sorted according to the primary key(s) and event time combination. diff --git a/docs/user_guides/fs/feature_view/feature-vectors.md b/docs/user_guides/fs/feature_view/feature-vectors.md index 125e3615b5..3a2fff6606 100644 --- a/docs/user_guides/fs/feature_view/feature-vectors.md +++ b/docs/user_guides/fs/feature_view/feature-vectors.md @@ -15,19 +15,27 @@ If you need to get more familiar with the concept of feature vectors, you can re You can get back feature vectors from either python or java client by providing the primary key value(s) for the feature view. Note that filters defined in feature view and training data will not be applied when feature vectors are returned. -If you need to retrieve a complete value of feature vectors without missing values, the required `entry` are [FeatureView.primary_keys][hsfs.feature_view.FeatureView.primary_keys]. -Alternative, you can provide the primary key of the feature groups as the key of the entry. -It is also possible to provide a subset of the entry, which will be discussed [below](#partial-feature-retrieval). +If you need to retrieve a complete value of feature vectors without missing values, the required `serving_keys` are [FeatureView.primary_keys][hsfs.feature_view.FeatureView.primary_keys]. +Alternative, you can provide the primary key of the feature groups as the key of the serving keys. +It is also possible to provide a subset of the serving keys, which will be discussed [below](#partial-feature-retrieval). + +!!! note "`entry` was renamed to `serving_keys`" + The argument used to be called `entry`, on `get_feature_vector`, `get_feature_vectors`, + `get_inference_helper` and `get_inference_helpers`. + `entry` still works and takes the same value, but it emits a `DeprecationWarning` and will + be removed in a future release. + Passing both names in one call is an error. + Positional calls such as `get_feature_vector({"pk1": 1})` are unaffected. === "Python" ```python # get a single vector - feature_view.get_feature_vector(entry={"pk1": 1, "pk2": 2}) + feature_view.get_feature_vector(serving_keys={"pk1": 1, "pk2": 2}) # get multiple vectors feature_view.get_feature_vectors( - entry=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}] + serving_keys=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}] ) ``` @@ -47,7 +55,7 @@ It is also possible to provide a subset of the entry, which will be discussed [b featureView.getFeatureVectors(Lists.newArrayList(entry1, entry2)); ``` -### Required entry +### Required serving keys Starting from python client v3.4, you can specify different values for the primary key of the same name which exists in multiple feature groups but are not joint by the same name. The table below summarises the value of `primary_keys` in different settings. @@ -89,7 +97,7 @@ Take the above example assuming the feature view consists of two joined feature ```python # get a single vector - feature_view.get_feature_vector(entry={"pk1": 1, "pk2": 2}) + feature_view.get_feature_vector(serving_keys={"pk1": 1, "pk2": 2}) ``` === "Java" @@ -111,7 +119,7 @@ When retrieving a batch of vectors, the behaviour is slightly different. ```python # get multiple vectors feature_view.get_feature_vectors( - entry=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}] + serving_keys=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}] ) ``` @@ -137,18 +145,18 @@ If you are aware of missing features, you can use the [*passed features*](#passe ### Partial feature retrieval If your model can handle missing value or if you want to impute the missing value, you can get back feature vectors with partial values using python client starting from version 3.4 (Note that this does not apply to java client.). -In the example below, let's say you join 2 feature groups by `fg1.join(fg2, left_on=["pk1"], right_on=["pk2"])`, required keys of the `entry` are `pk1` and `pk2`. +In the example below, let's say you join 2 feature groups by `fg1.join(fg2, left_on=["pk1"], right_on=["pk2"])`, required keys of `serving_keys` are `pk1` and `pk2`. If `pk2` is not provided, this returns feature values from the first feature group and null values from the second feature group when using the option `allow_missing=True`, otherwise it raises exception. === "Python" ```python # get a single vector with - feature_view.get_feature_vector(entry={"pk1": 1}, allow_missing=True) + feature_view.get_feature_vector(serving_keys={"pk1": 1}, allow_missing=True) # get multiple vectors feature_view.get_feature_vectors( - entry=[ + serving_keys=[ {"pk1": 1}, {"pk1": 3}, ], @@ -184,12 +192,12 @@ Please note that passed features is only available in the python client but not ```python # get a single vector feature_view.get_feature_vector( - entry={"pk1": 1, "pk2": 2}, passed_features={"feature_a": "value_a"} + serving_keys={"pk1": 1, "pk2": 2}, passed_features={"feature_a": "value_a"} ) # get multiple vectors feature_view.get_feature_vectors( - entry=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}], + serving_keys=[{"pk1": 1, "pk2": 2}, {"pk1": 3, "pk2": 4}, {"pk1": 5, "pk2": 6}], passed_features=[ {"feature_a": "value_a1"}, {"feature_a": "value_a2"}, @@ -210,7 +218,7 @@ In this second case, you do not have to provide the primary key value for that f # in this case feature_b and feature_c feature_view.get_feature_vector( - entry={"pk1": 1}, + serving_keys={"pk1": 1}, passed_features={ "feature_a": "value_a", "feature_b": "value_b", @@ -231,12 +239,12 @@ However, you can retrieve the untransformed feature vectors without applying mod ```python # Fetching untransformed feature vector. untransformed_feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, transform=False + serving_keys={"id": 1}, transform=False ) # Fetching untransformed feature vectors. untransformed_feature_vectors = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], transform=False + serving_keys=[{"id": 1}, {"id": 2}], transform=False ) ``` @@ -250,10 +258,10 @@ To achieve this, set the parameters `transform` and `on_demand_features` to `Fa ```python untransformed_feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, transform=False, on_demand_features=False + serving_keys={"id": 1}, transform=False, on_demand_features=False ) untransformed_feature_vectors = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False + serving_keys=[{"id": 1}, {"id": 2}], transform=False, on_demand_features=False ) ``` @@ -267,7 +275,7 @@ After [defining a transformation function using a context variable](../transform ```python # Passing context variable to IN-MEMORY Training Dataset. batch_data = feature_view.get_feature_vectors( - entry=[{"pk1": 1}], transformation_context={"context_parameter": 10} + serving_keys=[{"pk1": 1}], transformation_context={"context_parameter": 10} ) ``` @@ -320,12 +328,12 @@ my_feature_view.init_serving( # this will fetch a feature vector via REST try: my_feature_view.get_feature_vector( - entry={"pk1": 1, "pk2": 2}, + serving_keys={"pk1": 1, "pk2": 2}, ) except TimeoutException: # if the REST client times out, the SQL client will be used my_feature_view.get_feature_vector( - entry={"pk1": 1, "pk2": 2}, force_sql=True + serving_keys={"pk1": 1, "pk2": 2}, force_sql=True ) ``` diff --git a/docs/user_guides/fs/feature_view/future-batch-data.md b/docs/user_guides/fs/feature_view/future-batch-data.md new file mode 100644 index 0000000000..7c13aa88e3 --- /dev/null +++ b/docs/user_guides/fs/feature_view/future-batch-data.md @@ -0,0 +1,273 @@ +# Batch data for future timestamps + +## Why a time range is not enough + +`get_batch_data(start_time, end_time)` filters the feature view's root feature group, the left-most feature group of its query, and joins the rest onto the rows it finds. +That works whenever the root feature group has an observation for every entity and time you want to score. +It returns nothing for the future, because the future has no observations yet. + +An air quality feature view is the common shape. +It joins `air_quality` observations to a `weather` feature group that holds both observations and a forecast. +`get_batch_data(start_time=tomorrow, end_time=tomorrow + 7 days)` returns zero rows, even though `weather` holds all seven forecast days, because `air_quality` holds none of them. +The usual workaround is to read the forecast feature group directly, which loses the feature view's joins, its feature selection and its transformations. + +Instead, tell the feature view which rows you want to predict for. +`spine_df` is the set of rows to predict for: one row per entity and moment, carrying the serving keys and the prediction time. +It replaces the root feature group as the anchor of the query, and every feature group is looked up as of each row's own time. + +## Retrieving batch data for future timestamps + +```python +import datetime + +import pandas as pd +from hsfs.constructor.prediction_times import PredictionTimes + +tomorrow = datetime.date.today() + datetime.timedelta(days=1) + +entities = pd.DataFrame( + [{"country": "sweden", "city": "stockholm", "street": "sveavagen"}] +) +schedule = PredictionTimes.every("daily", offset="00:00", start=tomorrow, count=7) + +batch_data = feature_view.get_batch_data( + spine_df=schedule.cross(entities, event_time="date"), +) +``` + +The result has one row per entity per prediction time, so the example returns seven rows. +Rows come back in the order of `spine_df`, and within an entity in prediction-time order, so the frame can be handed to a model and its predictions joined back positionally. + +Every feature is taken from the most recent row at or before its prediction time. +For a forecast row dated in the future, that is the forecast for that day. +A feature group with no matching row contributes `NULL` rather than removing the row, the same as any left join, so an entity the feature store has never seen still comes back with the features that do resolve. + +## Choosing the entities + +`spine_df` accepts a pandas or polars DataFrame, a list of dictionaries, or, under the Spark engine, a Spark DataFrame. +A Spark DataFrame is refused under the Python engine, which has no session to evaluate it. +A Spark spine is collected to the driver to be registered as a session temporary view, which is what the Spark spine path has always done, so size the frame to the entities you are scoring. +A Spark DataFrame also has no row order, so the positional zip-back below does not apply to one: join predictions back on the serving keys instead. +Its columns may be: + +- the feature view's required serving keys, which identify the entity; +- any column of the root feature group, which is then used as supplied instead of being looked up. + +The second kind is the same idea as a passed feature in an online deployment. +If the root feature group holds a `pm25` column and `spine_df` carries one, the value you passed is returned and no lookup is made for it. + +Omitting some serving keys is allowed and warns, because a model may be able to infer what is missing. +Omitting all of them is an error: nothing in the feature view can then be looked up, which is a mistake rather than an empty result. +A column that matches neither a serving key nor a root feature is an error naming the columns that are accepted. + +## Choosing the timestamps + +The frame carries one timestamp per row, under the root feature group's event time column. +Build it yourself if you already have the rows, or let `PredictionTimes` build a schedule and cross it with your entities. +`cross` is worth preferring over rolling your own: it fixes the row order, entities as given and ascending in time within each, which is the order the result comes back in. + +`PredictionTimes` builds the set of timestamps three ways. + +```python +from hsfs.constructor.prediction_times import PredictionTimes + +# A named interval with an offset. Intervals: hourly, daily, weekly, monthly. +PredictionTimes.every("daily", offset="08:00", start=tomorrow, count=7) +PredictionTimes.every("weekly", offset="mon:09:30", start=tomorrow, count=4) + +# A cron expression, in the five-field Vixie dialect, where 0 and 7 are both Sunday. +PredictionTimes.cron("0 8 * * 1-5", start=tomorrow, count=10) + +# An explicit list, for a schedule no rule describes. +PredictionTimes.of([datetime.datetime(2026, 3, 1, 8, 0)]) +``` + + +Times are local, and daylight saving is resolved the way a scheduler resolves it. +A local time that does not exist on the day the clocks go forward is skipped. +A local time that occurs twice on the day they go back is taken at its first occurrence. + +!!! note + The prediction time in the returned frame is the time you asked for, not the event time of the row that matched it. + A prediction time of 08:00 matching a forecast row written at 00:00 comes back as 08:00. + +## Latest feature values for every entity + +One row per entity, all at the same instant, is the offline equivalent of `get_feature_vectors`. +Capture the timestamp once so every entity is read at the same moment rather than each drifting. + +```python +import datetime + +now = datetime.datetime.now(datetime.timezone.utc) + +spine = pd.DataFrame({"entity_id": [1, 2, 3]}) +spine["event_time"] = now # named after the root feature group's event time column + +latest = feature_view.get_batch_data(spine_df=spine) +``` + +To cover every entity the feature store knows about rather than a list you maintain, read them off the feature view's root feature group. +`get_root_fg()` returns the feature group the view is anchored on, and `read_primary_keys()` returns its distinct primary key values, one row per entity. + +```python +import datetime +from hsfs.constructor.prediction_times import PredictionTimes + +fg = feature_view.get_root_fg() +now = datetime.datetime.now(datetime.timezone.utc) + +spine = PredictionTimes.of([now]).cross(fg.read_primary_keys(), event_time=fg.event_time) +latest = feature_view.get_batch_data(spine_df=spine) +``` + +`read_primary_keys()` returns entities and no time, so it is not a `spine_df` on its own and passing it directly is refused. +Crossing it with one instant is what makes it one. +It reads the key columns of the whole feature group to take the distinct rows, so the cost scales with the feature group rather than with the number of entities, and it returns the root's keys only: a joined feature group keyed on something the root does not carry is not covered by it and comes back `NULL`. + +There is no implicit "as of now": the time is always in the frame. +A wall-clock default would make the same call return different rows on a re-run, and a training dataset materialized that way could never be reproduced. +Event times are kept to the millisecond, so sub-millisecond precision in the timestamp you pass is dropped rather than rejected. + +## Bounding how stale a feature may be + +An as-of lookup carries the last value forward for ever. +If a forecast is missing for one day, that day silently inherits the previous day's weather, and the frame gives no sign of it. + +`max_feature_age` bounds how old a matched row may be, relative to the prediction time. +A row older than the bound is returned as `NULL`, so the gap is visible to you and to the model. + +```python +# Set when the view is created, so it applies to every read anchored on a spine_df, +# batch inference and training data alike. +feature_view = fs.create_feature_view( + name="air_quality_fv", + query=query, + max_feature_age=datetime.timedelta(days=1), +) + +batch_data = feature_view.get_batch_data(spine_df=spine) +``` + +One bound covers the whole view: every feature group it reads is held to the same limit. +It is read-only after creation and stored with the view. +That is deliberate: if it could be changed per call, a training set and an inference read could be built with different bounds, which is the training/serving skew a feature view exists to prevent. +The bound is applied from the stored value, not from anything the read sends, so a read cannot opt out of it. +Read it back with `feature_view.max_feature_age`, which returns a `timedelta` or `None` when the view is unbounded. + +## Keys and event time in the result + +For a normal `get_batch_data` call, `primary_key` and `event_time` default to `False`. +For a call with `spine_df` they default to `True`, because without the keys and the prediction time the frame does not say which row belongs to which entity or day. +Pass `False` explicitly to leave them out, which is what you want when the model consumes the frame directly. + +```python +batch_data = feature_view.get_batch_data( + spine_df=spine, + primary_key=False, + event_time=False, +) +``` + +!!! note + When feature groups in the view share a column name, key columns come back fully qualified as `___`. + This is how `get_batch_data` has always named ambiguous key columns; it is not specific to this call. + +## Training data from the same rows + +The same mechanism builds training data. +Pass `spine_df` to `training_data`, `train_test_split`, `train_validation_test_split` or any of the `create_*` methods, and the query is anchored on your rows instead of on the root feature group. + +```python +train_x, test_x, train_y, test_y = feature_view.train_test_split( + test_size=0.2, + spine_df=labels, # keys, an event time per row, and the label +) +``` + +One difference from a batch read: columns the feature view does not define are carried through to the output untouched, which is how the label rides along. +A batch read stays strict about unknown columns, because inference has no labels and a mistyped column there is worth catching. + +A column named like a feature the view looks up from a joined feature group is refused rather than carried through. +The view reads that feature from the feature store, and carrying the frame's copy as well would put two columns of one name in the result. +Features of the root feature group are different: a root feature in the frame is a passed feature and takes the place of the lookup. +A root column the view does not select is refused as well, by the client rather than after the round trip, since the view neither looks it up nor returns it. + +`max_feature_age` applies here too, because it belongs to the view rather than to the call. +It applies to every read anchored on a `spine_df` and to nothing else: an ordinary `get_batch_data` window and online serving through `get_feature_vector` are unchanged by it. +A training example built from a feature that stopped being produced is the same silent staleness as an inference row built from one, and it is worse: the model learns from it. + +```python +# the view was created with max_feature_age=timedelta(days=1), so any feature whose newest +# row is older than a day carries NULL rather than a stale value +train_x, test_x, train_y, test_y = feature_view.train_test_split( + test_size=0.2, spine_df=labels +) +``` + +The label is the caller's own column and is never nulled by the bound. + +This is what a spine group does, without having had to create the feature view with one. +`spine_df` and `spine` both replace the left side of the query, so passing both is an error. + +!!! note "Materialized training datasets built this way need the frame again" + A `create_*` call records the query and the fact that a `spine_df` anchored it, not the frame itself. + `get_training_data`, `get_train_test_split`, `get_train_validation_test_split`, `recreate_training_dataset` and the materialization job all refuse to run for such a version without a `spine_df`, so the dataset is never quietly rebuilt from the feature view's own rows under the same version number. + Keep the frame if you need to regenerate it, and pass it to those methods as well. + The reverse holds too: a version built from the feature view's rows is not recreated on a frame. + +The prediction time column is written in the root feature group's own event time type. +A `timestamp` root takes timestamps, a `date` root takes dates, and a `bigint` root takes epoch milliseconds, which is also how an integer column in `spine_df` is read whatever the root's type. +`PredictionTimes` reads an integer the same way. +Feature groups joined into the view may keep their event time in a different type from the root; the lookup converts theirs to the root's before comparing. + +## Limits and performance + +Offline feature groups only. +Online serving through `get_feature_vector` is unchanged and does not take prediction times. + +Both engines are supported. +The Hopsworks Query Service renders each lookup as a DuckDB `ASOF LEFT JOIN`, and Spark renders it as a ranked window over the same rows. +Both return the same frame. + +The size of `spine_df` is bounded by cluster limits, which an administrator sets: + +| Variable | Bounds | +| --- | --- | +| `featurestore_asof_spine_max_rows` | Rows, meaning entities multiplied by prediction times | +| `featurestore_asof_spine_max_bytes` | The serialized size of those rows | +| `featurestore_asof_spine_max_columns` | Columns in `spine_df` | +| `featurestore_asof_spine_max_file_age_ms` | How long a staged spine file survives before the backend reclaims it | + +A request over any of them is refused before it runs, with the limit named. +The file age is not a limit on a request: it is how long the file a read stages stays readable, and it has to outlive the longest materialization job, which reads the file after the call that wrote it has returned. +How far ahead a schedule may expand is not a cluster variable; it is the `max_horizon_days` argument of `PredictionTimes`, which defaults to ten years. +The client checks the row and column ceilings itself, so a frame that is too large is refused before it is written and uploaded rather than after the round trip. +Those client-side ceilings are the shipped defaults; raising the cluster variables above them means raising the client's too. + +The limits are per request. +There is no cap on how many spine reads a user or a project may have in flight at once, so on a shared cluster a single caller can occupy the query service with repeated large reads. +Size the variables for the concurrency you expect, rather than for one request in isolation. + +Without a `lookback`, each feature group is scanned from its first row up to the last prediction time. +The upper bound excludes forecast rows beyond your horizon, but it does not bound history. +On a large feature group, `lookback` is what bounds the work, and `max_feature_age` bounds how many candidate rows each lookup considers. +A `lookback` passed together with `spine_df`, or recorded on the training dataset, restricts which rows of each feature group are candidates. +It never removes a row of `spine_df`: a spine row whose lookup finds nothing inside the window comes back with `NULL` features. + +Spark ranks every candidate row per spine row before keeping the newest, so its intermediate size is the number of eligible history rows across the spine, not the number of spine rows. +A hot key with deep history multiplied by many prediction times is where that grows; `lookback` and `max_feature_age` are the two bounds on it. + +## Restrictions + +- The feature view's joins must be `LEFT` or `INNER`. + A `RIGHT` or `FULL` join keeps source rows that have no prediction time to align to. +- Feature groups joined through another feature group are not supported. +- Every feature group in the view needs an event time, because an as-of lookup has nothing to order on without one. +- `spine_df` cannot be combined with `start_time` and `end_time`. + The frame's own timestamps define the time axis. +- A feature view created with a spine group uses `spine=` instead; the two cannot be combined. +- A filter on a column the entities supply is refused, because it would drop rows you asked to predict for. + A filter on the root feature group's event time is the exception: it bounds candidate rows, the same way a `lookback` does. +- A filter on a feature of a joined feature group restricts that feature group's candidate rows, wherever in the query it was added. + A single predicate that references two feature groups, such as an `OR` across them, is refused, because there is no one lookup it can restrict without changing its meaning. diff --git a/docs/user_guides/fs/feature_view/model-dependent-transformations.md b/docs/user_guides/fs/feature_view/model-dependent-transformations.md index 61d980c660..6e7e73352d 100644 --- a/docs/user_guides/fs/feature_view/model-dependent-transformations.md +++ b/docs/user_guides/fs/feature_view/model-dependent-transformations.md @@ -146,7 +146,7 @@ Model-dependent transformation functions can also be manually applied to a featu # Get untransformed feature Vector feature_vector = fv.get_feature_vector( - entry={"index": 10}, transform=False, return_type="pandas" + serving_keys={"index": 10}, transform=False, return_type="pandas" ) # Apply Model Dependent transformations @@ -164,12 +164,12 @@ To achieve this, set the `transform` parameter to False. ```python # Fetching untransformed feature vector. untransformed_feature_vector = feature_view.get_feature_vector( - entry={"id": 1}, transform=False + serving_keys={"id": 1}, transform=False ) # Fetching untransformed feature vectors. untransformed_feature_vectors = feature_view.get_feature_vectors( - entry=[{"id": 1}, {"id": 2}], transform=False + serving_keys=[{"id": 1}, {"id": 2}], transform=False ) # Fetching untransformed batch data. diff --git a/docs/user_guides/fs/transformation_functions.md b/docs/user_guides/fs/transformation_functions.md index 25047b227b..73113409b4 100644 --- a/docs/user_guides/fs/transformation_functions.md +++ b/docs/user_guides/fs/transformation_functions.md @@ -424,7 +424,7 @@ For online serving, spawning the worker pool during the first request would add fv.init_serving(training_dataset_version=1, n_processes=2) # Served using the pool of two workers spawned at init time. - vector = fv.get_feature_vector(entry={"id": 1}) + vector = fv.get_feature_vector(serving_keys={"id": 1}) ``` The worker pool start method defaults to `fork` on Linux and `spawn` on macOS and Windows. Set the `HOPSWORKS_TF_POOL_START_METHOD` environment variable to `fork`, `forkserver`, or `spawn` to override it. diff --git a/mkdocs.yml b/mkdocs.yml index 68e3f7aec6..d1c3fb824e 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -102,6 +102,7 @@ nav: - Overview: user_guides/fs/feature_view/overview.md - Training data: user_guides/fs/feature_view/training-data.md - Batch data: user_guides/fs/feature_view/batch-data.md + - Batch data for future timestamps: user_guides/fs/feature_view/future-batch-data.md - Feature vectors: user_guides/fs/feature_view/feature-vectors.md - Feature server: user_guides/fs/feature_view/feature-server.md - Query: user_guides/fs/feature_view/query.md