Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions docs/user_guides/fs/feature_group/feature_monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,35 @@ Additionally, you can specify the percentage of feature data on which statistics

See the API reference for [`FeatureMonitoringConfig.with_detection_window`][hsfs.core.feature_monitoring_config.FeatureMonitoringConfig.with_detection_window].

#### Time basis of the windows

Rolling windows select rows by an event-time feature when the Feature Group declares one, and by commit time otherwise.
The `event_time` parameter of `create_scheduled_statistics` and `create_feature_monitoring` overrides that default for the whole configuration, detection and reference windows alike.
Pass a feature name to use another timestamp, date or epoch feature of the Feature Group, or `False` to select rows by commit time.

=== "Python"

```python
# windows over the transaction time, the Feature Group event_time (default)
fg_monitoring_config = trans_fg.create_feature_monitoring(
name="trans_fg_amount_monitoring",
)

# windows over another time feature of the Feature Group
fg_monitoring_config = trans_fg.create_feature_monitoring(
name="trans_fg_amount_monitoring_by_settlement",
event_time="settlement_date",
)

# windows over the time the rows were written (commit time)
fg_monitoring_config = trans_fg.create_feature_monitoring(
name="trans_fg_amount_monitoring_by_commit",
event_time=False,
)
```

See [Time basis](../feature_monitoring/scheduled_statistics.md#time-basis) for how the two bases differ.

### Step 4: (Optional) Define a reference window

When setting up feature monitoring for a Feature Group, you can compare the detection statistics against a reference window of feature data.
Expand Down
16 changes: 13 additions & 3 deletions docs/user_guides/fs/feature_group/statistics.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,15 +93,25 @@ Users can schedule periodic statistics computation that take into consideration
By default, the `compute_statistics` method computes statistics on the most recent version of the data available in a feature group.
Users can provide a specific time using the `wallclock_time` parameter, to compute the statistics for a previous version of the data.

Hopsworks can compute statistics of external feature groups.
As external feature groups are read only from an Hopsworks perspective, statistics computation can be triggered using the `compute_statistics` method.

=== "Python"

```python
fg.compute_statistics(wallclock_time="20220611 20:00")
```

### External feature groups

External feature groups own the same built-in `ingestion_stats` configuration as cached and stream feature groups, but no data is ingested into Hopsworks for them, so it never runs on its own.
Calling `compute_statistics` on the external feature group, or clicking "Compute statistics" in the UI, runs it: the statistics job reads the external source and profiles it like an internal feature group.
Saving an external feature group with statistics enabled runs it once as well.
External feature groups have no commit history, so their statistics carry the computation time only and `compute_statistics` takes no time argument.

=== "Python"

```python
external_fg.compute_statistics()
```

## Inspect statistics

You can also create a new feature group through the UI.
16 changes: 16 additions & 0 deletions docs/user_guides/fs/feature_monitoring/scheduled_statistics.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,22 @@ Taking a Feature Group as an example, the figure above describes how these windo
- A _rolling window_ covering a variable subset of feature data (e.g., feature data written last week).
It helps you analyze the properties of **newly inserted feature data**.

### Time basis

A rolling window needs a notion of time to decide which rows fall inside it.
Hopsworks supports two bases, chosen once per configuration and shared by the detection and reference windows:

- _Event time_: rows are selected by the value of an event-time feature, so a window such as "last week" contains the rows whose event time falls in that week regardless of when they were written.
Backfills and late arrivals land in the window of their event time.
This is the default for Feature Groups and Feature Views that declare an `event_time`, and it also works on Feature Groups without time travel and on external Feature Groups.
- _Commit time_: rows are selected by the time they were written to the Feature Group, using time travel.
A window such as "last week" contains the rows committed during that week.
This is the default when no event-time feature is declared, and it requires a time-travel enabled Feature Group.

An expanding window reads the latest snapshot on either basis, with no time filter.
Statistics computed on an event-time window are stored with the event-time bounds of the window, and statistics computed on a commit-time window with its commit bounds.
The `event_time` parameter of `create_scheduled_statistics` and `create_feature_monitoring` selects the basis; see the guides linked below.

See more details on how to define a detection window for your Feature Groups and Feature Views in the Feature Monitoring Guides for [Feature Groups](../feature_group/feature_monitoring.md) and [Feature Views](../feature_view/feature_monitoring.md).

!!! info "Next steps"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,9 @@ Taking a Feature View as an example, the figure above describes how these window
- A _specific value_.
It helps you target the analysis of feature data to a **specific feature and statistics metric**.

Rolling and expanding reference windows use the same time basis as the detection window of the configuration, either the event-time feature or the commit time.
See [Time basis](scheduled_statistics.md#time-basis) in the scheduled statistics guide.

See more details on how to define a reference window for your Feature Groups and Training Datasets in the Feature Monitoring guides for [Feature Groups](../feature_group/feature_monitoring.md) and [Feature Views](../feature_view/feature_monitoring.md).

## Comparison criteria
Expand Down
26 changes: 26 additions & 0 deletions docs/user_guides/fs/feature_view/feature_monitoring.md
Original file line number Diff line number Diff line change
Expand Up @@ -133,6 +133,31 @@ Additionally, you can specify the percentage of feature data on which statistics

See the API reference for [`FeatureMonitoringConfig.with_detection_window`][hsfs.core.feature_monitoring_config.FeatureMonitoringConfig.with_detection_window].

#### Time basis of the windows

Rolling windows select rows by the event-time feature of the Feature View's left Feature Group when it declares one, and by commit time otherwise.
The `event_time` parameter of `create_scheduled_statistics` and `create_feature_monitoring` overrides that default for the whole configuration, detection and reference windows alike.
Pass the name of a timestamp, date or epoch feature from any Feature Group in the query, or `False` to select rows by commit time.
With event time the joined Feature Groups contribute their current rows, whereas with commit time the same commit interval is applied to every Feature Group in the query.

=== "Python"

```python
# windows over a time feature of a joined Feature Group
fm_monitoring_config = trans_fv.create_feature_monitoring(
name="trans_fv_amount_monitoring_by_event_time",
event_time="datetime",
)

# windows over the time the rows were written (commit time)
fm_monitoring_config = trans_fv.create_feature_monitoring(
name="trans_fv_amount_monitoring_by_commit",
event_time=False,
)
```

See [Time basis](../feature_monitoring/scheduled_statistics.md#time-basis) for how the two bases differ.

### Step 6: (Optional) Define a reference window

When setting up feature monitoring for a Feature View, the reference can be either a reference window of feature data or a training dataset.
Expand Down Expand Up @@ -251,6 +276,7 @@ A Feature View can also monitor the inference data of a model served in producti
This is the feature-view entry point to [Model Monitoring](../../mlops/model_monitoring/index.md).

It targets the feature view's logging feature group, so feature logging must be enabled with `feature_view.enable_logging()`, and filters the detection window by the given model name and version.
The windows select inference rows by their `log_time`, the time the prediction was logged.
The reference defaults to the training dataset version used to train the model.

=== "Python"
Expand Down
Loading