[FSTORE-1903] Backwards compatibility and upgrade tests for Feature Logging

docs/user_guides/fs/feature_view/feature_logging.md

@@ -11,7 +11,7 @@ You can log either transformed or/and untransformed features values.
 ### Enabling Feature Logging
 
 To enable logging, set `logging_enabled=True` when creating the feature view.
-Two feature groups will be created for storing transformed and untransformed features, but they are not visible in the UI.
+A single feature group storing both transformed and untransformed features will be created, but it is not visible in the UI.
 The logged features will be written to the offline feature store every hour by scheduled materialization jobs which are created automatically.
 
 ```python
@@ -200,7 +200,8 @@ feature_view.resume_logging()
 
 Besides the scheduled materialization job, you can materialize logs from Kafka to the offline store on demand.
 This does not pause the scheduled job.
-By default, it materializes both transformed and untransformed logs, optionally specifying whether to materialize transformed (transformed=True) or untransformed (transformed=False) logs.
+Feature views that still have the pre-4.6 pair of logging feature groups (see the upgrade compatibility section below) materialize both logs by default, and `transformed=True` or `transformed=False` selects one of them.
+Feature views with the combined layout have a single log, so the `transformed` argument makes no difference there.
 
 ### Materialize Logs
 
@@ -209,26 +210,50 @@ Materialize logs and optionally wait for the process to complete.
 ```python
 # Materialize logs and wait for completion
 materialization_result = feature_view.materialize_log(wait=True)
-# Materialize only transformed log entries
+# Pre-4.6 pair of logging feature groups only: materialize the transformed log
 feature_view.materialize_log(wait=True, transformed=True)
 ```
 
 ## Deleting Logs
 
 When log data is no longer needed, you might want to delete it to free up space and maintain data hygiene.
-This operation deletes the feature groups and recreates new ones.
+This operation deletes the logging feature group and recreates a new one.
 Scheduled materialization job and log timeline are reset as well.
 
 ### Delete Logs
 
-Remove all log entries (both transformed and untransformed logs), optionally specifying whether to delete transformed (transformed=True) or untransformed (transformed=False) logs.
+Remove all log entries.
+On a feature view that still has the pre-4.6 pair of logging feature groups, `delete_log()` deletes both and recreates the log in the combined layout; passing `transformed=True` or `transformed=False` does the same, because the pair can only be replaced as a whole.
+On the combined layout, `delete_log(transformed=True)` has nothing to delete and does nothing.
 
 ```python
 # Delete all log entries
 feature_view.delete_log()
+```
+
+## Upgrade Compatibility with Pre-4.6 Feature Logging
+
+Hopsworks 4.6 changed the feature logging layout: transformed and untransformed features are logged into one combined feature group instead of a separate pair, labels are logged as `predicted_<label>` columns, and the model identity is stored in `model_name` and `model_version` columns instead of a single `hsml_model` column.
+
+Feature views that enabled logging before the upgrade keep their original pair of logging feature groups unchanged.
+Model deployments and batch jobs that still run a pre-4.6 client keep logging to those feature views without any code change or downtime.
+Clients from 4.6 onwards also keep working against them: predictions and the model identity are written into the original columns, and `feature_view.read_log(model_name=..., model_version=...)` filters on the original `hsml_model` column.
+Calling `feature_view.delete_log()` on such a feature view deletes the original pair and recreates the logs in the combined layout, because the deleted logs are recreated with the current schema.
+
+Enabling logging on a feature view created after the upgrade requires a 4.6 or later client.
+A pre-4.6 client cannot produce the combined layout, so its `feature_view.log(...)` calls against such feature views fail instead of writing incomplete rows.
+
+The first positional parameter of `feature_view.log()` changed in 4.6 from `untransformed_features` to `logging_data`.
+On feature views with the pre-4.6 pair of logging feature groups, positional calls written for the old signature are detected and keep working.
+
+```python
+# Positional call style written for pre-4.6 clients, still working on
+# feature views that predate the upgrade:
+feature_view.log(features, predictions)
 
-# Delete only transformed log entries
-feature_view.delete_log(transformed=True)
+# Equivalent call that works on every feature view; use this form when
+# migrating code to a 4.6 or later client:
+feature_view.log(features, predictions=predictions)
 ```
 
 ## Summary