diff --git a/website/docs/engine-spark/options.md b/website/docs/engine-spark/options.md index c9d9a4614a..c62a90103c 100644 --- a/website/docs/engine-spark/options.md +++ b/website/docs/engine-spark/options.md @@ -17,6 +17,7 @@ The following Spark configurations can be used to control read behavior for both | `spark.sql.fluss.scan.startup.mode` | `full` | The startup mode when reading a Fluss table. Supported values: **Note:** This option only affects Structured Streaming reads, and only `latest` mode is currently supported there. Batch reads ignore it: a plain batch read is always the full table, and a time-range batch read is requested per query (see below). | | `spark.sql.fluss.read.optimized` | `false` | If `true`, Spark will only read data from the data lake snapshot or KV snapshot, without merging log changes. This can improve read performance but may return stale data for primary key tables. | | `spark.sql.fluss.scan.poll.timeout` | `10000ms` | The timeout for the log scanner to poll records. | +| `spark.sql.fluss.scan.maxRecordsPerPartition` | (none) | The maximum number of records per Spark input partition for batch log-table reads. Accepts a positive `Long`. When configured, each bounded Fluss bucket offset range that exceeds the value is split into multiple Spark input partitions. Disabled by default; it changes scan parallelism without changing the rows returned. | ## Per-Query Read Options