andygrove commented on code in PR #7226: URL: https://github.com/apache/datafusion-comet/pull/7226#discussion_r4388828092
##########
docs/source/user-guide/latest/iceberg.md:
##########
@@ -485,9 +485,11 @@ A write is eligible only when ALL of the following hold:
| `write.parquet.page-version`
| unset or `v1`
|
| `write.parquet.shred-variants`
| unset or
`false` (Spark 4.x / Iceberg 1.11 resolve this into every parquet write)
|
| `write.parquet.variant-inference-buffer-size`
| any value
(only meaningful when shredding, which is gated)
|
-| `write.parquet.bloom-filter-enabled.column.<col>`
| unset or
`false`
|
| `write.metadata.metrics.*`
| any value
(manifest metrics are re-derived on the JVM with Iceberg's own logic)
|
| `write.spark.fanout.enabled`
| any value (the
native writer implements both clustered and fanout modes)
|
+| `write.parquet.bloom-filter-enabled.column.<col>`
| `true` or
`false`; native eligibility requires identifiable Iceberg 1.5+. Unknown
runtimes and unsupported legacy settings use Java. On Iceberg 1.11+, an
explicit NDV enables the column even when this value is `false`. Enabled
columns use the native writer when
`spark.comet.write.iceberg.bloomFilterFolding.enabled=true`
|
+| `write.parquet.bloom-filter-fpp.column.<col>` /
`write.parquet.bloom-filter-ndv.column.<col>`
| Iceberg 1.5 ignores FPP and NDV; 1.8–1.10 interpret FPP but
ignore NDV; 1.11 interprets both. Explicit FPP on 1.6/1.7 uses Java. For
interpreted properties on a column named by an `enabled` property, FPP is
finite and strictly between 0 and 1 (default `0.01`), and NDV is a positive
Java long no greater than `Long.MAX_VALUE / 8`.
|
Review Comment:
This row says what 1.5, 1.8 to 1.10 and 1.11 do with NDV, but not 1.6 and
1.7. Both of those ignore NDV and only interpret FPP through the independent
validation pass. Could you say so here, so the row covers every version in the
runtime table below?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
