jimczi commented on code in PR #16418:
URL: https://github.com/apache/lucene/pull/16418#discussion_r3669244727
##########
lucene/core/src/java/org/apache/lucene/index/IndexWriterConfig.java:
##########
@@ -324,6 +338,43 @@ public boolean getReaderPooling() {
return readerPooling;
}
+ /**
+ * When enabled, a doc-values update that only sets values (no removals) is
written as a sparse
+ * "delta" generation holding just the updated documents and overlaid on the
existing column at
+ * read time, rather than rewriting the whole column. This trades some read
cost for much lower
+ * write amplification on frequently updated fields. Enabled by default;
pass {@code false} to
+ * keep the classic full-column rewrite.
+ *
+ * <p>A commit whose segments carry sparse delta generations is written at a
bumped segments-file
+ * version, so a Lucene version that predates this feature refuses to open
it rather than
+ * misreading a delta as the whole column. A segment merge (including a
force-merge) flattens the
+ * overlay back to a plain column, so once all overlays are merged away
older versions can read
+ * the index again. Only takes effect when IndexWriter is first created.
+ *
+ * @lucene.experimental
+ */
+ public IndexWriterConfig setIncrementalDocValuesUpdates(boolean
incrementalDocValuesUpdates) {
Review Comment:
Done in 3eb7cf1: collapsed into a single `setMaxDocValuesOverlays(int)`, 0
disables the feature and any value > 0 enables it with that overlay depth.
##########
lucene/core/src/java/org/apache/lucene/index/LiveIndexWriterConfig.java:
##########
@@ -90,6 +90,12 @@ public class LiveIndexWriterConfig {
/** True if calls to {@link IndexWriter#close()} should first do a commit. */
protected boolean commitOnClose = IndexWriterConfig.DEFAULT_COMMIT_ON_CLOSE;
+ /** True if set-only doc-values updates are written as sparse delta
generations. */
+ protected volatile boolean incrementalDocValuesUpdates;
Review Comment:
Good catch, dropped it in 3eb7cf1. The setter is on IndexWriterConfig
(creation-only), so the write happens-before the flush thread reads it via the
writer construction, no volatile needed.
##########
lucene/core/src/java/org/apache/lucene/index/IndexWriterConfig.java:
##########
@@ -324,6 +338,43 @@ public boolean getReaderPooling() {
return readerPooling;
}
+ /**
+ * When enabled, a doc-values update that only sets values (no removals) is
written as a sparse
+ * "delta" generation holding just the updated documents and overlaid on the
existing column at
+ * read time, rather than rewriting the whole column. This trades some read
cost for much lower
+ * write amplification on frequently updated fields. Enabled by default;
pass {@code false} to
+ * keep the classic full-column rewrite.
+ *
+ * <p>A commit whose segments carry sparse delta generations is written at a
bumped segments-file
Review Comment:
Done in 3eb7cf1: the public API and docs now use "overlay" instead of
"generation".
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
