Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami merged PR #4195: URL: https://github.com/apache/solr/pull/4195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on PR #4195: URL: https://github.com/apache/solr/pull/4195#issuecomment-4113138567 All review comments have been addressed. I'll go ahead and merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2972935828 ## solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc: ## @@ -319,7 +319,7 @@ The number of bits to use for each quantized dimension value Accepted values: 4 (half byte) or 7 (unsigned byte). `NOTE:` Existing Solr 10 schemas may still contain legacy scalar-quantization parameters such as -`confidenceInterval`, `dynamicConfidenceInterval`, or `compress`. They are accepted for backward +`confidenceInterval`, `dynamicConfidenceInterval`, or `compress`. They are deprecated (i.e. parsed and ignored but not used) for backward Review Comment: Oof! I completely botched the language there during a previous refactor. Fixed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2972830093 ## solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc: ## @@ -285,3 +285,15 @@ Attention: * The `llm` module has been renamed to `language-models`. * The HNSW parameters `hnswMaxConnections` and `hnswBeamWidth` have been renamed to `hnswM` and `hnswEfConstruction`, respectively, so they must be updated accordingly in the schema.xml file. + +== Solr 10.1 + +=== Lucene Codec Change + +Solr 10.1 upgrades the underlying Lucene library from 10.3 to 10.4, which introduces a new index codec (`Lucene104`). +New index segments will be written using the `Lucene104` codec format. +Existing segments written with older codecs (e.g. `Lucene103`) will continue to be readable. + +WARNING: After upgrading to Solr 10.1, downgrading to an earlier Solr 10.0.x version may fail because the older version does not include the `Lucene104` codec needed to read the newly written segments. +If you require the ability to roll back, back up your indexes before upgrading. +A full reindex would be needed to downgrade after new segments have been written in Solr 10.1. Review Comment: I meant to communicate that IF there is no backup and you need to downgrade, a full reindex from source would be needed. Agree the missing "if no backup" pretext makes the sentence confusing. Will fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2972830093 ## solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc: ## @@ -285,3 +285,15 @@ Attention: * The `llm` module has been renamed to `language-models`. * The HNSW parameters `hnswMaxConnections` and `hnswBeamWidth` have been renamed to `hnswM` and `hnswEfConstruction`, respectively, so they must be updated accordingly in the schema.xml file. + +== Solr 10.1 + +=== Lucene Codec Change + +Solr 10.1 upgrades the underlying Lucene library from 10.3 to 10.4, which introduces a new index codec (`Lucene104`). +New index segments will be written using the `Lucene104` codec format. +Existing segments written with older codecs (e.g. `Lucene103`) will continue to be readable. + +WARNING: After upgrading to Solr 10.1, downgrading to an earlier Solr 10.0.x version may fail because the older version does not include the `Lucene104` codec needed to read the newly written segments. +If you require the ability to roll back, back up your indexes before upgrading. +A full reindex would be needed to downgrade after new segments have been written in Solr 10.1. Review Comment: I meant to communicate that IF there is no backup and you need to downgrade, a full reindex would be needed. Agree the missing "if no backup" pretext makes the sentence confusing. Will fix. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
dsmiley commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2972795257 ## solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc: ## @@ -285,3 +285,15 @@ Attention: * The `llm` module has been renamed to `language-models`. * The HNSW parameters `hnswMaxConnections` and `hnswBeamWidth` have been renamed to `hnswM` and `hnswEfConstruction`, respectively, so they must be updated accordingly in the schema.xml file. + +== Solr 10.1 + +=== Lucene Codec Change + +Solr 10.1 upgrades the underlying Lucene library from 10.3 to 10.4, which introduces a new index codec (`Lucene104`). +New index segments will be written using the `Lucene104` codec format. +Existing segments written with older codecs (e.g. `Lucene103`) will continue to be readable. + +WARNING: After upgrading to Solr 10.1, downgrading to an earlier Solr 10.0.x version may fail because the older version does not include the `Lucene104` codec needed to read the newly written segments. +If you require the ability to roll back, back up your indexes before upgrading. +A full reindex would be needed to downgrade after new segments have been written in Solr 10.1. Review Comment: This sentence reads clearly yet I think it's message is confusing given what was said in the sentences before. For example, if we do backup-restore, why would a full reindex be needed? And you just said there may be errors for Solr 10 reading the indexes so, how would re-indexing (or any indexing or anything at all) work once you've seen such errors? You could drop this sentence entirely and I think we've already communicated what's needed. ## solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc: ## @@ -285,3 +285,15 @@ Attention: * The `llm` module has been renamed to `language-models`. * The HNSW parameters `hnswMaxConnections` and `hnswBeamWidth` have been renamed to `hnswM` and `hnswEfConstruction`, respectively, so they must be updated accordingly in the schema.xml file. + +== Solr 10.1 + +=== Lucene Codec Change + +Solr 10.1 upgrades the underlying Lucene library from 10.3 to 10.4, which introduces a new index codec (`Lucene104`). +New index segments will be written using the `Lucene104` codec format. +Existing segments written with older codecs (e.g. `Lucene103`) will continue to be readable. Review Comment: Suggest simpler: Older segments will continue to be readable. ## solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc: ## @@ -319,7 +319,7 @@ The number of bits to use for each quantized dimension value Accepted values: 4 (half byte) or 7 (unsigned byte). `NOTE:` Existing Solr 10 schemas may still contain legacy scalar-quantization parameters such as -`confidenceInterval`, `dynamicConfidenceInterval`, or `compress`. They are accepted for backward +`confidenceInterval`, `dynamicConfidenceInterval`, or `compress`. They are deprecated (i.e. parsed and ignored but not used) for backward Review Comment: ignored implies not used. I suggest slightly simpler wording: > They are now deprecated and ignored. New schemas should ... You might even drop the "New schemas should..." sentence as it seems obvious (to me). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2972681355 ## dev-docs/lucene-upgrade.md: ## Review Comment: Ah that one! Added changes in `major-changes-in-solr-10.adoc` and `Lucene-upgrade.md` to that effect. Thanks. ## dev-docs/lucene-upgrade.md: ## Review Comment: Ah that one! Added changes in `major-changes-in-solr-10.adoc` and `lucene-upgrade.md` to that effect. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2972681355 ## dev-docs/lucene-upgrade.md: ## Review Comment: Ah that one! Added changes in major-changes-in-solr-10.adoc and Lucene-upgrade.md to that effect. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2971053975 ## dev-docs/lucene-upgrade.md: ## Review Comment: I don't think I remember a discussion thread regarding this. Quick search on the dev list wasn't helpful either. No mention of codec changes in the current `solr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc` either. Do you have a link to the discussion ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
dsmiley commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2972327858 ## dev-docs/lucene-upgrade.md: ## Review Comment: Here's the [message](https://lists.apache.org/thread/olvrx0fn8475p0nlwxx4g05yktk16pvr) I wrote in response to Jan's thread. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on PR #4195: URL: https://github.com/apache/solr/pull/4195#issuecomment-4105649951 Thank you for the review @dsmiley and @cpoerschke. I have incorporated all the comments except one by David (pending clarification). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2971053975 ## dev-docs/lucene-upgrade.md: ## Review Comment: I don't think I remember a discussion thread regarding this. Quick search on the dev list wasn't helpful either. No mention of codec changes in the current `olr/solr-ref-guide/modules/upgrade-notes/pages/major-changes-in-solr-10.adoc` either. Do you have a link to the discussion ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2971053975 ## dev-docs/lucene-upgrade.md: ## Review Comment: I don't think I remember a discussion thread regarding this. Quick search on the dev list wasn't helpful either. Do you have a link to the discussion ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
cpoerschke commented on code in PR #4195: URL: https://github.com/apache/solr/pull/4195#discussion_r2966955348 ## solr/solr-ref-guide/modules/query-guide/pages/dense-vector-search.adoc: ## @@ -318,50 +318,10 @@ The number of bits to use for each quantized dimension value + Accepted values: 4 (half byte) or 7 (unsigned byte). -`confidenceInterval`:: -+ -[%autowidth,frame=none] -|=== -s|Optional |Default: `dimension-scaled` -|=== -+ -Statistically, outlier values are rarely meaningfully relevant to searches, so to increase the size of each bucket for -quantization (and therefore information gain) we can scale the quantization intervals to the middle n % of values and place the remaining -outliers in the outermost intervals. -+ -For example: 0.9 means scale interval sizes to the middle 90% of values -+ -If this param is omitted a default is used; scaled to the number of dimensions according to `1-1/(vector_dimensions + 1)` -+ -Accepted values: `FLOAT32` (within 0.9 and 1.0) - -`dynamicConfidenceInterval`:: -+ -[%autowidth,frame=none] -|=== -s|Optional |Default: `false` -|=== -+ -If set to true, enables dynamically determining confidence interval (per dimension) by sampling values each time a merge occurs. -+ -`NOTE: when this is enabled, it will take precedence over any value configured for confidenceInterval` -+ -Accepted values: `BOOLEAN` - -`compress`:: -+ -[%autowidth,frame=none] -|=== -s|Optional |Default: `false` -|=== -+ -If set to true, this will further pack multiple dimension values within a one byte alignment. This further decreases the -quantized vector disk storage size by 50% at some decode penalty. This does not affect the raw vector which is always -preserved when `stored` is true. -+ -`NOTE: this can only be enabled when bits=4` -+ -Accepted values: `BOOLEAN` +`NOTE:` Existing Solr 10 schemas may still contain legacy scalar-quantization parameters such as +`confidenceInterval`, `dynamicConfidenceInterval`, or `compress`. They are accepted for backward Review Comment: ```suggestion `confidenceInterval`, `dynamicConfidenceInterval`, or `compress`. They are deprecated (i.e. parsed and ignored but not used) for backward ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
cpoerschke commented on code in PR #4195:
URL: https://github.com/apache/solr/pull/4195#discussion_r2966943883
##
solr/core/src/java/org/apache/solr/schema/ScalarQuantizedDenseVectorField.java:
##
@@ -87,22 +90,17 @@ public void init(IndexSchema schema, Map
args) {
if (ofNullable(args.remove(DYNAMIC_CONFIDENCE_INTERVAL_PARAM))
.map(Boolean::parseBoolean)
.orElse(false)) {
- this.confidenceInterval =
Lucene99ScalarQuantizedVectorsFormat.DYNAMIC_CONFIDENCE_INTERVAL;
+ this.confidenceInterval = 0f;
}
super.init(schema, args);
}
@Override
public KnnVectorsFormat buildKnnVectorsFormat() {
-return new Lucene99HnswScalarQuantizedVectorsFormat(
-getHnswM(),
-getHnswEfConstruction(),
-DEFAULT_NUM_MERGE_WORKER,
-getBits(),
-useCompression(),
-getConfidenceInterval(),
Review Comment:
Perhaps reduce visibility of the accessors and/or mark them deprecated.
```
- public boolean useCompression() {
+ @Deprecated
+ @VisibleForTesting
+ boolean useCompression() {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
cpoerschke commented on code in PR #4195:
URL: https://github.com/apache/solr/pull/4195#discussion_r2966896209
##
solr/core/src/java/org/apache/solr/schema/BinaryQuantizedDenseVectorField.java:
##
@@ -17,12 +17,14 @@
package org.apache.solr.schema;
import org.apache.lucene.codecs.KnnVectorsFormat;
-import
org.apache.lucene.codecs.lucene102.Lucene102HnswBinaryQuantizedVectorsFormat;
+import
org.apache.lucene.codecs.lucene104.Lucene104HnswScalarQuantizedVectorsFormat;
+import
org.apache.lucene.codecs.lucene104.Lucene104ScalarQuantizedVectorsFormat.ScalarEncoding;
public class BinaryQuantizedDenseVectorField extends DenseVectorField {
@Override
public KnnVectorsFormat buildKnnVectorsFormat() {
-return new Lucene102HnswBinaryQuantizedVectorsFormat(getHnswM(),
getHnswEfConstruction());
+return new Lucene104HnswScalarQuantizedVectorsFormat(
+ScalarEncoding.SINGLE_BIT_QUERY_NIBBLE, getHnswM(),
getHnswEfConstruction());
Review Comment:
ref. https://github.com/apache/lucene/pull/15353
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
dsmiley commented on code in PR #4195:
URL: https://github.com/apache/solr/pull/4195#discussion_r2936759379
##
dev-docs/lucene-upgrade.md:
##
Review Comment:
Remember the discussion thread about a Lucene upgrade that includes a Codec
update, and needing to broadcast this fact in the upgrade notes page? This
markdown file should explicitly add this step.
##
solr/core/src/java/org/apache/solr/schema/ScalarQuantizedDenseVectorField.java:
##
@@ -76,6 +75,10 @@ public ScalarQuantizedDenseVectorField(
public void init(IndexSchema schema, Map args) {
this.bits =
ofNullable(args.remove(BITS_PARAM)).map(Integer::parseInt).orElse(DEFAULT_BITS);
+// Retain parsing of these ("compress", "confidenceInterval") legacy Solr
10 schema params for
+// compatibility even though
+// Lucene 10.4's scalar-quantized vector format no longer consumes them
directly. They are no-op
+// going forward.
Review Comment:
Then maybe lets mark the params deprecated with a version param of 10.1
##
solr/core/src/java/org/apache/solr/schema/ScalarQuantizedDenseVectorField.java:
##
@@ -76,6 +75,10 @@ public ScalarQuantizedDenseVectorField(
public void init(IndexSchema schema, Map args) {
this.bits =
ofNullable(args.remove(BITS_PARAM)).map(Integer::parseInt).orElse(DEFAULT_BITS);
+// Retain parsing of these ("compress", "confidenceInterval") legacy Solr
10 schema params for
+// compatibility even though
+// Lucene 10.4's scalar-quantized vector format no longer consumes them
directly. They are no-op
+// going forward.
Review Comment:
Also, use the DeprecationLog so that the user sees a warning if they provide
these params
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
-
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on PR #4195: URL: https://github.com/apache/solr/pull/4195#issuecomment-4059333470 Also adding @liangkaiwen since he worked on these quantization types for 10.0. Kevin, could you please take a look at the related changes? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on PR #4195: URL: https://github.com/apache/solr/pull/4195#issuecomment-4021671184 **ToDo**: Lucene104ScalarQuantizedVectorsFormat now defaults to 8 bits (ScalarEncoding.UNSIGNED_BYTE) instead of 7 bits in Lucene99ScalarQuantizedVectorsFormat earlier. Make that the default for ScalarQuantizedDenseVectorField too (ScalarQuantizedDenseVectorField.DEFAULT_BITS)? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on PR #4195: URL: https://github.com/apache/solr/pull/4195#issuecomment-4021512949 @dsmiley @alessandrobenedetti Requesting a review please. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on PR #4195: URL: https://github.com/apache/solr/pull/4195#issuecomment-4021455785 Additionally, Lucene104HnswScalarQuantizedVectorsFormat now supports 1,2,4,7 and 8 bits in the format as opposed to only 4 and 7 earlier. Guess that should be scoped under a separate PR with test and documentation changes instead of squashing everything together in this upgrade PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on PR #4195: URL: https://github.com/apache/solr/pull/4195#issuecomment-4021430260 Put Claude(Opus 4.6) and Codex (GPT 5.4) to work on compilation errors. Main test failures were around ScalarQuantizedDenseVectorField and BinaryQuantizedDenseVectorField due to breaking changes in Lucene104HnswScalarQuantizedVectorsFormat. Major changes: - Lucene104HnswScalarQuantizedVectorsFormat has moved to an encoding based API. It no longer accepts "confidenceInterval" or "compression" params. Hence made those no-op in ScalarQuantizedDenseVectorField and removed the same from tests and documentation. - Added a note in documentation to say that older Solr 10.x schema may contain those params but not to be used going forward. Existing schemas will continue to be supported. - There is no separate binary quantization format in Lucene 10.4. Binary quantization is now just another encoding type of Lucene104HnswScalarQuantizedVectorsFormat (encoding=ScalarEncoding.SINGLE_BIT_QUERY_NIBBLE). But we'll need to continue to expose it at the Solr level as a separate type for back compatibility. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
Re: [PR] Upgrade to Lucene 10.4 [solr]
rahulgoswami commented on PR #4195: URL: https://github.com/apache/solr/pull/4195#issuecomment-4015663187 WIP...fixing failures with DenseVectorField tests -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] - To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
