[
https://issues.apache.org/jira/browse/SOLR-17045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17777479#comment-17777479
]
Chris M. Hostetter commented on SOLR-17045:
-------------------------------------------
NOTE: I created this issue mainly for tracking & linking purposes, to help
users who may search for the problem when encountering the bug in the wild.
I'm not sure that there is much that can be done in the scope of a "bug fix"
for this problem.
I think the best long term solution is to change Solr to use
{{SchemaCodecFactory}} as it's implicit default -- but i will file & link a
distinct issue for that change since it has larger implications (and would
probably not be suitable for backporting to include in a bug-fix release)
> DenseVectorField w/ vectorDimension > 1024 no longer works by default
> ---------------------------------------------------------------------
>
> Key: SOLR-17045
> URL: https://issues.apache.org/jira/browse/SOLR-17045
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Affects Versions: 9.3
> Reporter: Chris M. Hostetter
> Priority: Major
>
> Starting with Solr 9.4, configuring a {{DenseVectorField}} w/
> {{vectorDimension}} > 1024 no longer works by default. There is no error on
> startup, but when indexing you'll get errors like...
> {noformat}
> 2> => org.apache.solr.common.SolrException: Exception writing
> document id 2 to the index; possible analysis error: Field [vector]vector's
> dimensions must be <= [1024]; got 1600
> 2> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:329)
> 2> org.apache.solr.common.SolrException: Exception writing document id 2 to
> the index; possible analysis error: Field [vector]vector's dimensions must be
> <= [1024]; got 1600
> 2> at
> org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:329)
> ~[solr-core-9.4.0.jar:9.4.0 71e101bb37497f730078d9afe1991b60d10bfe96 -
> stillalex - 2023-10-10 19:10:39]
> 2> at
> org.apache.solr.update.processor.RunUpdateProcessorFactory$RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:76)
> ~[solr-core-9.4.0.jar:9.4.0 71e101bb37497f730078d9afe1991b60d10bfe96 -
> stillalex - 2023-10-10 19:10:39]
> ...
> 2> Caused by: java.lang.IllegalArgumentException: Field [vector]vector's
> dimensions must be <= [1024]; got 1600
> 2> at
> org.apache.lucene.index.IndexingChain.validateMaxVectorDimension(IndexingChain.java:843)
> ~[lucene-core-9.8.0.jar:9.8.0 d914b3722bd5b8ef31ccf7e8ddc638a87fd648db -
> 2023-09-21 21:57:47]
> ...
> {noformat}
> This is because Lucene 9.8 moved the dimension size limitation to the codec –
> and while Solr 9.4's {{SchemaCodecFactory}} was updated to implement a
> per-field {{SolrDelegatingKnnVectorsFormat}} that respected the
> {{vectorDimension}} configured for each {{DenseVectorField}} *the
> {{SchemaCodecFactory}} _is not implicitly used by default in Solr_* – nor is
> it explicitly configured in the {{_default}} configset.
> {panel:title=Known Work Around}
> Existing {{DenseVectorField}} who encounter this error when upgrading to Solr
> >= 9.4, or new user attempting to use {{DenseVectorField}} with
> {{vectorDimension}} > 1024, need to explicitly configure
> {{SchemaCodecFactory}} in the solrconfig.xml for each collection.
> {panel}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@solr.apache.org
For additional commands, e-mail: issues-h...@solr.apache.org
