Hi Till & all, We are using KNN search in a SOLR cloud 9.0.0 cluster. We had played around with Spotify's Annoy but, apart from obviously # of documents vs memory size issues, the really big thing about KNN in SOLR is that we can mix queries (I'm still not 100% on how this really works because we have to really push TopK when mixing KNN queries with other variables but it does mostly work). We can't really go wild on this yet because it's just too CPU intensive right now (we;ve got enough RAM, 64GB, in every EC2 instance to cache the entire 2.5M documents which must be helping). The embeddings I'm holding in the dense vector fields are 512 in size so I'm thinking about trying to reduce to something like 64 using PCA maybe (I'm a bit afraid of losing accuracy but maybe the reduction will make it feasible to use KNN for more mainstream searches). Our embeddings are entirely inferenced outside of SOLR and then included in the SOLR record.
Derek On Thu, Feb 23, 2023 at 9:44 PM Till Kinstler <[email protected]> wrote: > Hi, > > I've been playing with "neural" / dense vector search in Solr 9 a bit > and find it very promising. > Currently I am calculating the vectors outside of Solr at indexing and > search time with a bunch of scripts using NLP models (text in, vectors > out...). Especially at search time, that's not exactly a handy solution, > because every client application would have to do this (or some sort of > proxy application between client applications and Solr, that would > manipulate requests (search terms out, vector in) on their way to Solr). > That's ok for my very basic prototype, but nothing else. > How are others solving this? Are there any best practices? Or even plans > to make Solr talk directly to ML models? > In Solr's traditional logic, I would imagine something like an analyzer, > that does the "dense vector creation" at indexing and search time. It > would have to use a ML model, pass data/searches in, get vectors out and > put them into a DenseVectorField. Just as traditional analyzers work. > The model could be a configurable ONNX model? > Is someone working on something like this? (I only found some related > comments in https://github.com/apache/solr/pull/1213) > > Till > > -- > Till Kinstler > Verbundzentrale des Gemeinsamen Bibliotheksverbundes (VZG) > Platz der Göttinger Sieben 1, D 37073 Göttingen > [email protected], http://www.gbv.de/ > -- -- Derek Conniffe Harvey Software Systems Ltd T/A HSSL Telephone (IRL): 086 856 3823 Telephone (US): (650) 443 8285 Skype: dconnrt Email: [email protected] *Disclaimer:* This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please delete it (if you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited). *Warning*: Although HSSL have taken reasonable precautions to ensure no viruses are present in this email, HSSL cannot accept responsibility for any loss or damage arising from the use of this email or attachments. P For the Environment, please only print this email if necessary.
