Re: How to Increase max vector size?

2022-02-17 Thread Michael Wechner
Not at the moment :-) I am using Lucene's vector search for https://ukatie.com to detect duplicated questions, whereas I am currently refactoring it, such that you can connect Katie with your own similarity search implementation, whereas I have done a very first prototype of a connector for

Re: How to Increase max vector size?

2022-02-16 Thread Michael Sokolov
Fair enough - are you planning to offer such a service;) sounds exciting! -Mike On Tue, Feb 15, 2022 at 6:00 PM Michael Wechner wrote: > true :-) when you are the one controlling the input of vectors, then a > method to disable the maximum limit would be sufficient. > > But I could imagine

Re: How to Increase max vector size?

2022-02-15 Thread Michael Wechner
true :-) when you are the one controlling the input of vectors, then a method to disable the maximum limit would be sufficient. But I could imagine when you offer Lucene as a service where people can for example configure their own "sentence embedding models" and you would like to offer a

Re: How to Increase max vector size?

2022-02-15 Thread Michael Sokolov
I don't think it makes sense to have a static variable maximum that you can change by calling a method. What purpose would it serve? On Tue, Feb 15, 2022, 2:39 PM Michael Wechner wrote: > Hi Alessandro > > No, I have not created a Jira ticket, but I would be happy to create one, > just let me

Re: How to Increase max vector size?

2022-02-15 Thread Michael Wechner
Hi Alessandro No, I have not created a Jira ticket, but I would be happy to create one, just let me know or please feel free to create one. I understand the concerns about the limits in general and I think it makes sense to have a default max dimensions limit, but I could imagine it needs

Re: How to Increase max vector size?

2022-02-15 Thread Alessandro Benedetti
Hi Michael, let's create a Jira ticket to use a higher value(if you haven't already). I would be happy to consider the patch/or do it myself but after 10/03. Once the pull request is ready (including the Javadoc documentation that clearly states that if you go above X it's at your own risk), we'll

How to Increase max vector size?

2022-02-12 Thread Michael Wechner
Hi I just tried to test the OpenAI model "text-similarity-davinci-001" with 12288 dimensions and receive the following error java.lang.IllegalArgumentException: vector numDimensions must be <= VectorValues.MAX_DIMENSIONS (=1024); got 12288     at