Re: [Proposal] Remove max number of dimensions for KNN vectors

2023-04-12 Thread Michael Wechner
Hi Kent Great, thank you very much! Will download it later today :-) All the best Michael Am 13.04.23 um 01:35 schrieb Kent Fitch: Hi Michael (and anyone else who wants just over 240K "real world" ada-002 vectors of dimension 1536), you are welcome to retrieve a tar.gz file which contains:

Re: [Proposal] Remove max number of dimensions for KNN vectors

2023-04-12 Thread Kent Fitch
Hi Michael (and anyone else who wants just over 240K "real world" ada-002 vectors of dimension 1536), you are welcome to retrieve a tar.gz file which contains: - 47K embeddings of Canberra Times news article text from 1994 - 38K embeddings of the first paragraphs of wikipedia articles about

Re: [Proposal] Remove max number of dimensions for KNN vectors

2023-04-12 Thread Michael Sokolov
Just addressing [1] I believe there is a simple workaround. Here's a unit test demonstrating: public void testExcessivelyLargeVector() throws Exception { IndexableFieldType vector2048 = new FieldType() { @Override public int vectorDimension() { return 2048; }

Re: [Proposal] Remove max number of dimensions for KNN vectors

2023-04-12 Thread Alessandro Benedetti
My tentative of listing here only a set of proposals to then vote, has unfortunately failed. I appreciate the discussion on better benchmarking hnsw but my feeling is that this discussion is orthogonal to the limit discussion itself, should we create a separate mail thread/github jira issue for

Re: [Proposal] Remove max number of dimensions for KNN vectors

2023-04-12 Thread Michael Wechner
thank you very much for your feedback! In a previous post (April 7) you wrote you could make availlable the 47K ada-002 vectors, which would be great! Would it make sense to setup a public gitub repo, such that others could use or also contribute vectors? Thanks Michael Wechner Am