Hey Michael, In short, it's being worked on :).
Could you point to the LinkedIN post? Is Nils talking about the model output quantized output or that their default output is easily compressible because of how the embeddings are built? I have done a bad job of linking back against that original issue the work that is being done: The initial implementation of adding int8 (really, its int7 because of signed bytes...): https://github.com/apache/lucene/pull/12582 A significant refactor to make adding new quantized storage easier: https://github.com/apache/lucene/pull/12729 Lucene already supports folks just giving it signed `byte[]` values. But this only gets so far. The additional work should get Lucene further down the road towards better lossy-compression for vectors. Thanks! Ben On Sat, Nov 4, 2023 at 4:07 AM Michael Wechner <michael.wech...@wyona.com> wrote: > Hi > > If I understand correctly some devs are working on introducing > quantization for vector search or at least considering it > > https://github.com/apache/lucene/issues/12497 > > Just being curious what is the status on this resp. is somebody working on > this actively? > > > It came to my mind, because Cohere recently made their new embedding model > "Embed v3" available > > https://txt.cohere.com/introducing-embed-v3/ > > whereas IIUC, Cohere intends to also provide embeddings optimized for > compression soon. > > Nils Reimers recently wrote on LinkedIn: > > ---- > "... what we see on the BioASQ dataset: > 4x - 99.99% search quality > 16x - 99.9% search quality > 32x - 95% search quality > 64x - 85% search quality > But it requires that the respective vector DB supports these modes, what > we currently work on with partners." > ---- > > This might be interesting for Lucene as well, resp. I am not sure whether > somebody at Lucene is already working on something like this. > > Thanks > > Michael >
