Re: Experience re OpenAI embeddings in combination with Lucene vector search

[Michael Wechner]

Hi Alessandro

I am mainly interested in detecting similarity, for example whether the 
following two sentences are similar resp. likely to mean the same thing

"How old are you?"
"What is your age?"

and that the following two sentences are not similar, resp. do not mean 
the same thing

"How old are you this year?"
"How old have you been last year?"

But also performance or how OpenAI embeddings compare for example with 
SBERT (https://sbert.net/docs/usage/semantic_textual_similarity.html)

Thanks

Michael



Am 12.02.22 um 20:41 schrieb Alessandro Benedetti:
Hi Michael, experience to what extent?
We have been exploring the area for a while given we contributed the 
first neural search milestone to Apache Solr.
What is your curiosity? Performance? Relevance impact? How to 
integrate it?
Regards

On Fri, 11 Feb 2022, 22:38 Michael Wechner, 
<michael.wech...@wyona.com> wrote:

    Hi

    Does anyone have experience using OpenAI embeddings in combination
    with Lucene vector search?

    https://beta.openai.com/docs/guides/embeddings|

    for example comparing performance re vector size

    ||https://api.openai.com/v1/engines/|||text-similarity-ada-001|/embeddings

    and

    
||https://api.openai.com/v1/engines/||||text-similarity-davinci-001||/embeddings

    ?

    ||
    |Thanks

    Michael