GitHub user zuozhiw added a comment to the discussion: Ambient "operator recommender" — predictive next-operator suggestions on the canvas
I like a stateless backend api just takes in the current state and returns recommendations. Stateful things are harder to get right. Back on to how this can be implemented, I'm mostly leaning towards LLM based recommendation. I'm not a huge fan of training a custom model for recommendations, if we use LLMs, we get a free ride as model capabilities increase. My main concern is that if the speed of LLMs are fast enough, also if we bring LLMs to the table, we might need to maintain the conversation history somewhere for it to be more cache friendly. I would really recommend looking into how cursor does code autocomplete. I know that cursor trains its own small mod (I think it's a small LLM model) and it's both fast and accurate. If we can get some more context on how cursor's autocomplete work today I'll be more comfortable. Also I would really like to see if there are any online articles discussing using LLMs to run code autocomplete. GitHub link: https://github.com/apache/texera/discussions/5240#discussioncomment-17174851 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
