Re: Introducing Comet, a plugin to accelerate Spark execution via DataFusion and Arrow

Hi Chao, Is there any example app/gist/repo which can help me use this plugin. I wanted to try out some realtime aggregate performance on top of parquet and spark dataframes. Thanks and Regards Praveen On Wed, Feb 14, 2024 at 9:20 AM Chao Sun wrote: > > Out of interest what are the

Re: Creating InMemory relations with data in ColumnarBatches

from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > > On Thu, 30 Mar 2023 at 19:56, praveen sinha > wrote: > >> Hi, >&

Creating InMemory relations with data in ColumnarBatches

Hi, I have been trying to implement InMemoryRelation based on spark ColumnarBatches, so far I have not been able to store the vectorised columnarbatch into the relation. Is there a way to achieve this without going with an intermediary representation like Arrow, so as to enable spark to do fast