Hi, my comments were for purposes of SQL, also most of other technologies like snowflake, and Redshift, and using KSQL directly to other sinks quite easily, without massive engineering, infact databricks is trying to play a catchup game in this market by coming out with GIU based ETL tools :)
Perhaps you should try other systems in the market first, that will give an unbiased view of databricks and SPARK being just over glamourised tool. The hope of extending SPARK with a separate easy to use query engine for deep learning and other AI systems is gone now with Ray, SPARK community now just defends the lack of support, and direction in this matter largely, which is a joke. Thanks and Regards, Gourav Sengupta On Mon, Aug 1, 2022 at 4:54 AM pengyh <pen...@web.de> wrote: > > I don't think so. we were using spark integarted with Kafka for > streaming computing and realtime reports. that just works. > > > > SPARK is now just an overhyped and overcomplicated ETL tool, nothing > > more, there is another distributed AI called as Ray, which should be the > > next billion dollar company instead of just building those features in > > SPARK natively using a different computation engine :) >
