Hi Gabor, It seems like it is better to upgrade my spark version .
Are there major changes in terms of streaming from spark 2.2 to spark 2.4? PS: I am using KafkaUtils api to create steam Thanks & Regards, Renu yadav On Fri, Mar 12, 2021 at 7:25 PM Renu Yadav <yren...@gmail.com> wrote: > Thanks Gabor, > This is very useful. > > Regards, > Renu Yadav > > On Fri, Mar 12, 2021 at 5:36 PM Gabor Somogyi <gabor.g.somo...@gmail.com> > wrote: > >> Kafka client upgrade is not a trivial change which may or may not work >> since new versions can contain incompatible API and/or behavior changes. >> I've collected how Spark evolved in terms of Kafka client and there I've >> gathered the breaking changes to make our life easier. >> Have a look and based on that you can make your choice: >> https://gist.github.com/gaborgsomogyi/3476c32d69ff2087ed5d7d031653c7a9 >> >> As a general suggestion it would be best to upgrade Spark as-is because >> we've added many fixes which one can face... >> >> Hope this helps! >> >> G >> >> >> On Fri, Mar 12, 2021 at 9:45 AM Renu Yadav <yren...@gmail.com> wrote: >> >>> Hi Team, >>> I am using spark -2.2 and spark_streamin_kafka 2.2 , which is pointing >>> to kafka-client 0.10 . How can I upgrade a kafka client to kafka 2.2.0 ? >>> >>> Thanks & Regards, >>> Renu Yadav >>> >>
