Unfortunately, Spark standalone mode doesn't support rolling update. All Spark components (master, worker, driver) must be updated to the same version. When using HA mode, the states persisted in zookeeper (or files if not using zookeeper) need to be cleaned because they are not compatible between versions.
Best Regards, Ryan On Wed, Jan 29, 2020 at 2:12 AM bsikander <behro...@gmail.com> wrote: > Anyone? > This question is not regarding my application running on top of Spark. > The question is about the upgrade of spark itself from 2.2 to 2.4. > > I expected atleast that spark would recover from upgrades gracefully and > recover its own persisted objects. > > > > -- > Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org > > -- Best Regards, Ryan
