Denis Magda created IGNITE-6163: ----------------------------------- Summary: Upgrade to Spark 2.2.0 Key: IGNITE-6163 URL: https://issues.apache.org/jira/browse/IGNITE-6163 Project: Ignite Issue Type: Improvement Reporter: Denis Magda Priority: Blocker Fix For: 2.2
Apache Ignite's Spark integration module has to be updated to Spark 2.2.0 to avoid issues like this: https://github.com/dmagda/IgniteSparkIoT/issues/1 In short, Spark 2.1.0 and Spark 2.2.0 modules can't be mixed together, otherwise, the exceptions like this will pop up: {code} Exception in thread "main" java.lang.NoSuchMethodError: org.apache.spark.ui.SparkUI.setStreamingJobProgressListener(Lorg/apache/spark/scheduler/SparkListener;)V at org.apache.spark.streaming.ui.StreamingTab.<init>(StreamingTab.scala:41) at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:192) at org.apache.spark.streaming.StreamingContext.<init>(StreamingContext.scala:85) at org.apache.spark.streaming.api.java.JavaStreamingContext.<init>(JavaStreamingContext.scala:138) at org.apache.ignite.iot.SparkStreamerStartup.main(SparkStreamerStartup.java:71) 17/08/22 18:27:25 INFO SparkContext: Invoking stop() from shutdown hook 17/08/22 18:27:25 INFO SparkUI: Stopped Spark web UI at http://10.0.1.6:4040 {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)