Looks like currently solution to update spark-stream jars/configurations is to
1) save current Kafka offsets somewhere (say zookeeper) 2) shutdown the cluster and restart 3) connect to Kafka with previously saved offset Assuming we're reading from Kafka which provides nice persistence and read-from-offset feature. Is there a better solution? Is it possible to reload jars dynamically? Resize cluster size dynamically (SPARK-3174)? A reasonable ask is to have a strategy to update or restart executors gracefully. Something like https://issues.apache.org/jira/browse/STORM-167 Any suggestion? -- Jianshi Huang LinkedIn: jianshi Twitter: @jshuang Github & Blog: http://huangjs.github.com/
