Re: java.io.NotSerializableException about SparkStreaming

2017-10-17 Thread 高佳翔
Hi Shengshan, In first code, ‘newAPIJobConfiguration’ is sharing across all rdds. So, it should be serializable. In second code, each rdd creates a new ‘mytest_config’ object and an individual ‘newAPIJobConfiguration’ instead of sharing the same object. So it can be non-serializable. If it’s

java.io.NotSerializableException about SparkStreaming

2017-10-17 Thread Shengshan Zhang
Hello guys! java.io.NotSerializableException troubles me a lot when i process data with spark. ``` val hbase_conf = HBaseConfiguration.create() hbase_conf.set("hbase.zookeeper.property.clientPort", "2181") hbase_conf.set("hbase.zookeeper.quorum", "hadoop-zk0.s.qima-inc.com