Re: convert JavaRDD<List> to JavaRDD

2017-04-05 Thread Jiang Jacky
There is no flattop just flatMap > On Apr 5, 2017, at 12:24 PM, Vinay Parekar wrote: > > I think flattop() will be helpful in this case . Correct me if I am wrong. > > From: Hamza HACHANI > Date: Wednesday, April 5, 2017 at 3:43 AM > To:

Re: strange behavior of spark 2.1.0

2017-04-02 Thread Jiang Jacky
lt to tell without details. > I believe one of the executors dies because of OOM or some Runtime Exception > (some unforeseen dirty data row). > Less probable is GC stop-the-world pause when incoming message rate increases > drastically. > > >> On Saturday, April 1, 201

strange behavior of spark 2.1.0

2017-04-01 Thread Jiang Jacky
Hello, Guys I am running the spark streaming in 2.1.0, the scala version is tried on 2.11.7 and 2.11.4. And it is consuming from JMS. Recently, I have get the following error *"ERROR scheduler.ReceiverTracker: Deregistered receiver for stream 0: Stopped by driver"* *This error can be occurred