Re: Accumulator value 0 in driver
Have you called collect() / count() on the RDD following flatMap() ? Cheers On Tue, Jul 21, 2015 at 8:47 AM, dlmar...@comcast.net wrote: I am using Accumulators in a JavaRDDLabeledPoint.flatMap() method. I have logged the localValue() of the Accumulator object and their values are non-zero. In the driver, after the .flatMap() method returns, the calling value() on the Accumulator yields 0. I am running 1.4.0 in yarn-client mode. Any pointers would be appreciated.
Re: Accumulator value 0 in driver
No, I have not. I will try that though, thank you. - Original Message - From: Ted Yu yuzhih...@gmail.com To: dlmarion dlmar...@comcast.net Cc: user user@spark.apache.org Sent: Tuesday, July 21, 2015 12:15:13 PM Subject: Re: Accumulator value 0 in driver Have you called collect() / count() on the RDD following flatMap() ? Cheers On Tue, Jul 21, 2015 at 8:47 AM, dlmar...@comcast.net wrote: I am using Accumulators in a JavaRDDLabeledPoint.flatMap() method. I have logged the localValue() of the Accumulator object and their values are non-zero. In the driver, after the .flatMap() method returns, the calling value() on the Accumulator yields 0. I am running 1.4.0 in yarn-client mode. Any pointers would be appreciated.
Re: Accumulator value 0 in driver
That did it, thanks. - Original Message - From: dlmar...@comcast.net To: Ted Yu yuzhih...@gmail.com Cc: user user@spark.apache.org Sent: Tuesday, July 21, 2015 1:15:37 PM Subject: Re: Accumulator value 0 in driver No, I have not. I will try that though, thank you. - Original Message - From: Ted Yu yuzhih...@gmail.com To: dlmarion dlmar...@comcast.net Cc: user user@spark.apache.org Sent: Tuesday, July 21, 2015 12:15:13 PM Subject: Re: Accumulator value 0 in driver Have you called collect() / count() on the RDD following flatMap() ? Cheers On Tue, Jul 21, 2015 at 8:47 AM, dlmar...@comcast.net wrote: I am using Accumulators in a JavaRDDLabeledPoint.flatMap() method. I have logged the localValue() of the Accumulator object and their values are non-zero. In the driver, after the .flatMap() method returns, the calling value() on the Accumulator yields 0. I am running 1.4.0 in yarn-client mode. Any pointers would be appreciated.
