Re: Use of Accumulators

2017-11-14 Thread Kedarnath Dixit
Yes! Thanks! ~Kedar Dixit Bigdata Analytics at Persistent Systems Ltd. From: Holden Karau <hol...@pigscanfly.ca> Sent: 14 November 2017 20:04:50 To: Kedarnath Dixit Cc: user@spark.apache.org Subject: Re: Use of Accumulators And where do you want t

RE: Use of Accumulators

2017-11-14 Thread Kedarnath Dixit
: Holden Karau [via Apache Spark User List] [mailto:ml+s1001560n29995...@n3.nabble.com] Sent: Tuesday, November 14, 2017 1:16 PM To: Kedarnath Dixit <kedarnath_di...@persistent.com<mailto:kedarnath_di...@persistent.com>> Subject: Re: Use of Accumulators So you want to set an accumulato

Use of Accumulators

2017-11-13 Thread Kedarnath Dixit
Hi, We need some way to toggle the flag of a variable in transformation. We are thinking to make use of spark Accumulators for this purpose. Can we use these as below: Variables -> Initial Value Variable1 -> 0 Variable2 -> 0 In one of the transformations if we need to make