simultaneous actions

[Kira]

Hi,

Can we run *simultaneous* actions on the *same RDD* ?; if yes how can this
be done ?

Thank you,
Regards



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/simultaneous-actions-tp25977.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Read Accumulator value while running

[Kira]

Hi, 

So i have an action on one RDD that is relatively long, let's call it ac1;
what i want to do is to execute another action (ac2) on the same RDD to see
the evolution of the first one (ac1); for this end i want to use an
accumulator and read it's value progressively to see the changes on it (on
the fly) while ac1 is always running. My problem is that the accumulator is
only updated once the ac1 has been finished, this is not helpful for me :/ . 

I ve seen  here

  
what may seem like a solution for me but it doesn t work : "While Spark
already offers support for asynchronous reduce (collect data from workers,
while not interrupting execution of a parallel transformation) through
accumulator" 

Another post suggested to use SparkListner to do that. 

are these solutions correct ? if yes, give me a simple exemple ? 
are there other solutions ? 

thank you. 
Regards



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Read-Accumulator-value-while-running-tp25960.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org

Concurrent Read of Accumulator's Value

[Kira]

Hi,

So i have an action on one RDD that is relatively long, let's call it ac1;
what i want to do is to execute another action (ac2) on the same RDD to see
the evolution of the first one (ac1); for this end i want to use an
accumulator and read it's value progressively to see the changes on it (on
the fly) while ac1 is always running. My problem is that the accumulator is
only updated once the ac1 has been finished, this is not helpful for me :/ .

I ve seen  here

  
what may seem like a solution for me but it doesn t work : "While Spark
already offers support for asynchronous reduce (collect data from workers,
while not interrupting execution of a parallel transformation) through
accumulator"

Another post suggested to use SparkListner to do that.

are these solutions correct ? if yes, give me a simple exemple ?
are there other solutions ?

thank you.
Regards



--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/Concurrent-Read-of-Accumulator-s-Value-tp25957.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

-
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org