GraphX Pregel: Access current iteration i

2021-11-08 Thread Jannik Rau
Hi all, I'm using the Pregel.apply function in an iterative graph algorithm. In the algorithm, the update of a vertex's value is treated differently in the first and last of the n total iterations. Calling Pregel.apply with maxIterations = 1, then with maxIterations = n - 2 and again with

Re: apache-spark-how to run distributed spark jobs in Apache-spark-standalone-cluster

2021-11-08 Thread Sean Owen
Yes, did you check the docs? https://spark.apache.org/docs/latest/spark-standalone.html On Mon, Nov 8, 2021 at 6:40 AM Dinakar Chennubotla wrote: > Hi All,i am dinakar and I am admin,i have question, > Question: " is it possible to run distributed spark jobs in Apache spark > standalone

apache-spark-how to run distributed spark jobs in Apache-spark-standalone-cluster

2021-11-08 Thread Dinakar Chennubotla
Hi All,i am dinakar and I am admin,i have question, Question: " is it possible to run distributed spark jobs in Apache spark standalone cluster".if yes, could someone, help with the docs or webpages. so that i can create and test it.Thanks in advance, Dinakar

3.1.2 Executor Initialization fails due to dep copy failure

2021-11-08 Thread Bode, Meikel, NMA-CFD
Hi all, I try to get Thrift2 on Spark 3.1.2 running on K8S with one executor for the moment. This works so far but it fails at executor side during initialization. The issue seems to be related to access restrictions on certain directories... But I am not sure. Please see errors marked in