[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23113 Adding @rxin and @gatorsmile fyi. We'll discuss this separately per policy. --- - To unsubscribe, e-mail:

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-25 Thread Tagar
Github user Tagar commented on the issue: https://github.com/apache/spark/pull/23113 `auth_token` is only used in accumulators update server https://github.com/apache/spark/search?q=auth_token_q=auth_token - relatively new code that was added in this commit -

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23113 Typo... auth_token ... Zeppelin supports it BTW. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-25 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/23113 I'm not sure if I follow the entire discussion here yet. I did spend a while trying to figure out if there is a race condition, and I'm pretty sure there is not. From your last comments, it looks

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23113 Also, it doesn't now then looks a race condition or anything. Please fix the PR description and JIRA accodingly. It looks potentially confusing. ---

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23113 auto token should be set to use Spark. If that's not set, it's a misuse of Spark - we should throw an exception. It's weird that we allow here it without the token here specifically. ---

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-24 Thread Tagar
Github user Tagar commented on the issue: https://github.com/apache/spark/pull/23113 @HyukjinKwon This is somehow related to the fact that it is reproducible when PySpark was created using an existing py4j gateway (for example, it reproduces from within Zeppelin)

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23113 > race condition explained in https://issues.apache.org/jira/browse/SPARK-26019 How race condition happens? Can you clarify it in PR description. I think @viirya's analysis is matched

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-21 Thread Tagar
Github user Tagar commented on the issue: https://github.com/apache/spark/pull/23113 @squito please help review this PR Thank you. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23113 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23113 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23113: [SPARK-26019][PYTHON] Fix race condition in accumulators...

2018-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23113 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional