[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2017-05-17 Thread Allen George (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014930#comment-16014930 ] Allen George commented on SPARK-17463: -- [~zsxwing] You're absolutely right: my apologies for the

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2017-05-17 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014799#comment-16014799 ] Shixiong Zhu commented on SPARK-17463: -- [~allengeorge] it's protected by

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2017-05-17 Thread Allen George (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16014564#comment-16014564 ] Allen George commented on SPARK-17463: -- [~zsxwing] So, we're hitting this problem on spark 2.0.2.

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2017-01-12 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15821106#comment-15821106 ] Sunil Rangwani commented on SPARK-17463: Hi [~zsxwing] My recordKey that I add to

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2017-01-09 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813205#comment-15813205 ] Shixiong Zhu commented on SPARK-17463: -- [~sunil.rangwani] could you have a simple reproducer? I ran

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2017-01-05 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15800979#comment-15800979 ] Sunil Rangwani commented on SPARK-17463: [~zsxwing] Do you agree this bug should be reopened? >

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2017-01-01 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15791469#comment-15791469 ] Sunil Rangwani commented on SPARK-17463: Hi [~zsxwing] Below is how I am using the Collection

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-12-30 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15788152#comment-15788152 ] Shixiong Zhu commented on SPARK-17463: -- [~sunil.rangwani] Could you provide the codes that use

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-12-30 Thread Sunil Rangwani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15787809#comment-15787809 ] Sunil Rangwani commented on SPARK-17463: Hi I get the same error with Spark 2.0.2 on EMR 5.2.0 I

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-11-07 Thread Harish (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15646453#comment-15646453 ] Harish commented on SPARK-17463: I was able to figure out the issue, its not related to this bug. >

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-19 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15589729#comment-15589729 ] Shixiong Zhu commented on SPARK-17463: -- I could not figure out the cause. Is it easy to reproduce in

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-11 Thread Harish (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566515#comment-15566515 ] Harish commented on SPARK-17463: My second approach was: def testfunc(keys, vals, columnsToStandardize):

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-11 Thread Harish (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566427#comment-15566427 ] Harish commented on SPARK-17463: No i dont have any code like that. I use pyspark .. Please find my code

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-11 Thread Harish (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566413#comment-15566413 ] Harish commented on SPARK-17463: Ok. thanks for the update. Do we have any work around for the second

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-11 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566391#comment-15566391 ] Shixiong Zhu commented on SPARK-17463: -- Do you have a reproducer? I saw `at

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566299#comment-15566299 ] Sean Owen commented on SPARK-17463: --- No, that change came after, and is part of a different JIRA that

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566300#comment-15566300 ] Sean Owen commented on SPARK-17463: --- No, that change came after, and is part of a different JIRA that

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-11 Thread Harish (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566206#comment-15566206 ] Harish commented on SPARK-17463: Is this fix is part of the https://github.com/apache/spark/pull/15371

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-11 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15566104#comment-15566104 ] Sean Owen commented on SPARK-17463: --- What do you mean? this has been released already in 2.0.1. >

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-11 Thread Harish (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15565493#comment-15565493 ] Harish commented on SPARK-17463: It looks like a show stopper for my current project. Can you please let

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-06 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15553134#comment-15553134 ] Shixiong Zhu commented on SPARK-17463: -- [~eseyfe] It's a different issue. Could you create a new

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-05 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550561#comment-15550561 ] Apache Spark commented on SPARK-17463: -- User 'seyfe' has created a pull request for this issue:

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-10-05 Thread Ergin Seyfe (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15550398#comment-15550398 ] Ergin Seyfe commented on SPARK-17463: - Hi [~zsxwing], I think this bug is not fully fixed. I still

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15485420#comment-15485420 ] Apache Spark commented on SPARK-17463: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-09-12 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484769#comment-15484769 ] Apache Spark commented on SPARK-17463: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-09-12 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15484720#comment-15484720 ] Shixiong Zhu commented on SPARK-17463: -- [~joshrosen] I think we can just leave LongAccum as it is.

[jira] [Commented] (SPARK-17463) Serialization of accumulators in heartbeats is not thread-safe

2016-09-08 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15475536#comment-15475536 ] Josh Rosen commented on SPARK-17463: [~zsxwing], FYI, since you're good at these types of RPC