[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-04-22 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253518#comment-15253518 ] Sun Rui commented on SPARK-13178: - This is fixed as the SparkR unit tests can pass after removing the

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-04-22 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253517#comment-15253517 ] Apache Spark commented on SPARK-13178: -- User 'sun-rui' has created a pull request for this issue:

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-04-20 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250268#comment-15250268 ] Shivaram Venkataraman commented on SPARK-13178: --- [~sunrui] [~yinxusen] Now that

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-24 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15162877#comment-15162877 ] Sun Rui commented on SPARK-13178: - Remember to clean the code at

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-06 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136132#comment-15136132 ] Xusen Yin commented on SPARK-13178: --- Cheers for the good news! :) > RRDD faces with concurrency issue

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-06 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135671#comment-15135671 ] Sun Rui commented on SPARK-13178: - The root cause is that RRDD.compute() uses some instance variables. If

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-04 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133363#comment-15133363 ] Xusen Yin commented on SPARK-13178: --- Yes, it works, we can use read.json to load a DataFrame that

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-04 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132718#comment-15132718 ] Xusen Yin commented on SPARK-13178: --- Thanks! I'll try it. > RRDD faces with concurrency issue in case

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-03 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131430#comment-15131430 ] Xusen Yin commented on SPARK-13178: --- Ping [~mengxr] [~shivaram] to know about the concurrency issue. I

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-03 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131436#comment-15131436 ] Shivaram Venkataraman commented on SPARK-13178: --- Hmm this is tricky to debug -- A higher

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-03 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131455#comment-15131455 ] Xusen Yin commented on SPARK-13178: --- I don't zip RRDD with itself. Actually, the bug exists when I

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-03 Thread Xusen Yin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131463#comment-15131463 ] Xusen Yin commented on SPARK-13178: --- We can work around with just adding a cache for the "df". But it

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-03 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131556#comment-15131556 ] Shivaram Venkataraman commented on SPARK-13178: --- Ah I see - so the problem is that

[jira] [Commented] (SPARK-13178) RRDD faces with concurrency issue in case of rdd.zip(rdd).count()

2016-02-03 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131634#comment-15131634 ] Sun Rui commented on SPARK-13178: - [~xusen] Could you first use a DataFrame created from something like