[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253518#comment-15253518
]
Sun Rui commented on SPARK-13178:
-
This is fixed as the SparkR unit tests can pass after removing the
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253517#comment-15253517
]
Apache Spark commented on SPARK-13178:
--
User 'sun-rui' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15250268#comment-15250268
]
Shivaram Venkataraman commented on SPARK-13178:
---
[~sunrui] [~yinxusen] Now that
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15162877#comment-15162877
]
Sun Rui commented on SPARK-13178:
-
Remember to clean the code at
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15136132#comment-15136132
]
Xusen Yin commented on SPARK-13178:
---
Cheers for the good news! :)
> RRDD faces with concurrency issue
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15135671#comment-15135671
]
Sun Rui commented on SPARK-13178:
-
The root cause is that RRDD.compute() uses some instance variables. If
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133363#comment-15133363
]
Xusen Yin commented on SPARK-13178:
---
Yes, it works, we can use read.json to load a DataFrame that
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15132718#comment-15132718
]
Xusen Yin commented on SPARK-13178:
---
Thanks! I'll try it.
> RRDD faces with concurrency issue in case
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131430#comment-15131430
]
Xusen Yin commented on SPARK-13178:
---
Ping [~mengxr] [~shivaram] to know about the concurrency issue. I
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131436#comment-15131436
]
Shivaram Venkataraman commented on SPARK-13178:
---
Hmm this is tricky to debug -- A higher
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131455#comment-15131455
]
Xusen Yin commented on SPARK-13178:
---
I don't zip RRDD with itself. Actually, the bug exists when I
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131463#comment-15131463
]
Xusen Yin commented on SPARK-13178:
---
We can work around with just adding a cache for the "df". But it
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131556#comment-15131556
]
Shivaram Venkataraman commented on SPARK-13178:
---
Ah I see - so the problem is that
[
https://issues.apache.org/jira/browse/SPARK-13178?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131634#comment-15131634
]
Sun Rui commented on SPARK-13178:
-
[~xusen] Could you first use a DataFrame created from something like
14 matches
Mail list logo