[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2020-01-15 Thread ShivaKumar SS (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17016200#comment-17016200 ] ShivaKumar SS commented on SPARK-26570: --- Unfortunately I have hit the same issue. 

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-11-05 Thread Gautam Pulla (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16967947#comment-16967947 ] Gautam Pulla commented on SPARK-26570: -- I'm hitting a similar issue - but at a more

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-14 Thread fengchaoge (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16951437#comment-16951437 ] fengchaoge commented on SPARK-26570: It's just a snapshot of an instant, and it's ac

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-14 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16951223#comment-16951223 ] L. C. Hsieh commented on SPARK-26570: - First from the stacktrace, I was thinking it

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-13 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950686#comment-16950686 ] Sean R. Owen commented on SPARK-26570: -- That doesn't help much, especially as it's

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-13 Thread fengchaoge (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950675#comment-16950675 ] fengchaoge commented on SPARK-26570: !image-2019-10-14-10-51-28-374.png! > Out of m

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-13 Thread fengchaoge (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950672#comment-16950672 ] fengchaoge commented on SPARK-26570: !image-2019-10-14-10-48-05-063.png! !image-201

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-13 Thread fengchaoge (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950658#comment-16950658 ] fengchaoge commented on SPARK-26570: hello, [~srowen] [~viirya] jmap logs is shown

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-13 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950406#comment-16950406 ] L. C. Hsieh commented on SPARK-26570: - I think the collected number of filestatus ca

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-13 Thread Sean R. Owen (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950370#comment-16950370 ] Sean R. Owen commented on SPARK-26570: -- Not sure, those objects aren't large, and 7

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-10-13 Thread fengchaoge (Jira)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16950276#comment-16950276 ] fengchaoge commented on SPARK-26570: hello, [~deshanxiao] [~srowen] spark2.4.3 may a

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-03-01 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782137#comment-16782137 ] Sean Owen commented on SPARK-26570: --- How big can these be? are you saying they're larg

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-01-09 Thread deshanxiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16739083#comment-16739083 ] deshanxiao commented on SPARK-26570: [~hyukjin.kwon] OK, I will try it. Thank you!

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-01-09 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16738955#comment-16738955 ] Hyukjin Kwon commented on SPARK-26570: -- Would you be able to test this in upper ver

[jira] [Commented] (SPARK-26570) Out of memory when InMemoryFileIndex bulkListLeafFiles

2019-01-08 Thread deshanxiao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16737061#comment-16737061 ] deshanxiao commented on SPARK-26570: !screenshot-1.png! > Out of memory when InMe