[jira] [Resolved] (SPARK-21817) Pass FSPermissions to LocatedFileStatus from InMemoryFileIndex

2017-08-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs resolved SPARK-21817. Resolution: Invalid This was caused by a change in a stable/evolving interface which previously

[jira] [Commented] (SPARK-21817) Pass FSPermissions to LocatedFileStatus from InMemoryFileIndex

2017-08-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138905#comment-16138905 ] Ewan Higgs commented on SPARK-21817: {quote} Ewan: do a patch there with a new test method (where?) &

[jira] [Commented] (SPARK-21817) Pass FSPermissions to LocatedFileStatus from InMemoryFileIndex

2017-08-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138253#comment-16138253 ] Ewan Higgs commented on SPARK-21817: {quote}Can this be accomplished with a change that's still

[jira] [Updated] (SPARK-21817) Pass FSPermissions to LocatedFileStatus from InMemoryFileIndex

2017-08-23 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated SPARK-21817: --- Attachment: SPARK-21817.001.patch Attaching simple fix that will no longer NPE on Hadoop head. >

[jira] [Created] (SPARK-21817) Pass FSPermissions to LocatedFileStatus from InMemoryFileIndex

2017-08-23 Thread Ewan Higgs (JIRA)
Ewan Higgs created SPARK-21817: -- Summary: Pass FSPermissions to LocatedFileStatus from InMemoryFileIndex Key: SPARK-21817 URL: https://issues.apache.org/jira/browse/SPARK-21817 Project: Spark

[jira] [Commented] (SPARK-13434) Reduce Spark RandomForest memory footprint

2016-02-22 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15157105#comment-15157105 ] Ewan Higgs commented on SPARK-13434: Hi Sean, This is using jmap with the {{-histo:live}} argument

[jira] [Commented] (SPARK-13434) Reduce Spark RandomForest memory footprint

2016-02-22 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15156921#comment-15156921 ] Ewan Higgs commented on SPARK-13434: SPARK-3728 is titled with a similar intent to this, but the

[jira] [Updated] (SPARK-13434) Reduce Spark RandomForest memory footprint

2016-02-22 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated SPARK-13434: --- Attachment: heap-usage.log Heap usage of RandomForest sampled with {{jmap -histo:live }} every 5

[jira] [Updated] (SPARK-13434) Reduce Spark RandomForest memory footprint

2016-02-22 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs updated SPARK-13434: --- Attachment: rf-heap-usage.png JConsole output of memory use with 1.3G file. > Reduce Spark

[jira] [Created] (SPARK-13434) Reduce Spark RandomForest memory footprint

2016-02-22 Thread Ewan Higgs (JIRA)
Ewan Higgs created SPARK-13434: -- Summary: Reduce Spark RandomForest memory footprint Key: SPARK-13434 URL: https://issues.apache.org/jira/browse/SPARK-13434 Project: Spark Issue Type:

[jira] [Commented] (SPARK-5836) Highlight in Spark documentation that by default Spark does not delete its temporary files

2015-12-03 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15037593#comment-15037593 ] Ewan Higgs commented on SPARK-5836: --- [~tdas] {quote} The only case there may be issues is when the

[jira] [Resolved] (SPARK-5300) Spark loads file partitions in inconsistent order on native filesystems

2015-04-27 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ewan Higgs resolved SPARK-5300. --- Resolution: Won't Fix I submitted a fix at the FileSystem level based on comments in the mailing

[jira] [Commented] (SPARK-5300) Spark loads file partitions in inconsistent order on native filesystems

2015-01-27 Thread Ewan Higgs (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5300?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14293201#comment-14293201 ] Ewan Higgs commented on SPARK-5300: --- The PR appears to have been rejected on the grounds

[jira] [Created] (SPARK-5300) Spark loads file partitions in inconsistent order on native filesystems

2015-01-17 Thread Ewan Higgs (JIRA)
Ewan Higgs created SPARK-5300: - Summary: Spark loads file partitions in inconsistent order on native filesystems Key: SPARK-5300 URL: https://issues.apache.org/jira/browse/SPARK-5300 Project: Spark