[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2014-03-11 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13930815#comment-13930815 ] Colin Patrick McCabe commented on HDFS-4461: +1 for the backport. Note that I

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2014-03-11 Thread Kihwal Lee (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13930966#comment-13930966 ] Kihwal Lee commented on HDFS-4461: -- Thanks, Colin. I've checked it into branch-0.23.

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-06-19 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687843#comment-13687843 ] Hudson commented on HDFS-4461: -- Integrated in Hadoop-Yarn-trunk #245 (See

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-06-19 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687944#comment-13687944 ] Hudson commented on HDFS-4461: -- Integrated in Hadoop-Hdfs-trunk #1435 (See

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-06-19 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687989#comment-13687989 ] Hudson commented on HDFS-4461: -- Integrated in Hadoop-Mapreduce-trunk #1462 (See

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-06-18 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687122#comment-13687122 ] Todd Lipcon commented on HDFS-4461: --- FWIW we are running into this again with another

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-06-18 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687130#comment-13687130 ] Todd Lipcon commented on HDFS-4461: --- todd@todd-w510:~$ cat /tmp/c A few quick comments on

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-06-18 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687360#comment-13687360 ] Hadoop QA commented on HDFS-4461: - {color:green}+1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-06-18 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687393#comment-13687393 ] Todd Lipcon commented on HDFS-4461: --- +1 DirectoryScanner: volume path

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-06-18 Thread Hudson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13687514#comment-13687514 ] Hudson commented on HDFS-4461: -- Integrated in Hadoop-trunk-Commit #3975 (See

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-02-12 Thread Harsh J (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13577326#comment-13577326 ] Harsh J commented on HDFS-4461: --- A general memory increase between 1.0 - 2.0 can also be

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-02-01 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13568901#comment-13568901 ] Suresh Srinivas commented on HDFS-4461: --- bq. This has been causing out-of-memory

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-02-01 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13568920#comment-13568920 ] Colin Patrick McCabe commented on HDFS-4461: bq. I doubt that the directory

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-02-01 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13568930#comment-13568930 ] Suresh Srinivas commented on HDFS-4461: --- bq. we analyzed a DN heap dump from a

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-02-01 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13568959#comment-13568959 ] Colin Patrick McCabe commented on HDFS-4461: If someone is running with around

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-02-01 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13568979#comment-13568979 ] Suresh Srinivas commented on HDFS-4461: --- bq. If someone is running with around

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-02-01 Thread Todd Lipcon (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13568989#comment-13568989 ] Todd Lipcon commented on HDFS-4461: --- Looks like we can cut the memory usage in half again

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-02-01 Thread Andy Isaacson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569013#comment-13569013 ] Andy Isaacson commented on HDFS-4461: - bq. A server generally has a lot of String

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-02-01 Thread Andy Isaacson (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569083#comment-13569083 ] Andy Isaacson commented on HDFS-4461: - The actual OOM backtrace is on the DN thread:

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-02-01 Thread Suresh Srinivas (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13569198#comment-13569198 ] Suresh Srinivas commented on HDFS-4461: --- I think my earlier comments perhaps are not

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-01-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13568320#comment-13568320 ] Hadoop QA commented on HDFS-4461: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-01-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13568391#comment-13568391 ] Hadoop QA commented on HDFS-4461: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-01-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13568410#comment-13568410 ] Hadoop QA commented on HDFS-4461: - {color:red}-1 overall{color}. Here are the results of

[jira] [Commented] (HDFS-4461) DirectoryScanner: volume path prefix takes up memory for every block that is scanned

2013-01-31 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-4461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13568465#comment-13568465 ] Hadoop QA commented on HDFS-4461: - {color:red}-1 overall{color}. Here are the results of