[jira] [Commented] (HDFS-1312) Re-balance disks within a Datanode

2012-09-27 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13465219#comment-13465219 ] Scott Carey commented on HDFS-1312: --- Isn't the datanode internal block placement policy a

[jira] [Commented] (HDFS-395) DFS Scalability: Incremental block reports

2011-07-01 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-395?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13058866#comment-13058866 ] Scott Carey commented on HDFS-395: -- {quote} from my earlier experience with developing loca

[jira] Commented: (HDFS-1114) Reducing NameNode memory usage by an alternate hash table

2010-06-13 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12878462#action_12878462 ] Scott Carey commented on HDFS-1114: --- if you are using a power of two hash table, you can a

[jira] Commented: (HDFS-1114) Reducing NameNode memory usage by an alternate hash table

2010-06-16 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12879633#action_12879633 ] Scott Carey commented on HDFS-1114: --- bq. # Capacity should be divided by a reference size

[jira] Commented: (HDFS-1114) Reducing NameNode memory usage by an alternate hash table

2010-06-18 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1114?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880423#action_12880423 ] Scott Carey commented on HDFS-1114: --- bq. This is a good point. Is there a way to determine

[jira] Commented: (HDFS-1224) Stale connection makes node miss append

2010-06-21 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12881008#action_12881008 ] Scott Carey commented on HDFS-1224: --- bq. Even so, does this cause any actual problems asid

[jira] Commented: (HDFS-1257) Race condition introduced by HADOOP-5124

2010-06-24 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12882254#action_12882254 ] Scott Carey commented on HDFS-1257: --- Is ConcurrentHashMap an option here? It is significa

[jira] Commented: (HDFS-1287) Why TreeSet is used when collecting block information FSDataSet::getBlockReport

2010-07-09 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886774#action_12886774 ] Scott Carey commented on HDFS-1287: --- As a general rule, if you need an ordered set or map

[jira] Commented: (HDFS-1094) Intelligent block placement policy to decrease probability of block loss

2010-07-09 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12886797#action_12886797 ] Scott Carey commented on HDFS-1094: --- bq. I've had some general discussions with Dhruba ab

[jira] Commented: (HDFS-1094) Intelligent block placement policy to decrease probability of block loss

2010-07-10 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12887136#action_12887136 ] Scott Carey commented on HDFS-1094: --- The "node group" versus Rodrigo's definition: Node g

[jira] Commented: (HDFS-1094) Intelligent block placement policy to decrease probability of block loss

2010-07-19 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12889910#action_12889910 ] Scott Carey commented on HDFS-1094: --- This needs to change "p" from a constant, to a functi

[jira] Commented: (HDFS-1094) Intelligent block placement policy to decrease probability of block loss

2010-07-19 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1289#action_1289 ] Scott Carey commented on HDFS-1094: --- bq. I think we are more interested in permanent failu

[jira] Commented: (HDFS-1312) Datanode storage directories fill unevenly

2010-07-21 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12890961#action_12890961 ] Scott Carey commented on HDFS-1312: --- a roulette-like algorithm for choosing which disk to

[jira] Commented: (HDFS-1547) Improve decommission mechanism

2011-01-07 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1547?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12978710#action_12978710 ] Scott Carey commented on HDFS-1547: --- I like Todd's proposal to have only one file, that li

[jira] [Assigned] (HDFS-1615) seek() on closed DFS input stream throws NPE

2011-05-11 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey reassigned HDFS-1615: - Assignee: Scott Carey (was: Harsh J Chouraria) > seek() on closed DFS input stream throws NPE > -

[jira] [Updated] (HDFS-1615) seek() on closed DFS input stream throws NPE

2011-05-11 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated HDFS-1615: -- Attachment: HDFS-1615.patch Patch contains simple fix, and unit test that fails without the fix. > seek

[jira] [Updated] (HDFS-1615) seek() on closed DFS input stream throws NPE

2011-05-11 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated HDFS-1615: -- Status: Patch Available (was: Open) > seek() on closed DFS input stream throws NPE > --

[jira] [Commented] (HDFS-1615) seek() on closed DFS input stream throws NPE

2011-05-11 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031988#comment-13031988 ] Scott Carey commented on HDFS-1615: --- No problem -- JIRA race condition. Your error messa

[jira] [Commented] (HDFS-1615) seek() on closed DFS input stream throws NPE

2011-05-11 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13031992#comment-13031992 ] Scott Carey commented on HDFS-1615: --- Actually, I think it was my fault -- I thought it wa

[jira] Commented: (HDFS-297) Implement a pure Java CRC32 calculator

2009-07-02 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12726651#action_12726651 ] Scott Carey commented on HDFS-297: -- {quote}The seed indicated in the comments doesn't line u

[jira] Commented: (HDFS-297) Implement a pure Java CRC32 calculator

2009-07-08 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728849#action_12728849 ] Scott Carey commented on HDFS-297: -- {quote}Is this new code written from scratch or has it b

[jira] Commented: (HDFS-297) Implement a pure Java CRC32 calculator

2009-07-08 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12728852#action_12728852 ] Scott Carey commented on HDFS-297: -- {quote}I think we now have the fastest crc32 in the west

[jira] Updated: (HDFS-148) timeout when writing dfs file causes infinite loop when closing the file

2010-05-21 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Carey updated HDFS-148: - Affects Version/s: 0.20.2 Priority: Critical (was: Major) This is affecting me on CDH2, there

[jira] Commented: (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

2010-05-23 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870470#action_12870470 ] Scott Carey commented on HDFS-1172: --- {quote} This doesn't cause major issues, but we do en

[jira] Commented: (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

2010-05-24 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12870749#action_12870749 ] Scott Carey commented on HDFS-1172: --- Perhaps when the write pipeline completes, it should

[jira] Commented: (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

2010-05-25 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12871283#action_12871283 ] Scott Carey commented on HDFS-1172: --- I run with min replication = 2, yet see this all the

[jira] Commented: (HDFS-1172) Blocks in newly completed files are considered under-replicated too quickly

2010-05-25 Thread Scott Carey (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1172?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12871385#action_12871385 ] Scott Carey commented on HDFS-1172: --- bq. # Scott's solution of making the primary DN send