[jira] [Updated] (HDFS-7960) The full block report should prune zombie storages even if they're not empty

2015-03-19 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7960: --- Attachment: HDFS-7960.002.patch > The full block report should prune zombie storages ev

[jira] [Updated] (HDFS-7960) The full block report should prune zombie storages even if they're not empty

2015-03-19 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7960: --- Attachment: HDFS-7960.001.patch patch to get jenkins run. will put up a unit test soon

[jira] [Updated] (HDFS-7960) The full block report should prune zombie storages even if they're not empty

2015-03-19 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7960: --- Status: Patch Available (was: Open) > The full block report should prune zombie storag

[jira] [Commented] (HDFS-7961) Trigger full block report after hot swapping disk

2015-03-19 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14370569#comment-14370569 ] Colin Patrick McCabe commented on HDFS-7961: maybe {{testFullBlockReportAfterHo

[jira] [Updated] (HDFS-7960) The full block report should prune zombie storages even if they're not empty

2015-03-19 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7960: --- Description: The full block report should prune zombie storages even if they're not emp

[jira] [Updated] (HDFS-7960) The full block report should prune zombie storages even if they're not empty

2015-03-19 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7960: --- Summary: The full block report should prune zombie storages even if they're not empty

[jira] [Commented] (HDFS-7960) The full block report should contain a "numStoragesOnDn" field that allows the NameNode to prune old storages

2015-03-19 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14370336#comment-14370336 ] Colin Patrick McCabe commented on HDFS-7960: Just a note here: the reason why H

[jira] [Updated] (HDFS-7960) The full block report should contain a "numStoragesOnDn" field that allows the NameNode to prune old storages

2015-03-19 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7960: --- Description: The full block report should contain a "numStoragesOnDn" field that a

[jira] [Updated] (HDFS-7960) NameNode should prune storages that are no longer existed on DataNode

2015-03-19 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7960: --- Assignee: Colin Patrick McCabe (was: Lei (Eddy) Xu) > NameNode should prune storages t

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2015-03-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368360#comment-14368360 ] Colin Patrick McCabe commented on HDFS-7878: Thanks, [~sershe]. I like version

[jira] [Updated] (HDFS-7929) inotify unable fetch pre-upgrade edit log segments once upgrade starts

2015-03-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7929: --- Resolution: Fixed Fix Version/s: 2.7.0 Target Version/s: 2.7.0

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

2015-03-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368324#comment-14368324 ] Colin Patrick McCabe commented on HDFS-6658: I didn't get a chance to look at t

[jira] [Updated] (HDFS-7054) Make DFSOutputStream tracing more fine-grained

2015-03-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7054: --- Resolution: Fixed Fix Version/s: 2.7.0 Status: Resolved (was: Patch A

[jira] [Commented] (HDFS-7054) Make DFSOutputStream tracing more fine-grained

2015-03-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368278#comment-14368278 ] Colin Patrick McCabe commented on HDFS-7054: {{TestLazyPersistFiles}} test fail

[jira] [Commented] (HDFS-7929) inotify unable fetch pre-upgrade edit log segments once upgrade starts

2015-03-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368277#comment-14368277 ] Colin Patrick McCabe commented on HDFS-7929: +1. Thanks for adding the test.

[jira] [Commented] (HDFS-6841) Use Time.monotonicNow() wherever applicable instead of Time.now()

2015-03-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14368194#comment-14368194 ] Colin Patrick McCabe commented on HDFS-6841: [~vinayrpet], do you have an updat

[jira] [Updated] (HDFS-7054) Make DFSOutputStream tracing more fine-grained

2015-03-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7054: --- Attachment: HDFS-7054.003.patch This adds a unit test, and also fixes a bug. the bug w

[jira] [Commented] (HDFS-7054) Make DFSOutputStream tracing more fine-grained

2015-03-18 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14367693#comment-14367693 ] Colin Patrick McCabe commented on HDFS-7054: bq. When would parents come back e

[jira] [Commented] (HDFS-7847) Modify NNThroughputBenchmark to be able to operate on a remote NameNode

2015-03-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365842#comment-14365842 ] Colin Patrick McCabe commented on HDFS-7847: {{getNamenodeProtocolProxy}}, {{ge

[jira] [Commented] (HDFS-7054) Make DFSOutputStream tracing more fine-grained

2015-03-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365801#comment-14365801 ] Colin Patrick McCabe commented on HDFS-7054: The "new" findbugs warnings are al

[jira] [Issue Comment Deleted] (HDFS-7922) ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors

2015-03-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7922: --- Comment: was deleted (was: Please tell me how i can use batch file HDFS-1783 https://i

[jira] [Issue Comment Deleted] (HDFS-7922) ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors

2015-03-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7922: --- Comment: was deleted (was: bq.Please tell me how i can use batch file HDFS-1783 [~Rosa]

[jira] [Updated] (HDFS-7940) Add tracing to DFSClient#setQuotaByStorageType

2015-03-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7940: --- Resolution: Fixed Fix Version/s: 2.7.0 Target Version/s: 2.7.0

[jira] [Commented] (HDFS-7940) Add tracing to DFSClient#setQuotaByStorageType

2015-03-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365637#comment-14365637 ] Colin Patrick McCabe commented on HDFS-7940: +1. Thanks, [~rakeshr]. > Add tr

[jira] [Commented] (HDFS-7938) OpensslSecureRandom.c pthread_threadid_np usage signature is wrong on 32-bit Mac

2015-03-17 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14365638#comment-14365638 ] Colin Patrick McCabe commented on HDFS-7938: Thanks, [~kiranmr]. > OpensslSecu

[jira] [Commented] (HDFS-7902) Expose truncate API for libwebhdfs

2015-03-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364457#comment-14364457 ] Colin Patrick McCabe commented on HDFS-7902: {code} 130 int createUrlForNnT

[jira] [Commented] (HDFS-7838) Expose truncate API for libhdfs

2015-03-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364455#comment-14364455 ] Colin Patrick McCabe commented on HDFS-7838: +1. Thanks, Yi. > Expose truncat

[jira] [Commented] (HDFS-7054) Make DFSOutputStream tracing more fine-grained

2015-03-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364418#comment-14364418 ] Colin Patrick McCabe commented on HDFS-7054: v2 fixes a missing import > Make

[jira] [Commented] (HDFS-7929) inotify unable fetch pre-upgrade edit log segments once upgrade starts

2015-03-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364411#comment-14364411 ] Colin Patrick McCabe commented on HDFS-7929: The approach looks reasonable here

[jira] [Created] (HDFS-7938) OpensslSecureRandom.c pthread_threadid_np usage signature is wrong on 32-bit Mac

2015-03-16 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-7938: -- Summary: OpensslSecureRandom.c pthread_threadid_np usage signature is wrong on 32-bit Mac Key: HDFS-7938 URL: https://issues.apache.org/jira/browse/HDFS-7938

[jira] [Commented] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364392#comment-14364392 ] Colin Patrick McCabe commented on HDFS-7915: Good catch. I committed to branch

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2015-03-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14364389#comment-14364389 ] Colin Patrick McCabe commented on HDFS-7878: Hi [~jingzhao], I understand that

[jira] [Updated] (HDFS-7054) Make DFSOutputStream tracing more fine-grained

2015-03-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7054: --- Attachment: HDFS-7054.002.patch > Make DFSOutputStream tracing more fine-grained >

[jira] [Updated] (HDFS-7054) Make DFSOutputStream tracing more fine-grained

2015-03-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7054: --- Target Version/s: 2.7.0 (was: 2.6.0) Status: Patch Available (was: Open)

[jira] [Updated] (HDFS-7054) Make DFSOutputStream tracing more fine-grained

2015-03-16 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7054: --- Attachment: HDFS-7054.001.patch > Make DFSOutputStream tracing more fine-grained >

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-14 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Resolution: Fixed Status: Resolved (was: Patch Available) > The DataNode can s

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Status: Patch Available (was: Reopened) > The DataNode can sometimes allocate a ShortC

[jira] [Reopened] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe reopened HDFS-7915: oops, I just saw that jenkins didn't run on v6 yet. sigh... > The DataNode can sometime

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Resolution: Fixed Fix Version/s: 2.7.0 Status: Resolved (was: Patch A

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Attachment: HDFS-7915.006.patch > The DataNode can sometimes allocate a ShortCircuitShm

[jira] [Comment Edited] (HDFS-7878) API - expose an unique file identifier

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361303#comment-14361303 ] Colin Patrick McCabe edited comment on HDFS-7878 at 3/13/15 11:24 PM: ---

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361303#comment-14361303 ] Colin Patrick McCabe commented on HDFS-7878: bq. Jing wrote: Could you please a

[jira] [Resolved] (HDFS-1841) Enforce read-only permissions in FUSE open()

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-1841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe resolved HDFS-1841. Resolution: Duplicate duplicate of HDFS-4139 from 2012 > Enforce read-only permissio

[jira] [Commented] (HDFS-7261) storageMap is accessed without synchronization in DatanodeDescriptor#updateHeartbeatState()

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361282#comment-14361282 ] Colin Patrick McCabe commented on HDFS-7261: thanks, [~brahmareddy]. 403

[jira] [Commented] (HDFS-7838) Expose truncate API for libhdfs

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361279#comment-14361279 ] Colin Patrick McCabe commented on HDFS-7838: The libhdfs part looks correct, bu

[jira] [Commented] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361264#comment-14361264 ] Colin Patrick McCabe commented on HDFS-7915: I did answer that comment, here:

[jira] [Commented] (HDFS-7922) ShortCircuitCache#close is not releasing ScheduledThreadPoolExecutors

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361255#comment-14361255 ] Colin Patrick McCabe commented on HDFS-7922: Note that this is never a problem

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Attachment: HDFS-7915.005.patch updated patch > The DataNode can sometimes allocate a

[jira] [Commented] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-13 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14361208#comment-14361208 ] Colin Patrick McCabe commented on HDFS-7915: bq. 1. I think we should look hard

[jira] [Created] (HDFS-7923) The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages

2015-03-12 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-7923: -- Summary: The DataNodes should rate-limit their full block reports by asking the NN on heartbeat messages Key: HDFS-7923 URL: https://issues.apache.org/jira/browse/HDFS

[jira] [Commented] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-12 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359377#comment-14359377 ] Colin Patrick McCabe commented on HDFS-7915: bq. cnauroth asked: Thanks for the

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-12 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Attachment: HDFS-7915.004.patch > The DataNode can sometimes allocate a ShortCircuitShm

[jira] [Commented] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-12 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359369#comment-14359369 ] Colin Patrick McCabe commented on HDFS-7915: I found another problem here. To

[jira] [Commented] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-12 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359351#comment-14359351 ] Colin Patrick McCabe commented on HDFS-7915: bq. Here bld is set to SUCCESS sta

[jira] [Updated] (HDFS-7722) DataNode#checkDiskError should also remove Storage when error is found.

2015-03-12 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7722: --- Resolution: Fixed Fix Version/s: 2.7.0 Status: Resolved (was: Patch A

[jira] [Commented] (HDFS-7722) DataNode#checkDiskError should also remove Storage when error is found.

2015-03-12 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14359180#comment-14359180 ] Colin Patrick McCabe commented on HDFS-7722: +1. Thanks, Eddy. > DataNode#che

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-11 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Attachment: HDFS-7915.002.patch I added the log message. The test should now fail when

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-11 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Attachment: HDFS-7915.001.patch Fix up the error handling in DataXceiver. A nested try

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Attachment: HDFS-7915.001.patch > The DataNode can sometimes allocate a ShortCircuitShm

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Status: Patch Available (was: Open) > The DataNode can sometimes allocate a ShortCircu

[jira] [Updated] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7915: --- Attachment: (was: HDFS-7915.001.patch) > The DataNode can sometimes allocate a Shor

[jira] [Commented] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356156#comment-14356156 ] Colin Patrick McCabe commented on HDFS-7915: I think putting the "unregister sl

[jira] [Created] (HDFS-7915) The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error

2015-03-10 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-7915: -- Summary: The DataNode can sometimes allocate a ShortCircuitShm slot and fail to tell the DFSClient about it because of a network error Key: HDFS-7915 URL: https://issu

[jira] [Updated] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.

2015-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7830: --- Resolution: Fixed Fix Version/s: 2.7.0 Status: Resolved (was: Patch A

[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.

2015-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14356068#comment-14356068 ] Colin Patrick McCabe commented on HDFS-7830: +1. Thanks, [~eddyxu]. > DataNod

[jira] [Commented] (HDFS-7846) Create off-heap BlocksMap and BlockData structures

2015-03-10 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7846?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14355701#comment-14355701 ] Colin Patrick McCabe commented on HDFS-7846: bq. Yi asked: For original BlockIn

[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14354087#comment-14354087 ] Colin Patrick McCabe commented on HDFS-7830: Thanks, [~eddyxu]. +1 pending jen

[jira] [Updated] (HDFS-7846) Create off-heap BlocksMap and BlockData structures

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7846: --- Attachment: HDFS-7846-scl.001.patch > Create off-heap BlocksMap and BlockData structure

[jira] [Work started] (HDFS-7846) Create off-heap BlocksMap and BlockData structures

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-7846 started by Colin Patrick McCabe. -- > Create off-heap BlocksMap and BlockData structures > ---

[jira] [Updated] (HDFS-7846) Create off-heap BlocksMap and BlockData structures

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7846?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7846: --- Summary: Create off-heap BlocksMap and BlockData structures (was: Create off-heap Bloc

[jira] [Commented] (HDFS-7847) Modify NNThroughputBenchmark to be able to operate on a remote NameNode

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353892#comment-14353892 ] Colin Patrick McCabe commented on HDFS-7847: DFSClient.java: this change adds t

[jira] [Updated] (HDFS-7909) Improve TestMemoryManager and TestProbingHashTable

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7909: --- Assignee: Charles Lamb > Improve TestMemoryManager and TestProbingHashTable > -

[jira] [Commented] (HDFS-7844) Create an off-heap hash table implementation

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353823#comment-14353823 ] Colin Patrick McCabe commented on HDFS-7844: I opened HDFS-7909 for the test im

[jira] [Created] (HDFS-7909) Improve TestMemoryManager and TestProbingHashTable

2015-03-09 Thread Colin Patrick McCabe (JIRA)
Colin Patrick McCabe created HDFS-7909: -- Summary: Improve TestMemoryManager and TestProbingHashTable Key: HDFS-7909 URL: https://issues.apache.org/jira/browse/HDFS-7909 Project: Hadoop HDFS

[jira] [Commented] (HDFS-7844) Create an off-heap hash table implementation

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353817#comment-14353817 ] Colin Patrick McCabe commented on HDFS-7844: bq. charles wrote: Thanks Colin, +

[jira] [Updated] (HDFS-7844) Create an off-heap hash table implementation

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7844: --- Attachment: HDFS-7844-scl.004.patch > Create an off-heap hash table implementation > --

[jira] [Commented] (HDFS-7844) Create an off-heap hash table implementation

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353791#comment-14353791 ] Colin Patrick McCabe commented on HDFS-7844: bq. Stack wrote: High level, any n

[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353769#comment-14353769 ] Colin Patrick McCabe commented on HDFS-7830: {code} 420 if (!exceptions

[jira] [Commented] (HDFS-7879) hdfs.dll does not export functions of the public libhdfs API

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353760#comment-14353760 ] Colin Patrick McCabe commented on HDFS-7879: Thanks, [~wheat9] and [~cnauroth].

[jira] [Resolved] (HDFS-3742) libhdfs symbol visibility issues

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-3742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe resolved HDFS-3742. Resolution: Duplicate Fix Version/s: 2.7.0 implemented in HDFS-7879 > libhdfs

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353755#comment-14353755 ] Colin Patrick McCabe commented on HDFS-7878: TOCTOU defined here: http://en.wik

[jira] [Commented] (HDFS-7878) API - expose an unique file identifier

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353753#comment-14353753 ] Colin Patrick McCabe commented on HDFS-7878: Thanks for tackling this! One big

[jira] [Commented] (HDFS-7838) Expose truncate API for libhdfs

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353421#comment-14353421 ] Colin Patrick McCabe commented on HDFS-7838: FileSystem.java says: {code} *

[jira] [Commented] (HDFS-7830) DataNode does not release the volume lock when adding a volume fails.

2015-03-09 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14353396#comment-14353396 ] Colin Patrick McCabe commented on HDFS-7830: {code} 108 } finally { 109

[jira] [Commented] (HDFS-7844) Create an off-heap hash table implementation

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351290#comment-14351290 ] Colin Patrick McCabe commented on HDFS-7844: thanks, Charles. I think we can c

[jira] [Commented] (HDFS-7722) DataNode#checkDiskError should also remove Storage when error is found.

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351152#comment-14351152 ] Colin Patrick McCabe commented on HDFS-7722: Eddy and I had an offline discussi

[jira] [Commented] (HDFS-7844) Create an off-heap hash table implementation

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351122#comment-14351122 ] Colin Patrick McCabe commented on HDFS-7844: bq. Several lines bust the 80 char

[jira] [Updated] (HDFS-7844) Create an off-heap hash table implementation

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7844: --- Attachment: HDFS-7844-scl.003.patch > Create an off-heap hash table implementation > --

[jira] [Commented] (HDFS-7758) Retire FsDatasetSpi#getVolumes() and use FsDatasetSpi#getVolumeRefs() instead

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7758?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351058#comment-14351058 ] Colin Patrick McCabe commented on HDFS-7758: {code} /** * Returns a list o

[jira] [Updated] (HDFS-7844) Create an off-heap hash table implementation

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7844: --- Attachment: HDFS-7844-scl.002.patch > Create an off-heap hash table implementation > --

[jira] [Commented] (HDFS-6695) Investigate using Java 7's nonblocking file I/O in BlockReaderLocal to implement read timeouts

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350939#comment-14350939 ] Colin Patrick McCabe commented on HDFS-6695: And sending an {{Interrupt}} to a

[jira] [Commented] (HDFS-6658) Namenode memory optimization - Block replicas list

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350936#comment-14350936 ] Colin Patrick McCabe commented on HDFS-6658: [~clamb] and I have been discussin

[jira] [Commented] (HDFS-7836) BlockManager Scalability Improvements

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350878#comment-14350878 ] Colin Patrick McCabe commented on HDFS-7836: I don't think the block map size i

[jira] [Commented] (HDFS-6450) Support non-positional hedged reads in HDFS

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-6450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350869#comment-14350869 ] Colin Patrick McCabe commented on HDFS-6450: I think it would be possible to su

[jira] [Commented] (HDFS-7844) Create an off-heap hash table implementation

2015-03-06 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350828#comment-14350828 ] Colin Patrick McCabe commented on HDFS-7844: bq. Open addressing (probing) is u

[jira] [Commented] (HDFS-7838) Expose truncate API for libhdfs

2015-03-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349283#comment-14349283 ] Colin Patrick McCabe commented on HDFS-7838: Thanks for looking at this, Yi. {

[jira] [Commented] (HDFS-7261) storageMap is accessed without synchronization in DatanodeDescriptor#updateHeartbeatState()

2015-03-05 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14349245#comment-14349245 ] Colin Patrick McCabe commented on HDFS-7261: Thanks, Brahma. It looks like sto

[jira] [Commented] (HDFS-7836) BlockManager Scalability Improvements

2015-03-04 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347999#comment-14347999 ] Colin Patrick McCabe commented on HDFS-7836: I think it makes sense to have tot

[jira] [Commented] (HDFS-7844) Create an off-heap hash table implementation

2015-03-04 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14347903#comment-14347903 ] Colin Patrick McCabe commented on HDFS-7844: {{CommonConfigurationKeys.java}}:

[jira] [Updated] (HDFS-7844) Create an off-heap hash table implementation

2015-03-04 Thread Colin Patrick McCabe (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Patrick McCabe updated HDFS-7844: --- Attachment: HDFS-7844-scl.001.patch > Create an off-heap hash table implementation > --

<    6   7   8   9   10   11   12   13   14   15   >