[jira] [Commented] (HDFS-10434) Fix intermittent test failure of TestDataNodeErasureCodingMetrics

2016-05-23 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296176#comment-15296176 ] Li Bo commented on HDFS-10434: -- Thanks for [~rakeshr]'s detailed explanation. The situation described will

[jira] [Commented] (HDFS-10434) Fix intermittent test failure of TestDataNodeErasureCodingMetrics

2016-05-23 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15296112#comment-15296112 ] Li Bo commented on HDFS-10434: -- Thanks for [~rakeshr] finding the problem.

[jira] [Commented] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-16 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15284223#comment-15284223 ] Li Bo commented on HDFS-8449: - Thanks for Kai's review and commit. I have just resolve this jira. > Add tasks

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-16 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Resolution: Fixed Target Version/s: (was: ) Status: Resolved (was: Patch Available) > Add

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-15 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-v12.patch Upload v12(same with v11) to trigger the test. > Add tasks count metrics to

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-09 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Status: Patch Available (was: In Progress) > Add tasks count metrics to datanode for ECWorker >

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-09 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Status: In Progress (was: Patch Available) To trigger the test > Add tasks count metrics to datanode for ECWorker

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-09 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-v11.patch v11 fix checkstyle problems > Add tasks count metrics to datanode for ECWorker >

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-08 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-v10.patch Rename the patch to trigger the test. > Add tasks count metrics to datanode for

[jira] [Updated] (HDFS-8901) Use ByteBuffer in striping positional read

2016-05-06 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8901: Attachment: HDFS-8901-v9.patch fix the bugs in TestParalledShortCircuitLegacyRead > Use ByteBuffer in striping

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-06 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-010.patch Fix the problems according to Kai's comments > Add tasks count metrics to datanode

[jira] [Commented] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-05 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15272031#comment-15272031 ] Li Bo commented on HDFS-8449: - Thanks very much for Kai's review. bq. Could you enhance

[jira] [Updated] (HDFS-8901) Use ByteBuffer in striping positional read

2016-05-04 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8901: Attachment: HDFS-8901-v8.patch rebase the patch to latest code; fix the unsupported method problem > Use ByteBuffer

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-04 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-009.patch > Add tasks count metrics to datanode for ECWorker >

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-03 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-008.patch > Add tasks count metrics to datanode for ECWorker >

[jira] [Commented] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-02 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15268124#comment-15268124 ] Li Bo commented on HDFS-8449: - Update patch 007 to fix some minor problems. > Add tasks count metrics to

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-05-02 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-007.patch > Add tasks count metrics to datanode for ECWorker >

[jira] [Commented] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-04-27 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15261472#comment-15261472 ] Li Bo commented on HDFS-8449: - The two failed unit tests have no relation with this patch and will be solved

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-04-27 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-006.patch > Add tasks count metrics to datanode for ECWorker >

[jira] [Commented] (HDFS-10317) dfs.domain.socket.path is not set in TestShortCircuitLocalRead.testReadWithRemoteBlockReader

2016-04-21 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10317?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15253201#comment-15253201 ] Li Bo commented on HDFS-10317: -- I use Intellij IDEA to run the test and the problem occurs, but it will not

[jira] [Created] (HDFS-10317) dfs.domain.socket.path is not set in TestShortCircuitLocalRead.testReadWithRemoteBlockReader

2016-04-20 Thread Li Bo (JIRA)
Li Bo created HDFS-10317: Summary: dfs.domain.socket.path is not set in TestShortCircuitLocalRead.testReadWithRemoteBlockReader Key: HDFS-10317 URL: https://issues.apache.org/jira/browse/HDFS-10317 Project:

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-04-20 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-005.patch Thanks Kai's review. There're several other ec related metrics to add, so I think

[jira] [Commented] (HDFS-10307) Fix a bug in TestShortCircuitLocalRead

2016-04-19 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15247349#comment-15247349 ] Li Bo commented on HDFS-10307: -- mark it as duplicated. > Fix a bug in TestShortCircuitLocalRead >

[jira] [Updated] (HDFS-10307) Fix a bug in TestShortCircuitLocalRead

2016-04-19 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-10307: - Resolution: Duplicate Status: Resolved (was: Patch Available) > Fix a bug in TestShortCircuitLocalRead >

[jira] [Updated] (HDFS-10307) Fix a bug in TestShortCircuitLocalRead

2016-04-19 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-10307: - Status: Patch Available (was: Open) > Fix a bug in TestShortCircuitLocalRead >

[jira] [Updated] (HDFS-10307) Fix a bug in TestShortCircuitLocalRead

2016-04-19 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-10307: - Attachment: HDFS-10307-001.patch > Fix a bug in TestShortCircuitLocalRead >

[jira] [Created] (HDFS-10307) Fix a bug in TestShortCircuitLocalRead

2016-04-19 Thread Li Bo (JIRA)
Li Bo created HDFS-10307: Summary: Fix a bug in TestShortCircuitLocalRead Key: HDFS-10307 URL: https://issues.apache.org/jira/browse/HDFS-10307 Project: Hadoop HDFS Issue Type: Bug

[jira] [Commented] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-04-18 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15245211#comment-15245211 ] Li Bo commented on HDFS-8449: - hi, [~drankye], could you help me review the newly updated patch? Thanks > Add

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2016-04-18 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-004.patch > Add tasks count metrics to datanode for ECWorker >

[jira] [Commented] (HDFS-10258) Erasure Coding: support small cluster whose #DataNode < # (Blocks in a BlockGroup)

2016-04-14 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15242287#comment-15242287 ] Li Bo commented on HDFS-10258: -- Thanks for Kai's idea, I will try to find a solution with the lowest cost.

[jira] [Updated] (HDFS-10258) Erasure Coding: support small cluster whose #DataNode < # (Blocks in a BlockGroup)

2016-04-05 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-10258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-10258: - Description: Currently EC has not supported small clusters whose datanode number is smaller than the block

[jira] [Created] (HDFS-10258) Erasure Coding: support small cluster whose #DataNode < # (Blocks in a BlockGroup)

2016-04-05 Thread Li Bo (JIRA)
Li Bo created HDFS-10258: Summary: Erasure Coding: support small cluster whose #DataNode < # (Blocks in a BlockGroup) Key: HDFS-10258 URL: https://issues.apache.org/jira/browse/HDFS-10258 Project: Hadoop

[jira] [Commented] (HDFS-9822) Erasure Coding: Avoids scheduling multiple reconstruction tasks for a striped block at the same time

2016-03-08 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15186510#comment-15186510 ] Li Bo commented on HDFS-9822: - hi,Rakesh After reading the code of {{UnderReplicationBlocks}} I am still a

[jira] [Updated] (HDFS-9826) Erasure Coding: Postpone the recovery work for a configurable time period

2016-03-08 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9826: Status: Patch Available (was: Open) > Erasure Coding: Postpone the recovery work for a configurable time period >

[jira] [Updated] (HDFS-9826) Erasure Coding: Postpone the recovery work for a configurable time period

2016-03-08 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9826: Attachment: HDFS-9826-002.patch > Erasure Coding: Postpone the recovery work for a configurable time period >

[jira] [Updated] (HDFS-9826) Erasure Coding: Postpone the recovery work for a configurable time period

2016-02-29 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9826: Attachment: HDFS-9826-001.patch upload an initial patch without test > Erasure Coding: Postpone the recovery work

[jira] [Commented] (HDFS-9657) Schedule EC tasks at proper time to reduce the impact of recovery traffic

2016-02-18 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152003#comment-15152003 ] Li Bo commented on HDFS-9657: - Thanks Zhe’s comment. 90% errors are transient and without permanent data loss,

[jira] [Updated] (HDFS-9826) Erasure Coding: Postpone the recovery work for a configurable time period

2016-02-18 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9826: Description: Currently NameNode prepares recovering when finding an under replicated block group. This is

[jira] [Created] (HDFS-9826) Erasure Coding: Postpone the recovery work for a configurable time period

2016-02-18 Thread Li Bo (JIRA)
Li Bo created HDFS-9826: --- Summary: Erasure Coding: Postpone the recovery work for a configurable time period Key: HDFS-9826 URL: https://issues.apache.org/jira/browse/HDFS-9826 Project: Hadoop HDFS

[jira] [Updated] (HDFS-9657) Schedule EC tasks at proper time to reduce the impact of recovery traffic

2016-02-02 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9657: Attachment: HDFS-9657-002.patch Currently only implements one policy, i.e ECRecoveryPolicyTimeSegment. I think it

[jira] [Updated] (HDFS-9657) Schedule EC tasks at proper time to reduce the impact of recovery traffic

2016-02-02 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9657: Status: Patch Available (was: Open) > Schedule EC tasks at proper time to reduce the impact of recovery traffic >

[jira] [Updated] (HDFS-9657) Schedule EC tasks at proper time to reduce the impact of recovery traffic

2016-01-27 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9657: Attachment: HDFS-9657-001.patch > Schedule EC tasks at proper time to reduce the impact of recovery traffic >

[jira] [Created] (HDFS-9657) Schedule EC tasks at proper time to reduce the impact of recovery traffic

2016-01-18 Thread Li Bo (JIRA)
Li Bo created HDFS-9657: --- Summary: Schedule EC tasks at proper time to reduce the impact of recovery traffic Key: HDFS-9657 URL: https://issues.apache.org/jira/browse/HDFS-9657 Project: Hadoop HDFS

[jira] [Updated] (HDFS-9374) Inform user when a file with corrupted data blocks are read

2016-01-14 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9374: Attachment: HDFS-9374-001.patch Another way is to add the decode info to ReadStatistics. > Inform user when a file

[jira] [Reopened] (HDFS-7346) Erasure Coding: perform stripping erasure encoding work given block reader and writer

2016-01-13 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo reopened HDFS-7346: - > Erasure Coding: perform stripping erasure encoding work given block reader > and writer >

[jira] [Updated] (HDFS-7346) Erasure Coding: perform stripping erasure encoding work given block reader and writer

2016-01-13 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-7346: Release Note: (was: The jira is very old and close it because we'll not handle it in the near future.) > Erasure

[jira] [Resolved] (HDFS-7346) Erasure Coding: perform stripping erasure encoding work given block reader and writer

2016-01-12 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo resolved HDFS-7346. - Resolution: Won't Fix Release Note: The jira is very old and close it because we'll not handle it in the

[jira] [Resolved] (HDFS-8889) Erasure Coding: cover more test situations of datanode failure during client writing

2016-01-12 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo resolved HDFS-8889. - Resolution: Won't Fix Release Note: There has been enough test cases for the datanode failures. > Erasure

[jira] [Updated] (HDFS-8762) Erasure Coding: the log of each streamer should show its index

2016-01-11 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8762: Resolution: Duplicate Status: Resolved (was: Patch Available) Other jiras have added the necessary index.

[jira] [Work stopped] (HDFS-8171) Extend BlockSender to support multiple block data source

2016-01-11 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HDFS-8171 stopped by Li Bo. --- > Extend BlockSender to support multiple block data source >

[jira] [Updated] (HDFS-9373) Show friendly information to user when client succeeds the writing with some failed streamers

2015-12-16 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9373: Attachment: HDFS-9373-003.patch Sorry for the checkstyle error.Update patch 003 to fix it. > Show friendly

[jira] [Commented] (HDFS-9529) Extend Erasure Code to support POWER Chip acceleration

2015-12-08 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9529?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15047924#comment-15047924 ] Li Bo commented on HDFS-9529: - hi, wqijun Hadoop-11887 is under "Hadoop common" and this JIRA is trying to

[jira] [Updated] (HDFS-9373) Show friendly information to user when client succeeds the writing with some failed streamers

2015-12-07 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9373: Attachment: HDFS-9373-002.patch Thanks Zhe and Daniel’s review. Just update the patch according the newest trunk

[jira] [Resolved] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-11-23 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo resolved HDFS-8704. - Resolution: Cannot Reproduce > Erasure Coding: client fails to write large file when one datanode fails >

[jira] [Commented] (HDFS-9382) Track the acks for the packets which are sent from ErasureCodingWorker as part of reconstruction work

2015-11-16 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15008062#comment-15008062 ] Li Bo commented on HDFS-9382: - Unifying the readers and writers is very complicated due to the different

[jira] [Updated] (HDFS-9373) Show friendly information to user when client succeeds the writing with some failed streamers

2015-11-11 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9373: Attachment: HDFS-9373-001.patch > Show friendly information to user when client succeeds the writing with some >

[jira] [Updated] (HDFS-7781) Use block erasure coder in client stripping

2015-11-11 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-7781: Parent Issue: HDFS-8030 (was: HDFS-8031) > Use block erasure coder in client stripping >

[jira] [Reopened] (HDFS-7781) Use block erasure coder in client stripping

2015-11-11 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo reopened HDFS-7781: - > Use block erasure coder in client stripping > --- > > Key:

[jira] [Updated] (HDFS-9373) Show friendly information to user when client succeeds the writing with some failed streamers

2015-11-08 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9373?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-9373: Description: When not more than PARITY_NUM streamers fail for a block group, the client may still succeed to write

[jira] [Created] (HDFS-9373) Show friendly information to user when client succeeds the writing with some failed streamers

2015-11-04 Thread Li Bo (JIRA)
Li Bo created HDFS-9373: --- Summary: Show friendly information to user when client succeeds the writing with some failed streamers Key: HDFS-9373 URL: https://issues.apache.org/jira/browse/HDFS-9373 Project:

[jira] [Created] (HDFS-9374) Inform user when a file with corrupted data blocks are read

2015-11-04 Thread Li Bo (JIRA)
Li Bo created HDFS-9374: --- Summary: Inform user when a file with corrupted data blocks are read Key: HDFS-9374 URL: https://issues.apache.org/jira/browse/HDFS-9374 Project: Hadoop HDFS Issue Type:

[jira] [Updated] (HDFS-8410) Add computation time metrics to datanode for ECWorker

2015-10-27 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8410: Attachment: HDFS-8410-002.patch > Add computation time metrics to datanode for ECWorker >

[jira] [Updated] (HDFS-8410) Add computation time metrics to datanode for ECWorker

2015-10-27 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8410: Summary: Add computation time metrics to datanode for ECWorker (was: Add time count metrics to datanode for

[jira] [Commented] (HDFS-8410) Add computation time metrics to datanode for ECWorker

2015-10-27 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14976021#comment-14976021 ] Li Bo commented on HDFS-8410: - Patch 002 reduces metrics number grom 3 to 2. The time metrics make user know

[jira] [Updated] (HDFS-8529) Add blocks count metrics to datanode for ECWorker

2015-10-27 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8529: Attachment: HDFS-8529-002.patch > Add blocks count metrics to datanode for ECWorker >

[jira] [Resolved] (HDFS-7653) Block Readers and Writers used in both client side and datanode side

2015-10-21 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo resolved HDFS-7653. - Resolution: Won't Fix Release Note: The design and implementation have changed a lot after this issue

[jira] [Resolved] (HDFS-7679) Erasure Coding: unifying common constructs like coding work, block reader and block writer across client and DataNode

2015-10-21 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7679?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo resolved HDFS-7679. - Resolution: Won't Fix > Erasure Coding: unifying common constructs like coding work, block reader and > block

[jira] [Resolved] (HDFS-7781) Use block erasure coder in client stripping

2015-10-21 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-7781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo resolved HDFS-7781. - Resolution: Won't Fix > Use block erasure coder in client stripping > ---

[jira] [Updated] (HDFS-8411) Add bytes count metrics to datanode for ECWorker

2015-10-21 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8411: Attachment: HDFS-8411-003.patch > Add bytes count metrics to datanode for ECWorker >

[jira] [Updated] (HDFS-8411) Add bytes count metrics to datanode for ECWorker

2015-10-15 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8411?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8411: Attachment: HDFS-8411-002.patch When some datanodes are corrupted, all their blocks are to be reconstructed by

[jira] [Commented] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2015-10-13 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14954670#comment-14954670 ] Li Bo commented on HDFS-8449: - Thanks [~rakeshr]'s review. The failed tests seem unrelated with this patch.

[jira] [Updated] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-10-09 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8704: Status: In Progress (was: Patch Available) > Erasure Coding: client fails to write large file when one datanode

[jira] [Commented] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-10-08 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14949702#comment-14949702 ] Li Bo commented on HDFS-8704: - Hi, Zhe I will give further check and test about this case. If no problem, I

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2015-09-29 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-002.patch reduce metrics number from 3 to 2. The count of failed tasks can be calculated by

[jira] [Updated] (HDFS-8449) Add tasks count metrics to datanode for ECWorker

2015-09-29 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8449?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8449: Attachment: HDFS-8449-003.patch Thanks Rakesh for the careful review. Update a new patch to fix the problems. > Add

[jira] [Commented] (HDFS-9040) Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests to Coordinator)

2015-09-22 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902207#comment-14902207 ] Li Bo commented on HDFS-9040: - I have just done a test with and without HDFS-9040-HDFS-7285.003.patch. The

[jira] [Commented] (HDFS-9040) Erasure coding: Refactor DFSStripedOutputStream (Move Namenode RPC Requests to Coordinator)

2015-09-17 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14805014#comment-14805014 ] Li Bo commented on HDFS-9040: - Thanks for the deep discussion here. One point confused me: bq. Only based on

[jira] [Commented] (HDFS-9040) Erasure coding: A BlockGroupDataStreamer to rule all internal blocks streamers

2015-09-14 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14743182#comment-14743182 ] Li Bo commented on HDFS-9040: - Thanks Walter and Jing ‘s work. In Jing’s patch, {{allocateNewBlock}} is called

[jira] [Commented] (HDFS-9040) Erasure coding: A BlockGroupDataStreamer to rule all internal blocks streamers

2015-09-09 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-9040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14738076#comment-14738076 ] Li Bo commented on HDFS-9040: - Implementing a block group writer is a better solution but you have to copy a

[jira] [Updated] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-09-08 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8704: Attachment: HDFS-8704-HDFS-7285-008.patch > Erasure Coding: client fails to write large file when one datanode fails

[jira] [Commented] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-09-06 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14733248#comment-14733248 ] Li Bo commented on HDFS-8704: - I have tried replacing the failed streamer with a new one. When replacing, the

[jira] [Updated] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-09-06 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8704: Attachment: HDFS-8704-HDFS-7285-007.patch > Erasure Coding: client fails to write large file when one datanode fails

[jira] [Commented] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-09-06 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14732314#comment-14732314 ] Li Bo commented on HDFS-8704: - Thanks for [~walter.k.su] and [~zhz]' s review. The cause of failed test cases

[jira] [Commented] (HDFS-8383) Tolerate multiple failures in DFSStripedOutputStream

2015-09-06 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14732320#comment-14732320 ] Li Bo commented on HDFS-8383: - Thanks [~walter.k.su] for the work! I have just read the code and find some

[jira] [Commented] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-08-28 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14718235#comment-14718235 ] Li Bo commented on HDFS-8704: - The two failed test cases are about insufficient datanodes. They

[jira] [Updated] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-08-27 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8704: Attachment: HDFS-8704-HDFS-7285-006.patch Update the patch based on current code of 7285 Erasure Coding: client

[jira] [Commented] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-08-26 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14712581#comment-14712581 ] Li Bo commented on HDFS-8704: - Thanks Zhe’s review. The handle of multiple block groups for

[jira] [Updated] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-08-26 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8704: Description: I test current code on a 5-node cluster using RS(3,2). When a datanode is corrupt, client succeeds to

[jira] [Commented] (HDFS-8838) Tolerate datanode failures in DFSStripedOutputStream when the data length is small

2015-08-25 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710695#comment-14710695 ] Li Bo commented on HDFS-8838: - hi, Walter Su 我看了下branch, 跟7285相关的有 remotes/origin/HDFS-7285

[jira] [Updated] (HDFS-8889) Erasure Coding: cover more test situations of datanode failure during client writing

2015-08-24 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8889: Attachment: HDFS-8889-HDFS-7285-001.patch Erasure Coding: cover more test situations of datanode failure during

[jira] [Commented] (HDFS-8838) Tolerate datanode failures in DFSStripedOutputStream when the data length is small

2015-08-24 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708942#comment-14708942 ] Li Bo commented on HDFS-8838: - hi, Jing, some error still exists when applying to newest code.

[jira] [Commented] (HDFS-8838) Tolerate datanode failures in DFSStripedOutputStream when the data length is small

2015-08-24 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14708943#comment-14708943 ] Li Bo commented on HDFS-8838: - hi, Jing, some error still exists when applying to newest code.

[jira] [Updated] (HDFS-8889) Erasure Coding: cover more test situations of datanode failure during client writing

2015-08-12 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8889?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8889: Description: Currently 9 streamers are working together for the client writing. A small number of failed datanodes

[jira] [Commented] (HDFS-8889) Erasure Coding: cover more test situations of datanode failure during client writing

2015-08-12 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14693047#comment-14693047 ] Li Bo commented on HDFS-8889: - Currently 9 streamers are working together for the client

[jira] [Created] (HDFS-8889) Erasure Coding: cover more test situations of datanode failure during client writing

2015-08-11 Thread Li Bo (JIRA)
Li Bo created HDFS-8889: --- Summary: Erasure Coding: cover more test situations of datanode failure during client writing Key: HDFS-8889 URL: https://issues.apache.org/jira/browse/HDFS-8889 Project: Hadoop HDFS

[jira] [Commented] (HDFS-8857) Erasure Coding: Fix ArrayIndexOutOfBoundsException in TestWriteStripedFileWithFailure

2015-08-05 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14659363#comment-14659363 ] Li Bo commented on HDFS-8857: - Thanks Kai for the review. I have just committed the code to

[jira] [Commented] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-08-05 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14654873#comment-14654873 ] Li Bo commented on HDFS-8704: - The failure of

[jira] [Updated] (HDFS-8857) Erasure Coding: Fix ArrayIndexOutOfBoundsException in TestWriteStripedFileWithFailure

2015-08-05 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8857: Resolution: Fixed Status: Resolved (was: Patch Available) Erasure Coding: Fix

[jira] [Updated] (HDFS-8704) Erasure Coding: client fails to write large file when one datanode fails

2015-08-04 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Bo updated HDFS-8704: Attachment: HDFS-8704-HDFS-7285-005.patch Erasure Coding: client fails to write large file when one datanode fails

[jira] [Commented] (HDFS-8838) Tolerate datanode failures in DFSStripedOutputStream when the data length is small

2015-08-04 Thread Li Bo (JIRA)
[ https://issues.apache.org/jira/browse/HDFS-8838?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14653136#comment-14653136 ] Li Bo commented on HDFS-8838: - hi, [~walter.k.su] and [~szetszwo], could you help me review the

  1   2   3   4   >