[jira] [Commented] (HDFS-8658) Improve retry attempt log in DFSOutputStream#completeFile with current attempt number

2015-12-21 Thread Kanaka Kumar Avvaru (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15067586#comment-15067586
 ] 

Kanaka Kumar Avvaru commented on HDFS-8658:
---

[~ajisakaa], Please have a look at updated patch

> Improve retry attempt log in DFSOutputStream#completeFile  with current 
> attempt number
> --
>
> Key: HDFS-8658
> URL: https://issues.apache.org/jira/browse/HDFS-8658
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Kanaka Kumar Avvaru
>Assignee: Kanaka Kumar Avvaru
>Priority: Minor
> Attachments: HDFS-8658-00.patch, HDFS-8658-01.patch
>
>
> In a scenario when block update was delayed from DN to NN, following client 
> log is observed which is confusing that file close failed after two attempts 
> but it tried 5 times actually. So, it's better to log the current and 
> remaining attempt counts.
> {code}
> 15/06/23 17:05:10 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> 15/06/23 17:05:16 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> copyFromLocal: Unable to close file because the last block does not have 
> enough number of replicas.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-8658) Improve retry attempt log in DFSOutputStream#completeFile with current attempt number

2015-12-16 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059942#comment-15059942
 ] 

Hadoop QA commented on HDFS-8658:
-

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s 
{color} | {color:blue} Docker mode activated. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s 
{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s 
{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 20m 
25s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 2m 23s 
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 33s 
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
29s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 17s 
{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
24s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m 
13s {color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 23s 
{color} | {color:green} trunk passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 1s 
{color} | {color:green} trunk passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 
26s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 49s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 49s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 13s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 13s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 31s 
{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m 
27s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 
0s {color} | {color:green} Patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m 
15s {color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 19s 
{color} | {color:green} the patch passed with JDK v1.8.0_66 {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m 3s 
{color} | {color:green} the patch passed with JDK v1.7.0_91 {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m 10s 
{color} | {color:green} hadoop-hdfs-client in the patch passed with JDK 
v1.8.0_66. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m 31s 
{color} | {color:green} hadoop-hdfs-client in the patch passed with JDK 
v1.7.0_91. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 
49s {color} | {color:green} Patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 61m 55s {color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:0ca8df7 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12777977/HDFS-8658-01.patch |
| JIRA Issue | HDFS-8658 |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux 72e340fb86b4 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed 
Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/persona

[jira] [Commented] (HDFS-8658) Improve retry attempt log in DFSOutputStream#completeFile with current attempt number

2015-12-16 Thread Kanaka Kumar Avvaru (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15059882#comment-15059882
 ] 

Kanaka Kumar Avvaru commented on HDFS-8658:
---

Thanks for taking up a look at my patch [~ajisakaa].  Sorry for late response. 
Yes logs may confuse when debug level is not enabled. I have updated patch to 
use INFO level log for every attempt. 

Now the logs should look like 
{code}
INFO: Could not complete  retrying... current attempt is 1 out of 5
INFO: Could not complete  retrying... current attempt is 2 out of 5
INFO: Could not complete  retrying... current attempt is 3 out of 5
INFO: Could not complete  retrying... current attempt is 4 out of 5
INFO: Could not complete  retrying... current attempt is 5 out of 5 
{code}


> Improve retry attempt log in DFSOutputStream#completeFile  with current 
> attempt number
> --
>
> Key: HDFS-8658
> URL: https://issues.apache.org/jira/browse/HDFS-8658
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Kanaka Kumar Avvaru
>Assignee: Kanaka Kumar Avvaru
>Priority: Minor
> Attachments: HDFS-8658-00.patch, HDFS-8658-01.patch
>
>
> In a scenario when block update was delayed from DN to NN, following client 
> log is observed which is confusing that file close failed after two attempts 
> but it tried 5 times actually. So, it's better to log the current and 
> remaining attempt counts.
> {code}
> 15/06/23 17:05:10 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> 15/06/23 17:05:16 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> copyFromLocal: Unable to close file because the last block does not have 
> enough number of replicas.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-8658) Improve retry attempt log in DFSOutputStream#completeFile with current attempt number

2015-10-27 Thread Akira AJISAKA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14977802#comment-14977802
 ] 

Akira AJISAKA commented on HDFS-8658:
-

After the patch, the log would be more confusing for me. The log is as follows:
{noformat}
WARN: Could not complete  retrying... current attempt is 2 out of 5
WARN: Could not complete  retrying... current attempt is 4 out of 5
{noformat}
Users might think "1st and 3rd attempts have done correctly, so why 2nd and 4th 
attempts are failing?" or "Why 1st and 3rd attempts are not logged (or why they 
are debug log)?"

IMO, it would be better to keep it as-is, or print all the attempts in the same 
log level.

> Improve retry attempt log in DFSOutputStream#completeFile  with current 
> attempt number
> --
>
> Key: HDFS-8658
> URL: https://issues.apache.org/jira/browse/HDFS-8658
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Kanaka Kumar Avvaru
>Assignee: Kanaka Kumar Avvaru
>Priority: Minor
> Attachments: HDFS-8658-00.patch
>
>
> In a scenario when block update was delayed from DN to NN, following client 
> log is observed which is confusing that file close failed after two attempts 
> but it tried 5 times actually. So, it's better to log the current and 
> remaining attempt counts.
> {code}
> 15/06/23 17:05:10 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> 15/06/23 17:05:16 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> copyFromLocal: Unable to close file because the last block does not have 
> enough number of replicas.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-8658) Improve retry attempt log in DFSOutputStream#completeFile with current attempt number

2015-09-16 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14790704#comment-14790704
 ] 

Hadoop QA commented on HDFS-8658:
-

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  18m  9s | Pre-patch trunk compilation is 
healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any 
@author tags. |
| {color:red}-1{color} | tests included |   0m  0s | The patch doesn't appear 
to include any new or modified tests.  Please justify why no new tests are 
needed for this patch. Also please list what manual steps were performed to 
verify this patch. |
| {color:green}+1{color} | javac |   8m  0s | There were no new javac warning 
messages. |
| {color:green}+1{color} | javadoc |  10m 16s | There were no new javadoc 
warning messages. |
| {color:green}+1{color} | release audit |   0m 24s | The applied patch does 
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle |   1m 21s | There were no new checkstyle 
issues. |
| {color:green}+1{color} | whitespace |   0m  0s | The patch has no lines that 
end in whitespace. |
| {color:green}+1{color} | install |   1m 28s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 33s | The patch built with 
eclipse:eclipse. |
| {color:green}+1{color} | findbugs |   2m 28s | The patch does not introduce 
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native |   3m 10s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 159m 15s | Tests failed in hadoop-hdfs. |
| | | 205m  7s | |
\\
\\
|| Reason || Tests ||
| Timed out tests | org.apache.hadoop.hdfs.server.mover.TestStorageMover |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12741618/HDFS-8658-00.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / bf2f2b4 |
| hadoop-hdfs test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/12482/artifact/patchprocess/testrun_hadoop-hdfs.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/12482/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/12482/console |


This message was automatically generated.

> Improve retry attempt log in DFSOutputStream#completeFile  with current 
> attempt number
> --
>
> Key: HDFS-8658
> URL: https://issues.apache.org/jira/browse/HDFS-8658
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Kanaka Kumar Avvaru
>Assignee: Kanaka Kumar Avvaru
>Priority: Minor
> Attachments: HDFS-8658-00.patch
>
>
> In a scenario when block update was delayed from DN to NN, following client 
> log is observed which is confusing that file close failed after two attempts 
> but it tried 5 times actually. So, it's better to log the current and 
> remaining attempt counts.
> {code}
> 15/06/23 17:05:10 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> 15/06/23 17:05:16 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> copyFromLocal: Unable to close file because the last block does not have 
> enough number of replicas.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HDFS-8658) Improve retry attempt log in DFSOutputStream#completeFile with current attempt number

2015-09-16 Thread Kanaka Kumar Avvaru (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14768913#comment-14768913
 ] 

Kanaka Kumar Avvaru commented on HDFS-8658:
---

Long lasting not reviewed JIRA. [~jingzhao], can you please review changes?

> Improve retry attempt log in DFSOutputStream#completeFile  with current 
> attempt number
> --
>
> Key: HDFS-8658
> URL: https://issues.apache.org/jira/browse/HDFS-8658
> Project: Hadoop HDFS
>  Issue Type: Sub-task
>Reporter: Kanaka Kumar Avvaru
>Assignee: Kanaka Kumar Avvaru
>Priority: Minor
> Attachments: HDFS-8658-00.patch
>
>
> In a scenario when block update was delayed from DN to NN, following client 
> log is observed which is confusing that file close failed after two attempts 
> but it tried 5 times actually. So, it's better to log the current and 
> remaining attempt counts.
> {code}
> 15/06/23 17:05:10 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> 15/06/23 17:05:16 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> copyFromLocal: Unable to close file because the last block does not have 
> enough number of replicas.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Auto-Re: [jira] [Commented] (HDFS-8658) Improve retry attempt log in DFSOutputStream#completeFile with current attempt number

2015-06-24 Thread wsb 
您的邮件已收到！谢谢！

[jira] [Commented] (HDFS-8658) Improve retry attempt log in DFSOutputStream#completeFile with current attempt number

2015-06-24 Thread kanaka kumar avvaru (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600728#comment-14600728
 ] 

kanaka kumar avvaru commented on HDFS-8658:
---

New test cases are not required as the patch adds/modifies only log messages. 
Test failures in the build are not related to this patch.

> Improve retry attempt log in DFSOutputStream#completeFile  with current 
> attempt number
> --
>
> Key: HDFS-8658
> URL: https://issues.apache.org/jira/browse/HDFS-8658
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: kanaka kumar avvaru
>Assignee: kanaka kumar avvaru
>Priority: Minor
> Attachments: HDFS-8658-00.patch
>
>
> In a scenario when block update was delayed from DN to NN, following client 
> log is observed which is confusing that file close failed after two attempts 
> but it tried 5 times actually. So, it's better to log the current and 
> remaining attempt counts.
> {code}
> 15/06/23 17:05:10 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> 15/06/23 17:05:16 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> copyFromLocal: Unable to close file because the last block does not have 
> enough number of replicas.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Auto-Re: [jira] [Commented] (HDFS-8658) Improve retry attempt log in DFSOutputStream#completeFile with current attempt number

2015-06-24 Thread wsb 
您的邮件已收到！谢谢！

[jira] [Commented] (HDFS-8658) Improve retry attempt log in DFSOutputStream#completeFile with current attempt number

2015-06-24 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-8658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14599793#comment-14599793
 ] 

Hadoop QA commented on HDFS-8658:
-

\\
\\
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | pre-patch |  18m 16s | Pre-patch trunk compilation is 
healthy. |
| {color:green}+1{color} | @author |   0m  0s | The patch does not contain any 
@author tags. |
| {color:red}-1{color} | tests included |   0m  0s | The patch doesn't appear 
to include any new or modified tests.  Please justify why no new tests are 
needed for this patch. Also please list what manual steps were performed to 
verify this patch. |
| {color:green}+1{color} | javac |   7m 38s | There were no new javac warning 
messages. |
| {color:green}+1{color} | javadoc |   9m 51s | There were no new javadoc 
warning messages. |
| {color:green}+1{color} | release audit |   0m 21s | The applied patch does 
not increase the total number of release audit warnings. |
| {color:green}+1{color} | checkstyle |   2m 21s | There were no new checkstyle 
issues. |
| {color:green}+1{color} | whitespace |   0m  0s | The patch has no lines that 
end in whitespace. |
| {color:green}+1{color} | install |   1m 35s | mvn install still works. |
| {color:green}+1{color} | eclipse:eclipse |   0m 33s | The patch built with 
eclipse:eclipse. |
| {color:green}+1{color} | findbugs |   3m 18s | The patch does not introduce 
any new Findbugs (version 3.0.0) warnings. |
| {color:green}+1{color} | native |   3m 19s | Pre-build of native portion |
| {color:red}-1{color} | hdfs tests | 118m 14s | Tests failed in hadoop-hdfs. |
| | | 165m 28s | |
\\
\\
|| Reason || Tests ||
| Failed unit tests | 
hadoop.hdfs.server.datanode.fsdataset.impl.TestRbwSpaceReservation |
| Timed out tests | 
org.apache.hadoop.hdfs.server.blockmanagement.TestBlockReportRateLimiting |
|   | 
org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistFiles |
\\
\\
|| Subsystem || Report/Notes ||
| Patch URL | 
http://issues.apache.org/jira/secure/attachment/12741618/HDFS-8658-00.patch |
| Optional Tests | javadoc javac unit findbugs checkstyle |
| git revision | trunk / 2ba6465 |
| hadoop-hdfs test log | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11464/artifact/patchprocess/testrun_hadoop-hdfs.txt
 |
| Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11464/testReport/ |
| Java | 1.7.0_55 |
| uname | Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP 
PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/11464/console |


This message was automatically generated.

> Improve retry attempt log in DFSOutputStream#completeFile  with current 
> attempt number
> --
>
> Key: HDFS-8658
> URL: https://issues.apache.org/jira/browse/HDFS-8658
> Project: Hadoop HDFS
>  Issue Type: Improvement
>Reporter: kanaka kumar avvaru
>Assignee: kanaka kumar avvaru
>Priority: Minor
> Attachments: HDFS-8658-00.patch
>
>
> In a scenario when block update was delayed from DN to NN, following client 
> log is observed which is confusing that file close failed after two attempts 
> but it tried 5 times actually. So, it's better to log the current and 
> remaining attempt counts.
> {code}
> 15/06/23 17:05:10 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> 15/06/23 17:05:16 INFO hdfs.DFSClient: Could not complete 
> /write2/tst5484._COPYING_ retrying...
> copyFromLocal: Unable to close file because the last block does not have 
> enough number of replicas.
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)