[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2018-03-16 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402580#comment-16402580
 ] 

Íñigo Goiri commented on HDFS-12514:


Is there a unit test we can add? Ideally we could reset it but mocking the 
exception is fine too.

> Cancelled HedgedReads cause block to be marked as suspect on Windows
> 
>
> Key: HDFS-12514
> URL: https://issues.apache.org/jira/browse/HDFS-12514
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, hdfs-client
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-12514-branch-2.000.patch, HDFS-12514.001.patch, 
> HDFS-12514.002.patch, HDFS-12514.003.patch
>
>
> DFSClient with hedged reads enabled will often close previous spawned 
> connections if it successfully reads from one of them. This can result in 
> DataNode's BlockSender getting a socket exception and wrongly marking the 
> block as suspect and to be rescanned for errors.
> This patch is aimed at adding windows specific network related exception 
> messages to be ignored in BlockSender.sendPacket.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2018-03-16 Thread genericqa (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402508#comment-16402508
 ] 

genericqa commented on HDFS-12514:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
32s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 
48s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
4s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
52s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
13s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
12m  4s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
10s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
52s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated 
0 new + 37 unchanged - 1 fixed = 37 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m 
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
11m  7s{color} | {color:green} patch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 31s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
24s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}171m 54s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | 
hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-12514 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12914901/HDFS-12514.003.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  |
| uname | Linux 216f8f28cc25 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 
11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | trunk / 08ff158 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_151 |
| findbugs | v3.1.0-RC1 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/23518/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/23518/testReport/ |
| Max. process+thread count | 2598 (vs. ulimit of 1) |
| modules | C:

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2018-03-16 Thread Lukas Majercak (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402261#comment-16402261
 ] 

Lukas Majercak commented on HDFS-12514:
---

Reuploaded patch 003 to trigger a build.

> Cancelled HedgedReads cause block to be marked as suspect on Windows
> 
>
> Key: HDFS-12514
> URL: https://issues.apache.org/jira/browse/HDFS-12514
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, hdfs-client
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-12514-branch-2.000.patch, HDFS-12514.001.patch, 
> HDFS-12514.002.patch, HDFS-12514.003.patch
>
>
> DFSClient with hedged reads enabled will often close previous spawned 
> connections if it successfully reads from one of them. This can result in 
> DataNode's BlockSender getting a socket exception and wrongly marking the 
> block as suspect and to be rescanned for errors.
> This patch is aimed at adding windows specific network related exception 
> messages to be ignored in BlockSender.sendPacket.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2018-03-13 Thread Lukas Majercak (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397660#comment-16397660
 ] 

Lukas Majercak commented on HDFS-12514:
---

Fixed the trunk patch and added one for branch-2.

> Cancelled HedgedReads cause block to be marked as suspect on Windows
> 
>
> Key: HDFS-12514
> URL: https://issues.apache.org/jira/browse/HDFS-12514
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, hdfs-client
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-12514-branch-2.000.patch, HDFS-12514.001.patch, 
> HDFS-12514.002.patch, HDFS-12514.003.patch
>
>
> DFSClient with hedged reads enabled will often close previous spawned 
> connections if it successfully reads from one of them. This can result in 
> DataNode's BlockSender getting a socket exception and wrongly marking the 
> block as suspect and to be rescanned for errors.
> This patch is aimed at adding windows specific network related exception 
> messages to be ignored in BlockSender.sendPacket.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2018-03-13 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397173#comment-16397173
 ] 

Íñigo Goiri commented on HDFS-12514:


FindBugs is complaining because I forgot to remove the {{e.getMessage()}} from 
the main class.
In any case, would you mind providing a fixed version?
trunk changed and won't apply to the others; could you also provide patches for 
branch-2 and others? 

> Cancelled HedgedReads cause block to be marked as suspect on Windows
> 
>
> Key: HDFS-12514
> URL: https://issues.apache.org/jira/browse/HDFS-12514
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, hdfs-client
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-12514.001.patch, HDFS-12514.002.patch
>
>
> DFSClient with hedged reads enabled will often close previous spawned 
> connections if it successfully reads from one of them. This can result in 
> DataNode's BlockSender getting a socket exception and wrongly marking the 
> block as suspect and to be rescanned for errors.
> This patch is aimed at adding windows specific network related exception 
> messages to be ignored in BlockSender.sendPacket.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2018-03-12 Thread genericqa (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396492#comment-16396492
 ] 

genericqa commented on HDFS-12514:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
25s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 
27s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
51s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
45s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
57s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 
10m 52s{color} | {color:green} branch has no errors when building and testing 
our client artifacts. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
47s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
53s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
42s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated 
0 new + 37 unchanged - 1 fixed = 37 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedclient {color} | {color:green}  
9m 57s{color} | {color:green} patch has no errors when building and testing our 
client artifacts. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m 
54s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 
unchanged - 0 fixed = 1 total (was 0) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 15s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
23s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}137m 20s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:hadoop-hdfs-project/hadoop-hdfs |
|  |  Dead store to ioem in 
org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(ByteBuffer, int, 
OutputStream, boolean, DataTransferThrottler)  At 
BlockSender.java:org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(ByteBuffer,
 int, OutputStream, boolean, DataTransferThrottler)  At BlockSender.java:[line 
661] |
| Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure |
|   | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes |
|   | hadoop.hdfs.web.TestWebHdfsTimeouts |
|   | hadoop.hdfs.server.namenode.TestDecommissioningStatus |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f |
| JIRA Issue | HDFS-12514 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12914195/HDFS-12514.002.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  shadedclient  findbugs  checkstyle  |
| uname | Linux c999e80fbfd2

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2018-03-12 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396357#comment-16396357
 ] 

Íñigo Goiri commented on HDFS-12514:


[^HDFS-12514.001.patch] doesn't apply anymore; rebased and posted  
[^HDFS-12514.002.patch].
If it comes clean, I'd like to commit this.

> Cancelled HedgedReads cause block to be marked as suspect on Windows
> 
>
> Key: HDFS-12514
> URL: https://issues.apache.org/jira/browse/HDFS-12514
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, hdfs-client
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-12514.001.patch, HDFS-12514.002.patch
>
>
> DFSClient with hedged reads enabled will often close previous spawned 
> connections if it successfully reads from one of them. This can result in 
> DataNode's BlockSender getting a socket exception and wrongly marking the 
> block as suspect and to be rescanned for errors.
> This patch is aimed at adding windows specific network related exception 
> messages to be ignored in BlockSender.sendPacket.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2018-03-12 Thread genericqa (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396349#comment-16396349
 ] 

genericqa commented on HDFS-12514:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m  
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m  5s{color} 
| {color:red} HDFS-12514 does not apply to trunk. Rebase required? Wrong 
Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | HDFS-12514 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12888086/HDFS-12514.001.patch |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/23437/console |
| Powered by | Apache Yetus 0.8.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Cancelled HedgedReads cause block to be marked as suspect on Windows
> 
>
> Key: HDFS-12514
> URL: https://issues.apache.org/jira/browse/HDFS-12514
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, hdfs-client
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-12514.001.patch
>
>
> DFSClient with hedged reads enabled will often close previous spawned 
> connections if it successfully reads from one of them. This can result in 
> DataNode's BlockSender getting a socket exception and wrongly marking the 
> block as suspect and to be rescanned for errors.
> This patch is aimed at adding windows specific network related exception 
> messages to be ignored in BlockSender.sendPacket.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2018-03-12 Thread Lukas Majercak (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396336#comment-16396336
 ] 

Lukas Majercak commented on HDFS-12514:
---

[~shahrs87],  "Software caused connection abort" will not be in the exception 
message, the same way as "Connection reset by peer" is not in the message for 
*WSAECONNRESET.*

> Cancelled HedgedReads cause block to be marked as suspect on Windows
> 
>
> Key: HDFS-12514
> URL: https://issues.apache.org/jira/browse/HDFS-12514
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, hdfs-client
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
>Priority: Major
> Attachments: HDFS-12514.001.patch
>
>
> DFSClient with hedged reads enabled will often close previous spawned 
> connections if it successfully reads from one of them. This can result in 
> DataNode's BlockSender getting a socket exception and wrongly marking the 
> block as suspect and to be rescanned for errors.
> This patch is aimed at adding windows specific network related exception 
> messages to be ignored in BlockSender.sendPacket.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2017-10-02 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188404#comment-16188404
 ] 

Íñigo Goiri commented on HDFS-12514:


Thanks [~shahrs87] for the comment.
It's true that we should check the rest of the string.
At this point I would move away from the {{startsWith}} pattern into a regexp 
though.


> Cancelled HedgedReads cause block to be marked as suspect on Windows
> 
>
> Key: HDFS-12514
> URL: https://issues.apache.org/jira/browse/HDFS-12514
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, hdfs-client
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
> Attachments: HDFS-12514.001.patch
>
>
> DFSClient with hedged reads enabled will often close previous spawned 
> connections if it successfully reads from one of them. This can result in 
> DataNode's BlockSender getting a socket exception and wrongly marking the 
> block as suspect and to be rescanned for errors.
> This patch is aimed at adding windows specific network related exception 
> messages to be ignored in BlockSender.sendPacket.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2017-10-02 Thread Rushabh S Shah (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188384#comment-16188384
 ] 

Rushabh S Shah commented on HDFS-12514:
---

bq. This is related to what HDFS-10627 and HDFS-7686. Rushabh S Shah, Colin P. 
McCabe, opinions on this?
[~elgoiri] The list of exceptions to be ignored seems  good to me.

I have one minor comment on the patch.
{noformat}
+  // WSAECONNABORTED
+  "An established connection was aborted by the software in your " +
+  "host machine",
{noformat}
I haven't developed on Windows environment so this comment might not make sense.
>From the following windows documentation 
>https://msdn.microsoft.com/en-us/library/windows/desktop/ms740668(v=vs.85).aspx
it says,
|WSAECONNABORTED| 10053 |Software caused connection abort.
An established connection was aborted by the software in your host computer, 
possibly due to a data transmission time-out or protocol error.|

Will the exception message start with {{An established connection was aborted}} 
or {{Software caused connection abort.}}
If it does starts with {{An established connection was aborted by the software 
in your host machine}}, then check the last few words in documentation.
It is {{host computer}}. But in the patch it is {{host machine}} so the 
{{String#startsWith(prefix)}} will fail.




> Cancelled HedgedReads cause block to be marked as suspect on Windows
> 
>
> Key: HDFS-12514
> URL: https://issues.apache.org/jira/browse/HDFS-12514
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, hdfs-client
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
> Attachments: HDFS-12514.001.patch
>
>
> DFSClient with hedged reads enabled will often close previous spawned 
> connections if it successfully reads from one of them. This can result in 
> DataNode's BlockSender getting a socket exception and wrongly marking the 
> block as suspect and to be rescanned for errors.
> This patch is aimed at adding windows specific network related exception 
> messages to be ignored in BlockSender.sendPacket.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2017-09-29 Thread JIRA 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186305#comment-16186305
 ] 

Íñigo Goiri commented on HDFS-12514:


This is related to what HDFS-10627 and HDFS-7686. [~shahrs87], [~cmccabe], 
opinions on this?


> Cancelled HedgedReads cause block to be marked as suspect on Windows
> 
>
> Key: HDFS-12514
> URL: https://issues.apache.org/jira/browse/HDFS-12514
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: datanode, hdfs-client
>Reporter: Lukas Majercak
>Assignee: Lukas Majercak
> Attachments: HDFS-12514.001.patch
>
>
> DFSClient with hedged reads enabled will often close previous spawned 
> connections if it successfully reads from one of them. This can result in 
> DataNode's BlockSender getting a socket exception and wrongly marking the 
> block as suspect and to be rescanned for errors.
> This patch is aimed at adding windows specific network related exception 
> messages to be ignored in BlockSender.sendPacket.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org

[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows

2017-09-20 Thread Hadoop QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16173686#comment-16173686
 ] 

Hadoop QA commented on HDFS-12514:
--

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue}  0m 
22s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red}  0m  
0s{color} | {color:red} The patch doesn't appear to include any new or modified 
tests. Please justify why no new tests are needed for this patch. Also please 
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} trunk Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m 
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
56s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
41s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  1m  
0s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  1m 
54s{color} | {color:green} trunk passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
45s{color} | {color:green} trunk passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
36s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated 
0 new + 37 unchanged - 1 fixed = 37 total (was 38) {color} |
| {color:green}+1{color} | {color:green} mvnsite {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  2m  
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
43s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m  2s{color} 
| {color:red} hadoop-hdfs in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}121m 47s{color} | 
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | hadoop.hdfs.TestReadStripedFileWithMissingBlocks |
|   | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover |
|   | hadoop.hdfs.TestDFSClientRetries |
|   | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure |
\\
\\
|| Subsystem || Report/Notes ||
| Docker |  Image:yetus/hadoop:71bbb86 |
| JIRA Issue | HDFS-12514 |
| JIRA Patch URL | 
https://issues.apache.org/jira/secure/attachment/12888086/HDFS-12514.001.patch |
| Optional Tests |  asflicense  compile  javac  javadoc  mvninstall  mvnsite  
unit  findbugs  checkstyle  |
| uname | Linux bd1dd4759747 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 
12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh 
|
| git revision | trunk / ce943eb |
| Default Java | 1.8.0_144 |
| findbugs | v3.1.0-RC1 |
| unit | 
https://builds.apache.org/job/PreCommit-HDFS-Build/21246/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 |
|  Test Results | 
https://builds.apache.org/job/PreCommit-HDFS-Build/21246/testReport/ |
| modules | C: hadoop-hdfs-project/hadoop-hdfs U: 
hadoop-hdfs-project/hadoop-hdfs |
| Console output | 
https://builds.apache.org/job/PreCommit-HDFS-Build/21246/console |
| Powered by | Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org |


This message was automatically generated.



> Cancelled HedgedReads cause block to be marked as suspect on Windows
>