[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402580#comment-16402580 ] Íñigo Goiri commented on HDFS-12514: Is there a unit test we can add? Ideally we could reset it but mocking the exception is fine too. > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514-branch-2.000.patch, HDFS-12514.001.patch, > HDFS-12514.002.patch, HDFS-12514.003.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402508#comment-16402508 ] genericqa commented on HDFS-12514: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 32s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 18m 48s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 4s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 52s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 13s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 12m 4s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 10s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 58s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m 14s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m 7s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 52s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 37 unchanged - 1 fixed = 37 total (was 38) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 13s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 11m 7s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 19s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 31s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 24s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}171m 54s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestSafeModeWithStripedFileWithRandomECPolicy | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f | | JIRA Issue | HDFS-12514 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12914901/HDFS-12514.003.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux 216f8f28cc25 3.13.0-135-generic #184-Ubuntu SMP Wed Oct 18 11:55:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/patchprocess/precommit/personality/provided.sh | | git revision | trunk / 08ff158 | | maven | version: Apache Maven 3.3.9 | | Default Java | 1.8.0_151 | | findbugs | v3.1.0-RC1 | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/23518/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/23518/testReport/ | | Max. process+thread count | 2598 (vs. ulimit of 1) | | modules | C:
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16402261#comment-16402261 ] Lukas Majercak commented on HDFS-12514: --- Reuploaded patch 003 to trigger a build. > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514-branch-2.000.patch, HDFS-12514.001.patch, > HDFS-12514.002.patch, HDFS-12514.003.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397660#comment-16397660 ] Lukas Majercak commented on HDFS-12514: --- Fixed the trunk patch and added one for branch-2. > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514-branch-2.000.patch, HDFS-12514.001.patch, > HDFS-12514.002.patch, HDFS-12514.003.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16397173#comment-16397173 ] Íñigo Goiri commented on HDFS-12514: FindBugs is complaining because I forgot to remove the {{e.getMessage()}} from the main class. In any case, would you mind providing a fixed version? trunk changed and won't apply to the others; could you also provide patches for branch-2 and others? > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514.001.patch, HDFS-12514.002.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396492#comment-16396492 ] genericqa commented on HDFS-12514: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 25s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 15m 27s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 51s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 45s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 57s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 10m 52s{color} | {color:green} branch has no errors when building and testing our client artifacts. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 47s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 53s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 54s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 49s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 42s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 37 unchanged - 1 fixed = 37 total (was 38) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} shadedclient {color} | {color:green} 9m 57s{color} | {color:green} patch has no errors when building and testing our client artifacts. {color} | | {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m 54s{color} | {color:red} hadoop-hdfs-project/hadoop-hdfs generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 52s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 89m 15s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 23s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}137m 20s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-hdfs-project/hadoop-hdfs | | | Dead store to ioem in org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(ByteBuffer, int, OutputStream, boolean, DataTransferThrottler) At BlockSender.java:org.apache.hadoop.hdfs.server.datanode.BlockSender.sendPacket(ByteBuffer, int, OutputStream, boolean, DataTransferThrottler) At BlockSender.java:[line 661] | | Failed junit tests | hadoop.hdfs.TestDFSStripedOutputStreamWithFailure | | | hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes | | | hadoop.hdfs.web.TestWebHdfsTimeouts | | | hadoop.hdfs.server.namenode.TestDecommissioningStatus | \\ \\ || Subsystem || Report/Notes || | Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hadoop:d4cc50f | | JIRA Issue | HDFS-12514 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12914195/HDFS-12514.002.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit shadedclient findbugs checkstyle | | uname | Linux c999e80fbfd2
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396357#comment-16396357 ] Íñigo Goiri commented on HDFS-12514: [^HDFS-12514.001.patch] doesn't apply anymore; rebased and posted [^HDFS-12514.002.patch]. If it comes clean, I'd like to commit this. > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514.001.patch, HDFS-12514.002.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396349#comment-16396349 ] genericqa commented on HDFS-12514: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 0s{color} | {color:blue} Docker mode activated. {color} | | {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color} | {color:red} HDFS-12514 does not apply to trunk. Rebase required? Wrong Branch? See https://wiki.apache.org/hadoop/HowToContribute for help. {color} | \\ \\ || Subsystem || Report/Notes || | JIRA Issue | HDFS-12514 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12888086/HDFS-12514.001.patch | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/23437/console | | Powered by | Apache Yetus 0.8.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514.001.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16396336#comment-16396336 ] Lukas Majercak commented on HDFS-12514: --- [~shahrs87], "Software caused connection abort" will not be in the exception message, the same way as "Connection reset by peer" is not in the message for *WSAECONNRESET.* > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak >Priority: Major > Attachments: HDFS-12514.001.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188404#comment-16188404 ] Íñigo Goiri commented on HDFS-12514: Thanks [~shahrs87] for the comment. It's true that we should check the rest of the string. At this point I would move away from the {{startsWith}} pattern into a regexp though. > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak > Attachments: HDFS-12514.001.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16188384#comment-16188384 ] Rushabh S Shah commented on HDFS-12514: --- bq. This is related to what HDFS-10627 and HDFS-7686. Rushabh S Shah, Colin P. McCabe, opinions on this? [~elgoiri] The list of exceptions to be ignored seems good to me. I have one minor comment on the patch. {noformat} + // WSAECONNABORTED + "An established connection was aborted by the software in your " + + "host machine", {noformat} I haven't developed on Windows environment so this comment might not make sense. >From the following windows documentation >https://msdn.microsoft.com/en-us/library/windows/desktop/ms740668(v=vs.85).aspx it says, |WSAECONNABORTED| 10053 |Software caused connection abort. An established connection was aborted by the software in your host computer, possibly due to a data transmission time-out or protocol error.| Will the exception message start with {{An established connection was aborted}} or {{Software caused connection abort.}} If it does starts with {{An established connection was aborted by the software in your host machine}}, then check the last few words in documentation. It is {{host computer}}. But in the patch it is {{host machine}} so the {{String#startsWith(prefix)}} will fail. > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak > Attachments: HDFS-12514.001.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186305#comment-16186305 ] Íñigo Goiri commented on HDFS-12514: This is related to what HDFS-10627 and HDFS-7686. [~shahrs87], [~cmccabe], opinions on this? > Cancelled HedgedReads cause block to be marked as suspect on Windows > > > Key: HDFS-12514 > URL: https://issues.apache.org/jira/browse/HDFS-12514 > Project: Hadoop HDFS > Issue Type: Bug > Components: datanode, hdfs-client >Reporter: Lukas Majercak >Assignee: Lukas Majercak > Attachments: HDFS-12514.001.patch > > > DFSClient with hedged reads enabled will often close previous spawned > connections if it successfully reads from one of them. This can result in > DataNode's BlockSender getting a socket exception and wrongly marking the > block as suspect and to be rescanned for errors. > This patch is aimed at adding windows specific network related exception > messages to be ignored in BlockSender.sendPacket. -- This message was sent by Atlassian JIRA (v6.4.14#64029) - To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
[jira] [Commented] (HDFS-12514) Cancelled HedgedReads cause block to be marked as suspect on Windows
[ https://issues.apache.org/jira/browse/HDFS-12514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16173686#comment-16173686 ] Hadoop QA commented on HDFS-12514: -- | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m 22s{color} | {color:blue} Docker mode activated. {color} | || || || || {color:brown} Prechecks {color} || | {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m 0s{color} | {color:green} The patch does not contain any @author tags. {color} | | {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m 0s{color} | {color:red} The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color} | || || || || {color:brown} trunk Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 14m 54s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 56s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 41s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 1m 0s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m 54s{color} | {color:green} trunk passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 45s{color} | {color:green} trunk passed {color} | || || || || {color:brown} Patch Compile Tests {color} || | {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m 57s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m 53s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m 36s{color} | {color:green} hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 37 unchanged - 1 fixed = 37 total (was 38) {color} | | {color:green}+1{color} | {color:green} mvnsite {color} | {color:green} 0m 59s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m 0s{color} | {color:green} The patch has no whitespace issues. {color} | | {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m 2s{color} | {color:green} the patch passed {color} | | {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m 43s{color} | {color:green} the patch passed {color} | || || || || {color:brown} Other Tests {color} || | {color:red}-1{color} | {color:red} unit {color} | {color:red} 93m 2s{color} | {color:red} hadoop-hdfs in the patch failed. {color} | | {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m 17s{color} | {color:green} The patch does not generate ASF License warnings. {color} | | {color:black}{color} | {color:black} {color} | {color:black}121m 47s{color} | {color:black} {color} | \\ \\ || Reason || Tests || | Failed junit tests | hadoop.hdfs.TestReadStripedFileWithMissingBlocks | | | hadoop.hdfs.server.namenode.ha.TestPipelinesFailover | | | hadoop.hdfs.TestDFSClientRetries | | | hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure | \\ \\ || Subsystem || Report/Notes || | Docker | Image:yetus/hadoop:71bbb86 | | JIRA Issue | HDFS-12514 | | JIRA Patch URL | https://issues.apache.org/jira/secure/attachment/12888086/HDFS-12514.001.patch | | Optional Tests | asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle | | uname | Linux bd1dd4759747 3.13.0-129-generic #178-Ubuntu SMP Fri Aug 11 12:48:20 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /testptch/hadoop/patchprocess/precommit/personality/provided.sh | | git revision | trunk / ce943eb | | Default Java | 1.8.0_144 | | findbugs | v3.1.0-RC1 | | unit | https://builds.apache.org/job/PreCommit-HDFS-Build/21246/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt | | Test Results | https://builds.apache.org/job/PreCommit-HDFS-Build/21246/testReport/ | | modules | C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs | | Console output | https://builds.apache.org/job/PreCommit-HDFS-Build/21246/console | | Powered by | Apache Yetus 0.6.0-SNAPSHOT http://yetus.apache.org | This message was automatically generated. > Cancelled HedgedReads cause block to be marked as suspect on Windows >