[jira] [Work logged] (HDFS-16061) DFTestUtil.waitReplication can produce false positives

2021-06-19 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-16061?focusedWorklogId=612353&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-612353
 ]

ASF GitHub Bot logged work on HDFS-16061:
-

Author: ASF GitHub Bot
Created on: 20/Jun/21 05:40
Start Date: 20/Jun/21 05:40
Worklog Time Spent: 10m 
  Work Description: ayushtkn commented on pull request #3095:
URL: https://github.com/apache/hadoop/pull/3095#issuecomment-864502846


   Thanx @amahussein for the contribution and @jbrennan333 for the review.
   
   Sorry, wanted to pull this to lower branches as well, got stuck with some 
internal stuff, couldn't find time to compile and push. 


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 612353)
Time Spent: 1h  (was: 50m)

> DFTestUtil.waitReplication can produce false positives
> --
>
> Key: HDFS-16061
> URL: https://issues.apache.org/jira/browse/HDFS-16061
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> While checking the intermittent failure in 
> TestBalancerRPCDelay#testBalancerRPCDelayQpsDefault described in HDFS-15146, 
> I found that the implementation of waitReplication is incorrect.
> In the last iteration, when {{correctReplFactor}} is {{false}}, the thread 
> sleeps for 1 second, then a {{TimeoutException}} is thrown without check 
> whether the replication was complete in the last second.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDFS-16061) DFTestUtil.waitReplication can produce false positives

2021-06-19 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-16061?focusedWorklogId=612349&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-612349
 ]

ASF GitHub Bot logged work on HDFS-16061:
-

Author: ASF GitHub Bot
Created on: 20/Jun/21 04:52
Start Date: 20/Jun/21 04:52
Worklog Time Spent: 10m 
  Work Description: ayushtkn merged pull request #3095:
URL: https://github.com/apache/hadoop/pull/3095


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 612349)
Time Spent: 50m  (was: 40m)

> DFTestUtil.waitReplication can produce false positives
> --
>
> Key: HDFS-16061
> URL: https://issues.apache.org/jira/browse/HDFS-16061
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> While checking the intermittent failure in 
> TestBalancerRPCDelay#testBalancerRPCDelayQpsDefault described in HDFS-15146, 
> I found that the implementation of waitReplication is incorrect.
> In the last iteration, when {{correctReplFactor}} is {{false}}, the thread 
> sleeps for 1 second, then a {{TimeoutException}} is thrown without check 
> whether the replication was complete in the last second.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDFS-16061) DFTestUtil.waitReplication can produce false positives

2021-06-16 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-16061?focusedWorklogId=611759&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-611759
 ]

ASF GitHub Bot logged work on HDFS-16061:
-

Author: ASF GitHub Bot
Created on: 16/Jun/21 07:09
Start Date: 16/Jun/21 07:09
Worklog Time Spent: 10m 
  Work Description: ayushtkn commented on a change in pull request #3095:
URL: https://github.com/apache/hadoop/pull/3095#discussion_r652409047



##
File path: 
hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/DFSTestUtil.java
##
@@ -206,6 +206,8 @@
   private static final String[] dirNames = {
 "zero", "one", "two", "three", "four", "five", "six", "seven", "eight", 
"nine"
   };
+
+  private static final int WAIT_REPLICATION_ATTEMPTS = 40;

Review comment:
   Why you have pulled this up? Means nobody apart from `waitReplication ` 
method is using it? We could have kept it there itself, the variable is 
private, no one can change it also




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 611759)
Time Spent: 40m  (was: 0.5h)

> DFTestUtil.waitReplication can produce false positives
> --
>
> Key: HDFS-16061
> URL: https://issues.apache.org/jira/browse/HDFS-16061
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
>  Labels: pull-request-available
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> While checking the intermittent failure in 
> TestBalancerRPCDelay#testBalancerRPCDelayQpsDefault described in HDFS-15146, 
> I found that the implementation of waitReplication is incorrect.
> In the last iteration, when {{correctReplFactor}} is {{false}}, the thread 
> sleeps for 1 second, then a {{TimeoutException}} is thrown without check 
> whether the replication was complete in the last second.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org



[jira] [Work logged] (HDFS-16061) DFTestUtil.waitReplication can produce false positives

2021-06-14 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-16061?focusedWorklogId=610353&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-610353
 ]

ASF GitHub Bot logged work on HDFS-16061:
-

Author: ASF GitHub Bot
Created on: 14/Jun/21 07:42
Start Date: 14/Jun/21 07:42
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on pull request #3095:
URL: https://github.com/apache/hadoop/pull/3095#issuecomment-859933114


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 36s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  0s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 2 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  34m  3s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m 44s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   1m 31s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   1m 19s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 52s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   1m 10s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 45s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   4m 15s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  22m 17s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 32s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 39s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |   1m 39s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 27s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |   1m 27s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   1m  8s |  |  
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 86 unchanged - 2 
fixed = 86 total (was 88)  |
   | +1 :green_heart: |  mvnsite  |   1m 38s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   1m  2s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 47s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   4m 30s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  22m 33s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  | 251m 16s | 
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3095/2/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
 |  hadoop-hdfs in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 48s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 356m  0s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.hdfs.TestDecommissionWithStripedBackoffMonitor 
|
   |   | hadoop.hdfs.TestDFSStripedInputStream |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3095/2/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3095 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux fa237f688cc1 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 
11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / 4040ad64a4537d47e3633d23daaaefe7e9d7192a |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoo

[jira] [Work logged] (HDFS-16061) DFTestUtil.waitReplication can produce false positives

2021-06-10 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-16061?focusedWorklogId=609915&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609915
 ]

ASF GitHub Bot logged work on HDFS-16061:
-

Author: ASF GitHub Bot
Created on: 10/Jun/21 20:24
Start Date: 10/Jun/21 20:24
Worklog Time Spent: 10m 
  Work Description: hadoop-yetus commented on pull request #3095:
URL: https://github.com/apache/hadoop/pull/3095#issuecomment-859025279


   :broken_heart: **-1 overall**
   
   
   
   
   
   
   | Vote | Subsystem | Runtime |  Logfile | Comment |
   |::|--:|:|::|:---:|
   | +0 :ok: |  reexec  |   0m 32s |  |  Docker mode activated.  |
    _ Prechecks _ |
   | +1 :green_heart: |  dupname  |   0m  0s |  |  No case conflicting files 
found.  |
   | +0 :ok: |  codespell  |   0m  1s |  |  codespell was not available.  |
   | +1 :green_heart: |  @author  |   0m  0s |  |  The patch does not contain 
any @author tags.  |
   | +1 :green_heart: |  test4tests  |   0m  0s |  |  The patch appears to 
include 2 new or modified test files.  |
    _ trunk Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |  30m 41s |  |  trunk passed  |
   | +1 :green_heart: |  compile  |   1m 23s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  compile  |   1m 16s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  checkstyle  |   1m  2s |  |  trunk passed  |
   | +1 :green_heart: |  mvnsite  |   1m 24s |  |  trunk passed  |
   | +1 :green_heart: |  javadoc  |   0m 55s |  |  trunk passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 31s |  |  trunk passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   3m  9s |  |  trunk passed  |
   | +1 :green_heart: |  shadedclient  |  16m 18s |  |  branch has no errors 
when building and testing our client artifacts.  |
    _ Patch Compile Tests _ |
   | +1 :green_heart: |  mvninstall  |   1m 11s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m 11s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javac  |   1m 11s |  |  the patch passed  |
   | +1 :green_heart: |  compile  |   1m  6s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  javac  |   1m  6s |  |  the patch passed  |
   | +1 :green_heart: |  blanks  |   0m  0s |  |  The patch has no blanks 
issues.  |
   | +1 :green_heart: |  checkstyle  |   0m 53s |  |  
hadoop-hdfs-project/hadoop-hdfs: The patch generated 0 new + 86 unchanged - 2 
fixed = 86 total (was 88)  |
   | +1 :green_heart: |  mvnsite  |   1m 11s |  |  the patch passed  |
   | +1 :green_heart: |  javadoc  |   0m 46s |  |  the patch passed with JDK 
Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04  |
   | +1 :green_heart: |  javadoc  |   1m 23s |  |  the patch passed with JDK 
Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10  |
   | +1 :green_heart: |  spotbugs  |   3m  8s |  |  the patch passed  |
   | +1 :green_heart: |  shadedclient  |  16m  7s |  |  patch has no errors 
when building and testing our client artifacts.  |
    _ Other Tests _ |
   | -1 :x: |  unit  | 252m 56s | 
[/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt](https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3095/1/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt)
 |  hadoop-hdfs in the patch passed.  |
   | +1 :green_heart: |  asflicense  |   0m 45s |  |  The patch does not 
generate ASF License warnings.  |
   |  |   | 336m 55s |  |  |
   
   
   | Reason | Tests |
   |---:|:--|
   | Failed junit tests | hadoop.hdfs.TestFileCorruption |
   |   | hadoop.hdfs.TestDecommissionWithBackoffMonitor |
   
   
   | Subsystem | Report/Notes |
   |--:|:-|
   | Docker | ClientAPI=1.41 ServerAPI=1.41 base: 
https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-3095/1/artifact/out/Dockerfile
 |
   | GITHUB PR | https://github.com/apache/hadoop/pull/3095 |
   | Optional Tests | dupname asflicense compile javac javadoc mvninstall 
mvnsite unit shadedclient spotbugs checkstyle codespell |
   | uname | Linux a756d5c27042 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 
05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux |
   | Build tool | maven |
   | Personality | dev-support/bin/hadoop.sh |
   | git revision | trunk / da16c59f102f81ff2be92b85f10f487fb7b2541c |
   | Default Java | Private Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   | Multi-JDK versions | 
/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.11+9-Ubuntu-0ubuntu2.20.04 
/usr/lib/jvm/java-8-openjdk-amd64:Private 
Build-1.8.0_292-8u292-b10-0ubuntu1~20.04-b10 |
   |  Test Results | 
https://ci-hadoop.apache.org/job/hadoop-multibranc

[jira] [Work logged] (HDFS-16061) DFTestUtil.waitReplication can produce false positives

2021-06-10 Thread ASF GitHub Bot (Jira)


 [ 
https://issues.apache.org/jira/browse/HDFS-16061?focusedWorklogId=609742&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-609742
 ]

ASF GitHub Bot logged work on HDFS-16061:
-

Author: ASF GitHub Bot
Created on: 10/Jun/21 14:46
Start Date: 10/Jun/21 14:46
Worklog Time Spent: 10m 
  Work Description: amahussein opened a new pull request #3095:
URL: https://github.com/apache/hadoop/pull/3095


   HDFS-16061 DFTestUtil.waitReplication can produce false positives
   
   While checking the intermittent failure in 
`TestBalancerRPCDelay#testBalancerRPCDelayQpsDefault` described in HDFS-15146, 
I found that the implementation of waitReplication is incorrect.
   In the last iteration, when correctReplFactor is false, the thread sleeps 
for 1 second, then a TimeoutException is thrown without check whether the 
replication was complete in the last second.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 609742)
Remaining Estimate: 0h
Time Spent: 10m

> DFTestUtil.waitReplication can produce false positives
> --
>
> Key: HDFS-16061
> URL: https://issues.apache.org/jira/browse/HDFS-16061
> Project: Hadoop HDFS
>  Issue Type: Bug
>  Components: hdfs
>Reporter: Ahmed Hussein
>Assignee: Ahmed Hussein
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> While checking the intermittent failure in 
> TestBalancerRPCDelay#testBalancerRPCDelayQpsDefault described in HDFS-15146, 
> I found that the implementation of waitReplication is incorrect.
> In the last iteration, when {{correctReplFactor}} is {{false}}, the thread 
> sleeps for 1 second, then a {{TimeoutException}} is thrown without check 
> whether the replication was complete in the last second.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org