[jira] [Updated] (HBASE-26074) testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on branch-1
[
https://issues.apache.org/jira/browse/HBASE-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharath Vissapragada updated HBASE-26074:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on
> branch-1
> --
>
> Key: HBASE-26074
> URL: https://issues.apache.org/jira/browse/HBASE-26074
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.7.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Labels: regression
> Fix For: 1.7.1
>
>
> In prep-ing for 1.7.1, I noticed that this test has been consistently failing
> on branch-1 forever.
> {noformat}
> Regression
> health checks / yetus jdk8 hadoop2 checks /
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego
> Failing for the past 1 build (Since Failed#145 )
> Took 0.79 sec.
> Error Message
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:743)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:702)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at
> org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:189)
> at
> org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:347)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.connect(LogLevel.java:268)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.process(LogLevel.java:284)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.doGetLevel(LogLevel.java:227)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.sendLogLevelRequest(LogLevel.java:123)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.run(LogLevel.java:107)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.getLevel(TestLogLevel.java:349)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.access$000(TestLogLevel.java:68)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:325)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:322)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:277)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego(TestLogLevel.java:451)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> Stacktrace
> java.lang.AssertionError:
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at
[GitHub] [hbase] bharathv merged pull request #3466: HBASE-26074: Fix testLogLevelByHttps/testLogLevelByHttpsWithSpnego
bharathv merged pull request #3466: URL: https://github.com/apache/hbase/pull/3466 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3467: [branch-1] HBASE-26075 Replication is stuck due to zero length wal file in oldWA…
Apache-HBase commented on pull request #3467: URL: https://github.com/apache/hbase/pull/3467#issuecomment-876921021 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 58s | branch-1 passed | | +1 :green_heart: | compile | 0m 39s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 46s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 1m 43s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 48s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 40s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 3m 9s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 6s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 52s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | compile | 0m 46s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 46s | the patch passed | | -1 :x: | checkstyle | 1m 31s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 50s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 27s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 42s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 2m 55s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 136m 26s | hbase-server in the patch passed. | | +1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | | | | 178m 20s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3467/2/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3467 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2cb649870283 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-3467/out/precommit/personality/provided.sh | | git revision | branch-1 / 5ece9cef | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3467/2/artifact/out/diff-checkstyle-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3467/2/testReport/ | | Max. process+thread count | 4123 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3467/2/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this
[GitHub] [hbase] shahrs87 commented on pull request #3467: [branch-1] HBASE-26075 Replication is stuck due to zero length wal file in oldWA…
shahrs87 commented on pull request #3467: URL: https://github.com/apache/hbase/pull/3467#issuecomment-876881625 @bharathv could you please review the patch ? Thank you ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 commented on pull request #3467: [branch-1] HBASE-26075 Replication is stuck due to zero length wal file in oldWA…
shahrs87 commented on pull request #3467: URL: https://github.com/apache/hbase/pull/3467#issuecomment-876880875 Failing test succeeded locally ``` [INFO] --- maven-surefire-plugin:2.22.2:test (default-test) @ hbase-server --- [INFO] [INFO] --- [INFO] T E S T S [INFO] --- [INFO] Running org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.35 s - in org.apache.hadoop.hbase.replication.TestReplicationKillSlaveRS [INFO] [INFO] Results: [INFO] [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (HBASE-26076) Support favoredNodes when do compaction offload
Yulin Niu created HBASE-26076: - Summary: Support favoredNodes when do compaction offload Key: HBASE-26076 URL: https://issues.apache.org/jira/browse/HBASE-26076 Project: HBase Issue Type: Sub-task Reporter: Yulin Niu Assignee: Yulin Niu 1.If we have favoredNodes for region, We use this policy 2.If we not have favoredNodes, we add the regionServer request compaction as favoredNodes when do compact on compaction server, to guarantee locality -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26074) testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on branch-1
[
https://issues.apache.org/jira/browse/HBASE-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377728#comment-17377728
]
Reid Chan commented on HBASE-26074:
---
Yeah..., it's a JDK issue.
> testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on
> branch-1
> --
>
> Key: HBASE-26074
> URL: https://issues.apache.org/jira/browse/HBASE-26074
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.7.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Labels: regression
> Fix For: 1.7.1
>
>
> In prep-ing for 1.7.1, I noticed that this test has been consistently failing
> on branch-1 forever.
> {noformat}
> Regression
> health checks / yetus jdk8 hadoop2 checks /
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego
> Failing for the past 1 build (Since Failed#145 )
> Took 0.79 sec.
> Error Message
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:743)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:702)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at
> org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:189)
> at
> org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:347)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.connect(LogLevel.java:268)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.process(LogLevel.java:284)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.doGetLevel(LogLevel.java:227)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.sendLogLevelRequest(LogLevel.java:123)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.run(LogLevel.java:107)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.getLevel(TestLogLevel.java:349)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.access$000(TestLogLevel.java:68)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:325)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:322)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:277)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego(TestLogLevel.java:451)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> Stacktrace
> java.lang.AssertionError:
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at
[GitHub] [hbase] Apache-HBase commented on pull request #3467: [branch-1] HBASE-26075 Replication is stuck due to zero length wal file in oldWA…
Apache-HBase commented on pull request #3467: URL: https://github.com/apache/hbase/pull/3467#issuecomment-876863316 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 0m 37s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 56s | branch-1 passed | | +1 :green_heart: | compile | 0m 40s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 46s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 1m 41s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 49s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 41s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 3m 6s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 3s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 53s | the patch passed | | +1 :green_heart: | compile | 0m 41s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 41s | the patch passed | | +1 :green_heart: | compile | 0m 45s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 45s | the patch passed | | -1 :x: | checkstyle | 1m 30s | hbase-server: The patch generated 1 new + 0 unchanged - 0 fixed = 1 total (was 0) | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 49s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 28s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 32s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 41s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 3m 0s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 136m 41s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | | | | 178m 38s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.replication.TestReplicationKillSlaveRS | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3467/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3467 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2a4eee3cee31 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-3467/out/precommit/personality/provided.sh | | git revision | branch-1 / 5ece9cef | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | checkstyle | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3467/1/artifact/out/diff-checkstyle-hbase-server.txt | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3467/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3467/1/testReport/ | | Max. process+thread count | 4489 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3467/1/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated.
[GitHub] [hbase] mokai87 commented on pull request #3458: HBASE-25700 Enhance znode parent validation when add_peer
mokai87 commented on pull request #3458: URL: https://github.com/apache/hbase/pull/3458#issuecomment-876823277 > Please fix the checkstyle issues then I will merge it. > > Thanks. Fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] shahrs87 opened a new pull request #3467: HBASE-26075 Replication is stuck due to zero length wal file in oldWA…
shahrs87 opened a new pull request #3467: URL: https://github.com/apache/hbase/pull/3467 …Ls directory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3466: HBASE-26074: Fix testLogLevelByHttps/testLogLevelByHttpsWithSpnego
Apache-HBase commented on pull request #3466: URL: https://github.com/apache/hbase/pull/3466#issuecomment-876758232 :broken_heart: **-1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 6m 42s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 1s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | | +1 :green_heart: | test4tests | 0m 0s | The patch appears to include 1 new or modified test files. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 10m 5s | branch-1 passed | | +1 :green_heart: | compile | 0m 41s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | compile | 0m 45s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | checkstyle | 1m 44s | branch-1 passed | | +1 :green_heart: | shadedjars | 3m 2s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 52s | branch-1 passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 41s | branch-1 passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +0 :ok: | spotbugs | 3m 5s | Used deprecated FindBugs config; considering switching to SpotBugs. | | +1 :green_heart: | findbugs | 3m 2s | branch-1 passed | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 51s | the patch passed | | +1 :green_heart: | compile | 0m 39s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javac | 0m 39s | the patch passed | | +1 :green_heart: | compile | 0m 44s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | javac | 0m 44s | the patch passed | | +1 :green_heart: | checkstyle | 1m 31s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | shadedjars | 2m 50s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | hadoopcheck | 4m 36s | Patch does not cause any errors with Hadoop 2.8.5 2.9.2. | | +1 :green_heart: | javadoc | 0m 30s | the patch passed with JDK Azul Systems, Inc.-1.8.0_262-b19 | | +1 :green_heart: | javadoc | 0m 42s | the patch passed with JDK Azul Systems, Inc.-1.7.0_272-b10 | | +1 :green_heart: | findbugs | 2m 52s | the patch passed | ||| _ Other Tests _ | | -1 :x: | unit | 135m 42s | hbase-server in the patch failed. | | +1 :green_heart: | asflicense | 0m 39s | The patch does not generate ASF License warnings. | | | | 183m 47s | | | Reason | Tests | |---:|:--| | Failed junit tests | hadoop.hbase.client.TestAdmin1 | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3466/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3466 | | JIRA Issue | HBASE-26074 | | Optional Tests | dupname asflicense javac javadoc unit spotbugs findbugs shadedjars hadoopcheck hbaseanti checkstyle compile | | uname | Linux 39509f3bcfd7 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-agent/workspace/Base-PreCommit-GitHub-PR_PR-3466/out/precommit/personality/provided.sh | | git revision | branch-1 / 5ece9cef | | Default Java | Azul Systems, Inc.-1.7.0_272-b10 | | Multi-JDK versions | /usr/lib/jvm/zulu-8-amd64:Azul Systems, Inc.-1.8.0_262-b19 /usr/lib/jvm/zulu-7-amd64:Azul Systems, Inc.-1.7.0_272-b10 | | unit | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3466/1/artifact/out/patch-unit-hbase-server.txt | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3466/1/testReport/ | | Max. process+thread count | 4181 (vs. ulimit of 1) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3466/1/console | | versions | git=1.9.1 maven=3.0.5 findbugs=3.0.1 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:
[GitHub] [hbase] saintstack commented on a change in pull request #3407: HBASE-26018 Perf improvement in L1 cache - Optimistic call to buffer.retain()
saintstack commented on a change in pull request #3407:
URL: https://github.com/apache/hbase/pull/3407#discussion_r666514850
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruAdaptiveBlockCache.java
##
@@ -646,14 +639,16 @@ private long updateSizeMetrics(LruCachedBlock cb, boolean
evict) {
@Override
public Cacheable getBlock(BlockCacheKey cacheKey, boolean caching, boolean
repeat,
boolean updateCacheMetrics) {
-LruCachedBlock cb = map.computeIfPresent(cacheKey, (key, val) -> {
- // It will be referenced by RPC path, so increase here. NOTICE: Must do
the retain inside
- // this block. because if retain outside the map#computeIfPresent, the
evictBlock may remove
- // the block and release, then we're retaining a block with refCnt=0
which is disallowed.
- // see HBASE-22422.
- val.getBuffer().retain();
- return val;
-});
+LruCachedBlock cb = map.get(cacheKey);
+if (cb != null) {
+ try {
+cb.getBuffer().retain();
Review comment:
#1 sounds good.
#2 yeah, it can get interesting. The computeIfPresent made reasoning easier
for sure.
Running w/ #get instead of #computeIfPresent -- even though it incorrect --
changed the locking profile of a loaded process; before the change, the
blockage in computeIfPresent was the biggest blockage. After, biggest locking
consumer was elsewhere and much more insignificant percentage
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3407: HBASE-26018 Perf improvement in L1 cache - Optimistic call to buffer.retain()
virajjasani commented on a change in pull request #3407:
URL: https://github.com/apache/hbase/pull/3407#discussion_r666482218
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruAdaptiveBlockCache.java
##
@@ -646,14 +639,16 @@ private long updateSizeMetrics(LruCachedBlock cb, boolean
evict) {
@Override
public Cacheable getBlock(BlockCacheKey cacheKey, boolean caching, boolean
repeat,
boolean updateCacheMetrics) {
-LruCachedBlock cb = map.computeIfPresent(cacheKey, (key, val) -> {
- // It will be referenced by RPC path, so increase here. NOTICE: Must do
the retain inside
- // this block. because if retain outside the map#computeIfPresent, the
evictBlock may remove
- // the block and release, then we're retaining a block with refCnt=0
which is disallowed.
- // see HBASE-22422.
- val.getBuffer().retain();
- return val;
-});
+LruCachedBlock cb = map.get(cacheKey);
+if (cb != null) {
+ try {
+cb.getBuffer().retain();
Review comment:
I think for possibility#2 in above, we stand a chance where buffer with
non-zero refCount is not in the cache. I see, let me see what alternatives we
have for this case.
Although I still think that same case can happen even today.
getBlock does retain() which will bring refCount of BB to 2, while getBlock
is busy updating stats, eviction thread can evict block from cache and it does
release() which will bring refCount of BB to 1. So even in this case, we can
positive refCount buffer which is evicted from cache.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3407: HBASE-26018 Perf improvement in L1 cache - Optimistic call to buffer.retain()
virajjasani commented on a change in pull request #3407:
URL: https://github.com/apache/hbase/pull/3407#discussion_r666482218
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruAdaptiveBlockCache.java
##
@@ -646,14 +639,16 @@ private long updateSizeMetrics(LruCachedBlock cb, boolean
evict) {
@Override
public Cacheable getBlock(BlockCacheKey cacheKey, boolean caching, boolean
repeat,
boolean updateCacheMetrics) {
-LruCachedBlock cb = map.computeIfPresent(cacheKey, (key, val) -> {
- // It will be referenced by RPC path, so increase here. NOTICE: Must do
the retain inside
- // this block. because if retain outside the map#computeIfPresent, the
evictBlock may remove
- // the block and release, then we're retaining a block with refCnt=0
which is disallowed.
- // see HBASE-22422.
- val.getBuffer().retain();
- return val;
-});
+LruCachedBlock cb = map.get(cacheKey);
+if (cb != null) {
+ try {
+cb.getBuffer().retain();
Review comment:
I think for possibility#2 in above, we stand a chance where buffer with
non-zero refCount is not in the cache. I see, let me see what alternatives we
have for this case.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] virajjasani commented on a change in pull request #3407: HBASE-26018 Perf improvement in L1 cache - Optimistic call to buffer.retain()
virajjasani commented on a change in pull request #3407:
URL: https://github.com/apache/hbase/pull/3407#discussion_r666473427
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruAdaptiveBlockCache.java
##
@@ -646,14 +639,16 @@ private long updateSizeMetrics(LruCachedBlock cb, boolean
evict) {
@Override
public Cacheable getBlock(BlockCacheKey cacheKey, boolean caching, boolean
repeat,
boolean updateCacheMetrics) {
-LruCachedBlock cb = map.computeIfPresent(cacheKey, (key, val) -> {
- // It will be referenced by RPC path, so increase here. NOTICE: Must do
the retain inside
- // this block. because if retain outside the map#computeIfPresent, the
evictBlock may remove
- // the block and release, then we're retaining a block with refCnt=0
which is disallowed.
- // see HBASE-22422.
- val.getBuffer().retain();
- return val;
-});
+LruCachedBlock cb = map.get(cacheKey);
+if (cb != null) {
+ try {
+cb.getBuffer().retain();
Review comment:
It is purged from cache by:
```
protected long evictBlock(LruCachedBlock block, boolean
evictedByEvictionProcess) {
LruCachedBlock previous = map.remove(block.getCacheKey());===>
removed from map
if (previous == null) {
return 0;
}
updateSizeMetrics(block, true);
long val = elements.decrementAndGet();
if (LOG.isTraceEnabled()) {
long size = map.size();
assertCounterSanity(size, val);
}
if (block.getBuffer().getBlockType().isData()) {
dataBlockElements.decrement();
}
if (evictedByEvictionProcess) {
// When the eviction of the block happened because of invalidation of
HFiles, no need to
// update the stats counter.
stats.evicted(block.getCachedTime(), block.getCacheKey().isPrimary());
if (victimHandler != null) {
victimHandler.cacheBlock(block.getCacheKey(), block.getBuffer());
}
}
// Decrease the block's reference count, and if refCount is 0, then
it'll auto-deallocate. DO
// NOT move this up because if do that then the victimHandler may access
the buffer with
// refCnt = 0 which is disallowed.
previous.getBuffer().release(); >
buffer released
return block.heapSize();
}
```
Based on above mentioned eviction code, we have below mentioned
possibilities when eviction and getBlock happens for the same block at the same
time:
1. getBlock retrieves block from map, eviction removes it from map, eviction
does release(), getBlock does retain() and encounters IllegalRefCount
Exception, we handler it with this patch and treat it as cache miss.
2. getBlock retrieves block from map, eviction removes it from map, getBlock
does retain(), eviction does release(). Since getBlock retain() was successful,
it proceeds as successful cache hit, which happens even today with
computeIfPresent. Subsequent getBlock call will return null as block was
evicted previously.
3. eviction removes from map, getBlock gets null, it's clear cache miss.
I think we seem good here. WDYT @saintstack?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3458: HBASE-25700 Enhance znode parent validation when add_peer
Apache-HBase commented on pull request #3458: URL: https://github.com/apache/hbase/pull/3458#issuecomment-876677596 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 34s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 15s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 30s | master passed | | +1 :green_heart: | compile | 1m 40s | master passed | | +1 :green_heart: | shadedjars | 9m 14s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 4s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 14s | the patch passed | | +1 :green_heart: | compile | 1m 35s | the patch passed | | +1 :green_heart: | javac | 1m 35s | the patch passed | | +1 :green_heart: | shadedjars | 9m 3s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 2s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 31s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 208m 30s | hbase-server in the patch passed. | | | | 246m 30s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3458/7/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3458 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 25b532c10aef 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e65fc9226f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3458/7/testReport/ | | Max. process+thread count | 3378 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3458/7/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3458: HBASE-25700 Enhance znode parent validation when add_peer
Apache-HBase commented on pull request #3458: URL: https://github.com/apache/hbase/pull/3458#issuecomment-876673804 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 10s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 17s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 5m 2s | master passed | | +1 :green_heart: | compile | 1m 49s | master passed | | +1 :green_heart: | shadedjars | 9m 5s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 8s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 13s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 44s | the patch passed | | +1 :green_heart: | compile | 1m 48s | the patch passed | | +1 :green_heart: | javac | 1m 48s | the patch passed | | +1 :green_heart: | shadedjars | 9m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 1m 7s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 1m 38s | hbase-client in the patch passed. | | +1 :green_heart: | unit | 201m 11s | hbase-server in the patch passed. | | | | 240m 23s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3458/7/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3458 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 7b532f510f4e 4.15.0-142-generic #146-Ubuntu SMP Tue Apr 13 01:11:19 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e65fc9226f | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3458/7/testReport/ | | Max. process+thread count | 3454 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3458/7/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] saintstack commented on a change in pull request #3407: HBASE-26018 Perf improvement in L1 cache - Optimistic call to buffer.retain()
saintstack commented on a change in pull request #3407:
URL: https://github.com/apache/hbase/pull/3407#discussion_r666432156
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/io/hfile/LruAdaptiveBlockCache.java
##
@@ -646,14 +639,16 @@ private long updateSizeMetrics(LruCachedBlock cb, boolean
evict) {
@Override
public Cacheable getBlock(BlockCacheKey cacheKey, boolean caching, boolean
repeat,
boolean updateCacheMetrics) {
-LruCachedBlock cb = map.computeIfPresent(cacheKey, (key, val) -> {
- // It will be referenced by RPC path, so increase here. NOTICE: Must do
the retain inside
- // this block. because if retain outside the map#computeIfPresent, the
evictBlock may remove
- // the block and release, then we're retaining a block with refCnt=0
which is disallowed.
- // see HBASE-22422.
- val.getBuffer().retain();
- return val;
-});
+LruCachedBlock cb = map.get(cacheKey);
+if (cb != null) {
+ try {
+cb.getBuffer().retain();
Review comment:
And if it is purge from cache by a background thread, we'll have a cb w/
a non-zero refcount that is not in the cache? Will that work?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (HBASE-26074) testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on branch-1
[
https://issues.apache.org/jira/browse/HBASE-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharath Vissapragada updated HBASE-26074:
-
Status: Patch Available (was: Open)
> testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on
> branch-1
> --
>
> Key: HBASE-26074
> URL: https://issues.apache.org/jira/browse/HBASE-26074
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.7.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Labels: regression
> Fix For: 1.7.1
>
>
> In prep-ing for 1.7.1, I noticed that this test has been consistently failing
> on branch-1 forever.
> {noformat}
> Regression
> health checks / yetus jdk8 hadoop2 checks /
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego
> Failing for the past 1 build (Since Failed#145 )
> Took 0.79 sec.
> Error Message
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:743)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:702)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at
> org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:189)
> at
> org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:347)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.connect(LogLevel.java:268)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.process(LogLevel.java:284)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.doGetLevel(LogLevel.java:227)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.sendLogLevelRequest(LogLevel.java:123)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.run(LogLevel.java:107)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.getLevel(TestLogLevel.java:349)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.access$000(TestLogLevel.java:68)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:325)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:322)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:277)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego(TestLogLevel.java:451)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> Stacktrace
> java.lang.AssertionError:
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at
[GitHub] [hbase] bharathv opened a new pull request #3466: HBASE-26074: Fix testLogLevelByHttps/testLogLevelByHttpsWithSpnego
bharathv opened a new pull request #3466: URL: https://github.com/apache/hbase/pull/3466 See the jira for context. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26069) Remove HStore.compactRecentForTestingAssumingDefaultPolicy and DefaultCompactor.compactForTesting
[
https://issues.apache.org/jira/browse/HBASE-26069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377550#comment-17377550
]
Hudson commented on HBASE-26069:
Results for branch branch-2
[build #296 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/296/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/296/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/296/JDK8_20Nightly_20Build_20Report_20_28Hadoop2_29/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/296/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/296/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(x) {color:red}-1 client integration test{color}
-- Something went wrong with this stage, [check relevant console
output|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-2/296//console].
> Remove HStore.compactRecentForTestingAssumingDefaultPolicy and
> DefaultCompactor.compactForTesting
> -
>
> Key: HBASE-26069
> URL: https://issues.apache.org/jira/browse/HBASE-26069
> Project: HBase
> Issue Type: Improvement
> Components: Compaction, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2
>
>
> We should try our best to not include testing code in normal code.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] saintstack commented on a change in pull request #3436: HBASE-26036 DBB released too early in HRegion.get() and dirty data for some operations
saintstack commented on a change in pull request #3436:
URL: https://github.com/apache/hbase/pull/3436#discussion_r666398359
##
File path:
hbase-common/src/main/java/org/apache/hadoop/hbase/io/DeallocateRewriteByteBuffAllocator.java
##
@@ -0,0 +1,59 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.hadoop.hbase.io;
+
+import java.nio.ByteBuffer;
+import org.apache.hadoop.hbase.util.Bytes;
+import org.apache.yetus.audience.InterfaceAudience;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * A ByteBuffAllocator that rewrite the bytebuffers right after released.
+ * It can be used for test whether there are prematurely releasing backing
bytebuffers.
+ */
+@InterfaceAudience.Private
+public class DeallocateRewriteByteBuffAllocator extends ByteBuffAllocator {
+ private static final Logger LOG = LoggerFactory.getLogger(
+DeallocateRewriteByteBuffAllocator.class);
+
+ DeallocateRewriteByteBuffAllocator(boolean reservoirEnabled, int
maxBufCount, int bufSize,
+int minSizeForReservoirUse) {
+super(reservoirEnabled, maxBufCount, bufSize, minSizeForReservoirUse);
+ }
+
+ @Override
+ protected void putbackBuffer(ByteBuffer buf) {
+if (buf.capacity() != bufSize || (reservoirEnabled ^ buf.isDirect())) {
+ LOG.warn("Trying to put a buffer, not created by this pool! Will be just
ignored");
+ return;
+}
+buf.clear();
+byte[] tmp = generateTmpBytes(buf.capacity());
+buf.put(tmp, 0, tmp.length);
+super.putbackBuffer(buf);
+ }
+
+ private byte[] generateTmpBytes(int length) {
+StringBuilder result = new StringBuilder();
+while (result.length() < length) {
+ result.append("-");
+}
+return Bytes.toBytes(result.substring(0, length));
+ }
+}
Review comment:
Great
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/coprocessor/MultiRowMutationEndpoint.java
##
@@ -221,22 +223,27 @@ private boolean matches(Region region,
ClientProtos.Condition condition) throws
get.setTimeRange(timeRange.getMin(), timeRange.getMax());
}
-List result = region.get(get, false);
boolean matches = false;
-if (filter != null) {
- if (!result.isEmpty()) {
-matches = true;
- }
-} else {
- boolean valueIsNull = comparator.getValue() == null ||
comparator.getValue().length == 0;
- if (result.isEmpty() && valueIsNull) {
-matches = true;
- } else if (result.size() > 0 && result.get(0).getValueLength() == 0 &&
valueIsNull) {
-matches = true;
- } else if (result.size() == 1 && !valueIsNull) {
-Cell kv = result.get(0);
-int compareResult = PrivateCellUtil.compareValue(kv, comparator);
-matches = matches(op, compareResult);
+try (RegionScanner scanner = region.getScanner(new Scan(get))) {
+ // NOTE: Please don't use HRegion.get() instead,
+ // because it will copy cells to heap. See HBASE-26036
Review comment:
Great
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -7541,6 +7554,9 @@ void prepareGet(final Get get) throws IOException {
() -> createRegionSpan("Region.get"));
}
+ /**
+ * This method will return onheap cells, for more details, please see
HBASE-26036.
Review comment:
Nit: Only do if you make a new PR. I think you would call out that this
can be an EXPENSIVE call. It may mean an extra copy from offheap to onheap
buffers.
Otherwise, this looks great.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (HBASE-26075) Replication is stuck due to zero length wal file in oldWALs directory.
[
https://issues.apache.org/jira/browse/HBASE-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377535#comment-17377535
]
Rushabh Shah commented on HBASE-26075:
--
I am working now to create patch. Will create PR for branch-1 first and then
master branches. Will update here ASAP.
> Replication is stuck due to zero length wal file in oldWALs directory.
> --
>
> Key: HBASE-26075
> URL: https://issues.apache.org/jira/browse/HBASE-26075
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.3.5, 2.4.4
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Critical
>
> Recently we encountered a case where size of log queue was increasing to
> around 300 in few region servers in our production environment.
> There were 295 wals in the oldWALs directory for that region server and the
> *first file* was a 0 length file.
> Replication was throwing the following error.
> {noformat}
> 2021-07-05 03:06:32,757 ERROR [20%2C1625185107182,1]
> regionserver.ReplicationSourceWALReaderThread - Failed to read stream of
> replication entries
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream$WALEntryStreamRuntimeException:
> java.io.EOFException: hdfs:///hbase/oldWALs/ not a
> SequenceFile
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:112)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReaderThread.run(ReplicationSourceWALReaderThread.java:156)
> Caused by: java.io.EOFException:
> hdfs:///hbase/oldWALs/ not a SequenceFile
> at
> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1934)
> at
> org.apache.hadoop.io.SequenceFile$Reader.initialize(SequenceFile.java:1893)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1842)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1856)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:70)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.reset(SequenceFileLogReader.java:168)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.initReader(SequenceFileLogReader.java:177)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:66)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:313)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:277)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:265)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:424)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:352)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.handleFileNotFound(WALEntryStream.java:341)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:359)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openNextLog(WALEntryStream.java:316)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.checkReader(WALEntryStream.java:306)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:207)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:110)
> ... 1 more
> {noformat}
> We fixed similar error via HBASE-25536 but the zero length file was in
> recovered sources.
> There were more logs after the above stack trace.
> {noformat}
> 2021-07-05 03:06:32,757 WARN [20%2C1625185107182,1]
> regionserver.ReplicationSourceWALReaderThread - Couldn't get file length
> information about log
> hdfs:///hbase/WALs/
> 2021-07-05 03:06:32,754 INFO [20%2C1625185107182,1]
> regionserver.WALEntryStream - Log hdfs:///hbase/WALs/
> was moved to hdfs:///hbase/oldWALs/
> {noformat}
> There is a special logic in ReplicationSourceWALReader thread to handle 0
> length files but we only look in WALs directory and not in oldWALs directory.
> {code}
> private boolean handleEofException(Exception e, WALEntryBatch batch) {
> PriorityBlockingQueue queue = logQueue.getQueue(walGroupId);
> // Dump the log even if logQueue size is 1 if the source is from
> recovered Source
> // since we don't add current log to recovered source queue so it is safe
> to remove.
> if ((e instanceof EOFException || e.getCause() instanceof EOFException) &&
> (source.isRecovered() || queue.size() > 1) &&
[jira] [Commented] (HBASE-26075) Replication is stuck due to zero length wal file in oldWALs directory.
[
https://issues.apache.org/jira/browse/HBASE-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377533#comment-17377533
]
Bharath Vissapragada commented on HBASE-26075:
--
Hoping to get an RC out before end of this week (unless there are any
unforeseen delays in the process), any chance this can be committed before
then? I can help review the patch.
> Replication is stuck due to zero length wal file in oldWALs directory.
> --
>
> Key: HBASE-26075
> URL: https://issues.apache.org/jira/browse/HBASE-26075
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.3.5, 2.4.4
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Critical
>
> Recently we encountered a case where size of log queue was increasing to
> around 300 in few region servers in our production environment.
> There were 295 wals in the oldWALs directory for that region server and the
> *first file* was a 0 length file.
> Replication was throwing the following error.
> {noformat}
> 2021-07-05 03:06:32,757 ERROR [20%2C1625185107182,1]
> regionserver.ReplicationSourceWALReaderThread - Failed to read stream of
> replication entries
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream$WALEntryStreamRuntimeException:
> java.io.EOFException: hdfs:///hbase/oldWALs/ not a
> SequenceFile
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:112)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReaderThread.run(ReplicationSourceWALReaderThread.java:156)
> Caused by: java.io.EOFException:
> hdfs:///hbase/oldWALs/ not a SequenceFile
> at
> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1934)
> at
> org.apache.hadoop.io.SequenceFile$Reader.initialize(SequenceFile.java:1893)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1842)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1856)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:70)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.reset(SequenceFileLogReader.java:168)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.initReader(SequenceFileLogReader.java:177)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:66)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:313)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:277)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:265)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:424)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:352)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.handleFileNotFound(WALEntryStream.java:341)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:359)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openNextLog(WALEntryStream.java:316)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.checkReader(WALEntryStream.java:306)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:207)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:110)
> ... 1 more
> {noformat}
> We fixed similar error via HBASE-25536 but the zero length file was in
> recovered sources.
> There were more logs after the above stack trace.
> {noformat}
> 2021-07-05 03:06:32,757 WARN [20%2C1625185107182,1]
> regionserver.ReplicationSourceWALReaderThread - Couldn't get file length
> information about log
> hdfs:///hbase/WALs/
> 2021-07-05 03:06:32,754 INFO [20%2C1625185107182,1]
> regionserver.WALEntryStream - Log hdfs:///hbase/WALs/
> was moved to hdfs:///hbase/oldWALs/
> {noformat}
> There is a special logic in ReplicationSourceWALReader thread to handle 0
> length files but we only look in WALs directory and not in oldWALs directory.
> {code}
> private boolean handleEofException(Exception e, WALEntryBatch batch) {
> PriorityBlockingQueue queue = logQueue.getQueue(walGroupId);
> // Dump the log even if logQueue size is 1 if the source is from
> recovered Source
> // since we don't add current log to recovered source queue so it is safe
> to remove.
> if ((e instanceof EOFException || e.getCause()
[jira] [Commented] (HBASE-25720) Sync WAL stuck when prepare flush cache will prevent flush cache and cause OOM
[ https://issues.apache.org/jira/browse/HBASE-25720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377532#comment-17377532 ] Michael Stack commented on HBASE-25720: --- Anything in the log before your png? That shows perhaps how or why the WAL system is stuck? A jstack? Thanks [~Xiaolin Ha] > Sync WAL stuck when prepare flush cache will prevent flush cache and cause OOM > -- > > Key: HBASE-25720 > URL: https://issues.apache.org/jira/browse/HBASE-25720 > Project: HBase > Issue Type: Improvement >Affects Versions: 1.4.13 >Reporter: Xiaolin Ha >Assignee: Xiaolin Ha >Priority: Major > Attachments: prepare-flush-cache-stuck.png > > > We call HRegion#doSyncOfUnflushedWALChanges when preparing to flush cache. > But this WAL sync may stuck, and abort the flush of cache. > !prepare-flush-cache-stuck.png|width=519,height=246! > If we cannot aware of this problem in time, RS will OOM kill. > I think we should force abort RS when sync stuck in preparing, like in > committing snapshots. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (HBASE-26074) testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on branch-1
[
https://issues.apache.org/jira/browse/HBASE-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377523#comment-17377523
]
Bharath Vissapragada commented on HBASE-26074:
--
I figured out the issue, it appears the difference is in the SSL libraries
shipped by open jdk (or oracle jdks, commonly used by the developers) and the
azul JDKs used in the yetus test runs. If we carefully notice the bottom of the
stack on the server side doing the SSL hand shakes, we can see the difference.
Open JDK's SSL libs has special handling that recognizes it is an HTTP
connection.
{noformat}
2021-07-08 10:25:33,541 WARN [984520532@qtp-1977083615-0] log.Slf4jLog(89):
EXCEPTION
javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at
sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:710)
at sun.security.ssl.InputRecord.read(InputRecord.java:527)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
at
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at
org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:708)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2021-07-08 10:25:33,542 WARN [1227853087@qtp-1977083615-2] log.Slf4jLog(89):
EXCEPTION
javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection?
at
sun.security.ssl.InputRecord.handleUnknownRecord(InputRecord.java:710)
at sun.security.ssl.InputRecord.read(InputRecord.java:527)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:975)
at
sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1367)
at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1395)
at
sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:1379)
at
org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:708)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2021-07-08 10:25:33,543 INFO [Time-limited test] log.Slf4jLog(67): Stopped
SslSocketConnectorSecure@localhost:0
{noformat}
Whereas the azul JDK uses a different version of ssl lib that treats it as a
generic SSL failure and resets the connection.
{noformat}
javax.net.ssl.SSLException: Unsupported or unrecognized SSL message
at
sun.security.ssl.SSLSocketInputRecord.handleUnknownRecord(SSLSocketInputRecord.java:440)
at
sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:175)
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:110)
at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1197)
at
sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1106)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:398)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:370)
at
org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:708)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2021-07-08 13:56:33,322 WARN [189676137@qtp-1691391859-0] log.Slf4jLog(89):
EXCEPTION
javax.net.ssl.SSLException: Unsupported or unrecognized SSL message
at
sun.security.ssl.SSLSocketInputRecord.handleUnknownRecord(SSLSocketInputRecord.java:440)
at
sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:175)
at sun.security.ssl.SSLTransport.decode(SSLTransport.java:110)
at sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1197)
at
sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1106)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:398)
at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:370)
at
org.mortbay.jetty.security.SslSocketConnector$SslConnection.run(SslSocketConnector.java:708)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2021-07-08 13:56:33,323 INFO [Time-limited test] log.Slf4jLog(67): Stopped
SslSocketConnectorSecure@localhost:0
{noformat}
I downloaded the azul JDK from
https://www.azul.com/downloads/?version=java-8-lts=macos=x86-64-bit=jdk=true
(tests seem to use `zulu8.48.0.51-ca-jdk8.0.26`) and then I can reproduce the
same error on my mac book. Let me create a patch to generically assert the
error.
[~reidchan] FYI (long pending branch-1 unit test issue).
> testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on
> branch-1
>
[jira] [Updated] (HBASE-26075) Replication is stuck due to zero length wal file in oldWALs directory.
[
https://issues.apache.org/jira/browse/HBASE-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rushabh Shah updated HBASE-26075:
-
Description:
Recently we encountered a case where size of log queue was increasing to around
300 in few region servers in our production environment.
There were 295 wals in the oldWALs directory for that region server and the
*first file* was a 0 length file.
Replication was throwing the following error.
{noformat}
2021-07-05 03:06:32,757 ERROR [20%2C1625185107182,1]
regionserver.ReplicationSourceWALReaderThread - Failed to read stream of
replication entries
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream$WALEntryStreamRuntimeException:
java.io.EOFException: hdfs:///hbase/oldWALs/ not a
SequenceFile
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:112)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReaderThread.run(ReplicationSourceWALReaderThread.java:156)
Caused by: java.io.EOFException: hdfs:///hbase/oldWALs/
not a SequenceFile
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1934)
at
org.apache.hadoop.io.SequenceFile$Reader.initialize(SequenceFile.java:1893)
at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1842)
at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1856)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:70)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.reset(SequenceFileLogReader.java:168)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.initReader(SequenceFileLogReader.java:177)
at
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:66)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:313)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:277)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:265)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:424)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:352)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.handleFileNotFound(WALEntryStream.java:341)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:359)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openNextLog(WALEntryStream.java:316)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.checkReader(WALEntryStream.java:306)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:207)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:110)
... 1 more
{noformat}
We fixed similar error via HBASE-25536 but the zero length file was in
recovered sources.
There were more logs after the above stack trace.
{noformat}
2021-07-05 03:06:32,757 WARN [20%2C1625185107182,1]
regionserver.ReplicationSourceWALReaderThread - Couldn't get file length
information about log
hdfs:///hbase/WALs/
2021-07-05 03:06:32,754 INFO [20%2C1625185107182,1]
regionserver.WALEntryStream - Log hdfs:///hbase/WALs/
was moved to hdfs:///hbase/oldWALs/
{noformat}
There is a special logic in ReplicationSourceWALReader thread to handle 0
length files but we only look in WALs directory and not in oldWALs directory.
{code}
private boolean handleEofException(Exception e, WALEntryBatch batch) {
PriorityBlockingQueue queue = logQueue.getQueue(walGroupId);
// Dump the log even if logQueue size is 1 if the source is from recovered
Source
// since we don't add current log to recovered source queue so it is safe
to remove.
if ((e instanceof EOFException || e.getCause() instanceof EOFException) &&
(source.isRecovered() || queue.size() > 1) && this.eofAutoRecovery) {
Path head = queue.peek();
try {
if (fs.getFileStatus(head).getLen() == 0) {
// head of the queue is an empty log file
LOG.warn("Forcing removal of 0 length log in queue: {}", head);
logQueue.remove(walGroupId);
currentPosition = 0;
if (batch != null) {
// After we removed the WAL from the queue, we should try shipping
the existing batch of
// entries
addBatchToShippingQueue(batch);
}
return true;
}
} catch (IOException ioe) {
LOG.warn("Couldn't get file length information about log " +
queue.peek(), ioe);
} catch (InterruptedException ie) {
LOG.trace("Interrupted while adding WAL batch to ship queue");
[jira] [Commented] (HBASE-26075) Replication is stuck due to zero length wal file in oldWALs directory.
[
https://issues.apache.org/jira/browse/HBASE-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377509#comment-17377509
]
Rushabh Shah commented on HBASE-26075:
--
[~bharathv] I know you are almost finished with 1.7.1 release process. I think
this is a critical issue which will block replication for days. Do you think we
can add this to 1.7.1 release ?
> Replication is stuck due to zero length wal file in oldWALs directory.
> --
>
> Key: HBASE-26075
> URL: https://issues.apache.org/jira/browse/HBASE-26075
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.3.5, 2.4.4
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Critical
>
> Recently we encountered a case where size of log queue was increasing to
> around 300 in few region servers in our production environment.
> There were 295 wals in the oldWALs directory for that region server and the
> *first file* was a 0 length file.
> Replication was throwing the following error.
> {noformat}
> 2021-07-05 03:06:32,757 ERROR [20%2C1625185107182,1]
> regionserver.ReplicationSourceWALReaderThread - Failed to read stream of
> replication entries
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream$WALEntryStreamRuntimeException:
> java.io.EOFException: hdfs:///hbase/oldWALs/ not a
> SequenceFile
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:112)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReaderThread.run(ReplicationSourceWALReaderThread.java:156)
> Caused by: java.io.EOFException:
> hdfs:///hbase/oldWALs/ not a SequenceFile
> at
> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1934)
> at
> org.apache.hadoop.io.SequenceFile$Reader.initialize(SequenceFile.java:1893)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1842)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1856)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:70)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.reset(SequenceFileLogReader.java:168)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.initReader(SequenceFileLogReader.java:177)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:66)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:313)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:277)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:265)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:424)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:352)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.handleFileNotFound(WALEntryStream.java:341)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:359)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openNextLog(WALEntryStream.java:316)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.checkReader(WALEntryStream.java:306)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:207)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:110)
> ... 1 more
> {noformat}
> We fixed similar error via HBASE-25536 but the zero length file was in
> recovered sources.
> There were more logs after the above stack trace.
> {noformat}
> 2021-07-05 03:06:32,757 WARN [20%2C1625185107182,1]
> regionserver.ReplicationSourceWALReaderThread - Couldn't get file length
> information about log
> hdfs:///hbase/WALs/
> 2021-07-05 03:06:32,754 INFO [20%2C1625185107182,1]
> regionserver.WALEntryStream - Log hdfs:///hbase/WALs/
> was moved to hdfs:///hbase/oldWALs/
> {noformat}
> There is a special logic in ReplicationSourceWALReader thread to handle 0
> length files but we only look in WALs directory and not in oldWALs directory.
> {code}
> private boolean handleEofException(Exception e, WALEntryBatch batch) {
> PriorityBlockingQueue queue = logQueue.getQueue(walGroupId);
> // Dump the log even if logQueue size is 1 if the source is from
> recovered Source
> // since we don't add current log to recovered source queue so it is safe
> to remove.
> if ((e instanceof EOFException || e.getCause() instanceof
[jira] [Updated] (HBASE-26075) Replication is stuck due to zero length wal file in oldWALs directory.
[
https://issues.apache.org/jira/browse/HBASE-26075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rushabh Shah updated HBASE-26075:
-
Affects Version/s: 3.0.0-alpha-1
2.3.5
2.4.4
> Replication is stuck due to zero length wal file in oldWALs directory.
> --
>
> Key: HBASE-26075
> URL: https://issues.apache.org/jira/browse/HBASE-26075
> Project: HBase
> Issue Type: Bug
> Components: Replication, wal
>Affects Versions: 3.0.0-alpha-1, 1.7.0, 2.3.5, 2.4.4
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Critical
>
> Recently we encountered a case where size of log queue was increasing to
> around 300 in few region servers in our production environment.
> There were 295 wals in the oldWALs directory for that region server and the
> *first file* was a 0 length file.
> Replication was throwing the following error.
> {noformat}
> 2021-07-05 03:06:32,757 ERROR [20%2C1625185107182,1]
> regionserver.ReplicationSourceWALReaderThread - Failed to read stream of
> replication entries
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream$WALEntryStreamRuntimeException:
> java.io.EOFException: hdfs:///hbase/oldWALs/ not a
> SequenceFile
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:112)
> at
> org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReaderThread.run(ReplicationSourceWALReaderThread.java:156)
> Caused by: java.io.EOFException:
> hdfs:///hbase/oldWALs/ not a SequenceFile
> at
> org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1934)
> at
> org.apache.hadoop.io.SequenceFile$Reader.initialize(SequenceFile.java:1893)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1842)
> at
> org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1856)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:70)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.reset(SequenceFileLogReader.java:168)
> at
> org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.initReader(SequenceFileLogReader.java:177)
> at
> org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:66)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:313)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:277)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:265)
> at
> org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:424)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:352)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.handleFileNotFound(WALEntryStream.java:341)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:359)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openNextLog(WALEntryStream.java:316)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.checkReader(WALEntryStream.java:306)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:207)
> at
> org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:110)
> ... 1 more
> {noformat}
> We fixed similar error via HBASE-25536 but the zero length file was in
> recovered sources.
> There were more logs after the above stack trace.
> {noformat}
> 2021-07-05 03:06:32,757 WARN [20%2C1625185107182,1]
> regionserver.ReplicationSourceWALReaderThread - Couldn't get file length
> information about log
> hdfs:///hbase/WALs/
> 2021-07-05 03:06:32,754 INFO [20%2C1625185107182,1]
> regionserver.WALEntryStream - Log hdfs:///hbase/WALs/
> was moved to hdfs:///hbase/oldWALs/
> {noformat}
> There is a special logic in ReplicationSourceWALReader thread to handle 0
> length files but we only look in WALs directory and not in oldWALs directory.
> {code}
> private boolean handleEofException(Exception e, WALEntryBatch batch) {
> PriorityBlockingQueue queue = logQueue.getQueue(walGroupId);
> // Dump the log even if logQueue size is 1 if the source is from
> recovered Source
> // since we don't add current log to recovered source queue so it is safe
> to remove.
> if ((e instanceof EOFException || e.getCause() instanceof EOFException) &&
> (source.isRecovered() || queue.size() > 1) && this.eofAutoRecovery) {
> Path head = queue.peek();
> try
[jira] [Created] (HBASE-26075) Replication is stuck due to zero length wal file in oldWALs directory.
Rushabh Shah created HBASE-26075:
Summary: Replication is stuck due to zero length wal file in
oldWALs directory.
Key: HBASE-26075
URL: https://issues.apache.org/jira/browse/HBASE-26075
Project: HBase
Issue Type: Bug
Components: Replication, wal
Affects Versions: 1.7.0
Reporter: Rushabh Shah
Assignee: Rushabh Shah
Recently we encountered a case where size of log queue was increasing to around
300 in few region servers in our production environment.
There were 295 wals in the oldWALs directory for that region server and the
*first file* was a 0 length file.
Replication was throwing the following error.
{noformat}
2021-07-05 03:06:32,757 ERROR [20%2C1625185107182,1]
regionserver.ReplicationSourceWALReaderThread - Failed to read stream of
replication entries
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream$WALEntryStreamRuntimeException:
java.io.EOFException: hdfs:///hbase/oldWALs/ not a
SequenceFile
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:112)
at
org.apache.hadoop.hbase.replication.regionserver.ReplicationSourceWALReaderThread.run(ReplicationSourceWALReaderThread.java:156)
Caused by: java.io.EOFException: hdfs:///hbase/oldWALs/
not a SequenceFile
at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1934)
at
org.apache.hadoop.io.SequenceFile$Reader.initialize(SequenceFile.java:1893)
at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1842)
at
org.apache.hadoop.io.SequenceFile$Reader.(SequenceFile.java:1856)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader$WALReader.(SequenceFileLogReader.java:70)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.reset(SequenceFileLogReader.java:168)
at
org.apache.hadoop.hbase.regionserver.wal.SequenceFileLogReader.initReader(SequenceFileLogReader.java:177)
at
org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:66)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:313)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:277)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:265)
at
org.apache.hadoop.hbase.wal.WALFactory.createReader(WALFactory.java:424)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:352)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.handleFileNotFound(WALEntryStream.java:341)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openReader(WALEntryStream.java:359)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.openNextLog(WALEntryStream.java:316)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.checkReader(WALEntryStream.java:306)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.tryAdvanceEntry(WALEntryStream.java:207)
at
org.apache.hadoop.hbase.replication.regionserver.WALEntryStream.hasNext(WALEntryStream.java:110)
... 1 more
{noformat}
We fixed similar error via HBASE-25536 but the zero length file was in
recovered sources.
There were more logs after the above stack trace.
{noformat}
2021-07-05 03:06:32,757 WARN [20%2C1625185107182,1]
regionserver.ReplicationSourceWALReaderThread - Couldn't get file length
information about log
hdfs:///hbase/WALs/
2021-07-05 03:06:32,754 INFO [20%2C1625185107182,1]
regionserver.WALEntryStream - Log hdfs:///hbase/WALs/
was moved to hdfs:///hbase/oldWALs/
{noformat}
There is a special logic in ReplicationSourceWALReader thread to handle 0
length files but we only look in WALs directory and not in oldWALs directory.
{code}
private boolean handleEofException(Exception e, WALEntryBatch batch) {
PriorityBlockingQueue queue = logQueue.getQueue(walGroupId);
// Dump the log even if logQueue size is 1 if the source is from recovered
Source
// since we don't add current log to recovered source queue so it is safe
to remove.
if ((e instanceof EOFException || e.getCause() instanceof EOFException) &&
(source.isRecovered() || queue.size() > 1) && this.eofAutoRecovery) {
Path head = queue.peek();
try {
if (fs.getFileStatus(head).getLen() == 0) {
// head of the queue is an empty log file
LOG.warn("Forcing removal of 0 length log in queue: {}", head);
logQueue.remove(walGroupId);
currentPosition = 0;
if (batch != null) {
// After we removed the WAL from the queue, we should try shipping
the existing batch of
// entries
addBatchToShippingQueue(batch);
}
return true;
}
}
[jira] [Commented] (HBASE-26074) testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on branch-1
[
https://issues.apache.org/jira/browse/HBASE-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377499#comment-17377499
]
Bharath Vissapragada commented on HBASE-26074:
--
Runs ok locally (mac), looks like the issue is only on CI.
> testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on
> branch-1
> --
>
> Key: HBASE-26074
> URL: https://issues.apache.org/jira/browse/HBASE-26074
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.7.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Labels: regression
> Fix For: 1.7.1
>
>
> In prep-ing for 1.7.1, I noticed that this test has been consistently failing
> on branch-1 forever.
> {noformat}
> Regression
> health checks / yetus jdk8 hadoop2 checks /
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego
> Failing for the past 1 build (Since Failed#145 )
> Took 0.79 sec.
> Error Message
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:743)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:702)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at
> org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:189)
> at
> org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:347)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.connect(LogLevel.java:268)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.process(LogLevel.java:284)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.doGetLevel(LogLevel.java:227)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.sendLogLevelRequest(LogLevel.java:123)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.run(LogLevel.java:107)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.getLevel(TestLogLevel.java:349)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.access$000(TestLogLevel.java:68)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:325)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:322)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:277)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego(TestLogLevel.java:451)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> Stacktrace
> java.lang.AssertionError:
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at
[jira] [Updated] (HBASE-26074) testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on branch-1
[
https://issues.apache.org/jira/browse/HBASE-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bharath Vissapragada updated HBASE-26074:
-
Labels: regression (was: )
> testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on
> branch-1
> --
>
> Key: HBASE-26074
> URL: https://issues.apache.org/jira/browse/HBASE-26074
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.7.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Labels: regression
> Fix For: 1.7.1
>
>
> In prep-ing for 1.7.1, I noticed that this test has been consistently failing
> on branch-1 forever.
> {noformat}
> Regression
> health checks / yetus jdk8 hadoop2 checks /
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego
> Failing for the past 1 build (Since Failed#145 )
> Took 0.79 sec.
> Error Message
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:743)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:702)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at
> org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:189)
> at
> org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:347)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.connect(LogLevel.java:268)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.process(LogLevel.java:284)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.doGetLevel(LogLevel.java:227)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.sendLogLevelRequest(LogLevel.java:123)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.run(LogLevel.java:107)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.getLevel(TestLogLevel.java:349)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.access$000(TestLogLevel.java:68)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:325)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:322)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:277)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego(TestLogLevel.java:451)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> Stacktrace
> java.lang.AssertionError:
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at
[jira] [Commented] (HBASE-26074) testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on branch-1
[
https://issues.apache.org/jira/browse/HBASE-26074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377491#comment-17377491
]
Bharath Vissapragada commented on HBASE-26074:
--
Sample jenkins job (that will expire soon anyway):
https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/145/
> testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on
> branch-1
> --
>
> Key: HBASE-26074
> URL: https://issues.apache.org/jira/browse/HBASE-26074
> Project: HBase
> Issue Type: Bug
> Components: test
>Affects Versions: 1.7.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: 1.7.1
>
>
> In prep-ing for 1.7.1, I noticed that this test has been consistently failing
> on branch-1 forever.
> {noformat}
> Regression
> health checks / yetus jdk8 hadoop2 checks /
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego
> Failing for the past 1 build (Since Failed#145 )
> Took 0.79 sec.
> Error Message
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:743)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:702)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
> at
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> at
> org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:189)
> at
> org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:347)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.connect(LogLevel.java:268)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.process(LogLevel.java:284)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.doGetLevel(LogLevel.java:227)
> at
> org.apache.hadoop.hbase.http.log.LogLevel$CLI.sendLogLevelRequest(LogLevel.java:123)
> at org.apache.hadoop.hbase.http.log.LogLevel$CLI.run(LogLevel.java:107)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.getLevel(TestLogLevel.java:349)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.access$000(TestLogLevel.java:68)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:325)
> at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:322)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:322)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:277)
> at
> org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego(TestLogLevel.java:451)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
> at
> org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.lang.Thread.run(Thread.java:748)
> Stacktrace
> java.lang.AssertionError:
> Expected to find 'Unexpected end of file from server' but got unexpected
> exception:java.net.SocketException: Connection reset
> at java.net.SocketInputStream.read(SocketInputStream.java:210)
> at
[jira] [Created] (HBASE-26074) testLogLevelByHttps/testLogLevelByHttpsWithSpnego consistently failing on branch-1
Bharath Vissapragada created HBASE-26074:
Summary: testLogLevelByHttps/testLogLevelByHttpsWithSpnego
consistently failing on branch-1
Key: HBASE-26074
URL: https://issues.apache.org/jira/browse/HBASE-26074
Project: HBase
Issue Type: Bug
Components: test
Affects Versions: 1.7.0
Reporter: Bharath Vissapragada
Assignee: Bharath Vissapragada
Fix For: 1.7.1
In prep-ing for 1.7.1, I noticed that this test has been consistently failing
on branch-1 forever.
{noformat}
Regression
health checks / yetus jdk8 hadoop2 checks /
org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego
Failing for the past 1 build (Since Failed#145 )
Took 0.79 sec.
Error Message
Expected to find 'Unexpected end of file from server' but got unexpected
exception:java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:743)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:702)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1593)
at
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1498)
at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
at
org.apache.hadoop.security.authentication.client.KerberosAuthenticator.authenticate(KerberosAuthenticator.java:189)
at
org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:347)
at org.apache.hadoop.hbase.http.log.LogLevel$CLI.connect(LogLevel.java:268)
at org.apache.hadoop.hbase.http.log.LogLevel$CLI.process(LogLevel.java:284)
at org.apache.hadoop.hbase.http.log.LogLevel$CLI.doGetLevel(LogLevel.java:227)
at
org.apache.hadoop.hbase.http.log.LogLevel$CLI.sendLogLevelRequest(LogLevel.java:123)
at org.apache.hadoop.hbase.http.log.LogLevel$CLI.run(LogLevel.java:107)
at
org.apache.hadoop.hbase.http.log.TestLogLevel.getLevel(TestLogLevel.java:349)
at
org.apache.hadoop.hbase.http.log.TestLogLevel.access$000(TestLogLevel.java:68)
at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:325)
at org.apache.hadoop.hbase.http.log.TestLogLevel$1.run(TestLogLevel.java:322)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1844)
at
org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:322)
at
org.apache.hadoop.hbase.http.log.TestLogLevel.testDynamicLogLevel(TestLogLevel.java:277)
at
org.apache.hadoop.hbase.http.log.TestLogLevel.testLogLevelByHttpsWithSpnego(TestLogLevel.java:451)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:298)
at
org.junit.internal.runners.statements.FailOnTimeout$CallableStatement.call(FailOnTimeout.java:292)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.lang.Thread.run(Thread.java:748)
Stacktrace
java.lang.AssertionError:
Expected to find 'Unexpected end of file from server' but got unexpected
exception:java.net.SocketException: Connection reset
at java.net.SocketInputStream.read(SocketInputStream.java:210)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:743)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:678)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:702)
at
[GitHub] [hbase] Apache-HBase commented on pull request #3458: HBASE-25700 Enhance znode parent validation when add_peer
Apache-HBase commented on pull request #3458: URL: https://github.com/apache/hbase/pull/3458#issuecomment-876562354 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 34s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | hbaseanti | 0m 0s | Patch does not have any anti-patterns. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ master Compile Tests _ | | +0 :ok: | mvndep | 0m 14s | Maven dependency ordering for branch | | +1 :green_heart: | mvninstall | 4m 25s | master passed | | +1 :green_heart: | compile | 4m 36s | master passed | | +1 :green_heart: | checkstyle | 1m 37s | master passed | | +1 :green_heart: | spotbugs | 3m 20s | master passed | ||| _ Patch Compile Tests _ | | +0 :ok: | mvndep | 0m 12s | Maven dependency ordering for patch | | +1 :green_heart: | mvninstall | 4m 13s | the patch passed | | +1 :green_heart: | compile | 4m 30s | the patch passed | | +1 :green_heart: | javac | 4m 30s | the patch passed | | +1 :green_heart: | checkstyle | 1m 38s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +1 :green_heart: | hadoopcheck | 22m 12s | Patch does not cause any errors with Hadoop 3.1.2 3.2.1 3.3.0. | | +1 :green_heart: | spotbugs | 4m 11s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 25s | The patch does not generate ASF License warnings. | | | | 62m 48s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3458/7/artifact/yetus-general-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3458 | | Optional Tests | dupname asflicense javac spotbugs hadoopcheck hbaseanti checkstyle compile | | uname | Linux 2e7624c0e3fa 4.15.0-128-generic #131-Ubuntu SMP Wed Dec 9 06:57:35 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e65fc9226f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Max. process+thread count | 86 (vs. ulimit of 3) | | modules | C: hbase-client hbase-server U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3458/7/console | | versions | git=2.17.1 maven=3.6.3 spotbugs=4.2.2 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26070) Tooling to auto-convert 1.7.0 serialized tables to 1.7.1 compatible tables
[
https://issues.apache.org/jira/browse/HBASE-26070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377464#comment-17377464
]
Hudson commented on HBASE-26070:
Results for branch branch-1
[build #145 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/145/]:
(x) *{color:red}-1 overall{color}*
details (if available):
(x) {color:red}-1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/145//General_Nightly_Build_Report/]
(/) {color:green}+1 jdk7 checks{color}
-- For more information [see jdk7
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/145//JDK7_Nightly_Build_Report/]
(x) {color:red}-1 jdk8 hadoop2 checks{color}
-- For more information [see jdk8 (hadoop2)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/branch-1/145//JDK8_Nightly_Build_Report_(Hadoop2)/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
> Tooling to auto-convert 1.7.0 serialized tables to 1.7.1 compatible tables
> --
>
> Key: HBASE-26070
> URL: https://issues.apache.org/jira/browse/HBASE-26070
> Project: HBase
> Issue Type: Sub-task
>Affects Versions: 1.7.0
>Reporter: Bharath Vissapragada
>Assignee: Bharath Vissapragada
>Priority: Major
> Fix For: 1.7.1
>
>
> As discussed in the parent issue, 1.7.0 introduced an incompatible table
> serialization change. We need tooling that auto converts these into 1.7.1
> compatible table descriptors and table states.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] virajjasani commented on a change in pull request #3372: HBASE-25986 set default value of normalization enabled from hbase site
virajjasani commented on a change in pull request #3372:
URL: https://github.com/apache/hbase/pull/3372#discussion_r666286329
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptor.java
##
@@ -275,9 +275,9 @@
/**
* Check if normalization enable flag of the table is true. If flag is false
- * then no region normalizer won't attempt to normalize this table.
+ * then region normalizer won't attempt to normalize this table.
*
- * @return true if region normalization is enabled for this table
Review comment:
nit: I think this one looks concise and sufficient, good to keep it as
is.
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizer.java
##
@@ -36,6 +36,7 @@
*/
@InterfaceAudience.Private
interface RegionNormalizer extends Configurable {
+ String NORMALIZATION_ENABLED_CONF_KEY = "hbase.table.normalization.enabled";
Review comment:
Good to define this directly in `RegionNormalizerWorker`
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizerWorker.java
##
@@ -71,6 +73,12 @@
this.splitPlanCount = 0;
this.mergePlanCount = 0;
this.rateLimiter = loadRateLimiter(configuration);
+this.defaultNormalizerValue = extractDefaultNormalizerValue(configuration);
+ }
+
+ private boolean extractDefaultNormalizerValue(final Configuration
configuration) {
+String s =
configuration.get(RegionNormalizer.NORMALIZATION_ENABLED_CONF_KEY);
+return Boolean.valueOf(s);
Review comment:
Simplify to `Boolean.parseBoolean()`?
##
File path:
hbase-client/src/main/java/org/apache/hadoop/hbase/client/TableDescriptorBuilder.java
##
@@ -866,11 +860,11 @@ public ModifyableTableDescriptor setMergeEnabled(final
boolean isEnable) {
* Check if normalization enable flag of the table is true. If flag is
false
* then no region normalizer won't attempt to normalize this table.
*
- * @return true if region normalization is enabled for this table
Review comment:
same as above
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizerWorker.java
##
@@ -181,10 +189,19 @@ public void run() {
final TableDescriptor tblDesc;
try {
tblDesc = masterServices.getTableDescriptors().get(tableName);
- if (tblDesc != null && !tblDesc.isNormalizationEnabled()) {
-LOG.debug("Skipping table {} because normalization is disabled in its
table properties.",
- tableName);
-return Collections.emptyList();
+ boolean normalizationEnabled;
+ if (tblDesc != null) {
+String defined =
tblDesc.getValue(TableDescriptorBuilder.NORMALIZATION_ENABLED);
+if(defined != null) {
+ normalizationEnabled = tblDesc.isNormalizationEnabled();
+} else {
+ normalizationEnabled = this.defaultNormalizerValue;
+}
+if (!normalizationEnabled) {
+ LOG.debug("Skipping table {} because normalization is disabled in
its table properties.",
+tableName);
Review comment:
Let's make it
```
if (!normalizationEnabled) {
LOG.debug("Skipping table {} because normalization is disabled in
its table properties and normalization is also disabled at table level by
default (config: {}).",
tableName, NORMALIZATION_ENABLED_CONF_KEY);
return Collections.emptyList();
}
```
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/master/normalizer/RegionNormalizerWorker.java
##
@@ -55,6 +56,7 @@
private final RateLimiter rateLimiter;
private final long[] skippedCount;
+ private final boolean defaultNormalizerValue;
Review comment:
Good to rename it to `defaultNormalizerTableLevel` or something similar
to indicate that this value is going to be used if not overridden by
`NORMALIZATION_ENABLED` at table descriptor level. Good to add this as comment
also.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3460: HBASE-26064 Introduce a StoreFileTracker to abstract the stor…
Apache9 commented on pull request #3460: URL: https://github.com/apache/hbase/pull/3460#issuecomment-876465025 https://docs.google.com/document/d/16Nr1Fn3VaXuz1g1FTiME-bnGR3qVK5B-raXshOkDLcY/edit?usp=sharing I've finish the first version of the design doc. PTAL. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (HBASE-26069) Remove HStore.compactRecentForTestingAssumingDefaultPolicy and DefaultCompactor.compactForTesting
[
https://issues.apache.org/jira/browse/HBASE-26069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377406#comment-17377406
]
Hudson commented on HBASE-26069:
Results for branch master
[build #340 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> Remove HStore.compactRecentForTestingAssumingDefaultPolicy and
> DefaultCompactor.compactForTesting
> -
>
> Key: HBASE-26069
> URL: https://issues.apache.org/jira/browse/HBASE-26069
> Project: HBase
> Issue Type: Improvement
> Components: Compaction, test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-2
>
>
> We should try our best to not include testing code in normal code.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-25516) [JDK17] reflective access Field.class.getDeclaredField("modifiers") not supported
[
https://issues.apache.org/jira/browse/HBASE-25516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377405#comment-17377405
]
Hudson commented on HBASE-25516:
Results for branch master
[build #340 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> [JDK17] reflective access Field.class.getDeclaredField("modifiers") not
> supported
> -
>
> Key: HBASE-25516
> URL: https://issues.apache.org/jira/browse/HBASE-25516
> Project: HBase
> Issue Type: Sub-task
> Components: Filesystem Integration
>Affects Versions: 2.3.3
> Environment: Windows 10, JavaSE11, pom dependencies:
> {code:java}
>
> org.apache.hbase
> hbase-testing-util
> 2.3.3
>
>
> junit
> junit
> 4.12
> {code}
>Reporter: Leon Bein
>Assignee: Wei-Chiu Chuang
>Priority: Major
> Labels: jdk11
> Fix For: 3.0.0-alpha-2
>
>
> The reflective access
> {code:java}
> Field.class.getDeclaredField("modifiers")
> {code}
> in HFileSystem.java:334 leads to a warning (and probably error?):
>
> {code:java}
> java.lang.NoSuchFieldException: modifiers
> at java.base/java.lang.Class.getDeclaredField(Class.java:2417)
> at
> org.apache.hadoop.hbase.fs.HFileSystem.addLocationsOrderInterceptor(HFileSystem.java:334)
> at
> org.apache.hadoop.hbase.fs.HFileSystem.addLocationsOrderInterceptor(HFileSystem.java:291)
> at org.apache.hadoop.hbase.fs.HFileSystem.(HFileSystem.java:96)
> at org.apache.hadoop.hbase.fs.HFileSystem.get(HFileSystem.java:465)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getTestFileSystem(HBaseTestingUtility.java:3330)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getNewDataTestDirOnTestFS(HBaseTestingUtility.java:565)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.setupDataTestDirOnTestFS(HBaseTestingUtility.java:554)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getDataTestDirOnTestFS(HBaseTestingUtility.java:527)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.getDefaultRootDirPath(HBaseTestingUtility.java:1415)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.createRootDir(HBaseTestingUtility.java:1446)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:1157)
> at
> org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:1144)
> at foo.Main.main(Main.java:11)
> {code}
> when running the following code:
>
> {code:java}
> public static void main(String[] args) throws Exception {
> HBaseTestingUtility utility = new
> HBaseTestingUtility(HBaseConfiguration.create());
>
> utility.startMiniCluster(StartMiniClusterOption.builder().numRegionServers(3).build());
> }{code}
> From my knowledge this results from the more restrictive reflection
> protection of java.base classes in the newer java versions.
>
> Related to HBASE-22972
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (HBASE-26068) The last assertion in TestHStore.testRefreshStoreFilesNotChanged is wrong
[
https://issues.apache.org/jira/browse/HBASE-26068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17377404#comment-17377404
]
Hudson commented on HBASE-26068:
Results for branch master
[build #340 on
builds.a.o|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/]:
(/) *{color:green}+1 overall{color}*
details (if available):
(/) {color:green}+1 general checks{color}
-- For more information [see general
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/General_20Nightly_20Build_20Report/]
(/) {color:green}+1 jdk8 hadoop3 checks{color}
-- For more information [see jdk8 (hadoop3)
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/JDK8_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 jdk11 hadoop3 checks{color}
-- For more information [see jdk11
report|https://ci-hadoop.apache.org/job/HBase/job/HBase%20Nightly/job/master/340/JDK11_20Nightly_20Build_20Report_20_28Hadoop3_29/]
(/) {color:green}+1 source release artifact{color}
-- See build output for details.
(/) {color:green}+1 client integration test{color}
> The last assertion in TestHStore.testRefreshStoreFilesNotChanged is wrong
> -
>
> Key: HBASE-26068
> URL: https://issues.apache.org/jira/browse/HBASE-26068
> Project: HBase
> Issue Type: Bug
> Components: test
>Reporter: Duo Zhang
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 2.3.6, 3.0.0-alpha-2, 2.4.5
>
>
> The number of call times will not be reset so the last assertion should not
> be 0, and why it could pass now is because we pass null to the verification
> while we will never call this method with null...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [hbase] sunhelly commented on a change in pull request #3140: HBASE-25720 Sync WAL stuck when prepare flush cache will prevent flus…
sunhelly commented on a change in pull request #3140:
URL: https://github.com/apache/hbase/pull/3140#discussion_r666086645
##
File path:
hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java
##
@@ -2846,7 +2846,13 @@ protected PrepareFlushResult
internalPrepareFlushCache(WAL wal, long myseqid,
String s = "Finished memstore snapshotting " + this + ", syncing WAL and
waiting on mvcc, " +
"flushsize=" + totalSizeOfFlushableStores;
status.setStatus(s);
-doSyncOfUnflushedWALChanges(wal, getRegionInfo());
+
+try {
+ doSyncOfUnflushedWALChanges(wal, getRegionInfo());
+} catch (Throwable t) {
+ status.abort("Sync unflushed WAL changes failed: " +
StringUtils.stringifyException(t));
+ fatalForFlushCache(t);
Review comment:
@Apache9 We currently only abort RS when there are critical problems,
such as the WAL stuck? The above IOExceptions can only abort the flush in mem
and should not prevent the next flushes, right? Here the abort is caused by the
wal stucks who are ahead of the committing flush wal sync stucks. What do you
think?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2984: HBASE-25680 Non-idempotent test in TestReplicationHFileCleaner
Apache-HBase commented on pull request #2984: URL: https://github.com/apache/hbase/pull/2984#issuecomment-876191463 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 7s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 5m 6s | master passed | | +1 :green_heart: | compile | 1m 21s | master passed | | +1 :green_heart: | shadedjars | 9m 6s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 45s | master passed | | -0 :warning: | patch | 10m 1s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 4m 47s | the patch passed | | +1 :green_heart: | compile | 1m 17s | the patch passed | | +1 :green_heart: | javac | 1m 17s | the patch passed | | +1 :green_heart: | shadedjars | 9m 18s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 201m 48s | hbase-server in the patch passed. | | | | 237m 14s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2984/2/artifact/yetus-jdk11-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2984 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 184f8e841a84 4.15.0-136-generic #140-Ubuntu SMP Thu Jan 28 05:20:47 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e65fc9226f | | Default Java | AdoptOpenJDK-11.0.10+9 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2984/2/testReport/ | | Max. process+thread count | 3161 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2984/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache9 commented on pull request #3458: HBASE-25700 Enhance znode parent validation when add_peer
Apache9 commented on pull request #3458: URL: https://github.com/apache/hbase/pull/3458#issuecomment-876184321 Please fix the checkstyle issues then I will merge it. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #3465: HBASE-26071: Document HBASE-26021 and upgrade considerations for 1.7.…
Apache-HBase commented on pull request #3465: URL: https://github.com/apache/hbase/pull/3465#issuecomment-876177755 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 7m 16s | Docker mode activated. | ||| _ Prechecks _ | | +1 :green_heart: | dupname | 0m 0s | No case conflicting files found. | | +1 :green_heart: | @author | 0m 0s | The patch does not contain any @author tags. | ||| _ branch-1 Compile Tests _ | | +1 :green_heart: | mvninstall | 9m 55s | branch-1 passed | | +0 :ok: | refguide | 4m 1s | branch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 1m 58s | the patch passed | | +1 :green_heart: | whitespace | 0m 0s | The patch has no whitespace issues. | | +0 :ok: | refguide | 3m 20s | patch has no errors when building the reference guide. See footer for rendered docs, which you should manually inspect. | ||| _ Other Tests _ | | +1 :green_heart: | asflicense | 0m 45s | The patch does not generate ASF License warnings. | | | | 28m 42s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3465/1/artifact/out/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/3465 | | JIRA Issue | HBASE-26071 | | Optional Tests | dupname asflicense refguide | | uname | Linux 65064d862aa0 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | /home/jenkins/jenkins-home/workspace/Base-PreCommit-GitHub-PR_PR-3465/out/precommit/personality/provided.sh | | git revision | branch-1 / 5ece9cef | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3465/1/artifact/out/branch-site/book.html | | refguide | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3465/1/artifact/out/patch-site/book.html | | Max. process+thread count | 86 (vs. ulimit of 1) | | modules | C: . U: . | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-3465/1/console | | versions | git=1.9.1 maven=3.0.5 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] sunhelly commented on pull request #3436: HBASE-26036 DBB released too early in HRegion.get() and dirty data for some operations
sunhelly commented on pull request #3436: URL: https://github.com/apache/hbase/pull/3436#issuecomment-876171301 @saintstack @Apache9 @anoopsjohn PTAL, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] bharathv opened a new pull request #3465: HBASE-26071: Document HBASE-26021 and upgrade considerations for 1.7.…
bharathv opened a new pull request #3465: URL: https://github.com/apache/hbase/pull/3465 …0/1.7.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [hbase] Apache-HBase commented on pull request #2984: HBASE-25680 Non-idempotent test in TestReplicationHFileCleaner
Apache-HBase commented on pull request #2984: URL: https://github.com/apache/hbase/pull/2984#issuecomment-876158713 :confetti_ball: **+1 overall** | Vote | Subsystem | Runtime | Comment | |::|--:|:|:| | +0 :ok: | reexec | 1m 28s | Docker mode activated. | | -0 :warning: | yetus | 0m 3s | Unprocessed flag(s): --brief-report-file --spotbugs-strict-precheck --whitespace-eol-ignore-list --whitespace-tabs-ignore-list --quick-hadoopcheck | ||| _ Prechecks _ | ||| _ master Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 57s | master passed | | +1 :green_heart: | compile | 1m 1s | master passed | | +1 :green_heart: | shadedjars | 8m 3s | branch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 41s | master passed | | -0 :warning: | patch | 8m 55s | Used diff version of patch file. Binary files and potentially other changes not applied. Please rebase and squash commits if necessary. | ||| _ Patch Compile Tests _ | | +1 :green_heart: | mvninstall | 3m 43s | the patch passed | | +1 :green_heart: | compile | 1m 2s | the patch passed | | +1 :green_heart: | javac | 1m 2s | the patch passed | | +1 :green_heart: | shadedjars | 8m 4s | patch has no errors when building our shaded downstream artifacts. | | +1 :green_heart: | javadoc | 0m 36s | the patch passed | ||| _ Other Tests _ | | +1 :green_heart: | unit | 145m 59s | hbase-server in the patch passed. | | | | 176m 40s | | | Subsystem | Report/Notes | |--:|:-| | Docker | ClientAPI=1.41 ServerAPI=1.41 base: https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2984/2/artifact/yetus-jdk8-hadoop3-check/output/Dockerfile | | GITHUB PR | https://github.com/apache/hbase/pull/2984 | | Optional Tests | javac javadoc unit shadedjars compile | | uname | Linux 438c1052b087 4.15.0-112-generic #113-Ubuntu SMP Thu Jul 9 23:41:39 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux | | Build tool | maven | | Personality | dev-support/hbase-personality.sh | | git revision | master / e65fc9226f | | Default Java | AdoptOpenJDK-1.8.0_282-b08 | | Test Results | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2984/2/testReport/ | | Max. process+thread count | 5039 (vs. ulimit of 3) | | modules | C: hbase-server U: hbase-server | | Console output | https://ci-hadoop.apache.org/job/HBase/job/HBase-PreCommit-GitHub-PR/job/PR-2984/2/console | | versions | git=2.17.1 maven=3.6.3 | | Powered by | Apache Yetus 0.12.0 https://yetus.apache.org | This message was automatically generated. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@hbase.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (HBASE-26073) Avoid merging regions if tables has not reached the state where regions are not split because of number of regions
[ https://issues.apache.org/jira/browse/HBASE-26073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aman Poonia updated HBASE-26073: Summary: Avoid merging regions if tables has not reached the state where regions are not split because of number of regions (was: Avoid merging regions if tables has not reached the state where numbers of regions are not split because of number of regions) > Avoid merging regions if tables has not reached the state where regions are > not split because of number of regions > -- > > Key: HBASE-26073 > URL: https://issues.apache.org/jira/browse/HBASE-26073 > Project: HBase > Issue Type: Improvement > Components: Normalizer >Affects Versions: 1.7.0, 2.4.4 >Reporter: Aman Poonia >Assignee: Aman Poonia >Priority: Minor > > we have a table on a cluster with 100 regions with default split policy > (SteppingSplitPolicy). This is a small table and will not get loaded with too > much data. Now if region size of table is smaller than the normalizer target > region size than there are chances that normalizer will consider the regions > for merges. But since the number of regions are small split policy will > trigger split on next flush. This is a continuous loop and our cluster will > be busy in these two actions. > We plan to consider number of regions and number of regions in creating plans > for normalizer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-26073) Avoid merging regions if tables has not reached the state where numbers of regions are not split because of number of regions
Aman Poonia created HBASE-26073: --- Summary: Avoid merging regions if tables has not reached the state where numbers of regions are not split because of number of regions Key: HBASE-26073 URL: https://issues.apache.org/jira/browse/HBASE-26073 Project: HBase Issue Type: Improvement Components: Normalizer Affects Versions: 2.4.4, 1.7.0 Reporter: Aman Poonia Assignee: Aman Poonia we have a table on a cluster with 100 regions with default split policy (SteppingSplitPolicy). This is a small table and will not get loaded with too much data. Now if region size of table is smaller than the normalizer target region size than there are chances that normalizer will consider the regions for merges. But since the number of regions are small split policy will trigger split on next flush. This is a continuous loop and our cluster will be busy in these two actions. We plan to consider number of regions and number of regions in creating plans for normalizer. -- This message was sent by Atlassian Jira (v8.3.4#803005)
