[jira] [Resolved] (HBASE-25927) Fix the log messages by not stringifying the exceptions in log
[ https://issues.apache.org/jira/browse/HBASE-25927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bharath Vissapragada resolved HBASE-25927. -- Resolution: Fixed > Fix the log messages by not stringifying the exceptions in log > -- > > Key: HBASE-25927 > URL: https://issues.apache.org/jira/browse/HBASE-25927 > Project: HBase > Issue Type: Bug > Components: logging >Affects Versions: 3.0.0-alpha-1, 2.5.0, 2.4.4 >Reporter: Sandeep Pal >Assignee: Sandeep Pal >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.5.0, 2.4.4 > > > There are few places where we stringify the exceptions and log, instead we > should just pass them as a parameter to see the stack trace in good format. > For example: > https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceWALReader.java#L175 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25948) Remove deprecated ZK command 'rmr' in hbase-cleanup.sh
mokai created HBASE-25948: - Summary: Remove deprecated ZK command 'rmr' in hbase-cleanup.sh Key: HBASE-25948 URL: https://issues.apache.org/jira/browse/HBASE-25948 Project: HBase Issue Type: Task Components: scripts Affects Versions: 3.0.0-alpha-1, 2.5.0 Reporter: mokai Deprecated command 'rmr' was removed from ZooKeeper since 3.6.0(see: ZOOKEEPER-3411). It's better to use 'deleteall' instead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25938) The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one
[ https://issues.apache.org/jira/browse/HBASE-25938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25938. --- Fix Version/s: 2.4.4 2.3.6 2.5.0 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2.3+. Thanks [~niuyulin] for reviewig. > The SnapshotOfRegionAssignmentFromMeta.initialize call in > FavoredNodeLoadBalancer is just a dummy one > - > > Key: HBASE-25938 > URL: https://issues.apache.org/jira/browse/HBASE-25938 > Project: HBase > Issue Type: Bug > Components: Balancer, FavoredNodes >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.4 > > > After we introduced FavoredNodesManager, we do not need to load this > information every time when balancing, as all the updates to favored nodes > will go to FavoredNodesManager and it can update its in memory state > accordingly. > The current SnapshotOfRegionAssignmentFromMeta.initialize is just a dummy one > in FavoredNodeLoadBalancer, as we never use it in the method. > It will introduce a full scan on hbase:meta so let's just remove it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25943) [hbase-thirdparty] Bump dependencis in hbase-thirdparty
[ https://issues.apache.org/jira/browse/HBASE-25943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25943. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~niuyulin] for reviewing. > [hbase-thirdparty] Bump dependencis in hbase-thirdparty > --- > > Key: HBASE-25943 > URL: https://issues.apache.org/jira/browse/HBASE-25943 > Project: HBase > Issue Type: Task > Components: dependencies, thirdparty >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: thirdparty-3.5.1 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25947) Backport 'HBASE-25894 Improve the performance for region load and region count related cost functions' to branch-2.4 and branch-2.3
Duo Zhang created HBASE-25947: - Summary: Backport 'HBASE-25894 Improve the performance for region load and region count related cost functions' to branch-2.4 and branch-2.3 Key: HBASE-25947 URL: https://issues.apache.org/jira/browse/HBASE-25947 Project: HBase Issue Type: Sub-task Components: Balancer, Performance Reporter: Duo Zhang Assignee: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25946) [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.5.1
Duo Zhang created HBASE-25946: - Summary: [hbase-thirdparty] Generate CHANGES.md and RELEASENOTES.md for 3.5.1 Key: HBASE-25946 URL: https://issues.apache.org/jira/browse/HBASE-25946 Project: HBase Issue Type: Sub-task Components: thirdparty Reporter: Duo Zhang Fix For: thirdparty-3.5.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25945) [hbase-thirdparty] Set version as 3.5.1 in prep for first RC
Duo Zhang created HBASE-25945: - Summary: [hbase-thirdparty] Set version as 3.5.1 in prep for first RC Key: HBASE-25945 URL: https://issues.apache.org/jira/browse/HBASE-25945 Project: HBase Issue Type: Sub-task Components: thirdparty Reporter: Duo Zhang Fix For: thirdparty-3.5.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25944) [hbase-thirdparty] Release 3.5.1
Duo Zhang created HBASE-25944: - Summary: [hbase-thirdparty] Release 3.5.1 Key: HBASE-25944 URL: https://issues.apache.org/jira/browse/HBASE-25944 Project: HBase Issue Type: Umbrella Reporter: Duo Zhang Assignee: Duo Zhang -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25943) [hbase-thirdparty] Bump dependencis in hbase-thirdparty
Duo Zhang created HBASE-25943: - Summary: [hbase-thirdparty] Bump dependencis in hbase-thirdparty Key: HBASE-25943 URL: https://issues.apache.org/jira/browse/HBASE-25943 Project: HBase Issue Type: Task Components: dependencies, thirdparty Reporter: Duo Zhang Assignee: Duo Zhang Fix For: thirdparty-3.5.1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25942) Get rid of null regioninfo in wrapped connection exceptions
Bharath Vissapragada created HBASE-25942:
Summary: Get rid of null regioninfo in wrapped connection
exceptions
Key: HBASE-25942
URL: https://issues.apache.org/jira/browse/HBASE-25942
Project: HBase
Issue Type: Improvement
Components: logging
Reporter: Bharath Vissapragada
Assignee: Bharath Vissapragada
I was debugging something and noticed this,
{noformat}
org.apache.hbase.thirdparty.com.google.protobuf.ServiceException:
java.io.IOException: Call to address=localhost:59142null failed on local
exception: java.io.IOException: Connection reset by peer
{noformat}
`null` there is regioninfo and because this call has nothing set on it, it was
confusing where that null is coming from, added recently via HBASE-25735.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25941) TestRESTServerSSL fails because of jdk bug
Michael Stack created HBASE-25941: - Summary: TestRESTServerSSL fails because of jdk bug Key: HBASE-25941 URL: https://issues.apache.org/jira/browse/HBASE-25941 Project: HBase Issue Type: Sub-task Components: test Reporter: Michael Stack [~weijing329] identified issue in TestRESTServerSSL when using jdk8 252+. It came up in comment in the parent issue. I verified it fails for me using jdk8 v292. Here is the failure ```[INFO] Running org.apache.hadoop.hbase.rest.TestRESTServerSSL [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 1.4 s <<< FAILURE! - in org.apache.hadoop.hbase.rest.TestRESTServerSSL [ERROR] org.apache.hadoop.hbase.rest.TestRESTServerSSL Time elapsed: 1.387 s <<< ERROR! java.security.NoSuchAlgorithmException: unrecognized algorithm name: PBEWithSHA1AndDESede at org.apache.hadoop.hbase.rest.TestRESTServerSSL.beforeClass(TestRESTServerSSL.java:74)``` For workaround, see https://github.com/bcgit/bc-java/issues/941 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25940) Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO
Michael Stack created HBASE-25940:
-
Summary: Update Compression/TestCompressionTest: LZ4, SNAPPY, LZO
Key: HBASE-25940
URL: https://issues.apache.org/jira/browse/HBASE-25940
Project: HBase
Issue Type: Sub-task
Reporter: Michael Stack
LZ4 was changed in hadoop-3.3.1 to use a more amenable library, one that did
not require gymnastics installing lz4 native libs everywhere; rather, you just
add a jar to the classpath. See HADOOP-17292.
Similar was done for SNAPPY. See HADOOP-17125.
What this means is that our TestCompressionTest passes for hadoop before 3.3.1
but at 3.3.1, the assert that SNAPPY and LZ4 compressors should fail -- because
no lib installed -- now no longer asserts.
While in here, LZO is GPL and requires extra install to setup [1]. When
TestCompressionTest runs, it emits the below for the LZO check. The check is
kinda useless.
{code}
2021-05-28T10:05:36,513 WARN [Time-limited test] util.CompressionTest(75):
Can't instantiate codec: lzo
org.apache.hadoop.hbase.DoNotRetryIOException: Compression algorithm 'lzo'
previously failed test.
{code}
I think best thing for now is to comment out the asserts that LZ4 and SNAPPY do
NOT work when binary cannot be found -- since this holds only if hadoop <
3.3..1; the test is a little weak anyways.
1.
https://stackoverflow.com/questions/23441142/class-com-hadoop-compression-lzo-lzocodec-not-found-for-spark-on-cdh-5
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (HBASE-25651) NORMALIZER_TARGET_REGION_SIZE needs a unit in its name
[
https://issues.apache.org/jira/browse/HBASE-25651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reopened HBASE-25651:
--
Re-opening for backport to branch-2.
> NORMALIZER_TARGET_REGION_SIZE needs a unit in its name
> --
>
> Key: HBASE-25651
> URL: https://issues.apache.org/jira/browse/HBASE-25651
> Project: HBase
> Issue Type: Bug
> Components: Normalizer
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Rahul Kumar
>Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> On a per-table basis, the normalizer can be configured with a specific region
> size target via {{NORMALIZER_TARGET_REGION_SIZE}}. This value is used in a
> context where size values are in the megabyte scale. However, this
> configuration name does not mention "megabyte" or "mb" anywhere, so it's very
> easy for an operator to get the configured value wrong by many orders of
> magnitude.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25910) Fix TestClusterPortAssignment.testClusterPortAssignment test and re-enable it.
[ https://issues.apache.org/jira/browse/HBASE-25910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Viraj Jasani resolved HBASE-25910. -- Fix Version/s: 2.4.4 2.3.6 2.5.0 Hadoop Flags: Reviewed Resolution: Fixed Thanks for the contribution [~vli02]. > Fix TestClusterPortAssignment.testClusterPortAssignment test and re-enable it. > -- > > Key: HBASE-25910 > URL: https://issues.apache.org/jira/browse/HBASE-25910 > Project: HBase > Issue Type: Test > Components: flakies, test >Reporter: Victor Li >Assignee: Victor Li >Priority: Minor > Fix For: 3.0.0-alpha-1, 2.5.0, 2.3.6, 2.4.4 > > Original Estimate: 24h > Remaining Estimate: 24h > > This test assigned with random unused ports for master and region server > ports and start a cluster, however it can fail as the previously unsigned > port might be taken by other application at the moment when cluster start to > bind the ports. > Server has an option to allow auto assign an port if the port is unavailable. > This option needs to be disabled in this test in order to test explicit port > assignment. > The exception for the failure of starting server needs to change to > IOException and there is a need to set a retry count instead of infinite > retry. > Related ticket: https://issues.apache.org/jira/browse/HBASE-24342 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25939) Move more tests code for StochasticLoadBalancer to hbase-balancer module
Duo Zhang created HBASE-25939: - Summary: Move more tests code for StochasticLoadBalancer to hbase-balancer module Key: HBASE-25939 URL: https://issues.apache.org/jira/browse/HBASE-25939 Project: HBase Issue Type: Sub-task Components: Balancer, test Reporter: Duo Zhang Assignee: Duo Zhang Fix For: 3.0.0-alpha-1 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25861) Correct the usage of Configuration#addDeprecation
[ https://issues.apache.org/jira/browse/HBASE-25861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25861. --- Hadoop Flags: Reviewed Resolution: Fixed Re-resolving. Issue addressed over in HBASE-25928 > Correct the usage of Configuration#addDeprecation > - > > Key: HBASE-25861 > URL: https://issues.apache.org/jira/browse/HBASE-25861 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.5.0 >Reporter: Baiqiang Zhao >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > > When I was solving HBASE-25745 > ([PR3139|https://github.com/apache/hbase/pull/3139]), I found that our use of > Configuration#addDeprecation API was wrong. > > At present, we will call Configuration#addDeprecation in the static block for > the deprecated configuration. But after testing, it is found that this does > not complete backward compatibility. When user upgrades HBase and does not > change the deprecated configuration to the new configuration, he will find > that the deprecated configuration does not effect, which may not be > consistent with expectations. The specific test results can be seen in the PR > above, and we can found the calling order of Configuration#addDeprecation is > very important. > > Configuration#addDeprecation is a Hadoop API, looking through the Hadoop > source code, we will find that before creating the Configuration object, the > addDeprecatedKeys() method will be called first: > [https://github.com/apache/hadoop/blob/b93e448f9aa66689f1ce5059f6cdce8add130457/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/HdfsConfiguration.java#L34] > . -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25928) TestHBaseConfiguration#testDeprecatedConfigurations is broken with Hadoop 3.3
[ https://issues.apache.org/jira/browse/HBASE-25928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Stack resolved HBASE-25928. --- Fix Version/s: 2.5.0 3.0.0-alpha-1 Hadoop Flags: Reviewed Resolution: Fixed Pushed to branch-2 and master. Thanks for finding the issue [~weichiu] and thanks for the fix [~DeanZ] > TestHBaseConfiguration#testDeprecatedConfigurations is broken with Hadoop 3.3 > - > > Key: HBASE-25928 > URL: https://issues.apache.org/jira/browse/HBASE-25928 > Project: HBase > Issue Type: Bug >Affects Versions: 3.0.0-alpha-1, 2.5.0 >Reporter: Wei-Chiu Chuang >Assignee: Baiqiang Zhao >Priority: Major > Fix For: 3.0.0-alpha-1, 2.5.0 > > > The test TestHBaseConfiguration#testDeprecatedConfigurations was added > recently by HBASE-25861 to address the usage of Hadoop Configuration > addDeprecations API. > However, the API's behavior was changed to fix a bug. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25938) The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one
Duo Zhang created HBASE-25938: - Summary: The SnapshotOfRegionAssignmentFromMeta.initialize call in FavoredNodeLoadBalancer is just a dummy one Key: HBASE-25938 URL: https://issues.apache.org/jira/browse/HBASE-25938 Project: HBase Issue Type: Bug Components: Balancer, FavoredNodes Reporter: Duo Zhang After we introduced FavoredNodesManager, we do not need to load this information every time when balancing, as all the updates to favored nodes will go to FavoredNodesManager and it can update its in memory state accordingly. The current SnapshotOfRegionAssignmentFromMeta.initialize is just a dummy one in FavoredNodeLoadBalancer, as we never use it in the method. It will introduce a full scan on hbase:meta so let's just remove it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25937) Clarify UnknownRegionException
David Mollitor created HBASE-25937: -- Summary: Clarify UnknownRegionException Key: HBASE-25937 URL: https://issues.apache.org/jira/browse/HBASE-25937 Project: HBase Issue Type: Improvement Reporter: David Mollitor Assignee: David Mollitor UnknownRegionException seems to accept a "region name" but it's actually a normal Exception message (and is used that way). Fix this to be "message" and add a "cause" capability as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (HBASE-25936) Improve Logging Statement Formatting
David Mollitor created HBASE-25936:
--
Summary: Improve Logging Statement Formatting
Key: HBASE-25936
URL: https://issues.apache.org/jira/browse/HBASE-25936
Project: HBase
Issue Type: Improvement
Reporter: David Mollitor
Assignee: David Mollitor
I reviewed most of the TRACE/DEBUG log statements and wanted to clean up the
first tier issues:
* Use logging anchors {{{}}} where previously was concatenating (without
logging guard)
* Remove String.format and use logging anchors {{{}}}
* Remove logging guards around statements which use anchors (kind of defeats
the purpose of anchors)
* Remove logging guards around log statements which have no parameters (not
concatenation)
* Remove explicit calls to toString() that defeat the purpose of anchors
* Remove Objects.toString() - logging framework will handle {{null}} values
* Remove StringUtils ExceptionStackTrace logging - logging framework will
handle pretty-print of Exception messages and stack traces
* Remove logging guards around INFO messages. They will just about always be
printed in production systems, no need to try to protect them from happening
The goal here is, maybe a tiny bump in performance, but to reduce the size of
the code and to make logging statement more clear to developers.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-25935) Data is not getting replicated to secondary replicas after altering a table to enable region replication (async Wal replication)
chiranjeevi created HBASE-25935:
---
Summary: Data is not getting replicated to secondary replicas
after altering a table to enable region replication (async Wal replication)
Key: HBASE-25935
URL: https://issues.apache.org/jira/browse/HBASE-25935
Project: HBase
Issue Type: Bug
Components: read replicas
Reporter: chiranjeevi
Data is not getting replicated to secondary replicas after altering a table to
enable region replication (async Wal replication)
*Test Steps:*
1)Create table with one replica
create 'table1','cf'
2)Alter table with region replica=3
alter 'table1',\{REGION_REPLICATION => '3'}
3)write data to table1
*Observation:*
1)Data is not getting replicated to secondary replicas this is happening when
alter table after the peer is created for region replication.
2)In table UI page also you can observe write requests showing for secondary
replicas=0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25718) Backport HBASE-25705 to branch-2
[ https://issues.apache.org/jira/browse/HBASE-25718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaolin Ha resolved HBASE-25718. Fix Version/s: 2.5.0 Resolution: Fixed Pushed to branch-2. Thanks [~mokai87] for contributing, and thanks [~ndimiduk] for reviewing. > Backport HBASE-25705 to branch-2 > > > Key: HBASE-25718 > URL: https://issues.apache.org/jira/browse/HBASE-25718 > Project: HBase > Issue Type: Sub-task > Components: rsgroup >Affects Versions: 2.4.2 >Reporter: mokai >Assignee: mokai >Priority: Minor > Fix For: 2.5.0 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-25651) NORMALIZER_TARGET_REGION_SIZE needs a unit in its name
[
https://issues.apache.org/jira/browse/HBASE-25651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiaolin Ha resolved HBASE-25651.
Fix Version/s: 3.0.0-alpha-1
Resolution: Fixed
Pushed to master.
Thanks [~rkrahul324] for contributing,
and thanks [~ndimiduk] [~DeanZ] for reviewing.
> NORMALIZER_TARGET_REGION_SIZE needs a unit in its name
> --
>
> Key: HBASE-25651
> URL: https://issues.apache.org/jira/browse/HBASE-25651
> Project: HBase
> Issue Type: Bug
> Components: Normalizer
>Affects Versions: 2.3.0
>Reporter: Nick Dimiduk
>Assignee: Rahul Kumar
>Priority: Major
> Fix For: 3.0.0-alpha-1
>
>
> On a per-table basis, the normalizer can be configured with a specific region
> size target via {{NORMALIZER_TARGET_REGION_SIZE}}. This value is used in a
> context where size values are in the megabyte scale. However, this
> configuration name does not mention "megabyte" or "mb" anywhere, so it's very
> easy for an operator to get the configured value wrong by many orders of
> magnitude.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25931) Move FavoredNodeManager to hbase-balancer module
[ https://issues.apache.org/jira/browse/HBASE-25931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-25931. --- Hadoop Flags: Reviewed Resolution: Fixed Merged to master. Thanks [~niuyulin] and [~stack] for reviewing. > Move FavoredNodeManager to hbase-balancer module > > > Key: HBASE-25931 > URL: https://issues.apache.org/jira/browse/HBASE-25931 > Project: HBase > Issue Type: Sub-task > Components: Balancer, FavoredNodes >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-1 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
