[jira] [Commented] (HBASE-25307) ThreadLocal pooling leads to NullPointerException
[
https://issues.apache.org/jira/browse/HBASE-25307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17240718#comment-17240718
]
Balazs Meszaros commented on HBASE-25307:
-
bq. This is not easy, it relates to the logic for the cleanup of the
connections, where we need to set and check the last touch time along with the
modification to the PoolMap.
Yes, we would need a functionality like {{close()}} and {{drainTo()}} which
ensure no new connection can be created and every existing connection is closed
properly.
I am not against further refactoring, the previous implementation wasn't very
good.
> ThreadLocal pooling leads to NullPointerException
> -
>
> Key: HBASE-25307
> URL: https://issues.apache.org/jira/browse/HBASE-25307
> Project: HBase
> Issue Type: Bug
> Components: Client
>Affects Versions: 3.0.0-alpha-1
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
> Fix For: 3.0.0-alpha-1, 2.4.0, 2.2.7, 2.3.4
>
>
> We got NPE after setting {{hbase.client.ipc.pool.type}} to {{thread-local}}:
> {noformat}
> 20/11/18 01:53:04 ERROR yarn.ApplicationMaster: User class threw exception:
> java.lang.NullPointerException
> java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.close(AbstractRpcClient.java:496)
> at
> org.apache.hadoop.hbase.client.ConnectionImplementation.close(ConnectionImplementation.java:1944)
> at
> org.apache.hadoop.hbase.mapreduce.TableInputFormatBase.close(TableInputFormatBase.java:660)
> {noformat}
> The root cause of the issue is probably at
> {{PoolMap.ThreadLocalPool.values()}}:
> {code:java}
> public Collection values() {
> List values = new ArrayList<>();
> values.add(get());
> return values;
> }
> {code}
> It adds {{null}} into the collection if the current thread does not have any
> resources which leads to NPE later.
> I traced the usages of values() and it should return every resource, not just
> that one which is attached to the caller thread.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-25236) [hbase-connectors] Run package phase on spark modules
[ https://issues.apache.org/jira/browse/HBASE-25236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros resolved HBASE-25236. - Fix Version/s: connector-1.0.1 Resolution: Fixed > [hbase-connectors] Run package phase on spark modules > - > > Key: HBASE-25236 > URL: https://issues.apache.org/jira/browse/HBASE-25236 > Project: HBase > Issue Type: Bug > Components: hbase-connectors >Affects Versions: connector-1.0.1 >Reporter: Peter Somogyi >Assignee: Peter Somogyi >Priority: Major > Fix For: connector-1.0.1 > > > The precommit job fails if a change is made in the spark module because the > protobuf generator plugin runs in package phase. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-23592) Refactor tests in hbase-kafka-proxy in hbase-connectors
[
https://issues.apache.org/jira/browse/HBASE-23592?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros resolved HBASE-23592.
-
Fix Version/s: connector-1.0.1
Resolution: Fixed
> Refactor tests in hbase-kafka-proxy in hbase-connectors
> ---
>
> Key: HBASE-23592
> URL: https://issues.apache.org/jira/browse/HBASE-23592
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Trivial
> Fix For: connector-1.0.1
>
>
> The tests in {{hbase-kafka-proxy}} within {{hbase-connectors}} should be
> refactored to
> * move the usage of the character set to {{StandardCharsets}}
> * remove printing the stacktrace
> * simplification of the asserts
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (HBASE-23608) Remove redundant groupId from spark module in hbase-connectors
[
https://issues.apache.org/jira/browse/HBASE-23608?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-23608:
Fix Version/s: connector-1.0.1
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Remove redundant groupId from spark module in hbase-connectors
> --
>
> Key: HBASE-23608
> URL: https://issues.apache.org/jira/browse/HBASE-23608
> Project: HBase
> Issue Type: Improvement
>Reporter: Jan Hentschel
>Assignee: Jan Hentschel
>Priority: Trivial
> Fix For: connector-1.0.1
>
>
> The {{groupId}} declaration in the {{spark}} module within
> {{hbase-connectors}} is redundant and should be removed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-24230) Support user-defined version timestamp when bulk load data
[ https://issues.apache.org/jira/browse/HBASE-24230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros resolved HBASE-24230. - Fix Version/s: connector-1.0.1 Resolution: Fixed > Support user-defined version timestamp when bulk load data > -- > > Key: HBASE-24230 > URL: https://issues.apache.org/jira/browse/HBASE-24230 > Project: HBase > Issue Type: Improvement > Components: hbase-connectors >Affects Versions: 1.0.0 >Reporter: Xiao Zhang >Assignee: Xiao Zhang >Priority: Minor > Fix For: connector-1.0.1 > > > In hbase-connectors-1.0.0, loading data in bulk, only the current system time > can be used as the KeyValue version timestamp. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-26789) Automatically add default security headers to http/rest if SSL enabled
[ https://issues.apache.org/jira/browse/HBASE-26789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros resolved HBASE-26789. - Resolution: Fixed > Automatically add default security headers to http/rest if SSL enabled > -- > > Key: HBASE-26789 > URL: https://issues.apache.org/jira/browse/HBASE-26789 > Project: HBase > Issue Type: Improvement > Components: REST, UI >Affects Versions: 2.0.6, 2.1.10, 2.2.7, 3.0.0-alpha-2 >Reporter: Andor Molnar >Assignee: Andor Molnar >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.11 > > > In the previous ticket https://issues.apache.org/jira/browse/HBASE-23303 we > implemented these security headers as optional which had to explicitly > enabled in the config. > With this change the headers will automatically be added with meaningful > default values if SSL is enabled. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-26789) Automatically add default security headers to http/rest if SSL enabled
[ https://issues.apache.org/jira/browse/HBASE-26789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-26789: Fix Version/s: 2.5.0 3.0.0-alpha-3 2.4.11 > Automatically add default security headers to http/rest if SSL enabled > -- > > Key: HBASE-26789 > URL: https://issues.apache.org/jira/browse/HBASE-26789 > Project: HBase > Issue Type: Improvement > Components: REST, UI >Affects Versions: 2.0.6, 2.1.10, 2.2.7, 3.0.0-alpha-2 >Reporter: Andor Molnar >Assignee: Andor Molnar >Priority: Major > Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.11 > > > In the previous ticket https://issues.apache.org/jira/browse/HBASE-23303 we > implemented these security headers as optional which had to explicitly > enabled in the config. > With this change the headers will automatically be added with meaningful > default values if SSL is enabled. -- This message was sent by Atlassian Jira (v8.20.1#820001)
[jira] [Updated] (HBASE-28345) Close HBase connection on exit from HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-28345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-28345: Fix Version/s: 3.0.0 > Close HBase connection on exit from HBase Shell > --- > > Key: HBASE-28345 > URL: https://issues.apache.org/jira/browse/HBASE-28345 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.17 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 2.4.18, 3.0.0, 2.5.8, 3.0.0-beta-2 > > > When using Netty for the ZK client, hbase shell hangs on exit. > This is caused by the non-deamon Netty threads that ZK creates. > Wheter ZK should create daemon threads for Netty or not is debatable, but > explicitly closing the connection in hbase shell on exit fixes the issue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28345) Close HBase connection on exit from HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-28345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros resolved HBASE-28345. - Resolution: Fixed [~bbeaudreault] Thanks, fixed. > Close HBase connection on exit from HBase Shell > --- > > Key: HBASE-28345 > URL: https://issues.apache.org/jira/browse/HBASE-28345 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.17 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 2.4.18, 3.0.0, 2.5.8, 3.0.0-beta-2 > > > When using Netty for the ZK client, hbase shell hangs on exit. > This is caused by the non-deamon Netty threads that ZK creates. > Wheter ZK should create daemon threads for Netty or not is debatable, but > explicitly closing the connection in hbase shell on exit fixes the issue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28254) Flaky test: TestTableShell
[
https://issues.apache.org/jira/browse/HBASE-28254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28254:
Fix Version/s: 3.0.0-beta-2
(was: 3.0.0-beta-1)
> Flaky test: TestTableShell
> --
>
> Key: HBASE-28254
> URL: https://issues.apache.org/jira/browse/HBASE-28254
> Project: HBase
> Issue Type: Test
> Components: flakies, integration tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 2.5.8, 3.0.0-beta-2
>
>
> The test is running the following Ruby commands:
> {noformat}
> # Instert test data
> @test_table.put(1, "x:a", 1)
> @test_table.put(2, "x:raw1", 11)
> @test_table.put(2, "x:raw1", 11)
> @test_table.put(2, "x:raw1", 11)
> @test_table.put(2, "x:raw1", 11)
> {noformat}
> And validate the versions with:
> {noformat}
> args = { VERSIONS => 10, RAW => true } # Since 4 versions of row with rowkey
> 2 is been added, we can use any number >= 4 for VERSIONS to scan all 4
> versions.
> num_rows = 0
> @test_table._scan_internal(args) do # Raw Scan
> num_rows += 1
> end
> # 5 since , 1 from row key '1' and other 4 from row key '4'
> assert_equal(num_rows, 5,
> 'Num rows scanned without RAW/VERSIONS are not 5')
> {noformat}
> Which sometimes (almost always on fast machines) fails, because it only finds
> 3 versions out of 4. I believe this due to commands are running too fast and
> inserts data with same timestamp, so HBase cannot distinguish them. I'd like
> to add some sleep between puts to fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28254) Flaky test: TestTableShell
[
https://issues.apache.org/jira/browse/HBASE-28254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28254:
Fix Version/s: 4.0.0-alpha-1
> Flaky test: TestTableShell
> --
>
> Key: HBASE-28254
> URL: https://issues.apache.org/jira/browse/HBASE-28254
> Project: HBase
> Issue Type: Test
> Components: flakies, integration tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 4.0.0-alpha-1, 2.5.8, 3.0.0-beta-2
>
>
> The test is running the following Ruby commands:
> {noformat}
> # Instert test data
> @test_table.put(1, "x:a", 1)
> @test_table.put(2, "x:raw1", 11)
> @test_table.put(2, "x:raw1", 11)
> @test_table.put(2, "x:raw1", 11)
> @test_table.put(2, "x:raw1", 11)
> {noformat}
> And validate the versions with:
> {noformat}
> args = { VERSIONS => 10, RAW => true } # Since 4 versions of row with rowkey
> 2 is been added, we can use any number >= 4 for VERSIONS to scan all 4
> versions.
> num_rows = 0
> @test_table._scan_internal(args) do # Raw Scan
> num_rows += 1
> end
> # 5 since , 1 from row key '1' and other 4 from row key '4'
> assert_equal(num_rows, 5,
> 'Num rows scanned without RAW/VERSIONS are not 5')
> {noformat}
> Which sometimes (almost always on fast machines) fails, because it only finds
> 3 versions out of 4. I believe this due to commands are running too fast and
> inserts data with same timestamp, so HBase cannot distinguish them. I'd like
> to add some sleep between puts to fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28345) Close HBase connection on exit from HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-28345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-28345: Fix Version/s: 4.0.0-alpha-1 (was: 3.0.0) > Close HBase connection on exit from HBase Shell > --- > > Key: HBASE-28345 > URL: https://issues.apache.org/jira/browse/HBASE-28345 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.17 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 2.4.18, 4.0.0-alpha-1, 2.5.8, 3.0.0-beta-2 > > > When using Netty for the ZK client, hbase shell hangs on exit. > This is caused by the non-deamon Netty threads that ZK creates. > Wheter ZK should create daemon threads for Netty or not is debatable, but > explicitly closing the connection in hbase shell on exit fixes the issue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28274) Flaky test: TestFanOutOneBlockAsyncDFSOutput (Part 2)
[
https://issues.apache.org/jira/browse/HBASE-28274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28274:
Fix Version/s: 4.0.0-alpha-1
3.0.0-beta-2
(was: 3.0.0-beta-1)
> Flaky test: TestFanOutOneBlockAsyncDFSOutput (Part 2)
> -
>
> Key: HBASE-28274
> URL: https://issues.apache.org/jira/browse/HBASE-28274
> Project: HBase
> Issue Type: Test
> Components: flakies, integration tests, test
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 4.0.0-alpha-1, 2.5.8, 3.0.0-beta-2
>
>
> The following test sometimes fails for me when running locally with Maven:
> TestFanOutOneBlockAsyncDFSOutput.testRecover()
> I can't really figure out the reason, but it's probably a side effect of the
> preceding test: testConnectToDatanodeFailed(). This test also restarts one of
> the datanodes in the MiniDFS cluster just like testRecover() and it somehow
> causes the failure.
> {noformat}
> java.lang.AssertionError: flush should fail
> at org.junit.Assert.fail(Assert.java:89)
> at
> org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput.testRecover(TestFanOutOneBlockAsyncDFSOutput.java:154)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method){noformat}
> The flush() call is testRecover() should fail, because we restart one of the
> DN in the DFS cluster which is expected to brake connection. It succeeds
> though if the preceding test already restarted a DN. No matter which DN we
> restart, even if they're different, the error occurs.
> I also tried to add CLUSTER.waitDatanodeFullyStarted() at the end of
> testConnectToDatanodeFailed(), looks like it made the tests slightly more
> stable, but didn't help fully.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28353) Close HBase connection on implicit exit from HBase shell
[ https://issues.apache.org/jira/browse/HBASE-28353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-28353: Fix Version/s: 2.6.0 2.4.18 4.0.0-alpha-1 2.5.8 3.0.0-beta-2 Resolution: Fixed Status: Resolved (was: Patch Available) > Close HBase connection on implicit exit from HBase shell > > > Key: HBASE-28353 > URL: https://issues.apache.org/jira/browse/HBASE-28353 > Project: HBase > Issue Type: Bug > Components: shell >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 2.4.18, 4.0.0-alpha-1, 2.5.8, 3.0.0-beta-2 > > > The fix in HBASE-28345 only works when the exit function is explicitly called. > It does not work when scripts are piped in in non-interactive mode. > Hook the connection close into Ruby at_exit instead of the exit shell command. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28340) Add trust/key store type to ZK TLS settings handled by HBase
[
https://issues.apache.org/jira/browse/HBASE-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28340:
Fix Version/s: 4.0.0-alpha-1
3.0.0-beta-2
> Add trust/key store type to ZK TLS settings handled by HBase
>
>
> Key: HBASE-28340
> URL: https://issues.apache.org/jira/browse/HBASE-28340
> Project: HBase
> Issue Type: Sub-task
> Components: Zookeeper
>Affects Versions: 2.4.17, 3.0.0-beta-1, 2.5.7
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0-alpha-1, 3.0.0-beta-2
>
>
> Let's add the following settings as well. Last time we missed it.
> {noformat}
> zookeeper.ssl.keyStore.type
> zookeeper.ssl.trustStore.type{noformat}
> Handle them in hbase-site.xml as:
> {noformat}
> hbase.zookeeper.property.ssl.keyStore.type
> hbase.zookeeper.property.ssl.trustStore.type{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28340) Add trust/key store type to ZK TLS settings handled by HBase
[
https://issues.apache.org/jira/browse/HBASE-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28340:
Fix Version/s: 2.6.0
2.5.8
> Add trust/key store type to ZK TLS settings handled by HBase
>
>
> Key: HBASE-28340
> URL: https://issues.apache.org/jira/browse/HBASE-28340
> Project: HBase
> Issue Type: Sub-task
> Components: Zookeeper
>Affects Versions: 2.4.17, 3.0.0-beta-1, 2.5.7
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.6.0, 4.0.0-alpha-1, 2.5.8, 3.0.0-beta-2
>
>
> Let's add the following settings as well. Last time we missed it.
> {noformat}
> zookeeper.ssl.keyStore.type
> zookeeper.ssl.trustStore.type{noformat}
> Handle them in hbase-site.xml as:
> {noformat}
> hbase.zookeeper.property.ssl.keyStore.type
> hbase.zookeeper.property.ssl.trustStore.type{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28340) Add trust/key store type to ZK TLS settings handled by HBase
[
https://issues.apache.org/jira/browse/HBASE-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28340:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add trust/key store type to ZK TLS settings handled by HBase
>
>
> Key: HBASE-28340
> URL: https://issues.apache.org/jira/browse/HBASE-28340
> Project: HBase
> Issue Type: Sub-task
> Components: Zookeeper
>Affects Versions: 2.4.17, 3.0.0-beta-1, 2.5.7
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.6.0, 4.0.0-alpha-1, 2.5.8, 3.0.0-beta-2
>
>
> Let's add the following settings as well. Last time we missed it.
> {noformat}
> zookeeper.ssl.keyStore.type
> zookeeper.ssl.trustStore.type{noformat}
> Handle them in hbase-site.xml as:
> {noformat}
> hbase.zookeeper.property.ssl.keyStore.type
> hbase.zookeeper.property.ssl.trustStore.type{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28340) Add trust/key store type to ZK TLS settings handled by HBase
[
https://issues.apache.org/jira/browse/HBASE-28340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28340:
Fix Version/s: 2.4.18
> Add trust/key store type to ZK TLS settings handled by HBase
>
>
> Key: HBASE-28340
> URL: https://issues.apache.org/jira/browse/HBASE-28340
> Project: HBase
> Issue Type: Sub-task
> Components: Zookeeper
>Affects Versions: 2.4.17, 3.0.0-beta-1, 2.5.7
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Labels: pull-request-available
> Fix For: 2.6.0, 2.4.18, 4.0.0-alpha-1, 2.5.8, 3.0.0-beta-2
>
>
> Let's add the following settings as well. Last time we missed it.
> {noformat}
> zookeeper.ssl.keyStore.type
> zookeeper.ssl.trustStore.type{noformat}
> Handle them in hbase-site.xml as:
> {noformat}
> hbase.zookeeper.property.ssl.keyStore.type
> hbase.zookeeper.property.ssl.trustStore.type{noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-28152) Replace scala.util.parsing.json with org.json4s.jackson which used in Spark too
[ https://issues.apache.org/jira/browse/HBASE-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros reassigned HBASE-28152: --- Assignee: Attila Zsolt Piros > Replace scala.util.parsing.json with org.json4s.jackson which used in Spark > too > --- > > Key: HBASE-28152 > URL: https://issues.apache.org/jira/browse/HBASE-28152 > Project: HBase > Issue Type: New Feature > Components: spark >Affects Versions: connector-1.0.0 >Reporter: Attila Zsolt Piros >Assignee: Attila Zsolt Piros >Priority: Major > > In https://issues.apache.org/jira/browse/HBASE-28137 to support Spark 3.4 the > scala-parser-combinators was added as direct dependency to HBase Spark > Connector. > This was needed as Spark 3.4 is not using scala-parser-combinators and it is > not inherited as transitive dependency. > But this solution has a disadvantage. As the HBase Spark Connector assembly > jar does not include any 3rd party libraries the scala-parser-combinators > must be added to the spark classpath for HBase Spark Connector to work. > A much better solution is to replace scala.util.parsing.json with > org.json4s.jackson which is used by Spark core, see > https://github.com/apache/spark/blob/branch-3.4/core/pom.xml#L279-L280. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28152) Replace scala.util.parsing.json with org.json4s.jackson which used in Spark too
[ https://issues.apache.org/jira/browse/HBASE-28152?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros resolved HBASE-28152. - Fix Version/s: hbase-connectors-1.0.1 Resolution: Fixed > Replace scala.util.parsing.json with org.json4s.jackson which used in Spark > too > --- > > Key: HBASE-28152 > URL: https://issues.apache.org/jira/browse/HBASE-28152 > Project: HBase > Issue Type: New Feature > Components: spark >Affects Versions: connector-1.0.0 >Reporter: Attila Zsolt Piros >Assignee: Attila Zsolt Piros >Priority: Major > Fix For: hbase-connectors-1.0.1 > > > In https://issues.apache.org/jira/browse/HBASE-28137 to support Spark 3.4 the > scala-parser-combinators was added as direct dependency to HBase Spark > Connector. > This was needed as Spark 3.4 is not using scala-parser-combinators and it is > not inherited as transitive dependency. > But this solution has a disadvantage. As the HBase Spark Connector assembly > jar does not include any 3rd party libraries the scala-parser-combinators > must be added to the spark classpath for HBase Spark Connector to work. > A much better solution is to replace scala.util.parsing.json with > org.json4s.jackson which is used by Spark core, see > https://github.com/apache/spark/blob/branch-3.4/core/pom.xml#L279-L280. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28247) Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM test flags
[
https://issues.apache.org/jira/browse/HBASE-28247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28247:
Fix Version/s: 2.6.0
3.0.0-beta-1
2.5.7
> Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM
> test flags
>
>
> Key: HBASE-28247
> URL: https://issues.apache.org/jira/browse/HBASE-28247
> Project: HBase
> Issue Type: Bug
> Components: java
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-beta-1, 2.5.7
>
>
> While testing with JDK17 we have found that we need to add
> {noformat}
> --add-exports java.base/sun.net.dns=ALL-UNNAMED
> --add-exports java.base/sun.net.util=ALL-UNNAMED
> {noformat}
> on top of what is already defined in _hbase-surefire.jdk11.flags_ , otherwise
> RS and Master startup fails in the Hadoop security code.
> While this does not affect the test suite (at least not the commonly run
> tests), I consider hbase-surefire.jdk11.flags to be an unoffical resource to
> getting HBase to run on newer JDK versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28247) Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM test flags
[
https://issues.apache.org/jira/browse/HBASE-28247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28247:
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM
> test flags
>
>
> Key: HBASE-28247
> URL: https://issues.apache.org/jira/browse/HBASE-28247
> Project: HBase
> Issue Type: Bug
> Components: java
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-beta-1, 2.5.7
>
>
> While testing with JDK17 we have found that we need to add
> {noformat}
> --add-exports java.base/sun.net.dns=ALL-UNNAMED
> --add-exports java.base/sun.net.util=ALL-UNNAMED
> {noformat}
> on top of what is already defined in _hbase-surefire.jdk11.flags_ , otherwise
> RS and Master startup fails in the Hadoop security code.
> While this does not affect the test suite (at least not the commonly run
> tests), I consider hbase-surefire.jdk11.flags to be an unoffical resource to
> getting HBase to run on newer JDK versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28252) Add sun.net.dns and sun.net.util to the JDK11+ module exports in the hbase script
[ https://issues.apache.org/jira/browse/HBASE-28252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-28252: Fix Version/s: 2.6.0 3.0.0-beta-1 2.5.7 > Add sun.net.dns and sun.net.util to the JDK11+ module exports in the hbase > script > - > > Key: HBASE-28252 > URL: https://issues.apache.org/jira/browse/HBASE-28252 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 2.6.0, 3.0.0-beta-1, 2.5.7 > > > As noted in HBASE-28247, HBase can run into module permission issues that are > not handled in the current JDK11 options in the hbase startup script. > The surefire test config also includes some JDK17 specific options, we should > also add those as needed. > We are not yet aware of any additional JVM options required by Java 21. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28247) Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM test flags
[
https://issues.apache.org/jira/browse/HBASE-28247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28247:
Fix Version/s: (was: 2.6.0)
(was: 3.0.0-beta-1)
(was: 2.5.7)
> Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM
> test flags
>
>
> Key: HBASE-28247
> URL: https://issues.apache.org/jira/browse/HBASE-28247
> Project: HBase
> Issue Type: Bug
> Components: java
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
>
> While testing with JDK17 we have found that we need to add
> {noformat}
> --add-exports java.base/sun.net.dns=ALL-UNNAMED
> --add-exports java.base/sun.net.util=ALL-UNNAMED
> {noformat}
> on top of what is already defined in _hbase-surefire.jdk11.flags_ , otherwise
> RS and Master startup fails in the Hadoop security code.
> While this does not affect the test suite (at least not the commonly run
> tests), I consider hbase-surefire.jdk11.flags to be an unoffical resource to
> getting HBase to run on newer JDK versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Reopened] (HBASE-28247) Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM test flags
[
https://issues.apache.org/jira/browse/HBASE-28247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros reopened HBASE-28247:
-
> Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM
> test flags
>
>
> Key: HBASE-28247
> URL: https://issues.apache.org/jira/browse/HBASE-28247
> Project: HBase
> Issue Type: Bug
> Components: java
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
> Fix For: 2.6.0, 3.0.0-beta-1, 2.5.7
>
>
> While testing with JDK17 we have found that we need to add
> {noformat}
> --add-exports java.base/sun.net.dns=ALL-UNNAMED
> --add-exports java.base/sun.net.util=ALL-UNNAMED
> {noformat}
> on top of what is already defined in _hbase-surefire.jdk11.flags_ , otherwise
> RS and Master startup fails in the Hadoop security code.
> While this does not affect the test suite (at least not the commonly run
> tests), I consider hbase-surefire.jdk11.flags to be an unoffical resource to
> getting HBase to run on newer JDK versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-25058) Export necessary modules when running under JDK11
[
https://issues.apache.org/jira/browse/HBASE-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-25058:
Fix Version/s: 2.4.18
> Export necessary modules when running under JDK11
> -
>
> Key: HBASE-25058
> URL: https://issues.apache.org/jira/browse/HBASE-25058
> Project: HBase
> Issue Type: Sub-task
> Components: Performance, rpc
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.4.0
>Reporter: Nick Dimiduk
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.18
>
>
> Running on JDK11, I noticed this in the debug logging. It looks like netty is
> using {{sun.misc.Unsafe}} instead of the new location on jdk11.
> {noformat}
> 2020-09-17 00:21:53,318 DEBUG
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent0:
> jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable
> java.lang.IllegalAccessException: class
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent0$6
> cannot access class jdk.internal.misc.Unsafe (in module java.base) because
> module java.base does not export jdk.internal.misc to unnamed module @78461bc4
> at
> java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:361)
> at
> java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:591)
> at java.base/java.lang.reflect.Method.invoke(Method.java:558)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent0$6.run(PlatformDependent0.java:335)
> at java.base/java.security.AccessController.doPrivileged(Native
> Method)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent0.(PlatformDependent0.java:326)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:289)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent.(PlatformDependent.java:92)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:228)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.(Native.java:58)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.Epoll.(Epoll.java:39)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.(EpollEventLoop.java:51)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoopGroup.newChild(EpollEventLoopGroup.java:150)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoopGroup.newChild(EpollEventLoopGroup.java:35)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.MultithreadEventExecutorGroup.(MultithreadEventExecutorGroup.java:84)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.MultithreadEventExecutorGroup.(MultithreadEventExecutorGroup.java:58)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.MultithreadEventExecutorGroup.(MultithreadEventExecutorGroup.java:47)
> at
> org.apache.hbase.thirdparty.io.netty.channel.MultithreadEventLoopGroup.(MultithreadEventLoopGroup.java:59)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:112)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:99)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:76)
> at
> org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.(NettyEventLoopGroupConfig.java:58)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupNetty(HRegionServer.java:713)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:569)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:524)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:3054)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:236)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
> at
> org.apache.hadoop.hbase.util.ServerComman
[jira] [Commented] (HBASE-25058) Export necessary modules when running under JDK11
[
https://issues.apache.org/jira/browse/HBASE-25058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17795721#comment-17795721
]
Balazs Meszaros commented on HBASE-25058:
-
[~mrzhao]: [~stoty] did the backport and it is landed in branch-2.4.
> Export necessary modules when running under JDK11
> -
>
> Key: HBASE-25058
> URL: https://issues.apache.org/jira/browse/HBASE-25058
> Project: HBase
> Issue Type: Sub-task
> Components: Performance, rpc
>Affects Versions: 3.0.0-alpha-1, 2.3.0, 2.4.0
>Reporter: Nick Dimiduk
>Assignee: Duo Zhang
>Priority: Major
> Fix For: 2.5.0, 3.0.0-alpha-3, 2.4.18
>
>
> Running on JDK11, I noticed this in the debug logging. It looks like netty is
> using {{sun.misc.Unsafe}} instead of the new location on jdk11.
> {noformat}
> 2020-09-17 00:21:53,318 DEBUG
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent0:
> jdk.internal.misc.Unsafe.allocateUninitializedArray(int): unavailable
> java.lang.IllegalAccessException: class
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent0$6
> cannot access class jdk.internal.misc.Unsafe (in module java.base) because
> module java.base does not export jdk.internal.misc to unnamed module @78461bc4
> at
> java.base/jdk.internal.reflect.Reflection.newIllegalAccessException(Reflection.java:361)
> at
> java.base/java.lang.reflect.AccessibleObject.checkAccess(AccessibleObject.java:591)
> at java.base/java.lang.reflect.Method.invoke(Method.java:558)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent0$6.run(PlatformDependent0.java:335)
> at java.base/java.security.AccessController.doPrivileged(Native
> Method)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent0.(PlatformDependent0.java:326)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent.isAndroid(PlatformDependent.java:289)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.PlatformDependent.(PlatformDependent.java:92)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.loadNativeLibrary(Native.java:228)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.Native.(Native.java:58)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.Epoll.(Epoll.java:39)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.(EpollEventLoop.java:51)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoopGroup.newChild(EpollEventLoopGroup.java:150)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoopGroup.newChild(EpollEventLoopGroup.java:35)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.MultithreadEventExecutorGroup.(MultithreadEventExecutorGroup.java:84)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.MultithreadEventExecutorGroup.(MultithreadEventExecutorGroup.java:58)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.MultithreadEventExecutorGroup.(MultithreadEventExecutorGroup.java:47)
> at
> org.apache.hbase.thirdparty.io.netty.channel.MultithreadEventLoopGroup.(MultithreadEventLoopGroup.java:59)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:112)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:99)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoopGroup.(EpollEventLoopGroup.java:76)
> at
> org.apache.hadoop.hbase.util.NettyEventLoopGroupConfig.(NettyEventLoopGroupConfig.java:58)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.setupNetty(HRegionServer.java:713)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.(HRegionServer.java:569)
> at org.apache.hadoop.hbase.master.HMaster.(HMaster.java:524)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at
> java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490)
> at
> org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:3054)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:236)
> at
> org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:140)
> at org.apache.had
[jira] [Updated] (HBASE-27510) Should use 'org.apache.hbase.thirdparty.io.netty.tryReflectionSetAccessible'
[ https://issues.apache.org/jira/browse/HBASE-27510?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-27510: Fix Version/s: 2.4.18 > Should use 'org.apache.hbase.thirdparty.io.netty.tryReflectionSetAccessible' > > > Key: HBASE-27510 > URL: https://issues.apache.org/jira/browse/HBASE-27510 > Project: HBase > Issue Type: Bug >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 2.6.0, 2.5.3, 2.4.18, 3.0.0-beta-1 > > > We current use '-Dio.netty.tryReflectionSetAccessible=true' in pom, but after > shading, it should be > '-Dorg.apache.hbase.thirdparty.io.netty.tryReflectionSetAccessible=true' now. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28252) Add sun.net.dns and sun.net.util to the JDK11+ module exports in the hbase script
[ https://issues.apache.org/jira/browse/HBASE-28252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-28252: Resolution: Fixed Status: Resolved (was: Patch Available) > Add sun.net.dns and sun.net.util to the JDK11+ module exports in the hbase > script > - > > Key: HBASE-28252 > URL: https://issues.apache.org/jira/browse/HBASE-28252 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7 > > > As noted in HBASE-28247, HBase can run into module permission issues that are > not handled in the current JDK11 options in the hbase startup script. > The surefire test config also includes some JDK17 specific options, we should > also add those as needed. > We are not yet aware of any additional JVM options required by Java 21. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28252) Add sun.net.dns and sun.net.util to the JDK11+ module exports in the hbase script
[ https://issues.apache.org/jira/browse/HBASE-28252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-28252: Fix Version/s: 2.4.18 > Add sun.net.dns and sun.net.util to the JDK11+ module exports in the hbase > script > - > > Key: HBASE-28252 > URL: https://issues.apache.org/jira/browse/HBASE-28252 > Project: HBase > Issue Type: Bug > Components: scripts >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7 > > > As noted in HBASE-28247, HBase can run into module permission issues that are > not handled in the current JDK11 options in the hbase startup script. > The surefire test config also includes some JDK17 specific options, we should > also add those as needed. > We are not yet aware of any additional JVM options required by Java 21. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-28247) Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM test flags
[
https://issues.apache.org/jira/browse/HBASE-28247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros resolved HBASE-28247.
-
Resolution: Fixed
> Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM
> test flags
>
>
> Key: HBASE-28247
> URL: https://issues.apache.org/jira/browse/HBASE-28247
> Project: HBase
> Issue Type: Bug
> Components: java
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> While testing with JDK17 we have found that we need to add
> {noformat}
> --add-exports java.base/sun.net.dns=ALL-UNNAMED
> --add-exports java.base/sun.net.util=ALL-UNNAMED
> {noformat}
> on top of what is already defined in _hbase-surefire.jdk11.flags_ , otherwise
> RS and Master startup fails in the Hadoop security code.
> While this does not affect the test suite (at least not the commonly run
> tests), I consider hbase-surefire.jdk11.flags to be an unoffical resource to
> getting HBase to run on newer JDK versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-28247) Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM test flags
[
https://issues.apache.org/jira/browse/HBASE-28247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-28247:
Fix Version/s: 2.6.0
2.4.18
3.0.0-beta-1
2.5.7
> Add java.base/sun.net.dns and java.base/sun.net.util export to jdk11 JVM
> test flags
>
>
> Key: HBASE-28247
> URL: https://issues.apache.org/jira/browse/HBASE-28247
> Project: HBase
> Issue Type: Bug
> Components: java
>Affects Versions: 2.6.0, 3.0.0-alpha-4, 2.4.17, 2.5.6
>Reporter: Istvan Toth
>Assignee: Istvan Toth
>Priority: Minor
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.7
>
>
> While testing with JDK17 we have found that we need to add
> {noformat}
> --add-exports java.base/sun.net.dns=ALL-UNNAMED
> --add-exports java.base/sun.net.util=ALL-UNNAMED
> {noformat}
> on top of what is already defined in _hbase-surefire.jdk11.flags_ , otherwise
> RS and Master startup fails in the Hadoop security code.
> While this does not affect the test suite (at least not the commonly run
> tests), I consider hbase-surefire.jdk11.flags to be an unoffical resource to
> getting HBase to run on newer JDK versions.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-20563) Use parameterized logging in hbase-common
[ https://issues.apache.org/jira/browse/HBASE-20563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-20563: Resolution: Won't Fix Status: Resolved (was: Patch Available) [~zhangduo] closing it with won't fix. > Use parameterized logging in hbase-common > - > > Key: HBASE-20563 > URL: https://issues.apache.org/jira/browse/HBASE-20563 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha-1 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Fix For: 3.0.0-beta-1 > > Attachments: HBASE-20563.master.001.patch > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-20562) [UMBRELLA] Use parameterized logging
[
https://issues.apache.org/jira/browse/HBASE-20562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros resolved HBASE-20562.
-
Resolution: Won't Fix
> [UMBRELLA] Use parameterized logging
>
>
> Key: HBASE-20562
> URL: https://issues.apache.org/jira/browse/HBASE-20562
> Project: HBase
> Issue Type: Umbrella
>Affects Versions: 3.0.0-alpha-1
>Reporter: Balazs Meszaros
>Priority: Major
>
> We should use parameterized log message feature of slf4j/logback. (Use {}
> instead of string concatenation.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-20563) Use parameterized logging in hbase-common
[ https://issues.apache.org/jira/browse/HBASE-20563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-20563: Fix Version/s: (was: 3.0.0-beta-1) > Use parameterized logging in hbase-common > - > > Key: HBASE-20563 > URL: https://issues.apache.org/jira/browse/HBASE-20563 > Project: HBase > Issue Type: Sub-task >Affects Versions: 3.0.0-alpha-1 >Reporter: Balazs Meszaros >Assignee: Balazs Meszaros >Priority: Major > Attachments: HBASE-20563.master.001.patch > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Closed] (HBASE-20562) [UMBRELLA] Use parameterized logging
[
https://issues.apache.org/jira/browse/HBASE-20562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros closed HBASE-20562.
---
> [UMBRELLA] Use parameterized logging
>
>
> Key: HBASE-20562
> URL: https://issues.apache.org/jira/browse/HBASE-20562
> Project: HBase
> Issue Type: Umbrella
>Affects Versions: 3.0.0-alpha-1
>Reporter: Balazs Meszaros
>Priority: Major
>
> We should use parameterized log message feature of slf4j/logback. (Use {}
> instead of string concatenation.)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (HBASE-20549) Dead MBean: Hadoop:service=HBase,sub=Memory,name=RegionServer
[
https://issues.apache.org/jira/browse/HBASE-20549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17798212#comment-17798212
]
Balazs Meszaros commented on HBASE-20549:
-
[~zhangduo] No updates. I am fine if we drop this.
> Dead MBean: Hadoop:service=HBase,sub=Memory,name=RegionServer
> -
>
> Key: HBASE-20549
> URL: https://issues.apache.org/jira/browse/HBASE-20549
> Project: HBase
> Issue Type: Bug
>Reporter: Michael Stack
>Assignee: Balazs Meszaros
>Priority: Major
> Fix For: 3.0.0-beta-1
>
> Attachments: HBASE-20549.master.001.patch, Screen Shot 2018-05-08 at
> 10.05.57 PM.png
>
>
> I'm studying flushing and blocking memstores. I came across the MBean
> Hadoop:service=HBase,sub=Memory,name=RegionServer It looks promising with
> metrics on blocked flush time, etc. See below. All metrics are zero. I tried
> to get it going. It is connected to the heap tuner facitlity that we might
> want to turn on. I managed to get the latter going w/ below configs (need
> to dump out more on what this thing is doing because it is tricky enabling it)
> {code:java}
>
>
>
> hbase.regionserver.global.memstore.size.min.range
>
> 0.39
>
>
>
> hbase.regionserver.global.memstore.size.max.range
>
> 0.41
>
>
>
> hfile.block.cache.size.max.range
>
> 0.41
>
>
>
> hfile.block.cache.size.min.range
>
> 0.39
>
>
> {code}
> ... but still the metrics are not updated.
> We should make this bean work whether tuning is on or off or remove it.
> !Screen Shot 2018-05-08 at 10.05.57 PM.png|width=440,height=692!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28254) Flaky test: TestTableShell
[
https://issues.apache.org/jira/browse/HBASE-28254?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros resolved HBASE-28254.
-
Fix Version/s: 2.6.0
2.4.18
3.0.0-beta-1
2.5.8
Resolution: Fixed
> Flaky test: TestTableShell
> --
>
> Key: HBASE-28254
> URL: https://issues.apache.org/jira/browse/HBASE-28254
> Project: HBase
> Issue Type: Test
> Components: flakies, integration tests
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8
>
>
> The test is running the following Ruby commands:
> {noformat}
> # Instert test data
> @test_table.put(1, "x:a", 1)
> @test_table.put(2, "x:raw1", 11)
> @test_table.put(2, "x:raw1", 11)
> @test_table.put(2, "x:raw1", 11)
> @test_table.put(2, "x:raw1", 11)
> {noformat}
> And validate the versions with:
> {noformat}
> args = { VERSIONS => 10, RAW => true } # Since 4 versions of row with rowkey
> 2 is been added, we can use any number >= 4 for VERSIONS to scan all 4
> versions.
> num_rows = 0
> @test_table._scan_internal(args) do # Raw Scan
> num_rows += 1
> end
> # 5 since , 1 from row key '1' and other 4 from row key '4'
> assert_equal(num_rows, 5,
> 'Num rows scanned without RAW/VERSIONS are not 5')
> {noformat}
> Which sometimes (almost always on fast machines) fails, because it only finds
> 3 versions out of 4. I believe this due to commands are running too fast and
> inserts data with same timestamp, so HBase cannot distinguish them. I'd like
> to add some sleep between puts to fix it.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-28274) Flaky test: TestFanOutOneBlockAsyncDFSOutput (Part 2)
[
https://issues.apache.org/jira/browse/HBASE-28274?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros resolved HBASE-28274.
-
Fix Version/s: 2.6.0
2.4.18
3.0.0-beta-1
2.5.8
Resolution: Fixed
> Flaky test: TestFanOutOneBlockAsyncDFSOutput (Part 2)
> -
>
> Key: HBASE-28274
> URL: https://issues.apache.org/jira/browse/HBASE-28274
> Project: HBase
> Issue Type: Test
> Components: flakies, integration tests, test
>Reporter: Andor Molnar
>Assignee: Andor Molnar
>Priority: Major
> Fix For: 2.6.0, 2.4.18, 3.0.0-beta-1, 2.5.8
>
>
> The following test sometimes fails for me when running locally with Maven:
> TestFanOutOneBlockAsyncDFSOutput.testRecover()
> I can't really figure out the reason, but it's probably a side effect of the
> preceding test: testConnectToDatanodeFailed(). This test also restarts one of
> the datanodes in the MiniDFS cluster just like testRecover() and it somehow
> causes the failure.
> {noformat}
> java.lang.AssertionError: flush should fail
> at org.junit.Assert.fail(Assert.java:89)
> at
> org.apache.hadoop.hbase.io.asyncfs.TestFanOutOneBlockAsyncDFSOutput.testRecover(TestFanOutOneBlockAsyncDFSOutput.java:154)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method){noformat}
> The flush() call is testRecover() should fail, because we restart one of the
> DN in the DFS cluster which is expected to brake connection. It succeeds
> though if the preceding test already restarted a DN. No matter which DN we
> restart, even if they're different, the error occurs.
> I also tried to add CLUSTER.waitDatanodeFullyStarted() at the end of
> testConnectToDatanodeFailed(), looks like it made the tests slightly more
> stable, but didn't help fully.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (HBASE-28292) Make Delay prefetch property to be dynamically configured
[ https://issues.apache.org/jira/browse/HBASE-28292?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros reassigned HBASE-28292: --- Assignee: Abhishek Kothalikar > Make Delay prefetch property to be dynamically configured > - > > Key: HBASE-28292 > URL: https://issues.apache.org/jira/browse/HBASE-28292 > Project: HBase > Issue Type: Improvement >Reporter: Abhishek Kothalikar >Assignee: Abhishek Kothalikar >Priority: Major > > Rolling restart triggers region movement on the cluster while the > RegionServers are getting restarted. During this time, the temporary > RegionServers will start prefetching these files which are only hosted until > the source RegionServer is restarted. Hence, in this timing window, fetches > are executed on temporary region servers which takes few minutes. > Currrently, this behaviour is governed by hbase.hfile.prefetch.delay > Also, file-based block cache gives performance improvement on HBase clusters > using object storage. COD uses a large ephemeral disk for this purpose. > > Possible solution is, > > The number of prefetches on the cluster caused by these temporary region > movements during Rolling Upgrade (rolling restart) can be mininized by > introducing delay before the prefetch starts. > To achieve this, making the hbase.hfile.prefetch.delay dynamically > configurable. The interval will let the rolling restart complete. > Resultantly, prefetches on temporary region server can be minimized. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-28345) Close HBase connection on exit from HBase Shell
[ https://issues.apache.org/jira/browse/HBASE-28345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-28345: Fix Version/s: 2.6.0 2.4.18 2.5.8 3.0.0-beta-2 Resolution: Fixed Status: Resolved (was: Patch Available) > Close HBase connection on exit from HBase Shell > --- > > Key: HBASE-28345 > URL: https://issues.apache.org/jira/browse/HBASE-28345 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 2.4.17 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Major > Labels: pull-request-available > Fix For: 2.6.0, 2.4.18, 2.5.8, 3.0.0-beta-2 > > > When using Netty for the ZK client, hbase shell hangs on exit. > This is caused by the non-deamon Netty threads that ZK creates. > Wheter ZK should create daemon threads for Netty or not is debatable, but > explicitly closing the connection in hbase shell on exit fixes the issue. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27810) HBCK throws RejectedExecutionException when closing ZooKeeper resources
[ https://issues.apache.org/jira/browse/HBASE-27810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-27810: Fix Version/s: 2.6.0 3.0.0-alpha-4 2.5.5 > HBCK throws RejectedExecutionException when closing ZooKeeper resources > --- > > Key: HBASE-27810 > URL: https://issues.apache.org/jira/browse/HBASE-27810 > Project: HBase > Issue Type: Bug > Components: hbck >Affects Versions: 2.6.0, 3.0.0-alpha-3, 2.4.17, 2.5.4 >Reporter: Andor Molnar >Assignee: Andor Molnar >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4, 2.5.5 > > > HBCK throws RejectedExecutionException at the end of run, because the order > of closing ZooKeeper resources has been swapped in HBASE-27426. > In ZKWatcher.java close() method first it shuts down the zkEventProcessor and > when it fully shut down, it closes the RecoverableZooKeeper (the ZK client). > The watcher receives the close event which cannot be submitted to the event > processor and throws exception. > I think we need to check whether the executor is able to receive jobs before > submitting. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27406) Make "/prometheus" endpoint accessible from HBase UI
[ https://issues.apache.org/jira/browse/HBASE-27406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-27406: Fix Version/s: 2.6.0 3.0.0-alpha-4 > Make "/prometheus" endpoint accessible from HBase UI > > > Key: HBASE-27406 > URL: https://issues.apache.org/jira/browse/HBASE-27406 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 2.6.0, 3.0.0-alpha-3 >Reporter: Luca Kovacs >Assignee: Luca Kovacs >Priority: Minor > Fix For: 2.6.0, 3.0.0-alpha-4 > > > Prometheus metrics feature were added in HBASE-20904, but it's not in the UI > yet, so the only option to use it is the URL. > I would like to make a change in the web UI, so it can be accessed via a > dropdown menu there and I would like to include the description option, so > metrics can be viewed with or without description. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27406) Make "/prometheus" endpoint accessible from HBase UI
[ https://issues.apache.org/jira/browse/HBASE-27406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros resolved HBASE-27406. - Resolution: Fixed > Make "/prometheus" endpoint accessible from HBase UI > > > Key: HBASE-27406 > URL: https://issues.apache.org/jira/browse/HBASE-27406 > Project: HBase > Issue Type: Improvement > Components: UI >Affects Versions: 2.6.0, 3.0.0-alpha-3 >Reporter: Luca Kovacs >Assignee: Luca Kovacs >Priority: Minor > Fix For: 2.6.0, 3.0.0-alpha-4 > > > Prometheus metrics feature were added in HBASE-20904, but it's not in the UI > yet, so the only option to use it is the URL. > I would like to make a change in the web UI, so it can be accessed via a > dropdown menu there and I would like to include the description option, so > metrics can be viewed with or without description. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-27397) Spark-hbase support for 'startWith' predicate.
[ https://issues.apache.org/jira/browse/HBASE-27397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-27397: Fix Version/s: hbase-connectors-1.0.1 Resolution: Fixed Status: Resolved (was: Patch Available) > Spark-hbase support for 'startWith' predicate. > -- > > Key: HBASE-27397 > URL: https://issues.apache.org/jira/browse/HBASE-27397 > Project: HBase > Issue Type: New Feature > Components: hbase-connectors >Affects Versions: 3.0.0-alpha-3 >Reporter: Sergey Soldatov >Assignee: Richárd Antal >Priority: Minor > Labels: usability > Fix For: hbase-connectors-1.0.1 > > > Currently, spark-hbase connector doesn't support stringStartWith predicate > and completely ignores it. This is a disadvantage compared to the Apache > Phoenix connector and the old SHC (Hortonworks spark-hbase connector). It > would be nice if we also have this functionality in the actual connector. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27435) Make Prometheus metrics queryable
[ https://issues.apache.org/jira/browse/HBASE-27435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros resolved HBASE-27435. - Fix Version/s: 2.6.0 3.0.0-alpha-4 Resolution: Fixed > Make Prometheus metrics queryable > - > > Key: HBASE-27435 > URL: https://issues.apache.org/jira/browse/HBASE-27435 > Project: HBase > Issue Type: Improvement > Components: metrics >Reporter: Luca Kovacs >Assignee: Luca Kovacs >Priority: Minor > Fix For: 2.6.0, 3.0.0-alpha-4 > > > To provide the same features that JMX has, I would like to implement querying > in Prometheus metrics. > This includes the *_qry_* URL parameter which can be used with the Prometheus > name of the metric. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (HBASE-27673) Fix mTLS client authentication
Balazs Meszaros created HBASE-27673:
---
Summary: Fix mTLS client authentication
Key: HBASE-27673
URL: https://issues.apache.org/jira/browse/HBASE-27673
Project: HBase
Issue Type: Bug
Components: rpc
Affects Versions: 3.0.0-alpha-3
Reporter: Balazs Meszaros
Assignee: Balazs Meszaros
The exception what I get:
{noformat}
23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify host address:
127.0.0.1
javax.net.ssl.SSLPeerUnverifiedException: Certificate for <127.0.0.1> doesn't
match any of the subject alternative names: [***]
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchIPAddress(HBaseHostnameVerifier.java:144)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:117)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:143)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
...
23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify hostname:
localhost
javax.net.ssl.SSLPeerUnverifiedException: Certificate for doesn't
match any of the subject alternative names: [***]
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchDNSName(HBaseHostnameVerifier.java:159)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:119)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:171)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
...
23/02/22 15:18:06 WARN ipc.NettyRpcServer: Connection /100.100.124.2:47109;
caught unexpected downstream exception.
org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException:
javax.net.ssl.SSLHandshakeException: Failed to verify both host address and
host name
at
org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:499)
at
org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at
org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at
org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
at
org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
at
org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
at
org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at
org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:750)
Caused by: javax.net.ssl.SSLHandshakeException: Failed to verify both host
address and host name
at sun.security.ssl.Alert.createSSLException(Alert.java:131)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
at
sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkClientCerts(CertificateMessage.java:700)
at
sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:411)
at
sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:375)
at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
at sun.security.ssl
[jira] [Updated] (HBASE-27673) Fix mTLS client authentication
[
https://issues.apache.org/jira/browse/HBASE-27673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-27673:
Description:
The exception what I get:
{noformat}
23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify host address:
127.0.0.1
javax.net.ssl.SSLPeerUnverifiedException: Certificate for <127.0.0.1> doesn't
match any of the subject alternative names: [***]
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchIPAddress(HBaseHostnameVerifier.java:144)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:117)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:143)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
...
23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify hostname:
localhost
javax.net.ssl.SSLPeerUnverifiedException: Certificate for doesn't
match any of the subject alternative names: [***]
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchDNSName(HBaseHostnameVerifier.java:159)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:119)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:171)
at
org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
...
23/02/22 15:18:06 WARN ipc.NettyRpcServer: Connection /100.100.124.2:47109;
caught unexpected downstream exception.
org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException:
javax.net.ssl.SSLHandshakeException: Failed to verify both host address and
host name
at
org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:499)
at
org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
at
org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
at
org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
at
org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at
org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
at
org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
at
org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
at
org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at
org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at
org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.lang.Thread.run(Thread.java:750)
Caused by: javax.net.ssl.SSLHandshakeException: Failed to verify both host
address and host name
at sun.security.ssl.Alert.createSSLException(Alert.java:131)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
at
sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkClientCerts(CertificateMessage.java:700)
at
sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:411)
at
sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:375)
at sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:377)
at sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444)
at
sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:981)
at
sun.security.ssl.SSLEngineImpl$DelegatedTa
[jira] [Commented] (HBASE-27673) Fix mTLS client authentication
[
https://issues.apache.org/jira/browse/HBASE-27673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17694015#comment-17694015
]
Balazs Meszaros commented on HBASE-27673:
-
We are getting the remote address from {{SSLEngine}} at
{{HBaseTrustManager:97}}, so we have to create {{SSLEngine}} properly.
> Fix mTLS client authentication
> --
>
> Key: HBASE-27673
> URL: https://issues.apache.org/jira/browse/HBASE-27673
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 3.0.0-alpha-3
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
>
> The exception what I get:
> {noformat}
> 23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify host address:
> 127.0.0.1
> javax.net.ssl.SSLPeerUnverifiedException: Certificate for <127.0.0.1> doesn't
> match any of the subject alternative names: [***]
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchIPAddress(HBaseHostnameVerifier.java:144)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:117)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:143)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
> ...
> 23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify hostname:
> localhost
> javax.net.ssl.SSLPeerUnverifiedException: Certificate for doesn't
> match any of the subject alternative names: [***]
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchDNSName(HBaseHostnameVerifier.java:159)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:119)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:171)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
> ...
> 23/02/22 15:18:06 WARN ipc.NettyRpcServer: Connection /100.100.124.2:47109;
> caught unexpected downstream exception.
> org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException:
> javax.net.ssl.SSLHandshakeException: Failed to verify both host address and
> host name
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:499)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: javax.net.ssl.SSLHandshakeException: Failed to verify both host
> address and host name
> at sun.security.ssl.Alert.createSSLException(Alert.java:131)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java
[jira] [Updated] (HBASE-27673) Fix mTLS client hostname verification
[
https://issues.apache.org/jira/browse/HBASE-27673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-27673:
Summary: Fix mTLS client hostname verification (was: Fix mTLS client
authentication)
> Fix mTLS client hostname verification
> -
>
> Key: HBASE-27673
> URL: https://issues.apache.org/jira/browse/HBASE-27673
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 3.0.0-alpha-3
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
>
> The exception what I get:
> {noformat}
> 23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify host address:
> 127.0.0.1
> javax.net.ssl.SSLPeerUnverifiedException: Certificate for <127.0.0.1> doesn't
> match any of the subject alternative names: [***]
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchIPAddress(HBaseHostnameVerifier.java:144)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:117)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:143)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
> ...
> 23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify hostname:
> localhost
> javax.net.ssl.SSLPeerUnverifiedException: Certificate for doesn't
> match any of the subject alternative names: [***]
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchDNSName(HBaseHostnameVerifier.java:159)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:119)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:171)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
> ...
> 23/02/22 15:18:06 WARN ipc.NettyRpcServer: Connection /100.100.124.2:47109;
> caught unexpected downstream exception.
> org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException:
> javax.net.ssl.SSLHandshakeException: Failed to verify both host address and
> host name
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:499)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: javax.net.ssl.SSLHandshakeException: Failed to verify both host
> address and host name
> at sun.security.ssl.Alert.createSSLException(Alert.java:131)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
> at
> sun.security.ssl.CertificateMessage$T12CertificateConsumer.ch
[jira] [Resolved] (HBASE-27673) Fix mTLS client hostname verification
[
https://issues.apache.org/jira/browse/HBASE-27673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros resolved HBASE-27673.
-
Fix Version/s: 2.6.0
3.0.0-alpha-4
Resolution: Fixed
> Fix mTLS client hostname verification
> -
>
> Key: HBASE-27673
> URL: https://issues.apache.org/jira/browse/HBASE-27673
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 3.0.0-alpha-3
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
> Fix For: 2.6.0, 3.0.0-alpha-4
>
>
> The exception what I get:
> {noformat}
> 23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify host address:
> 127.0.0.1
> javax.net.ssl.SSLPeerUnverifiedException: Certificate for <127.0.0.1> doesn't
> match any of the subject alternative names: [***]
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchIPAddress(HBaseHostnameVerifier.java:144)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:117)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:143)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
> ...
> 23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify hostname:
> localhost
> javax.net.ssl.SSLPeerUnverifiedException: Certificate for doesn't
> match any of the subject alternative names: [***]
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchDNSName(HBaseHostnameVerifier.java:159)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:119)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:171)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
> ...
> 23/02/22 15:18:06 WARN ipc.NettyRpcServer: Connection /100.100.124.2:47109;
> caught unexpected downstream exception.
> org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException:
> javax.net.ssl.SSLHandshakeException: Failed to verify both host address and
> host name
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:499)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: javax.net.ssl.SSLHandshakeException: Failed to verify both host
> address and host name
> at sun.security.ssl.Alert.createSSLException(Alert.java:131)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
> at
> sun.security.ssl.Cert
[jira] [Work started] (HBASE-27673) Fix mTLS client hostname verification
[
https://issues.apache.org/jira/browse/HBASE-27673?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HBASE-27673 started by Balazs Meszaros.
---
> Fix mTLS client hostname verification
> -
>
> Key: HBASE-27673
> URL: https://issues.apache.org/jira/browse/HBASE-27673
> Project: HBase
> Issue Type: Bug
> Components: rpc
>Affects Versions: 3.0.0-alpha-3
>Reporter: Balazs Meszaros
>Assignee: Balazs Meszaros
>Priority: Major
>
> The exception what I get:
> {noformat}
> 23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify host address:
> 127.0.0.1
> javax.net.ssl.SSLPeerUnverifiedException: Certificate for <127.0.0.1> doesn't
> match any of the subject alternative names: [***]
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchIPAddress(HBaseHostnameVerifier.java:144)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:117)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:143)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
> ...
> 23/02/22 15:18:06 ERROR tls.HBaseTrustManager: Failed to verify hostname:
> localhost
> javax.net.ssl.SSLPeerUnverifiedException: Certificate for doesn't
> match any of the subject alternative names: [***]
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.matchDNSName(HBaseHostnameVerifier.java:159)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseHostnameVerifier.verify(HBaseHostnameVerifier.java:119)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.performHostVerification(HBaseTrustManager.java:171)
> at
> org.apache.hadoop.hbase.io.crypto.tls.HBaseTrustManager.checkClientTrusted(HBaseTrustManager.java:97)
> ...
> 23/02/22 15:18:06 WARN ipc.NettyRpcServer: Connection /100.100.124.2:47109;
> caught unexpected downstream exception.
> org.apache.hbase.thirdparty.io.netty.handler.codec.DecoderException:
> javax.net.ssl.SSLHandshakeException: Failed to verify both host address and
> host name
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:499)
> at
> org.apache.hbase.thirdparty.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440)
> at
> org.apache.hbase.thirdparty.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420)
> at
> org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:499)
> at
> org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
> at
> org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
> at
> org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
> at java.lang.Thread.run(Thread.java:750)
> Caused by: javax.net.ssl.SSLHandshakeException: Failed to verify both host
> address and host name
> at sun.security.ssl.Alert.createSSLException(Alert.java:131)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:324)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:267)
> at sun.security.ssl.TransportContext.fatal(TransportContext.java:262)
> at
> sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkClientCerts(CertificateMessage.java:700)
> at
> sun.security
[jira] [Resolved] (HBASE-27680) Bump hbase, hbase-thirdparty, hadoop and spark for hbase-connectors
[ https://issues.apache.org/jira/browse/HBASE-27680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros resolved HBASE-27680. - Fix Version/s: hbase-connectors-1.0.1 Resolution: Fixed > Bump hbase, hbase-thirdparty, hadoop and spark for hbase-connectors > --- > > Key: HBASE-27680 > URL: https://issues.apache.org/jira/browse/HBASE-27680 > Project: HBase > Issue Type: Task > Components: hbase-connectors >Reporter: Nihal Jain >Assignee: Nihal Jain >Priority: Major > Fix For: hbase-connectors-1.0.1 > > > Will update following as part of this JIRA > * HBase: 2.4.16 > * HBase thirdparty: 4.1.4 (See > [https://github.com/apache/hbase/blob/d1714710877653691e2125bd94b68a5b484a3a06/pom.xml#L634]) > * Hadoop3: 3.2.4 (We are bumping to latest stable 3.2.x) > * Hadoop2: 2.10.0 (See > * > [https://github.com/apache/hbase/blob/d1714710877653691e2125bd94b68a5b484a3a06/pom.xml#L543]) > * Spark: 3.2.3 > * Scala binary: 2.12 (See > [https://github.com/apache/spark/blob/b53c341e0fefbb33d115ab630369a18765b7763d/pom.xml#L164]) > > * Scala: 2.12.15 (See > [https://github.com/apache/spark/blob/b53c341e0fefbb33d115ab630369a18765b7763d/pom.xml#L163]) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27272) Enable code coverage reporting to SonarQube in hbase-connectors
[ https://issues.apache.org/jira/browse/HBASE-27272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros resolved HBASE-27272. - Fix Version/s: hbase-connectors-1.0.1 Resolution: Fixed > Enable code coverage reporting to SonarQube in hbase-connectors > --- > > Key: HBASE-27272 > URL: https://issues.apache.org/jira/browse/HBASE-27272 > Project: HBase > Issue Type: Task > Components: hbase-connectors >Reporter: Dóra Horváth >Assignee: Dóra Horváth >Priority: Minor > Fix For: hbase-connectors-1.0.1 > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (HBASE-27342) Use Hadoop Credentials API to retrieve passwords of TLS key/trust stores
[ https://issues.apache.org/jira/browse/HBASE-27342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17598334#comment-17598334 ] Balazs Meszaros commented on HBASE-27342: - [~andor] Can you do the backport for branch-2? There was a conflict when I tried to cherry-pick it. > Use Hadoop Credentials API to retrieve passwords of TLS key/trust stores > > > Key: HBASE-27342 > URL: https://issues.apache.org/jira/browse/HBASE-27342 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, security >Reporter: Andor Molnar >Assignee: Andor Molnar >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4 > > > Based on a discussion in the TLS Jira and mailing list, it would be > beneficial to protect the password of trust and key stores for TLS encryption > support in Netty RPC. > Hadoop Credentials API is built for the same purpose, extensible and simple > to use: > [https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (HBASE-22939) SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on.
[
https://issues.apache.org/jira/browse/HBASE-22939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros updated HBASE-22939:
Fix Version/s: 2.6.0
2.5.1
2.4.15
> SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned
> on.
> -
>
> Key: HBASE-22939
> URL: https://issues.apache.org/jira/browse/HBASE-22939
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1
>Reporter: Yiran Wu
>Assignee: Yiran Wu
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
> Attachments: HBASE-22939-v0.patch, HBASE-22939-v1.patch,
> HBASE-22939-v2.patch, HBASE-22939_branch-2.patch, HBASE-22939_branch-2.x.patch
>
>
> {code:java}
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(java.io.IOException):
> java.io.IOException: Wrong FS:
> hdfs://172.16.159.148:8020/tmp/bkldOutPut/fm1/327d2de5db4d4f0da667bfdf77105d4d,
> expected: hdfs://snake
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:433)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> Caused by: java.lang.IllegalArgumentException: Wrong FS:
> hdfs://172.16.159.148:8020/tmp/bkldOutPut/fm1/327d2de5db4d4f0da667bfdf77105d4d,
> expected: hdfs://snake
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:665)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:214)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1440)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1437)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1437)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at
> org.apache.hadoop.hbase.quotas.policies.AbstractViolationPolicyEnforcement.getFileSize(AbstractViolationPolicyEnforcement.java:95)
> at
> org.apache.hadoop.hbase.quotas.policies.MissingSnapshotViolationPolicyEnforcement.computeBulkLoadSize(MissingSnapshotViolationPolicyEnforcement.java:53)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.bulkLoadHFile(RSRpcServices.java:2407)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42004)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:374)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (HBASE-22939) SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned on.
[
https://issues.apache.org/jira/browse/HBASE-22939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Balazs Meszaros resolved HBASE-22939.
-
Resolution: Fixed
> SpaceQuotas- Bulkload from different hdfs failed when space quotas are turned
> on.
> -
>
> Key: HBASE-22939
> URL: https://issues.apache.org/jira/browse/HBASE-22939
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1
>Reporter: Yiran Wu
>Assignee: Yiran Wu
>Priority: Major
> Fix For: 2.6.0, 2.5.1, 3.0.0-alpha-4, 2.4.15
>
> Attachments: HBASE-22939-v0.patch, HBASE-22939-v1.patch,
> HBASE-22939-v2.patch, HBASE-22939_branch-2.patch, HBASE-22939_branch-2.x.patch
>
>
> {code:java}
> Caused by:
> org.apache.hadoop.hbase.ipc.RemoteWithExtrasException(java.io.IOException):
> java.io.IOException: Wrong FS:
> hdfs://172.16.159.148:8020/tmp/bkldOutPut/fm1/327d2de5db4d4f0da667bfdf77105d4d,
> expected: hdfs://snake
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:433)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:338)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:318)
> Caused by: java.lang.IllegalArgumentException: Wrong FS:
> hdfs://172.16.159.148:8020/tmp/bkldOutPut/fm1/327d2de5db4d4f0da667bfdf77105d4d,
> expected: hdfs://snake
> at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:665)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:214)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1440)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1437)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1437)
> at
> org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:442)
> at
> org.apache.hadoop.hbase.quotas.policies.AbstractViolationPolicyEnforcement.getFileSize(AbstractViolationPolicyEnforcement.java:95)
> at
> org.apache.hadoop.hbase.quotas.policies.MissingSnapshotViolationPolicyEnforcement.computeBulkLoadSize(MissingSnapshotViolationPolicyEnforcement.java:53)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.bulkLoadHFile(RSRpcServices.java:2407)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:42004)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:374)
> ... 3 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (HBASE-27342) Use Hadoop Credentials API to retrieve passwords of TLS key/trust stores
[ https://issues.apache.org/jira/browse/HBASE-27342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros updated HBASE-27342: Resolution: Fixed Status: Resolved (was: Patch Available) > Use Hadoop Credentials API to retrieve passwords of TLS key/trust stores > > > Key: HBASE-27342 > URL: https://issues.apache.org/jira/browse/HBASE-27342 > Project: HBase > Issue Type: Improvement > Components: IPC/RPC, security >Reporter: Andor Molnar >Assignee: Andor Molnar >Priority: Major > Fix For: 2.6.0, 3.0.0-alpha-4 > > > Based on a discussion in the TLS Jira and mailing list, it would be > beneficial to protect the password of trust and key stores for TLS encryption > support in Netty RPC. > Hadoop Credentials API is built for the same purpose, extensible and simple > to use: > [https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html] > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (HBASE-27395) Adding description to Prometheus metrics
[ https://issues.apache.org/jira/browse/HBASE-27395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Balazs Meszaros resolved HBASE-27395. - Fix Version/s: 2.6.0 3.0.0-alpha-4 Resolution: Fixed > Adding description to Prometheus metrics > > > Key: HBASE-27395 > URL: https://issues.apache.org/jira/browse/HBASE-27395 > Project: HBase > Issue Type: Improvement > Components: metrics >Affects Versions: 2.6.0, 3.0.0-alpha-3 >Reporter: Luca Kovacs >Assignee: Luca Kovacs >Priority: Minor > Fix For: 2.6.0, 3.0.0-alpha-4 > > > As /jmx endpoint provides enabling description for metrics by using > 'description=true' URL parameter I would like to implement this feature for > the /prometheus endpoint as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
