[jira] [Resolved] (HBASE-26174) Make rpc connection registry the default registry on 3.0.0
[ https://issues.apache.org/jira/browse/HBASE-26174?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26174. --- Fix Version/s: 3.0.0-alpha-2 Hadoop Flags: Reviewed Release Note: Now RpcConnectionRegistry is the default ConnectionRegistry. Assignee: Duo Zhang Resolution: Fixed > Make rpc connection registry the default registry on 3.0.0 > -- > > Key: HBASE-26174 > URL: https://issues.apache.org/jira/browse/HBASE-26174 > Project: HBase > Issue Type: Sub-task > Components: Client >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > Fix For: 3.0.0-alpha-2 > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-26181) Region server and master could use itself as ConnectionRegistry
[ https://issues.apache.org/jira/browse/HBASE-26181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Duo Zhang resolved HBASE-26181. --- Hadoop Flags: Reviewed Assignee: Duo Zhang Resolution: Fixed Pushed to master. Thanks [~bharathv] for reviewing. > Region server and master could use itself as ConnectionRegistry > --- > > Key: HBASE-26181 > URL: https://issues.apache.org/jira/browse/HBASE-26181 > Project: HBase > Issue Type: Sub-task > Components: master, regionserver >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Major > > As they already cached everything for connection registry in memory, the > cluster connection can fetch the in memory data directly instead of go to > zookeeper again. > This is an optimization to reduce the pressure on zookeeper. > For MasterRegistry, we do not want to use it as the ConnectionRegistry for > our cluster connection because: > // We use ZKConnectionRegistry for all the internal communication, > primarily for these reasons: > // - Decouples RS and master life cycles. RegionServers can continue be > up independent of > // masters' availability. > // - Configuration management for region servers (cluster internal) is > much simpler when adding > // new masters or removing existing masters, since only clients' config > needs to be updated. > // - We need to retain ZKConnectionRegistry for replication use anyway, > so we just extend it for > // other internal connections too. > The above comments are in our code, in the HRegionServer.cleanupConfiguration > method. > But since now, masters and regionservers both implement the ClientMetaService > interface, we are free to just let the ConnectionRegistry to make use of > these in memory information directly, instead of going to zookeeper again. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (HBASE-26184) TestTableSnapshotScanner.testMergeRegion error message is misleading.
[
https://issues.apache.org/jira/browse/HBASE-26184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wellington Chevreuil resolved HBASE-26184.
--
Resolution: Fixed
Merged to master branch.
> TestTableSnapshotScanner.testMergeRegion error message is misleading.
> -
>
> Key: HBASE-26184
> URL: https://issues.apache.org/jira/browse/HBASE-26184
> Project: HBase
> Issue Type: Bug
>Affects Versions: 3.0.0-alpha-1, 2.4.5
>Reporter: Wellington Chevreuil
>Assignee: Wellington Chevreuil
>Priority: Minor
> Fix For: 3.0.0-alpha-2
>
>
> In case of any exception thrown, this test wrongly reports a FNFE:
> {noformat}
> ...
> } catch (Exception e) {
> LOG.error("scan snapshot error", e);
> Assert.fail("Should not throw FileNotFoundException");
> Assert.assertTrue(e.getCause() != null);
> Assert.assertTrue(e.getCause().getCause() instanceof
> FileNotFoundException);
> } finally {
> ...{noformat}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-26211) [hbase-connectors] Pushdown filters in Spark do not work correctly with long types
Hristo Iliev created HBASE-26211:
Summary: [hbase-connectors] Pushdown filters in Spark do not work
correctly with long types
Key: HBASE-26211
URL: https://issues.apache.org/jira/browse/HBASE-26211
Project: HBase
Issue Type: Bug
Components: hbase-connectors
Affects Versions: 1.0.0
Reporter: Hristo Iliev
Reading from an HBase table and filtering on a LONG column does not seem to
work correctly.
{{Dataset df = spark.read()
.format("org.apache.hadoop.hbase.spark")
.option("hbase.columns.mapping", "id STRING :key, v LONG cf:v")
...
.load();
df.filter("v > 100").show();}}
Expected behaviour is to show rows where cf:v > 100, but instead an empty
dataset is shown.
Moreover, replacing {{"v > 100"}} with {{"v >= 100"}} results in a dataset
where some rows have values of v less than 100.
The problem appears to be that long values are decoded incorrectly as integers
in {{NaiveEncoder.filter}}:
{{case LongEnc | TimestampEnc =>
val in = Bytes.toInt(input, offset1)
val value = Bytes.toInt(filterBytes, offset2 + 1)
compare(in.compareTo(value), ops)}}
It looks like that error hasn’t been caught because
{{DynamicLogicExpressionSuite}} lack test cases with long values.
The erroneous code is also present in the master branch. We have extended the
test suite and implemented a quick fix and will PR on GitHub.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (HBASE-26212) Allow AuthUtil automatic renewal to be disabled
Josh Elser created HBASE-26212:
--
Summary: Allow AuthUtil automatic renewal to be disabled
Key: HBASE-26212
URL: https://issues.apache.org/jira/browse/HBASE-26212
Project: HBase
Issue Type: Improvement
Components: Client, security
Reporter: Josh Elser
Assignee: Josh Elser
Talking with [~bbende] who was looking at some "spam" in the NiFi log where
AuthUtil was complaining that it couldn't renew the UGI. This is did not cause
him problems (NiFi could always read/write to HBase), but it generated a lot of
noise in the NiFi log.
NiFi is special in that it's managing renewals on its own (for all services it
can communicate with), rather than letting each client do it on its own.
Specifically, one way they do this is by doing a keytab-based login via JAAS,
constructing a UGI object from that JAAS login, and then invoking HBase in a
normal UGI.doAs().
The problem comes in that AuthUtil _thinks_ that it is capable of renewing this
UGI instance on its own. AuthUtil can determine that the current UGI came from
a keytab, and thus thinks that it can renew it. However, this actually fails
because the LoginContext inside UGI *isn't* actually something that UGI can
renew (remember: because NiFI did it directly via JAAS and not via UGI)
{noformat}
2021-08-19 17:32:19,438 ERROR [Relogin service.Chore.1]
org.apache.hadoop.hbase.AuthUtil Got exception while trying to refresh
credentials: loginUserFromKeyTab must be done first
java.io.IOException: loginUserFromKeyTab must be done first
at
org.apache.hadoop.security.UserGroupInformation.reloginFromKeytab(UserGroupInformation.java:1194)
at
org.apache.hadoop.security.UserGroupInformation.checkTGTAndReloginFromKeytab(UserGroupInformation.java:1125)
at org.apache.hadoop.hbase.AuthUtil$1.chore(AuthUtil.java:206)
{noformat}
After talking with Bryan about this: we don't see a good way for HBase to
detect this specific "A UGI instance, but not created by UGI" case because the
LoginContext inside UGI is private. It is great that AuthUtil will
automatically try to renew keytab logins, even if not using
{{hbase.client.keytab.file}} and {{hbase.client.keytab.principal}}, so I don't
want to break that functionality{{.}}
NiFi is unique in this case that it is fully managing the renewals, so I think
the best path forward is to add an option which lets NiFi disable AuthUtil
since it knows it can safely do this. This should affect any others users (but
also give us an option if AuthUtil ever does cause problems).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (HBASE-26089) Support RegionCoprocessor on CompactionServer
[ https://issues.apache.org/jira/browse/HBASE-26089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yulin Niu resolved HBASE-26089. --- Resolution: Fixed > Support RegionCoprocessor on CompactionServer > - > > Key: HBASE-26089 > URL: https://issues.apache.org/jira/browse/HBASE-26089 > Project: HBase > Issue Type: Sub-task >Reporter: Yulin Niu >Assignee: Yulin Niu >Priority: Major > > We have two kinds of RegionCoprocessor: CoreCoprocessor or not. > 1.The CoreCoprocessor(AccessController, TokenProvider, > SecureBulkLoadEndpoint...), They have RegionEnvironmentForCoreCoprocessors > which expose RegionServerServices. And they involve authorization, > permission, etc, We don't need to support these CoreCoprocessor on > compactionServer > 2. Not CoreCoprocessor: They have RegionEnvironment which do not expose > RegionServerServices, only offer Connection/ OnlineRegions, We can propose > RegionEnvironmentOnCS, offer Connection, but the OnlineRegions may leave to > discuss -- This message was sent by Atlassian Jira (v8.3.4#803005)
