[jira] [Resolved] (HBASE-16493) MapReduce counters not updated with ScanMetrics
[ https://issues.apache.org/jira/browse/HBASE-16493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-16493. --- Resolution: Duplicate Duplicate of HBASE-16678 > MapReduce counters not updated with ScanMetrics > > > Key: HBASE-16493 > URL: https://issues.apache.org/jira/browse/HBASE-16493 > Project: HBase > Issue Type: Bug > Components: mapreduce >Affects Versions: 1.0.0, 2.0.0, 1.1.0 >Reporter: Jacobo Coll > > ScanMetrics were introduced in the [HBASE-4145]. These metrics were able to > work even in a parallel environment such as MapReduce. > The TableRecordReader creates a Scanner with a copy of the given "scan", > called "currentScan". The ScanMetrics are captured by the Scanner and > modifies the given Scan instance, "currentScan". The TableRecordReader, after > reading the last value, updates the job counters to aggregate them. > But since [HBASE-13030], the TableRecordReader reads the scanMetrics from the > object "scan" instead of using the "currentScan" -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-17203) unable to see the namespaces and tables when grant read-only privleges
[ https://issues.apache.org/jira/browse/HBASE-17203?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-17203. --- Resolution: Invalid This is correct. To see to namespaces a user needs to have Admin privilege. Please check https://hbase.apache.org/book.html#appendix_acl_matrix > unable to see the namespaces and tables when grant read-only privleges > -- > > Key: HBASE-17203 > URL: https://issues.apache.org/jira/browse/HBASE-17203 > Project: HBase > Issue Type: Task > Components: hbase, shell >Affects Versions: 1.0.0 >Reporter: sandeep vura > > Hi, > i have created namespaces and given grant read-only privileges. When i run > "list_namespace" on hbase shell i am unable to see the namespaces and also > the tables. > But namespaces and tables are able to see in HUE --> HBase browser. > Executed commands: > create_namespace 'q1' > grant '@hadooptest','R','@q1' > Please advise. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-17687) hive on hbase table and phoenix table can't be selected
[
https://issues.apache.org/jira/browse/HBASE-17687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi resolved HBASE-17687.
---
Resolution: Invalid
Hadoop Flags: (was: Incompatible change)
> hive on hbase table and phoenix table can't be selected
>
>
> Key: HBASE-17687
> URL: https://issues.apache.org/jira/browse/HBASE-17687
> Project: HBase
> Issue Type: Improvement
> Components: hbase
>Affects Versions: 1.0.2
> Environment: hadoop 2.7.2
> hbase 1.0.2
> phoenix 4.4
> hive 1.3
> all above are based on huawei FusionInsight HD(FusionInsight
> V100R002C60U10SPC001)
>Reporter: yunliangchen
> Original Estimate: 48h
> Remaining Estimate: 48h
>
> First , I created a table on phoenix, as this:
> ---
> DROP TABLE IF EXISTS bidwd_test01 CASCADE;
> CREATE TABLE IF NOT EXISTS bidwd_test01(
>rk VARCHAR,
>c1 integer,
>c2 VARCHAR,
>c3 VARCHAR,
>c4 VARCHAR
>constraint bidwd_test01_pk primary key(rk)
> )
> COMPRESSION='SNAPPY'
> ;
> ---
> And then , I upserted two rows into the table:
> ---
> upsert into bidwd_test01 values('001',1,'zhangsan','20170217','2017-02-17
> 12:34:22');
> upsert into bidwd_test01 values('002',2,'lisi','20170216','2017-02-16
> 12:34:22');
> ---
> At last , I scaned the table like this:
> ---
> select * from bidwd_test01;
> ---
> It's OK by now, but, I want to create a hive on hbase table ,that mapping to
> the phoenix table , the script likes this:
> ---
> USE BIDWD;
> DROP TABLE test01;
> CREATE EXTERNAL TABLE test01
> (
> rk string,
> id int,
> name string,
> datekey string,
> time_stamp string
> )
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler'
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,0:C1,0:C2,0:C3,0:C4")
> TBLPROPERTIES ("hbase.table.name" = "BIDWD_TEST01");
> ---
> So,I also try to insert some data into the table,and scan this table:
> ---
> set hive.execution.engine=mr;
> insert into test01 values('003',3,'lisi2','20170215','2017-02-15 12:34:22');
> select * from test01;
> ---
> But,there are some problems like this:
> +++--+-+--+--+
> | test01.rk | test01.id | test01.name | test01.datekey |
> test01.time_stamp |
> +++--+-+--+--+
> | 001| NULL | zhangsan | 20170217| 2017-02-17
> 12:34:22 |
> | 002| NULL | lisi | 20170216| 2017-02-16
> 12:34:22 |
> | 003| 3 | lisi2| 20170215| 2017-02-15
> 12:34:22 |
> +++--+-+--+--+
> the column "id" 's value was null,only the last row is ok.
> but,when I scan data in the phoenix ,there are some errors like this:
> Error: ERROR 201 (22000): Illegal data. Expected length of at least 115
> bytes, but had 31 (state=22000,code=201)
> java.sql.SQLException: ERROR 201 (22000): Illegal data. Expected length of at
> least 115 bytes, but had 31
> at
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:389)
> at
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
> at
> org.apache.phoenix.schema.KeyValueSchema.next(KeyValueSchema.java:211)
> at
> org.apache.phoenix.expression.ProjectedColumnExpression.evaluate(ProjectedColumnExpression.java:113)
> at
> org.apache.phoenix.compile.ExpressionProjector.getValue(ExpressionProjector.java:69)
> at
> org.apache.phoenix.jdbc.PhoenixResultSet.getString(PhoenixResultSet.java:591)
> at sqlline.Rows$Row.(Rows.java:183)
> at sqlline.BufferedRows.(BufferedRows.java:38)
> at sqlline.SqlLine.print(SqlLine.java:1546)
> at sqlline.Commands.execute(Commands.java:833)
> at sqlline.Commands.sql(Commands.java:732)
> at sqlline.SqlLine.dispatch(SqlLine.java:702)
> at sqlline.SqlLine.begin(SqlLine.java:575)
> at sqlline.SqlLine.start(SqlLine.java:292)
> at s
[jira] [Resolved] (HBASE-17468) unread messages in TCP connections - possible connection leak
[ https://issues.apache.org/jira/browse/HBASE-17468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-17468. --- Resolution: Duplicate Duplicate of HBASE-9393 > unread messages in TCP connections - possible connection leak > - > > Key: HBASE-17468 > URL: https://issues.apache.org/jira/browse/HBASE-17468 > Project: HBase > Issue Type: Bug >Reporter: Shridhar Sahukar >Priority: Critical > > We are running HBase 1.2.0-cdh5.7.1 (Cloudera distribution). > On our Hadoop cluster, we are seeing that each HBase region server has large > number of TCP connections to all the HDFS data nodes and all these > connections have unread data in socket buffers. Some of these connections are > also in CLOSE_WAIT or FIN_WAIT1 state while the rest are in ESTABLISHED state. > Looks like HBase is creating some connections requesting data from HDFS, but > its forgetting about those connections before it could read the data. Thus > the connections are left lingering around with large data stuck in their > receive buffers. Also, it seems HDFS closes these connections after a while, > but since there is data in receive buffer the connection is left in > CLOSE_WAIT/FIN_WAIT1 states. > Below is a snapshot from one of the region servers: > ## Total number of connections to HDFS (pid of region server is 143722) > [bda@md-bdadev-42 hbase]$ sudo netstat -anp|grep 143722 | wc -l > 827 > ## Connections that are not in ESTABLISHED state > [bda@md-bdadev-42 hbase]$ sudo netstat -anp|grep 143722 | grep -v ESTABLISHED > | wc -l > 344 > ##Snapshot of some of these connections: > tcp 133887 0 146.1.180.43:48533 146.1.180.40:50010 > ESTABLISHED 143722/java > tcp82934 0 146.1.180.43:59647 146.1.180.42:50010 > ESTABLISHED 143722/java > tcp0 0 146.1.180.43:50761 146.1.180.27:2181 > ESTABLISHED 143722/java > tcp 234084 0 146.1.180.43:58335 146.1.180.42:50010 > ESTABLISHED 143722/java > tcp 967667 0 146.1.180.43:56136 146.1.180.68:50010 > ESTABLISHED 143722/java > tcp 156037 0 146.1.180.43:59659 146.1.180.42:50010 > ESTABLISHED 143722/java > tcp 212488 0 146.1.180.43:56810 146.1.180.48:50010 > ESTABLISHED 143722/java > tcp61871 0 146.1.180.43:53593 146.1.180.35:50010 > ESTABLISHED 143722/java > tcp 121216 0 146.1.180.43:35324 146.1.180.38:50010 > ESTABLISHED 143722/java > tcp1 0 146.1.180.43:32982 146.1.180.42:50010 > CLOSE_WAIT 143722/java > tcp82934 0 146.1.180.43:42359 146.1.180.54:50010 > ESTABLISHED 143722/java > tcp 159422 0 146.1.180.43:59731 146.1.180.42:50010 > ESTABLISHED 143722/java > tcp 134573 0 146.1.180.43:60210 146.1.180.76:50010 > ESTABLISHED 143722/java > tcp82934 0 146.1.180.43:59713 146.1.180.42:50010 > ESTABLISHED 143722/java > tcp 135765 0 146.1.180.43:44412 146.1.180.29:50010 > ESTABLISHED 143722/java > tcp 161655 0 146.1.180.43:43117 146.1.180.42:50010 > ESTABLISHED 143722/java > tcp75990 0 146.1.180.43:59729 146.1.180.42:50010 > ESTABLISHED 143722/java > tcp78583 0 146.1.180.43:59971 146.1.180.42:50010 > ESTABLISHED 143722/java > tcp1 0 146.1.180.43:39893 146.1.180.67:50010 > CLOSE_WAIT 143722/java > tcp1 0 146.1.180.43:38834 146.1.180.47:50010 > CLOSE_WAIT 143722/java > tcp1 0 146.1.180.43:40707 146.1.180.50:50010 > CLOSE_WAIT 143722/java > tcp 106102 0 146.1.180.43:48208 146.1.180.75:50010 > ESTABLISHED 143722/java > tcp 332013 0 146.1.180.43:34795 146.1.180.37:50010 > ESTABLISHED 143722/java > tcp1 0 146.1.180.43:57644 146.1.180.67:50010 > CLOSE_WAIT 143722/java > tcp79119 0 146.1.180.43:54438 146.1.180.70:50010 > ESTABLISHED 143722/java > tcp77438 0 146.1.180.43:35259 146.1.180.38:50010 > ESTABLISHED 143722/java > tcp1 0 146.1.180.43:57579 146.1.180.41:50010 > CLOSE_WAIT 143722/java > tcp 318091 0 146.1.180.43:60124 146.1.180.42:50010 > ESTABLISHED 143722/java > tcp1 0 146.1.180.43:51715 146.1.180.70:50010 > CLOSE_WAIT 143722/java > tcp 126519 0 146.1.180.43:36389 146.1.180.49:50010 > ESTABLISHED 143722/java > tcp1 0 146.1.180.43:45656
[jira] [Created] (HBASE-18437) Revoke access permissions of a user from a table does not work as expected
Ashish Singhi created HBASE-18437:
-
Summary: Revoke access permissions of a user from a table does not
work as expected
Key: HBASE-18437
URL: https://issues.apache.org/jira/browse/HBASE-18437
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 1.1.12
Reporter: Ashish Singhi
Assignee: Ashish Singhi
A table for which a user was granted 'RW' permission. Now when we want to
revoke its 'W' permission only, code removes the user itself from that table
permissions.
Below is the test code which reproduces the issue.
{noformat}
@Test(timeout = 18)
public void testRevokeOnlySomePerms() throws Throwable {
TableName name = TableName.valueOf("testAgain");
HTableDescriptor htd = new HTableDescriptor(name);
HColumnDescriptor hcd = new HColumnDescriptor("cf");
htd.addFamily(hcd);
createTable(TEST_UTIL, htd);
TEST_UTIL.waitUntilAllRegionsAssigned(name);
try (Connection conn = ConnectionFactory.createConnection(conf)) {
AccessControlClient.grant(conn, name, USER_RO.getShortName(), null, null,
Action.READ, Action.WRITE);
ListMultimap tablePermissions =
AccessControlLists.getTablePermissions(conf, name);
// hbase user and USER_RO has permis
assertEquals(2, tablePermissions.size());
AccessControlClient.revoke(conn, name, USER_RO.getShortName(), null,
null, Action.READ, Action.WRITE);
tablePermissions = AccessControlLists.getTablePermissions(conf, name);
List userPerm =
tablePermissions.get(USER_RO.getShortName());
assertEquals(1, userPerm.size());
} finally {
deleteTable(TEST_UTIL, name);
}
}
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (HBASE-18935) Backport bug fixes to 1.3.x which are fixed in 1.2.x and 1.4.x
Ashish Singhi created HBASE-18935: - Summary: Backport bug fixes to 1.3.x which are fixed in 1.2.x and 1.4.x Key: HBASE-18935 URL: https://issues.apache.org/jira/browse/HBASE-18935 Project: HBase Issue Type: Umbrella Affects Versions: 1.3.1 Reporter: Ashish Singhi Fix For: 1.3.2 Backport the bug fixes which were fixed in 1.2.x and 1.4.x branch but missed in 1.3.x branch. I have used the below query to those jiras. *project = HBASE AND status in (Resolved, Closed) AND fixVersion in (1.4.0, 1.4.1) AND fixVersion in (1.2.7, 1.2.0, 1.2.1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6) AND fixVersion not in (1.3.0, 1.3.1, 1.3.2)* -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18936) Backport HBASE-16870 to branch-1.3
Ashish Singhi created HBASE-18936: - Summary: Backport HBASE-16870 to branch-1.3 Key: HBASE-18936 URL: https://issues.apache.org/jira/browse/HBASE-18936 Project: HBase Issue Type: Sub-task Affects Versions: 1.3.1 Reporter: Ashish Singhi Fix For: 1.3.2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18937) Backport HBASE-16815 to branch-1.3
Ashish Singhi created HBASE-18937: - Summary: Backport HBASE-16815 to branch-1.3 Key: HBASE-18937 URL: https://issues.apache.org/jira/browse/HBASE-18937 Project: HBase Issue Type: Sub-task Reporter: Ashish Singhi -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18938) Backport HBASE-16985 to branch-1.3
Ashish Singhi created HBASE-18938: - Summary: Backport HBASE-16985 to branch-1.3 Key: HBASE-18938 URL: https://issues.apache.org/jira/browse/HBASE-18938 Project: HBase Issue Type: Sub-task Reporter: Ashish Singhi -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-18939) Backport HBASE-16538 to branch-1.3
Ashish Singhi created HBASE-18939: - Summary: Backport HBASE-16538 to branch-1.3 Key: HBASE-18939 URL: https://issues.apache.org/jira/browse/HBASE-18939 Project: HBase Issue Type: Sub-task Affects Versions: 1.3.1 Reporter: Ashish Singhi Fix For: 1.3.2 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19796) ReplicationSynUp tool is not replicating the data if the WAL is moved to splititing directory
Ashish Singhi created HBASE-19796: - Summary: ReplicationSynUp tool is not replicating the data if the WAL is moved to splititing directory Key: HBASE-19796 URL: https://issues.apache.org/jira/browse/HBASE-19796 Project: HBase Issue Type: Bug Components: Replication Affects Versions: 1.3.1 Reporter: Ashish Singhi Assignee: Ashish Singhi In our test cluster we found that ReplictionSyncUp tool is not replicating the data from the source cluster RS WAL which is moved to WAL splitting directory to the peer cluster. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19905) ReplicationSyncUp tool will not exit if a peer replication is disabled
Ashish Singhi created HBASE-19905: - Summary: ReplicationSyncUp tool will not exit if a peer replication is disabled Key: HBASE-19905 URL: https://issues.apache.org/jira/browse/HBASE-19905 Project: HBase Issue Type: Bug Components: Replication Affects Versions: 1.3.1 Reporter: Ashish Singhi Assignee: Ashish Singhi -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20004) Client is not able to execute REST queries through browser in a secure cluster
Ashish Singhi created HBASE-20004: - Summary: Client is not able to execute REST queries through browser in a secure cluster Key: HBASE-20004 URL: https://issues.apache.org/jira/browse/HBASE-20004 Project: HBase Issue Type: Bug Components: REST Affects Versions: 1.3.1 Environment: Firefox browser is not able to negotiate REST queries with server in secure mode. Reporter: Ashish Singhi Assignee: Ashish Singhi -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20146) Regions are stuck to get opened when WAL is disabled
Ashish Singhi created HBASE-20146:
-
Summary: Regions are stuck to get opened when WAL is disabled
Key: HBASE-20146
URL: https://issues.apache.org/jira/browse/HBASE-20146
Project: HBase
Issue Type: Bug
Components: wal
Affects Versions: 1.3.1
Reporter: Ashish Singhi
On a running cluster we had set {{hbase.regionserver.hlog.enabled}} to false,
to disable the WAL for complete cluster, after restarting HBase service,
regions are not getting opened leading to HMaster abort as Namespace table
regions are not getting assigned.
jstack for region open:
{noformat}
"RS_OPEN_PRIORITY_REGION-BLR106595:16045-1" #159 prio=5 os_prio=0
tid=0x7fdfa4341000 nid=0x419d waiting on condition [0x7fdfa0467000]
java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
- parking to wait for <0x87554448> (a
java.util.concurrent.CountDownLatch$Sync)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
at org.apache.hadoop.hbase.wal.WALKey.getWriteEntry(WALKey.java:98)
at
org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeMarker(WALUtil.java:131)
at
org.apache.hadoop.hbase.regionserver.wal.WALUtil.writeRegionEventMarker(WALUtil.java:88)
at
org.apache.hadoop.hbase.regionserver.HRegion.writeRegionOpenMarker(HRegion.java:1026)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6849)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6803)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6774)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6730)
at org.apache.hadoop.hbase.regionserver.HRegion.openHRegion(HRegion.java:6681)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.openRegion(OpenRegionHandler.java:363)
at
org.apache.hadoop.hbase.regionserver.handler.OpenRegionHandler.process(OpenRegionHandler.java:129)
at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:129)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
This used to work with HBase 1.0.2 version.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-16499) slow replication for small HBase clusters
[
https://issues.apache.org/jira/browse/HBASE-16499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi resolved HBASE-16499.
---
Resolution: Fixed
> slow replication for small HBase clusters
> -
>
> Key: HBASE-16499
> URL: https://issues.apache.org/jira/browse/HBASE-16499
> Project: HBase
> Issue Type: Bug
> Components: Replication
>Reporter: Vikas Vishwakarma
>Assignee: Ashish Singhi
>Priority: Critical
> Fix For: 2.0.0
>
> Attachments: HBASE-16499-addendum.patch, HBASE-16499.patch,
> HBASE-16499.patch
>
>
> For small clusters 10-20 nodes we recently observed that replication is
> progressing very slowly when we do bulk writes and there is lot of lag
> accumulation on AgeOfLastShipped / SizeOfLogQueue. From the logs we observed
> that the number of threads used for shipping wal edits in parallel comes from
> the following equation in HBaseInterClusterReplicationEndpoint
> int n = Math.min(Math.min(this.maxThreads, entries.size()/100+1),
> replicationSinkMgr.getSinks().size());
> ...
> for (int i=0; i entryLists.add(new ArrayList(entries.size()/n+1)); <--
> batch size
> }
> ...
> for (int i=0; i .
> // RuntimeExceptions encountered here bubble up and are handled
> in ReplicationSource
> pool.submit(createReplicator(entryLists.get(i), i)); <--
> concurrency
> futures++;
> }
> }
> maxThreads is fixed & configurable and since we are taking min of the three
> values n gets decided based replicationSinkMgr.getSinks().size() when we have
> enough edits to replicate
> replicationSinkMgr.getSinks().size() is decided based on
> int numSinks = (int) Math.ceil(slaveAddresses.size() * ratio);
> where ratio is this.ratio = conf.getFloat("replication.source.ratio",
> DEFAULT_REPLICATION_SOURCE_RATIO);
> Currently DEFAULT_REPLICATION_SOURCE_RATIO is set to 10% so for small
> clusters of size 10-20 RegionServers the value we get for numSinks and hence
> n is very small like 1 or 2. This substantially reduces the pool concurrency
> used for shipping wal edits in parallel effectively slowing down replication
> for small clusters and causing lot of lag accumulation in AgeOfLastShipped.
> Sometimes it takes tens of hours to clear off the entire replication queue
> even after the client has finished writing on the source side.
> We are running tests by varying replication.source.ratio and have seen
> multi-fold improvement in total replication time (will update the results
> here). I wanted to propose here that we should increase the default value for
> replication.source.ratio also so that we have sufficient concurrency even for
> small clusters. We figured it out after lot of iterations and debugging so
> probably slightly higher default will save the trouble.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Resolved] (HBASE-5742) LoadIncrementalHFiles throws too generic of an exception
[ https://issues.apache.org/jira/browse/HBASE-5742?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-5742. -- Resolution: Not A Problem The constructor in the context no more exists. > LoadIncrementalHFiles throws too generic of an exception > > > Key: HBASE-5742 > URL: https://issues.apache.org/jira/browse/HBASE-5742 > Project: HBase > Issue Type: Improvement > Components: mapreduce >Reporter: David Capwell >Assignee: Ashish Singhi >Priority: Major > > In 0.90 the LoadIncrementalHFiles constructor did not throw an exception, now > it throws Exception. The constructor should ether not throw an exception or > throw ZooKeeperConnectionException, and MasterNotRunningException since those > come from the HBaseAdmin call. > https://github.com/apache/hbase/blob/trunk/src/main/java/org/apache/hadoop/hbase/mapreduce/LoadIncrementalHFiles.java#L105 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-20590) REST Java client is not able to negotiate with the server in the secure mode
Ashish Singhi created HBASE-20590: - Summary: REST Java client is not able to negotiate with the server in the secure mode Key: HBASE-20590 URL: https://issues.apache.org/jira/browse/HBASE-20590 Project: HBase Issue Type: Bug Components: REST, security Affects Versions: 1.3.1 Reporter: Ashish Singhi Assignee: Ashish Singhi -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (HBASE-11896) LoadIncrementalHFiles fails in secure mode if the namespace is specified
Ashish Singhi created HBASE-11896:
-
Summary: LoadIncrementalHFiles fails in secure mode if the
namespace is specified
Key: HBASE-11896
URL: https://issues.apache.org/jira/browse/HBASE-11896
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.3
Reporter: Ashish Singhi
Fix For: 1.0.0, 2.0.0
completebulkload tool is failing in secure mode when namespace is specified.
{noformat}
Caused by: java.lang.IllegalArgumentException: java.net.URISyntaxException:
Relative path in absolute URI:
hdfs__ns:a1__eb0l9esfa6ta4msuul40vk1kgts2hn0goo953geplsnjv1qbeq1a4sacvp07qcn6
at org.apache.hadoop.fs.Path.initialize(Path.java:206)
at org.apache.hadoop.fs.Path.(Path.java:172)
at org.apache.hadoop.fs.Path.(Path.java:94)
at
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.createStagingDir(SecureBulkLoadEndpoint.java:303)
at
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.createStagingDir(SecureBulkLoadEndpoint.java:296)
at
org.apache.hadoop.hbase.security.access.SecureBulkLoadEndpoint.prepareBulkLoad(SecureBulkLoadEndpoint.java:160)
at
org.apache.hadoop.hbase.protobuf.generated.SecureBulkLoadProtos$SecureBulkLoadService.callMethod(SecureBulkLoadProtos.java:4626)
at
org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:5696)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3332)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3314)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:29501)
{noformat}
Here I feel we should replace the ':' in table name
{code}
bulkToken = new SecureBulkLoadClient(table).prepareBulkLoad(table.getName())
{code}
Because at the later part of code we are creating staging directory using table
name.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12049) Help for alter command is a bit confusing
Ashish Singhi created HBASE-12049:
-
Summary: Help for alter command is a bit confusing
Key: HBASE-12049
URL: https://issues.apache.org/jira/browse/HBASE-12049
Project: HBase
Issue Type: Improvement
Components: shell
Affects Versions: 0.98.5
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Trivial
The help message shown for alter command is a bit confusing.
A part of current help message
{code}
Here is some help for this command:
Alter a table. Depending on the HBase setting
("hbase.online.schema.update.enable"),
the table must be disabled or not to be altered (see help 'disable').
You can add/modify/delete column families, as well as change table
configuration. Column families work similarly to create; column family
spec can either be a name string, or a dictionary with NAME attribute.
Dictionaries are described on the main help command output.
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12095) Master shuts down during log splitting on restart
Ashish Singhi created HBASE-12095: - Summary: Master shuts down during log splitting on restart Key: HBASE-12095 URL: https://issues.apache.org/jira/browse/HBASE-12095 Project: HBase Issue Type: Bug Affects Versions: 0.98.6 Reporter: Ashish Singhi Fix For: 0.98.7 I observed that when I have the following value set in my hbase-site.xml file hbase.regionserver.wal.encryption false hbase.regionserver.hlog.reader.impl org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader hbase.regionserver.hlog.writer.impl org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogWriter And while log splitting on hbase service restart, master shutdown with following exception. Exception in master log 2014-09-24 17:14:28,590 FATAL [master:host-10-18-40-18:6] master.HMaster: Master server abort: loaded coprocessors are: [org.apache.hadoop.hbase.security.access.AccessController] 2014-09-24 17:14:28,590 FATAL [master:host-10-18-40-18:6] master.HMaster: Unhandled exception. Starting shutdown. java.io.IOException: error or interrupted while splitting logs in [hdfs://10.18.40.18:8020/tmp/hbase-ashish/hbase/WALs/host-10-18-40-18,60020,1411558717849-splitting] Task = installed = 6 done = 0 error = 6 at org.apache.hadoop.hbase.master.SplitLogManager.splitLogDistributed(SplitLogManager.java:378) at org.apache.hadoop.hbase.master.MasterFileSystem.splitLog(MasterFileSystem.java:415) at org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:307) at org.apache.hadoop.hbase.master.MasterFileSystem.splitMetaLog(MasterFileSystem.java:298) at org.apache.hadoop.hbase.master.HMaster.splitMetaLogBeforeAssignment(HMaster.java:1071) at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:863) at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:612) at java.lang.Thread.run(Thread.java:745) Exception in region server log 2014-09-24 20:10:16,535 WARN [RS_LOG_REPLAY_OPS-host-10-18-40-18:60020-1] regionserver.SplitLogWorker: log splitting of WALs/host-10-18-40-18,60020,1411558717849-splitting/host-10-18-40-18%2C60020%2C1411558717849.1411558724316.meta failed, returning error java.io.IOException: Cannot get log reader at org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:161) at org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:89) at org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:660) at org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.getReader(HLogSplitter.java:569) at org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:282) at org.apache.hadoop.hbase.regionserver.wal.HLogSplitter.splitLogFile(HLogSplitter.java:225) at org.apache.hadoop.hbase.regionserver.SplitLogWorker$1.exec(SplitLogWorker.java:143) at org.apache.hadoop.hbase.regionserver.handler.HLogSplitterHandler.process(HLogSplitterHandler.java:82) at org.apache.hadoop.hbase.executor.EventHandler.run(EventHandler.java:128) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.UnsupportedOperationException: Unable to find suitable constructor for class org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec at org.apache.hadoop.hbase.util.ReflectionUtils.instantiateWithCustomCtor(ReflectionUtils.java:39) at org.apache.hadoop.hbase.regionserver.wal.WALCellCodec.create(WALCellCodec.java:101) at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.getCodec(ProtobufLogReader.java:242) at org.apache.hadoop.hbase.regionserver.wal.ProtobufLogReader.initAfterCompression(ProtobufLogReader.java:247) at org.apache.hadoop.hbase.regionserver.wal.SecureProtobufLogReader.initAfterCompression(SecureProtobufLogReader.java:138) at org.apache.hadoop.hbase.regionserver.wal.ReaderBase.init(ReaderBase.java:85) at org.apache.hadoop.hbase.regionserver.wal.HLogFactory.createReader(HLogFactory.java:113) ... 11 more Caused by: java.lang.NoSuchMethodException: org.apache.hadoop.hbase.regionserver.wal.SecureWALCellCodec.(org.apache.hadoop.conf.Configuration, org.apache.hadoop.hbase.regionserver.wal.CompressionContext) at java.lang.Class.getConstructor0(Class.java:2849) at java.lang.C
[jira] [Created] (HBASE-12096) In ZKSplitLog Coordination and AggregateImplementation replace enhaced for statements with basic for statement to avoid unnecessary object allocation
Ashish Singhi created HBASE-12096: - Summary: In ZKSplitLog Coordination and AggregateImplementation replace enhaced for statements with basic for statement to avoid unnecessary object allocation Key: HBASE-12096 URL: https://issues.apache.org/jira/browse/HBASE-12096 Project: HBase Issue Type: Sub-task Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor In ZKSplitLog Coordination classes and AggregateImplementation replace enhanced for statements with basic for statement to avoid unnecessary object allocation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12118) Explain how to grant permission to a namespace in grant command usage
Ashish Singhi created HBASE-12118: - Summary: Explain how to grant permission to a namespace in grant command usage Key: HBASE-12118 URL: https://issues.apache.org/jira/browse/HBASE-12118 Project: HBase Issue Type: Improvement Components: documentation, shell Affects Versions: 0.98.6.1 Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor In the grant command usage, How to grant permission to a namespace is missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12240) hbase-daemon.sh should remove pid file if process not found running
Ashish Singhi created HBASE-12240:
-
Summary: hbase-daemon.sh should remove pid file if process not
found running
Key: HBASE-12240
URL: https://issues.apache.org/jira/browse/HBASE-12240
Project: HBase
Issue Type: Bug
Components: Usability
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
{{stop-hbase.sh}} removes process pid file irrespective of whether process is
running or not.
Same should be the behavior with {{hbase-daemon.sh stop}} command.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12264) ImportTsv should fail fast if output is not specified and table does not exist
Ashish Singhi created HBASE-12264:
-
Summary: ImportTsv should fail fast if output is not specified and
table does not exist
Key: HBASE-12264
URL: https://issues.apache.org/jira/browse/HBASE-12264
Project: HBase
Issue Type: Improvement
Components: mapreduce
Affects Versions: 0.98.5
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
ImportTsv should fail fast if {{importtsv.bulk.output}} is not specified and
the specified table also does not exist
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-11192) HBase ClusterId File Empty Check Loggic
[
https://issues.apache.org/jira/browse/HBASE-11192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi resolved HBASE-11192.
---
Resolution: Duplicate
Duplicate of HBASE-11191
> HBase ClusterId File Empty Check Loggic
> ---
>
> Key: HBASE-11192
> URL: https://issues.apache.org/jira/browse/HBASE-11192
> Project: HBase
> Issue Type: Bug
> Environment: HBase 0.94+Hadoop2.2.0+Zookeeper3.4.5
>Reporter: sunjingtao
>
> if the clusterid file exists but empty ,then the following check logic in the
> MasterFileSystem.java has none effects.
> if (!FSUtils.checkClusterIdExists(fs, rd, c.getInt(
> HConstants.THREAD_WAKE_FREQUENCY, 10 * 1000))) {
> FSUtils.setClusterId(fs, rd, UUID.randomUUID().toString(), c.getInt(
> HConstants.THREAD_WAKE_FREQUENCY, 10 * 1000));
> }
> clusterId = FSUtils.getClusterId(fs, rd);
> because the checkClusterIdExists method only check the path .
> Path filePath = new Path(rootdir, HConstants.CLUSTER_ID_FILE_NAME);
> return fs.exists(filePath);
> in my case ,the file exists but is empty,so the readed clusterid is null
> which cause a nullPointerException:
> java.lang.NullPointerException
> at org.apache.hadoop.hbase.util.Bytes.toBytes(Bytes.java:441)
> at
> org.apache.hadoop.hbase.zookeeper.ClusterId.setClusterId(ClusterId.java:72)
> at
> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:581)
> at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:433)
> at java.lang.Thread.run(Thread.java:745)
> is this a bug?please make sure!
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12304) CellCounter will throw AIOBE when output directory is not specified
Ashish Singhi created HBASE-12304: - Summary: CellCounter will throw AIOBE when output directory is not specified Key: HBASE-12304 URL: https://issues.apache.org/jira/browse/HBASE-12304 Project: HBase Issue Type: Bug Components: mapreduce Affects Versions: 0.98.5 Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor CellCounter will throw ArrayIndexOutOfBoundsException when output directory is not specified instead it should display the usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12306) CellCounter output's wrong value for Total Families Across all Rows in output file
Ashish Singhi created HBASE-12306:
-
Summary: CellCounter output's wrong value for Total Families
Across all Rows in output file
Key: HBASE-12306
URL: https://issues.apache.org/jira/browse/HBASE-12306
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 0.98.5
Reporter: Ashish Singhi
Assignee: Ashish Singhi
CellCounter output's wrong value for {{Total Families Across all Rows}} in
output file.
The scan output of table looks like the following
{noformat}
r1 column=f:count,
timestamp=1413444040422, value=v1
r1 column=f1:count1,
timestamp=1413444045493, value=v1
r1 column=f2:count2,
timestamp=1413444051461, value=v1
r2 column=f:count,
timestamp=1413444073282, value=v2
r2 column=f1:count1,
timestamp=1413444077666, value=v2
r2 column=f2:count2,
timestamp=1413444083141, value=v2
r3 column=f:count,
timestamp=1413444184672, value=v3
{noformat}
and the output in the output file of CellCounter looks like this
{noformat}
Total Families Across all Rows 7
Total Qualifiers across all Rows7
Total ROWS 3
f 3
f,count 3
f1 2
f1,count1 2
f2 2
f2,count2 2
r1,f,count_Versions 1
r1,f1,count1_Versions 1
r1,f2,count2_Versions 1
r2,f,count_Versions 1
r2,f1,count1_Versions 1
r2,f2,count2_Versions 1
r3,f,count_Versions 1
{noformat}
The {{Total Families Across all Rows}} should be 3, but it is 7.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12348) Add family name in audit log for addColumn, deleteColumn and modifyColumn operations
Ashish Singhi created HBASE-12348:
-
Summary: Add family name in audit log for addColumn, deleteColumn
and modifyColumn operations
Key: HBASE-12348
URL: https://issues.apache.org/jira/browse/HBASE-12348
Project: HBase
Issue Type: Improvement
Components: security
Affects Versions: 0.98.5
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
Family name can be added in audit log for addColumn, deleteColumn and
modifyColumn operations similar to createTable operation.
{noformat}
create 't', 'd'
2014-10-27 20:41:54,303 TRACE
SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: Access
allowed for user ashish; reason: Global check allowed; remote address:
/10.18.40.106; request: createTable; context: (user=ashish, scope=t, family=d,
action=CREATE)
alter 't', NAME => 'd', VERSIONS => 5
2014-10-27 20:42:54,771 TRACE
SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: Access
allowed for user ashish; reason: Table permission granted; remote address:
/10.18.40.106; request: modifyColumn; context: (user=ashish, scope=t, family=,
action=ADMIN)
alter 't', 'd2'
2014-10-27 20:44:45,635 TRACE
SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: Access
allowed for user ashish; reason: Table permission granted; remote address:
/10.18.40.106; request: addColumn; context: (user=ashish, scope=t, family=,
action=ADMIN)
alter 't', NAME => 'd2', METHOD => 'delete'
2014-10-27 20:45:25,681 TRACE
SecurityLogger.org.apache.hadoop.hbase.security.access.AccessController: Access
allowed for user ashish; reason: Table permission granted; remote address:
/10.18.40.106; request: deleteColumn; context: (user=ashish, scope=t, family=,
action=ADMIN)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12357) TestHCM#testClusterStatus is continuosly failing in jenkins
Ashish Singhi created HBASE-12357:
-
Summary: TestHCM#testClusterStatus is continuosly failing in
jenkins
Key: HBASE-12357
URL: https://issues.apache.org/jira/browse/HBASE-12357
Project: HBase
Issue Type: Bug
Components: test
Reporter: Ashish Singhi
{noformat}Tests run: 21, Failures: 0, Errors: 1, Skipped: 2, Time elapsed:
187.475 sec <<< FAILURE! - in org.apache.hadoop.hbase.client.TestHCM
testClusterStatus(org.apache.hadoop.hbase.client.TestHCM) Time elapsed: 41.477
sec <<< ERROR!
java.lang.Exception: Unexpected exception,
expected but
was
at junit.framework.Assert.fail(Assert.java:57)
at org.apache.hadoop.hbase.Waiter.waitFor(Waiter.java:193)
at
org.apache.hadoop.hbase.HBaseTestingUtility.waitFor(HBaseTestingUtility.java:3537)
at
org.apache.hadoop.hbase.client.TestHCM.testClusterStatus(TestHCM.java:273)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12375) LoadIncrementalHFiles fails to load data in table when CF name starts with '_'
Ashish Singhi created HBASE-12375:
-
Summary: LoadIncrementalHFiles fails to load data in table when CF
name starts with '_'
Key: HBASE-12375
URL: https://issues.apache.org/jira/browse/HBASE-12375
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.5
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
We do not restrict user from creating a table having column family starting
with '_'.
So when user creates a table in such a way then LoadIncrementalHFiles will skip
those family data to load into the table.
{code}
// Skip _logs, etc
if (familyDir.getName().startsWith("_")) continue;
{code}
I think we should remove that check as I do not see any _logs directory being
created by the bulkload tool in the output directory.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12552) listSnapshots should lists only owned snapshots for a non-super user
Ashish Singhi created HBASE-12552: - Summary: listSnapshots should lists only owned snapshots for a non-super user Key: HBASE-12552 URL: https://issues.apache.org/jira/browse/HBASE-12552 Project: HBase Issue Type: Bug Components: snapshots Reporter: Ashish Singhi Assignee: Ashish Singhi Currently list_snapshots lists all the snapshots available for a non-super user which may not be correct, this should be applicable only for a user with admin rights. So I feel for a non-super user it should lists only the snapshots owned by it if any. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12553) request value is not consistent for restoreSnapshot in audit logs
Ashish Singhi created HBASE-12553:
-
Summary: request value is not consistent for restoreSnapshot in
audit logs
Key: HBASE-12553
URL: https://issues.apache.org/jira/browse/HBASE-12553
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
{code}
requirePermission("restoreSnapshot", hTableDescriptor.getTableName(),
null, null,
Permission.Action.ADMIN);
} else {
requirePermission("restore", Action.ADMIN);
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12594) Remove the usage of deprecated HbaseAdmin#getTableNames from shell list command
Ashish Singhi created HBASE-12594: - Summary: Remove the usage of deprecated HbaseAdmin#getTableNames from shell list command Key: HBASE-12594 URL: https://issues.apache.org/jira/browse/HBASE-12594 Project: HBase Issue Type: Bug Components: shell Affects Versions: 2.0.0 Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor With this fix one behavior change a non-super user will notice is that he/she will be able to see only tables to which they have create/admin permissions -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12601) Explain how to grant permission to a group in grant command usage
Ashish Singhi created HBASE-12601: - Summary: Explain how to grant permission to a group in grant command usage Key: HBASE-12601 URL: https://issues.apache.org/jira/browse/HBASE-12601 Project: HBase Issue Type: Improvement Components: security, shell Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor In the grant command usage, how to grant permission to a group is missing. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12624) Remove rename_snapshot.rb from code as there is no equivalent renameSnapshot api in Admin
Ashish Singhi created HBASE-12624: - Summary: Remove rename_snapshot.rb from code as there is no equivalent renameSnapshot api in Admin Key: HBASE-12624 URL: https://issues.apache.org/jira/browse/HBASE-12624 Project: HBase Issue Type: Task Components: shell Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor rename_sanpshot.rb fails as it looks for renameSnapshot api in admin class, which is not implemented as of now. May be we can remove it now and add it when we plan to implement that. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12634) Fix the AccessController#requireGlobalPermission(ns) with NS
Ashish Singhi created HBASE-12634: - Summary: Fix the AccessController#requireGlobalPermission(ns) with NS Key: HBASE-12634 URL: https://issues.apache.org/jira/browse/HBASE-12634 Project: HBase Issue Type: Bug Components: security Affects Versions: 0.98.8 Reporter: Ashish Singhi Assignee: Ashish Singhi The namespace argument passed to AccessController#requireGlobalPermission API to authorize namespace is actually not authorizing it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12653) Move TestRegionServerOnlineConfigChange & TestConfigurationManager to Junit4 tests
Ashish Singhi created HBASE-12653: - Summary: Move TestRegionServerOnlineConfigChange & TestConfigurationManager to Junit4 tests Key: HBASE-12653 URL: https://issues.apache.org/jira/browse/HBASE-12653 Project: HBase Issue Type: Improvement Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-12662) region_status.rb is failing with NoMethodError
Ashish Singhi created HBASE-12662:
-
Summary: region_status.rb is failing with NoMethodError
Key: HBASE-12662
URL: https://issues.apache.org/jira/browse/HBASE-12662
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
{noformat}
$/hbase-2.0.0-SNAPSHOT/bin> hbase org.jruby.Main region_status.rb --table t
2014-12-09 19:33:06,839 WARN [main] util.NativeCodeLoader: Unable to load
native-hadoop library for your platform... using builtin-java classes where
applicable
NoMethodError: undefined method `cache_blocks=' for
#
(root) at region_status.rb:93
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12681) truncate_preserve comand fails with undefined method `getTable' error
Ashish Singhi created HBASE-12681:
-
Summary: truncate_preserve comand fails with undefined method
`getTable' error
Key: HBASE-12681
URL: https://issues.apache.org/jira/browse/HBASE-12681
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.99.2, 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
{noformat}
hbase(main):002:0> truncate_preserve 'a'
Truncating 'a' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12690) list_quotas command is failing with not able to load Java class
Ashish Singhi created HBASE-12690:
-
Summary: list_quotas command is failing with not able to load Java
class
Key: HBASE-12690
URL: https://issues.apache.org/jira/browse/HBASE-12690
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 0.99.2, 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Fix For: 1.0.0, 2.0.0
{noformat}
hbase(main):004:0> list_quotas
OWNERQUOTAS
ERROR: cannot load Java class org.apache.hadoop.hbase.client.ConnectionFactor
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12692) NPE from SnapshotManager#stop
Ashish Singhi created HBASE-12692:
-
Summary: NPE from SnapshotManager#stop
Key: HBASE-12692
URL: https://issues.apache.org/jira/browse/HBASE-12692
Project: HBase
Issue Type: Bug
Components: snapshots
Affects Versions: 0.99.2, 0.98.9, 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
Fix For: 1.0.0, 2.0.0, 0.98.10
If there are some files in snapshot dir and snapshot is disabled. Then we will
stop master process. As pert of this stop got NPE.
{noformat}
Exception in thread "master/host-10-18-40-106/10.18.40.106:16020"
java.lang.NullPointerException
at
org.apache.hadoop.hbase.master.snapshot.SnapshotManager.stop(SnapshotManager.java:930)
at
org.apache.hadoop.hbase.procedure.MasterProcedureManagerHost.stop(MasterProcedureManagerHost.java:56)
at
org.apache.hadoop.hbase.master.HMaster.stopServiceThreads(HMaster.java:994)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1010)
at java.lang.Thread.run(Thread.java:745)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12804) ImportTsv fails to delete partition files created by it
Ashish Singhi created HBASE-12804:
-
Summary: ImportTsv fails to delete partition files created by it
Key: HBASE-12804
URL: https://issues.apache.org/jira/browse/HBASE-12804
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.9
Reporter: Ashish Singhi
Assignee: Ashish Singhi
{code}
fs.deleteOnExit(partitionsPath);
writePartitions(job.getConfiguration(), partitionsPath, splitPoints);
{code}
In the above code deleteOnExit will return without adding file in the
deleteOnExit set as the file is not created till then.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12811) NPE while scan a table with user associated with multiple groups.
Ashish Singhi created HBASE-12811:
-
Summary: NPE while scan a table with user associated with multiple
groups.
Key: HBASE-12811
URL: https://issues.apache.org/jira/browse/HBASE-12811
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.98.9
Reporter: Ashish Singhi
Assignee: Ashish Singhi
A user is associated with two groups.
{noformat}
/hbase/bin> groups ashish_test
ashish_test : defaultgroup ashish_test_1420524824527
{noformat}
One of its group is granted permission on a table as shown by user_permission
command.
{noformat}
hbase(main):005:0> user_permission 't1'
User
Table,Family,Qualifier:Permission
@ashish_test_1420524824527 t1,,: [Permission:
actions=EXEC,WRITE,CREATE]
@ashish_test_1420524824527 t1,d,: [Permission:
actions=EXEC,WRITE,CREATE]
hbase t1,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
3 row(s) in 0.3710 seconds
{noformat}
Now when this user try the scan the table, we get the following exception.
{noformat}
java.lang.NullPointerException
at
org.apache.hadoop.hbase.security.access.TablePermission.implies(TablePermission.java:215)
at
org.apache.hadoop.hbase.security.access.TableAuthManager.authorize(TableAuthManager.java:340)
at
org.apache.hadoop.hbase.security.access.TableAuthManager.authorize(TableAuthManager.java:332)
at
org.apache.hadoop.hbase.security.access.TableAuthManager.authorizeGroup(TableAuthManager.java:473)
at
org.apache.hadoop.hbase.security.access.TableAuthManager.authorize(TableAuthManager.java:490)
at
org.apache.hadoop.hbase.security.access.TableAuthManager.authorize(TableAuthManager.java:500)
at
org.apache.hadoop.hbase.security.access.AccessController.permissionGranted(AccessController.java:415)
at
org.apache.hadoop.hbase.security.access.AccessController.permissionGranted(AccessController.java:484)
at
org.apache.hadoop.hbase.security.access.AccessController.internalPreRead(AccessController.java:1504)
at
org.apache.hadoop.hbase.security.access.AccessController.preScannerOpen(AccessController.java:2027)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preScannerOpen(RegionCoprocessorHost.java:1987)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3102)
{noformat}
*Note:* Line numbers may not match.
Exception is coming because the other group of same user which has not been
granted permission on the table will have the TablePermission's table(name) as
null.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-12953) RegionServer is not functionally working with AysncRpcClient in secure mode
Ashish Singhi created HBASE-12953:
-
Summary: RegionServer is not functionally working with
AysncRpcClient in secure mode
Key: HBASE-12953
URL: https://issues.apache.org/jira/browse/HBASE-12953
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 2.0.0, 1.1.0
Reporter: Ashish Singhi
Priority: Critical
HBase version 2.0.0
Default value for {{hbase.rpc.client.impl}} is set to AsyncRpcClient.
When trying to install HBase with Kerberos, RegionServer is not working
functionally.
The following log is logged in its log file
{noformat}
2015-02-02 14:59:05,407 WARN [AsyncRpcChannel-pool1-t1]
channel.DefaultChannelPipeline: An exceptionCaught() event was fired, and it
reached at the tail of the pipeline. It usually means the last handler in the
pipeline did not handle the exception.
io.netty.channel.ChannelPipelineException:
org.apache.hadoop.hbase.security.SaslClientHandler.handlerAdded() has thrown an
exception; removed.
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:499)
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded(DefaultChannelPipeline.java:481)
at
io.netty.channel.DefaultChannelPipeline.addFirst0(DefaultChannelPipeline.java:114)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:97)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:235)
at
io.netty.channel.DefaultChannelPipeline.addFirst(DefaultChannelPipeline.java:214)
at
org.apache.hadoop.hbase.ipc.AsyncRpcChannel$2.operationComplete(AsyncRpcChannel.java:194)
at
org.apache.hadoop.hbase.ipc.AsyncRpcChannel$2.operationComplete(AsyncRpcChannel.java:157)
at
io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680)
at
io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603)
at
io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563)
at
io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:406)
at
io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:82)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:253)
at
io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:288)
at
io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:528)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
at
io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
at
io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:116)
at java.lang.Thread.run(Thread.java:745)
Caused by: javax.security.sasl.SaslException: GSS initiate failed [Caused by
GSSException: No valid credentials provided (Mechanism level: Failed to find
any Kerberos tgt)]
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
at
org.apache.hadoop.hbase.security.SaslClientHandler.handlerAdded(SaslClientHandler.java:154)
at
io.netty.channel.DefaultChannelPipeline.callHandlerAdded0(DefaultChannelPipeline.java:486)
... 20 more
Caused by: GSSException: No valid credentials provided (Mechanism level: Failed
to find any Kerberos tgt)
at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:147)
at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:121)
at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:223)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at
com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:193)
{noformat}
When set hbase.rpc.client.impl to RpcClientImpl, there seems to be no issue.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13002) Make encryption cipher configurable
Ashish Singhi created HBASE-13002: - Summary: Make encryption cipher configurable Key: HBASE-13002 URL: https://issues.apache.org/jira/browse/HBASE-13002 Project: HBase Issue Type: Improvement Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0, 1.0.1, 1.1.0, 0.98.11 Make encryption cipher configurable currently it is hard coded to AES -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13038) Fix the java doc continuously reported by Hadoop QA
Ashish Singhi created HBASE-13038: - Summary: Fix the java doc continuously reported by Hadoop QA Key: HBASE-13038 URL: https://issues.apache.org/jira/browse/HBASE-13038 Project: HBase Issue Type: Bug Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor Fix the java doc warning introduced after HBASE-12978 commit. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13048) Use hbase.crypto.wal.algorithm in SecureProtobufLogReader while decrypting the data
Ashish Singhi created HBASE-13048: - Summary: Use hbase.crypto.wal.algorithm in SecureProtobufLogReader while decrypting the data Key: HBASE-13048 URL: https://issues.apache.org/jira/browse/HBASE-13048 Project: HBase Issue Type: Bug Affects Versions: 0.98.10 Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor We are using hbase.crypto.wal.algorithm in SecureProtobufLogWriter for encrypting the data but we are using hbase.crypto.key.algorithm in SecureProtobufLogReader while decrypting the data, fix this typo. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13057) Provide client utility to easily enable and disable table replication
Ashish Singhi created HBASE-13057: - Summary: Provide client utility to easily enable and disable table replication Key: HBASE-13057 URL: https://issues.apache.org/jira/browse/HBASE-13057 Project: HBase Issue Type: New Feature Components: Replication Affects Versions: 0.98.10 Reporter: Ashish Singhi Assignee: Ashish Singhi Currently to enable replication for a table user has to 1) Disable the table 2) Set the replication scope 3) enable the table and also create a table with same name and CFs in peer cluster(s). To help user we can provide client API's to enable and disable table replication. So that user has to call only one API i.e., enable table replication which will, 1. Create a table with same name and CFs in the peer cluster(s) 2. Disable the table 3. Set the replication scope 4. Enable the table If user wants to disable table replication then he/she only has to call disable table replication API which will, 1. Disable the table 2. Set the replication scope 3. Enable the table. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13098) HBase Connection Control
Ashish Singhi created HBASE-13098: - Summary: HBase Connection Control Key: HBASE-13098 URL: https://issues.apache.org/jira/browse/HBASE-13098 Project: HBase Issue Type: New Feature Components: security Affects Versions: 0.98.10 Reporter: Ashish Singhi Assignee: Ashish Singhi It is desirable to set the limit on the number of client connections permitted to the HBase server by controlling with certain system variables/parameters. Too many connections to the HBase server imply too many queries and MR jobs running on HBase. This can slow down the performance of the system and lead to denial of service. Hence such connections need to be controlled. Using too many connections may just cause thrashing rather than get more useful work done. This is kind off inspired from http://www.ebaytechblog.com/2014/08/21/quality-of-service-in-hadoop/#.VO2JXXyUe9y -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13111) truncate_preserve command is failing with undefined method error
Ashish Singhi created HBASE-13111:
-
Summary: truncate_preserve command is failing with undefined
method error
Key: HBASE-13111
URL: https://issues.apache.org/jira/browse/HBASE-13111
Project: HBase
Issue Type: Bug
Components: shell
Affects Versions: 1.0.0, 2.0.0, 1.1.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
{noformat}
hbase(main):001:0> truncate_preserve 't1'
Truncating 't1' table (it may take a while):
ERROR: undefined method `getTable' for nil:NilClass
Here is some help for this command:
Disables, drops and recreates the specified table while still maintaing the
previous region boundaries.
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13112) quota.rb, security.rb and visibility_labels.rb leaks connection
Ashish Singhi created HBASE-13112: - Summary: quota.rb, security.rb and visibility_labels.rb leaks connection Key: HBASE-13112 URL: https://issues.apache.org/jira/browse/HBASE-13112 Project: HBase Issue Type: Bug Components: shell Affects Versions: 1.0.0, 2.0.0, 1.1.0 Reporter: Ashish Singhi Assignee: Ashish Singhi quota.rb, security.rb and visibility_labels.rb leaks connection as noted by [~ndimiduk] [here|https://issues.apache.org/jira/browse/HBASE-12833?focusedCommentId=14279465&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14279465] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13137) [0.98] Backport HBASE-13057 - Provide client utility to easily enable and disable table replication
Ashish Singhi created HBASE-13137: - Summary: [0.98] Backport HBASE-13057 - Provide client utility to easily enable and disable table replication Key: HBASE-13137 URL: https://issues.apache.org/jira/browse/HBASE-13137 Project: HBase Issue Type: Sub-task Components: Replication Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 0.98.11 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-12873) [investigate] shell SecurityAdmin and VisibilityLabelsAdmin probably leak connections
[ https://issues.apache.org/jira/browse/HBASE-12873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-12873. --- Resolution: Duplicate Fix Version/s: (was: 0.98.12) (was: 1.1.0) (was: 1.0.1) (was: 2.0.0) Duplicate of HBASE-13112. > [investigate] shell SecurityAdmin and VisibilityLabelsAdmin probably leak > connections > -- > > Key: HBASE-12873 > URL: https://issues.apache.org/jira/browse/HBASE-12873 > Project: HBase > Issue Type: Task > Components: shell >Affects Versions: 0.98.0 >Reporter: Nick Dimiduk >Priority: Minor > Labels: beginner, beginners > > I noticed this over on HBASE-12833. Should look into it and ensure that > proper cleanup is happening. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13205) [branch-1] Backport HBASE-11598 Add simple rpc throttling
Ashish Singhi created HBASE-13205: - Summary: [branch-1] Backport HBASE-11598 Add simple rpc throttling Key: HBASE-13205 URL: https://issues.apache.org/jira/browse/HBASE-13205 Project: HBase Issue Type: Task Components: security Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 1.1.0 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13218) Correct the syntax shown for using ExportSnapshot tool in the book
Ashish Singhi created HBASE-13218:
-
Summary: Correct the syntax shown for using ExportSnapshot tool in
the book
Key: HBASE-13218
URL: https://issues.apache.org/jira/browse/HBASE-13218
Project: HBase
Issue Type: Bug
Components: documentation
Reporter: Ashish Singhi
Assignee: Ashish Singhi
It is
{noformat}
$ bin/hbase class org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
MySnapshot -copy-to hdfs://srv2:8082/hbase -mappers 16
{noformat}
It should be
{noformat}
$ bin/hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot
MySnapshot -copy-to hdfs://srv2:8082/hbase -mappers 16
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13220) set_quota should fail for non-existing table and namespace
Ashish Singhi created HBASE-13220: - Summary: set_quota should fail for non-existing table and namespace Key: HBASE-13220 URL: https://issues.apache.org/jira/browse/HBASE-13220 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0 set_quota command to should throw exception to user when trying to set quota on a non-existing table or namespace same like grant command. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13226) Document enable_table_replication and disable_table_replication shell commands
Ashish Singhi created HBASE-13226: - Summary: Document enable_table_replication and disable_table_replication shell commands Key: HBASE-13226 URL: https://issues.apache.org/jira/browse/HBASE-13226 Project: HBase Issue Type: Sub-task Components: documentation Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13246) Correct the assertion for namespace permissions in tearDown method of TestAccessController
Ashish Singhi created HBASE-13246:
-
Summary: Correct the assertion for namespace permissions in
tearDown method of TestAccessController
Key: HBASE-13246
URL: https://issues.apache.org/jira/browse/HBASE-13246
Project: HBase
Issue Type: Bug
Components: security, test
Affects Versions: 0.98.11
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
In *TestAcessController#tearDown* we should use
{{TEST_TABLE.getNamespaceAsString()}} instead of
{{TEST_TABLE.getNameAsString()}} as the assertion is for namespace.
{code}
assertEquals(0, AccessControlLists.getNamespacePermissions(conf,
TEST_TABLE.getTableName().getNamespaceAsString()).size());
{code}
Some credit goes to [~srikanth235] to indirectly point this out
[here|https://issues.apache.org/jira/browse/HBASE-13241?focusedCommentId=14361995&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14361995]
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13304) hbase cleanup script with --cleanAll argument should clean up acls also
Ashish Singhi created HBASE-13304: - Summary: hbase cleanup script with --cleanAll argument should clean up acls also Key: HBASE-13304 URL: https://issues.apache.org/jira/browse/HBASE-13304 Project: HBase Issue Type: Bug Affects Versions: 0.98.12 Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13357) If maxTables/maxRegions exceeds in a namespace throw QuotaExceededException
Ashish Singhi created HBASE-13357: - Summary: If maxTables/maxRegions exceeds in a namespace throw QuotaExceededException Key: HBASE-13357 URL: https://issues.apache.org/jira/browse/HBASE-13357 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Ashish Singhi Priority: Minor Fix For: 2.0.0 If maximum number of tables or regions count exceeds in a namespace we are throwing DoNotRetryIOException instead we should throw QuotaExceededException. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13386) Backport HBASE-12601 to all active branches other than master
Ashish Singhi created HBASE-13386: - Summary: Backport HBASE-12601 to all active branches other than master Key: HBASE-13386 URL: https://issues.apache.org/jira/browse/HBASE-13386 Project: HBase Issue Type: Sub-task Components: documentation, shell Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13398) Document HBase Quota
Ashish Singhi created HBASE-13398: - Summary: Document HBase Quota Key: HBASE-13398 URL: https://issues.apache.org/jira/browse/HBASE-13398 Project: HBase Issue Type: Bug Components: documentation Reporter: Ashish Singhi As part of this we should document HBASE-11598 and HBASE-8410 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13402) [Backport 1.1] HBASE-12552: listSnapshots should list only owned snapshots for non-super user
Ashish Singhi created HBASE-13402: - Summary: [Backport 1.1] HBASE-12552: listSnapshots should list only owned snapshots for non-super user Key: HBASE-13402 URL: https://issues.apache.org/jira/browse/HBASE-13402 Project: HBase Issue Type: Bug Components: snapshots Affects Versions: 1.1.0 Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 1.1.0 After HBASE-11869 pushed in branch-1. It's time to push HBASE-12552 also. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13411) Misleading error message when request size quota limit exceeds
Ashish Singhi created HBASE-13411: - Summary: Misleading error message when request size quota limit exceeds Key: HBASE-13411 URL: https://issues.apache.org/jira/browse/HBASE-13411 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor Fix For: 2.0.0 User will get the same error message when either number of requests exceeds or request size exceeds. So its better we differentiate them. Thanks to [~mbertozzi] for confirming the same offline. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13438) [branch-1] Backport Basic quota support for namespaces
Ashish Singhi created HBASE-13438: - Summary: [branch-1] Backport Basic quota support for namespaces Key: HBASE-13438 URL: https://issues.apache.org/jira/browse/HBASE-13438 Project: HBase Issue Type: Task Affects Versions: 1.1.0 Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 1.1.0 This task for backporting HBASE-8410, HBase-12936 and HBASE-13357 to branch-1 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13564) Master MBeans are not published
Ashish Singhi created HBASE-13564: - Summary: Master MBeans are not published Key: HBASE-13564 URL: https://issues.apache.org/jira/browse/HBASE-13564 Project: HBase Issue Type: Bug Affects Versions: 1.0.1 Reporter: Ashish Singhi Assignee: Ashish Singhi Master MBeans are not published. For more details check the mailing thread on dev@h.a.o with subject "How to retrieve master metrics from JMX client". -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13593) Quota support for namespace should take restore and clone snapshot into account
Ashish Singhi created HBASE-13593: - Summary: Quota support for namespace should take restore and clone snapshot into account Key: HBASE-13593 URL: https://issues.apache.org/jira/browse/HBASE-13593 Project: HBase Issue Type: Sub-task Affects Versions: 1.1.0 Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0, 1.1.0 Quota support for namespace should take restore and clone snapshot into account. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13644) Expand AC testing coverage to include all combinations of scope and permissions for region interface
Ashish Singhi created HBASE-13644: - Summary: Expand AC testing coverage to include all combinations of scope and permissions for region interface Key: HBASE-13644 URL: https://issues.apache.org/jira/browse/HBASE-13644 Project: HBase Issue Type: Sub-task Components: security, test Reporter: Ashish Singhi Assignee: Ashish Singhi As of now, the tests in TestAccessController and TestAccessController2 doesn't cover all the combinations of Scope and Permissions. Ideally, we should have testing coverage for the entire region interface in [ACL matrix|https://hbase.apache.org/book/appendix_acl_matrix.html]. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13657) Improve test run time
Ashish Singhi created HBASE-13657:
-
Summary: Improve test run time
Key: HBASE-13657
URL: https://issues.apache.org/jira/browse/HBASE-13657
Project: HBase
Issue Type: Umbrella
Components: test
Reporter: Ashish Singhi
In some tests we are doing some operations in {{@Before}} and {{@After}}
annotated methods which are not really required to be done before and after
every test run, instead we can move them in {{@BeforeClass}} and
{{@AfterClass}} annotated methods respectively and hence improve the test run
time.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-13658) Improve the test run time for TestAccessController* classes
Ashish Singhi created HBASE-13658: - Summary: Improve the test run time for TestAccessController* classes Key: HBASE-13658 URL: https://issues.apache.org/jira/browse/HBASE-13658 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi Improve the test run time for TestAccessController* classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13659) Improve test run time for TestMetaWithReplicas class
Ashish Singhi created HBASE-13659: - Summary: Improve test run time for TestMetaWithReplicas class Key: HBASE-13659 URL: https://issues.apache.org/jira/browse/HBASE-13659 Project: HBase Issue Type: Sub-task Components: test Reporter: Ashish Singhi Assignee: Ashish Singhi In TestMetaWithReplicas, start and shutdown of mini cluster is done at start and end of every test in that class respectively, which makes the test class to take more time to complete. Instead we can start and stop the mini cluster only once per the class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13098) HBase Connection Control
[ https://issues.apache.org/jira/browse/HBASE-13098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-13098. --- Resolution: Not A Problem Number of connection/requests to a table/namespace can be controlled using quota. > HBase Connection Control > > > Key: HBASE-13098 > URL: https://issues.apache.org/jira/browse/HBASE-13098 > Project: HBase > Issue Type: New Feature > Components: security >Reporter: Ashish Singhi >Assignee: Ashish Singhi > Attachments: HBASE-13098.patch, HBase Connection Control.pdf > > > It is desirable to set the limit on the number of client connections > permitted to the HBase server by controlling with certain system > variables/parameters. Too many connections to the HBase server imply too many > queries and MR jobs running on HBase. This can slow down the performance of > the system and lead to denial of service. Hence such connections need to be > controlled. Using too many connections may just cause thrashing rather than > get more useful work done. > This is kind off inspired from > http://www.ebaytechblog.com/2014/08/21/quality-of-service-in-hadoop/#.VO2JXXyUe9y -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13820) Zookeeper is failing to start
Ashish Singhi created HBASE-13820:
-
Summary: Zookeeper is failing to start
Key: HBASE-13820
URL: https://issues.apache.org/jira/browse/HBASE-13820
Project: HBase
Issue Type: Bug
Affects Versions: 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Critical
Fix For: 2.0.0
{{bin/start-hbase.sh}} fails to start zookeeper and throws below exception
{noformat}
Exception in thread "main" java.lang.NullPointerException
at
org.apache.hadoop.hbase.zookeeper.ZKServerTool.main(ZKServerTool.java:45)
{noformat}
This have been broken with HBASE-13636
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-13818) manual region split from HBase shell, I found that split command acts incorrectly with hex split keys
[ https://issues.apache.org/jira/browse/HBASE-13818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-13818. --- Resolution: Invalid [~kukuzidian] please use the user mailing lists from http://hbase.apache.org/mail-lists.html if you facing any issues rather than raising jira next time. > manual region split from HBase shell, I found that split command acts > incorrectly with hex split keys > - > > Key: HBASE-13818 > URL: https://issues.apache.org/jira/browse/HBASE-13818 > Project: HBase > Issue Type: Bug > Components: shell >Affects Versions: 0.96.2 >Reporter: zhangjg > > manual region split from HBase shell, I found that split command acts > incorrectly with hex split keys > hbase(main):001:0> split > 'sdb,\x00\x00+Ug\xD60\x00\x00\x01\x00\x10\xC0,1432909366893.6b601fa4eb9e1244d049bde93e340736.' > SLF4J: Class path contains multiple SLF4J bindings. > SLF4J: Found binding in > [jar:file:/data/xiaoju/hbase-0.96.2-hadoop2/lib/phoenix-4.1.0-client-hadoop2.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/data/xiaoju/hbase-0.96.2-hadoop2/lib/slf4j-log4j12-1.6.4.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: Found binding in > [jar:file:/data/xiaoju/hadoop-2.2.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class] > SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an > explanation. > 2015-06-01 11:40:46,986 WARN [main] util.NativeCodeLoader: Unable to load > native-hadoop library for your platform... using builtin-java classes where > applicable > ERROR: Illegal character code:44, <,> at 3. User-space table qualifiers can > only contain 'alphanumeric characters': i.e. [a-zA-Z_0-9-.]: > sdb,"\x00\x00+Ug\xD60\x00\x00\x01\x00\x10\xC0",1432909366893.6b601fa4eb9e1244d049bde93e340736. > Here is some help for this command: > Split entire table or pass a region to split individual region. With the > second parameter, you can specify an explicit split key for the region. > Examples: > split 'tableName' > split 'namespace:tableName' > split 'regionName' # format: 'tableName,startKey,id' > split 'tableName', 'splitKey' > split 'regionName', 'splitKey' -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13925) Use zookeeper multi to clear znodes in ZKProcedureUtil
Ashish Singhi created HBASE-13925: - Summary: Use zookeeper multi to clear znodes in ZKProcedureUtil Key: HBASE-13925 URL: https://issues.apache.org/jira/browse/HBASE-13925 Project: HBase Issue Type: Bug Affects Versions: 0.98.13 Reporter: Ashish Singhi Assignee: Ashish Singhi Address the TODO in ZKProcedureUtil clearChildZNodes() and clearZNodes methods -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13954) Remove HTableInterface#getRowOrBefore related server side code
Ashish Singhi created HBASE-13954: - Summary: Remove HTableInterface#getRowOrBefore related server side code Key: HBASE-13954 URL: https://issues.apache.org/jira/browse/HBASE-13954 Project: HBase Issue Type: Sub-task Reporter: Ashish Singhi As part of HBASE-13214 review, [~anoop.hbase] had a review comment to remove all the server side related code for getRowOrBefore. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-13968) Remove deprecated methods from BufferedMutator class
Ashish Singhi created HBASE-13968: - Summary: Remove deprecated methods from BufferedMutator class Key: HBASE-13968 URL: https://issues.apache.org/jira/browse/HBASE-13968 Project: HBase Issue Type: Sub-task Reporter: Ashish Singhi Fix For: 2.0.0 As part of HBASE-13214 review, [~anoop.hbase] had a review comment on the review board to remove BufferedMutator#getWriteBuffer method as it deprecated since 1.0.0 So as part of this jira we can remove all the deprecated and unused methods from BufferedMutator class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-13659) Improve test run time for TestMetaWithReplicas class
[ https://issues.apache.org/jira/browse/HBASE-13659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-13659. --- Resolution: Won't Fix Assignee: (was: Ashish Singhi) Fix Version/s: (was: 1.2.1) (was: 1.3.0) (was: 1.1.2) (was: 2.0.0) I have closed this as Won't Fix, if it should be something else just let me know. Thanks. > Improve test run time for TestMetaWithReplicas class > > > Key: HBASE-13659 > URL: https://issues.apache.org/jira/browse/HBASE-13659 > Project: HBase > Issue Type: Sub-task > Components: test >Affects Versions: 1.1.0 >Reporter: Ashish Singhi > Attachments: HBASE-13659-branch-1.1-v1.patch, > HBASE-13659-branch-1.1.patch, HBASE-13659.patch, > org.apache.hadoop.hbase.client.TestMetaWithReplicas-output.txt > > > In TestMetaWithReplicas, start and shutdown of mini cluster is done at start > and end of every test in that class respectively, which makes the test class > to take more time to complete. Instead we can start and stop the mini cluster > only once per the class. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14021) Quota table has a wrong description on the UI
Ashish Singhi created HBASE-14021: - Summary: Quota table has a wrong description on the UI Key: HBASE-14021 URL: https://issues.apache.org/jira/browse/HBASE-14021 Project: HBase Issue Type: Bug Components: UI Affects Versions: 1.1.0 Reporter: Ashish Singhi Priority: Minor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14024) ImportTsv is not loading hbase-default.xml
Ashish Singhi created HBASE-14024:
-
Summary: ImportTsv is not loading hbase-default.xml
Key: HBASE-14024
URL: https://issues.apache.org/jira/browse/HBASE-14024
Project: HBase
Issue Type: Bug
Components: mapreduce
Affects Versions: 2.0.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Critical
Fix For: 2.0.0
ImportTsv job is failing with below exception
{noformat}
Exception in thread "main" java.lang.IllegalArgumentException: Can not create a
Path from a null string
at org.apache.hadoop.fs.Path.checkPathArg(Path.java:123)
at org.apache.hadoop.fs.Path.(Path.java:135)
at org.apache.hadoop.fs.Path.(Path.java:89)
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configurePartitioner(HFileOutputFormat2.java:591)
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:441)
at
org.apache.hadoop.hbase.mapreduce.HFileOutputFormat2.configureIncrementalLoad(HFileOutputFormat2.java:406)
at
org.apache.hadoop.hbase.mapreduce.ImportTsv.createSubmittableJob(ImportTsv.java:555)
at org.apache.hadoop.hbase.mapreduce.ImportTsv.run(ImportTsv.java:763)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:772)
{noformat}
{{hbase.fs.tmp.dir}} is set to a default value in hbase-default.xml.
I found that hbase configuration resources from its xml are not loaded into
conf object.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14047) Cleanup deprecated APIs from Cell class
Ashish Singhi created HBASE-14047: - Summary: Cleanup deprecated APIs from Cell class Key: HBASE-14047 URL: https://issues.apache.org/jira/browse/HBASE-14047 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0 Cleanup deprecated APIs from Cell class -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14105) Add shell tests for Snapshot
Ashish Singhi created HBASE-14105: - Summary: Add shell tests for Snapshot Key: HBASE-14105 URL: https://issues.apache.org/jira/browse/HBASE-14105 Project: HBase Issue Type: Sub-task Reporter: Ashish Singhi Assignee: Ashish Singhi -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-10748) hbase-daemon.sh fails to execute with 'sh' command
Ashish Singhi created HBASE-10748: - Summary: hbase-daemon.sh fails to execute with 'sh' command Key: HBASE-10748 URL: https://issues.apache.org/jira/browse/HBASE-10748 Project: HBase Issue Type: Bug Affects Versions: 0.94.11 Reporter: Ashish Singhi hostname:HBASE_HOME/bin # sh hbase-daemon.sh restart master *hbase-daemon.sh: line 188: hbase-daemon.sh: command not found* *hbase-daemon.sh: line 196: hbase-daemon.sh: command not found* -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-10780) HFilePrettyPrinter#processFile should return immediately if file does not exists.
Ashish Singhi created HBASE-10780:
-
Summary: HFilePrettyPrinter#processFile should return immediately
if file does not exists.
Key: HBASE-10780
URL: https://issues.apache.org/jira/browse/HBASE-10780
Project: HBase
Issue Type: Bug
Components: HFile
Affects Versions: 0.94.11
Reporter: Ashish Singhi
Priority: Minor
HFilePrettyPrinter#processFile should return immediately if file does not
exists same like HLogPrettyPrinter#run
{code}
if (!fs.exists(file)) {
System.err.println("ERROR, file doesnt exist: " + file);
}{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-14121) Remove getRowOrBefore related thrift side code
Ashish Singhi created HBASE-14121: - Summary: Remove getRowOrBefore related thrift side code Key: HBASE-14121 URL: https://issues.apache.org/jira/browse/HBASE-14121 Project: HBase Issue Type: Sub-task Components: Thrift Affects Versions: 2.0.0 Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0 HBASE-13954 removes HTableInterface#getRowOrBefore related server side code only thing pending now is thrift side code, which should be done as part of this jira. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14121) Remove getRowOrBefore related thrift side code
[ https://issues.apache.org/jira/browse/HBASE-14121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-14121. --- Resolution: Duplicate Assignee: (was: Ashish Singhi) Fix Version/s: (was: 2.0.0) Handled in HBASE-13954 itself. > Remove getRowOrBefore related thrift side code > -- > > Key: HBASE-14121 > URL: https://issues.apache.org/jira/browse/HBASE-14121 > Project: HBase > Issue Type: Sub-task > Components: Thrift >Affects Versions: 2.0.0 >Reporter: Ashish Singhi > > HBASE-13954 removes HTableInterface#getRowOrBefore related server side code > only thing pending now is thrift side code, which should be done as part of > this jira. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14154) DFS Replication should be configurable at column family level
Ashish Singhi created HBASE-14154: - Summary: DFS Replication should be configurable at column family level Key: HBASE-14154 URL: https://issues.apache.org/jira/browse/HBASE-14154 Project: HBase Issue Type: New Feature Reporter: Ashish Singhi Assignee: Ashish Singhi Priority: Minor There are cases where a user wants to have a control on the number of hfile copies he/she can have in the cluster. For eg: For a test table user would like to have only one copy instead of three(default). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14183) Scanning hbase meta table is failing in master branch
Ashish Singhi created HBASE-14183: - Summary: Scanning hbase meta table is failing in master branch Key: HBASE-14183 URL: https://issues.apache.org/jira/browse/HBASE-14183 Project: HBase Issue Type: Bug Affects Versions: 2.0.0 Reporter: Ashish Singhi Assignee: Ashish Singhi Fix For: 2.0.0 As part of HBASE-14047 cleanup this issue has been introduced. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14396) audit log should record the operation object
[ https://issues.apache.org/jira/browse/HBASE-14396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-14396. --- Resolution: Not A Problem > audit log should record the operation object > - > > Key: HBASE-14396 > URL: https://issues.apache.org/jira/browse/HBASE-14396 > Project: HBase > Issue Type: Bug >Reporter: sunhaitao > > Currently the hbase audit log only records the user and scope,we can't know > which table the user is operating on unless the scope is table. > It would be better to know what's going on if we record the exact table and > column family we are operating on besides the scope. > String logMessage = > "Access " + (result.isAllowed() ? "allowed" : "denied") + " for user " > + (result.getUser() != null ? result.getUser().getShortName() : > "UNKNOWN") > + "; reason: " + result.getReason() + "; remote address: " > + (remoteAddr != null ? remoteAddr : "") + "; request: " + > result.getRequest() > + "; context: " + result.toContextString(); -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (HBASE-14396) audit log should record the operation object
[ https://issues.apache.org/jira/browse/HBASE-14396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi reopened HBASE-14396: --- > audit log should record the operation object > - > > Key: HBASE-14396 > URL: https://issues.apache.org/jira/browse/HBASE-14396 > Project: HBase > Issue Type: Bug >Reporter: sunhaitao > > Currently the hbase audit log only records the user and scope,we can't know > which table the user is operating on unless the scope is table. > It would be better to know what's going on if we record the exact table and > column family we are operating on besides the scope. > String logMessage = > "Access " + (result.isAllowed() ? "allowed" : "denied") + " for user " > + (result.getUser() != null ? result.getUser().getShortName() : > "UNKNOWN") > + "; reason: " + result.getReason() + "; remote address: " > + (remoteAddr != null ? remoteAddr : "") + "; request: " + > result.getRequest() > + "; context: " + result.toContextString(); -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (HBASE-14419) Error: Could not find or load main class org.apache.hadoop.hbase.util.HBaseConfTool
[ https://issues.apache.org/jira/browse/HBASE-14419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-14419. --- Resolution: Invalid Please write in to u...@hbase.apache.org to help troubleshooting issues. Jira is for the project development tracker. > Error: Could not find or load main class > org.apache.hadoop.hbase.util.HBaseConfTool > --- > > Key: HBASE-14419 > URL: https://issues.apache.org/jira/browse/HBASE-14419 > Project: HBase > Issue Type: Bug > Environment: centos7 >Reporter: chenlong >Priority: Critical > > [root@localhost local]# ll > lrwxrwxrwx. 1 root root 11 Sep 12 21:34 hbase -> hbase-1.1.2 > drwxr-xr-x. 30 root root 4096 Sep 12 21:34 hbase-1.1.2 > [root@localhost local]# ./hbase/bin/start-hbase.sh > Error: Could not find or load main class > org.apache.hadoop.hbase.util.HBaseConfTool > Error: Could not find or load main class > org.apache.hadoop.hbase.zookeeper.ZKServerTool > starting master, logging to > /usr/local/hbase/logs/hbase-root-master-localhost.out > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option PermSize=128m; > support was removed in 8.0 > Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=128m; > support was removed in 8.0 > Error: Could not find or load main class > org.apache.hadoop.hbase.master.HMaster > starting regionserver, logging to > /usr/local/hbase/logs/hbase-root-1-regionserver-localhost.out > Error: Could not find or load main class > org.apache.hadoop.hbase.regionserver.HRegionServer > why show this error ? > It exist 。 > [root@localhost local]# find ./ -name HBaseConfTool.class > ./hbase-1.1.2/hbase-server/target/classes/org/apache/hadoop/hbase/util/HBaseConfTool.class > /etc/profle: > export JAVA_HOME=/usr/local/jdk1.8.0_20 > export HBASE_HOME=/usr/local/hbase > export PATH=$JAVA_HOME/bin:$HBASE_HOME/bin:$PATH > export > CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar:$HBASE_HOME/hbase-server/target/classes > I add $HBASE_HOME/hbase-server/target/classes, but it still not find class > file。version 0.98xx has no this problem，why the version 1.1.2 so eggache？ I > am just a newer，geting start follow official docs, but can not run。I am so > sad。。。sos。。。 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14491) ReplicationSource#countDistinctRowKeys code logic is not correct
Ashish Singhi created HBASE-14491:
-
Summary: ReplicationSource#countDistinctRowKeys code logic is not
correct
Key: HBASE-14491
URL: https://issues.apache.org/jira/browse/HBASE-14491
Project: HBase
Issue Type: Bug
Reporter: Ashish Singhi
Assignee: Ashish Singhi
Priority: Minor
{code}
Cell lastCell = cells.get(0);
for (int i = 0; i < edit.size(); i++) {
if (!CellUtil.matchingRow(cells.get(i), lastCell)) {
distinctRowKeys++;
}
}
{code}
The above logic for finding the distinct row keys in the list needs to be
corrected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-14491) ReplicationSource#countDistinctRowKeys code logic is not correct
[
https://issues.apache.org/jira/browse/HBASE-14491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashish Singhi resolved HBASE-14491.
---
Resolution: Fixed
Assignee: (was: Ashish Singhi)
Fixed by Enis as part of HBASE-14501.
> ReplicationSource#countDistinctRowKeys code logic is not correct
>
>
> Key: HBASE-14491
> URL: https://issues.apache.org/jira/browse/HBASE-14491
> Project: HBase
> Issue Type: Bug
>Reporter: Ashish Singhi
>Priority: Minor
>
> {code}
> Cell lastCell = cells.get(0);
> for (int i = 0; i < edit.size(); i++) {
> if (!CellUtil.matchingRow(cells.get(i), lastCell)) {
> distinctRowKeys++;
> }
> }
> {code}
> The above logic for finding the distinct row keys in the list needs to be
> corrected.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (HBASE-14685) Procedure Id is not set for MasterRpcServices#modifyTable
Ashish Singhi created HBASE-14685: - Summary: Procedure Id is not set for MasterRpcServices#modifyTable Key: HBASE-14685 URL: https://issues.apache.org/jira/browse/HBASE-14685 Project: HBase Issue Type: Sub-task Affects Versions: 2.0.0 Reporter: Ashish Singhi Fix For: 2.0.0 Procedure Id is not set for MasterRpcServices#modifyTable -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14937) Make rpc call timeout for replication adaptive
Ashish Singhi created HBASE-14937: - Summary: Make rpc call timeout for replication adaptive Key: HBASE-14937 URL: https://issues.apache.org/jira/browse/HBASE-14937 Project: HBase Issue Type: Improvement Reporter: Ashish Singhi Assignee: Ashish Singhi When peer cluster replication is disabled and lot of writes are happening in active cluster and later on peer cluster replication is enabled then there are chances that replication requests to peer cluster may time out. This is possible after HBASE-13153 and it can also happen with many and many WAL data replication still pending to replicate. Approach to this problem will be discussed in the comments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14938) Limit to and fro requests size from ZK in bulk loaded hfile replication
Ashish Singhi created HBASE-14938: - Summary: Limit to and fro requests size from ZK in bulk loaded hfile replication Key: HBASE-14938 URL: https://issues.apache.org/jira/browse/HBASE-14938 Project: HBase Issue Type: Improvement Reporter: Ashish Singhi Assignee: Ashish Singhi In ZK the maximum allowable size of the data array is 1 MB. Until we have fixed HBASE-10295 we need to handle this. Approach to this problem will be discussed in the comments section. Note: We have done internal testing with more than 3k nodes in ZK yet to be replicated. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14939) Document bulk loaded hfile replication
Ashish Singhi created HBASE-14939: - Summary: Document bulk loaded hfile replication Key: HBASE-14939 URL: https://issues.apache.org/jira/browse/HBASE-14939 Project: HBase Issue Type: Task Components: documentation Reporter: Ashish Singhi Assignee: Ashish Singhi After HBASE-13153 is committed we need to add that information under the Cluster Replication section in HBase book. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-14950) Create table with AC fails when quota is enabled
Ashish Singhi created HBASE-14950:
-
Summary: Create table with AC fails when quota is enabled
Key: HBASE-14950
URL: https://issues.apache.org/jira/browse/HBASE-14950
Project: HBase
Issue Type: Bug
Affects Versions: 0.98.16, 1.1.2, 2.0.0, 1.2.0, 1.3.0
Reporter: Ashish Singhi
Priority: Minor
Scenario:
1. Set hbase.quota.enabled to true
2. As per the [ACL matrix |
http://hbase.apache.org/book.html#appendix_acl_matrix] for create table, grant
'@group1', 'C', '@ns1'
3. create 't1', 'd' -- *Failed*
{noformat}
ERROR: java.io.IOException: Namespace Descriptor found null for default This is
unexpected.
at
org.apache.hadoop.hbase.namespace.NamespaceStateManager.checkAndUpdateNamespaceTableCount(NamespaceStateManager.java:170)
at
org.apache.hadoop.hbase.namespace.NamespaceAuditor.checkQuotaToCreateTable(NamespaceAuditor.java:76)
at
org.apache.hadoop.hbase.quotas.MasterQuotaManager.checkNamespaceTableAndRegionQuota(MasterQuotaManager.java:312)
at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1445)
at
org.apache.hadoop.hbase.master.MasterRpcServices.createTable(MasterRpcServices.java:428)
at
org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java:49404)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2136)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:107)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
{noformat}
When quota is enabled, then as part of createTable we internally also call
getNamespaceDescriptor which needs 'A' privilege.
So when quota is enabled we need both C and A permission to create a table. ACL
Matrix needs to be updated.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (HBASE-14957) issue in starting start-hbase.sh
[ https://issues.apache.org/jira/browse/HBASE-14957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashish Singhi resolved HBASE-14957. --- Resolution: Invalid Fix Version/s: (was: 0.94.9) > issue in starting start-hbase.sh > > > Key: HBASE-14957 > URL: https://issues.apache.org/jira/browse/HBASE-14957 > Project: HBase > Issue Type: Bug > Components: build >Affects Versions: 0.94.9 >Reporter: gaurav kandpal >Priority: Blocker > Original Estimate: 12h > Remaining Estimate: 12h > > for initializing nutch, after configuring nutch whent I am trying to kickoff > start-hbase.sh I am getting the below error. > /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../conf/hbase-env.sh: > line 101: $'\r': command not found > /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../conf/hbase-env.sh: > line 104: $'\r': command not found > /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../conf/hbase-env.sh: > line 107: $'\r': command not found > /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../conf/hbase-env.sh: > line 110: $'\r': command not found > /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../conf/hbase-env.sh: > line 115: $'\r': command not found > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > 'nrecognized VM option 'UseConcMarkSweepGC > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > 'nrecognized VM option 'UseConcMarkSweepGC > starting master, logging to > /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/../logs/hbase-Gaurav.Kandpal-master-gauravk.out > localhost: > /cygdrive/c/users/gaurav.kandpal/desktop/test/hbase-0.94.9.tar/hbase-0.94.9/hbase-0.94.9/bin/regionservers.sh: > line 64: ssh: command not found > reference URL is > https://gist.github.com/xrstf/b48a970098a8e76943b9 > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (HBASE-15018) Inconsistent way of handling TimeoutException in the rpc client implemenations
Ashish Singhi created HBASE-15018:
-
Summary: Inconsistent way of handling TimeoutException in the rpc
client implemenations
Key: HBASE-15018
URL: https://issues.apache.org/jira/browse/HBASE-15018
Project: HBase
Issue Type: Bug
Affects Versions: 1.1.0, 2.0.0, 1.2.0
Reporter: Ashish Singhi
Assignee: Ashish Singhi
If there is any rpc timeout when using RpcClientImpl then we wrap the exception
in IOE and throw it,
{noformat}
2015-11-16 04:05:24,935 WARN [main-EventThread.replicationSource,peer2]
regionserver.HBaseInterClusterReplicationEndpoint: Can't replicate because of a
local or network error:
java.io.IOException: Call to host-XX:16040 failed on local exception:
org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=510, waitTime=180001,
operationTimeout=18 expired.
at
org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1271)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1239)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
at
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.replicateWALEntry(AdminProtos.java:25690)
at
org.apache.hadoop.hbase.protobuf.ReplicationProtbufUtil.replicateWALEntry(ReplicationProtbufUtil.java:77)
at
org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:322)
at
org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:308)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=510,
waitTime=180001, operationTimeout=18 expired.
at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1213)
... 10 more
{noformat}
But that isn't case with AsyncRpcClient, we don't wrap and throw
CallTimeoutException as it is.
{noformat}
2015-12-21 14:27:33,093 WARN
[RS_OPEN_REGION-host-XX:16201-0.replicationSource.host-XX%2C16201%2C1450687255593,1]
regionserver.HBaseInterClusterReplicationEndpoint: Can't replicate because of
a local or network error:
org.apache.hadoop.hbase.ipc.CallTimeoutException: callId=2,
method=ReplicateWALEntry, rpcTimeout=60, param {TODO: class
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$ReplicateWALEntryRequest}
at
org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:257)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217)
at
org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:295)
at
org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.replicateWALEntry(AdminProtos.java:23707)
at
org.apache.hadoop.hbase.protobuf.ReplicationProtbufUtil.replicateWALEntry(ReplicationProtbufUtil.java:73)
at
org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:387)
at
org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:370)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
{noformat}
I think we should have same behavior across both the implementations.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
