[jira] [Commented] (HBASE-10576) Custom load balancer to co-locate the regions of two tables which are having same split keys
[
https://issues.apache.org/jira/browse/HBASE-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980728#comment-13980728
]
rajeshbabu commented on HBASE-10576:
If I understand the [~jeffreyz] idea in the above
comment(https://issues.apache.org/jira/browse/HBASE-10576?focusedCommentId=13978998page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13978998)
correctly,
I think we can do the same without any changes in the kernel.
The steps are like this
1) Before creating index table we need to disable user table.
2) create index table with the same split keys as user table and disable it. So
next time onwards index regions wont be assigned anymore by AM.(Need not have
any special state like shadow)
3) Enable user table. While opening user region we can get corresponing index
region from meta,htd from namenode and just open it with below method
{code}
public static HRegion openHRegion(final HRegionInfo info,
final HTableDescriptor htd, final HLog wal,
final Configuration conf)
throws IOException {
return openHRegion(info, htd, wal, conf, null, null);
}
{code}
After taht maintain user region and index region mapping.
4) While scanning if query conditions involves covering indexes we can just
scan index region in the hooks and skip scanning user region through bypass.
Otherwise get rowkeys from index region and seek to rowkey in the user region
and get required information.
Custom load balancer to co-locate the regions of two tables which are having
same split keys
Key: HBASE-10576
URL: https://issues.apache.org/jira/browse/HBASE-10576
Project: HBase
Issue Type: Sub-task
Components: Balancer
Reporter: rajeshbabu
Assignee: rajeshbabu
Attachments: HBASE-10536_v2.patch, HBASE-10576.patch
To support local indexing both user table and index table should have same
split keys. This issue to provide custom balancer to colocate the regions of
two tables which are having same split keys.
This helps in Phoenix as well.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980729#comment-13980729 ] Mikhail Antonov commented on HBASE-10985: - Looking at the patch, a few things needs some thinking. Actually, those are things similar in design to closing/opening region, unless I'm missing something? So I would try to unify some requirements to consensus classes, I guess. May be add to design doc too. That said, some of the items below may be a bit tricky to achieve at first cut.. 1) ConsensusProvider should only be instantiated / initialize()-d / start()-ed / stop()-ped once in Server impl (HMaster or HRegionServer) 2) SplitTransactionConsensus shouldn't use ZK api, i.e. shouldn't do any of this: - import ZK classes - take params like znodeVersion or ZK watcher (znode version and stuff like that we can encapsulate in classes like SplitTransactionProposal, or SplitTransactionDetails, and this SplitTransactionDetails class needs to be constructed from..protobuf request or something like that, while ZKWatcher should be retrieved from ZkConsensusProvider internally in consensus class), - throw KeeperException (we may need to introduce ConsensusException? didn't see the need so far, but sure we may need to) - have methods named after ZK concepts (like transitionNode(), should be..I don't know..finishTransaction() or something like that?) 3) I guess we shouldn't need to pass Server or RegionServerServices into consensus class methods, we should be able to retrieve them from consensusProvider. 4) SplitTransaction which we abstract shouldn't reference ZK API like KeeperException (I guess it was easier for closing regions somewhat, but here we need to either handle all KeeperException-s in ZkSplitTransactionConsensus, or add ConsensusException). Let me know if I missed something. Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10985) Decouple Split Transaction from Zookeeper
[ https://issues.apache.org/jira/browse/HBASE-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980730#comment-13980730 ] Mikhail Antonov commented on HBASE-10985: - I just meant if we have like.. a checklist for abstracting-zk-patches, reviews would be easier for the folks, I guess. Decouple Split Transaction from Zookeeper - Key: HBASE-10985 URL: https://issues.apache.org/jira/browse/HBASE-10985 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Sergey Soldatov Attachments: HBASE-10985.patch, HBASE-10985.patch, HBASE-10985.patch As part of HBASE-10296 SplitTransaction should be decoupled from Zookeeper. This is an initial patch for review. At the moment the consensus provider placed directly to SplitTransaction to minimize affected code. In the ideal world it should be done in HServer. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-10576) Custom load balancer to co-locate the regions of two tables which are having same split keys
[
https://issues.apache.org/jira/browse/HBASE-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980728#comment-13980728
]
rajeshbabu edited comment on HBASE-10576 at 4/25/14 6:06 AM:
-
If I understand the [~jeffreyz] idea in the above
comment(https://issues.apache.org/jira/browse/HBASE-10576?focusedCommentId=13978998page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13978998)
correctly,
I think we can do the same without any changes in the kernel.
The steps are like this
1) Before creating index table we need to disable user table.
2) create index table with the same split keys as user table and disable it. So
next time onwards index regions wont be assigned anymore by AM.(Need not have
any special state like shadow)
3) Enable user table. While opening user region we can get corresponing index
region from meta,htd from namenode and just open it with below method
{code}
public static HRegion openHRegion(final HRegionInfo info,
final HTableDescriptor htd, final HLog wal,
final Configuration conf)
throws IOException {
return openHRegion(info, htd, wal, conf, null, null);
}
{code}
After that maintain user region and index region mapping.
4) While scanning if query conditions involves covering indexes we can just
scan index region in the hooks and skip scanning user region by bypass.
Otherwise get rowkeys from index region and seek to rowkey in the user region
and get required information.
was (Author: rajesh23):
If I understand the [~jeffreyz] idea in the above
comment(https://issues.apache.org/jira/browse/HBASE-10576?focusedCommentId=13978998page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13978998)
correctly,
I think we can do the same without any changes in the kernel.
The steps are like this
1) Before creating index table we need to disable user table.
2) create index table with the same split keys as user table and disable it. So
next time onwards index regions wont be assigned anymore by AM.(Need not have
any special state like shadow)
3) Enable user table. While opening user region we can get corresponing index
region from meta,htd from namenode and just open it with below method
{code}
public static HRegion openHRegion(final HRegionInfo info,
final HTableDescriptor htd, final HLog wal,
final Configuration conf)
throws IOException {
return openHRegion(info, htd, wal, conf, null, null);
}
{code}
After taht maintain user region and index region mapping.
4) While scanning if query conditions involves covering indexes we can just
scan index region in the hooks and skip scanning user region through bypass.
Otherwise get rowkeys from index region and seek to rowkey in the user region
and get required information.
Custom load balancer to co-locate the regions of two tables which are having
same split keys
Key: HBASE-10576
URL: https://issues.apache.org/jira/browse/HBASE-10576
Project: HBase
Issue Type: Sub-task
Components: Balancer
Reporter: rajeshbabu
Assignee: rajeshbabu
Attachments: HBASE-10536_v2.patch, HBASE-10576.patch
To support local indexing both user table and index table should have same
split keys. This issue to provide custom balancer to colocate the regions of
two tables which are having same split keys.
This helps in Phoenix as well.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (HBASE-10289) Avoid random port usage by default JMX Server. Create Custome JMX server
[ https://issues.apache.org/jira/browse/HBASE-10289?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13979398#comment-13979398 ] Qiang Tian edited comment on HBASE-10289 at 4/25/14 6:07 AM: - hi [~nijel], /proc/sys/net/ipv4/ip_local_port_range looks not so good as it limits port resource. I just happen to find when com.sun.management.jmxremote.local.only=false is set, there is only 1 random port, i.e.: export HBASE_MASTER_OPTS=$HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=61100 -Dcom.sun.management .jmxremote.local.only=false without com.sun.management.jmxremote.local.only=false: [root@test tmp]# netstat -nltp |grep 61100 tcp0 0 :::61100:::* LISTEN 1989249/java [root@test tmp]# netstat -nltp |grep 1989249 tcp0 0 :::61100:::* LISTEN 1989249/java tcp0 0 :::4159 :::* LISTEN 1989249/java tcp0 0 :::192.168.1.101:6 :::* LISTEN 1989249/java tcp0 0 :::61320:::* LISTEN 1989249/java tcp0 0 :::60010:::* LISTEN 1989249/java with com.sun.management.jmxremote.local.only=false [root@test tmp]# netstat -nltp |grep 61100 tcp0 0 :::61100:::* LISTEN 2021776/java [root@test tmp]# netstat -nltp |grep 2021776 tcp0 0 :::61100:::* LISTEN 2021776/java tcp0 0 :::2174 :::* LISTEN 2021776/java tcp0 0 :::192.168.1.101:6 :::* LISTEN 2021776/java tcp0 0 :::60010:::* LISTEN 2021776/java I tried jconsole can work locally and remotely. could you also have a try? ps below is the description: http://www.oracle.com/technetwork/java/javase/compatibility-417013.html Area: JMX Synopsis: New Property for JMX RMI Connector Server Description: The new property, com.sun.management.jmxremote.local.only, when true (the default) indicates that the local JMX RMI connector will only accept connection requests from local interfaces. Setting this property to false restores JDK 6 behavior, but is not recommended because the local JMX RMI connector server will accept connection requests from both local and remote interfaces. For remote management, the remote JMX RMI connector server should be used with authentication and SLL/TLS encyrption enabled. Nature of Incompatibility: behavioral Regarding to the RMI server port, we could: a)using parameter com.sun.management.jmxremote.rmi.port after upgrade to jdk7. this is the simplest way. b)using existing artifcat catalina-jmx-remote.jar c)implement by ourselves as you mentioned. was (Author: tianq): hi [~nijel], /proc/sys/net/ipv4/ip_local_port_range looks not good solution as it limits port resource. I just happen to find when com.sun.management.jmxremote.local.only=false is set, there is only 1 random port, i.e.: export HBASE_MASTER_OPTS=$HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=61100 -Dcom.sun.management .jmxremote.local.only=false without com.sun.management.jmxremote.local.only=false: [root@test tmp]# netstat -nltp |grep 61100 tcp0 0 :::61100:::* LISTEN 1989249/java [root@test tmp]# netstat -nltp |grep 1989249 tcp0 0 :::61100:::* LISTEN 1989249/java tcp0 0 :::4159 :::* LISTEN 1989249/java tcp0 0 :::9.181.64.235:6 :::* LISTEN 1989249/java tcp0 0 :::61320:::* LISTEN 1989249/java tcp0 0 :::60010:::* LISTEN 1989249/java with com.sun.management.jmxremote.local.only=false [root@test tmp]# netstat -nltp |grep 61100 tcp0 0 :::61100:::* LISTEN 2021776/java [root@test tmp]# netstat -nltp |grep 2021776 tcp0 0 :::61100:::* LISTEN 2021776/java tcp0 0 :::2174 :::* LISTEN 2021776/java tcp0 0 :::9.181.64.235:6 :::* LISTEN 2021776/java tcp0 0
[jira] [Assigned] (HBASE-11073) Hide ZooKeeperWatcher and dependent ZK Listeners inside of a consensus impl
[ https://issues.apache.org/jira/browse/HBASE-11073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov reassigned HBASE-11073: --- Assignee: Mikhail Antonov Hide ZooKeeperWatcher and dependent ZK Listeners inside of a consensus impl --- Key: HBASE-11073 URL: https://issues.apache.org/jira/browse/HBASE-11073 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Affects Versions: 0.99.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov (copied from the design doc): Redesign ZooKeeperListeners to abstract them from ZooKeeper API. Current design: The ZooKeeperListener interface is implemented by a number of classes and has 4 low-level API methods nodeCreated(), nodeDeleted(), nodeDataChanged(), nodeChildrenChanged() ZooKeeperWatcher maintains list of listeners and upon receiving ZK event calls all registered listeners and passes them affected znode’s path. Proposed design: Create new interface ConsensusHandler that contains a single method, handle(Agreement agreement). Modify the classes currently implementing the ZooKeeperListener interface so they implement the new ConsensusHandler interface. In their handle(...) method they will typecast the agreement instance to specific type (like AssignmentAgreement), extract the information encoded in it, and do their work Create an interface called AgreementLearner which will maintain a list of all ConsensusHandlers as a map of znode prefixes to handler instance. The ZK implementation, ZkAgreementLearner, will be used to dispatch events to the appropriate namespaces in the ZooKeeperWatcher by parsing the znodes-level event, constructing Agreement objects of proper type and dispatching them to appropriate ConsensusHandlers. Modify ZooKeeperWatcher: remove the internal collection of ZooKeeperListeners move znode prefixes to ZkAgreementLearner class in the handle(...) method pass the event to ZkAgreementLearner for further dispatching and execution. The following is a list of listeners to take care of (some are described in the items above already): ZKLeaderManager ZooKeeperNodeTracker TableHFileArchiveTracker ActiveMasterManager AssignmentManager SplitLogManager ZKProcedureUtil SplitLogWorker ZKPermissionWatcher ZKSecretWatcher ZKNamespaceManager DeletionListener DrainingServerTracker RecoveringRegionWatcher RegionServerTracker ReplicationPeer MasterAddressTracker MetaRegionTracker ClusterStatusTracker LoadBalancerTracker -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11053) Change DeleteTracker APIs to work with Cell
[ https://issues.apache.org/jira/browse/HBASE-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11053: --- Status: Patch Available (was: Open) Change DeleteTracker APIs to work with Cell --- Key: HBASE-11053 URL: https://issues.apache.org/jira/browse/HBASE-11053 Project: HBase Issue Type: Sub-task Affects Versions: 0.98.1, 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11053.patch, HBASE-11053_1.patch DeleteTracker interface (marked as Private) should work with Cells. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11053) Change DeleteTracker APIs to work with Cell
[ https://issues.apache.org/jira/browse/HBASE-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11053: --- Attachment: HBASE-11053_1.patch Resubmitting for QA to run again. Change DeleteTracker APIs to work with Cell --- Key: HBASE-11053 URL: https://issues.apache.org/jira/browse/HBASE-11053 Project: HBase Issue Type: Sub-task Affects Versions: 0.98.1, 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11053.patch, HBASE-11053_1.patch DeleteTracker interface (marked as Private) should work with Cells. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11053) Change DeleteTracker APIs to work with Cell
[ https://issues.apache.org/jira/browse/HBASE-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11053: --- Status: Open (was: Patch Available) Change DeleteTracker APIs to work with Cell --- Key: HBASE-11053 URL: https://issues.apache.org/jira/browse/HBASE-11053 Project: HBase Issue Type: Sub-task Affects Versions: 0.98.1, 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11053.patch, HBASE-11053_1.patch DeleteTracker interface (marked as Private) should work with Cells. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10278) Provide better write predictability
[ https://issues.apache.org/jira/browse/HBASE-10278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980752#comment-13980752 ] Jonathan Hsieh commented on HBASE-10278: mhlog is with the multihlog patch in (on a trunk branch from last week) and on by default. trunk was yesterday's trunk. The two numbers are the average and stdev of 10 of the same consecutive runs. 99 is for the 99%tile, and 99.99 is for the 99.99%tile as reported by ITMTTR. So to interpret the most interesting results with the patch has a higher percentage that are adversely affected with high latencies. The 99%tile latency for a puts when killing an RS using multihlog is 5.1ms, with a stddev of 0.3ms. The 99.99%tile is 37647.2ms average with a 9888.0ms stddev. using the old log it is 735.2 and 673.7 ms respetively. Next time i run this I'll also get the worst cases, my guess is that they are the same or there is some constant extra latency due to two wals in the multilog case. Provide better write predictability --- Key: HBASE-10278 URL: https://issues.apache.org/jira/browse/HBASE-10278 Project: HBase Issue Type: New Feature Reporter: Himanshu Vashishtha Assignee: Himanshu Vashishtha Attachments: 10278-trunk-v2.1.patch, 10278-trunk-v2.1.patch, 10278-wip-1.1.patch, Multiwaldesigndoc.pdf, SwitchWriterFlow.pptx Currently, HBase has one WAL per region server. Whenever there is any latency in the write pipeline (due to whatever reasons such as n/w blip, a node in the pipeline having a bad disk, etc), the overall write latency suffers. Jonathan Hsieh and I analyzed various approaches to tackle this issue. We also looked at HBASE-5699, which talks about adding concurrent multi WALs. Along with performance numbers, we also focussed on design simplicity, minimum impact on MTTR Replication, and compatibility with 0.96 and 0.98. Considering all these parameters, we propose a new HLog implementation with WAL Switching functionality. Please find attached the design doc for the same. It introduces the WAL Switching feature, and experiments/results of a prototype implementation, showing the benefits of this feature. The second goal of this work is to serve as a building block for concurrent multiple WALs feature. Please review the doc. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11054) Create new hook in StoreScanner to help user creating his own delete tracker
[ https://issues.apache.org/jira/browse/HBASE-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11054: --- Status: Open (was: Patch Available) Create new hook in StoreScanner to help user creating his own delete tracker Key: HBASE-11054 URL: https://issues.apache.org/jira/browse/HBASE-11054 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11054.patch In case of Visibility labels, to support deletes we need to consider the visibility expression per version inorder to identify the matching visibility expression. Tracking the visibility tags along with the delete tracker would make things easier rather than handling during scans/compactions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11054) Create new hook in StoreScanner to help user creating his own delete tracker
[ https://issues.apache.org/jira/browse/HBASE-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11054: --- Attachment: HBASE-11054_1.patch Updated patch. Renames the preHook to postHook as now this hook is called after the ScanDeleteTracker is created and the same is passed as param to the hook. If no new DeleteTracker is created in the hook the original hook is returned back. Create new hook in StoreScanner to help user creating his own delete tracker Key: HBASE-11054 URL: https://issues.apache.org/jira/browse/HBASE-11054 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11054.patch, HBASE-11054_1.patch In case of Visibility labels, to support deletes we need to consider the visibility expression per version inorder to identify the matching visibility expression. Tracking the visibility tags along with the delete tracker would make things easier rather than handling during scans/compactions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Comment Edited] (HBASE-11054) Create new hook in StoreScanner to help user creating his own delete tracker
[
https://issues.apache.org/jira/browse/HBASE-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980755#comment-13980755
]
ramkrishna.s.vasudevan edited comment on HBASE-11054 at 4/25/14 7:17 AM:
-
Updated patch. Renames the preHook to postHook as now this hook is called
after the ScanDeleteTracker is created and the same is passed as param to the
hook.
If no new DeleteTracker is created in the delete tracker the original hook is
returned back.
{Edit} hook to deleteTracker.
was (Author: ram_krish):
Updated patch. Renames the preHook to postHook as now this hook is called
after the ScanDeleteTracker is created and the same is passed as param to the
hook.
If no new DeleteTracker is created in the hook the original hook is returned
back.
Create new hook in StoreScanner to help user creating his own delete tracker
Key: HBASE-11054
URL: https://issues.apache.org/jira/browse/HBASE-11054
Project: HBase
Issue Type: Sub-task
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.0, 0.98.2
Attachments: HBASE-11054.patch, HBASE-11054_1.patch
In case of Visibility labels, to support deletes we need to consider the
visibility expression per version inorder to identify the matching visibility
expression.
Tracking the visibility tags along with the delete tracker would make things
easier rather than handling during scans/compactions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (HBASE-11054) Create new hook in StoreScanner to help user creating his own delete tracker
[
https://issues.apache.org/jira/browse/HBASE-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980755#comment-13980755
]
ramkrishna.s.vasudevan edited comment on HBASE-11054 at 4/25/14 7:18 AM:
-
Updated patch. Renames the preHook to postHook as now this hook is called
after the ScanDeleteTracker is created and the same is passed as param to the
hook.
If no new DeleteTracker is created in the hook the original deletetracker is
returned back.
{Edit} hook to deleteTracker.
was (Author: ram_krish):
Updated patch. Renames the preHook to postHook as now this hook is called
after the ScanDeleteTracker is created and the same is passed as param to the
hook.
If no new DeleteTracker is created in the delete tracker the original hook is
returned back.
{Edit} hook to deleteTracker.
Create new hook in StoreScanner to help user creating his own delete tracker
Key: HBASE-11054
URL: https://issues.apache.org/jira/browse/HBASE-11054
Project: HBase
Issue Type: Sub-task
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.0, 0.98.2
Attachments: HBASE-11054.patch, HBASE-11054_1.patch
In case of Visibility labels, to support deletes we need to consider the
visibility expression per version inorder to identify the matching visibility
expression.
Tracking the visibility tags along with the delete tracker would make things
easier rather than handling during scans/compactions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11054) Create new hook in StoreScanner to help user creating his own delete tracker
[ https://issues.apache.org/jira/browse/HBASE-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11054: --- Status: Patch Available (was: Open) Create new hook in StoreScanner to help user creating his own delete tracker Key: HBASE-11054 URL: https://issues.apache.org/jira/browse/HBASE-11054 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11054.patch, HBASE-11054_1.patch In case of Visibility labels, to support deletes we need to consider the visibility expression per version inorder to identify the matching visibility expression. Tracking the visibility tags along with the delete tracker would make things easier rather than handling during scans/compactions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11074) Have PE emit histogram stats as it runs rather than dump once at end of test
[
https://issues.apache.org/jira/browse/HBASE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980766#comment-13980766
]
Hadoop QA commented on HBASE-11074:
---
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641873/11074.txt
against trunk revision .
ATTACHMENT ID: 12641873
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9397//console
This message is automatically generated.
Have PE emit histogram stats as it runs rather than dump once at end of test
Key: HBASE-11074
URL: https://issues.apache.org/jira/browse/HBASE-11074
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: stack
Assignee: stack
Priority: Minor
Attachments: 11074.txt
PE emits progress reading and writing. Add to the progress emission current
histogram snapshot readings. Means don' t have to wait till test completes
to get idea of latencies. Here is sample:
{code}
2014-04-24 22:47:28,085 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/188730/1048576, Min=137.00, Mean=578.74, Max=5152756.00, StdDev=11884.79,
95th=1590.00, 99th=2950.68
2014-04-24 22:47:32,465 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/199215/1048576, Min=137.00, Mean=570.19, Max=5152756.00, StdDev=11591.56,
95th=1543.00, 99th=2911.00
2014-04-24 22:47:37,334 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/209700/1048576, Min=137.00, Mean=564.80, Max=5152756.00, StdDev=11317.96,
95th=1480.00, 99th=2863.00
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10576) Custom load balancer to co-locate the regions of two tables which are having same split keys
[ https://issues.apache.org/jira/browse/HBASE-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980824#comment-13980824 ] rajeshbabu commented on HBASE-10576: [~stack] bq. Please tell me more how this balancer works. It is not clear skimming the code. 'colocate' is how you term data and index tables going together? Main idea of the balancer is to select same plan for a user region and it's corresponding index region at any time. The custom balancer is wrapper on top of normal balancer like StochasticLoadBalancer(which will be used as delegator). The balancer maintains the region plans of user table and corresponding index table in a map.We call it as co-location info. When AM requests for a user(index) regions plans first we will check any plans available for corresponding index(user) regions in the co-location info. If available same will be selected as plans otherwise generate the plans in any fashion like round-robin/random/retain by the delegator and add them to co-location info. When a region is offline the corresponding region plan will be removed from the co-location info. Thanks. Custom load balancer to co-locate the regions of two tables which are having same split keys Key: HBASE-10576 URL: https://issues.apache.org/jira/browse/HBASE-10576 Project: HBase Issue Type: Sub-task Components: Balancer Reporter: rajeshbabu Assignee: rajeshbabu Attachments: HBASE-10536_v2.patch, HBASE-10576.patch To support local indexing both user table and index table should have same split keys. This issue to provide custom balancer to colocate the regions of two tables which are having same split keys. This helps in Phoenix as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11053) Change DeleteTracker APIs to work with Cell
[
https://issues.apache.org/jira/browse/HBASE-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980843#comment-13980843
]
Hadoop QA commented on HBASE-11053:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641877/HBASE-11053_1.patch
against trunk revision .
ATTACHMENT ID: 12641877
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 8
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9398//console
This message is automatically generated.
Change DeleteTracker APIs to work with Cell
---
Key: HBASE-11053
URL: https://issues.apache.org/jira/browse/HBASE-11053
Project: HBase
Issue Type: Sub-task
Affects Versions: 0.98.1, 0.99.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.0, 0.98.2
Attachments: HBASE-11053.patch, HBASE-11053_1.patch
DeleteTracker interface (marked as Private) should work with Cells.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Comment Edited] (HBASE-10576) Custom load balancer to co-locate the regions of two tables which are having same split keys
[
https://issues.apache.org/jira/browse/HBASE-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980728#comment-13980728
]
rajeshbabu edited comment on HBASE-10576 at 4/25/14 9:34 AM:
-
If I understand the [~jeffreyz] idea in the above
comment(https://issues.apache.org/jira/browse/HBASE-10576?focusedCommentId=13978998page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13978998)
correctly,
I think we can do the same without any changes in the kernel.
The steps are like this
1) Before creating index table we need to disable user table.
2) create index table with the same split keys as user table and disable it. So
next time onwards index regions wont be assigned anymore by AM.(Need not have
any special state like shadow)
3) Enable user table. While opening user region we can get corresponing index
region from meta,htd from namenode and just open it with below method
{code}
public static HRegion openHRegion(final HRegionInfo info,
final HTableDescriptor htd, final HLog wal,
final Configuration conf)
throws IOException {
return openHRegion(info, htd, wal, conf, null, null);
}
{code}
After that maintain user region and index region mapping.
4) While scanning if query conditions involves covering indexes we can just
scan index region in the hooks and skip scanning user region by bypass.
Otherwise get rowkeys from index region and seek to rowkey in the user region
and get required information.
I will do prototype of this and see any problems down the like mainly during
split or merge. Mostly there should not be any problem.
what do you say [~jeffreyz]?
was (Author: rajesh23):
If I understand the [~jeffreyz] idea in the above
comment(https://issues.apache.org/jira/browse/HBASE-10576?focusedCommentId=13978998page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13978998)
correctly,
I think we can do the same without any changes in the kernel.
The steps are like this
1) Before creating index table we need to disable user table.
2) create index table with the same split keys as user table and disable it. So
next time onwards index regions wont be assigned anymore by AM.(Need not have
any special state like shadow)
3) Enable user table. While opening user region we can get corresponing index
region from meta,htd from namenode and just open it with below method
{code}
public static HRegion openHRegion(final HRegionInfo info,
final HTableDescriptor htd, final HLog wal,
final Configuration conf)
throws IOException {
return openHRegion(info, htd, wal, conf, null, null);
}
{code}
After that maintain user region and index region mapping.
4) While scanning if query conditions involves covering indexes we can just
scan index region in the hooks and skip scanning user region by bypass.
Otherwise get rowkeys from index region and seek to rowkey in the user region
and get required information.
Custom load balancer to co-locate the regions of two tables which are having
same split keys
Key: HBASE-10576
URL: https://issues.apache.org/jira/browse/HBASE-10576
Project: HBase
Issue Type: Sub-task
Components: Balancer
Reporter: rajeshbabu
Assignee: rajeshbabu
Attachments: HBASE-10536_v2.patch, HBASE-10576.patch
To support local indexing both user table and index table should have same
split keys. This issue to provide custom balancer to colocate the regions of
two tables which are having same split keys.
This helps in Phoenix as well.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11053) Change DeleteTracker APIs to work with Cell
[ https://issues.apache.org/jira/browse/HBASE-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-11053: --- Attachment: HBASE-11053_0.98.patch Change DeleteTracker APIs to work with Cell --- Key: HBASE-11053 URL: https://issues.apache.org/jira/browse/HBASE-11053 Project: HBase Issue Type: Sub-task Affects Versions: 0.98.1, 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11053.patch, HBASE-11053_0.98.patch, HBASE-11053_1.patch DeleteTracker interface (marked as Private) should work with Cells. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11053) Change DeleteTracker APIs to work with Cell
[ https://issues.apache.org/jira/browse/HBASE-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980852#comment-13980852 ] ramkrishna.s.vasudevan commented on HBASE-11053: Committed to trunk. If I commit this to 0.98 one thing is as there is no Cell related changes in 0.98, so passing kv to the DeleteTracker would make it do Kv.getqualifierOffset() , kv.getQualifierLength(), kv.getTimeStamp(), kv.getTypeByte() once again which is already extracted out in the SQM itself. [~apurtell] But if we do a visibility deletion strategy based on DeleteTracker, then this change is very much and later would be easier in 0.99. Also would help in extracting the tag details easily from the cell itself. Would wait before committing to 0.98. Change DeleteTracker APIs to work with Cell --- Key: HBASE-11053 URL: https://issues.apache.org/jira/browse/HBASE-11053 Project: HBase Issue Type: Sub-task Affects Versions: 0.98.1, 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11053.patch, HBASE-11053_0.98.patch, HBASE-11053_1.patch DeleteTracker interface (marked as Private) should work with Cells. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11054) Create new hook in StoreScanner to help user creating his own delete tracker
[
https://issues.apache.org/jira/browse/HBASE-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980896#comment-13980896
]
Hadoop QA commented on HBASE-11054:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641881/HBASE-11054_1.patch
against trunk revision .
ATTACHMENT ID: 12641881
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:red}-1 core tests{color}. The patch failed these unit tests:
org.apache.hadoop.hbase.security.visibility.TestVisibilityLabelsWithDistributedLogReplay
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9399//console
This message is automatically generated.
Create new hook in StoreScanner to help user creating his own delete tracker
Key: HBASE-11054
URL: https://issues.apache.org/jira/browse/HBASE-11054
Project: HBase
Issue Type: Sub-task
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.0, 0.98.2
Attachments: HBASE-11054.patch, HBASE-11054_1.patch
In case of Visibility labels, to support deletes we need to consider the
visibility expression per version inorder to identify the matching visibility
expression.
Tracking the visibility tags along with the delete tracker would make things
easier rather than handling during scans/compactions.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11053) Change DeleteTracker APIs to work with Cell
[
https://issues.apache.org/jira/browse/HBASE-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980900#comment-13980900
]
Hadoop QA commented on HBASE-11053:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641902/HBASE-11053_0.98.patch
against trunk revision .
ATTACHMENT ID: 12641902
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9400//console
This message is automatically generated.
Change DeleteTracker APIs to work with Cell
---
Key: HBASE-11053
URL: https://issues.apache.org/jira/browse/HBASE-11053
Project: HBase
Issue Type: Sub-task
Affects Versions: 0.98.1, 0.99.0
Reporter: ramkrishna.s.vasudevan
Assignee: ramkrishna.s.vasudevan
Fix For: 0.99.0, 0.98.2
Attachments: HBASE-11053.patch, HBASE-11053_0.98.patch,
HBASE-11053_1.patch
DeleteTracker interface (marked as Private) should work with Cells.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11075) TestVisibilityLabelsWithDistributedLogReplay is failing in Precommit builds frequently
ramkrishna.s.vasudevan created HBASE-11075: -- Summary: TestVisibilityLabelsWithDistributedLogReplay is failing in Precommit builds frequently Key: HBASE-11075 URL: https://issues.apache.org/jira/browse/HBASE-11075 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Latest precommit builds I could see TestVisibilityLabelsWithDistributedLogReplay failing frequently. Need to identify the root cause and fix it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11075) TestVisibilityLabelsWithDistributedLogReplay is failing in Precommit builds frequently
[ https://issues.apache.org/jira/browse/HBASE-11075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980902#comment-13980902 ] ramkrishna.s.vasudevan commented on HBASE-11075: Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9399//testReport/ Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/9377//testReport/ These report cannot be opened now but these are instances where it failed. See HBASE-11053 and HBASE-11054. TestVisibilityLabelsWithDistributedLogReplay is failing in Precommit builds frequently -- Key: HBASE-11075 URL: https://issues.apache.org/jira/browse/HBASE-11075 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Latest precommit builds I could see TestVisibilityLabelsWithDistributedLogReplay failing frequently. Need to identify the root cause and fix it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11053) Change DeleteTracker APIs to work with Cell
[ https://issues.apache.org/jira/browse/HBASE-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980954#comment-13980954 ] Hudson commented on HBASE-11053: SUCCESS: Integrated in HBase-TRUNK #5115 (See [https://builds.apache.org/job/HBase-TRUNK/5115/]) HBASE-11053- Change DeleteTracker APIs to work with Cell (Ram) (ramkrishna: rev 1589963) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/DeleteTracker.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanDeleteTracker.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/ScanQueryMatcher.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestScanDeleteTracker.java Change DeleteTracker APIs to work with Cell --- Key: HBASE-11053 URL: https://issues.apache.org/jira/browse/HBASE-11053 Project: HBase Issue Type: Sub-task Affects Versions: 0.98.1, 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11053.patch, HBASE-11053_0.98.patch, HBASE-11053_1.patch DeleteTracker interface (marked as Private) should work with Cells. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11006) Cannot create 'data' table
[ https://issues.apache.org/jira/browse/HBASE-11006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980967#comment-13980967 ] Gustavo Anatoly commented on HBASE-11006: - Thanks, [~mbertozzi] Cannot create 'data' table -- Key: HBASE-11006 URL: https://issues.apache.org/jira/browse/HBASE-11006 Project: HBase Issue Type: Bug Affects Versions: 0.96.0 Reporter: Gregory Hart Assignee: Gustavo Anatoly Labels: noob Fix For: 0.99.0 Attachments: HBASE-11006-v2.patch, HBASE-11006.patch Starting with HBase 0.96, it is not possible to create a table named 'data'. When a user tries to create a table named 'data', the table is added to the metastore but the region server fails to open the region. To reproduce, I just run “hbase shell” and type in: create ‘data’, ‘cf’ This was also verified on Apache HBase trunk by Enis from the Hortornworks forums. His comment on the bug: There has been a change in the directory layout in HBase 0.96 for introducing namespaces. The HBase root directory now keeps all table data under the sub-directory “data”. I suspect the error is somewhere there. More info can be found on the thread in the Hortornworks forums: http://hortonworks.com/community/forums/topic/cannot-create-data-table/ -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11074) Have PE emit histogram stats as it runs rather than dump once at end of test
[
https://issues.apache.org/jira/browse/HBASE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13980996#comment-13980996
]
Nicolas Liochon commented on HBASE-11074:
-
+1
Have PE emit histogram stats as it runs rather than dump once at end of test
Key: HBASE-11074
URL: https://issues.apache.org/jira/browse/HBASE-11074
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: stack
Assignee: stack
Priority: Minor
Attachments: 11074.txt
PE emits progress reading and writing. Add to the progress emission current
histogram snapshot readings. Means don' t have to wait till test completes
to get idea of latencies. Here is sample:
{code}
2014-04-24 22:47:28,085 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/188730/1048576, Min=137.00, Mean=578.74, Max=5152756.00, StdDev=11884.79,
95th=1590.00, 99th=2950.68
2014-04-24 22:47:32,465 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/199215/1048576, Min=137.00, Mean=570.19, Max=5152756.00, StdDev=11591.56,
95th=1543.00, 99th=2911.00
2014-04-24 22:47:37,334 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/209700/1048576, Min=137.00, Mean=564.80, Max=5152756.00, StdDev=11317.96,
95th=1480.00, 99th=2863.00
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11055) Extends the sampling size
[ https://issues.apache.org/jira/browse/HBASE-11055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nicolas Liochon updated HBASE-11055: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed to trunk, thanks for the review, Nick Stack. Extends the sampling size - Key: HBASE-11055 URL: https://issues.apache.org/jira/browse/HBASE-11055 Project: HBase Issue Type: Bug Components: test Affects Versions: 0.99.0 Reporter: Nicolas Liochon Assignee: Nicolas Liochon Priority: Minor Fix For: 0.99.0 Attachments: 11055.v1.patch The latency is measured with a 1K buffer. It's not very comfortable when you're looking at 99.999% latency on multiple millions of operation. Unfortunately, the constructor is protected, so we need to hack a little... -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11076) Update refguide on getting 0.94.x to run on hadoop 2.2.0+
Ted Yu created HBASE-11076: -- Summary: Update refguide on getting 0.94.x to run on hadoop 2.2.0+ Key: HBASE-11076 URL: https://issues.apache.org/jira/browse/HBASE-11076 Project: HBase Issue Type: Task Reporter: Ted Yu http://hbase.apache.org/book.html#d248e643 contains steps for rebuilding 0.94 code base to run on hadoop 2.2.0+ However, the files under src/main/java/org/apache/hadoop/hbase/protobuf/generated were produced by protoc 2.4.0 These files need to be regenerated. See http://search-hadoop.com/m/DHED4j7Um02/HBase+0.94+on+hadoop+2.2.0subj=Re+HBase+0+94+on+hadoop+2+2+0+2+4+0+ This issue is to update refguide with this regeneration step. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10997) Add a modulo argument to PE to constrain the key range
[ https://issues.apache.org/jira/browse/HBASE-10997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981107#comment-13981107 ] Nick Dimiduk commented on HBASE-10997: -- HBASE-9953 lets me think in terms of total GiB data size instead of ((rows * clients) / million rows-per-GiB). Given a total target size and a known rowsize, it will fill in the blank for --rows. This one limits the range of values a client will use. Sorry, caffeine hasn't set in yet; I'm not sure how this is useful. I guess it's for hotspotting whichever regions are hosting those first $(modulo) rows, is that it? In short, I think they're compatible. Add a modulo argument to PE to constrain the key range -- Key: HBASE-10997 URL: https://issues.apache.org/jira/browse/HBASE-10997 Project: HBase Issue Type: Improvement Reporter: stack Assignee: stack Priority: Trivial Fix For: 0.99.0 Attachments: modulo.txt Helps w/ keeping PE inside block cache but same number of clients. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Resolved] (HBASE-10923) Control where to put meta region
[ https://issues.apache.org/jira/browse/HBASE-10923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang resolved HBASE-10923. - Resolution: Won't Fix Close it as Won't Fix. Let's keep meta together with master for now. Control where to put meta region Key: HBASE-10923 URL: https://issues.apache.org/jira/browse/HBASE-10923 Project: HBase Issue Type: Improvement Reporter: Jimmy Xiang Assignee: Jimmy Xiang There is a concern on placing meta regions on the master, as in the comments of HBASE-10569. I was thinking we should have a configuration for a load balancer to decide where to put it. Adjusting this configuration we can control whether to put the meta on master, or other region server. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11059) ZK-less region assignment
[ https://issues.apache.org/jira/browse/HBASE-11059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981153#comment-13981153 ] Jimmy Xiang commented on HBASE-11059: - I don't have a writeup yet. Currently, I was thinking to make sure region assignment works without ZK to simplify the state machine, improve the assignment performance, make it more reliable, so that we can support more regions, and do HA a little bit easier later on. As to pin some system tables to somewhere, like the active master, it's been done in a different issue. For backward compatibility, the ZK part will still be there for a while. Yes, it should be of some help. ZK-less region assignment - Key: HBASE-11059 URL: https://issues.apache.org/jira/browse/HBASE-11059 Project: HBase Issue Type: Improvement Components: master, Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang It seems that most people don't like region assignment with ZK (HBASE-5487), which causes many uncertainties. This jira is to support ZK-less region assignment. We need to make sure this patch doesn't break backward compatibility/rolling upgrade. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-9953) PerformanceEvaluation: Decouple data size from client concurrency
[
https://issues.apache.org/jira/browse/HBASE-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-9953:
Attachment: HBASE-9953.01.patch
Refreshing patch after a short hiatus. Added description for new --size command
to help text.
PerformanceEvaluation: Decouple data size from client concurrency
-
Key: HBASE-9953
URL: https://issues.apache.org/jira/browse/HBASE-9953
Project: HBase
Issue Type: Test
Components: test
Reporter: Nick Dimiduk
Priority: Minor
Attachments: HBASE-9953.00.patch, HBASE-9953.01.patch
PerfEval tool provides a {{--rows=R}} for specifying the number of records to
work with and requires the user provide a value of N, used as the concurrency
level. From what I can tell, every concurrent process will interact with R
rows. In order to perform an apples-to-apples test, the user must
re-calculate the value R for every new value of N.
Instead, I propose accepting a {{--size=S}} for the amount of data to
interact with and let PerfEval divide that amongst the N clients on the
user's behalf.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9953) PerformanceEvaluation: Decouple data size from client concurrency
[
https://issues.apache.org/jira/browse/HBASE-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981171#comment-13981171
]
Nick Dimiduk commented on HBASE-9953:
-
[~jmspaggi]
bq. 2) If both (opts.size == DEFAULT_OPTS.size) and (opts.perClientRunRows ==
DEFAULT_OPTS.perClientRunRows) then opts.totalRows is never set?
Yes, that's fine; in this case the default totalRows value of 1024 * 1024 is
used.
PerformanceEvaluation: Decouple data size from client concurrency
-
Key: HBASE-9953
URL: https://issues.apache.org/jira/browse/HBASE-9953
Project: HBase
Issue Type: Test
Components: test
Reporter: Nick Dimiduk
Priority: Minor
Attachments: HBASE-9953.00.patch, HBASE-9953.01.patch
PerfEval tool provides a {{--rows=R}} for specifying the number of records to
work with and requires the user provide a value of N, used as the concurrency
level. From what I can tell, every concurrent process will interact with R
rows. In order to perform an apples-to-apples test, the user must
re-calculate the value R for every new value of N.
Instead, I propose accepting a {{--size=S}} for the amount of data to
interact with and let PerfEval divide that amongst the N clients on the
user's behalf.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-9953) PerformanceEvaluation: Decouple data size from client concurrency
[
https://issues.apache.org/jira/browse/HBASE-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-9953:
Component/s: (was: test)
Performance
PerformanceEvaluation: Decouple data size from client concurrency
-
Key: HBASE-9953
URL: https://issues.apache.org/jira/browse/HBASE-9953
Project: HBase
Issue Type: Test
Components: Performance
Reporter: Nick Dimiduk
Priority: Minor
Attachments: HBASE-9953.00.patch, HBASE-9953.01.patch
PerfEval tool provides a {{--rows=R}} for specifying the number of records to
work with and requires the user provide a value of N, used as the concurrency
level. From what I can tell, every concurrent process will interact with R
rows. In order to perform an apples-to-apples test, the user must
re-calculate the value R for every new value of N.
Instead, I propose accepting a {{--size=S}} for the amount of data to
interact with and let PerfEval divide that amongst the N clients on the
user's behalf.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-9953) PerformanceEvaluation: Decouple data size from client concurrency
[
https://issues.apache.org/jira/browse/HBASE-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk updated HBASE-9953:
Status: Patch Available (was: Open)
PerformanceEvaluation: Decouple data size from client concurrency
-
Key: HBASE-9953
URL: https://issues.apache.org/jira/browse/HBASE-9953
Project: HBase
Issue Type: Test
Components: Performance
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Priority: Minor
Attachments: HBASE-9953.00.patch, HBASE-9953.01.patch
PerfEval tool provides a {{--rows=R}} for specifying the number of records to
work with and requires the user provide a value of N, used as the concurrency
level. From what I can tell, every concurrent process will interact with R
rows. In order to perform an apples-to-apples test, the user must
re-calculate the value R for every new value of N.
Instead, I propose accepting a {{--size=S}} for the amount of data to
interact with and let PerfEval divide that amongst the N clients on the
user's behalf.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (HBASE-9953) PerformanceEvaluation: Decouple data size from client concurrency
[
https://issues.apache.org/jira/browse/HBASE-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nick Dimiduk reassigned HBASE-9953:
---
Assignee: Nick Dimiduk
PerformanceEvaluation: Decouple data size from client concurrency
-
Key: HBASE-9953
URL: https://issues.apache.org/jira/browse/HBASE-9953
Project: HBase
Issue Type: Test
Components: Performance
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Priority: Minor
Attachments: HBASE-9953.00.patch, HBASE-9953.01.patch
PerfEval tool provides a {{--rows=R}} for specifying the number of records to
work with and requires the user provide a value of N, used as the concurrency
level. From what I can tell, every concurrent process will interact with R
rows. In order to perform an apples-to-apples test, the user must
re-calculate the value R for every new value of N.
Instead, I propose accepting a {{--size=S}} for the amount of data to
interact with and let PerfEval divide that amongst the N clients on the
user's behalf.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9953) PerformanceEvaluation: Decouple data size from client concurrency
[
https://issues.apache.org/jira/browse/HBASE-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981174#comment-13981174
]
Nick Dimiduk commented on HBASE-9953:
-
ping [~nkeywal], [~stack].
PerformanceEvaluation: Decouple data size from client concurrency
-
Key: HBASE-9953
URL: https://issues.apache.org/jira/browse/HBASE-9953
Project: HBase
Issue Type: Test
Components: Performance
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Priority: Minor
Attachments: HBASE-9953.00.patch, HBASE-9953.01.patch
PerfEval tool provides a {{--rows=R}} for specifying the number of records to
work with and requires the user provide a value of N, used as the concurrency
level. From what I can tell, every concurrent process will interact with R
rows. In order to perform an apples-to-apples test, the user must
re-calculate the value R for every new value of N.
Instead, I propose accepting a {{--size=S}} for the amount of data to
interact with and let PerfEval divide that amongst the N clients on the
user's behalf.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11074) Have PE emit histogram stats as it runs rather than dump once at end of test
[
https://issues.apache.org/jira/browse/HBASE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981172#comment-13981172
]
Nick Dimiduk commented on HBASE-11074:
--
Good by me.
Have PE emit histogram stats as it runs rather than dump once at end of test
Key: HBASE-11074
URL: https://issues.apache.org/jira/browse/HBASE-11074
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: stack
Assignee: stack
Priority: Minor
Attachments: 11074.txt
PE emits progress reading and writing. Add to the progress emission current
histogram snapshot readings. Means don' t have to wait till test completes
to get idea of latencies. Here is sample:
{code}
2014-04-24 22:47:28,085 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/188730/1048576, Min=137.00, Mean=578.74, Max=5152756.00, StdDev=11884.79,
95th=1590.00, 99th=2950.68
2014-04-24 22:47:32,465 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/199215/1048576, Min=137.00, Mean=570.19, Max=5152756.00, StdDev=11591.56,
95th=1543.00, 99th=2911.00
2014-04-24 22:47:37,334 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/209700/1048576, Min=137.00, Mean=564.80, Max=5152756.00, StdDev=11317.96,
95th=1480.00, 99th=2863.00
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11038) Filtered scans can bypass metrics collection
[ https://issues.apache.org/jira/browse/HBASE-11038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Dimiduk updated HBASE-11038: - Resolution: Fixed Status: Resolved (was: Patch Available) Committed to 0.96, 0.98, and trunk. Thanks for having a look [~stack]. Filtered scans can bypass metrics collection Key: HBASE-11038 URL: https://issues.apache.org/jira/browse/HBASE-11038 Project: HBase Issue Type: Bug Components: Scanners Affects Versions: 0.96.2, 0.98.1, 0.99.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 0.99.0, 0.98.2, 0.96.3 Attachments: HBASE-11038.00.patch, HBASE-11038.01.96.patch, HBASE-11038.01.98.patch, HBASE-11038.01.patch In RegionScannerImpl#nextRaw, after a batch of results are retrieved, delegates to the filter regarding continuation of the scan. If filterAllRemaining returns true, the method exits immediately, without calling MetricsRegion#updateNextScan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10873) Control number of regions assigned to backup masters
[ https://issues.apache.org/jira/browse/HBASE-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jimmy Xiang updated HBASE-10873: Resolution: Fixed Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Integrated into trunk. Thanks. Control number of regions assigned to backup masters Key: HBASE-10873 URL: https://issues.apache.org/jira/browse/HBASE-10873 Project: HBase Issue Type: Improvement Components: Balancer Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.99.0 Attachments: hbase-10873.patch, hbase-10873_v2.patch, hbase-10873_v3.patch By default, a backup master is treated just like another regionserver. So it can host as many regions as other regionserver does. When the backup master becomes the active one, region balancer needs to move those user regions on this master to other region servers. To minimize the impact, it's better not to assign too many regions on backup masters. It may not be good to leave the backup masters idle and not host any region either. We should make this adjustable so that users can control how many regions to assign to each backup master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10892) Add support for globs in user_permission
[
https://issues.apache.org/jira/browse/HBASE-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981246#comment-13981246
]
Matteo Bertozzi commented on HBASE-10892:
-
+1 for me
Add support for globs in user_permission
Key: HBASE-10892
URL: https://issues.apache.org/jira/browse/HBASE-10892
Project: HBase
Issue Type: Improvement
Components: shell
Affects Versions: 0.99.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Fix For: 0.99.0, 0.98.2
Attachments: HBASE-10892.v0.diff, HBASE-10892.v1.patch
It would be nice for {{user_permission}} to show all the permissions for all
the tables or a subset of tables if a glob (regex) is provided.
{code}
hbase user_permission '*'
User Table,Family,Qualifier:Permission
esteban x,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
hbase1y,,: [Permission:
actions=READ,WRITE]
hbase2z,,: [Permission:
actions=READ,WRITE]
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11074) Have PE emit histogram stats as it runs rather than dump once at end of test
[
https://issues.apache.org/jira/browse/HBASE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981243#comment-13981243
]
Andrew Purtell commented on HBASE-11074:
+1 nice, please commit to all branches !
Have PE emit histogram stats as it runs rather than dump once at end of test
Key: HBASE-11074
URL: https://issues.apache.org/jira/browse/HBASE-11074
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: stack
Assignee: stack
Priority: Minor
Attachments: 11074.txt
PE emits progress reading and writing. Add to the progress emission current
histogram snapshot readings. Means don' t have to wait till test completes
to get idea of latencies. Here is sample:
{code}
2014-04-24 22:47:28,085 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/188730/1048576, Min=137.00, Mean=578.74, Max=5152756.00, StdDev=11884.79,
95th=1590.00, 99th=2950.68
2014-04-24 22:47:32,465 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/199215/1048576, Min=137.00, Mean=570.19, Max=5152756.00, StdDev=11591.56,
95th=1543.00, 99th=2911.00
2014-04-24 22:47:37,334 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/209700/1048576, Min=137.00, Mean=564.80, Max=5152756.00, StdDev=11317.96,
95th=1480.00, 99th=2863.00
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11075) TestVisibilityLabelsWithDistributedLogReplay is failing in Precommit builds frequently
[ https://issues.apache.org/jira/browse/HBASE-11075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981247#comment-13981247 ] Andrew Purtell commented on HBASE-11075: Firing up the looping test rig. Will report back if I see this there. TestVisibilityLabelsWithDistributedLogReplay is failing in Precommit builds frequently -- Key: HBASE-11075 URL: https://issues.apache.org/jira/browse/HBASE-11075 Project: HBase Issue Type: Bug Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Latest precommit builds I could see TestVisibilityLabelsWithDistributedLogReplay failing frequently. Need to identify the root cause and fix it. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11054) Create new hook in StoreScanner to help user creating his own delete tracker
[ https://issues.apache.org/jira/browse/HBASE-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981252#comment-13981252 ] Andrew Purtell commented on HBASE-11054: Also agree that removing trackers and replacing with filters is a good follow on goal [~lhofhansl] and [~ram_krish] Create new hook in StoreScanner to help user creating his own delete tracker Key: HBASE-11054 URL: https://issues.apache.org/jira/browse/HBASE-11054 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11054.patch, HBASE-11054_1.patch In case of Visibility labels, to support deletes we need to consider the visibility expression per version inorder to identify the matching visibility expression. Tracking the visibility tags along with the delete tracker would make things easier rather than handling during scans/compactions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11054) Create new hook in StoreScanner to help user creating his own delete tracker
[ https://issues.apache.org/jira/browse/HBASE-11054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981254#comment-13981254 ] Andrew Purtell commented on HBASE-11054: +1 on latest patch Create new hook in StoreScanner to help user creating his own delete tracker Key: HBASE-11054 URL: https://issues.apache.org/jira/browse/HBASE-11054 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11054.patch, HBASE-11054_1.patch In case of Visibility labels, to support deletes we need to consider the visibility expression per version inorder to identify the matching visibility expression. Tracking the visibility tags along with the delete tracker would make things easier rather than handling during scans/compactions. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10892) Add support for globs in user_permission
[
https://issues.apache.org/jira/browse/HBASE-10892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated HBASE-10892:
---
Fix Version/s: 0.98.2
0.99.0
Yeah we need to fix the source of those zombies in precommit builds. It's not
this patch.
Will commit shortly unless objection. I will fix the long lines at commit time.
Add support for globs in user_permission
Key: HBASE-10892
URL: https://issues.apache.org/jira/browse/HBASE-10892
Project: HBase
Issue Type: Improvement
Components: shell
Affects Versions: 0.99.0
Reporter: Esteban Gutierrez
Assignee: Esteban Gutierrez
Fix For: 0.99.0, 0.98.2
Attachments: HBASE-10892.v0.diff, HBASE-10892.v1.patch
It would be nice for {{user_permission}} to show all the permissions for all
the tables or a subset of tables if a glob (regex) is provided.
{code}
hbase user_permission '*'
User Table,Family,Qualifier:Permission
esteban x,,: [Permission:
actions=READ,WRITE,EXEC,CREATE,ADMIN]
hbase1y,,: [Permission:
actions=READ,WRITE]
hbase2z,,: [Permission:
actions=READ,WRITE]
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-11074) Have PE emit histogram stats as it runs rather than dump once at end of test
[
https://issues.apache.org/jira/browse/HBASE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-11074:
--
Resolution: Fixed
Fix Version/s: 0.99.0
Hadoop Flags: Reviewed
Status: Resolved (was: Patch Available)
Only makes sense in trunk since that has the histogram support. Committed
there. Thanks for reviews.
Have PE emit histogram stats as it runs rather than dump once at end of test
Key: HBASE-11074
URL: https://issues.apache.org/jira/browse/HBASE-11074
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: stack
Assignee: stack
Priority: Minor
Fix For: 0.99.0
Attachments: 11074.txt
PE emits progress reading and writing. Add to the progress emission current
histogram snapshot readings. Means don' t have to wait till test completes
to get idea of latencies. Here is sample:
{code}
2014-04-24 22:47:28,085 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/188730/1048576, Min=137.00, Mean=578.74, Max=5152756.00, StdDev=11884.79,
95th=1590.00, 99th=2950.68
2014-04-24 22:47:32,465 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/199215/1048576, Min=137.00, Mean=570.19, Max=5152756.00, StdDev=11591.56,
95th=1543.00, 99th=2911.00
2014-04-24 22:47:37,334 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/209700/1048576, Min=137.00, Mean=564.80, Max=5152756.00, StdDev=11317.96,
95th=1480.00, 99th=2863.00
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10932) Improve RowCounter to allow mapper number set/control
[
https://issues.apache.org/jira/browse/HBASE-10932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981271#comment-13981271
]
Jean-Daniel Cryans commented on HBASE-10932:
bq. But I didn't quite catch the point of job scheduler, in my understanding
job scheduler is cluster-level and cannot be configured per-job, right?
Well, by using a scheduler, you can constrain certain types of jobs so that
they don't run as fast as they can. For example, with the fair scheduler you
can configure a pool (let's call it the slow pool) to have only {{maxMaps}}
running concurrently on the cluster. Then, when you run your {{RowCounter}}
jobs and whatnot, you can tie them automatically to the slow pool. Hadoop
cluster operators usually know how to use a scheduler, whereas having to rely
on the person who runs the jobs to configure them correctly can lead to human
errors like oops I forgot to pass the maps configuration to my row counter and
now the website is down.
It also works well if you have two users who want to concurrently run a row
counter; they'll both get in the slow pool and only two mappers will run
(alternating between the two jobs, unless you set different weights because one
user is more important than the other, etc etc). If you were to rely on
individual users specifying the correct number of maps, and they both set their
job to use two, then you'd have four mappers running. Back to square one.
Anyways, all of this to say that there's a more generic way of doing this, and
it already exists. Can we close this jira, [~carp84]?
Improve RowCounter to allow mapper number set/control
-
Key: HBASE-10932
URL: https://issues.apache.org/jira/browse/HBASE-10932
Project: HBase
Issue Type: Improvement
Components: mapreduce
Reporter: Yu Li
Assignee: Yu Li
Priority: Minor
Attachments: HBASE-10932_v1.patch, HBASE-10932_v2.patch
The typical use case of RowCounter is to do some kind of data integrity
checking, like after exporting some data from RDBMS to HBase, or from one
HBase cluster to another, making sure the row(record) number matches. Such
check commonly won't require much on response time.
Meanwhile, based on current impl, RowCounter will launch one mapper per
region, and each mapper will send one scan request. Assuming the table is
kind of big like having tens of regions, and the cpu core number of the whole
MR cluster is also enough, the parallel scan requests sent by mapper would be
a real burden for the HBase cluster.
So in this JIRA, we're proposing to make rowcounter support an additional
option --maps to specify mapper number, and make each mapper able to scan
more than one region of the target table.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9953) PerformanceEvaluation: Decouple data size from client concurrency
[
https://issues.apache.org/jira/browse/HBASE-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981273#comment-13981273
]
stack commented on HBASE-9953:
--
+1 Commit and I'll try it. Let me explain over in the modulo issue how it does
similar but I like your take better (After trying this, may back out the modulo
arg)
PerformanceEvaluation: Decouple data size from client concurrency
-
Key: HBASE-9953
URL: https://issues.apache.org/jira/browse/HBASE-9953
Project: HBase
Issue Type: Test
Components: Performance
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Priority: Minor
Attachments: HBASE-9953.00.patch, HBASE-9953.01.patch
PerfEval tool provides a {{--rows=R}} for specifying the number of records to
work with and requires the user provide a value of N, used as the concurrency
level. From what I can tell, every concurrent process will interact with R
rows. In order to perform an apples-to-apples test, the user must
re-calculate the value R for every new value of N.
Instead, I propose accepting a {{--size=S}} for the amount of data to
interact with and let PerfEval divide that amongst the N clients on the
user's behalf.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-9953) PerformanceEvaluation: Decouple data size from client concurrency
[
https://issues.apache.org/jira/browse/HBASE-9953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981287#comment-13981287
]
Hadoop QA commented on HBASE-9953:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12641941/HBASE-9953.01.patch
against trunk revision .
ATTACHMENT ID: 12641941
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 3 new
or modified tests.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated 8
warning messages.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 lineLengths{color}. The patch does not introduce lines
longer than 100
{color:green}+1 site{color}. The mvn site goal succeeds with this patch.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-thrift.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
Findbugs warnings:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
Console output:
https://builds.apache.org/job/PreCommit-HBASE-Build/9401//console
This message is automatically generated.
PerformanceEvaluation: Decouple data size from client concurrency
-
Key: HBASE-9953
URL: https://issues.apache.org/jira/browse/HBASE-9953
Project: HBase
Issue Type: Test
Components: Performance
Reporter: Nick Dimiduk
Assignee: Nick Dimiduk
Priority: Minor
Attachments: HBASE-9953.00.patch, HBASE-9953.01.patch
PerfEval tool provides a {{--rows=R}} for specifying the number of records to
work with and requires the user provide a value of N, used as the concurrency
level. From what I can tell, every concurrent process will interact with R
rows. In order to perform an apples-to-apples test, the user must
re-calculate the value R for every new value of N.
Instead, I propose accepting a {{--size=S}} for the amount of data to
interact with and let PerfEval divide that amongst the N clients on the
user's behalf.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10997) Add a modulo argument to PE to constrain the key range
[ https://issues.apache.org/jira/browse/HBASE-10997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981297#comment-13981297 ] stack commented on HBASE-10997: --- bq. I'm not sure how this is useful. I use it like this. I want to run a PE that does random reads and that runs for a good while, long enough for the latency pattern to emerge. I want to run ten clients. Block cache is small. I want the random reads to stay in block cache. As is, the random reads range over a namespace that is clients X rows. To stay inside block cache, I can set rows small but my test is over before a pattern is well established. If I set rows to 1B, then clients will get 1B rows but if I add the modulo, the keys will be confined to the lower part of the range (dependent on what modulo is). If I want to bust block cache but stay inside the os cache, I up the modulo till just before i/o comes on. I like your 'size' better than my modulo. I'll remove this if your size works. Thanks. Sorry for committing this w/o waiting on review. Didn't think anyone cared (missed your size issue). Add a modulo argument to PE to constrain the key range -- Key: HBASE-10997 URL: https://issues.apache.org/jira/browse/HBASE-10997 Project: HBase Issue Type: Improvement Reporter: stack Assignee: stack Priority: Trivial Fix For: 0.99.0 Attachments: modulo.txt Helps w/ keeping PE inside block cache but same number of clients. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10576) Custom load balancer to co-locate the regions of two tables which are having same split keys
[ https://issues.apache.org/jira/browse/HBASE-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981310#comment-13981310 ] stack commented on HBASE-10576: --- [~rajesh23] So, you would only require changes in the balancer? Have you tried it? Is your scheme immune to the issues [~jeffreyz] talks of above? Thanks. Custom load balancer to co-locate the regions of two tables which are having same split keys Key: HBASE-10576 URL: https://issues.apache.org/jira/browse/HBASE-10576 Project: HBase Issue Type: Sub-task Components: Balancer Reporter: rajeshbabu Assignee: rajeshbabu Attachments: HBASE-10536_v2.patch, HBASE-10576.patch To support local indexing both user table and index table should have same split keys. This issue to provide custom balancer to colocate the regions of two tables which are having same split keys. This helps in Phoenix as well. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10581) ACL znode are left without PBed during upgrading hbase0.94* to hbase0.96+
[ https://issues.apache.org/jira/browse/HBASE-10581?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981318#comment-13981318 ] Enis Soztutar commented on HBASE-10581: --- Ok thanks. Just wanted to make sure that there is not any other issue. ACL znode are left without PBed during upgrading hbase0.94* to hbase0.96+ - Key: HBASE-10581 URL: https://issues.apache.org/jira/browse/HBASE-10581 Project: HBase Issue Type: Bug Affects Versions: 0.98.0, 0.96.0, 0.96.1 Reporter: Jeffrey Zhong Assignee: Jeffrey Zhong Priority: Critical Fix For: 0.96.2, 0.98.1, 0.99.0 Attachments: hbase-10581.patch ACL znodes are left in the upgrade process when upgrading 0.94 to 0.96+ Those 0.94 znodes will choke HMaster because their data aren't PBed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11070) [AccessController] Restore early-out access denial if the user has no access at the table or CF level
[ https://issues.apache.org/jira/browse/HBASE-11070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981344#comment-13981344 ] Enis Soztutar commented on HBASE-11070: --- I think it makes sense to have a separate permission for being able to read only granted cells. Also thinking more about this, we want the lack of default READ perm would prevent the users from launching scanners. If the admin wants to allow some users to read some cells than she can explicitly set it so. [AccessController] Restore early-out access denial if the user has no access at the table or CF level - Key: HBASE-11070 URL: https://issues.apache.org/jira/browse/HBASE-11070 Project: HBase Issue Type: Task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.3 We want to support two different use cases for cell ACLs: 1. The user can see all cells in a table or CF unless a cell ACL denies access 2. The user cannot see any cells in a table or CF unless a cell ACL grants access For the sake of flexibility we made it a toggle on an operation by operation basis. However this changed the behavior of the AccessController with respect to how requests for which a user has no grant at the table or CF level are handled. Prior to the cell ACL changes if a user had no grant at the table or CF level, they would see an AccessDeniedException. We can't do that if we want cell ACLs to provide exceptional access. Subsequent to the cell ACL changes if a user has no grant at the table or CF level, there is no exception, they simply won't see any cells except those granting exceptional access at the cell level. This also brings the AccessController semantics in line with those of the new VisibilityController. Feedback on dev@ is this change is a bridge too far for at least three reasons. First, it is surprising (Enis and Vandana). Second, the audit trail is affected or missing (Enis). Third, it allows any user on the cluster to mount targeted queries against all tables looking for timing differences, that depending on schema design could possibly leak the existence in row keys of sensitive information, or leak the size of the table (Todd). Although we can't prevent timing attacks in general we can limit the scope of what a user can explore by restoring early-out access denial if the user has no access at the table or CF level. We can make early-out access denial if the user has no access at the table or CF level configurable on a per table basis. Setting the default to false, with a release note and paragraph in the security guide explaining how to reintroduce the old behavior, would address the above and not introduce another surprising change among 0.98 releases. If the consensus is (presumably a milder) surprise due to this change is fine, then the default could be true -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11071) Abstract HM admin table handlers from ZK
[ https://issues.apache.org/jira/browse/HBASE-11071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981398#comment-13981398 ] stack commented on HBASE-11071: --- [~cos] I added you. Abstract HM admin table handlers from ZK Key: HBASE-11071 URL: https://issues.apache.org/jira/browse/HBASE-11071 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Mikhail Antonov Abstract table admin handlers, including: - CreateTableHandler - DeleteTableHandler - EnableTableHandler - DisableTableHandler -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11058) call_id seems never present in response
[
https://issues.apache.org/jira/browse/HBASE-11058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981399#comment-13981399
]
Jean-Marc Spaggiari commented on HBASE-11058:
-
Hi [~charlenehelios], can you please provide more details on the context? What
are you trying to do, what do you retrieve, etc.?
Thanks.
call_id seems never present in response
---
Key: HBASE-11058
URL: https://issues.apache.org/jira/browse/HBASE-11058
Project: HBase
Issue Type: Bug
Reporter: Charlene Sun
version used: 0.98.1
call_id seems never present in response, could someone take a look and help?
message ResponseHeader {
optional uint32 call_id = 1;
message RequestHeader {
// Monotonically increasing call_id to keep track of RPC requests and their
response
optional uint32 call_id = 1;
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11050) Replace empty catch block in TestHLog#testFailedToCreateHLogIfParentRenamed with @Test(expected=)
[
https://issues.apache.org/jira/browse/HBASE-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981410#comment-13981410
]
Jean-Marc Spaggiari commented on HBASE-11050:
-
Since this has been committed, can we close this JIRA?
Replace empty catch block in TestHLog#testFailedToCreateHLogIfParentRenamed
with @Test(expected=)
--
Key: HBASE-11050
URL: https://issues.apache.org/jira/browse/HBASE-11050
Project: HBase
Issue Type: Task
Reporter: Gustavo Anatoly
Assignee: Gustavo Anatoly
Priority: Trivial
Fix For: 0.99.0
Attachments: HBASE-11050.patch
This change refactor TestHLog#testFailedToCreateHLogIfParentRenamed. The test
basically create {{HLogFactory.createWALWriter(fs, path, conf);}} and after
that parent {{path}} is renamed followed to another call
{{HLogFactory.createWALWriter(fs, path, conf);}} in this moment is expected
IOException, because the parent {{path}} doesn't exist more.
The second call monitored by a block try-catch with an empty {{catch}}:
{code}
try {
HLogFactory.createWALWriter(fs, path, conf);
fail(It should fail to create the new WAL);
} catch (IOException ioe) {
// expected, good.
}
{code}
The patch proposed removes the {{try-catch}} and use
{{@Test(expected=IOException.class)}} to capture exception produced by the
test.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11070) [AccessController] Restore early-out access denial if the user has no access at the table or CF level
[ https://issues.apache.org/jira/browse/HBASE-11070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981423#comment-13981423 ] Andrew Purtell commented on HBASE-11070: bq. I think it makes sense to have a separate permission for being able to read only granted cells. We can tinker and see how this works out. See above comment about WRITE being granted independent of READ. Do we want the same kind of separate permission for being able to write only granted cells? bq. Also thinking more about this, we want the lack of default READ perm would prevent the users from launching scanners. This could be done pretty easily by varying the permissions tests in preGet and preExists versus preScannerOpen. [AccessController] Restore early-out access denial if the user has no access at the table or CF level - Key: HBASE-11070 URL: https://issues.apache.org/jira/browse/HBASE-11070 Project: HBase Issue Type: Task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.3 We want to support two different use cases for cell ACLs: 1. The user can see all cells in a table or CF unless a cell ACL denies access 2. The user cannot see any cells in a table or CF unless a cell ACL grants access For the sake of flexibility we made it a toggle on an operation by operation basis. However this changed the behavior of the AccessController with respect to how requests for which a user has no grant at the table or CF level are handled. Prior to the cell ACL changes if a user had no grant at the table or CF level, they would see an AccessDeniedException. We can't do that if we want cell ACLs to provide exceptional access. Subsequent to the cell ACL changes if a user has no grant at the table or CF level, there is no exception, they simply won't see any cells except those granting exceptional access at the cell level. This also brings the AccessController semantics in line with those of the new VisibilityController. Feedback on dev@ is this change is a bridge too far for at least three reasons. First, it is surprising (Enis and Vandana). Second, the audit trail is affected or missing (Enis). Third, it allows any user on the cluster to mount targeted queries against all tables looking for timing differences, that depending on schema design could possibly leak the existence in row keys of sensitive information, or leak the size of the table (Todd). Although we can't prevent timing attacks in general we can limit the scope of what a user can explore by restoring early-out access denial if the user has no access at the table or CF level. We can make early-out access denial if the user has no access at the table or CF level configurable on a per table basis. Setting the default to false, with a release note and paragraph in the security guide explaining how to reintroduce the old behavior, would address the above and not introduce another surprising change among 0.98 releases. If the consensus is (presumably a milder) surprise due to this change is fine, then the default could be true -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10576) Custom load balancer to co-locate the regions of two tables which are having same split keys
[
https://issues.apache.org/jira/browse/HBASE-10576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981427#comment-13981427
]
Jeffrey Zhong commented on HBASE-10576:
---
[~rajesh23]
{quote}
(Need not have any special state like shadow)
{quote}
You can't reuse disabled state because a client can't talk to a region in a
disabled table. Introducing a new state like shadow, I think it's cleaner.
{quote}
I will do prototype of this
{quote}
Please do.
[~saint@gmail.com] Just changing balancer alone isn't enough. Because even
we send out region assignment requests simultaneously with same destination RS,
there is no guarantee which assignment will happen firstly, when they will
happen complete and if they can both succeed.
With this coprocessor approach, since both region open the same time, we can
even atomically update both their location info in meta table with a single
batch. So clients can see both of them in a location at the same time.
[~giacomotaylor] The new proposal is to enforce strong co-locating. We still
need same split key prefix for the index regions. There are other ways
without requiring same split key/prefix but they're not clean. Since there is
an entry in meta table for the index region with shadow state, a client can
scan the region directly.
Thanks.
Custom load balancer to co-locate the regions of two tables which are having
same split keys
Key: HBASE-10576
URL: https://issues.apache.org/jira/browse/HBASE-10576
Project: HBase
Issue Type: Sub-task
Components: Balancer
Reporter: rajeshbabu
Assignee: rajeshbabu
Attachments: HBASE-10536_v2.patch, HBASE-10576.patch
To support local indexing both user table and index table should have same
split keys. This issue to provide custom balancer to colocate the regions of
two tables which are having same split keys.
This helps in Phoenix as well.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11077) [AccessController] Restore compatible early-out access denial
Andrew Purtell created HBASE-11077: -- Summary: [AccessController] Restore compatible early-out access denial Key: HBASE-11077 URL: https://issues.apache.org/jira/browse/HBASE-11077 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.2 See parent for the whole story. For 0.98, to start, just put back the early out that was removed in 0.98.0 and allow it to be overridden with a table attribute. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11078) [AccessController] Consider new permission for read visible
Andrew Purtell created HBASE-11078: -- Summary: [AccessController] Consider new permission for read visible Key: HBASE-11078 URL: https://issues.apache.org/jira/browse/HBASE-11078 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Fix For: 0.99.0 See parent for the whole story. Consider a new permission with the semantics being able to read only granted cells, perhaps called READ_VISIBLE. Maybe consider a symmetric new permission for writes. The lack of default READ perm should prevent users from launching scanners. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11070) [AccessController] Restore early-out access denial if the user has no access at the table or CF level
[ https://issues.apache.org/jira/browse/HBASE-11070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981442#comment-13981442 ] Andrew Purtell commented on HBASE-11070: Opened subtasks [AccessController] Restore early-out access denial if the user has no access at the table or CF level - Key: HBASE-11070 URL: https://issues.apache.org/jira/browse/HBASE-11070 Project: HBase Issue Type: Task Reporter: Andrew Purtell Assignee: Andrew Purtell Fix For: 0.99.0, 0.98.3 We want to support two different use cases for cell ACLs: 1. The user can see all cells in a table or CF unless a cell ACL denies access 2. The user cannot see any cells in a table or CF unless a cell ACL grants access For the sake of flexibility we made it a toggle on an operation by operation basis. However this changed the behavior of the AccessController with respect to how requests for which a user has no grant at the table or CF level are handled. Prior to the cell ACL changes if a user had no grant at the table or CF level, they would see an AccessDeniedException. We can't do that if we want cell ACLs to provide exceptional access. Subsequent to the cell ACL changes if a user has no grant at the table or CF level, there is no exception, they simply won't see any cells except those granting exceptional access at the cell level. This also brings the AccessController semantics in line with those of the new VisibilityController. Feedback on dev@ is this change is a bridge too far for at least three reasons. First, it is surprising (Enis and Vandana). Second, the audit trail is affected or missing (Enis). Third, it allows any user on the cluster to mount targeted queries against all tables looking for timing differences, that depending on schema design could possibly leak the existence in row keys of sensitive information, or leak the size of the table (Todd). Although we can't prevent timing attacks in general we can limit the scope of what a user can explore by restoring early-out access denial if the user has no access at the table or CF level. We can make early-out access denial if the user has no access at the table or CF level configurable on a per table basis. Setting the default to false, with a release note and paragraph in the security guide explaining how to reintroduce the old behavior, would address the above and not introduce another surprising change among 0.98 releases. If the consensus is (presumably a milder) surprise due to this change is fine, then the default could be true -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11074) Have PE emit histogram stats as it runs rather than dump once at end of test
[
https://issues.apache.org/jira/browse/HBASE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981448#comment-13981448
]
Andrew Purtell commented on HBASE-11074:
Why not backport the histogram support? Should tools like this have the same
functionality cross branches so we can compare them?
Have PE emit histogram stats as it runs rather than dump once at end of test
Key: HBASE-11074
URL: https://issues.apache.org/jira/browse/HBASE-11074
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: stack
Assignee: stack
Priority: Minor
Fix For: 0.99.0
Attachments: 11074.txt
PE emits progress reading and writing. Add to the progress emission current
histogram snapshot readings. Means don' t have to wait till test completes
to get idea of latencies. Here is sample:
{code}
2014-04-24 22:47:28,085 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/188730/1048576, Min=137.00, Mean=578.74, Max=5152756.00, StdDev=11884.79,
95th=1590.00, 99th=2950.68
2014-04-24 22:47:32,465 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/199215/1048576, Min=137.00, Mean=570.19, Max=5152756.00, StdDev=11591.56,
95th=1543.00, 99th=2911.00
2014-04-24 22:47:37,334 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/209700/1048576, Min=137.00, Mean=564.80, Max=5152756.00, StdDev=11317.96,
95th=1480.00, 99th=2863.00
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11074) Have PE emit histogram stats as it runs rather than dump once at end of test
[
https://issues.apache.org/jira/browse/HBASE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981453#comment-13981453
]
stack commented on HBASE-11074:
---
bq. Why not backport the histogram support?
Sorry. Working on trunk. PE seems to be under a bit of churn at the moment
(folks are using it). Lets backport the whole thing when it settles.
Have PE emit histogram stats as it runs rather than dump once at end of test
Key: HBASE-11074
URL: https://issues.apache.org/jira/browse/HBASE-11074
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: stack
Assignee: stack
Priority: Minor
Fix For: 0.99.0
Attachments: 11074.txt
PE emits progress reading and writing. Add to the progress emission current
histogram snapshot readings. Means don' t have to wait till test completes
to get idea of latencies. Here is sample:
{code}
2014-04-24 22:47:28,085 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/188730/1048576, Min=137.00, Mean=578.74, Max=5152756.00, StdDev=11884.79,
95th=1590.00, 99th=2950.68
2014-04-24 22:47:32,465 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/199215/1048576, Min=137.00, Mean=570.19, Max=5152756.00, StdDev=11591.56,
95th=1543.00, 99th=2911.00
2014-04-24 22:47:37,334 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/209700/1048576, Min=137.00, Mean=564.80, Max=5152756.00, StdDev=11317.96,
95th=1480.00, 99th=2863.00
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-10932) Improve RowCounter to allow mapper number set/control
[ https://issues.apache.org/jira/browse/HBASE-10932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981441#comment-13981441 ] Yu Li commented on HBASE-10932: --- Hi [~jdcryans], If we follow this logic, do you mean the -m option of DistCp also useless? IMHO, the configuration of job scheduler in JT/Yarn is the server-side configuration, while the -m option is the client-side configuration, and both are necessary. Back to the scheduler discussion, I believe job scheduler could only limit the max resource one user could use, and it depends on the user to decide how he uses the resource assigned to him. Like in the example you gave, what if the slow pool have 4 slots while only one user submit a rowcounter and he prefers only 2 maps running in parallel? I'm afraid asking the cluster operator to create another slow pool with only 2 slots is not a good solution. In a common hbase ETL application, user would need to first do distcp, then bulkload, then rowcounter to check data integrity, and he would prefer distcp to run as fast as possible w/ low scan workload during rowcounter. In this case, he would need to submit the distcp job to the fast queue while the rowcounter job to the slow queue? And he also needs to get access to both queues... Anyway, this is a real requirement from user in our product env, and I'm just trying to contribute this to community in case this can help other users. But if you still think it useless, just go ahead and close it, you're the boss after all. :-) And no matter what decision made, thanks for your time on reviewing this JIRA and discussion. Improve RowCounter to allow mapper number set/control - Key: HBASE-10932 URL: https://issues.apache.org/jira/browse/HBASE-10932 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Yu Li Assignee: Yu Li Priority: Minor Attachments: HBASE-10932_v1.patch, HBASE-10932_v2.patch The typical use case of RowCounter is to do some kind of data integrity checking, like after exporting some data from RDBMS to HBase, or from one HBase cluster to another, making sure the row(record) number matches. Such check commonly won't require much on response time. Meanwhile, based on current impl, RowCounter will launch one mapper per region, and each mapper will send one scan request. Assuming the table is kind of big like having tens of regions, and the cpu core number of the whole MR cluster is also enough, the parallel scan requests sent by mapper would be a real burden for the HBase cluster. So in this JIRA, we're proposing to make rowcounter support an additional option --maps to specify mapper number, and make each mapper able to scan more than one region of the target table. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11074) Have PE emit histogram stats as it runs rather than dump once at end of test
[
https://issues.apache.org/jira/browse/HBASE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981462#comment-13981462
]
Nick Dimiduk commented on HBASE-11074:
--
According to git, 0.96 and 0.98 have equivalent patches applied. Trunk is ahead
by at least: HBASE-11055, HBASE-11026, HBASE-11000, HBASE-11000, HBASE-10997,
HBASE-10788.
Have PE emit histogram stats as it runs rather than dump once at end of test
Key: HBASE-11074
URL: https://issues.apache.org/jira/browse/HBASE-11074
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: stack
Assignee: stack
Priority: Minor
Fix For: 0.99.0
Attachments: 11074.txt
PE emits progress reading and writing. Add to the progress emission current
histogram snapshot readings. Means don' t have to wait till test completes
to get idea of latencies. Here is sample:
{code}
2014-04-24 22:47:28,085 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/188730/1048576, Min=137.00, Mean=578.74, Max=5152756.00, StdDev=11884.79,
95th=1590.00, 99th=2950.68
2014-04-24 22:47:32,465 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/199215/1048576, Min=137.00, Mean=570.19, Max=5152756.00, StdDev=11591.56,
95th=1543.00, 99th=2911.00
2014-04-24 22:47:37,334 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/209700/1048576, Min=137.00, Mean=564.80, Max=5152756.00, StdDev=11317.96,
95th=1480.00, 99th=2863.00
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Resolved] (HBASE-10932) Improve RowCounter to allow mapper number set/control
[ https://issues.apache.org/jira/browse/HBASE-10932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans resolved HBASE-10932. Resolution: Won't Fix Resolving as won't fix. If you want to work on a more general solution, like adding this option to the TIF, please open a new jira. Thanks. Improve RowCounter to allow mapper number set/control - Key: HBASE-10932 URL: https://issues.apache.org/jira/browse/HBASE-10932 Project: HBase Issue Type: Improvement Components: mapreduce Reporter: Yu Li Assignee: Yu Li Priority: Minor Attachments: HBASE-10932_v1.patch, HBASE-10932_v2.patch The typical use case of RowCounter is to do some kind of data integrity checking, like after exporting some data from RDBMS to HBase, or from one HBase cluster to another, making sure the row(record) number matches. Such check commonly won't require much on response time. Meanwhile, based on current impl, RowCounter will launch one mapper per region, and each mapper will send one scan request. Assuming the table is kind of big like having tens of regions, and the cpu core number of the whole MR cluster is also enough, the parallel scan requests sent by mapper would be a real burden for the HBase cluster. So in this JIRA, we're proposing to make rowcounter support an additional option --maps to specify mapper number, and make each mapper able to scan more than one region of the target table. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11074) Have PE emit histogram stats as it runs rather than dump once at end of test
[
https://issues.apache.org/jira/browse/HBASE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981487#comment-13981487
]
Hudson commented on HBASE-11074:
FAILURE: Integrated in HBase-TRUNK #5117 (See
[https://builds.apache.org/job/HBase-TRUNK/5117/])
HBASE-11074 Have PE emit histogram stats as it runs rather than dump once at
end of test (stack: rev 1590085)
*
/hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/PerformanceEvaluation.java
Have PE emit histogram stats as it runs rather than dump once at end of test
Key: HBASE-11074
URL: https://issues.apache.org/jira/browse/HBASE-11074
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: stack
Assignee: stack
Priority: Minor
Fix For: 0.99.0
Attachments: 11074.txt
PE emits progress reading and writing. Add to the progress emission current
histogram snapshot readings. Means don' t have to wait till test completes
to get idea of latencies. Here is sample:
{code}
2014-04-24 22:47:28,085 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/188730/1048576, Min=137.00, Mean=578.74, Max=5152756.00, StdDev=11884.79,
95th=1590.00, 99th=2950.68
2014-04-24 22:47:32,465 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/199215/1048576, Min=137.00, Mean=570.19, Max=5152756.00, StdDev=11591.56,
95th=1543.00, 99th=2911.00
2014-04-24 22:47:37,334 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/209700/1048576, Min=137.00, Mean=564.80, Max=5152756.00, StdDev=11317.96,
95th=1480.00, 99th=2863.00
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HBASE-11038) Filtered scans can bypass metrics collection
[ https://issues.apache.org/jira/browse/HBASE-11038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981486#comment-13981486 ] Hudson commented on HBASE-11038: FAILURE: Integrated in HBase-TRUNK #5117 (See [https://builds.apache.org/job/HBase-TRUNK/5117/]) HBASE-11038 Filtered scans can bypass metrics collection (ndimiduk: rev 1590067) * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Filtered scans can bypass metrics collection Key: HBASE-11038 URL: https://issues.apache.org/jira/browse/HBASE-11038 Project: HBase Issue Type: Bug Components: Scanners Affects Versions: 0.96.2, 0.98.1, 0.99.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 0.99.0, 0.98.2, 0.96.3 Attachments: HBASE-11038.00.patch, HBASE-11038.01.96.patch, HBASE-11038.01.98.patch, HBASE-11038.01.patch In RegionScannerImpl#nextRaw, after a batch of results are retrieved, delegates to the filter regarding continuation of the scan. If filterAllRemaining returns true, the method exits immediately, without calling MetricsRegion#updateNextScan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10873) Control number of regions assigned to backup masters
[ https://issues.apache.org/jira/browse/HBASE-10873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981485#comment-13981485 ] Hudson commented on HBASE-10873: FAILURE: Integrated in HBase-TRUNK #5117 (See [https://builds.apache.org/job/HBase-TRUNK/5117/]) HBASE-10873 Control number of regions assigned to backup masters (jxiang: rev 1590078) * /hbase/trunk/hbase-common/src/main/resources/hbase-default.xml * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/ServerManager.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/BaseLoadBalancer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/ClusterLoadState.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/FavoredNodeLoadBalancer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/SimpleLoadBalancer.java * /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/balancer/StochasticLoadBalancer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/BalancerTestBase.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestBaseLoadBalancer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestDefaultLoadBalancer.java * /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/master/balancer/TestStochasticLoadBalancer.java Control number of regions assigned to backup masters Key: HBASE-10873 URL: https://issues.apache.org/jira/browse/HBASE-10873 Project: HBase Issue Type: Improvement Components: Balancer Reporter: Jimmy Xiang Assignee: Jimmy Xiang Fix For: 0.99.0 Attachments: hbase-10873.patch, hbase-10873_v2.patch, hbase-10873_v3.patch By default, a backup master is treated just like another regionserver. So it can host as many regions as other regionserver does. When the backup master becomes the active one, region balancer needs to move those user regions on this master to other region servers. To minimize the impact, it's better not to assign too many regions on backup masters. It may not be good to leave the backup masters idle and not host any region either. We should make this adjustable so that users can control how many regions to assign to each backup master. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11079) Normalize test tools across branches
Andrew Purtell created HBASE-11079: -- Summary: Normalize test tools across branches Key: HBASE-11079 URL: https://issues.apache.org/jira/browse/HBASE-11079 Project: HBase Issue Type: Umbrella Reporter: Andrew Purtell Will be a challenge wherever the branches vary functionally, but it would be good to normalize the test tools (LoadTestTool and PerformanceEvaluation) as much as possible among the active branches so we can compare them. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11071) Abstract HM admin table handlers from ZK
[ https://issues.apache.org/jira/browse/HBASE-11071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-11071: Assignee: Konstantin Boudnik Abstract HM admin table handlers from ZK Key: HBASE-11071 URL: https://issues.apache.org/jira/browse/HBASE-11071 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Reporter: Mikhail Antonov Assignee: Konstantin Boudnik Abstract table admin handlers, including: - CreateTableHandler - DeleteTableHandler - EnableTableHandler - DisableTableHandler -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11074) Have PE emit histogram stats as it runs rather than dump once at end of test
[
https://issues.apache.org/jira/browse/HBASE-11074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981496#comment-13981496
]
Andrew Purtell commented on HBASE-11074:
Thanks [~stack] and [~ndimiduk]. I filed HBASE-11079 and will look for more
related.
Have PE emit histogram stats as it runs rather than dump once at end of test
Key: HBASE-11074
URL: https://issues.apache.org/jira/browse/HBASE-11074
Project: HBase
Issue Type: Improvement
Components: Performance
Reporter: stack
Assignee: stack
Priority: Minor
Fix For: 0.99.0
Attachments: 11074.txt
PE emits progress reading and writing. Add to the progress emission current
histogram snapshot readings. Means don' t have to wait till test completes
to get idea of latencies. Here is sample:
{code}
2014-04-24 22:47:28,085 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/188730/1048576, Min=137.00, Mean=578.74, Max=5152756.00, StdDev=11884.79,
95th=1590.00, 99th=2950.68
2014-04-24 22:47:32,465 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/199215/1048576, Min=137.00, Mean=570.19, Max=5152756.00, StdDev=11591.56,
95th=1543.00, 99th=2911.00
2014-04-24 22:47:37,334 INFO [TestClient-0] hbase.PerformanceEvaluation:
0/209700/1048576, Min=137.00, Mean=564.80, Max=5152756.00, StdDev=11317.96,
95th=1480.00, 99th=2863.00
{code}
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Assigned] (HBASE-11072) Abstract WAL splitting from ZK
[ https://issues.apache.org/jira/browse/HBASE-11072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov reassigned HBASE-11072: --- Assignee: Mikhail Antonov Abstract WAL splitting from ZK -- Key: HBASE-11072 URL: https://issues.apache.org/jira/browse/HBASE-11072 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Affects Versions: 0.99.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov HM side: - SplitLogManager RS side: - SplitLogWorker - HLogSplitter and a few handler classes. This jira may need to be split further apart into smaller ones. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10915) Decouple region closing (HM and HRS) from ZK
[ https://issues.apache.org/jira/browse/HBASE-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-10915: Status: Open (was: Patch Available) Decouple region closing (HM and HRS) from ZK Key: HBASE-10915 URL: https://issues.apache.org/jira/browse/HBASE-10915 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Affects Versions: 0.99.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch Decouple region closing from ZK. Includes RS side (CloseRegionHandler), HM side (ClosedRegionHandler) and the code using (HRegionServer, RSRpcServices etc). May need small changes in AssignmentManager. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11059) ZK-less region assignment
[ https://issues.apache.org/jira/browse/HBASE-11059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981504#comment-13981504 ] Mikhail Antonov commented on HBASE-11059: - [~jxiang] thanks for desctiption! look forward for the writeup. ZK-less region assignment - Key: HBASE-11059 URL: https://issues.apache.org/jira/browse/HBASE-11059 Project: HBase Issue Type: Improvement Components: master, Region Assignment Reporter: Jimmy Xiang Assignee: Jimmy Xiang It seems that most people don't like region assignment with ZK (HBASE-5487), which causes many uncertainties. This jira is to support ZK-less region assignment. We need to make sure this patch doesn't break backward compatibility/rolling upgrade. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10915) Decouple region closing (HM and HRS) from ZK
[ https://issues.apache.org/jira/browse/HBASE-10915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mikhail Antonov updated HBASE-10915: Status: Patch Available (was: Open) Decouple region closing (HM and HRS) from ZK Key: HBASE-10915 URL: https://issues.apache.org/jira/browse/HBASE-10915 Project: HBase Issue Type: Sub-task Components: Consensus, Zookeeper Affects Versions: 0.99.0 Reporter: Mikhail Antonov Assignee: Mikhail Antonov Attachments: HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch, HBASE-10915.patch Decouple region closing from ZK. Includes RS side (CloseRegionHandler), HM side (ClosedRegionHandler) and the code using (HRegionServer, RSRpcServices etc). May need small changes in AssignmentManager. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11008) Align bulk load, flush, and compact to require Action.CREATE
[ https://issues.apache.org/jira/browse/HBASE-11008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-11008: --- Release Note: preBulkLoadHFile now requires CREATE, which it effectively already needed since getTableDescriptor also requires it which is what LoadIncrementalHFiles is doing before bulk loading. compact and flush can now be issued by users with CREATE permission. I committed to 0.96 and up. I'm waiting on [~lhofhansl]'s +1 for 0.94 (or I can create a backport jira). Align bulk load, flush, and compact to require Action.CREATE Key: HBASE-11008 URL: https://issues.apache.org/jira/browse/HBASE-11008 Project: HBase Issue Type: Improvement Components: security Reporter: Jean-Daniel Cryans Assignee: Jean-Daniel Cryans Fix For: 0.99.0, 0.98.2, 0.96.3, 0.94.20 Attachments: HBASE-11008-0.94.patch, HBASE-11008-v2.patch, HBASE-11008-v3.patch, HBASE-11008.patch Over in HBASE-10958 we noticed that it might make sense to require Action.CREATE for bulk load, flush, and compact since it is also required for things like enable and disable. This means the following changes: - preBulkLoadHFile goes from WRITE to CREATE - compact/flush go from ADMIN to ADMIN or CREATE -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11038) Filtered scans can bypass metrics collection
[ https://issues.apache.org/jira/browse/HBASE-11038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981541#comment-13981541 ] Hudson commented on HBASE-11038: FAILURE: Integrated in HBase-0.98-on-Hadoop-1.1 #280 (See [https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/280/]) HBASE-11038 Filtered scans can bypass metrics collection (ndimiduk: rev 1590068) * /hbase/branches/0.98/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Filtered scans can bypass metrics collection Key: HBASE-11038 URL: https://issues.apache.org/jira/browse/HBASE-11038 Project: HBase Issue Type: Bug Components: Scanners Affects Versions: 0.96.2, 0.98.1, 0.99.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 0.99.0, 0.98.2, 0.96.3 Attachments: HBASE-11038.00.patch, HBASE-11038.01.96.patch, HBASE-11038.01.98.patch, HBASE-11038.01.patch In RegionScannerImpl#nextRaw, after a batch of results are retrieved, delegates to the filter regarding continuation of the scan. If filterAllRemaining returns true, the method exits immediately, without calling MetricsRegion#updateNextScan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11050) Replace empty catch block in TestHLog#testFailedToCreateHLogIfParentRenamed with @Test(expected=)
[
https://issues.apache.org/jira/browse/HBASE-11050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981566#comment-13981566
]
Ted Yu commented on HBASE-11050:
I think this would be closed when hbase 1.0 is released.
Replace empty catch block in TestHLog#testFailedToCreateHLogIfParentRenamed
with @Test(expected=)
--
Key: HBASE-11050
URL: https://issues.apache.org/jira/browse/HBASE-11050
Project: HBase
Issue Type: Task
Reporter: Gustavo Anatoly
Assignee: Gustavo Anatoly
Priority: Trivial
Fix For: 0.99.0
Attachments: HBASE-11050.patch
This change refactor TestHLog#testFailedToCreateHLogIfParentRenamed. The test
basically create {{HLogFactory.createWALWriter(fs, path, conf);}} and after
that parent {{path}} is renamed followed to another call
{{HLogFactory.createWALWriter(fs, path, conf);}} in this moment is expected
IOException, because the parent {{path}} doesn't exist more.
The second call monitored by a block try-catch with an empty {{catch}}:
{code}
try {
HLogFactory.createWALWriter(fs, path, conf);
fail(It should fail to create the new WAL);
} catch (IOException ioe) {
// expected, good.
}
{code}
The patch proposed removes the {{try-catch}} and use
{{@Test(expected=IOException.class)}} to capture exception produced by the
test.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Created] (HBASE-11080) TestZKSecretWatcher#testKeyUpdate occasionally fails
Ted Yu created HBASE-11080:
--
Summary: TestZKSecretWatcher#testKeyUpdate occasionally fails
Key: HBASE-11080
URL: https://issues.apache.org/jira/browse/HBASE-11080
Project: HBase
Issue Type: Test
Affects Versions: 0.98.1
Reporter: Ted Yu
Priority: Minor
From
https://builds.apache.org/job/HBase-0.98-on-Hadoop-1.1/280/testReport/junit/org.apache.hadoop.hbase.security.token/TestZKSecretWatcher/testKeyUpdate/
:
{code}
java.lang.AssertionError
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertNotNull(Assert.java:621)
at org.junit.Assert.assertNotNull(Assert.java:631)
at
org.apache.hadoop.hbase.security.token.TestZKSecretWatcher.testKeyUpdate(TestZKSecretWatcher.java:221)
{code}
Here is the assertion that failed:
{code}
assertNotNull(newMaster);
{code}
Looks like new master did not come up within 5 tries.
One potential fix is to increase the number of attempts.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Updated] (HBASE-10960) Enhance HBase Thrift 1 to include append and checkAndPut operations
[ https://issues.apache.org/jira/browse/HBASE-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-10960: --- Attachment: hbase-10960.v3.patch Attaching +1'ed patch for hadoopqa to test. Enhance HBase Thrift 1 to include append and checkAndPut operations --- Key: HBASE-10960 URL: https://issues.apache.org/jira/browse/HBASE-10960 Project: HBase Issue Type: Improvement Components: Thrift Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Fix For: 0.98.2 Attachments: HBASE-10960.patch, hbase-10960.v3.patch Both append, and checkAndPut functionalities are available in Thrift 2 interface, but not in Thrift. So, adding the support for these functionalities in Thrift1 too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10960) Enhance HBase Thrift 1 to include append and checkAndPut operations
[ https://issues.apache.org/jira/browse/HBASE-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-10960: --- Fix Version/s: (was: 0.98.2) 0.99.0 Hadoop Flags: Reviewed Status: Patch Available (was: Open) Enhance HBase Thrift 1 to include append and checkAndPut operations --- Key: HBASE-10960 URL: https://issues.apache.org/jira/browse/HBASE-10960 Project: HBase Issue Type: Improvement Components: Thrift Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Fix For: 0.99.0 Attachments: HBASE-10960.patch, hbase-10960.v3.patch Both append, and checkAndPut functionalities are available in Thrift 2 interface, but not in Thrift. So, adding the support for these functionalities in Thrift1 too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-5697) Audit HBase for usage of deprecated hadoop 0.20.x property names.
[ https://issues.apache.org/jira/browse/HBASE-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981597#comment-13981597 ] Jonathan Hsieh commented on HBASE-5697: --- np. I'm +1 for the patch if it applies and passes. Can you upload an updated version and kick off the hadoopqa build bot again? I'd like to see the build work before I commit. Audit HBase for usage of deprecated hadoop 0.20.x property names. - Key: HBASE-5697 URL: https://issues.apache.org/jira/browse/HBASE-5697 Project: HBase Issue Type: Task Reporter: Jonathan Hsieh Assignee: Srikanth Srungarapu Labels: noob Attachments: HBASE-5697.patch, HBASE-5697_v2.patch, deprecated_properties Many xml config properties in Hadoop have changed in 0.23. We should audit hbase to insulate it from hadoop property name changes. Here is a list of the hadoop property name changes: http://hadoop.apache.org/common/docs/r0.23.1/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11053) Change DeleteTracker APIs to work with Cell
[ https://issues.apache.org/jira/browse/HBASE-11053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981582#comment-13981582 ] Andrew Purtell commented on HBASE-11053: bq. If I commit this to 0.98 one thing is as there is no Cell related changes in 0.98, so passing kv to the DeleteTracker would make it do Kv.getqualifierOffset() , kv.getQualifierLength(), kv.getTimeStamp(), kv.getTypeByte() once again which is already extracted out in the SQM itself. Yes, well we need this in 0.98 so please consider making the change and attach what you commit here. +1 if what you describe above is the only change necessary. Change DeleteTracker APIs to work with Cell --- Key: HBASE-11053 URL: https://issues.apache.org/jira/browse/HBASE-11053 Project: HBase Issue Type: Sub-task Affects Versions: 0.98.1, 0.99.0 Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 0.99.0, 0.98.2 Attachments: HBASE-11053.patch, HBASE-11053_0.98.patch, HBASE-11053_1.patch DeleteTracker interface (marked as Private) should work with Cells. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10958) [dataloss] Bulk loading with seqids can prevent some log entries from being replayed
[ https://issues.apache.org/jira/browse/HBASE-10958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jean-Daniel Cryans updated HBASE-10958: --- Release Note: Bulk loading with sequence IDs, an option in late 0.94 releases and the default since 0.96.0, will now trigger a flush per region that loads an HFile (if there's data that needs to flushed). Committed to 0.96 and up. Like for HBASE-11008, I'm waiting to commit to 0.94 or I can open a backport jira. [dataloss] Bulk loading with seqids can prevent some log entries from being replayed Key: HBASE-10958 URL: https://issues.apache.org/jira/browse/HBASE-10958 Project: HBase Issue Type: Bug Affects Versions: 0.96.2, 0.98.1, 0.94.18 Reporter: Jean-Daniel Cryans Assignee: Jean-Daniel Cryans Priority: Blocker Fix For: 0.99.0, 0.98.2, 0.96.3, 0.94.20 Attachments: HBASE-10958-0.94.patch, HBASE-10958-less-intrusive-hack-0.96.patch, HBASE-10958-quick-hack-0.96.patch, HBASE-10958-v2.patch, HBASE-10958-v3.patch, HBASE-10958.patch We found an issue with bulk loads causing data loss when assigning sequence ids (HBASE-6630) that is triggered when replaying recovered edits. We're nicknaming this issue *Blindspot*. The problem is that the sequence id given to a bulk loaded file is higher than those of the edits in the region's memstore. When replaying recovered edits, the rule to skip some of them is that they have to be _lower than the highest sequence id_. In other words, the edits that have a sequence id lower than the highest one in the store files *should* have also been flushed. This is not the case with bulk loaded files since we now have an HFile with a sequence id higher than unflushed edits. The log recovery code takes this into account by simply skipping the bulk loaded files, but this bulk loaded status is *lost* on compaction. The edits in the logs that have a sequence id lower than the bulk loaded file that got compacted are put in a blind spot and are skipped during replay. Here's the easiest way to recreate this issue: - Create an empty table - Put one row in it (let's say it gets seqid 1) - Bulk load one file (it gets seqid 2). I used ImporTsv and set hbase.mapreduce.bulkload.assign.sequenceNumbers. - Bulk load a second file the same way (it gets seqid 3). - Major compact the table (the new file has seqid 3 and isn't considered bulk loaded). - Kill the region server that holds the table's region. - Scan the table once the region is made available again. The first row, at seqid 1, will be missing since the HFile with seqid 3 makes us believe that everything that came before it was flushed. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-10960) Enhance HBase Thrift 1 to include append and checkAndPut operations
[ https://issues.apache.org/jira/browse/HBASE-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981617#comment-13981617 ] Jonathan Hsieh commented on HBASE-10960: Since this only affects thrift, I ran TestThriftServer with the patch applied and it passed. Thanks for the patch Srikanth, committing to trunk. Enhance HBase Thrift 1 to include append and checkAndPut operations --- Key: HBASE-10960 URL: https://issues.apache.org/jira/browse/HBASE-10960 Project: HBase Issue Type: Improvement Components: Thrift Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Fix For: 0.99.0 Attachments: HBASE-10960.patch, hbase-10960.v3.patch Both append, and checkAndPut functionalities are available in Thrift 2 interface, but not in Thrift. So, adding the support for these functionalities in Thrift1 too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10960) Enhance HBase Thrift 1 to include append and checkAndPut operations
[ https://issues.apache.org/jira/browse/HBASE-10960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Hsieh updated HBASE-10960: --- Resolution: Fixed Status: Resolved (was: Patch Available) Enhance HBase Thrift 1 to include append and checkAndPut operations --- Key: HBASE-10960 URL: https://issues.apache.org/jira/browse/HBASE-10960 Project: HBase Issue Type: Improvement Components: Thrift Reporter: Srikanth Srungarapu Assignee: Srikanth Srungarapu Fix For: 0.99.0 Attachments: HBASE-10960.patch, hbase-10960.v3.patch Both append, and checkAndPut functionalities are available in Thrift 2 interface, but not in Thrift. So, adding the support for these functionalities in Thrift1 too. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11038) Filtered scans can bypass metrics collection
[ https://issues.apache.org/jira/browse/HBASE-11038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981626#comment-13981626 ] Hudson commented on HBASE-11038: SUCCESS: Integrated in hbase-0.96 #394 (See [https://builds.apache.org/job/hbase-0.96/394/]) HBASE-11038 Filtered scans can bypass metrics collection (ndimiduk: rev 1590069) * /hbase/branches/0.96/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegion.java Filtered scans can bypass metrics collection Key: HBASE-11038 URL: https://issues.apache.org/jira/browse/HBASE-11038 Project: HBase Issue Type: Bug Components: Scanners Affects Versions: 0.96.2, 0.98.1, 0.99.0 Reporter: Nick Dimiduk Assignee: Nick Dimiduk Priority: Minor Fix For: 0.99.0, 0.98.2, 0.96.3 Attachments: HBASE-11038.00.patch, HBASE-11038.01.96.patch, HBASE-11038.01.98.patch, HBASE-11038.01.patch In RegionScannerImpl#nextRaw, after a batch of results are retrieved, delegates to the filter regarding continuation of the scan. If filterAllRemaining returns true, the method exits immediately, without calling MetricsRegion#updateNextScan. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Created] (HBASE-11081) Trunk Master won't start; looking for Constructor that takes conf only
stack created HBASE-11081: - Summary: Trunk Master won't start; looking for Constructor that takes conf only Key: HBASE-11081 URL: https://issues.apache.org/jira/browse/HBASE-11081 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.99.0 Committing the Consensus Infra, we broke starting master. Small fix so constructMaster passes in a ConsensusProvider. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-11081) Trunk Master won't start; looking for Constructor that takes conf only
[ https://issues.apache.org/jira/browse/HBASE-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-11081: -- Attachment: 11081.txt Small patch. Tried it on cluster and it works. Going to commit. Trunk Master won't start; looking for Constructor that takes conf only -- Key: HBASE-11081 URL: https://issues.apache.org/jira/browse/HBASE-11081 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.99.0 Attachments: 11081.txt Committing the Consensus Infra, we broke starting master. Small fix so constructMaster passes in a ConsensusProvider. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-5697) Audit HBase for usage of deprecated hadoop 0.20.x property names.
[ https://issues.apache.org/jira/browse/HBASE-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-5697: --- Fix Version/s: 0.99.0 Audit HBase for usage of deprecated hadoop 0.20.x property names. - Key: HBASE-5697 URL: https://issues.apache.org/jira/browse/HBASE-5697 Project: HBase Issue Type: Task Reporter: Jonathan Hsieh Assignee: Srikanth Srungarapu Labels: noob Fix For: 0.99.0 Attachments: HBASE-5697.patch, HBASE-5697_v2.patch, deprecated_properties Many xml config properties in Hadoop have changed in 0.23. We should audit hbase to insulate it from hadoop property name changes. Here is a list of the hadoop property name changes: http://hadoop.apache.org/common/docs/r0.23.1/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10918) [VisibilityController] System table backed ScanLabelGenerator
[ https://issues.apache.org/jira/browse/HBASE-10918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-10918: --- Status: Open (was: Patch Available) [VisibilityController] System table backed ScanLabelGenerator -- Key: HBASE-10918 URL: https://issues.apache.org/jira/browse/HBASE-10918 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Critical Fix For: 0.99.0, 0.98.2 Attachments: HBASE-10918.patch A ScanLabelGenerator that retrieves a static set of authorizations for a user or group from a new HBase system table, and insures these auths are part of the effective set. Useful for forcing a baseline set of auths for a user. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11079) Normalize test tools across branches
[ https://issues.apache.org/jira/browse/HBASE-11079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981637#comment-13981637 ] Nick Dimiduk commented on HBASE-11079: -- Re: PerfEval, 0.94 has been left behind. I tried backporting patches on a couple instances; it's a struggle. May be worth trying a blanket paste-over from 0.96. Normalize test tools across branches Key: HBASE-11079 URL: https://issues.apache.org/jira/browse/HBASE-11079 Project: HBase Issue Type: Umbrella Reporter: Andrew Purtell Will be a challenge wherever the branches vary functionally, but it would be good to normalize the test tools (LoadTestTool and PerformanceEvaluation) as much as possible among the active branches so we can compare them. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10918) [VisibilityController] System table backed ScanLabelGenerator
[ https://issues.apache.org/jira/browse/HBASE-10918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-10918: --- Status: Patch Available (was: Open) [VisibilityController] System table backed ScanLabelGenerator -- Key: HBASE-10918 URL: https://issues.apache.org/jira/browse/HBASE-10918 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Critical Fix For: 0.99.0, 0.98.2 Attachments: HBASE-10918.patch, HBASE-10918_1.patch A ScanLabelGenerator that retrieves a static set of authorizations for a user or group from a new HBase system table, and insures these auths are part of the effective set. Useful for forcing a baseline set of auths for a user. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-10918) [VisibilityController] System table backed ScanLabelGenerator
[ https://issues.apache.org/jira/browse/HBASE-10918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell updated HBASE-10918: --- Attachment: HBASE-10918_1.patch Patch 10918_1 adds a new test. Passes locally. What I will commit soon. [VisibilityController] System table backed ScanLabelGenerator -- Key: HBASE-10918 URL: https://issues.apache.org/jira/browse/HBASE-10918 Project: HBase Issue Type: Sub-task Reporter: Andrew Purtell Assignee: Andrew Purtell Priority: Critical Fix For: 0.99.0, 0.98.2 Attachments: HBASE-10918.patch, HBASE-10918_1.patch A ScanLabelGenerator that retrieves a static set of authorizations for a user or group from a new HBase system table, and insures these auths are part of the effective set. Useful for forcing a baseline set of auths for a user. -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Updated] (HBASE-5697) Audit HBase for usage of deprecated hadoop 0.20.x property names.
[ https://issues.apache.org/jira/browse/HBASE-5697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Srikanth Srungarapu updated HBASE-5697: --- Attachment: HBASE-5697_v3.patch Uploading the same patch for HadoopQA bot to pick it up. Audit HBase for usage of deprecated hadoop 0.20.x property names. - Key: HBASE-5697 URL: https://issues.apache.org/jira/browse/HBASE-5697 Project: HBase Issue Type: Task Reporter: Jonathan Hsieh Assignee: Srikanth Srungarapu Labels: noob Fix For: 0.99.0 Attachments: HBASE-5697.patch, HBASE-5697_v2.patch, HBASE-5697_v3.patch, deprecated_properties Many xml config properties in Hadoop have changed in 0.23. We should audit hbase to insulate it from hadoop property name changes. Here is a list of the hadoop property name changes: http://hadoop.apache.org/common/docs/r0.23.1/hadoop-project-dist/hadoop-common/DeprecatedProperties.html -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (HBASE-11081) Trunk Master won't start; looking for Constructor that takes conf only
[ https://issues.apache.org/jira/browse/HBASE-11081?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13981652#comment-13981652 ] Mikhail Antonov commented on HBASE-11081: - Ouch :( Should we have a second constructor in HMaster jsut as for HRS? Trunk Master won't start; looking for Constructor that takes conf only -- Key: HBASE-11081 URL: https://issues.apache.org/jira/browse/HBASE-11081 Project: HBase Issue Type: Bug Reporter: stack Assignee: stack Fix For: 0.99.0 Attachments: 11081.txt Committing the Consensus Infra, we broke starting master. Small fix so constructMaster passes in a ConsensusProvider. -- This message was sent by Atlassian JIRA (v6.2#6252)
