Re: How to make HBase survive a laptop standby
Could you clarify what you mean by 'network partition' ? First blush... not a good idea. Can you please explain how a laptop suspend is an issue for those running HBase in a production environment? I really want to encourage the committers to think more like product owners because HBase is now being presented to the enterprise by Cloudera and Amazon. (MapR has M7 and I haven't seen either IBM or Pivotal in the wild.) On Jun 4, 2014, at 10:30 AM, Cosmin Lehene cleh...@adobe.com wrote: Thanks Andy, This is in lines with what I was asking/thinking of. To separate concerns a bit: The question is if HRegionSever instead of exiting could have a standby state (maintaining the current region state would be optional depending whether we have shadow regions or not) where it doesn¹t serve requests, but still visible to the cluster. This way we could ³wake it up² if we decide that is safe (e.g. we remain consistent). This scenarios seems to solve both my laptop standby problem as well as (potentially fast) recovery after a network partition. Cosmin On 6/3/14, 3:42 PM, Andrew Purtell apurt...@apache.org wrote: Although a dev lapop suspend and resume is equivalent to a total network failure for the elapsed time, with all that implies, perhaps after HBASE-10070 goes in it could be possible for a RS to stay up with regions switched to read only replica mode, reconnect, and perhaps the master upon discovering all regions still available on the cluster but in read only mode (no primary), it could reassign primaries - the net effect being indeed no process needs restarting by a supervisor. On Tue, Jun 3, 2014 at 3:27 PM, Andrew Purtell apurt...@apache.org wrote: Stating the obvious you just need to restart the RegionServer because it shut down. We use ZooKeeper for tracking server liveness and from the ZooKeeper perspective a sufficiently long time elapsed without heartbeat such that the RegionServer's session expired. We've left this option to date to the user to do with supervisory scripts, e.g. Puppet / Chef / Daemontools. I suppose a RegionServer could try and reinitialize as a new process or the ./bin/hbase script could do this if you ask. On Tue, Jun 3, 2014 at 2:46 PM, Cosmin Lehene cleh...@adobe.com wrote: I just realized that, for years, I've been countlessly restarted hbase every time my laptop gets out of standby. I know well why I do this, but I also know I could probably not do it and that I don't have to do with Hadoop or Zookeeper or other services and I wish I wouldn't need to with Hbase either. So short term, I'd like to know if there a better way already. Long term I think this is a bigger, more fundamental resiliency aspect that perhaps is not trivial, but probably worth thinking about in the real deployments context and I wonder if there's something that tries to solve this already. Thoughts? Thanks, Cosmin -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White) -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
[jira] [Created] (HBASE-11300) Wrong permission check for checkAndPut in AccessController
Liu Shaohui created HBASE-11300:
---
Summary: Wrong permission check for checkAndPut in AccessController
Key: HBASE-11300
URL: https://issues.apache.org/jira/browse/HBASE-11300
Project: HBase
Issue Type: Bug
Components: security
Affects Versions: 0.99.0
Reporter: Liu Shaohui
Assignee: Liu Shaohui
Priority: Minor
For the checkAndPut operation, the AccessController only checks the read and
write permission for the family and qualifier to check, but ignores the write
permission for the family map of put. What's more, we don't need the write
permission for the family and qualifier to check.
See the code AccessController.java #1538
{code}
Mapbyte[],? extends Collectionbyte[] families = makeFamilyMap(family,
qualifier);
User user = getActiveUser();
AuthResult authResult = permissionGranted(OpType.CHECK_AND_PUT, user, env,
families,
Action.READ, Action.WRITE);
{code}
Same problem for checkAndDelete operation.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
Re: How to make HBase survive a laptop standby
On Thu, Jun 5, 2014 at 5:05 AM, Michael Segel msegel_had...@hotmail.com wrote: Could you clarify what you mean by 'network partition' ? First blush... not a good idea. Can you please explain how a laptop suspend is an issue for those running HBase in a production environment? Cosmin did not say that Michael. St.Ack
coprocessor piggy-back on compaction logic
hi, folks, I am thinking about the possibility to implement a table statistic collector feature by plugging in exiting compaction logic through coprocessor. Wondering someone can give me a couple pointer/jira that already piggy-back on compaction? so I don't need to reinvent the wheel? Many thanks Demai
Re: coprocessor piggy-back on compaction logic
HBASE-7958 --- Jesse Yates @jesse_yates jyates.github.com On Thu, Jun 5, 2014 at 11:43 AM, Demai Ni nid...@gmail.com wrote: hi, folks, I am thinking about the possibility to implement a table statistic collector feature by plugging in exiting compaction logic through coprocessor. Wondering someone can give me a couple pointer/jira that already piggy-back on compaction? so I don't need to reinvent the wheel? Many thanks Demai
Re: coprocessor piggy-back on compaction logic
Jesse, you are fast. :-) thanks a lot... Demai On Thu, Jun 5, 2014 at 11:45 AM, Jesse Yates jesse.k.ya...@gmail.com wrote: HBASE-7958 --- Jesse Yates @jesse_yates jyates.github.com On Thu, Jun 5, 2014 at 11:43 AM, Demai Ni nid...@gmail.com wrote: hi, folks, I am thinking about the possibility to implement a table statistic collector feature by plugging in exiting compaction logic through coprocessor. Wondering someone can give me a couple pointer/jira that already piggy-back on compaction? so I don't need to reinvent the wheel? Many thanks Demai
HConstants.ALL_VERSIONS
Mates,
Seems that HConstants.ALL_VERSIONS is not used anymore. We have
HContants.FOREVER used to disable TTL, I will have expected ALL_VERSIONS
to be the same for the versions. However, it seems to not be used anywhere.
Should we just remove it from HContants? Or should we make sure it's used?
On the Scan class we have this:
public Scan setMaxVersions() {
this.maxVersions = Integer.MAX_VALUE;
return this;
}
Should we change it by HConstants.ALL_VERSIONS instead of
Integer.MAX_VALUE? It's the same value but more clear? And should we make
sure on the compaction side we take care of this in case we want to never
delete any version?
JM
Re: coprocessor piggy-back on compaction logic
Would be awesome if you pick that up Demai! On Thu, Jun 5, 2014 at 11:47 AM, Demai Ni nid...@gmail.com wrote: Jesse, you are fast. :-) thanks a lot... Demai On Thu, Jun 5, 2014 at 11:45 AM, Jesse Yates jesse.k.ya...@gmail.com wrote: HBASE-7958 --- Jesse Yates @jesse_yates jyates.github.com On Thu, Jun 5, 2014 at 11:43 AM, Demai Ni nid...@gmail.com wrote: hi, folks, I am thinking about the possibility to implement a table statistic collector feature by plugging in exiting compaction logic through coprocessor. Wondering someone can give me a couple pointer/jira that already piggy-back on compaction? so I don't need to reinvent the wheel? Many thanks Demai -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
Re: coprocessor piggy-back on compaction logic
Andrew, I will definitely take a look, would be a good one to play with ... Demai On Thu, Jun 5, 2014 at 12:20 PM, Andrew Purtell apurt...@apache.org wrote: Would be awesome if you pick that up Demai! On Thu, Jun 5, 2014 at 11:47 AM, Demai Ni nid...@gmail.com wrote: Jesse, you are fast. :-) thanks a lot... Demai On Thu, Jun 5, 2014 at 11:45 AM, Jesse Yates jesse.k.ya...@gmail.com wrote: HBASE-7958 --- Jesse Yates @jesse_yates jyates.github.com On Thu, Jun 5, 2014 at 11:43 AM, Demai Ni nid...@gmail.com wrote: hi, folks, I am thinking about the possibility to implement a table statistic collector feature by plugging in exiting compaction logic through coprocessor. Wondering someone can give me a couple pointer/jira that already piggy-back on compaction? so I don't need to reinvent the wheel? Many thanks Demai -- Best regards, - Andy Problems worthy of attack prove their worth by hitting back. - Piet Hein (via Tom White)
