[jira] [Commented] (HBASE-19090) Add config 'hbase.systemtables.compacting.memstore.type' to docs
[ https://issues.apache.org/jira/browse/HBASE-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220055#comment-16220055 ] Chia-Ping Tsai commented on HBASE-19090: +1 > Add config 'hbase.systemtables.compacting.memstore.type' to docs > > > Key: HBASE-19090 > URL: https://issues.apache.org/jira/browse/HBASE-19090 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19090.patch > > > As discussed in parent JIRA the new config > 'hbase.systemtables.compacting.memstore.type' has to be added to docs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220053#comment-16220053 ] stack edited comment on HBASE-18770 at 10/26/17 6:53 AM: - Thanks for reviews so far. Helps. I think I have a pattern now. Methods that are bypassable are given the same basic signature; i.e. the return is a bypass boolean and an extra 'result' param is what we return if bypass is true: e.g: default boolean preGetOp(ObserverContext c, Get get, List result) throws IOException This doesn't deviate too much from what was in place for a few methods already. No more Optional. Don't need it anymore. Was being used so a boolean return could be overloaded to signify what to return on bypass. We simplified so we don't need this anymore. I thought I could purge the 'complete' background flag and processing too (complete is how a CP says skip outstanding CPs in current execution chain, usually used to force bypass result) but looks like purge would cause a ripple that would take me too long to come back from so punting for now; will keep on w/ the complete though it is crimped now such that it only applies to the few methods that support bypass. Just a status. Am down in the depths doing grunt work to make convertion. More detail to follow. was (Author: stack): Thanks for reviews so far. I think I have a pattern now. Methods that are bypassable are given the same basic signature; i.e. the return is a bypass boolean and an extra 'result' param is what we return if bypass is true: e.g: default boolean preGetOp(ObserverContext c, Get get, List result) throws IOException This doesn't deviate too much from what was in place for a few methods already. No more Optional. Don't need it anymore. Was being used so a boolean return could be overloaded to signify what to return on bypass. We simplified so we don't need this anymore. I thought I could purge the 'complete' background flag and processing too (complete is how a CP says skip outstanding CPs in current execution chain, usually used to force bypass result) but looks like purge would cause a ripple that would take me too long to come back from so punting for now; will keep on w/ the complete though it is crimped now such that it only applies to the few methods that support bypass. Just a status. Am down in the depths doing grunt work to make convertion. More detail to follow. > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220053#comment-16220053 ] stack commented on HBASE-18770: --- Thanks for reviews so far. I think I have a pattern now. Methods that are bypassable are given the same basic signature; i.e. the return is a bypass boolean and an extra 'result' param is what we return if bypass is true: e.g: default boolean preGetOp(ObserverContext c, Get get, List result) throws IOException This doesn't deviate too much from what was in place for a few methods already. No more Optional. Don't need it anymore. Was being used so a boolean return could be overloaded to signify what to return on bypass. We simplified so we don't need this anymore. I thought I could purge the 'complete' background flag and processing too (complete is how a CP says skip outstanding CPs in current execution chain, usually used to force bypass result) but looks like purge would cause a ripple that would take me too long to come back from so punting for now; will keep on w/ the complete though it is crimped now such that it only applies to the few methods that support bypass. Just a status. Am down in the depths doing grunt work to make convertion. More detail to follow. > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19048) Cleanup MasterObserver hooks which takes IA private params
[
https://issues.apache.org/jira/browse/HBASE-19048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220044#comment-16220044
]
stack commented on HBASE-19048:
---
Thanks lads. Let me review imports and then commit.
> Cleanup MasterObserver hooks which takes IA private params
> --
>
> Key: HBASE-19048
> URL: https://issues.apache.org/jira/browse/HBASE-19048
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Anoop Sam John
>Assignee: stack
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-19048.master.001.patch,
> HBASE-19048.master.002.patch, HBASE-19048.master.003.patch,
> HBASE-19048.master.003.patch
>
>
> These are the ones in MasterObserver
> {code}
> preAbortProcedure- ProcedureExecutor
> postGetProcedures- Procedure
> postGetLocks - LockedResource
> preRequestLock - LockType
> postRequestLock - LockType
> preLockHeartbeat - LockProcedure
> postLockHeartbeat- LockProcedure
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
stack updated HBASE-13346:
--
Attachment: HBASE-13346.master.011.patch
Retry. Previous didn't seem to get notice.
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch, HBASE-13346.master.004.patch,
> HBASE-13346.master.005.patch, HBASE-13346.master.006.patch,
> HBASE-13346.master.007.patch, HBASE-13346.master.008.patch,
> HBASE-13346.master.009.patch, HBASE-13346.master.010.patch,
> HBASE-13346.master.011.patch, HBASE-13346.master.011.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19047) CP exposed Scanner types should not extend Shipper
[ https://issues.apache.org/jira/browse/HBASE-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220038#comment-16220038 ] Duo Zhang commented on HBASE-19047: --- What if a user just opens a new RegionScanner and return it and then close the one you passed in? This may be done by calling instanceof on the returned RegionScanner, but what if user wraps it before returning it? And I think RegionScanner is way too complicated to implement by a CP users, so maybe just like the Region/Store way? You can give me a Region/Store, but I can safely cast it to HRegion/HStore as I'm sure there is no other implementation. The only exception is InternalScanner, which we allow user to do wrapping. Thanks. > CP exposed Scanner types should not extend Shipper > -- > > Key: HBASE-19047 > URL: https://issues.apache.org/jira/browse/HBASE-19047 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19047.patch, HBASE-19047_V2.patch, > HBASE-19047_V2.patch > > > Shipper is a IA.Private interface and very much internal.. > Right now CP exposed RegionScanner is extending this and so exposing the > shipped() method. This by mistake is called, can harm the correctness of the > cells in the Results. > preScannerOpen() allowing to return a new Scanner is also problematic now. > This can allow users to create a Region scanner from Region and then wrap it > and return back (Well same can be done by postScannerOpen also), it can so > happen that the wrapper is not implementing the shipped() properly. In any > way exposing the shipped () is problematic. > Solution Steps > 1. Remove preScannerOpen() , the use case I can think of is wrapping the > original scanner. The original scanner can be created by Region.getScanner > way only.. May be no need to remove this hook. Just remove the ability for > it to return a RegionScanner instance. Call this with the Scan object and > the CP can change the Scan object if they want. > 2. Let RegionScanner not extending Shipper but only RegionScannerImpl > implements this > 3. We have ref to the RegionScanner created by core and let that be used by > RegionScannerShippedCallBack when the post hook doing a wrap. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220036#comment-16220036 ] stack commented on HBASE-19092: --- What [~anoop.hbase] said. > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-beta-1 > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220034#comment-16220034 ] ramkrishna.s.vasudevan commented on HBASE-19092: Am ok. But I think along with that we will also do some Util method clean up? > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-beta-1 > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19091) Code annotation wrote "BinaryComparator" instead of "LongComparator"
[ https://issues.apache.org/jira/browse/HBASE-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qilin Cao updated HBASE-19091: -- Status: Patch Available (was: Open) > Code annotation wrote "BinaryComparator" instead of "LongComparator" > > > Key: HBASE-19091 > URL: https://issues.apache.org/jira/browse/HBASE-19091 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 3.0.0 >Reporter: Qilin Cao >Assignee: Qilin Cao >Priority: Minor > Attachments: HBASE-19091-v1.patch > > > LongComparator class code annotation wrote "BinaryComparator" instead of > "LongComparator" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
[ https://issues.apache.org/jira/browse/HBASE-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220032#comment-16220032 ] Anoop Sam John commented on HBASE-19092: This is about making some new things for CP expose and not changing or shutting down already exposed CP classes/APIs. So it is ok to do after alpha-4 IMO. If u can make the patch quickly, we can get that in for Alpha-4 also. I assure you a quick review once patch is there :-) > Make Tag IA.LimitedPrivate and expose for CPs > - > > Key: HBASE-19092 > URL: https://issues.apache.org/jira/browse/HBASE-19092 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-beta-1 > > > We need to make tags as LimitedPrivate as some use cases are trying to use > tags like timeline server. The same topic was discussed in dev@ and also in > HBASE-18995. > Shall we target this for beta1 - cc [~saint@gmail.com]. > So once we do this all related Util methods and APIs should also move to > LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19033) Allow CP users to change versions and TTL before opening StoreScanner
[ https://issues.apache.org/jira/browse/HBASE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220030#comment-16220030 ] Duo Zhang commented on HBASE-19033: --- Yes I plan to add these methods back. But instead of returning an InternalScanner, I prefer to provide a modifiable 'ScanOptions' to let CP users change the max versions and TTL. For in memory compaction, I agree that we need to introduce new hooks. Mind opening a jira? Thanks. > Allow CP users to change versions and TTL before opening StoreScanner > - > > Key: HBASE-19033 > URL: https://issues.apache.org/jira/browse/HBASE-19033 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > > See the discussion in HBASE-19001. Changing versions and TTL is safe for > flush/compaction so we can expose them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19091) Code annotation wrote "BinaryComparator" instead of "LongComparator"
[ https://issues.apache.org/jira/browse/HBASE-19091?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Qilin Cao updated HBASE-19091: -- Attachment: HBASE-19091-v1.patch submit a patch! > Code annotation wrote "BinaryComparator" instead of "LongComparator" > > > Key: HBASE-19091 > URL: https://issues.apache.org/jira/browse/HBASE-19091 > Project: HBase > Issue Type: Improvement > Components: Client >Affects Versions: 3.0.0 >Reporter: Qilin Cao >Assignee: Qilin Cao >Priority: Minor > Attachments: HBASE-19091-v1.patch > > > LongComparator class code annotation wrote "BinaryComparator" instead of > "LongComparator" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19047) CP exposed Scanner types should not extend Shipper
[ https://issues.apache.org/jira/browse/HBASE-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-19047: --- Attachment: HBASE-19047_V2.patch > CP exposed Scanner types should not extend Shipper > -- > > Key: HBASE-19047 > URL: https://issues.apache.org/jira/browse/HBASE-19047 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19047.patch, HBASE-19047_V2.patch, > HBASE-19047_V2.patch > > > Shipper is a IA.Private interface and very much internal.. > Right now CP exposed RegionScanner is extending this and so exposing the > shipped() method. This by mistake is called, can harm the correctness of the > cells in the Results. > preScannerOpen() allowing to return a new Scanner is also problematic now. > This can allow users to create a Region scanner from Region and then wrap it > and return back (Well same can be done by postScannerOpen also), it can so > happen that the wrapper is not implementing the shipped() properly. In any > way exposing the shipped () is problematic. > Solution Steps > 1. Remove preScannerOpen() , the use case I can think of is wrapping the > original scanner. The original scanner can be created by Region.getScanner > way only.. May be no need to remove this hook. Just remove the ability for > it to return a RegionScanner instance. Call this with the Scan object and > the CP can change the Scan object if they want. > 2. Let RegionScanner not extending Shipper but only RegionScannerImpl > implements this > 3. We have ref to the RegionScanner created by core and let that be used by > RegionScannerShippedCallBack when the post hook doing a wrap. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18844) Release hbase-2.0.0-alpha-4; Theme "Coprocessor API Cleanup"
[ https://issues.apache.org/jira/browse/HBASE-18844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220029#comment-16220029 ] stack commented on HBASE-18844: --- Try @busbey suggestion for src tgz: git archive --format=tar.gz --output=/path/to/rc/artifacts/hbase-2.0.0-alpha-3.tar.gz --prefix=hbase-2.0.0-alpha-3/ 2.0.0-alpha-3RC0.2 Compare to what assembly src.tgz generates. > Release hbase-2.0.0-alpha-4; Theme "Coprocessor API Cleanup" > > > Key: HBASE-18844 > URL: https://issues.apache.org/jira/browse/HBASE-18844 > Project: HBase > Issue Type: Bug >Reporter: stack >Assignee: stack > Fix For: 2.0.0-alpha-4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19047) CP exposed Scanner types should not extend Shipper
[ https://issues.apache.org/jira/browse/HBASE-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220026#comment-16220026 ] Anoop Sam John commented on HBASE-19047: [~Apache9] U ok to commit this? On post hook not allow return RegionScanner, I think we need a discuss 1st. Seems QA did not run. > CP exposed Scanner types should not extend Shipper > -- > > Key: HBASE-19047 > URL: https://issues.apache.org/jira/browse/HBASE-19047 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19047.patch, HBASE-19047_V2.patch > > > Shipper is a IA.Private interface and very much internal.. > Right now CP exposed RegionScanner is extending this and so exposing the > shipped() method. This by mistake is called, can harm the correctness of the > cells in the Results. > preScannerOpen() allowing to return a new Scanner is also problematic now. > This can allow users to create a Region scanner from Region and then wrap it > and return back (Well same can be done by postScannerOpen also), it can so > happen that the wrapper is not implementing the shipped() properly. In any > way exposing the shipped () is problematic. > Solution Steps > 1. Remove preScannerOpen() , the use case I can think of is wrapping the > original scanner. The original scanner can be created by Region.getScanner > way only.. May be no need to remove this hook. Just remove the ability for > it to return a RegionScanner instance. Call this with the Scan object and > the CP can change the Scan object if they want. > 2. Let RegionScanner not extending Shipper but only RegionScannerImpl > implements this > 3. We have ref to the RegionScanner created by core and let that be used by > RegionScannerShippedCallBack when the post hook doing a wrap. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19092) Make Tag IA.LimitedPrivate and expose for CPs
ramkrishna.s.vasudevan created HBASE-19092: -- Summary: Make Tag IA.LimitedPrivate and expose for CPs Key: HBASE-19092 URL: https://issues.apache.org/jira/browse/HBASE-19092 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Assignee: ramkrishna.s.vasudevan Fix For: 2.0.0-beta-1 We need to make tags as LimitedPrivate as some use cases are trying to use tags like timeline server. The same topic was discussed in dev@ and also in HBASE-18995. Shall we target this for beta1 - cc [~saint@gmail.com]. So once we do this all related Util methods and APIs should also move to LimitedPrivate Util classes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19091) Code annotation wrote "BinaryComparator" instead of "LongComparator"
Qilin Cao created HBASE-19091: - Summary: Code annotation wrote "BinaryComparator" instead of "LongComparator" Key: HBASE-19091 URL: https://issues.apache.org/jira/browse/HBASE-19091 Project: HBase Issue Type: Improvement Components: Client Affects Versions: 3.0.0 Reporter: Qilin Cao Assignee: Qilin Cao Priority: Minor LongComparator class code annotation wrote "BinaryComparator" instead of "LongComparator" -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220010#comment-16220010 ] Anoop Sam John commented on HBASE-18995: bq.So any name for the InternalCellUtil ? Will change it and then upload to RB. I think this is ok. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19033) Allow CP users to change versions and TTL before opening StoreScanner
[ https://issues.apache.org/jira/browse/HBASE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220008#comment-16220008 ] Anoop Sam John commented on HBASE-19033: So for control of TTL/versions etc at time of flush/compaction, we have to add back these hooks? In mail thread u say so. But in another jira we discussed that the preFlush/Compaction hook can change the scan info and get things done right? On in memory compaction - The flatten and merge op will not remove any versions based on TTL/version. Only the compact mode will do this. That time we will open up scanner (with SQM) for reading the cells to be preserved and SQM will remove some TTL/version expired cells. So u mean this wont call the preFlush kind of hooks and so we are in trouble? hmm.. may be new one hook at least would be needed then, > Allow CP users to change versions and TTL before opening StoreScanner > - > > Key: HBASE-19033 > URL: https://issues.apache.org/jira/browse/HBASE-19033 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > > See the discussion in HBASE-19001. Changing versions and TTL is safe for > flush/compaction so we can expose them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220005#comment-16220005 ] ramkrishna.s.vasudevan commented on HBASE-18995: Ok makes sense. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16220005#comment-16220005 ] ramkrishna.s.vasudevan edited comment on HBASE-18995 at 10/26/17 5:54 AM: -- Ok makes sense. So any name for the InternalCellUtil ? Will change it and then upload to RB. was (Author: ram_krish): Ok makes sense. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219994#comment-16219994 ] Anoop Sam John commented on HBASE-18995: I would say make this new Internal Util class as Private for this jira and commit. Tag making LP and making a LP util class can be done as part of another issue. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-15410) Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList return SEEK_NEXT_USING_HINT
[ https://issues.apache.org/jira/browse/HBASE-15410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219987#comment-16219987 ] Zheng Hu commented on HBASE-15410: -- [~tedyu], Any concern about the addendum ? > Utilize the max seek value when all Filters in MUST_PASS_ALL FilterList > return SEEK_NEXT_USING_HINT > --- > > Key: HBASE-15410 > URL: https://issues.apache.org/jira/browse/HBASE-15410 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Ted Yu > Labels: filter, perfomance > Fix For: 2.0.0-alpha-4, HBASE-18410 > > Attachments: 15410-wip.patch, 15410.branch-1.patch, 15410.v1.patch, > 15410.v2.patch, 15410.v3.patch, 15410.v4.patch, HBASE-15410.addendum.v1.patch > > > As Preston mentioned in the comment in HBASE-15243: > https://issues.apache.org/jira/browse/HBASE-15243?focusedCommentId=15143557&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15143557 > An optimization for filters returning a SEEK_NEXT_USING_HINT would be to seek > to the highest hint (Any previous/lower row won't be accepted by the filter > returning that seek). > This JIRA is to explore this potential optimization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19071) Import from Hbase version 0.94.27 to higher version 1.2.1 not working
[ https://issues.apache.org/jira/browse/HBASE-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjeet Singh updated HBASE-19071: -- Description: Data migration from one cluster to another cluster in same N/W, but with a different version of hbase one is 0.94.27 (source cluster hbase) and another is destination cluster hbase version is 1.2.1. is not working. I used below command to take backup of hbase table on source cluster is: ./hbase org.apache.hadoop.hbase.mapreduce.Export /data/backupData/ and as a result below files were genrated by using above command:- drwxr-xr-x 3 root root4096 Dec 9 2016 _logs -rw-r--r-- 1 root root 788227695 Dec 16 2016 part-m-0 -rw-r--r-- 1 root root 1098757026 Dec 16 2016 part-m-1 -rw-r--r-- 1 root root 906973626 Dec 16 2016 part-m-2 -rw-r--r-- 1 root root 1981769314 Dec 16 2016 part-m-3 -rw-r--r-- 1 root root 2099785782 Dec 16 2016 part-m-4 -rw-r--r-- 1 root root 4118835540 Dec 16 2016 part-m-5 -rw-r--r-- 1 root root 14217981341 Dec 16 2016 part-m-6 -rw-r--r-- 1 root root 0 Dec 16 2016 _SUCCESS I have copy above files to destination cluster by using scp command and put them into destination cluster HDFS (It's because of two different version of Haddop destination cluster hadoop is 1.2.1 and destination is having Hadoop 2.0 ) First I get HDFS files to local linux and use scp command to put them into destination cluster. I import these file in another Hbase version (1.2.1) in order to restore these files I am assuming I have to move these files in destination cluster and have to run below command hbase org.apache.hadoop.hbase.mapreduce.Import sudo -u hdfs hbase org.apache.hadoop.hbase.mapreduce.Import test_table hdfs://:8020/data/ExportedFiles I am getting below error 17/10/23 16:13:50 INFO mapreduce.Job: Task Id : attempt_1505781444745_0070_m_03_0, Status : FAILED Error: java.io.IOException: keyvalues=NONE read 2 bytes, should read 121347 at org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2306) at org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.nextKeyValue(SequenceFileRecordReader.java:78) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556) at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) was: Data migration from one cluster to another cluster in same N/W, but with a different version of hbase one is 0.94.27 (source cluster hbase) and another is destination cluster hbase version is 1.2.1. is not working. I used below command to take backup of hbase table on source cluster is: ./hbase org.apache.hadoop.hbase.mapreduce.Export /data/backupData/ and as a result below files were genrated by using above command:- drwxr-xr-x 3 root root4096 Dec 9 2016 _logs -rw-r--r-- 1 root root 788227695 Dec 16 2016 part-m-0 -rw-r--r-- 1 root root 1098757026 Dec 16 2016 part-m-1 -rw-r--r-- 1 root root 906973626 Dec 16 2016 part-m-2 -rw-r--r-- 1 root root 1981769314 Dec 16 2016 part-m-3 -rw-r--r-- 1 root root 2099785782 Dec 16 2016 part-m-4 -rw-r--r-- 1 root root 4118835540 Dec 16 2016 part-m-5 -rw-r--r-- 1 root root 14217981341 Dec 16 2016 part-m-6 -rw-r--r-- 1 root root 0 Dec 16 2016 _SUCCESS I have copy above files to destination cluster by using scp command and put them into destination cluster HDFS (It's because of two different version of Haddop destination cluster hadoop is 1.2.1 and destination is having Hadoop 2.0 ) First I get HDFS files to linux and use scp command to destination cluster and I put to destination cluster I import these file in another Hbase version (1.2.1) in order to restore these files I am assuming I have to move these files in destination cluster and have to run below command hbase org.apache.hadoop.hbase.mapreduce.Import sudo -u hdfs hbase org.apache.hadoop.hbase.mapreduce.Import test_table hdfs://:8020/data/ExportedFiles I am getting below error 17/10/23 16:13:50 INFO mapreduce.Job: Task Id : attempt_1505781444745_0070_m_03_0, Status : FAILED Error: java.io.IOException: keyvalue
[jira] [Assigned] (HBASE-19090) Add config 'hbase.systemtables.compacting.memstore.type' to docs
[ https://issues.apache.org/jira/browse/HBASE-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan reassigned HBASE-19090: -- Assignee: ramkrishna.s.vasudevan > Add config 'hbase.systemtables.compacting.memstore.type' to docs > > > Key: HBASE-19090 > URL: https://issues.apache.org/jira/browse/HBASE-19090 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19090.patch > > > As discussed in parent JIRA the new config > 'hbase.systemtables.compacting.memstore.type' has to be added to docs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19090) Add config 'hbase.systemtables.compacting.memstore.type' to docs
[ https://issues.apache.org/jira/browse/HBASE-19090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-19090: --- Attachment: HBASE-19090.patch > Add config 'hbase.systemtables.compacting.memstore.type' to docs > > > Key: HBASE-19090 > URL: https://issues.apache.org/jira/browse/HBASE-19090 > Project: HBase > Issue Type: Sub-task >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19090.patch > > > As discussed in parent JIRA the new config > 'hbase.systemtables.compacting.memstore.type' has to be added to docs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19071) Import from Hbase version 0.94.27 to higher version 1.2.1 not working
[ https://issues.apache.org/jira/browse/HBASE-19071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manjeet Singh updated HBASE-19071: -- Description: Data migration from one cluster to another cluster in same N/W, but with a different version of hbase one is 0.94.27 (source cluster hbase) and another is destination cluster hbase version is 1.2.1. is not working. I used below command to take backup of hbase table on source cluster is: ./hbase org.apache.hadoop.hbase.mapreduce.Export /data/backupData/ and as a result below files were genrated by using above command:- drwxr-xr-x 3 root root4096 Dec 9 2016 _logs -rw-r--r-- 1 root root 788227695 Dec 16 2016 part-m-0 -rw-r--r-- 1 root root 1098757026 Dec 16 2016 part-m-1 -rw-r--r-- 1 root root 906973626 Dec 16 2016 part-m-2 -rw-r--r-- 1 root root 1981769314 Dec 16 2016 part-m-3 -rw-r--r-- 1 root root 2099785782 Dec 16 2016 part-m-4 -rw-r--r-- 1 root root 4118835540 Dec 16 2016 part-m-5 -rw-r--r-- 1 root root 14217981341 Dec 16 2016 part-m-6 -rw-r--r-- 1 root root 0 Dec 16 2016 _SUCCESS I have copy above files to destination cluster by using scp command and put them into destination cluster HDFS (It's because of two different version of Haddop destination cluster hadoop is 1.2.1 and destination is having Hadoop 2.0 ) First I get HDFS files to linux and use scp command to destination cluster and I put to destination cluster I import these file in another Hbase version (1.2.1) in order to restore these files I am assuming I have to move these files in destination cluster and have to run below command hbase org.apache.hadoop.hbase.mapreduce.Import sudo -u hdfs hbase org.apache.hadoop.hbase.mapreduce.Import test_table hdfs://:8020/data/ExportedFiles I am getting below error 17/10/23 16:13:50 INFO mapreduce.Job: Task Id : attempt_1505781444745_0070_m_03_0, Status : FAILED Error: java.io.IOException: keyvalues=NONE read 2 bytes, should read 121347 at org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2306) at org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.nextKeyValue(SequenceFileRecordReader.java:78) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:556) at org.apache.hadoop.mapreduce.task.MapContextImpl.nextKeyValue(MapContextImpl.java:80) at org.apache.hadoop.mapreduce.lib.map.WrappedMapper$Context.nextKeyValue(WrappedMapper.java:91) at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144) at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787) at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341) at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:422) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693) at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158) was: Data migration from one cluster to another cluster in same N/W, but with a different version of hbase one is 0.94.27 (source cluster hbase) and another is destination cluster hbase version is 1.2.1. is not working. I have used below command to take backup of hbase table on source cluster is: ./hbase org.apache.hadoop.hbase.mapreduce.Export SPDBRebuild /data/backupData/ and as a result below files were genrated by using above command:- drwxr-xr-x 3 root root4096 Dec 9 2016 _logs -rw-r--r-- 1 root root 788227695 Dec 16 2016 part-m-0 -rw-r--r-- 1 root root 1098757026 Dec 16 2016 part-m-1 -rw-r--r-- 1 root root 906973626 Dec 16 2016 part-m-2 -rw-r--r-- 1 root root 1981769314 Dec 16 2016 part-m-3 -rw-r--r-- 1 root root 2099785782 Dec 16 2016 part-m-4 -rw-r--r-- 1 root root 4118835540 Dec 16 2016 part-m-5 -rw-r--r-- 1 root root 14217981341 Dec 16 2016 part-m-6 -rw-r--r-- 1 root root 0 Dec 16 2016 _SUCCESS I import these file in another Hbase version (1.2.1) in order to restore these files I am assuming I have to move these files in destination cluster and have to run below command hbase org.apache.hadoop.hbase.mapreduce.Import sudo -u hdfs hbase org.apache.hadoop.hbase.mapreduce.Import test_table hdfs://:8020/data/ExportedFiles I am getting below error 17/10/23 16:13:50 INFO mapreduce.Job: Task Id : attempt_1505781444745_0070_m_03_0, Status : FAILED Error: java.io.IOException: keyvalues=NONE read 2 bytes, should read 121347 at org.apache.hadoop.io.SequenceFile$Reader.getCurrentValue(SequenceFile.java:2306) at org.apache.hadoop.mapreduce.lib.input.SequenceFileRecordReader.nextKeyValue(SequenceFileRecordReader.java:78) at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader
[jira] [Commented] (HBASE-18870) Hbase Backup should set the details to MR jobs
[ https://issues.apache.org/jira/browse/HBASE-18870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219984#comment-16219984 ] Vishal Khandelwal commented on HBASE-18870: --- Thanks [~te...@apache.org] for review. Updated the patch after incorporating comments and removing the white space warning. > Hbase Backup should set the details to MR jobs > -- > > Key: HBASE-18870 > URL: https://issues.apache.org/jira/browse/HBASE-18870 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Minor > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-18870.branch-2.v1.patch, > HBASE-18870.branch-2.v2.patch > > > For Incremental and Full Backups job names should be set correctly. It would > make the debug and monitoring better. > Current : > incremental -> WALPlayer_1506332227619 > ExportSnapshot-snapshot_1506323760362_default_t1 > Proposed : > incremental -> Backup_Incremental__ > Full -> Backups_Full__ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18870) Hbase Backup should set the details to MR jobs
[ https://issues.apache.org/jira/browse/HBASE-18870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vishal Khandelwal updated HBASE-18870: -- Attachment: HBASE-18870.branch-2.v2.patch > Hbase Backup should set the details to MR jobs > -- > > Key: HBASE-18870 > URL: https://issues.apache.org/jira/browse/HBASE-18870 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0 >Reporter: Vishal Khandelwal >Assignee: Vishal Khandelwal >Priority: Minor > Labels: backup > Fix For: 2.0.0 > > Attachments: HBASE-18870.branch-2.v1.patch, > HBASE-18870.branch-2.v2.patch > > > For Incremental and Full Backups job names should be set correctly. It would > make the debug and monitoring better. > Current : > incremental -> WALPlayer_1506332227619 > ExportSnapshot-snapshot_1506323760362_default_t1 > Proposed : > incremental -> Backup_Incremental__ > Full -> Backups_Full__ -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18994) Decide if META/System tables should use Compacting Memstore or Default Memstore
[ https://issues.apache.org/jira/browse/HBASE-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219980#comment-16219980 ] ramkrishna.s.vasudevan commented on HBASE-18994: Oh . Just seeing your comment. Wil push the the hbase-default.xml change as part of sub-JIRA. > Decide if META/System tables should use Compacting Memstore or Default > Memstore > --- > > Key: HBASE-18994 > URL: https://issues.apache.org/jira/browse/HBASE-18994 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18994.patch, HBASE-18994_1.patch, > HBASE-18994_2.patch, HBASE-18994_3.patch > > > HBASE-18992 brought this topic. Currently META is also using Compacting > Memstore. We should decide if system tables can go with Default memstore only. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19090) Add config 'hbase.systemtables.compacting.memstore.type' to docs
ramkrishna.s.vasudevan created HBASE-19090: -- Summary: Add config 'hbase.systemtables.compacting.memstore.type' to docs Key: HBASE-19090 URL: https://issues.apache.org/jira/browse/HBASE-19090 Project: HBase Issue Type: Sub-task Reporter: ramkrishna.s.vasudevan Fix For: 2.0.0-alpha-4 As discussed in parent JIRA the new config 'hbase.systemtables.compacting.memstore.type' has to be added to docs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18994) Decide if META/System tables should use Compacting Memstore or Default Memstore
[ https://issues.apache.org/jira/browse/HBASE-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-18994: --- Resolution: Fixed Status: Resolved (was: Patch Available) Pushed to master and branch-2. Thanks for the reivews. Will raise a subJIRA to add to docs/hbase-default.xml. Yes we can start a dev@ mail to discuss on the creation of tables in system namespace. > Decide if META/System tables should use Compacting Memstore or Default > Memstore > --- > > Key: HBASE-18994 > URL: https://issues.apache.org/jira/browse/HBASE-18994 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18994.patch, HBASE-18994_1.patch, > HBASE-18994_2.patch, HBASE-18994_3.patch > > > HBASE-18992 brought this topic. Currently META is also using Compacting > Memstore. We should decide if system tables can go with Default memstore only. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18994) Decide if META/System tables should use Compacting Memstore or Default Memstore
[ https://issues.apache.org/jira/browse/HBASE-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219975#comment-16219975 ] stack edited comment on HBASE-18994 at 10/26/17 5:16 AM: - +1 on commit. Add to hbase-default.xml on commit. Add a nice description. This'll make it show up in docs (we import hbase-default.xml when we generate the refguide). was (Author: stack): +1 on commit. Add to hbase-default.xml on commit. Add a nice description. This'll make it show up in docs. > Decide if META/System tables should use Compacting Memstore or Default > Memstore > --- > > Key: HBASE-18994 > URL: https://issues.apache.org/jira/browse/HBASE-18994 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18994.patch, HBASE-18994_1.patch, > HBASE-18994_2.patch, HBASE-18994_3.patch > > > HBASE-18992 brought this topic. Currently META is also using Compacting > Memstore. We should decide if system tables can go with Default memstore only. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18994) Decide if META/System tables should use Compacting Memstore or Default Memstore
[ https://issues.apache.org/jira/browse/HBASE-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219975#comment-16219975 ] stack commented on HBASE-18994: --- +1 on commit. Add to hbase-default.xml on commit. Add a nice description. This'll make it show up in docs. > Decide if META/System tables should use Compacting Memstore or Default > Memstore > --- > > Key: HBASE-18994 > URL: https://issues.apache.org/jira/browse/HBASE-18994 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18994.patch, HBASE-18994_1.patch, > HBASE-18994_2.patch, HBASE-18994_3.patch > > > HBASE-18992 brought this topic. Currently META is also using Compacting > Memstore. We should decide if system tables can go with Default memstore only. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18994) Decide if META/System tables should use Compacting Memstore or Default Memstore
[
https://issues.apache.org/jira/browse/HBASE-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ramkrishna.s.vasudevan updated HBASE-18994:
---
Release Note:
Added a new config 'hbase.systemtables.compacting.memstore.type" for the
system tables. By default all the system tables will have 'NONE' as the type
and so it will be using the default memstore by default.
{code}
hbase.systemtables.compacting.memstore.type
NONE
{code}
was:Added a new config 'hbase.systemtables.compacting.memstore.type" for the
system tables. By default all the system tables will have 'NONE' as the type
and so it will be using the default memstore by default.
Description:
HBASE-18992 brought this topic. Currently META is also using Compacting
Memstore. We should decide if system tables can go with Default memstore only.
was:HBASE-18992 brought this topic. Currently META is also using Compacting
Memstore. We should decide if system tables can go with Default memstore only.
> Decide if META/System tables should use Compacting Memstore or Default
> Memstore
> ---
>
> Key: HBASE-18994
> URL: https://issues.apache.org/jira/browse/HBASE-18994
> Project: HBase
> Issue Type: Improvement
>Affects Versions: 2.0.0-alpha-3
>Reporter: ramkrishna.s.vasudevan
>Assignee: ramkrishna.s.vasudevan
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18994.patch, HBASE-18994_1.patch,
> HBASE-18994_2.patch, HBASE-18994_3.patch
>
>
> HBASE-18992 brought this topic. Currently META is also using Compacting
> Memstore. We should decide if system tables can go with Default memstore only.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18994) Decide if META/System tables should use Compacting Memstore or Default Memstore
[ https://issues.apache.org/jira/browse/HBASE-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-18994: --- Hadoop Flags: Reviewed Release Note: Added a new config 'hbase.systemtables.compacting.memstore.type" for the system tables. By default all the system tables will have 'NONE' as the type and so it will be using the default memstore by default. > Decide if META/System tables should use Compacting Memstore or Default > Memstore > --- > > Key: HBASE-18994 > URL: https://issues.apache.org/jira/browse/HBASE-18994 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18994.patch, HBASE-18994_1.patch, > HBASE-18994_2.patch, HBASE-18994_3.patch > > > HBASE-18992 brought this topic. Currently META is also using Compacting > Memstore. We should decide if system tables can go with Default memstore only. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18994) Decide if META/System tables should use Compacting Memstore or Default Memstore
[ https://issues.apache.org/jira/browse/HBASE-18994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219972#comment-16219972 ] ramkrishna.s.vasudevan commented on HBASE-18994: So for now I will add a release note and commit this? So we need a doc update I believe and is there a need to add to hbase-default.xml? Let me know I shall commit this then. Thanks all for the reviews. > Decide if META/System tables should use Compacting Memstore or Default > Memstore > --- > > Key: HBASE-18994 > URL: https://issues.apache.org/jira/browse/HBASE-18994 > Project: HBase > Issue Type: Improvement >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18994.patch, HBASE-18994_1.patch, > HBASE-18994_2.patch, HBASE-18994_3.patch > > > HBASE-18992 brought this topic. Currently META is also using Compacting > Memstore. We should decide if system tables can go with Default memstore only. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
[ https://issues.apache.org/jira/browse/HBASE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219969#comment-16219969 ] Sean Busbey commented on HBASE-19089: - please make sure any change works with the nightly test for source artifacts. > Fix the list of included moduleSets in src and binary tars > -- > > Key: HBASE-19089 > URL: https://issues.apache.org/jira/browse/HBASE-19089 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19089.master.001.patch > > > List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite > a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. > The core issue is duplication involved in list. Let me try to get > rid of it by using a shared list and including it using -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219960#comment-16219960 ] ramkrishna.s.vasudevan commented on HBASE-18995: For the CP expose thing I agree - like not all would be needed. But still if it is exposed to CP do you think it will be misused? Making Tag LimitedPRivate should be a seperate task and yes when we do that all the related Util APIs also should move. So do you say that for this JIRA itself we need split and first ensure the Tag is made LP and then move related classes to that and then create another PRivate Util class to move all other APIs to it? bq.The ones with offset, length and all, ya lets keep in private util If we are going to have for CPs then we can have all versions and types of matchingXXX in the CP version only IMHO. bq.. Any APIs which were not released in 1.x branches, we can remove from public CellUtil if needed. No deprecation cycle for them Yes this has been done. Infact the API that take byteRange and write to specific byte etc can actually directly removed since I don't think anyone will even use it but since we follow some version control mechanism I thought for 2.0 lets go this way of deprecate and then remove them though it may be unnecessary. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18925) Need updated mockito for using java optional
[ https://issues.apache.org/jira/browse/HBASE-18925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-18925: - Attachment: HBASE-18925.master.005.patch > Need updated mockito for using java optional > > > Key: HBASE-18925 > URL: https://issues.apache.org/jira/browse/HBASE-18925 > Project: HBase > Issue Type: Improvement >Reporter: Appy >Assignee: Appy > Fix For: 2.0.0-beta-1 > > Attachments: HBASE-18925.master.001.patch, > HBASE-18925.master.002.patch, HBASE-18925.master.002.patch, > HBASE-18925.master.003.patch, HBASE-18925.master.004.patch, > HBASE-18925.master.005.patch > > > Came up when i was trying to test HBASE-18878. > It kept failing because mock of RpcCall returned null where return type was > Optional. > Instead, we want it to return Optional.empty(). > New mockito versions support this (and other java8 things) - > https://github.com/mockito/mockito/wiki/What%27s-new-in-Mockito-2 > We use mockito-all which was last released in Dec2014. However, mockito-core > has had more than 50 releases after that > (https://mvnrepository.com/artifact/org.mockito/mockito-core). > We need to change our deps from mockito-all to mockito-core. > However that comes with fair breakages, so this is not a simple task of > changing pom files. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19047) CP exposed Scanner types should not extend Shipper
[ https://issues.apache.org/jira/browse/HBASE-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219916#comment-16219916 ] Anoop Sam John commented on HBASE-19047: post hook I believe people were using to create wrapper over the passed RegionScanner object. Ya as u said it may be possible to chnage Scan object and do what u want. Extra /special filter. But what abt some other op.. Like make a wrapper and so the next ops happening via that. And do some extra accounting or so in the wrapper.. Some counters or so?I tend to continue this support of post hook return a wrapper. At least for this jira lets keep this.. If post also not allow wrap, me must do the discuss in dev@. Am sure many using the Region scanner wrapping. > CP exposed Scanner types should not extend Shipper > -- > > Key: HBASE-19047 > URL: https://issues.apache.org/jira/browse/HBASE-19047 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19047.patch, HBASE-19047_V2.patch > > > Shipper is a IA.Private interface and very much internal.. > Right now CP exposed RegionScanner is extending this and so exposing the > shipped() method. This by mistake is called, can harm the correctness of the > cells in the Results. > preScannerOpen() allowing to return a new Scanner is also problematic now. > This can allow users to create a Region scanner from Region and then wrap it > and return back (Well same can be done by postScannerOpen also), it can so > happen that the wrapper is not implementing the shipped() properly. In any > way exposing the shipped () is problematic. > Solution Steps > 1. Remove preScannerOpen() , the use case I can think of is wrapping the > original scanner. The original scanner can be created by Region.getScanner > way only.. May be no need to remove this hook. Just remove the ability for > it to return a RegionScanner instance. Call this with the Scan object and > the CP can change the Scan object if they want. > 2. Let RegionScanner not extending Shipper but only RegionScannerImpl > implements this > 3. We have ref to the RegionScanner created by core and let that be used by > RegionScannerShippedCallBack when the post hook doing a wrap. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-17436) Add facility to provide more information for Other Regions seen on Master UI
[ https://issues.apache.org/jira/browse/HBASE-17436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219908#comment-16219908 ] Ted Yu commented on HBASE-17436: HttpServer.java has been moved to hbase-http module. Mind rebasing ? > Add facility to provide more information for Other Regions seen on Master UI > > > Key: HBASE-17436 > URL: https://issues.apache.org/jira/browse/HBASE-17436 > Project: HBase > Issue Type: Improvement >Reporter: Ted Yu >Assignee: Janos Gub > Labels: ui, usability > Fix For: 2.0.0 > > Attachments: HBASE-17436-v2.patch, HBASE-17436-v5.patch, > HBASE-17436-v6.patch, HBASE-17436-v6.patch, HBASE-17436-v7.patch, > HBASE-17436-v8.patch, HBASE-17436.patch, HBASE-17779-v3.patch, > HBASE-17779-v4.patch, Screen Shot 2017-04-24 at 8.43.31 PM.png, Screen Shot > 2017-04-26 at 4.39.49 PM.png, initial.patch, regionstates.png, > table-selected.png, tableregions.png > > > [~rpednekar] and I were looking at a case where the count displayed under > Other Regions was high (~1200). > Since the table page just maintains a count instead of List of region names, > it is very difficult for user to determine which regions belong to this > category. > We should provide facility to provide more details for this category (LOG, > JMX, etc). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18386) When the zookeeper host cannot be resolved, provide better error message
[
https://issues.apache.org/jira/browse/HBASE-18386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated HBASE-18386:
---
Labels: fault-tolerance (was: )
> When the zookeeper host cannot be resolved, provide better error message
>
>
> Key: HBASE-18386
> URL: https://issues.apache.org/jira/browse/HBASE-18386
> Project: HBase
> Issue Type: Sub-task
>Reporter: Ted Yu
>Priority: Minor
> Labels: fault-tolerance
>
> Currently, when the zookeeper host cannot be resolved, we would see:
> {code}
> 2017-07-15 19:42:09,367:3811(0x7f9ee2ffd700):ZOO_ERROR@getaddrs@613:
> getaddrinfo: No such file or directory
> *** Aborted at 1500147729 (unix time) try "date -d @1500147729" if you are
> using GNU date ***
> PC: @ 0x6f4c28 zoo_get
> *** SIGSEGV (@0x260) received by PID 3811 (TID 0x7f9ee2ffd700) from PID 608;
> stack trace: ***
> @ 0x7f9eee0923d0 (unknown)
> @ 0x6f4c28 zoo_get
> @ 0xb4 hbase::LocationCache::ReadMetaLocation()
> @ 0x449ec4 std::_Function_handler<>::_M_invoke()
> @ 0x55ec12 wangle::ThreadPoolExecutor::runTask()
> @ 0x54dfca wangle::CPUThreadPoolExecutor::threadRun()
> @ 0x55f892 std::_Function_handler<>::_M_invoke()
> {code}
> There should be more intuitive error message so that user knows what to do
> next.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19088) move_tables_rsgroup will throw an exception when the table is disabled
[
https://issues.apache.org/jira/browse/HBASE-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guangxu Cheng updated HBASE-19088:
--
Attachment: HBASE-19088.branch-1.001.patch
the patch looks good.Upload patch for branch-1.Thanks:)
> move_tables_rsgroup will throw an exception when the table is disabled
> --
>
> Key: HBASE-19088
> URL: https://issues.apache.org/jira/browse/HBASE-19088
> Project: HBase
> Issue Type: Bug
> Components: rsgroup
>Affects Versions: 3.0.0
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Attachments: HBASE-19088.branch-1.001.patch,
> HBASE-19088.master.001.patch, HBASE-19088.master.002.patch
>
>
> When the table is disabled, *move_tables_rsgroup* will throw an exception,
> but the table has been moved to the new group successfully.
> {code:java}
> hbase(main):009:0> move_tables_rsgroup 'default',['t1']
> ERROR: java.io.IOException
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:465)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:134)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:325)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:305)
> Caused by: java.lang.NullPointerException
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProcedureProtos$MoveRegionStateData$Builder.setSourceServer(MasterProcedureProtos.java:26022)
> at
> org.apache.hadoop.hbase.master.assignment.MoveRegionProcedure.serializeStateData(MoveRegionProcedure.java:133)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureUtil.convertToProtoProcedure(ProcedureUtil.java:198)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeEntry(ProcedureWALFormat.java:211)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.ProcedureWALFormat.writeInsert(ProcedureWALFormat.java:222)
> at
> org.apache.hadoop.hbase.procedure2.store.wal.WALProcedureStore.insert(WALProcedureStore.java:470)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:866)
> at
> org.apache.hadoop.hbase.procedure2.ProcedureExecutor.submitProcedure(ProcedureExecutor.java:835)
> at
> org.apache.hadoop.hbase.master.procedure.ProcedureSyncWait.submitAndWaitProcedure(ProcedureSyncWait.java:120)
> at
> org.apache.hadoop.hbase.master.assignment.AssignmentManager.move(AssignmentManager.java:557)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminServer.moveTables(RSGroupAdminServer.java:413)
> at
> org.apache.hadoop.hbase.rsgroup.RSGroupAdminEndpoint$RSGroupAdminServiceImpl.moveTables(RSGroupAdminEndpoint.java:190)
> at
> org.apache.hadoop.hbase.protobuf.generated.RSGroupAdminProtos$RSGroupAdminService.callMethod(RSGroupAdminProtos.java:12786)
> at
> org.apache.hadoop.hbase.master.MasterRpcServices.execMasterService(MasterRpcServices.java:786)
> at
> org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$MasterService$2.callBlockingMethod(MasterProtos.java)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:406)
> ... 3 more
> Reassign tables from one RegionServer group to another.
> Example:
> hbase> move_tables_rsgroup 'dest',['table1','table2']
> Took 0.0297 seconds
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-12941) CompactionRequestor - a private interface class with no users
[ https://issues.apache.org/jira/browse/HBASE-12941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Drob updated HBASE-12941: -- Resolution: Not A Problem Status: Resolved (was: Patch Available) This looks like it is used to me. If we want to do cleanup here (which we might) then let's get a more compelling justification. Closing as not a problem due to lack of activity. > CompactionRequestor - a private interface class with no users > - > > Key: HBASE-12941 > URL: https://issues.apache.org/jira/browse/HBASE-12941 > Project: HBase > Issue Type: Bug > Components: regionserver >Reporter: ryan rawson > Attachments: HBASE-12941-master.patch, HBASE-12941.patch > > > CompactionRequestor is a 'interface audience private' class with no users in > the HBase code base. Unused things should be deleted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19047) CP exposed Scanner types should not extend Shipper
[ https://issues.apache.org/jira/browse/HBASE-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219847#comment-16219847 ] Duo Zhang commented on HBASE-19047: --- I wonder whether we still need to give user the ability to extend RegionScanner. Let's just also remove the return value of postScannerOpen? I think the Scan object is powerful enough to do what you want, especially that you can use a Filter. Thanks. > CP exposed Scanner types should not extend Shipper > -- > > Key: HBASE-19047 > URL: https://issues.apache.org/jira/browse/HBASE-19047 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19047.patch, HBASE-19047_V2.patch > > > Shipper is a IA.Private interface and very much internal.. > Right now CP exposed RegionScanner is extending this and so exposing the > shipped() method. This by mistake is called, can harm the correctness of the > cells in the Results. > preScannerOpen() allowing to return a new Scanner is also problematic now. > This can allow users to create a Region scanner from Region and then wrap it > and return back (Well same can be done by postScannerOpen also), it can so > happen that the wrapper is not implementing the shipped() properly. In any > way exposing the shipped () is problematic. > Solution Steps > 1. Remove preScannerOpen() , the use case I can think of is wrapping the > original scanner. The original scanner can be created by Region.getScanner > way only.. May be no need to remove this hook. Just remove the ability for > it to return a RegionScanner instance. Call this with the Scan object and > the CP can change the Scan object if they want. > 2. Let RegionScanner not extending Shipper but only RegionScannerImpl > implements this > 3. We have ref to the RegionScanner created by core and let that be used by > RegionScannerShippedCallBack when the post hook doing a wrap. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19033) Allow CP users to change versions and TTL before opening StoreScanner
[ https://issues.apache.org/jira/browse/HBASE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219832#comment-16219832 ] Duo Zhang commented on HBASE-19033: --- There are RegionObserver changes so it has to be done before alpha4. Will give out the patch today. > Allow CP users to change versions and TTL before opening StoreScanner > - > > Key: HBASE-19033 > URL: https://issues.apache.org/jira/browse/HBASE-19033 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > > See the discussion in HBASE-19001. Changing versions and TTL is safe for > flush/compaction so we can expose them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
[ https://issues.apache.org/jira/browse/HBASE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219826#comment-16219826 ] Mike Drob commented on HBASE-19089: --- Not sure I understand what this patch is doing... I don't see common-moduleset-includes.xml referenced from any of the other files? > Fix the list of included moduleSets in src and binary tars > -- > > Key: HBASE-19089 > URL: https://issues.apache.org/jira/browse/HBASE-19089 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19089.master.001.patch > > > List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite > a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. > The core issue is duplication involved in list. Let me try to get > rid of it by using a shared list and including it using -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
[ https://issues.apache.org/jira/browse/HBASE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19089: - Attachment: HBASE-19089.master.001.patch > Fix the list of included moduleSets in src and binary tars > -- > > Key: HBASE-19089 > URL: https://issues.apache.org/jira/browse/HBASE-19089 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > Attachments: HBASE-19089.master.001.patch > > > List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite > a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. > The core issue is duplication involved in list. Let me try to get > rid of it by using a shared list and including it using -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18906) Provide Region#waitForFlushes API
[ https://issues.apache.org/jira/browse/HBASE-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219811#comment-16219811 ] Mike Drob commented on HBASE-18906: --- bq. No.. it wont throw InterruptedException. It is not an interruption case. I was trying to follow same way as that of Object.wait(timeout). Confused here... [Object.wait(long)|https://docs.oracle.com/javase/8/docs/api/java/lang/Object.html#wait-long-] definitely throws InterruptedEx... Not if timeout exceeded, but if the thread is interrupted it needs to stop waiting, not swallow the exception. Oh, are you saying that when we get a notify while waiting, we can't tell if it is an interrupt or if the action completed? That seems bad. > Provide Region#waitForFlushes API > - > > Key: HBASE-18906 > URL: https://issues.apache.org/jira/browse/HBASE-18906 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18906.patch > > > Expose an API for the CPs to wait for all on going flushes in a Region. The > API should support taking a time out. > Background > While reviewing HBASE-18183, Andy pointed out that Phoenix uses > waitForFlushesAndCompactions and/or waitForFlushes for diff reasons. This > issue is to see why they need them and whether alternate ways are possible. > This seems to be too much internal stuff and a normal CP hook calling these > would be dangerous. > If there are alternate ways for Phoenix not to use this and not landing in > issues (As said by Andy) we should suggest/fix for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19033) Allow CP users to change versions and TTL before opening StoreScanner
[ https://issues.apache.org/jira/browse/HBASE-19033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219805#comment-16219805 ] Mike Drob commented on HBASE-19033: --- StoreScanner is IA.Private, do the changes happen there or on CPs? If this is changes on StoreScanner itself then can do that later (like b1) > Allow CP users to change versions and TTL before opening StoreScanner > - > > Key: HBASE-19033 > URL: https://issues.apache.org/jira/browse/HBASE-19033 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Duo Zhang >Priority: Blocker > Fix For: 2.0.0-alpha-4 > > > See the discussion in HBASE-19001. Changing versions and TTL is safe for > flush/compaction so we can expose them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18624) Added support for clearing BlockCache based on table name
[ https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18624: -- Status: In Progress (was: Patch Available) > Added support for clearing BlockCache based on table name > - > > Key: HBASE-18624 > URL: https://issues.apache.org/jira/browse/HBASE-18624 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.0, 2.0.0 >Reporter: Ajay Jadhav >Assignee: Zach York > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18624.branch-1.001.patch, > HBASE-18624.master.001.patch, HBASE-18624.master.002.patch, > HBASE-18624.master.003.patch, HBASE-18624.master.004.patch, > HBASE-18624.master.005.patch > > > Bulk loading the primary HBase cluster triggers a lot of compactions > resulting in archival/ creation > of multiple HFiles. This process will cause a lot of items to become stale in > replica’s BlockCache. > This patch will help users to clear the block cache for a given table by > either using shell or API. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18624) Added support for clearing BlockCache based on table name
[ https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18624: -- Status: Patch Available (was: In Progress) > Added support for clearing BlockCache based on table name > - > > Key: HBASE-18624 > URL: https://issues.apache.org/jira/browse/HBASE-18624 > Project: HBase > Issue Type: Sub-task >Affects Versions: 1.3.0, 2.0.0 >Reporter: Ajay Jadhav >Assignee: Zach York > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18624.branch-1.001.patch, > HBASE-18624.master.001.patch, HBASE-18624.master.002.patch, > HBASE-18624.master.003.patch, HBASE-18624.master.004.patch, > HBASE-18624.master.005.patch > > > Bulk loading the primary HBase cluster triggers a lot of compactions > resulting in archival/ creation > of multiple HFiles. This process will cause a lot of items to become stale in > replica’s BlockCache. > This patch will help users to clear the block cache for a given table by > either using shell or API. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19002) Introduce more examples to show how to intercept normal region operations
[
https://issues.apache.org/jira/browse/HBASE-19002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219733#comment-16219733
]
Ted Yu commented on HBASE-19002:
Josh - you're so fast :-)
{code}
+public void postFlush(
+ObserverContext c, Store store,
StoreFile resultFile,
+FlushLifeCycleTracker tracker) throws IOException {
+ flushCounter.increment();
{code}
This hook has store passed to it (compared to postFlush() above this). I wonder
if a different counter than flushCounter can be used to track the occurrences.
{code}
+public class ValueRewritingObserver implements RegionObserver,
RegionCoprocessor {
{code}
Please add description on what the observer does.
{code}
+// TODO: this is incontrived, other types will happen in real life
+cellBuilder.setType(DataType.Put);
{code}
How about using the same type as that of the cell ?
Please clear the cellBuilder after build() is called.
Thanks
> Introduce more examples to show how to intercept normal region operations
> -
>
> Key: HBASE-19002
> URL: https://issues.apache.org/jira/browse/HBASE-19002
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Duo Zhang
>Assignee: Josh Elser
>Priority: Minor
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-19002.001.branch-2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-19024) provide a configurable option to hsync WAL edits to the disk for better durability
[
https://issues.apache.org/jira/browse/HBASE-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219728#comment-16219728
]
Andrew Purtell edited comment on HBASE-19024 at 10/25/17 11:30 PM:
---
Mostly lgtm, one minor nit:
{code}
diff --git a/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
index a272fc8..c1a50bf 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
@@ -1327,6 +1327,10 @@ public final class HConstants {
public static final String NOT_IMPLEMENTED = "Not implemented";
+ public static final String USE_HSYNC_CONF_KEY = "hbase.wal.use.hsync";
+
+ public static final boolean DEFAULT_USE_HSYNC = false;
+
private HConstants() {
// Can't be instantiated with this ctor.
}
{code}
It's better to include the name of the thing that changes behavior when this is
specified, like maybe:
{code}
+ public static final String WAL_HSYNC_CONF_KEY = "hbase.wal.hsync";
+ public static final boolean DEFAULT_WAL_HSYNC = false;
{code}
was (Author: apurtell):
Mostly lgtm, one minor nit:
{code}
diff --git a/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
index a272fc8..c1a50bf 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
@@ -1327,6 +1327,10 @@ public final class HConstants {
public static final String NOT_IMPLEMENTED = "Not implemented";
+ public static final String USE_HSYNC_CONF_KEY = "hbase.wal.use.hsync";
+
+ public static final boolean DEFAULT_USE_HSYNC = false;
+
private HConstants() {
// Can't be instantiated with this ctor.
}
{code}
Like maybe:
{code}
+ public static final String WAL_HSYNC_CONF_KEY = "hbase.wal.hsync";
+ public static final boolean DEFAULT_WAL_HSYNC = false;
{code}
> provide a configurable option to hsync WAL edits to the disk for better
> durability
> --
>
> Key: HBASE-19024
> URL: https://issues.apache.org/jira/browse/HBASE-19024
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Environment:
>Reporter: Vikas Vishwakarma
>Assignee: Harshal Jain
> Attachments: master.patch
>
>
> At present we do not have an option to hsync WAL edits to the disk for better
> durability. In our local tests we see 10-15% latency impact of using hsync
> instead of hflush which is not very high.
> We should have a configurable option to hysnc WAL edits instead of just
> sync/hflush which will call the corresponding API on the hadoop side.
> Currently HBase handles both SYNC_WAL and FSYNC_WAL as the same calling
> FSDataOutputStream sync/hflush on the hadoop side. This can be modified to
> let FSYNC_WAL call hsync on the hadoop side instead of sync/hflush. We can
> keep the default value to sync as the current behavior and hsync can be
> enabled based on explicit configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19024) provide a configurable option to hsync WAL edits to the disk for better durability
[
https://issues.apache.org/jira/browse/HBASE-19024?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219728#comment-16219728
]
Andrew Purtell commented on HBASE-19024:
Mostly lgtm, one minor nit:
{code}
diff --git a/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
b/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
index a272fc8..c1a50bf 100644
--- a/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
+++ b/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
@@ -1327,6 +1327,10 @@ public final class HConstants {
public static final String NOT_IMPLEMENTED = "Not implemented";
+ public static final String USE_HSYNC_CONF_KEY = "hbase.wal.use.hsync";
+
+ public static final boolean DEFAULT_USE_HSYNC = false;
+
private HConstants() {
// Can't be instantiated with this ctor.
}
{code}
Like maybe:
{code}
+ public static final String WAL_HSYNC_CONF_KEY = "hbase.wal.hsync";
+ public static final boolean DEFAULT_WAL_HSYNC = false;
{code}
> provide a configurable option to hsync WAL edits to the disk for better
> durability
> --
>
> Key: HBASE-19024
> URL: https://issues.apache.org/jira/browse/HBASE-19024
> Project: HBase
> Issue Type: Improvement
> Components: wal
> Environment:
>Reporter: Vikas Vishwakarma
>Assignee: Harshal Jain
> Attachments: master.patch
>
>
> At present we do not have an option to hsync WAL edits to the disk for better
> durability. In our local tests we see 10-15% latency impact of using hsync
> instead of hflush which is not very high.
> We should have a configurable option to hysnc WAL edits instead of just
> sync/hflush which will call the corresponding API on the hadoop side.
> Currently HBase handles both SYNC_WAL and FSYNC_WAL as the same calling
> FSDataOutputStream sync/hflush on the hadoop side. This can be modified to
> let FSYNC_WAL call hsync on the hadoop side instead of sync/hflush. We can
> keep the default value to sync as the current behavior and hsync can be
> enabled based on explicit configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19002) Introduce more examples to show how to intercept normal region operations
[ https://issues.apache.org/jira/browse/HBASE-19002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19002: --- Status: Patch Available (was: Open) > Introduce more examples to show how to intercept normal region operations > - > > Key: HBASE-19002 > URL: https://issues.apache.org/jira/browse/HBASE-19002 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Josh Elser >Priority: Minor > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19002.001.branch-2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19002) Introduce more examples to show how to intercept normal region operations
[ https://issues.apache.org/jira/browse/HBASE-19002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser updated HBASE-19002: --- Attachment: HBASE-19002.001.branch-2.patch .001 How's this? > Introduce more examples to show how to intercept normal region operations > - > > Key: HBASE-19002 > URL: https://issues.apache.org/jira/browse/HBASE-19002 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Ted Yu >Priority: Minor > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19002.001.branch-2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HBASE-19002) Introduce more examples to show how to intercept normal region operations
[ https://issues.apache.org/jira/browse/HBASE-19002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Elser reassigned HBASE-19002: -- Assignee: Josh Elser (was: Ted Yu) > Introduce more examples to show how to intercept normal region operations > - > > Key: HBASE-19002 > URL: https://issues.apache.org/jira/browse/HBASE-19002 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Josh Elser >Priority: Minor > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19002.001.branch-2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18624) Added support for clearing BlockCache based on table name
[ https://issues.apache.org/jira/browse/HBASE-18624?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York updated HBASE-18624: -- Attachment: HBASE-18624.master.005.patch > Added support for clearing BlockCache based on table name > - > > Key: HBASE-18624 > URL: https://issues.apache.org/jira/browse/HBASE-18624 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0, 1.3.0 >Reporter: Ajay Jadhav >Assignee: Zach York > Fix For: 2.0.0, 1.4.0 > > Attachments: HBASE-18624.branch-1.001.patch, > HBASE-18624.master.001.patch, HBASE-18624.master.002.patch, > HBASE-18624.master.003.patch, HBASE-18624.master.004.patch, > HBASE-18624.master.005.patch > > > Bulk loading the primary HBase cluster triggers a lot of compactions > resulting in archival/ creation > of multiple HFiles. This process will cause a lot of items to become stale in > replica’s BlockCache. > This patch will help users to clear the block cache for a given table by > either using shell or API. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19087) Remove redundant characters from the audit log
[
https://issues.apache.org/jira/browse/HBASE-19087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219550#comment-16219550
]
Hadoop QA commented on HBASE-19087:
---
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
20s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 7m
4s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
44s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
54m 26s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
30s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} unit {color} | {color:red}115m 8s{color}
| {color:red} hbase-server in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
19s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}192m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19087 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12893983/HBASE-19087.master.001.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux e39e5d186a05 3.13.0-119-generic #166-Ubuntu SMP Wed May 3
12:18:55 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 28d81295f3 |
| Default Java | 1.8.0_141 |
| unit |
https://builds.apache.org/job/PreCommit-HBASE-Build/9405/artifact/patchprocess/patch-unit-hbase-server.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9405/testReport/ |
| modules | C: hbase-server U: hbase-server |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9405/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> Remove redundant characters from the audit log
> --
>
>
[jira] [Comment Edited] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[
https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219413#comment-16219413
]
stack edited comment on HBASE-18770 at 10/25/17 9:22 PM:
-
After Andrew comment up on RB made me think, and indeed we can do bypass simply
and the same pattern for all methods; i.e. return true if you want to bypass,
false otherwise.
List of methods that return bypass or not from (RegionObserver):
{code}
default boolean preGetOp(ObserverContext c,
Get get,
List bypassResult) throws IOException
default boolean preExists(ObserverContext c,
Get get,
MutableBoolean bypassResult) throws IOException
default boolean prePut(ObserverContext c, Put
put, WALEdit edit,
Durability durability) throws IOException
default boolean preDelete(ObserverContext c,
Delete delete,
WALEdit edit, Durability durability) throws IOException
@Deprecated
default boolean
prePrepareTimeStampForDeleteVersion(ObserverContext
c,
Mutation mutation, Cell cell, byte[] byteNow, Get get) throws IOException
default boolean
preCheckAndPutAfterRowLock(ObserverContext c,
byte[] row, byte[] family, byte[] qualifier, CompareOperator op,
ByteArrayComparable comparator, Put put, MutableBoolean bypassResult)
throws IOException
default void preCheckAndDelete(ObserverContext
c,
byte[] row, byte[] family, byte[] qualifier, CompareOperator op,
ByteArrayComparable comparator, Delete delete, boolean result) throws
IOException
default boolean
preAppendAfterRowLock(ObserverContext c,
Append append, Result bypassResult) throws IOException
default boolean
preIncrementAfterRowLock(ObserverContext c,
Increment increment, Result bypassResult) throws IOException
{code}
... and I was going to try and fix...
default boolean preScannerNext(ObserverContext
c, InternalScanner s,
List result, int limit, boolean hasNext) throws IOException
default boolean
postScannerFilterRow(ObserverContext c,
InternalScanner s, Cell curRowCell, boolean hasMore) throws IOException
These seem to be doing bypass but doesn't say so in javadoc, only in
client-code when it calls.
was (Author: stack):
After Andrew comment up on RB made me think, and indeed we can do bypass simply
and the same pattern for all methods; i.e. return true if you want to bypass,
false otherwise.
List of methods that return bypass or not from (RegionObserver):
{code}
default boolean preGetOp(ObserverContext c,
Get get,
List bypassResult) throws IOException
default boolean preExists(ObserverContext c,
Get get,
MutableBoolean bypassResult) throws IOException
default boolean prePut(ObserverContext c, Put
put, WALEdit edit,
Durability durability) throws IOException
default boolean preDelete(ObserverContext c,
Delete delete,
WALEdit edit, Durability durability) throws IOException
@Deprecated
default boolean
prePrepareTimeStampForDeleteVersion(ObserverContext
c,
Mutation mutation, Cell cell, byte[] byteNow, Get get) throws IOException
default boolean
preCheckAndPutAfterRowLock(ObserverContext c,
byte[] row, byte[] family, byte[] qualifier, CompareOperator op,
ByteArrayComparable comparator, Put put, MutableBoolean bypassResult)
throws IOException
default Optional
preCheckAndDeleteAfterRowLock(ObserverContext c,
byte[] row, byte[] family, byte[] qualifier, CompareOperator op,
ByteArrayComparable comparator, Delete delete, MutableBoolean
bypassResult) throws IOException
default boolean
preAppendAfterRowLock(ObserverContext c,
Append append, Result bypassResult) throws IOException
default boolean
preIncrementAfterRowLock(ObserverContext c,
Increment increment, Result bypassResult) throws IOException
{code}
... and I was going to try and fix...
default boolean preScannerNext(ObserverContext
c, InternalScanner s,
List result, int limit, boolean hasNext) throws IOException
default boolean
postScannerFilterRow(ObserverContext c,
InternalScanner s, Cell curRowCell, boolean hasMore) throws IOException
These seem to be doing bypass but doesn't say so in javadoc, only in
client-code when it calls.
> Remove bypass method in ObserverContext and implement the 'bypass' logic case
> by case
> -
>
> Key: HBASE-18770
> URL: https://issues.apache.org/jira/browse/HBASE-18770
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18770.master.001.patch
>
>
> http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1
--
This message was s
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219543#comment-16219543 ] stack commented on HBASE-18770: --- You are right [~mdrob] Edited my comment. Thanks. > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
[ https://issues.apache.org/jira/browse/HBASE-19089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Appy updated HBASE-19089: - Description: List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. The core issue is duplication involved in list. Let me try to get rid of it by using a shared list and including it using was: List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. The core issue is duplication of list. Let me try to get rid of it by using shaded list of includes. > Fix the list of included moduleSets in src and binary tars > -- > > Key: HBASE-19089 > URL: https://issues.apache.org/jira/browse/HBASE-19089 > Project: HBase > Issue Type: Bug >Reporter: Appy >Assignee: Appy > > List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite > a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. > The core issue is duplication involved in list. Let me try to get > rid of it by using a shared list and including it using -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (HBASE-19089) Fix the list of included moduleSets in src and binary tars
Appy created HBASE-19089: Summary: Fix the list of included moduleSets in src and binary tars Key: HBASE-19089 URL: https://issues.apache.org/jira/browse/HBASE-19089 Project: HBase Issue Type: Bug Reporter: Appy Assignee: Appy List of moduleSets included in src.xml and hadoop-two-compat.xml differ quite a lot. Particularly, hadoop-two-compat.xml is missing quite a few modules. The core issue is duplication of list. Let me try to get rid of it by using shaded list of includes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[
https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219496#comment-16219496
]
Mike Drob commented on HBASE-18770:
---
{quote}
default Optional
preCheckAndDeleteAfterRowLock(ObserverContext c,
{quote}
I think you missed one on cleanup.
> Remove bypass method in ObserverContext and implement the 'bypass' logic case
> by case
> -
>
> Key: HBASE-18770
> URL: https://issues.apache.org/jira/browse/HBASE-18770
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Duo Zhang
>Assignee: stack
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18770.master.001.patch
>
>
> http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[ https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-18770: -- Priority: Critical (was: Major) Release Note: Removes blanket bypass mechanism (Observer#bypass). Instead, a curated ubset of methods are bypassable. Removes Observer#complete, a means of having one CPs answer overrule any others if a chain of Coprocessors. > Remove bypass method in ObserverContext and implement the 'bypass' logic case > by case > - > > Key: HBASE-18770 > URL: https://issues.apache.org/jira/browse/HBASE-18770 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: stack >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18770.master.001.patch > > > http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19030) nightly runs should attempt to log test results after archiving
[ https://issues.apache.org/jira/browse/HBASE-19030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219464#comment-16219464 ] Hudson commented on HBASE-19030: FAILURE: Integrated in Jenkins build HBase-1.5 #116 (See [https://builds.apache.org/job/HBase-1.5/116/]) HBASE-19030 nightly runs should attempt to log test results after (busbey: rev 19f74284c1db07edfb489a1f1d3e2931a3612b61) * (edit) dev-support/Jenkinsfile > nightly runs should attempt to log test results after archiving > --- > > Key: HBASE-19030 > URL: https://issues.apache.org/jira/browse/HBASE-19030 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13, 2.0.0-alpha-4 > > Attachments: HBASE-19030.0.patch > > > right now on the nightly tests the first post-action we do is log junit > results. due to current limitations of Jenkins DSL, if this step fails none > of the other post actions will happen. > Since we might not make junit test results, e.g. in the case of a timeout of > yetus itself, we should log the junit results after we've saved whatever we > can of yetus output. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Penzes updated HBASE-13346:
-
Status: Open (was: Patch Available)
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch, HBASE-13346.master.004.patch,
> HBASE-13346.master.005.patch, HBASE-13346.master.006.patch,
> HBASE-13346.master.007.patch, HBASE-13346.master.008.patch,
> HBASE-13346.master.009.patch, HBASE-13346.master.010.patch,
> HBASE-13346.master.011.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Penzes updated HBASE-13346:
-
Status: Patch Available (was: Open)
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch, HBASE-13346.master.004.patch,
> HBASE-13346.master.005.patch, HBASE-13346.master.006.patch,
> HBASE-13346.master.007.patch, HBASE-13346.master.008.patch,
> HBASE-13346.master.009.patch, HBASE-13346.master.010.patch,
> HBASE-13346.master.011.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-15753) Site does not build with the instructions in the book
[
https://issues.apache.org/jira/browse/HBASE-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219460#comment-16219460
]
Misty Stanley-Jones commented on HBASE-15753:
-
Yeah, you still have to do
{code:java}
mvn install
{code}
before you can build the docs for the first time. I thought we fixed this.
Anyway after I do
{code:java}
mvn install
{code}
I am able to build the docs with just
{code:java}
mvn site -DskipTests
{code}
and no errors.
> Site does not build with the instructions in the book
> -
>
> Key: HBASE-15753
> URL: https://issues.apache.org/jira/browse/HBASE-15753
> Project: HBase
> Issue Type: Bug
> Components: community, documentation, website
>Reporter: Enis Soztutar
>Assignee: Misty Stanley-Jones
>
> Originally reported by [~clarax98007] in HBASE-15337.
> Instructions in the book say to run:
> {code}
> mvn site -DskipTests
> {code}
> But it fails with javadoc related errors.
> Seems that we are using this in the jenkins job that [~misty] had setup
> (https://builds.apache.org/job/hbase_generate_website/):
> {code}mvn site -DskipTests -Dmaven.javadoc.skip=true -Dcheckstyle.skip=true
> -Dfindbugs.skip=true {code}
> We should either fix the javadoc, or update the instructions.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219458#comment-16219458
]
Tamas Penzes commented on HBASE-13346:
--
hi [~openinx],
fixed. Also other places where I've added IOException without a real cause.
Thanks for pointing it out.
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch, HBASE-13346.master.004.patch,
> HBASE-13346.master.005.patch, HBASE-13346.master.006.patch,
> HBASE-13346.master.007.patch, HBASE-13346.master.008.patch,
> HBASE-13346.master.009.patch, HBASE-13346.master.010.patch,
> HBASE-13346.master.011.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Penzes updated HBASE-13346:
-
Status: Patch Available (was: Open)
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch, HBASE-13346.master.004.patch,
> HBASE-13346.master.005.patch, HBASE-13346.master.006.patch,
> HBASE-13346.master.007.patch, HBASE-13346.master.008.patch,
> HBASE-13346.master.009.patch, HBASE-13346.master.010.patch,
> HBASE-13346.master.011.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Penzes updated HBASE-13346:
-
Attachment: HBASE-13346.master.011.patch
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch, HBASE-13346.master.004.patch,
> HBASE-13346.master.005.patch, HBASE-13346.master.006.patch,
> HBASE-13346.master.007.patch, HBASE-13346.master.008.patch,
> HBASE-13346.master.009.patch, HBASE-13346.master.010.patch,
> HBASE-13346.master.011.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-13346) Clean up Filter package for post 1.0 s/KeyValue/Cell/g
[
https://issues.apache.org/jira/browse/HBASE-13346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tamas Penzes updated HBASE-13346:
-
Status: Open (was: Patch Available)
> Clean up Filter package for post 1.0 s/KeyValue/Cell/g
> --
>
> Key: HBASE-13346
> URL: https://issues.apache.org/jira/browse/HBASE-13346
> Project: HBase
> Issue Type: Bug
> Components: API, Filters
>Affects Versions: 2.0.0
>Reporter: Lars George
>Assignee: Tamas Penzes
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-13346.master.001.patch,
> HBASE-13346.master.002.patch, HBASE-13346.master.003.patch,
> HBASE-13346.master.003.patch, HBASE-13346.master.004.patch,
> HBASE-13346.master.005.patch, HBASE-13346.master.006.patch,
> HBASE-13346.master.007.patch, HBASE-13346.master.008.patch,
> HBASE-13346.master.009.patch, HBASE-13346.master.010.patch,
> HBASE-13346.master.011.patch
>
>
> Since we have a bit of a messy Filter API with KeyValue vs Cell reference
> mixed up all over the place, I recommend cleaning this up once and for all.
> There should be no {{KeyValue}} (or {{kv}}, {{kvs}} etc.) in any method or
> parameter name.
> This includes deprecating and renaming filters too, for example
> {{FirstKeyOnlyFilter}}, which really should be named {{FirstKeyValueFilter}}
> as it does _not_ just return the key, but the entire cell. It should be
> deprecated and renamed to {{FirstCellFilter}} (or {{FirstColumnFilter}} if
> you prefer).
> In general we should clarify and settle on {{KeyValue}} vs {{Cell}} vs
> {{Column}} in our naming. The latter two are the only ones going forward with
> the public API, and are used synonymous. We should carefully check which is
> better suited (is it really a specific cell, or the newest cell, aka the
> newest column value) and settle on a naming schema.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19002) Introduce more examples to show how to intercept normal region operations
[ https://issues.apache.org/jira/browse/HBASE-19002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219457#comment-16219457 ] Josh Elser commented on HBASE-19002: I think the other reviews are under control. Let me get some examples put together too. > Introduce more examples to show how to intercept normal region operations > - > > Key: HBASE-19002 > URL: https://issues.apache.org/jira/browse/HBASE-19002 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Duo Zhang >Assignee: Ted Yu >Priority: Minor > Fix For: 2.0.0-alpha-4 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18770) Remove bypass method in ObserverContext and implement the 'bypass' logic case by case
[
https://issues.apache.org/jira/browse/HBASE-18770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219413#comment-16219413
]
stack commented on HBASE-18770:
---
After Andrew comment up on RB made me think, and indeed we can do bypass simply
and the same pattern for all methods; i.e. return true if you want to bypass,
false otherwise.
List of methods that return bypass or not from (RegionObserver):
{code}
default boolean preGetOp(ObserverContext c,
Get get,
List bypassResult) throws IOException
default boolean preExists(ObserverContext c,
Get get,
MutableBoolean bypassResult) throws IOException
default boolean prePut(ObserverContext c, Put
put, WALEdit edit,
Durability durability) throws IOException
default boolean preDelete(ObserverContext c,
Delete delete,
WALEdit edit, Durability durability) throws IOException
@Deprecated
default boolean
prePrepareTimeStampForDeleteVersion(ObserverContext
c,
Mutation mutation, Cell cell, byte[] byteNow, Get get) throws IOException
default boolean
preCheckAndPutAfterRowLock(ObserverContext c,
byte[] row, byte[] family, byte[] qualifier, CompareOperator op,
ByteArrayComparable comparator, Put put, MutableBoolean bypassResult)
throws IOException
default Optional
preCheckAndDeleteAfterRowLock(ObserverContext c,
byte[] row, byte[] family, byte[] qualifier, CompareOperator op,
ByteArrayComparable comparator, Delete delete, MutableBoolean
bypassResult) throws IOException
default boolean
preAppendAfterRowLock(ObserverContext c,
Append append, Result bypassResult) throws IOException
default boolean
preIncrementAfterRowLock(ObserverContext c,
Increment increment, Result bypassResult) throws IOException
{code}
... and I was going to try and fix...
default boolean preScannerNext(ObserverContext
c, InternalScanner s,
List result, int limit, boolean hasNext) throws IOException
default boolean
postScannerFilterRow(ObserverContext c,
InternalScanner s, Cell curRowCell, boolean hasMore) throws IOException
These seem to be doing bypass but doesn't say so in javadoc, only in
client-code when it calls.
> Remove bypass method in ObserverContext and implement the 'bypass' logic case
> by case
> -
>
> Key: HBASE-18770
> URL: https://issues.apache.org/jira/browse/HBASE-18770
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Duo Zhang
>Assignee: stack
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18770.master.001.patch
>
>
> http://search-hadoop.com/m/HBase/YGbbXd0RDCIHSC1
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19048) Cleanup MasterObserver hooks which takes IA private params
[
https://issues.apache.org/jira/browse/HBASE-19048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219412#comment-16219412
]
Chia-Ping Tsai commented on HBASE-19048:
+1. nit: There are many unused imports. Please remove them before commit.
> Cleanup MasterObserver hooks which takes IA private params
> --
>
> Key: HBASE-19048
> URL: https://issues.apache.org/jira/browse/HBASE-19048
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Anoop Sam John
>Assignee: stack
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-19048.master.001.patch,
> HBASE-19048.master.002.patch, HBASE-19048.master.003.patch,
> HBASE-19048.master.003.patch
>
>
> These are the ones in MasterObserver
> {code}
> preAbortProcedure- ProcedureExecutor
> postGetProcedures- Procedure
> postGetLocks - LockedResource
> preRequestLock - LockType
> postRequestLock - LockType
> preLockHeartbeat - LockProcedure
> postLockHeartbeat- LockProcedure
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18602) rsgroup cleanup unassign code
[
https://issues.apache.org/jira/browse/HBASE-18602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219360#comment-16219360
]
Chia-Ping Tsai commented on HBASE-18602:
Could you attach the patch for branch-1 and branch-1.4?
> rsgroup cleanup unassign code
> -
>
> Key: HBASE-18602
> URL: https://issues.apache.org/jira/browse/HBASE-18602
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Wang, Xinglong
>Assignee: Wang, Xinglong
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18602-master-v1.patch,
> HBASE-18602-master-v2.patch, HBASE-18602-master-v3.patch,
> HBASE-18602-master-v3.patch, HBASE-18602-master-v3.patch,
> HBASE-18602-master-v4.patch
>
>
> While walking through rsgroup code, I found that variable misplacedRegions
> has never been added any element into. This makes the unassign region code is
> not functional. And according to my test, it is actually unnecessary to do
> that.
> RSGroupBasedLoadBalancer.java
> {code:java}
> private Map> correctAssignments(
>Map> existingAssignments)
> throws HBaseIOException{
> Map> correctAssignments = new TreeMap<>();
> List misplacedRegions = new LinkedList<>();
> correctAssignments.put(LoadBalancer.BOGUS_SERVER_NAME, new
> LinkedList<>());
> for (Map.Entry> assignments :
> existingAssignments.entrySet()){
> ServerName sName = assignments.getKey();
> correctAssignments.put(sName, new LinkedList<>());
> List regions = assignments.getValue();
> for (HRegionInfo region : regions) {
> RSGroupInfo info = null;
> try {
> info = rsGroupInfoManager.getRSGroup(
> rsGroupInfoManager.getRSGroupOfTable(region.getTable()));
> } catch (IOException exp) {
> LOG.debug("RSGroup information null for region of table " +
> region.getTable(),
> exp);
> }
> if ((info == null) || (!info.containsServer(sName.getAddress( {
> correctAssignments.get(LoadBalancer.BOGUS_SERVER_NAME).add(region);
> } else {
> correctAssignments.get(sName).add(region);
> }
> }
> }
> //TODO bulk unassign?
> //unassign misplaced regions, so that they are assigned to correct groups.
> for(HRegionInfo info: misplacedRegions) {
> try {
> this.masterServices.getAssignmentManager().unassign(info);
> } catch (IOException e) {
> throw new HBaseIOException(e);
> }
> }
> return correctAssignments;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-19047) CP exposed Scanner types should not extend Shipper
[ https://issues.apache.org/jira/browse/HBASE-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anoop Sam John updated HBASE-19047: --- Attachment: HBASE-19047_V2.patch Added comments around the Shipper type casting. > CP exposed Scanner types should not extend Shipper > -- > > Key: HBASE-19047 > URL: https://issues.apache.org/jira/browse/HBASE-19047 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19047.patch, HBASE-19047_V2.patch > > > Shipper is a IA.Private interface and very much internal.. > Right now CP exposed RegionScanner is extending this and so exposing the > shipped() method. This by mistake is called, can harm the correctness of the > cells in the Results. > preScannerOpen() allowing to return a new Scanner is also problematic now. > This can allow users to create a Region scanner from Region and then wrap it > and return back (Well same can be done by postScannerOpen also), it can so > happen that the wrapper is not implementing the shipped() properly. In any > way exposing the shipped () is problematic. > Solution Steps > 1. Remove preScannerOpen() , the use case I can think of is wrapping the > original scanner. The original scanner can be created by Region.getScanner > way only.. May be no need to remove this hook. Just remove the ability for > it to return a RegionScanner instance. Call this with the Scan object and > the CP can change the Scan object if they want. > 2. Let RegionScanner not extending Shipper but only RegionScannerImpl > implements this > 3. We have ref to the RegionScanner created by core and let that be used by > RegionScannerShippedCallBack when the post hook doing a wrap. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19047) CP exposed Scanner types should not extend Shipper
[ https://issues.apache.org/jira/browse/HBASE-19047?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219308#comment-16219308 ] Anoop Sam John commented on HBASE-19047: bq.Whats going on in Export? Calling Scanner CP hooks? Not sure.. Did not read that part of code in detail any way. Ya bit strange to see direct call to CP hooks. > CP exposed Scanner types should not extend Shipper > -- > > Key: HBASE-19047 > URL: https://issues.apache.org/jira/browse/HBASE-19047 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-19047.patch > > > Shipper is a IA.Private interface and very much internal.. > Right now CP exposed RegionScanner is extending this and so exposing the > shipped() method. This by mistake is called, can harm the correctness of the > cells in the Results. > preScannerOpen() allowing to return a new Scanner is also problematic now. > This can allow users to create a Region scanner from Region and then wrap it > and return back (Well same can be done by postScannerOpen also), it can so > happen that the wrapper is not implementing the shipped() properly. In any > way exposing the shipped () is problematic. > Solution Steps > 1. Remove preScannerOpen() , the use case I can think of is wrapping the > original scanner. The original scanner can be created by Region.getScanner > way only.. May be no need to remove this hook. Just remove the ability for > it to return a RegionScanner instance. Call this with the Scan object and > the CP can change the Scan object if they want. > 2. Let RegionScanner not extending Shipper but only RegionScannerImpl > implements this > 3. We have ref to the RegionScanner created by core and let that be used by > RegionScannerShippedCallBack when the post hook doing a wrap. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19048) Cleanup MasterObserver hooks which takes IA private params
[
https://issues.apache.org/jira/browse/HBASE-19048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219263#comment-16219263
]
Anoop Sam John commented on HBASE-19048:
+1. We can work with follow-on issue as needed. Also that test case issue u
mentioned. (Sleep)
> Cleanup MasterObserver hooks which takes IA private params
> --
>
> Key: HBASE-19048
> URL: https://issues.apache.org/jira/browse/HBASE-19048
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Anoop Sam John
>Assignee: stack
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-19048.master.001.patch,
> HBASE-19048.master.002.patch, HBASE-19048.master.003.patch,
> HBASE-19048.master.003.patch
>
>
> These are the ones in MasterObserver
> {code}
> preAbortProcedure- ProcedureExecutor
> postGetProcedures- Procedure
> postGetLocks - LockedResource
> preRequestLock - LockType
> postRequestLock - LockType
> preLockHeartbeat - LockProcedure
> postLockHeartbeat- LockProcedure
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HBASE-19000) Group multiple block cache clear requests per server
[ https://issues.apache.org/jira/browse/HBASE-19000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zach York reassigned HBASE-19000: - Assignee: Zach York > Group multiple block cache clear requests per server > > > Key: HBASE-19000 > URL: https://issues.apache.org/jira/browse/HBASE-19000 > Project: HBase > Issue Type: Sub-task >Reporter: Ted Yu >Assignee: Zach York > Labels: rpc > > During the review of HBASE-18624, I brought up the following: > There would be multiple regions on the same server whose block cache is to be > cleared. > Multiple block cache clear requests should be grouped per server to reduce > the number of RPC calls. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19088) move_tables_rsgroup will throw an exception when the table is disabled
[
https://issues.apache.org/jira/browse/HBASE-19088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219235#comment-16219235
]
Hadoop QA commented on HBASE-19088:
---
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 2m
49s{color} | {color:blue} Docker mode activated. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} hbaseanti {color} | {color:green} 0m
0s{color} | {color:green} Patch does not have any anti-patterns. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 1 new or modified test
files. {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
34s{color} | {color:green} branch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 4m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} mvneclipse {color} | {color:green} 0m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} shadedjars {color} | {color:green} 5m
13s{color} | {color:green} patch has no errors when building our shaded
downstream artifacts. {color} |
| {color:green}+1{color} | {color:green} hadoopcheck {color} | {color:green}
49m 43s{color} | {color:green} Patch does not cause any errors with Hadoop
2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha4. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 2m
52s{color} | {color:green} hbase-rsgroup in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
8s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 73m 19s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/hbase:eee3b01 |
| JIRA Issue | HBASE-19088 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12893970/HBASE-19088.master.002.patch
|
| Optional Tests | asflicense javac javadoc unit findbugs shadedjars
hadoopcheck hbaseanti checkstyle compile |
| uname | Linux f571155a5e0b 3.13.0-117-generic #164-Ubuntu SMP Fri Apr 7
11:05:26 UTC 2017 x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
|
| git revision | master / 28d81295f3 |
| Default Java | 1.8.0_141 |
| Test Results |
https://builds.apache.org/job/PreCommit-HBASE-Build/9404/testReport/ |
| modules | C: hbase-rsgroup U: hbase-rsgroup |
| Console output |
https://builds.apache.org/job/PreCommit-HBASE-Build/9404/console |
| Powered by | Apache Yetus 0.4.0 http://yetus.apache.org |
This message was automatically generated.
> move_tables_rsgroup will throw an exception when the table is disabled
> --
>
> Key: HBASE-19088
> URL: https://
[jira] [Commented] (HBASE-19048) Cleanup MasterObserver hooks which takes IA private params
[
https://issues.apache.org/jira/browse/HBASE-19048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219233#comment-16219233
]
stack commented on HBASE-19048:
---
+1 [~anoop.hbase]? I can file the follow-on.
> Cleanup MasterObserver hooks which takes IA private params
> --
>
> Key: HBASE-19048
> URL: https://issues.apache.org/jira/browse/HBASE-19048
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Anoop Sam John
>Assignee: stack
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-19048.master.001.patch,
> HBASE-19048.master.002.patch, HBASE-19048.master.003.patch,
> HBASE-19048.master.003.patch
>
>
> These are the ones in MasterObserver
> {code}
> preAbortProcedure- ProcedureExecutor
> postGetProcedures- Procedure
> postGetLocks - LockedResource
> preRequestLock - LockType
> postRequestLock - LockType
> preLockHeartbeat - LockProcedure
> postLockHeartbeat- LockProcedure
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19030) nightly runs should attempt to log test results after archiving
[ https://issues.apache.org/jira/browse/HBASE-19030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219218#comment-16219218 ] Hudson commented on HBASE-19030: SUCCESS: Integrated in Jenkins build HBase-1.1-JDK8 #2016 (See [https://builds.apache.org/job/HBase-1.1-JDK8/2016/]) HBASE-19030 nightly runs should attempt to log test results after (busbey: rev d0a3a747e448bbeddb96541194b5ed64b1ee8f20) * (edit) dev-support/Jenkinsfile > nightly runs should attempt to log test results after archiving > --- > > Key: HBASE-19030 > URL: https://issues.apache.org/jira/browse/HBASE-19030 > Project: HBase > Issue Type: Bug > Components: test >Reporter: Sean Busbey >Assignee: Sean Busbey >Priority: Critical > Fix For: 3.0.0, 1.4.0, 1.3.2, 1.5.0, 1.2.7, 1.1.13, 2.0.0-alpha-4 > > Attachments: HBASE-19030.0.patch > > > right now on the nightly tests the first post-action we do is log junit > results. due to current limitations of Jenkins DSL, if this step fails none > of the other post actions will happen. > Since we might not make junit test results, e.g. in the case of a timeout of > yetus itself, we should log the junit results after we've saved whatever we > can of yetus output. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219215#comment-16219215 ] Anoop Sam John commented on HBASE-18995: Just summarizing the points from RB comments and replies. 1. I was saying abt the need for a non public but LP for CP util class.. I did not see the new Util already with LP for CP. Then we will need 3 Utils. CellUtil public, an LP exposed one and a private one. We have many APIs which not even to be given to CPs. 2. Ya we better make Tag also LP and have the related methods moved to CP exposed Util class. May be we can even remove those APIs from public CellUtil with out deprecation cycle? Because no user would have been using it in client side as we never exposed Tags in client side. Ya CP side would have been using but any way lots of BC breaks around CP 3. Matching APIs ya lets keep the ones which take Cells as param and Cell and byte[] as params in LP class. It will be useful for CPs. The ones with offset, length and all, ya lets keep in private util 4. We have APIs like write parts of cells to OS etc. All these can be in private CellUtil. Those which work on ByteRange also. 5. Any APIs which were not released in 1.x branches, we can remove from public CellUtil if needed. No deprecation cycle for them Any other points. Feel free to split this entire thig as a follow on jira also as needed. The hard thing would be to get names for diff levels of exposed classes :-) > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219205#comment-16219205 ] ramkrishna.s.vasudevan commented on HBASE-18995: Resubmitting for QA. I have kept InternalCellUtil as is till I get a better name. CoprocessorCellUtil seems not good to me. This is mainly for QA to see any findbugs warning and test case issues. Because in last run there was a stack overflow exception because of one change. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-18995: --- Attachment: HBASE-18995-branch-2_1.patch > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-18995: --- Status: Patch Available (was: Open) > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch, HBASE-18995-branch-2_1.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ramkrishna.s.vasudevan updated HBASE-18995: --- Status: Open (was: Patch Available) > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18906) Provide Region#waitForFlushes API
[ https://issues.apache.org/jira/browse/HBASE-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219188#comment-16219188 ] Anoop Sam John commented on HBASE-18906: bq.Do you know if Phoenix still has access to memstore size? Ya it is possible to get what that part of code wants. If the wait op is been interrupted, we do Thread.currentThread().interrupt(); again. bq.Could we get a documentation update that tells me, as a consumer, what happens if the timeout is exceeded? (should be an InterruptedException, right?). Should Region#waitForFlushes(long) throws InterruptedException? No.. it wont throw InterruptedException. It is not an interruption case. I was trying to follow same way as that of Object.wait(timeout). If within time out notify happens, we come out of wait then. Or else at timeout time. But no Exception. So for which condition we were waiting, the callee is supposed to check that condition again once out of wait.But here it is bit diff case as we were waiting for flushes. We dont have any API to know whether flush is ongoing or not. For Phoenix case, it is the memstore size check which is fine. But here I think it would be better to return a boolean indicating whether it is flushes over or not. Sounds good? > Provide Region#waitForFlushes API > - > > Key: HBASE-18906 > URL: https://issues.apache.org/jira/browse/HBASE-18906 > Project: HBase > Issue Type: Sub-task > Components: Coprocessors >Reporter: Anoop Sam John >Assignee: Anoop Sam John > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18906.patch > > > Expose an API for the CPs to wait for all on going flushes in a Region. The > API should support taking a time out. > Background > While reviewing HBASE-18183, Andy pointed out that Phoenix uses > waitForFlushesAndCompactions and/or waitForFlushes for diff reasons. This > issue is to see why they need them and whether alternate ways are possible. > This seems to be too much internal stuff and a normal CP hook calling these > would be dangerous. > If there are alternate ways for Phoenix not to use this and not landing in > issues (As said by Andy) we should suggest/fix for them. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19008) Add missing equals or hashCode method(s) to stock Filter implementations
[
https://issues.apache.org/jira/browse/HBASE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219180#comment-16219180
]
Ted Yu commented on HBASE-19008:
Thanks for taking this, Jan.
> Add missing equals or hashCode method(s) to stock Filter implementations
>
>
> Key: HBASE-19008
> URL: https://issues.apache.org/jira/browse/HBASE-19008
> Project: HBase
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Jan Hentschel
>
> In HBASE-15410, [~mdrob] reminded me that Filter implementations may not
> write {{equals}} or {{hashCode}} method(s).
> This issue is to add missing {{equals}} or {{hashCode}} method(s) to stock
> Filter implementations such as KeyOnlyFilter.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18602) rsgroup cleanup unassign code
[
https://issues.apache.org/jira/browse/HBASE-18602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chia-Ping Tsai updated HBASE-18602:
---
Fix Version/s: 1.5.0
1.4.0
> rsgroup cleanup unassign code
> -
>
> Key: HBASE-18602
> URL: https://issues.apache.org/jira/browse/HBASE-18602
> Project: HBase
> Issue Type: Improvement
> Components: rsgroup
>Reporter: Wang, Xinglong
>Assignee: Wang, Xinglong
>Priority: Minor
> Fix For: 2.0.0, 1.4.0, 1.5.0
>
> Attachments: HBASE-18602-master-v1.patch,
> HBASE-18602-master-v2.patch, HBASE-18602-master-v3.patch,
> HBASE-18602-master-v3.patch, HBASE-18602-master-v3.patch,
> HBASE-18602-master-v4.patch
>
>
> While walking through rsgroup code, I found that variable misplacedRegions
> has never been added any element into. This makes the unassign region code is
> not functional. And according to my test, it is actually unnecessary to do
> that.
> RSGroupBasedLoadBalancer.java
> {code:java}
> private Map> correctAssignments(
>Map> existingAssignments)
> throws HBaseIOException{
> Map> correctAssignments = new TreeMap<>();
> List misplacedRegions = new LinkedList<>();
> correctAssignments.put(LoadBalancer.BOGUS_SERVER_NAME, new
> LinkedList<>());
> for (Map.Entry> assignments :
> existingAssignments.entrySet()){
> ServerName sName = assignments.getKey();
> correctAssignments.put(sName, new LinkedList<>());
> List regions = assignments.getValue();
> for (HRegionInfo region : regions) {
> RSGroupInfo info = null;
> try {
> info = rsGroupInfoManager.getRSGroup(
> rsGroupInfoManager.getRSGroupOfTable(region.getTable()));
> } catch (IOException exp) {
> LOG.debug("RSGroup information null for region of table " +
> region.getTable(),
> exp);
> }
> if ((info == null) || (!info.containsServer(sName.getAddress( {
> correctAssignments.get(LoadBalancer.BOGUS_SERVER_NAME).add(region);
> } else {
> correctAssignments.get(sName).add(region);
> }
> }
> }
> //TODO bulk unassign?
> //unassign misplaced regions, so that they are assigned to correct groups.
> for(HRegionInfo info: misplacedRegions) {
> try {
> this.masterServices.getAssignmentManager().unassign(info);
> } catch (IOException e) {
> throw new HBaseIOException(e);
> }
> }
> return correctAssignments;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HBASE-18984) [AMv2] Retain assignment does not work well in AMv2
[ https://issues.apache.org/jira/browse/HBASE-18984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yi Liang updated HBASE-18984: - Resolution: Duplicate Status: Resolved (was: Patch Available) > [AMv2] Retain assignment does not work well in AMv2 > --- > > Key: HBASE-18984 > URL: https://issues.apache.org/jira/browse/HBASE-18984 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-18984-V1-master.patch, Screen Shot 2017-10-10 at > 2.24.19 PM.png > > > work on 8.17 Retain assignment in > https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.epjn9nege80k > To reproduce this error, in hbase shell: > createTable t1 --> list_regions 't1' --> disable 't1' ---> enable 't1' --> > list_reigons 't1' (maybe you need to try enable/disable multiple times) > See attached images. same region assigned to different region servers -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-18984) [AMv2] Retain assignment does not work well in AMv2
[ https://issues.apache.org/jira/browse/HBASE-18984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219168#comment-16219168 ] Yi Liang commented on HBASE-18984: -- Mark this as duplicated of HBASE-19017, and will create a new jira to discuss the Writing those intermediate State(OPENING, CLOSING ..) into Meta. > [AMv2] Retain assignment does not work well in AMv2 > --- > > Key: HBASE-18984 > URL: https://issues.apache.org/jira/browse/HBASE-18984 > Project: HBase > Issue Type: Bug > Components: proc-v2 >Affects Versions: 2.0.0 >Reporter: Yi Liang >Assignee: Yi Liang > Fix For: 2.0.0 > > Attachments: HBASE-18984-V1-master.patch, Screen Shot 2017-10-10 at > 2.24.19 PM.png > > > work on 8.17 Retain assignment in > https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.epjn9nege80k > To reproduce this error, in hbase shell: > createTable t1 --> list_regions 't1' --> disable 't1' ---> enable 't1' --> > list_reigons 't1' (maybe you need to try enable/disable multiple times) > See attached images. same region assigned to different region servers -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HBASE-19029) Align RPC timout methods in Table and AsyncTableBase
[
https://issues.apache.org/jira/browse/HBASE-19029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219164#comment-16219164
]
Peter Somogyi commented on HBASE-19029:
---
Thanks for the review!
> Align RPC timout methods in Table and AsyncTableBase
>
>
> Key: HBASE-19029
> URL: https://issues.apache.org/jira/browse/HBASE-19029
> Project: HBase
> Issue Type: Sub-task
> Components: asyncclient, Client
>Affects Versions: 2.0.0-alpha-3
>Reporter: Peter Somogyi
>Assignee: Peter Somogyi
>Priority: Critical
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-19029.master.001.patch,
> HBASE-19029.master.002.patch, HBASE-19029.master.002.patch,
> HBASE-19029.master.003.patch, HBASE-19029.master.003.patch,
> HBASE-19029.master.003.patch, HBASE-19029.master.004.patch
>
>
> Table and AsyncTableBase have similar RPC timeout methods but the async
> version supports TimeUtils to be passed.
> To align these 2 interfaces lets depricate the existing methods in Table and
> add the ones that are currently in AsyncTableBase.
> These methods are the following:
> * long getRpcTimeout(TimeUnit unit)
> * long getReadRpcTimeout(TimeUnit unit)
> * long getWriteRpcTimeout(TimeUnit unit)
> * long getOperationTimeout(TimeUnit unit)
> We do not have {{long getScanTimeout(TimeUnit unit)}} since scan is handled
> differently.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-19087) Remove redundant characters from the audit log
[
https://issues.apache.org/jira/browse/HBASE-19087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219161#comment-16219161
]
Josh Elser commented on HBASE-19087:
+1 on QA. Looks fine to me -- did a quick grep over hbase-server on branch-2
and I don't see anything else needing a similar change.
> Remove redundant characters from the audit log
> --
>
> Key: HBASE-19087
> URL: https://issues.apache.org/jira/browse/HBASE-19087
> Project: HBase
> Issue Type: Bug
>Affects Versions: 2.0.0-alpha-4
>Reporter: Guangxu Cheng
>Assignee: Guangxu Cheng
> Fix For: 2.0.0-beta-1
>
> Attachments: HBASE-19087.master.001.patch,
> HBASE-19087.master.001.patch
>
>
> After HBASE-18878, the audit log contains redundant characters "Optional"
> which leading to unreadable
> {code:java}
> 2017-10-19 19:51:08,054 INFO
> [RpcServer.default.FPBQ.Fifo.handler=49,queue=4,port=8081] master.HMaster:
> Client=Optional[username]/Optional[/locahost] disable tablename
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (HBASE-18906) Provide Region#waitForFlushes API
[
https://issues.apache.org/jira/browse/HBASE-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219146#comment-16219146
]
Josh Elser edited comment on HBASE-18906 at 10/25/17 5:33 PM:
--
{code}
+ /**
+ * Wait for all current flushes of the region to complete
+ *
+ * @param timeout The maximum time to wait in milliseconds.
+ */
{code}
Could we get a documentation update that tells me, as a consumer, what happens
if the {{timeout}} is exceeded? (should be an InterruptedException, right?).
Should {{Region#waitForFlushes(long)}} {{throws InterruptedException}}?
edit: otherwise, looks fine to me. Nice, small+contained fix which should make
Phoenix's life a little easier.
was (Author: elserj):
{code}
+ /**
+ * Wait for all current flushes of the region to complete
+ *
+ * @param timeout The maximum time to wait in milliseconds.
+ */
{code}
Could we get a documentation update that tells me, as a consumer, what happens
if the {{timeout}} is exceeded? (should be an InterruptedException, right?).
Should {{Region#waitForFlushes(long)}} {{throws InterruptedException}}?
> Provide Region#waitForFlushes API
> -
>
> Key: HBASE-18906
> URL: https://issues.apache.org/jira/browse/HBASE-18906
> Project: HBase
> Issue Type: Sub-task
> Components: Coprocessors
>Reporter: Anoop Sam John
>Assignee: Anoop Sam John
> Fix For: 2.0.0-alpha-4
>
> Attachments: HBASE-18906.patch
>
>
> Expose an API for the CPs to wait for all on going flushes in a Region. The
> API should support taking a time out.
> Background
> While reviewing HBASE-18183, Andy pointed out that Phoenix uses
> waitForFlushesAndCompactions and/or waitForFlushes for diff reasons. This
> issue is to see why they need them and whether alternate ways are possible.
> This seems to be too much internal stuff and a normal CP hook calling these
> would be dangerous.
> If there are alternate ways for Phoenix not to use this and not landing in
> issues (As said by Andy) we should suggest/fix for them.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HBASE-18995) Move methods that are for internal usage from CellUtil to Private util class
[ https://issues.apache.org/jira/browse/HBASE-18995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16219149#comment-16219149 ] stack commented on HBASE-18995: --- bq. I thought it is better we allow CPs to use this? Probably will be helpful for cases like the timeline server projects where they need access to tags? Good point. > Move methods that are for internal usage from CellUtil to Private util class > > > Key: HBASE-18995 > URL: https://issues.apache.org/jira/browse/HBASE-18995 > Project: HBase > Issue Type: Sub-task >Affects Versions: 2.0.0-alpha-3 >Reporter: ramkrishna.s.vasudevan >Assignee: ramkrishna.s.vasudevan >Priority: Critical > Fix For: 2.0.0-alpha-4 > > Attachments: HBASE-18995-branch-2.patch > > > This was brought up long time back. We need to move some of the public APIs > from CellUtil to internal private Util class because they are used in some > internal flow and does not make sense to have it in a @public exposed Util > class. > The topic again came in HBASE-18945 RB comments also. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
