[
https://issues.apache.org/jira/browse/TRAFODION-1729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15080609#comment-15080609
]
liu ming commented on TRAFODION-1729:
-
Hi, Stack,
Yes, the patch is exactly what Trafodion need, if that can be merged into HBase
upstream, it will help T8 folks a lot!
Thank you for considering this. So you will do that change in HBase?
thanks,
Ming
> change the coprocessor deployment method
>
>
> Key: TRAFODION-1729
> URL: https://issues.apache.org/jira/browse/TRAFODION-1729
> Project: Apache Trafodion
> Issue Type: Improvement
> Components: dtm
>Reporter: liu ming
>Assignee: mashengchen
>
> I have a proposal to change our current HBase coprocessor configuration
> method.
> There are three ways to add a coprocessor to a HBase table:
> 1. Via editing hbase-site.xml, which will load coprocessor for ALL
> tables (Trafodion is using this method now)
> 2. Via HBase shell command
> 3. Via HTableDescriptor.addCoprocessor() java API
> Trafodion now is using the first method. I proposed to use method 3, I
> finished a prototype and test seems works well.
>
> Here are the reasons I propose for this change:
> At present, the Trafodion installer needs to modify the hbase-site.xml and
> then restart HBase instance for the configuration to take effect. This step
> not only complicate the installer but also let user think Trafodion is
> intrusive into underlying HBase system. It will be ideal if we can avoid this
> step. Another problem: in CDH, there is a concept called ‘region server
> group’ or something, so the settings will have to carefully handled by
> installer to apply to all groups. As we saw recently in WebRoot deployment,
> Trafodion failed due to this reason. All these are very error prone and
> complicate the Trafodion installer. Once CDH or HDP changed something,
> Trafodion may fail again.
>
> So I spent time to investigate why we need to restart HBase in order to
> install Trafodion.
> As I understand, there are 3 major reasons
> 1. To add hbase-trx coprocessors
> 2. To overload HRegion with TransactionalRegion
> 3. Various configuration settings, need to check one by one.
> The first configuration can be avoided by applying my proposed change.
> The second one, I look through the TransactionalRegion.java, and find out the
> only reason (now) is to overload the getScanner() method to be public so can
> be invoked by the coprocessor. And there are only 1 or 2 places that API is
> invoked in Trafodion code. I checked with Kevin and he proposed by using
> ‘java reflection’ we can also avoid this.
> All other configuration items to some extent look like ‘best to have’, but
> not ‘must to have’. And I also find two config items seems never been used:
> hbase.bulkload.staging.dir /hbase-staging (Suresh can confirm,
> but I search in all code, seems this is never used)
> hbase.regionserver.region.transactional.tlog true (Narendra can
> confirm, this is NEVER used, maybe a legacy config item?)
> Yes, by now, there are still some other config items seems cannot be avoided,
> but I hope we can find some way to remove them in the future. I am not trying
> to solve all issues right now, just want to start the effort to remove
> unnecessary hbase reconfiguration.
> For this example, Coprocessors can be added to a table at run time, no need
> to edit the hbase-site.xml and restart hbase. This is only the first step to
> try to remove the deep impact to the current HBase config and restart HBase.
>
> So I asked for your opinions about this change. If you think this is
> necessary, I will continue to file a JIRA and fix it.
>
> I strongly recommend to get rid of the step of ‘modify hbase-site.xml and
> restart your hbase’ for Trafodion installation, it should be an option , to
> tune the system to best suit Trafodion, but should not be a forced step. To
> be note: Apache Phoenix is also a SQL on HBase, its installation will change
> nothing of underlying HBase, very lightweight, no ‘intrude into’ the existing
> HBase system. Trafodion is considered to be heavy and intrusive in this
> manner, and I feel maybe we can change this.
>
> Should I start this discussion in the dev mail list?
>
> P.S. a list of changed config items. My proposal will remove the last one,
> hope we can get rid of all of them:
> hbase.master.distributed.log.splitting false
> hbase.snapshot.master.timeoutMillis 60
> hbase_regionserver_lease_period 60
> hbase.hregion.impl
> org.apache.hadoop.hbase.regionserver.transactional.TransactionalRegion
> hbase.regionserver.region.split.policy
> org.apache.hadoop.hbase.regionserver.ConstantSizeRegionSplitPolicy
