OK, I am starting with the connector that will rely on an abstract event listener. I can also work in parallel on the Beam integration, just let me know how you want to proceed.
Best, Sergio. On Thu, Jun 7, 2018 at 10:19 AM, Von Gosling <[email protected]> wrote: > IMO, Co-processor is seldom used in the production consider its heavy cost > and security problem. for simplicity, you could design and implement the > connector for HBase. > > BTW, do we plan to raise the priority of Beam Integration? > > > Best Regards, > Von Gosling > > 在 2018年6月5日，21:41，Sergio Esteves <[email protected]> 写道： > > n that keeps a database in synch with a rocketmq topic. > The first way that comes to my mind to listen to HBase updates on a table, > is by using co-processors (akin to BigTable Observers): https://blogs. > apache.org/hbase/entry/coprocessor_introduction > With co-processors it is possible to insert hooks for data manipulation > events (e.g., put, delete). It is also possible to intercept WAL > (Write-Ahead Log that keeps track of updates on a table) writing of events. > The co-processors reside on the server side and are distributed > automatically alongside HBase region servers. Co-processor custom > implementations require modifications in the HBase configuration. By not > using co-processors, it would be possible to access directly the WAL on > HDFS (in case WAL writing is not disabled). > > These are the two main ways (co-processors or accessing WAL directly) I > can see to listen to HBase table updates. Using co-processors seems the > most natural way to go. For example, implementing a WALObserver and keep > track of the last position that was replicated to the rocketmq topic. Do > you agree? > > >
