Re: Adding features to HBase Input Operators in Malhar-contrib

2016-03-28 Thread Sandeep Deshmukh
I shall do that in a day or two. Regards, Sandeep On Thu, Mar 24, 2016 at 6:10 PM, Bhupesh Chawda wrote: > Dear Community, > > Can anyone help review the pull request: > https://github.com/apache/incubator-apex-malhar/pull/212 > > Thanks. > > ~Bhupesh > > On Thu, Mar 17, 2016 at 4:16 PM, Bhupes

Re: Adding features to HBase Input Operators in Malhar-contrib

2016-03-24 Thread Bhupesh Chawda
Dear Community, Can anyone help review the pull request: https://github.com/apache/incubator-apex-malhar/pull/212 Thanks. ~Bhupesh On Thu, Mar 17, 2016 at 4:16 PM, Bhupesh Chawda wrote: > Hi, > > I have opened a pull request for the changes as described in the previous > emails. Here is the p

Re: Adding features to HBase Input Operators in Malhar-contrib

2016-03-20 Thread Bhupesh Chawda
Hi, I have opened a pull request for the changes as described in the previous emails. Here is the pull request: https://github.com/apache/incubator-apex-malhar/pull/212 Here is a short description of the changes: HBaseInputOperator - Takes care of HBaseStore and its connection. Got rid of HBaseO

Re: Adding features to HBase Input Operators in Malhar-contrib

2016-03-11 Thread Bhupesh Chawda
Hi All, In the current design of HBase input and output operators, the row key is hard-coded to be of String type. I foresee the following issue: - In case of numeric keys which are type casted to String, *incremental read* is problematic. For example, after reading key = 9, we may not be

Re: Adding features to HBase Input Operators in Malhar-contrib

2015-12-30 Thread Sandeep Deshmukh
Looks fine to me. Regards, Sandeep On Wed, Dec 30, 2015 at 7:34 PM, Bhupesh Chawda wrote: > Here is the final hierarchy I am considering: > > HBaseInputOperator - Takes care of HBaseStore and its connection. Got rid > of HBaseOperatorBase. > HBaseScanOperator - Takes care of scanning the ta

Re: Adding features to HBase Input Operators in Malhar-contrib

2015-12-30 Thread Bhupesh Chawda
Here is the final hierarchy I am considering: HBaseInputOperator - Takes care of HBaseStore and its connection. Got rid of HBaseOperatorBase. HBaseScanOperator - Takes care of scanning the table in a non-blocking manner. Exposes operationScan() and getTuple() as before. HBasePOJOInputO

Re: Adding features to HBase Input Operators in Malhar-contrib

2015-12-30 Thread Bhupesh Chawda
The class HBaseInputOperator seems to be quite old. HBaseStore seems to be having all the functionality provided by HBaseInputOperator and even more (including Kerberos authentication). It would be a good idea to avoid the usage of HBaseInputOperator going forward and use HBaseStore instead. I wi

Re: Adding features to HBase Input Operators in Malhar-contrib

2015-12-23 Thread Bhupesh Chawda
Thanks for the inputs. As an input operator, I am targeting just the Scan operation. Get operation may be supported better as a generic operator (like a query operator) which I can take up later. -Bhupesh On Tue, Dec 22, 2015 at 3:48 PM, Mohit Jotwani wrote: > +1 > > Regards, > Mohit > > On Tue

Re: Adding features to HBase Input Operators in Malhar-contrib

2015-12-22 Thread Mohit Jotwani
+1 Regards, Mohit On Tue, Dec 22, 2015 at 11:21 AM, Chinmay Kolhatkar wrote: > +1 for above. > I see that there is HbaseGetOperator but but its abstract no concrete > implementation of this I can find. > Are you going to implement of that too? > > Maybe the concrete implementation of HbaseGetOp

Re: Adding features to HBase Input Operators in Malhar-contrib

2015-12-21 Thread Chinmay Kolhatkar
+1 for above. I see that there is HbaseGetOperator but but its abstract no concrete implementation of this I can find. Are you going to implement of that too? Maybe the concrete implementation of HbaseGetOperator should have this. Also, I want to mention one thing about scan from my previous expe

Re: Adding features to HBase Input Operators in Malhar-contrib

2015-12-21 Thread Sandeep Deshmukh
+1 for this Bhupesh. Additionally, I would suggest to add support for; 1. Point query 2. Returning any row version The above two are key features of HBase and should be supported. Regards, Sandeep On Fri, Dec 18, 2015 at 4:39 PM, Bhupesh Chawda wrote: > Hi All, > > The current HBasePOJOInputO

Adding features to HBase Input Operators in Malhar-contrib

2015-12-18 Thread Bhupesh Chawda
Hi All, The current HBasePOJOInputOperator does not allow us to do the following: 1. Allow us to specify a set of "column family: column" and fetch data only for these columns. 2. Output format is currently a POJO. We need to have other output formats such that "columnFamily:column" r