How can I "use" a hbase co-processor from a User Defined Function?

2017-04-12 Thread Cheyenne Forbes
If I have a coprocessor of class "MyCoprocessor" which extends "BaseRegionObserverCoprocessor" is it possible to "access" it from a Phoenix UDF? Regards, Cheyenne O. Forbes

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-12 Thread Josh Elser
Since you're writing the UDF yourself, you can have it do anything you'd like. However, I wouldn't think that it would be a good idea to do a remote RPC for every potential row that you're processing in a query... On Wed, Apr 12, 2017 at 5:45 PM, Cheyenne Forbes wrote: > If I have a coprocessor

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-14 Thread Cheyenne Forbes
If the UDF will be used only in select queries "SELECT my_udf(column) FROM table WHERE id = 123" would it be possible to access the MyCoprocessor instance on the region server the row is on instead of doing a remote call? maybe like *MyCoprocessor cp = get_cp(MyCoprocessor.getName());*

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-14 Thread James Taylor
No, that's not possible. Thanks, James On Fri, Apr 14, 2017 at 9:48 AM Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > If the UDF will be used only in select queries "SELECT my_udf(column) FROM > table WHERE id = 123" would it be possible to access the MyCoprocessor > instance on the

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-14 Thread Cheyenne Forbes
would *my_udf* be executed on the region server that the row of the column that is passed to it?

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-14 Thread Cheyenne Forbes
would *my_udf* be executed on the region server that the row of the column that is passed to it is located on ?

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-17 Thread Sergey Soldatov
No. UDF function doesn't have any context where it's executed, so it can't obtain neither region instance nor coprocessor instance Thanks, Sergey On Fri, Apr 14, 2017 at 10:39 AM, Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > would *my_udf* be executed on the region server that the

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-17 Thread Cheyenne Forbes
so there is no way of getting HRegion in a UDF?

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-18 Thread Sergey Soldatov
Well, theoretically there is a way of having a coprocessor that will keep static public map of current rowkey processed by Phoenix and the correlated HRegion instance and get this HRegion using the key that is processed by evaluate function. But it's a completely wrong approach for both HBase and P

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-18 Thread James Taylor
Shorter answer is "no". Your UDF may be executed on the client side as well (depending on the query) and there is of course no HRegion available from the client. On Tue, Apr 18, 2017 at 11:10 AM Sergey Soldatov wrote: > Well, theoretically there is a way of having a coprocessor that will keep >

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-18 Thread Cheyenne Forbes
I am creating a plugin that uses Lucene to index text fields and I need to access *getConf()* and *getFilesystem()* of *HRegion, *the Lucene indexes are split with the regions so I need " *HRegion MyVar; ", *I am positive the UDF will run on the region server and not the client *.*

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-18 Thread Cheyenne Forbes
I am creating a plugin that uses Lucene to index text fields and I need to access *getConf()* and *getFilesystem()* of *HRegion, *the Lucene indexes are split with the regions so I need " *HRegion MyVar; ", *I am positive the UDF will run on the region server and not the client*.* Regards, Cheye

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-18 Thread Sergey Soldatov
I may be wrong, but you have chosen wrong approach. Such kind of integration need to be (should be) done on the Phoenix layer in the way like global/local indexes are implemented. Thanks, Sergey On Tue, Apr 18, 2017 at 12:34 PM, Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > I am cr

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-18 Thread Cheyenne Forbes
Could you explain a little more what you mean by that? Regards, Cheyenne O. Forbes On Tue, Apr 18, 2017 at 4:36 PM, Sergey Soldatov wrote: > I may be wrong, but you have chosen wrong approach. Such kind of > integration need to be (should be) done on the Phoenix layer in the way > like global

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-18 Thread Sergey Soldatov
I mean you need to modify Phoenix code itself to properly support such kind of features. Thanks, Sergey On Tue, Apr 18, 2017 at 3:52 PM, Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > Could you explain a little more what you mean by that? > > Regards, > > Cheyenne O. Forbes > > > On

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-19 Thread Cheyenne Forbes
I'd still need " *HRegion MyVar; ", *because I'd still need the name of the region where the row of the id passed to the UDF is located and the value returned my* "getFilesystem()" *of* "**HRegion", *what do you recommend that I do? Regards, Cheyenne O. Forbes On Tue, Apr 18, 2017 at 6:27 PM,

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-19 Thread James Taylor
Can you describe the functionality you're after at a high level in terms of a use case (rather than an implementation idea/detail) and we can discuss any options wrt potential new features? On Wed, Apr 19, 2017 at 8:53 AM Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > I'd still need

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-19 Thread Cheyenne Forbes
I created a hbase co-processor that stores/deletes text indexes with Lucene, the indexes are stored on HDFS (for back up, replication, etc.). The indexes "mirror" the regions so if the index for a column is at "hdfs://localhost:9000/hbase/region_name" the index is stored at "hdfs://localhost:9000/l

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-19 Thread Sergey Soldatov
How do you handle HBase region splits and merges with such architecture? Thanks, Sergey On Wed, Apr 19, 2017 at 9:22 AM, Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > I created a hbase co-processor that stores/deletes text indexes with > Lucene, the indexes are stored on HDFS (for

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-04-19 Thread Cheyenne Forbes
At postOpen the location of the lucene directory to be used for the region is set using the value of *"h_region.getRegionInfo().getEncodedName();" *so whenever prePut is called the index of the column is stored in the directory that was set during postOpen. So basically the lucene operations are "t

Re: How can I "use" a hbase co-processor from a User Defined Function?

2017-05-12 Thread Cheyenne Forbes
Any updates on how I'd go about getting *"**HRegion" *in a UDF? Regards, Cheyenne O. Forbes On Wed, Apr 19, 2017 at 6:03 PM, Cheyenne Forbes < cheyenne.osanu.for...@gmail.com> wrote: > At postOpen the location of the lucene directory to be used for the region > is set using the value of *"h_reg