bq. Will bulk load from Phoenix update the underlying Hbase table? Yes. instead of using importTSV try to use CSV bulkload only.
bq. Do I need to replace Phoenix view on Hbase as with CREATE TABLE? You can still keep VIEW. Regards, Ankit Singhal On Sun, Oct 23, 2016 at 6:37 PM, Mich Talebzadeh <mich.talebza...@gmail.com> wrote: > Thanks Ankit, > > couple of questions. > > > 1. Will bulk load from Phoenix update the underlying Hbase table? > 2. Do I need to replace Phoenix view on Hbase as with CREATE TABLE? > > regards > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.wordpress.com > > > *Disclaimer:* Use it at your own risk. Any and all responsibility for any > loss, damage or destruction of data or any other property which may arise > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 23 October 2016 at 13:52, Ankit Singhal <ankitsingha...@gmail.com> > wrote: > >> Rebuild is currently a costly operation as it will rebuild the complete >> index again and should be used when you think your index is corrupted and >> you are not aware about the timestamp when it got out of sync. >> >> Why can't you use Bulk loading tool[1] provided by Phoenix instead of >> using importTSV, as this tool will build the data for index table along >> with data table? >> https://phoenix.apache.org/bulk_dataload.html >> >> >> On Sat, Oct 22, 2016 at 11:52 PM, Mich Talebzadeh < >> mich.talebza...@gmail.com> wrote: >> >>> A workout I deployed was to rebuild the index immediately after bulk >>> load of data into Hbase table >>> >>> ALTER INDEX MARKETDATAHBASE_IDX1 ON "marketDataHbase" REBUILD; >>> >>> >>> >>> Dr Mich Talebzadeh >>> >>> >>> >>> LinkedIn * >>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>> >>> >>> >>> http://talebzadehmich.wordpress.com >>> >>> >>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>> any loss, damage or destruction of data or any other property which may >>> arise from relying on this email's technical content is explicitly >>> disclaimed. The author will in no case be liable for any monetary damages >>> arising from such loss, damage or destruction. >>> >>> >>> >>> On 22 October 2016 at 18:25, Mich Talebzadeh <mich.talebza...@gmail.com> >>> wrote: >>> >>>> Sorry Ted, >>>> >>>> This is the syntax for view >>>> >>>> create view "marketDataHbase" (PK VARCHAR PRIMARY KEY, >>>> “price_info”.”ticker” VARCHAR,"price_info"."timecreated" VARCHAR, >>>> "price_info"."price" VARCHAR); >>>> >>>> Thanks James for clarification. >>>> >>>> My understanding is that when one creates an index on a Phoenix view on >>>> an Hbase table, a local index file is created in Phoenix with data as is >>>> and that Phoenix table has no way of knowing if new data is added to Hbase >>>> table. >>>> >>>> From an operational point of view unless index in Phoenix is >>>> immediately updated after any append to Hbase, then it is pretty useless. >>>> >>>> In my case data is inserted to Hbase table. I am just using Phoenix for >>>> data queries (DQ) as opposed to inserts. >>>> >>>> Regards, >>>> >>>> >>>> >>>> >>>> >>>> >>>> >>>> Dr Mich Talebzadeh >>>> >>>> >>>> >>>> LinkedIn * >>>> https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>>> <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>>> >>>> >>>> >>>> http://talebzadehmich.wordpress.com >>>> >>>> >>>> *Disclaimer:* Use it at your own risk. Any and all responsibility for >>>> any loss, damage or destruction of data or any other property which may >>>> arise from relying on this email's technical content is explicitly >>>> disclaimed. The author will in no case be liable for any monetary damages >>>> arising from such loss, damage or destruction. >>>> >>>> >>>> >>>> On 22 October 2016 at 17:59, James Taylor <jamestay...@apache.org> >>>> wrote: >>>> >>>>> Hi Mich, >>>>> Phoenix indexes are only updated if you use Phoenix APIs to input the >>>>> data. >>>>> Thanks, >>>>> James >>>>> >>>>> >>>>> On Saturday, October 22, 2016, Ted Yu <yuzhih...@gmail.com> wrote: >>>>> >>>>>> The first statement creates index, not view. >>>>>> >>>>>> Can you check ? >>>>>> >>>>>> Cheers >>>>>> >>>>>> > On Oct 22, 2016, at 1:51 AM, Mich Talebzadeh < >>>>>> mich.talebza...@gmail.com> wrote: >>>>>> > >>>>>> > Hi, >>>>>> > >>>>>> > I have a Hbase table that is populated via >>>>>> > org.apache.hadoop.hbase.mapreduce.ImportTsv >>>>>> > through bulk load ever 15 minutes. This works fine. >>>>>> > >>>>>> > In Phoenix I created a view on this table >>>>>> > >>>>>> > jdbc:phoenix:rhes564:2181> create index marketDataHbase_idx on >>>>>> > "marketDataHbase" ("price_info"."ticker", "price_info"."price", >>>>>> > "price_info"."timecreated"); >>>>>> > >>>>>> > This also does what is supposed to do and shows correct count. >>>>>> > >>>>>> > I then created an index in Phoenix as below >>>>>> > >>>>>> > create index index_dx1 on "marketDataHbase" >>>>>> > ("price_info"."timecreated","price_info"."ticker", >>>>>> "price_info"."price"); >>>>>> > >>>>>> > that showed the records OK at that time. I verified this using >>>>>> explain >>>>>> > >>>>>> > >>>>>> > 0: jdbc:phoenix:rhes564:2181> explain select count(1) from >>>>>> > "marketDataHbase"; >>>>>> > +---------------------------------------------------------+ >>>>>> > | PLAN | >>>>>> > +---------------------------------------------------------+ >>>>>> > | CLIENT 1-CHUNK PARALLEL 1-WAY FULL SCAN OVER INDEX_DX1 | >>>>>> > | SERVER FILTER BY FIRST KEY ONLY | >>>>>> > | SERVER AGGREGATE INTO SINGLE ROW | >>>>>> > +---------------------------------------------------------+ >>>>>> > >>>>>> > Now the issue is that the above does not show new data since build >>>>>> in Hbase >>>>>> > table unless I do the following: >>>>>> > >>>>>> > 0: jdbc:phoenix:rhes564:2181> alter index INDEX_DX1 on >>>>>> "marketDataHbase" >>>>>> > rebuild; >>>>>> > >>>>>> > >>>>>> > Which is not what an index should do (The covered index should be >>>>>> > maintained automatically). >>>>>> > The simple issue is how to overcome this problem? >>>>>> > >>>>>> > As I understand the index in Phoenix ia another file independent of >>>>>> the >>>>>> > original phoenix view so I assume that this index file is not >>>>>> updated for >>>>>> > one reason or other? >>>>>> > >>>>>> > Thanks >>>>>> >>>>> >>>> >>> >> >
