RE: Phoenix as a source for Spark processing

2018-03-05 Thread Stepan Migunov
Hi Josh, thank you for response! Our cluster has 14 nodes (32 cores each/128 GB memory). The source Phoenix table contains about 1 billion records (100 columns). We start a Spark's job with about 100 executors. Spark executes SELECT from the source table (select 6 columns with DISTINCT) and

Re: Odd Syntax Error during Index creation on View

2018-03-05 Thread Josh Elser
Yes. If you're starting from a blank slate, please use the Phoenix SQL statements to create tables, not the HBase shell. On 3/5/18 4:33 AM, Dominic Egger wrote: Hi Phoenix Users So I have a somewhat baffling errors. I have create a view on the follwoing HBase Table: create 'xx:yy', {NAME =>

Re: Phoenix as a source for Spark processing

2018-03-05 Thread Josh Elser
Hi Stepan, Can you better ballpark the Phoenix-Spark performance you've seen (e.g. how much hardware do you have, how many spark executors did you use, how many region servers)? Also, what versions of software are you using? I don't think there are any firm guidelines on how you can solve

Re: AW: Duplicate Records Showing in Apache Phoenix

2018-03-05 Thread Azharuddin Shaikh
Thanks Martin for your reply. Actually after deleting system.stats also we are getting the same issue after few hours. It is occurring continuously. As of now, we have increased the guidepost width from 100MB to 5GB post which issue has not occurred but we are not confidence whether this is the

Re: Phoenix shows incorrect count as compared to Hbase count

2018-03-05 Thread Azharuddin Shaikh
Hi, We tried deleting system.stats for impacted table & restarted the hbase but again after few hours we are getting same issue. Not able to understand is this a bug with phoenix. We have now increased the guidepost width from 100MB to 5GB after which issue has not occured but still we are not

Odd Syntax Error during Index creation on View

2018-03-05 Thread Dominic Egger
Hi Phoenix Users So I have a somewhat baffling errors. I have create a view on the follwoing HBase Table: create 'xx:yy', {NAME => 'columns', COMPRESSION => 'SNAPPY'} , {SPLITS => ['1', '2', '3', '4', '5', '6', '7', '8', '9', 'a', 'b', 'c', 'd', 'e', 'f'] } So far this worked very well. But when

AW: Duplicate Records Showing in Apache Phoenix

2018-03-05 Thread Hengesbach, Martin
Hi, we had the same problem in our Phoenix 4.7 installation two or three times. We got it "solved" with a combination of "update statistics", "delete * from system.stats", rebuilding the indices of the table and restart of hbase. The problem is not reproducible at the moment and occurs very