Re: Why we change index state to PENDING_DISABLE on RegionMovedException

2019-09-10 Thread Vincent Poon
Normally you're right, this should get retried at the HBase layer and would be transparent. However as part of PHOENIX-4130, we have the hbase client only try the write once, so there's no chance to retry. We did that to avoid tying up rpc handlers on the server. Instead, we retry the entire Phoe

Re: Is there any way to using appropriate index automatically?

2019-08-20 Thread Vincent Poon
check out PHOENIX-5109 , it likely fixes your issue. Unfortunately it's targeted for 4.15.0 which hasn't been released yet. Maybe you can backport and see if it works for your query. On Tue, Aug 20, 2019 at 11:38 AM Ankit Singhal wrote: > CREATE local INDEX local_c_h_index ON test_phoenix.app (c

Re: Phoenix Index Scrutiny Tool

2019-08-12 Thread Vincent Poon
@Aleksandr did you delete rows from the data table, or the index table? The output you're showing says that you have orphaned rows in the index table - i.e. rows that exist only in the index table and have no corresponding row in the data table. If you deleted the original rows in the data table w

Re: Query optimization

2019-06-21 Thread Vincent Poon
? e.g. indexes, stats... On Wed, Jun 19, 2019 at 6:18 PM Alexander Batyrshin <0x62...@gmail.com> wrote: > Is it possible not to full scan table1 for ’table1.col = ?’, but do this > check only on subset table1.pk IN (…)? > > On 19 Jun 2019, at 23:31, Vincent Poon wrote: > > &#x

Re: Query optimization

2019-06-19 Thread Vincent Poon
'table1.col = ?' will be a full table scan of table1 unless you have a secondary index on table.col Check the explain plan to see if it's working as expected On Wed, Jun 19, 2019 at 7:43 AM Alexander Batyrshin <0x62...@gmail.com> wrote: > Hello, > We have 2 tables: > > Table1 - big one (2000M+ r

Re: Growth in table size and performance degradation on read-queries

2019-01-29 Thread Vincent Poon
is your max_versions set to 1 ? keep_deleted_cells? On Tue, Jan 29, 2019 at 10:41 AM talluri abhishek wrote: > Hi All, > > We are seeing a couple of issues on some of our Phoenix tables where the > size of the tables keep growing 2-3 times after around 2-3 days of > ingestion and the read perfo

Re: Query All Dynamic Columns

2018-12-26 Thread Vincent Poon
A lot of work is currently going into handling large numbers of views - splittable syscat, view management, etc... but agree that it's not ideal. There's currently no built-in way to do what you want AFAIK, but you can manage the columns yourself in a separate table: - store them all in a single c

Re: Phoenix perform full scan and ignore covered global index

2018-12-26 Thread Vincent Poon
I'm not able to repro this on latest 4.15.0 : 0: jdbc:phoenix:> explain select * from US_POPULATION where city = 'test'; +-+-++--+ |

Re: "upsert select" with "limit" clause

2018-12-20 Thread Vincent Poon
e 3000-5000 rows/sec for us. Any other reason upsert > with limit is so slow? > > Thanks, > Shawn > > On Wed, Dec 19, 2018 at 5:04 PM Vincent Poon > wrote: > >> Shawn, >> >> Took a quick look, I think what is happening is the UPSERT is done >> serial

Re: "upsert select" with "limit" clause

2018-12-19 Thread Vincent Poon
at 2:31 PM Vincent Poon wrote: > > Shawn, that sounds like a bug, I would file a JIRA. > > On Tue, Dec 18, 2018 at 12:33 PM Shawn Li wrote: > >> Hi Vincent & William, >> >> >> >> Below is the explain plan, both are PARALLEL excuted in plan: >&

Re: "upsert select" with "limit" clause

2018-12-18 Thread Vincent Poon
| > > | CLIENT 27-CHUNK 3600 ROWS 48114000 BYTES PARALLEL 18-WAY ROUND > ROBIN FULL SCAN OVER table2 | > > | SERVER 200 ROW > LIMIT > | > > | CLIENT 200 ROW LIMIT > > > > > > > > Thanks, > &

Re: "upsert select" with "limit" clause

2018-12-18 Thread Vincent Poon
ck to the other question. Can you explain what is underlying Phoenix >> implementation for "upsert select limit"? Why it is slower than without >> "limit" when insert a huge number (2m rows) like ""upsert into table2 >> select * from table1 limit 2,000,

Re: "upsert select" with "limit" clause

2018-12-17 Thread Vincent Poon
e higher memory >>>> usage is caused by using scan cache and memstore under the higher >>>> throughput. >>>> >>>> >>>>Jaanai Zhang >>>>Best regards! >>>> >>>>

Re: "upsert select" with "limit" clause

2018-12-12 Thread Vincent Poon
( > > state CHAR(2) NOT NULL, > > city VARCHAR, > > population BIGINT, > > … > > CONSTRAINT my_pk PRIMARY KEY (state)); > > > > Thanks, > > Shawn > > > > On Wed, Dec 12, 2018, 13:42 Vincent Poon >> Fo

Re: "upsert select" with "limit" clause

2018-12-12 Thread Vincent Poon
For #2, can you provide the table definition and the statement used? e.g. Is the table immutable, or does it have indexes? On Tue, Dec 11, 2018 at 6:08 PM Shawn/Xiang Li wrote: > Hi, > > > > 1. Want to check what is underlying running for limit clause used > in the following Upsert statem

[ANNOUNCE] Apache Phoenix 4.14.1 released

2018-11-14 Thread Vincent Poon
The Apache Phoenix team is pleased to announce the immediate availability of the 4.14.1 patch release. Apache Phoenix enables SQL-based OLTP and operational analytics for Apache Hadoop using Apache HBase as its backing store and providing integration with other projects in the Apache ecosystem such

Re: ABORTING region server and following HBase cluster "crash"

2018-11-02 Thread Vincent Poon
Indexes in Phoenix should not in theory cause any cluster outage. An index write failure should just disable the index, not cause a crash. In practice, there have been some bugs around race conditions, the most dangerous of which accidentally trigger a KillServerOnFailurePolicy which then potentia

Re: ON DUPLICATE KEY with Global Index

2018-10-09 Thread Vincent Poon
We do need to update the docs after PHOENIX-3925, which changed the behavior from 'recommended' to 'mandatory'. I'll update the docs now. On Tue, Oct 9, 2018 at 1:08 PM Ankit Singhal wrote: > We do not allow atomic upsert and throw the corresponding exception in the > cases documented under the

Re: Table dead lock: ERROR 1120 (XCL20): Writes to table blocked until index can be updated

2018-09-26 Thread Vincent Poon
We are planning a Phoenix 4.14.1 release which will have this fix On Wed, Sep 26, 2018 at 3:36 PM Batyrshin Alexander <0x62...@gmail.com> wrote: > Thank you. We will try somehow... > Is there any chance that this fix will be included in next release for > HBASE-1.4 (not 2.0)? > > On 27 Sep 2018,