[jira] [Updated] (PHOENIX-4849) UPSERT SELECT fails with stale region boundary exception after a split

2018-09-23 Thread Lars Hofhansl (JIRA) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated PHOENIX-4849:
---
Attachment: PHOENIX-4849-all.patch

> UPSERT SELECT fails with stale region boundary exception after a split
> --
>
> Key: PHOENIX-4849
> URL: https://issues.apache.org/jira/browse/PHOENIX-4849
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Akshita Malhotra
>Assignee: Thomas D'Silva
>Priority: Critical
> Attachments: PHOENIX-4849-4.x-HBase-1.4.05.patch, 
> PHOENIX-4849-all.patch, PHOENIX-4849-complete-1.4.txt, PHOENIX-4849-fix.txt, 
> PHOENIX-4849-v2.patch, PHOENIX-4849-v3.patch, PHOENIX-4849-v4.patch, 
> PHOENIX-4849-v5.patch, PHOENIX-4849.patch, SerialIterators.diff, SplitIT.patch
>
>
> UPSERT SELECT throws a StaleRegionBoundaryCacheException immediately after a 
> split. On the other hand, an upsert followed by a select for example works 
> absolutely fine
> org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108 
> (XCL08): Cache of region boundaries are out of date.
> at 
> org.apache.phoenix.exception.SQLExceptionCode$14.newException(SQLExceptionCode.java:365)
>  at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:150)
>  at 
> org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:183)
>  at 
> org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:167)
>  at 
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:134)
>  at 
> org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:153)
>  at 
> org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:228)
>  at 
> org.apache.phoenix.iterate.LookAheadResultIterator$1.advance(LookAheadResultIterator.java:47)
>  at 
> org.apache.phoenix.iterate.LookAheadResultIterator.init(LookAheadResultIterator.java:59)
>  at 
> org.apache.phoenix.iterate.LookAheadResultIterator.peek(LookAheadResultIterator.java:73)
>  at 
> org.apache.phoenix.iterate.SerialIterators$SerialIterator.nextIterator(SerialIterators.java:187)
>  at 
> org.apache.phoenix.iterate.SerialIterators$SerialIterator.currentIterator(SerialIterators.java:160)
>  at 
> org.apache.phoenix.iterate.SerialIterators$SerialIterator.peek(SerialIterators.java:218)
>  at 
> org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:100)
>  at 
> org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
>  at 
> org.apache.phoenix.iterate.DelegateResultIterator.next(DelegateResultIterator.java:44)
>  at 
> org.apache.phoenix.iterate.LimitingResultIterator.next(LimitingResultIterator.java:47)
>  at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:805)
>  at 
> org.apache.phoenix.compile.UpsertCompiler.upsertSelect(UpsertCompiler.java:219)
>  at 
> org.apache.phoenix.compile.UpsertCompiler$ClientUpsertSelectMutationPlan.execute(UpsertCompiler.java:1292)
>  at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:408)
>  at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:391)
>  at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>  at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:390)
>  at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:378)
>  at 
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:173)
>  at 
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:183)
>  at 
> org.apache.phoenix.end2end.UpsertSelectAfterSplitTest.upsertSelectData1(UpsertSelectAfterSplitTest.java:109)
>  at 
> org.apache.phoenix.end2end.UpsertSelectAfterSplitTest.testUpsertSelect(UpsertSelectAfterSplitTest.java:59)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498)
>  at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>  at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>  at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>  at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>  at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>  at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>  at 
>

[jira] [Updated] (PHOENIX-4849) UPSERT SELECT fails with stale region boundary exception after a split

2018-09-23 Thread Lars Hofhansl (JIRA) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated PHOENIX-4849:
---
Affects Version/s: (was: 4.13.0)
   4.14.0

> UPSERT SELECT fails with stale region boundary exception after a split
> --
>
> Key: PHOENIX-4849
> URL: https://issues.apache.org/jira/browse/PHOENIX-4849
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Akshita Malhotra
>Assignee: Thomas D'Silva
>Priority: Critical
> Attachments: PHOENIX-4849-4.x-HBase-1.4.05.patch, 
> PHOENIX-4849-complete-1.4.txt, PHOENIX-4849-fix.txt, PHOENIX-4849-v2.patch, 
> PHOENIX-4849-v3.patch, PHOENIX-4849-v4.patch, PHOENIX-4849-v5.patch, 
> PHOENIX-4849.patch, SerialIterators.diff, SplitIT.patch
>
>
> UPSERT SELECT throws a StaleRegionBoundaryCacheException immediately after a 
> split. On the other hand, an upsert followed by a select for example works 
> absolutely fine
> org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108 
> (XCL08): Cache of region boundaries are out of date.
> at 
> org.apache.phoenix.exception.SQLExceptionCode$14.newException(SQLExceptionCode.java:365)
>  at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:150)
>  at 
> org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:183)
>  at 
> org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:167)
>  at 
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:134)
>  at 
> org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:153)
>  at 
> org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:228)
>  at 
> org.apache.phoenix.iterate.LookAheadResultIterator$1.advance(LookAheadResultIterator.java:47)
>  at 
> org.apache.phoenix.iterate.LookAheadResultIterator.init(LookAheadResultIterator.java:59)
>  at 
> org.apache.phoenix.iterate.LookAheadResultIterator.peek(LookAheadResultIterator.java:73)
>  at 
> org.apache.phoenix.iterate.SerialIterators$SerialIterator.nextIterator(SerialIterators.java:187)
>  at 
> org.apache.phoenix.iterate.SerialIterators$SerialIterator.currentIterator(SerialIterators.java:160)
>  at 
> org.apache.phoenix.iterate.SerialIterators$SerialIterator.peek(SerialIterators.java:218)
>  at 
> org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:100)
>  at 
> org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
>  at 
> org.apache.phoenix.iterate.DelegateResultIterator.next(DelegateResultIterator.java:44)
>  at 
> org.apache.phoenix.iterate.LimitingResultIterator.next(LimitingResultIterator.java:47)
>  at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:805)
>  at 
> org.apache.phoenix.compile.UpsertCompiler.upsertSelect(UpsertCompiler.java:219)
>  at 
> org.apache.phoenix.compile.UpsertCompiler$ClientUpsertSelectMutationPlan.execute(UpsertCompiler.java:1292)
>  at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:408)
>  at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:391)
>  at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>  at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:390)
>  at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:378)
>  at 
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:173)
>  at 
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:183)
>  at 
> org.apache.phoenix.end2end.UpsertSelectAfterSplitTest.upsertSelectData1(UpsertSelectAfterSplitTest.java:109)
>  at 
> org.apache.phoenix.end2end.UpsertSelectAfterSplitTest.testUpsertSelect(UpsertSelectAfterSplitTest.java:59)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498)
>  at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>  at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>  at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>  at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>  at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>  at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>  at 
>

[jira] [Updated] (PHOENIX-4849) UPSERT SELECT fails with stale region boundary exception after a split

2018-09-23 Thread Lars Hofhansl (JIRA) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated PHOENIX-4849:
---
Affects Version/s: 4.13.0

> UPSERT SELECT fails with stale region boundary exception after a split
> --
>
> Key: PHOENIX-4849
> URL: https://issues.apache.org/jira/browse/PHOENIX-4849
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.14.0
>Reporter: Akshita Malhotra
>Assignee: Thomas D'Silva
>Priority: Critical
> Attachments: PHOENIX-4849-4.x-HBase-1.4.05.patch, 
> PHOENIX-4849-complete-1.4.txt, PHOENIX-4849-fix.txt, PHOENIX-4849-v2.patch, 
> PHOENIX-4849-v3.patch, PHOENIX-4849-v4.patch, PHOENIX-4849-v5.patch, 
> PHOENIX-4849.patch, SerialIterators.diff, SplitIT.patch
>
>
> UPSERT SELECT throws a StaleRegionBoundaryCacheException immediately after a 
> split. On the other hand, an upsert followed by a select for example works 
> absolutely fine
> org.apache.phoenix.schema.StaleRegionBoundaryCacheException: ERROR 1108 
> (XCL08): Cache of region boundaries are out of date.
> at 
> org.apache.phoenix.exception.SQLExceptionCode$14.newException(SQLExceptionCode.java:365)
>  at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:150)
>  at 
> org.apache.phoenix.util.ServerUtil.parseRemoteException(ServerUtil.java:183)
>  at 
> org.apache.phoenix.util.ServerUtil.parseServerExceptionOrNull(ServerUtil.java:167)
>  at 
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:134)
>  at 
> org.apache.phoenix.iterate.ScanningResultIterator.next(ScanningResultIterator.java:153)
>  at 
> org.apache.phoenix.iterate.TableResultIterator.next(TableResultIterator.java:228)
>  at 
> org.apache.phoenix.iterate.LookAheadResultIterator$1.advance(LookAheadResultIterator.java:47)
>  at 
> org.apache.phoenix.iterate.LookAheadResultIterator.init(LookAheadResultIterator.java:59)
>  at 
> org.apache.phoenix.iterate.LookAheadResultIterator.peek(LookAheadResultIterator.java:73)
>  at 
> org.apache.phoenix.iterate.SerialIterators$SerialIterator.nextIterator(SerialIterators.java:187)
>  at 
> org.apache.phoenix.iterate.SerialIterators$SerialIterator.currentIterator(SerialIterators.java:160)
>  at 
> org.apache.phoenix.iterate.SerialIterators$SerialIterator.peek(SerialIterators.java:218)
>  at 
> org.apache.phoenix.iterate.ConcatResultIterator.currentIterator(ConcatResultIterator.java:100)
>  at 
> org.apache.phoenix.iterate.ConcatResultIterator.next(ConcatResultIterator.java:117)
>  at 
> org.apache.phoenix.iterate.DelegateResultIterator.next(DelegateResultIterator.java:44)
>  at 
> org.apache.phoenix.iterate.LimitingResultIterator.next(LimitingResultIterator.java:47)
>  at org.apache.phoenix.jdbc.PhoenixResultSet.next(PhoenixResultSet.java:805)
>  at 
> org.apache.phoenix.compile.UpsertCompiler.upsertSelect(UpsertCompiler.java:219)
>  at 
> org.apache.phoenix.compile.UpsertCompiler$ClientUpsertSelectMutationPlan.execute(UpsertCompiler.java:1292)
>  at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:408)
>  at org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:391)
>  at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>  at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:390)
>  at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:378)
>  at 
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:173)
>  at 
> org.apache.phoenix.jdbc.PhoenixPreparedStatement.execute(PhoenixPreparedStatement.java:183)
>  at 
> org.apache.phoenix.end2end.UpsertSelectAfterSplitTest.upsertSelectData1(UpsertSelectAfterSplitTest.java:109)
>  at 
> org.apache.phoenix.end2end.UpsertSelectAfterSplitTest.testUpsertSelect(UpsertSelectAfterSplitTest.java:59)
>  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>  at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498)
>  at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>  at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>  at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>  at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>  at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
>  at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
>  at 
> org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
>  at

[jira] [Assigned] (PHOENIX-4885) After HBASE-20940 any local index query will open all HFiles of every Region involved in the query

2018-09-23 Thread Lars Hofhansl (JIRA) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl reassigned PHOENIX-4885:
--

Assignee: Lars Hofhansl

> After HBASE-20940 any local index query will open all HFiles of every Region 
> involved in the query
> --
>
> Key: PHOENIX-4885
> URL: https://issues.apache.org/jira/browse/PHOENIX-4885
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
>Priority: Major
> Fix For: 4.15.0, 4.14.1, 5.1.0
>
> Attachments: PHOENIX-4885-4.x-HBase-1.4.01.patch, 
> PHOENIX-4885-master.01.patch, PHOENIX-4885-quickfix.txt
>
>
> See HBASE-20940.
> [~vishk], [~apurtell]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4885) After HBASE-20940 any local index query will open all HFiles of every Region involved in the query

2018-09-23 Thread Lars Hofhansl (JIRA) 


 [ 
https://issues.apache.org/jira/browse/PHOENIX-4885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated PHOENIX-4885:
---
Attachment: PHOENIX-4885-master.01.patch

> After HBASE-20940 any local index query will open all HFiles of every Region 
> involved in the query
> --
>
> Key: PHOENIX-4885
> URL: https://issues.apache.org/jira/browse/PHOENIX-4885
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Lars Hofhansl
>Priority: Major
> Attachments: PHOENIX-4885-4.x-HBase-1.4.01.patch, 
> PHOENIX-4885-master.01.patch, PHOENIX-4885-quickfix.txt
>
>
> See HBASE-20940.
> [~vishk], [~apurtell]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [DISCUSS] Suggestions for Phoenix from HBaseCon Asia notes

2018-09-23 Thread la...@apache.org 
 100% agreement.
A bit worried about "boiling the ocean" and risking not getting done anything.
Speaking of modules. I would *love* if we had a simple HBase abstraction API 
and then a module for each version of HBase, rather than a different branch 
each.Most differences are presumably in coprocessors APIs, which should be able 
to be "wrapped away" with some indirection layer.

-- Lars

On Monday, September 17, 2018, 8:52:58 AM PDT, Josh Elser 
 wrote:  
 
 Maybe an implementation detail, but I'm a fan of having a devoted Maven 
module to "client-facing" API as opposed to an annotation-based 
approach. I find a separate module helps to catch problematic API design 
faster, and make it crystal clear what users should (and should not) be 
relying upon).

On 9/17/18 1:00 AM, la...@apache.org wrote:
>  I think we can start by implementing a tighter integration with Spark 
>through DataSource V2.That would make it quickly apparent what parts of 
>Phoenix would need direct access.
> Some parts just need a interface audience declaration (like Phoenix's basic 
> type system) and our agreement that we will change those only according to 
> semantic versioning. Otherwise (like the query plan) will need a bit more 
> thinking. Maybe that's the path to hook Calcite - just making that part up as 
> I write this...
> Perhaps turning the HBase interface into an API might not be so difficult 
> either. That would perhaps be a new client - strictly additional - client API.
> 
> A good Spark interface is in everybody's interest and I think is the best 
> avenue to figure out what's missing/needed.
> -- Lars
> 
>      On Wednesday, September 12, 2018, 12:47:21 PM PDT, Josh Elser 
> wrote:
>  
>  I like it, Lars. I like it very much.
> 
> Just the easy part of doing it... ;)
> 
> On 9/11/18 4:53 PM, la...@apache.org wrote:
>>    Sorry for coming a bit late to this. I've been thinking about some of 
>>lines for a bit.
>> It seems Phoenix serves 4 distinct purposes:
>> 1. Query parsing and compiling.2. A type system3. Query execution4. 
>> Efficient HBase interface
>> Each of these is useful by itself, but we do not expose these as stable 
>> interfaces.We have seen a lot of need to tie HBase into "higher level" 
>> service, such as Spark (and Presto, etc).
>> I think we can get a long way if we separate at least #1 (SQL) from the rest 
>> #2, #3, and #4 (Typed HBase Interface - THI).
>> Phoenix is used via SQL (#1), other tools such as Presto, Impala, Drill, 
>> Spark, etc, can interface efficiently with HBase via THI (#2, #3, and #4).
>> Thoughts?
>> -- Lars
>>        On Monday, August 27, 2018, 11:03:33 AM PDT, Josh Elser 
>> wrote:
>>    
>>    (bcc: dev@hbase, in case folks there have been waiting for me to send
>> this email to dev@phoenix)
>>
>> Hi,
>>
>> In case you missed it, there was an HBaseCon event held in Asia
>> recently. Stack took some great notes and shared them with the HBase
>> community. A few of them touched on Phoenix, directly or in a related
>> manner. I think they are good "criticisms" that are beneficial for us to
>> hear.
>>
>> 1. The phoenix-$version-client.jar size is prohibitively large
>>
>> In this day and age, I'm surprised that this is a big issue for people.
>> I know have a lot of cruft, most of which coming from hadoop. We have
>> gotten better here over recent releases, but I would guess that there is
>> more we can do.
>>
>> 2. Can Phoenix be the de-facto schema for SQL on HBase?
>>
>> We've long asserted "if you have to ask how Phoenix serializes data, you
>> shouldn't be do it" (a nod that you have to write lots of code). What if
>> we turn that on its head? Could we extract our PDataType serialization,
>> composite row-key, column encoding, etc into a minimal API that folks
>> with their own itches can use?
>>
>> With the growing integrations into Phoenix, we could embrace them by
>> providing an API to make what they're doing easier. In the same vein, we
>> cement ourselves as a cornerstone of doing it "correctly".
>>
>> 3. Better recommendations to users to not attempt certain queries.
>>
>> We definitively know that there are certain types of queries that
>> Phoenix cannot support well (compared to optimal Phoenix use-cases).
>> Users very commonly fall into such pitfalls on their own and this leaves
>> a bad taste in their mouth (thinking that the product "stinks").
>>
>> Can we do a better job of telling the user when and why it happened?
>> What would such a user-interaction model look like? Can we supplement
>> the "why" with instructions of what to do differently (even if in the
>> abstract)?
>>
>> 4. Phoenix-Calcite
>>
>> This was mentioned as a "nice to have". From what I understand, there
>> was nothing explicitly from with the implementation or approach, just
>> that it was a massive undertaking to continue with little immediate
>> gain. Would this be a boon for us to try to continue in some form? Are
>> there steps we can take that would help push us along the

Re: [DISCUSS] Suggestions for Phoenix from HBaseCon Asia notes

2018-09-23 Thread la...@apache.org 
I will point out that the thin client can have advantages too:1. The query 
engine pool can be sized independently of clients and region servers now.2. 
Appropriate machine/VM configurations can be picked that are optimal for query 
execution.3. Client and (region) servers can be upgraded entirely independently.
As Josh pointed out... As HBase commiter and PMC member I am unaware of any 
"native SQL" aspirations for HBase.

   On Wednesday, September 19, 2018, 11:07:09 AM PDT, Josh Elser 
 wrote:  
 
 On 9/18/18 12:08 PM, Jaanai Zhang wrote:
>>
>> I don't understand what performance issues you think exist based solely on
>> the above. Those numbers appear to be precisely in line with my
>> expectations. Can you please describe what issues you think exist?
>>
> 
> 1. Performance of the thick client has almost 1~4 time higher than the thin
> client, the performance of the thin client will be decreased when the
> number of concurrencies is increased.  For some applications of the web
> server, this is not enough.
> 2. An HA thin client.
> 3. SQL audit function
> 
> A lot of developer like using the thin client, which has a lower
> maintenance cost on the client.Sorry, that's all that comes to me. :)

The thin-client is always doing more work to execute the same query as 
the thick-client (shipping results to/from PQS), so it shouldn't be 
surprising that the thin-client is slower. This is the trade-off to do 
less in the client and also make a well-defined API for other language 
to talk to PQS.

Out of curiosity, did you increase the JVM heap for PQS or increase 
configuration property defaults for PQS to account for the increased 
concurrency?

> Please be more specific. Asking for "more documentation" doesn't help us
>> actually turn this around into more documentation. What are the specific
>> pain points you have experienced? What topics do you want to know more
>> about? Be as specific as possible.
>>
> 
> About documents:
> 1. I think that we cloud add documents about migrate tools and migrate
> cases since many users migrate from RDBMS(MYSQL/PG/SQL SERVER) to Phoenix
> for some applications of non-transaction.
> 2. How to design PK or indexes.
> 
> About pain points:
> The stability is a big problem. Most of the people use Phoenix as a common
> RDBMS, they are informal to execute a query, even if they don't know why
> server crash when a scan full table was executed, so define use boundary of
> Phoenix is important that rejects some query and reports it user's client.

A migration document would be a great. Something that can supplement the 
existing "Quick Start" document.

What kind of points would you want to have centralized about designing 
PK's or indexes?

> Are you referring to the hbase-spark (and thus, Spark SQL) integration? Or
>> something that some company is building?
>>
> 
> Some companies are building with SPARK SQL to access Phoenix to support
> OLAP and OLTP requirements. it will produce heavily load for HBase cluster
> when Spark reads Phoenix tables,  my co-workers want to directly read
> HFiles of Phoenix tables for some offline business, but that depends on
> more flexible Phoenix API.

Just beware in calling "HBase native sql" as this implies that this is 
something that is a part of Apache HBase (which is not the case).

I doubt anyone in the Phoenix community would take offense to saying 
that a basic read/write SQL-esque language on top of HBase would be much 
more simple/faster than Phoenix is now. The value that Phoenix provides 
is a _robust_ SQL implementation and a consistent secondary indexing 
support. Going beyond a "sql skin" and implementing a database 
management system is where Phoenix excels above the rest.

> Uh, I also got some feedback that some features import for users, For
> example, "alter table modify column" can avoid reloaded data again which is
> expensive operate for the massive data table. I had upload patches to JIRA(
> PHOENIX-4815 ), but
> nobody responds to me  :(.

You should already know that we're all volunteers here, with our own 
responsibilities. You can ask for assistance/help in reviews, but, as 
always, be respectful of everyone's time. This goes for code reviews, as 
well as new documentation.

> Now,  i devote to develop chaos test and PQS stability(it was developed on
> the branch of my company, these patches will contribute to the community
> after stable running ),  if you have any suggests, please tell to me what
> you thinking. I would appreciate your reply.

Would be happy to see what you create.