[jira] [Commented] (PHOENIX-3189) HBase/ZooKeeper connection leaks when providing principal/keytab in JDBC url

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451082#comment-15451082
 ] 

ASF GitHub Bot commented on PHOENIX-3189:
-

Github user joshelser commented on the issue:

https://github.com/apache/phoenix/pull/191
  
> Thanks for pursuing this tricky issue, @joshelser. I think what you have 
here is definitely an improvement and should be pulled in for 4.8.1

Thanks as always for your careful eye.

> but I do think we should look at doing a value-based equality check for 
User instead. There's a fair amount of overhead in what you're doing and 
Phoenix does not do connection pooling but relies on being able to 
quickly/cheaply get a connection. Would you have some cycles to investigate 
that a bit first?

To my point in the last comment, I believe it's become apparent that people 
are misusing this functionality. So, I believe there is some education that 
needs to happen about how clients of Phoenix who need to support multiple 
Kerberos users concurrently should interact with the system. I think this 
documentation is point 1. It flat-out shouldn't be used to concurrently support 
multiple Kerberos users -- developers must write their own code to do this 
correctly. It's difficult to handle this for them because of the required 
doAs() call. Can investigate this further too, maybe I'm overlooking something 
obvious.

I do need to make good on my promise to dig through Hadoop/UGI git-log and 
try to piece together why exactly UGI isn't doing value-based checks already. 
This is point 2.

I'd like to move ahead and push then (rather then let it rot on the vine), 
but will try to address both of these points this week if that's OK with you.


> HBase/ZooKeeper connection leaks when providing principal/keytab in JDBC url
> 
>
> Key: PHOENIX-3189
> URL: https://issues.apache.org/jira/browse/PHOENIX-3189
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Blocker
> Fix For: 4.9.0, 4.8.1
>
>
> We've been doing some more testing after PHOENIX-3126 and, with the help of 
> [~arpitgupta] and [~harsha_ch], we've found an issue in a test between Storm 
> and Phoenix.
> Storm was configured to create a JDBC Bolt, specifying the principal and 
> keytab in the JDBC URL, relying on PhoenixDriver to do the Kerberos login for 
> them. After PHOENIX-3126, a ZK server blacklisted the host running the bolt, 
> and we observed that there were over 140 active ZK threads in the JVM.
> This results in a subtle change where every time the client tries to get a 
> new Connection, we end up getting a new UGI instance (because the 
> {{ConnectionQueryServicesImpl#openConnection()}} always does a new login).
> If users are correctly caching Connections, there isn't an issue (best as I 
> can presently tell). However, if users rely on the getting the same 
> connection every time (the pre-PHOENIX-3126), they will saturate their local 
> JVM with connections and crash.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] phoenix issue #191: PHOENIX-3189 Perform Kerberos login before ConnectionInf...

[joshelser]

Github user joshelser commented on the issue:

https://github.com/apache/phoenix/pull/191
  
> Thanks for pursuing this tricky issue, @joshelser. I think what you have 
here is definitely an improvement and should be pulled in for 4.8.1

Thanks as always for your careful eye.

> but I do think we should look at doing a value-based equality check for 
User instead. There's a fair amount of overhead in what you're doing and 
Phoenix does not do connection pooling but relies on being able to 
quickly/cheaply get a connection. Would you have some cycles to investigate 
that a bit first?

To my point in the last comment, I believe it's become apparent that people 
are misusing this functionality. So, I believe there is some education that 
needs to happen about how clients of Phoenix who need to support multiple 
Kerberos users concurrently should interact with the system. I think this 
documentation is point 1. It flat-out shouldn't be used to concurrently support 
multiple Kerberos users -- developers must write their own code to do this 
correctly. It's difficult to handle this for them because of the required 
doAs() call. Can investigate this further too, maybe I'm overlooking something 
obvious.

I do need to make good on my promise to dig through Hadoop/UGI git-log and 
try to piece together why exactly UGI isn't doing value-based checks already. 
This is point 2.

I'd like to move ahead and push then (rather then let it rot on the vine), 
but will try to address both of these points this week if that's OK with you.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (PHOENIX-3189) HBase/ZooKeeper connection leaks when providing principal/keytab in JDBC url

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451068#comment-15451068
 ] 

ASF GitHub Bot commented on PHOENIX-3189:
-

Github user joshelser commented on the issue:

https://github.com/apache/phoenix/pull/191
  
> Did you get to look at the comment regarding the user login ?

Assuming you mean the below..

> With that said I am not sure that you can support multiple users and 
support renewals with the way the UGI works.
> If in the same JVM a driver is instantiated for User A and then another 
driver is instantiated for User B the last call to loginUserFromKeytab will set 
the the user information in the UGI.

Yes, the best way to approach this is that Phoenix's automatic Kerberos 
login should _only_ be used when there a single user accessing Phoenix at one 
time. I was thinking that Phoenix could likely use its own docs page on the 
interactions with Kerberos. That would be a good place to warn people and 
instruct them how they need to properly support multiple concurrent users on 
their own.

> Can you allow another to login with a different principal? Would that 
cause a security issue?
If we create one driver(One) with user A and then create another 
driver(Two) with user B the info in the UGI now is that of user B. So there can 
be a situation where driver One will be using credentials of user B.

My thinking is this (but I should probably test it): says userA uses 
phoenix, then userB starts using Phoenix. This implies that userA would be 
logged out and I would hope that subsequent interactions with Phoenix by the 
"human" userA would fail because its credentials are no longer there (the RPCs 
themselves would fail). In other words, my thinking is that the RPC 
authentication would protect us in Phoenix from this being an issue.


> HBase/ZooKeeper connection leaks when providing principal/keytab in JDBC url
> 
>
> Key: PHOENIX-3189
> URL: https://issues.apache.org/jira/browse/PHOENIX-3189
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Josh Elser
>Assignee: Josh Elser
>Priority: Blocker
> Fix For: 4.9.0, 4.8.1
>
>
> We've been doing some more testing after PHOENIX-3126 and, with the help of 
> [~arpitgupta] and [~harsha_ch], we've found an issue in a test between Storm 
> and Phoenix.
> Storm was configured to create a JDBC Bolt, specifying the principal and 
> keytab in the JDBC URL, relying on PhoenixDriver to do the Kerberos login for 
> them. After PHOENIX-3126, a ZK server blacklisted the host running the bolt, 
> and we observed that there were over 140 active ZK threads in the JVM.
> This results in a subtle change where every time the client tries to get a 
> new Connection, we end up getting a new UGI instance (because the 
> {{ConnectionQueryServicesImpl#openConnection()}} always does a new login).
> If users are correctly caching Connections, there isn't an issue (best as I 
> can presently tell). However, if users rely on the getting the same 
> connection every time (the pre-PHOENIX-3126), they will saturate their local 
> JVM with connections and crash.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] phoenix issue #191: PHOENIX-3189 Perform Kerberos login before ConnectionInf...

[joshelser]

Github user joshelser commented on the issue:

https://github.com/apache/phoenix/pull/191
  
> Did you get to look at the comment regarding the user login ?

Assuming you mean the below..

> With that said I am not sure that you can support multiple users and 
support renewals with the way the UGI works.
> If in the same JVM a driver is instantiated for User A and then another 
driver is instantiated for User B the last call to loginUserFromKeytab will set 
the the user information in the UGI.

Yes, the best way to approach this is that Phoenix's automatic Kerberos 
login should _only_ be used when there a single user accessing Phoenix at one 
time. I was thinking that Phoenix could likely use its own docs page on the 
interactions with Kerberos. That would be a good place to warn people and 
instruct them how they need to properly support multiple concurrent users on 
their own.

> Can you allow another to login with a different principal? Would that 
cause a security issue?
If we create one driver(One) with user A and then create another 
driver(Two) with user B the info in the UGI now is that of user B. So there can 
be a situation where driver One will be using credentials of user B.

My thinking is this (but I should probably test it): says userA uses 
phoenix, then userB starts using Phoenix. This implies that userA would be 
logged out and I would hope that subsequent interactions with Phoenix by the 
"human" userA would fail because its credentials are no longer there (the RPCs 
themselves would fail). In other words, my thinking is that the RPC 
authentication would protect us in Phoenix from this being an issue.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (PHOENIX-3218) First draft of Phoenix Tuning Guide

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15451055#comment-15451055
 ] 

Josh Elser commented on PHOENIX-3218:
-

Would it make sense to try to move all of the JVM and Operating-System config 
tunings (e.g. Nagle's, THP, garbage collection) into the HBase book and avoid 
trying to exhaustively cover that all here in Phoenix?

There are lots of nice things covered at the Phoenix level 
(configuration/implementation and data modeling) which are great here. I would 
want to detract from that.

> First draft of Phoenix Tuning Guide
> ---
>
> Key: PHOENIX-3218
> URL: https://issues.apache.org/jira/browse/PHOENIX-3218
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Peter Conrad
> Attachments: Phoenix-Tuning-Guide.md
>
>
> Here's a first draft of a Tuning Guide for Phoenix performance. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[Mujtaba Chohan (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450622#comment-15450622
 ] 

Mujtaba Chohan commented on PHOENIX-3228:
-

Yes it was originally set as index showed poor performance when data was not 
filtered on index key since index would span on fewer regions/region servers 
compared to base table otherwise. 

> Index table are configured with a custom/smaller MAX_FILESIZE
> -
>
> Key: PHOENIX-3228
> URL: https://issues.apache.org/jira/browse/PHOENIX-3228
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 4.9.0, 4.8.1
>
> Attachments: 3228-remove.txt, 3228.txt
>
>
> I do not think we should do this. The default of 10G is chosen to keep HBase 
> happy. For smaller tables or initially until the index gets large it might 
> lead to more index regions and hence be able to utilize more region server, 
> but generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3223) Add hadoop classpath to PQS classpath

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450598#comment-15450598
 ] 

Hudson commented on PHOENIX-3223:
-

FAILURE: Integrated in Jenkins build Phoenix-4.8-HBase-1.2 #2 (See 
[https://builds.apache.org/job/Phoenix-4.8-HBase-1.2/2/])
PHOENIX-3223 Add `hadoop classpath` to PQS classpath when available (elserj: 
rev 008883a5805c117fbd2209d911b336b44109eb9c)
* (edit) bin/queryserver.py


> Add hadoop classpath to PQS classpath
> -
>
> Key: PHOENIX-3223
> URL: https://issues.apache.org/jira/browse/PHOENIX-3223
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Sergio Peleato
>Assignee: Josh Elser
> Fix For: 4.9.0, 4.8.1
>
>
> Follow-on from PHOENIX-2369: running on Microsoft's Azure Data Lake Store 
> nets some ClassNotFoundExceptions on the corresponding classes. The library 
> is available via the Hadoop installation, but we miss it because PQS isn't 
> adding the {{hadoop classpath}} like {{sqlline.py}} is (only 
> {{$HADOOP_CONF_DIR}}, oops).
> Alicia's patch on PHOENIX-2369 already did the calculation work to get the 
> classpath; just need to simply add it to PQS' classpath now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450589#comment-15450589
 ] 

Lars Hofhansl commented on PHOENIX-3228:


Yeah... Will run the tests... Didn't even compile, yet :)
I'd expect that with small data size (but big enough to reach the smaller max 
size) you'd see the index table distributed over fewer regions. But that 
argument doesn't hold as then we'd also have to apply the same logic to the 
main table - something more smaller region might better in certain 
circumstances.

Perhaps the angst stems from the fact that a full scan over the index table 
could be slower than a full scan over the main table, since that table is 
probably spread over slightly more regions...?

> Index table are configured with a custom/smaller MAX_FILESIZE
> -
>
> Key: PHOENIX-3228
> URL: https://issues.apache.org/jira/browse/PHOENIX-3228
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 4.9.0, 4.8.1
>
> Attachments: 3228-remove.txt, 3228.txt
>
>
> I do not think we should do this. The default of 10G is chosen to keep HBase 
> happy. For smaller tables or initially until the index gets large it might 
> lead to more index regions and hence be able to utilize more region server, 
> but generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3223) Add hadoop classpath to PQS classpath

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450579#comment-15450579
 ] 

Hudson commented on PHOENIX-3223:
-

FAILURE: Integrated in Jenkins build Phoenix-master #1376 (See 
[https://builds.apache.org/job/Phoenix-master/1376/])
PHOENIX-3223 Add `hadoop classpath` to PQS classpath when available (elserj: 
rev 93a9c9187f63578b9da9864e06c0df3231c9f61c)
* (edit) bin/queryserver.py


> Add hadoop classpath to PQS classpath
> -
>
> Key: PHOENIX-3223
> URL: https://issues.apache.org/jira/browse/PHOENIX-3223
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Sergio Peleato
>Assignee: Josh Elser
> Fix For: 4.9.0, 4.8.1
>
>
> Follow-on from PHOENIX-2369: running on Microsoft's Azure Data Lake Store 
> nets some ClassNotFoundExceptions on the corresponding classes. The library 
> is available via the Hadoop installation, but we miss it because PQS isn't 
> adding the {{hadoop classpath}} like {{sqlline.py}} is (only 
> {{$HADOOP_CONF_DIR}}, oops).
> Alicia's patch on PHOENIX-2369 already did the calculation work to get the 
> classpath; just need to simply add it to PQS' classpath now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450542#comment-15450542
 ] 

James Taylor commented on PHOENIX-3228:
---

Can you do a test run too to make sure you're not breaking any tests, 
[~lhofhansl]?

> Index table are configured with a custom/smaller MAX_FILESIZE
> -
>
> Key: PHOENIX-3228
> URL: https://issues.apache.org/jira/browse/PHOENIX-3228
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 4.9.0, 4.8.1
>
> Attachments: 3228-remove.txt, 3228.txt
>
>
> I do not think we should do this. The default of 10G is chosen to keep HBase 
> happy. For smaller tables or initially until the index gets large it might 
> lead to more index regions and hence be able to utilize more region server, 
> but generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450533#comment-15450533
 ] 

James Taylor commented on PHOENIX-3228:
---

If this is causing problems (or even not helping), then I'm fine with removing 
it all. [~mujtabachohan]? Any memories about particular queries/use cases that 
require this?

> Index table are configured with a custom/smaller MAX_FILESIZE
> -
>
> Key: PHOENIX-3228
> URL: https://issues.apache.org/jira/browse/PHOENIX-3228
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 4.9.0, 4.8.1
>
> Attachments: 3228-remove.txt, 3228.txt
>
>
> I do not think we should do this. The default of 10G is chosen to keep HBase 
> happy. For smaller tables or initially until the index gets large it might 
> lead to more index regions and hence be able to utilize more region server, 
> but generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3218) First draft of Phoenix Tuning Guide

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450527#comment-15450527
 ] 

James Taylor commented on PHOENIX-3218:
---

Thanks for putting this together, [~pconrad]. I think it'd be useful to have a 
single guide for tuning that has pointers out to other reference material, as 
it's difficult for users to know where to look for this information (since it's 
scattered throughout). This is a good first step.

Here's some specific feedback:
- Given that primary key design is the most important factor, I think the guide 
should lead with that. Might need to be expanded too - there've been several 
really good presentations in the past on this (i.e. [~ivarley] did one a few 
years back).
{code}
## Primary Keys

The underlying row key design is the single most important factor in Phoenix 
performance, and it's important to get it right at design time because you 
can't patch it later.
{code}
- To help users understand this, I think one or more examples or use cases 
should be used throughout the guide. This will help users relate more to the 
information presented and see what changes need to be made for each topic.
- For the write-heavy section, you mention using HBase or Phoenix compression. 
Minor nit, but Phoenix provides no compression - we get that goodness directly 
from HBase. Also, compression is good across the board, both for reading and 
writing, because it reduces disk IO. I'd recommend calling that out independent 
of this section (maybe you already do?).
{code}
* For write-heavy data:
* Turn on HBase or Phoenix compression. 
{code}
- In the Improving parallelization section, can you add a link to 
https://phoenix.apache.org/update_statistics.html
{code}
## Improving parallelization

Given a region, what HBase knows is the start key and the key in the next 
region (the end key kind of)
{code}
- In general, more links into the HBase Reference Guide would be useful: 
https://hbase.apache.org/book.html

> First draft of Phoenix Tuning Guide
> ---
>
> Key: PHOENIX-3218
> URL: https://issues.apache.org/jira/browse/PHOENIX-3218
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Peter Conrad
> Attachments: Phoenix-Tuning-Guide.md
>
>
> Here's a first draft of a Tuning Guide for Phoenix performance. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[Lars Hofhansl (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated PHOENIX-3228:
---
Attachment: 3228-remove.txt

FWIW, here's a patch that removes everything.

> Index table are configured with a custom/smaller MAX_FILESIZE
> -
>
> Key: PHOENIX-3228
> URL: https://issues.apache.org/jira/browse/PHOENIX-3228
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 4.9.0, 4.8.1
>
> Attachments: 3228-remove.txt, 3228.txt
>
>
> I do not think we should do this. The default of 10G is chosen to keep HBase 
> happy. For smaller tables or initially until the index gets large it might 
> lead to more index regions and hence be able to utilize more region server, 
> but generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[Lars Hofhansl (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated PHOENIX-3228:
---
Attachment: (was: 3228-remove.txt)

> Index table are configured with a custom/smaller MAX_FILESIZE
> -
>
> Key: PHOENIX-3228
> URL: https://issues.apache.org/jira/browse/PHOENIX-3228
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 4.9.0, 4.8.1
>
> Attachments: 3228-remove.txt, 3228.txt
>
>
> I do not think we should do this. The default of 10G is chosen to keep HBase 
> happy. For smaller tables or initially until the index gets large it might 
> lead to more index regions and hence be able to utilize more region server, 
> but generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450500#comment-15450500
 ] 

Lars Hofhansl commented on PHOENIX-3228:


Are you OK with just removing this logic? [~giacomotaylor]

> Index table are configured with a custom/smaller MAX_FILESIZE
> -
>
> Key: PHOENIX-3228
> URL: https://issues.apache.org/jira/browse/PHOENIX-3228
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 4.9.0, 4.8.1
>
> Attachments: 3228-remove.txt, 3228.txt
>
>
> I do not think we should do this. The default of 10G is chosen to keep HBase 
> happy. For smaller tables or initially until the index gets large it might 
> lead to more index regions and hence be able to utilize more region server, 
> but generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (PHOENIX-3217) Phoenix sets MAX_FILESIZE table attribute for index tables

[Andrew Purtell (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Purtell resolved PHOENIX-3217.
-
Resolution: Duplicate

Dup of PHOENIX-3228

> Phoenix sets MAX_FILESIZE table attribute for index tables
> --
>
> Key: PHOENIX-3217
> URL: https://issues.apache.org/jira/browse/PHOENIX-3217
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Andrew Purtell
>
> Phoenix appears to set the HBase table attribute MAX_FILESIZE for index 
> tables. We should discuss this. This setting is a user tunable that impacts 
> splitting decisions. It should be set in conjunction with a split policy not 
> in isolation. Is it necessary to do this for index tables? Because this is an 
> important user tunable, overriding the value is likely to lead to surprise 
> and unexpected cluster behavior. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450492#comment-15450492
 ] 

James Taylor commented on PHOENIX-3228:
---

There's also this code in MetaDataClient that sets the MAX_FILESIZE for an 
index, based on the ratio of the index row size versus data row size:
{code}
} else if (tableType == PTableType.INDEX && indexId == null) {
if (tableProps.get(HTableDescriptor.MAX_FILESIZE) == null) {
int nIndexRowKeyColumns = isPK ? 1 : pkColumnsNames.size();
int nIndexKeyValueColumns = columns.size() - 
nIndexRowKeyColumns;
int nBaseRowKeyColumns = parent.getPKColumns().size() - 
(parent.getBucketNum() == null ? 0 : 1);
int nBaseKeyValueColumns = parent.getColumns().size() - 
parent.getPKColumns().size();
/*
 * Approximate ratio between index table size and data 
table size:
 * More or less equal to the ratio between the number of 
key value columns in each. We add one to
 * the key value column count to take into account our 
empty key value. We add 1/4 for any key
 * value data table column that was moved into the index 
table row key.
 */
double ratio = (1+nIndexKeyValueColumns + 
(nIndexRowKeyColumns - nBaseRowKeyColumns)/4d)/(1+nBaseKeyValueColumns);
HTableDescriptor descriptor = 
connection.getQueryServices().getTableDescriptor(parent.getPhysicalName().getBytes());
if (descriptor != null) { // Is null for connectionless
long maxFileSize = descriptor.getMaxFileSize();
if (maxFileSize == -1) { // If unset, use default
maxFileSize = HConstants.DEFAULT_MAX_FILE_SIZE;
}
tableProps.put(HTableDescriptor.MAX_FILESIZE, 
(long)(maxFileSize * ratio));
}
}
{code}



> Index table are configured with a custom/smaller MAX_FILESIZE
> -
>
> Key: PHOENIX-3228
> URL: https://issues.apache.org/jira/browse/PHOENIX-3228
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>Assignee: Lars Hofhansl
> Fix For: 4.9.0, 4.8.1
>
> Attachments: 3228-remove.txt, 3228.txt
>
>
> I do not think we should do this. The default of 10G is chosen to keep HBase 
> happy. For smaller tables or initially until the index gets large it might 
> lead to more index regions and hence be able to utilize more region server, 
> but generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[Lars Hofhansl (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated PHOENIX-3228:
---
Fix Version/s: 4.8.1
   4.9.0

> Index table are configured with a custom/smaller MAX_FILESIZE
> -
>
> Key: PHOENIX-3228
> URL: https://issues.apache.org/jira/browse/PHOENIX-3228
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
> Fix For: 4.9.0, 4.8.1
>
> Attachments: 3228.txt
>
>
> I do not think we should do this. The default of 10G is chosen to keep HBase 
> happy. For smaller tables or initially until the index gets large it might 
> lead to more index regions and hence be able to utilize more region server, 
> but generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450468#comment-15450468
 ] 

Lars Hofhansl commented on PHOENIX-3228:


For 4.8.1 we should set the default to 100% (and not set the property on the 
table if set to 100), for 4.9.0 I propose we simply remove the logic.

> Index table are configured with a custom/smaller MAX_FILESIZE
> -
>
> Key: PHOENIX-3228
> URL: https://issues.apache.org/jira/browse/PHOENIX-3228
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>
> I do not think we should do this. The default of 10G is chosen to keep HBase 
> happy. For smaller tables or initially until the index gets large it might 
> lead to more index regions and hence be able to utilize more region server, 
> but generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3224) Observations from large scale testing.

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450454#comment-15450454
 ] 

James Taylor commented on PHOENIX-3224:
---

Would be good to get more info on bigger joins. Using the USE_SORT_MERGE_JOIN 
hint, you should be able to do larger joins and perf would be equivalent to 
doing a sort on each of the tables being joined. Can we try some smaller tests 
first with this hint to make sure there's not something else going on? Please 
let us know if we're missing anything, [~maryannxue].

> Observations from large scale testing.
> --
>
> Key: PHOENIX-3224
> URL: https://issues.apache.org/jira/browse/PHOENIX-3224
> Project: Phoenix
>  Issue Type: Task
>Reporter: Lars Hofhansl
>
> We have a >1000 node physical cluster at our disposal for a short time, 
> before it'll be handed off to its intended use.
> Loaded a bunch of data (TPCs LINEITEM table, among others) and ran a bunch of 
> queries. Most tables are between 100G and 500G (uncompressed) and between 
> 600m and 2bn rows.
> The good news is that many things just worked. We sorted > 400G is < 5s with 
> HBase and Phoenix. Scans work. Joins work (as long as one side is kept under 
> 1m rows or so).
> For the issues we observers I'll file sub jiras under this.
> I'm going to write a lob post about this and attach a link here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PHOENIX-3228) Index table are configured with a custom/smaller MAX_FILESIZE

[Lars Hofhansl (JIRA)]

Lars Hofhansl created PHOENIX-3228:
--

 Summary: Index table are configured with a custom/smaller 
MAX_FILESIZE
 Key: PHOENIX-3228
 URL: https://issues.apache.org/jira/browse/PHOENIX-3228
 Project: Phoenix
  Issue Type: Sub-task
Reporter: Lars Hofhansl


I do not think we should do this. The default of 10G is chosen to keep HBase 
happy. For smaller tables or initially until the index gets large it might lead 
to more index regions and hence be able to utilize more region server, but 
generally, this is not the right thing to do.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3227) OFFSET is very slow at scale

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450425#comment-15450425
 ] 

James Taylor commented on PHOENIX-3227:
---

Not sure if [~an...@apache.org] has anything in mind to improve it, but if not 
we should close it.

> OFFSET is very slow at scale
> 
>
> Key: PHOENIX-3227
> URL: https://issues.apache.org/jira/browse/PHOENIX-3227
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>
> Perhaps this one is also expected since it is the client who has to count off 
> rows. In any case, we tried with an OFFSET of just 1000 and that took 
> 46s, regardless of how big the cluster at our disposal was.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3223) Add hadoop classpath to PQS classpath

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450377#comment-15450377
 ] 

ASF GitHub Bot commented on PHOENIX-3223:
-

Github user asfgit closed the pull request at:

https://github.com/apache/phoenix/pull/205


> Add hadoop classpath to PQS classpath
> -
>
> Key: PHOENIX-3223
> URL: https://issues.apache.org/jira/browse/PHOENIX-3223
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Sergio Peleato
>Assignee: Josh Elser
> Fix For: 4.9.0, 4.8.1
>
>
> Follow-on from PHOENIX-2369: running on Microsoft's Azure Data Lake Store 
> nets some ClassNotFoundExceptions on the corresponding classes. The library 
> is available via the Hadoop installation, but we miss it because PQS isn't 
> adding the {{hadoop classpath}} like {{sqlline.py}} is (only 
> {{$HADOOP_CONF_DIR}}, oops).
> Alicia's patch on PHOENIX-2369 already did the calculation work to get the 
> classpath; just need to simply add it to PQS' classpath now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] phoenix pull request #205: PHOENIX-3223 Add `hadoop classpath` to PQS classp...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/phoenix/pull/205


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (PHOENIX-3223) Add hadoop classpath to PQS classpath

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450361#comment-15450361
 ] 

Josh Elser commented on PHOENIX-3223:
-

Thanks [~sergey.soldatov]! Will commit shortly. I've tested this locally and it 
worked as expected for me.

> Add hadoop classpath to PQS classpath
> -
>
> Key: PHOENIX-3223
> URL: https://issues.apache.org/jira/browse/PHOENIX-3223
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Sergio Peleato
>Assignee: Josh Elser
> Fix For: 4.9.0, 4.8.1
>
>
> Follow-on from PHOENIX-2369: running on Microsoft's Azure Data Lake Store 
> nets some ClassNotFoundExceptions on the corresponding classes. The library 
> is available via the Hadoop installation, but we miss it because PQS isn't 
> adding the {{hadoop classpath}} like {{sqlline.py}} is (only 
> {{$HADOOP_CONF_DIR}}, oops).
> Alicia's patch on PHOENIX-2369 already did the calculation work to get the 
> classpath; just need to simply add it to PQS' classpath now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3227) OFFSET is very slow at scale

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450321#comment-15450321
 ] 

Lars Hofhansl commented on PHOENIX-3227:


Should I close this one [~giacomotaylor]?

> OFFSET is very slow at scale
> 
>
> Key: PHOENIX-3227
> URL: https://issues.apache.org/jira/browse/PHOENIX-3227
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>
> Perhaps this one is also expected since it is the client who has to count off 
> rows. In any case, we tried with an OFFSET of just 1000 and that took 
> 46s, regardless of how big the cluster at our disposal was.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Phoenix Tuning Guide [First Draft]

[Peter Conrad]

All,

I've uploaded a draft of the Phoenix Tuning Guide as an attachment to the
following Jira:
https://issues.apache.org/jira/browse/PHOENIX-3218

I welcome any feedback or comments.

Thanks,
Peter


Peter Conrad

Staff Technical Writer: Infrastructure | salesforce.com

Office: (415) 471-5265


[image: http://www.salesforce.com/signature]


--

[jira] [Commented] (PHOENIX-2645) Wildcard characters do not match newline characters

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450239#comment-15450239
 ] 

Hudson commented on PHOENIX-2645:
-

FAILURE: Integrated in Jenkins build Phoenix-master #1375 (See 
[https://builds.apache.org/job/Phoenix-master/1375/])
PHOENIX-2645 Wildcard characters do not match newline characters (twdsilva: rev 
ede483cf29c86c1acebc12d995966a838e4eaffb)
* (edit) 
phoenix-core/src/test/java/org/apache/phoenix/expression/LikeExpressionTest.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/expression/function/ByteBasedRegexpSplitFunction.java
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/LikeExpressionIT.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/expression/ByteBasedLikeExpression.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/expression/StringBasedLikeExpression.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/expression/function/ByteBasedRegexpReplaceFunction.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/expression/function/ByteBasedRegexpSubstrFunction.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/expression/function/StringBasedRegexpReplaceFunction.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/expression/function/StringBasedRegexpSubstrFunction.java


> Wildcard characters do not match newline characters
> ---
>
> Key: PHOENIX-2645
> URL: https://issues.apache.org/jira/browse/PHOENIX-2645
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.7.0
> Environment: Phoenix 4.7.0 on Calcite 1.5
>Reporter: Kevin Liew
>Assignee: Kevin Liew
>  Labels: newbie
> Fix For: 4.9.0, 4.8.1
>
>
> Wildcard characters do not match newline characters
> {code:sql}
> 0: jdbc:phoenix:thin:url=http://localhost:876> create table testnewline (pk 
> varchar(10) primary key)
> . . . . . . . . . . . . . . . . . . . . . . .> ;
> No rows affected (2.643 seconds)
> 0: jdbc:phoenix:thin:url=http://localhost:876> upsert into testnewline values 
> ('AA\nA');
> 1 row affected (0.079 seconds)
> 0: jdbc:phoenix:thin:url=http://localhost:876> select * from testnewline 
> where pk like 'AA%'
> . . . . . . . . . . . . . . . . . . . . . . .> ;
> ++
> | PK |
> ++
> | AA
> A   |
> ++
> 1 row selected (0.086 seconds)
> 0: jdbc:phoenix:thin:url=http://localhost:876> select * from testnewline 
> where pk like 'AA_A';
> ++
> | PK |
> ++
> ++
> No rows selected (0.053 seconds)
> 0: jdbc:phoenix:thin:url=http://localhost:876> select * from testnewline 
> where pk like 'AA%A';
> ++
> | PK |
> ++
> ++
> No rows selected (0.032 seconds)
> 0: jdbc:phoenix:thin:url=http://localhost:876> 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3163) Split during global index creation may cause ERROR 201 error.

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450220#comment-15450220
 ] 

James Taylor commented on PHOENIX-3163:
---

I think the split detection code on the client side is too hard to follow, 
doesn't handle all cases, and is too brittle. I think it should be cleaned up:
- handle aggregation case. We can know what row key we've last processed and we 
have the current aggregation state. We should just rerun the query from the 
correct row key and initialize the aggregation state with the current state.
- handle local index case. Same as above - just throw and catch at the topmost 
level (PhoenixResultSet) and then rerun the query (canceling the existing one)
- add clear APIs to handle case where the query can continue running. If UPSERT 
SELECT is a special case, we shouldn't handle it this way.

> Split during global index creation may cause ERROR 201 error.
> -
>
> Key: PHOENIX-3163
> URL: https://issues.apache.org/jira/browse/PHOENIX-3163
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Sergey Soldatov
>Assignee: Sergey Soldatov
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3163_v1.patch, PHOENIX-3163_v3.patch, 
> PHOENIX-3163_v4.patch
>
>
> When we create global index and split happen meanwhile there is a chance to 
> fail with ERROR 201:
> {noformat}
> 2016-08-08 15:55:17,248 INFO  [Thread-6] 
> org.apache.phoenix.iterate.BaseResultIterators(878): Failed to execute task 
> during cancel
> java.util.concurrent.ExecutionException: java.sql.SQLException: ERROR 201 
> (22000): Illegal data.
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.close(BaseResultIterators.java:872)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:809)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:713)
>   at 
> org.apache.phoenix.iterate.RoundRobinResultIterator.getIterators(RoundRobinResultIterator.java:176)
>   at 
> org.apache.phoenix.iterate.RoundRobinResultIterator.next(RoundRobinResultIterator.java:91)
>   at 
> org.apache.phoenix.compile.UpsertCompiler$2.execute(UpsertCompiler.java:815)
>   at 
> org.apache.phoenix.compile.DelegateMutationPlan.execute(DelegateMutationPlan.java:31)
>   at 
> org.apache.phoenix.compile.PostIndexDDLCompiler$1.execute(PostIndexDDLCompiler.java:124)
>   at 
> org.apache.phoenix.query.ConnectionQueryServicesImpl.updateData(ConnectionQueryServicesImpl.java:2823)
>   at 
> org.apache.phoenix.schema.MetaDataClient.buildIndex(MetaDataClient.java:1079)
>   at 
> org.apache.phoenix.schema.MetaDataClient.createIndex(MetaDataClient.java:1382)
>   at 
> org.apache.phoenix.compile.CreateIndexCompiler$1.execute(CreateIndexCompiler.java:85)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:343)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:331)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:330)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1440)
>   at 
> org.apache.phoenix.hbase.index.write.TestIndexWriter$1.run(TestIndexWriter.java:93)
> Caused by: java.sql.SQLException: ERROR 201 (22000): Illegal data.
>   at 
> org.apache.phoenix.exception.SQLExceptionCode$Factory$1.newException(SQLExceptionCode.java:441)
>   at 
> org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145)
>   at 
> org.apache.phoenix.schema.types.PDataType.newIllegalDataException(PDataType.java:287)
>   at 
> org.apache.phoenix.schema.types.PUnsignedSmallint$UnsignedShortCodec.decodeShort(PUnsignedSmallint.java:146)
>   at 
> org.apache.phoenix.schema.types.PSmallint.toObject(PSmallint.java:104)
>   at org.apache.phoenix.schema.types.PSmallint.toObject(PSmallint.java:28)
>   at 
> org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:980)
>   at 
> org.apache.phoenix.schema.types.PUnsignedSmallint.toObject(PUnsignedSmallint.java:102)
>   at 
> org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:980)
>   at 
> org.apache.phoenix.schema.types.PDataType.toObject(PDataType.java:992)
>   at 
> org.apache.phoenix.schema.types.PDataType.coerceBytes(PDataType.java:830)
>   at 
> org.apache.phoenix.schema.types.PDecimal.coerceBytes(PDecimal.java:342)
>   at 
> 

[jira] [Commented] (PHOENIX-3227) OFFSET is very slow at scale

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450202#comment-15450202
 ] 

James Taylor commented on PHOENIX-3227:
---

[~an...@apache.org]

> OFFSET is very slow at scale
> 
>
> Key: PHOENIX-3227
> URL: https://issues.apache.org/jira/browse/PHOENIX-3227
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>
> Perhaps this one is also expected since it is the client who has to count off 
> rows. In any case, we tried with an OFFSET of just 1000 and that took 
> 46s, regardless of how big the cluster at our disposal was.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3227) OFFSET is very slow at scale

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450197#comment-15450197
 ] 

James Taylor commented on PHOENIX-3227:
---

Yes, that's what I expected. FYI, we do push the OFFSET to the server when ever 
possible, but you still have to step through row by row (except on the server 
and then at least you don't return all the data back to the client). Also, the 
slowness is due to the query needing to be executed serially.

> OFFSET is very slow at scale
> 
>
> Key: PHOENIX-3227
> URL: https://issues.apache.org/jira/browse/PHOENIX-3227
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>
> Perhaps this one is also expected since it is the client who has to count off 
> rows. In any case, we tried with an OFFSET of just 1000 and that took 
> 46s, regardless of how big the cluster at our disposal was.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3224) Observations from large scale testing.

[Lars Hofhansl (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated PHOENIX-3224:
---
Description: 
We have a >1000 node physical cluster at our disposal for a short time, before 
it'll be handed off to its intended use.

Loaded a bunch of data (TPCs LINEITEM table, among others) and ran a bunch of 
queries. Most tables are between 100G and 500G (uncompressed) and between 600m 
and 2bn rows.

The good news is that many things just worked. We sorted > 400G is < 5s with 
HBase and Phoenix. Scans work. Joins work (as long as one side is kept under 1m 
rows or so).

For the issues we observers I'll file sub jiras under this.

I'm going to write a lob post about this and attach a link here.

  was:
We have a >1000 node physical cluster at our disposal for a short time, before 
it'll be handed off to its intended use.

Loaded a bunch of data (TPCs LINEITEM table, among others) and ran a bunch of 
queries. Most tables are between 100G and 500G (uncompressed) and between 600m 
and 2bn rows.

The good news is that many things just worked. We sorted > 400G is < 5s with 
HBase and Phoenix. Scans work. Joins work (as long as one side is kept under 1m 
rows or so).

For the issues we observers I'll file sub jiras under this.


> Observations from large scale testing.
> --
>
> Key: PHOENIX-3224
> URL: https://issues.apache.org/jira/browse/PHOENIX-3224
> Project: Phoenix
>  Issue Type: Task
>Reporter: Lars Hofhansl
>
> We have a >1000 node physical cluster at our disposal for a short time, 
> before it'll be handed off to its intended use.
> Loaded a bunch of data (TPCs LINEITEM table, among others) and ran a bunch of 
> queries. Most tables are between 100G and 500G (uncompressed) and between 
> 600m and 2bn rows.
> The good news is that many things just worked. We sorted > 400G is < 5s with 
> HBase and Phoenix. Scans work. Joins work (as long as one side is kept under 
> 1m rows or so).
> For the issues we observers I'll file sub jiras under this.
> I'm going to write a lob post about this and attach a link here.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PHOENIX-3227) OFFSET is very slow at scale

[Lars Hofhansl (JIRA)]

Lars Hofhansl created PHOENIX-3227:
--

 Summary: OFFSET is very slow at scale
 Key: PHOENIX-3227
 URL: https://issues.apache.org/jira/browse/PHOENIX-3227
 Project: Phoenix
  Issue Type: Sub-task
Reporter: Lars Hofhansl


Perhaps this one is also expected since it is the client who has to count off 
rows. In any case, we tried with an OFFSET of just 1000 and that took 46s, 
regardless of how big the cluster at our disposal was.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PHOENIX-3226) Phoenix should have an option to fail a query that would ship large amounts of data to the client.

[Lars Hofhansl (JIRA)]

Lars Hofhansl created PHOENIX-3226:
--

 Summary: Phoenix should have an option to fail a query that would 
ship large amounts of data to the client.
 Key: PHOENIX-3226
 URL: https://issues.apache.org/jira/browse/PHOENIX-3226
 Project: Phoenix
  Issue Type: Sub-task
Reporter: Lars Hofhansl


For many queries (merge joins, union all, some distincts, etc) Phoenix needs to 
ship data to the client. It will happily try to do that even when that requires 
shipping terrabytes of data to the poor client to do its final merge sweep.

Phoenix should have an option to detect these cases (from the stats and the 
query plan), and simply - and optionally - fail the query immediately with 
"Query too complicated", or "Query too slow", or something.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3225) Distinct Queries are slower than expected at scale.

[Lars Hofhansl (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lars Hofhansl updated PHOENIX-3225:
---
Summary: Distinct Queries are slower than expected at scale.  (was: 
Distinct Queries are slower than expected)

> Distinct Queries are slower than expected at scale.
> ---
>
> Key: PHOENIX-3225
> URL: https://issues.apache.org/jira/browse/PHOENIX-3225
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: Lars Hofhansl
>
> In our large scale tests we found that we can easily sort 400G on a few 100 
> machines, but that a simple DISTINCT would just time out. Perhaps that's 
> expected as we have to keep track of the unique values, but we should 
> investigate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PHOENIX-3225) Distinct Queries are slower than expected

[Lars Hofhansl (JIRA)]

Lars Hofhansl created PHOENIX-3225:
--

 Summary: Distinct Queries are slower than expected
 Key: PHOENIX-3225
 URL: https://issues.apache.org/jira/browse/PHOENIX-3225
 Project: Phoenix
  Issue Type: Sub-task
Reporter: Lars Hofhansl


In our large scale tests we found that we can easily sort 400G on a few 100 
machines, but that a simple DISTINCT would just time out. Perhaps that's 
expected as we have to keep track of the unique values, but we should 
investigate.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3224) Observations from large scale testing.

[Lars Hofhansl (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450118#comment-15450118
 ] 

Lars Hofhansl commented on PHOENIX-3224:


[~giacomotaylor], [~mujtabachohan], FYI.

> Observations from large scale testing.
> --
>
> Key: PHOENIX-3224
> URL: https://issues.apache.org/jira/browse/PHOENIX-3224
> Project: Phoenix
>  Issue Type: Task
>Reporter: Lars Hofhansl
>
> We have a >1000 node physical cluster at our disposal for a short time, 
> before it'll be handed off to its intended use.
> Loaded a bunch of data (TPCs LINEITEM table, among others) and ran a bunch of 
> queries. Most tables are between 100G and 500G (uncompressed) and between 
> 600m and 2bn rows.
> The good news is that many things just worked. We sorted > 400G is < 5s with 
> HBase and Phoenix. Scans work. Joins work (as long as one side is kept under 
> 1m rows or so).
> For the issues we observers I'll file sub jiras under this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3216) Kerberos ticket is not renewed when using Kerberos authentication with Phoenix JDBC driver

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450106#comment-15450106
 ] 

ASF GitHub Bot commented on PHOENIX-3216:
-

Github user dbahir closed the pull request at:

https://github.com/apache/phoenix/pull/203


> Kerberos ticket is not renewed when using Kerberos authentication with 
> Phoenix JDBC driver
> --
>
> Key: PHOENIX-3216
> URL: https://issues.apache.org/jira/browse/PHOENIX-3216
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.4.0, 4.5.0, 4.5.1, 4.6.0, 4.5.2, 4.8.0
> Environment: Kerberized
>Reporter: Dan Bahir
>Assignee: Dan Bahir
> Fix For: 4.9.0, 4.8.1
>
>
> When using Phoenix jdbc driver in a Kerberized environment and logging in 
> with a keytab is not automatically renewed.
> Expected:The ticket will be automatically renewed and the Phoenix driver will 
> be able to write to the database.
> Actual: The ticket is not renewed and driver loses access to the database.
> 2016-08-15 00:00:59.738 WARN  AbstractRpcClient 
> [hconnection-0x4763c727-metaLookup-shared--pool1-t686] - Exception 
> encountered 
> while connecting to the server : javax.security.sasl.Sa
> slException: GSS initiate failed [Caused by GSSException: No valid 
> credentials 
> provided (Mechanism level: Failed to find any Kerberos tgt)]
> 2016-08-15 00:00:59.739 ERROR AbstractRpcClient 
> [hconnection-0x4763c727-metaLookup-shared--pool1-t686] - SASL authentication 
> failed. The most likely cause is missing or invalid crede
> ntials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by 
> GSSException: 
> No valid credentials provided (Mechanism level: Failed to find any Kerberos 
> tgt)]
> at 
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java
> :211)
> at 
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClie
> nt.java:179)
> at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClie
> ntImpl.java:611)
> at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.ja
> va:156)
> at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:73
> 7)
> at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:73
> 4)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.ja



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3216) Kerberos ticket is not renewed when using Kerberos authentication with Phoenix JDBC driver

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15450107#comment-15450107
 ] 

ASF GitHub Bot commented on PHOENIX-3216:
-

Github user dbahir commented on the issue:

https://github.com/apache/phoenix/pull/203
  
Closing, this issue will be fixed by 
[#191](https://github.com/apache/phoenix/pull/191)


> Kerberos ticket is not renewed when using Kerberos authentication with 
> Phoenix JDBC driver
> --
>
> Key: PHOENIX-3216
> URL: https://issues.apache.org/jira/browse/PHOENIX-3216
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.4.0, 4.5.0, 4.5.1, 4.6.0, 4.5.2, 4.8.0
> Environment: Kerberized
>Reporter: Dan Bahir
>Assignee: Dan Bahir
> Fix For: 4.9.0, 4.8.1
>
>
> When using Phoenix jdbc driver in a Kerberized environment and logging in 
> with a keytab is not automatically renewed.
> Expected:The ticket will be automatically renewed and the Phoenix driver will 
> be able to write to the database.
> Actual: The ticket is not renewed and driver loses access to the database.
> 2016-08-15 00:00:59.738 WARN  AbstractRpcClient 
> [hconnection-0x4763c727-metaLookup-shared--pool1-t686] - Exception 
> encountered 
> while connecting to the server : javax.security.sasl.Sa
> slException: GSS initiate failed [Caused by GSSException: No valid 
> credentials 
> provided (Mechanism level: Failed to find any Kerberos tgt)]
> 2016-08-15 00:00:59.739 ERROR AbstractRpcClient 
> [hconnection-0x4763c727-metaLookup-shared--pool1-t686] - SASL authentication 
> failed. The most likely cause is missing or invalid crede
> ntials. Consider 'kinit'.
> javax.security.sasl.SaslException: GSS initiate failed [Caused by 
> GSSException: 
> No valid credentials provided (Mechanism level: Failed to find any Kerberos 
> tgt)]
> at 
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java
> :211)
> at 
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClie
> nt.java:179)
> at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClie
> ntImpl.java:611)
> at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.ja
> va:156)
> at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:73
> 7)
> at 
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:73
> 4)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.ja



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] phoenix issue #203: [PHOENIX-3216] Kerberos ticket is not renewed when using...

[dbahir]

Github user dbahir commented on the issue:

https://github.com/apache/phoenix/pull/203
  
Closing, this issue will be fixed by 
[#191](https://github.com/apache/phoenix/pull/191)


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] phoenix pull request #203: [PHOENIX-3216] Kerberos ticket is not renewed whe...

[dbahir]

Github user dbahir closed the pull request at:

https://github.com/apache/phoenix/pull/203


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (PHOENIX-3223) Add hadoop classpath to PQS classpath

[Sergey Soldatov (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449996#comment-15449996
 ] 

Sergey Soldatov commented on PHOENIX-3223:
--

LGTM. [~elserj] nice work :)

> Add hadoop classpath to PQS classpath
> -
>
> Key: PHOENIX-3223
> URL: https://issues.apache.org/jira/browse/PHOENIX-3223
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Sergio Peleato
>Assignee: Josh Elser
> Fix For: 4.9.0, 4.8.1
>
>
> Follow-on from PHOENIX-2369: running on Microsoft's Azure Data Lake Store 
> nets some ClassNotFoundExceptions on the corresponding classes. The library 
> is available via the Hadoop installation, but we miss it because PQS isn't 
> adding the {{hadoop classpath}} like {{sqlline.py}} is (only 
> {{$HADOOP_CONF_DIR}}, oops).
> Alicia's patch on PHOENIX-2369 already did the calculation work to get the 
> classpath; just need to simply add it to PQS' classpath now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3223) Add hadoop classpath to PQS classpath

[Josh Elser (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449987#comment-15449987
 ] 

Josh Elser commented on PHOENIX-3223:
-

[~sergey.soldatov], [~aliciashu], either of you have a chance to look at this? 
Pretty simple change.

For got to mention originally, thanks to [~speleato] for reporting this one in 
the first place.

> Add hadoop classpath to PQS classpath
> -
>
> Key: PHOENIX-3223
> URL: https://issues.apache.org/jira/browse/PHOENIX-3223
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 4.9.0, 4.8.1
>
>
> Follow-on from PHOENIX-2369: running on Microsoft's Azure Data Lake Store 
> nets some ClassNotFoundExceptions on the corresponding classes. The library 
> is available via the Hadoop installation, but we miss it because PQS isn't 
> adding the {{hadoop classpath}} like {{sqlline.py}} is (only 
> {{$HADOOP_CONF_DIR}}, oops).
> Alicia's patch on PHOENIX-2369 already did the calculation work to get the 
> classpath; just need to simply add it to PQS' classpath now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3223) Add hadoop classpath to PQS classpath

[Josh Elser (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Josh Elser updated PHOENIX-3223:

Reporter: Sergio Peleato  (was: Josh Elser)

> Add hadoop classpath to PQS classpath
> -
>
> Key: PHOENIX-3223
> URL: https://issues.apache.org/jira/browse/PHOENIX-3223
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Sergio Peleato
>Assignee: Josh Elser
> Fix For: 4.9.0, 4.8.1
>
>
> Follow-on from PHOENIX-2369: running on Microsoft's Azure Data Lake Store 
> nets some ClassNotFoundExceptions on the corresponding classes. The library 
> is available via the Hadoop installation, but we miss it because PQS isn't 
> adding the {{hadoop classpath}} like {{sqlline.py}} is (only 
> {{$HADOOP_CONF_DIR}}, oops).
> Alicia's patch on PHOENIX-2369 already did the calculation work to get the 
> classpath; just need to simply add it to PQS' classpath now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3223) Add hadoop classpath to PQS classpath

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449984#comment-15449984
 ] 

ASF GitHub Bot commented on PHOENIX-3223:
-

GitHub user joshelser opened a pull request:

https://github.com/apache/phoenix/pull/205

PHOENIX-3223 Add `hadoop classpath` to PQS classpath when available



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/joshelser/phoenix PHOENIX-3223-pqs-classpath

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/phoenix/pull/205.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #205


commit d65593b194f6bb89bc0e69e744d2c0e92971cdb8
Author: Josh Elser 
Date:   2016-08-30T19:58:50Z

PHOENIX-3223 Add `hadoop classpath` to PQS classpath when available




> Add hadoop classpath to PQS classpath
> -
>
> Key: PHOENIX-3223
> URL: https://issues.apache.org/jira/browse/PHOENIX-3223
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Josh Elser
>Assignee: Josh Elser
> Fix For: 4.9.0, 4.8.1
>
>
> Follow-on from PHOENIX-2369: running on Microsoft's Azure Data Lake Store 
> nets some ClassNotFoundExceptions on the corresponding classes. The library 
> is available via the Hadoop installation, but we miss it because PQS isn't 
> adding the {{hadoop classpath}} like {{sqlline.py}} is (only 
> {{$HADOOP_CONF_DIR}}, oops).
> Alicia's patch on PHOENIX-2369 already did the calculation work to get the 
> classpath; just need to simply add it to PQS' classpath now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] phoenix pull request #205: PHOENIX-3223 Add `hadoop classpath` to PQS classp...

[joshelser]

GitHub user joshelser opened a pull request:

https://github.com/apache/phoenix/pull/205

PHOENIX-3223 Add `hadoop classpath` to PQS classpath when available



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/joshelser/phoenix PHOENIX-3223-pqs-classpath

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/phoenix/pull/205.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #205


commit d65593b194f6bb89bc0e69e744d2c0e92971cdb8
Author: Josh Elser 
Date:   2016-08-30T19:58:50Z

PHOENIX-3223 Add `hadoop classpath` to PQS classpath when available




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[GitHub] phoenix pull request #:

[twdsilva]

Github user twdsilva commented on the pull request:


https://github.com/apache/phoenix/commit/ede483cf29c86c1acebc12d995966a838e4eaffb#commitcomment-18834750
  
This closes #199


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Resolved] (PHOENIX-2645) Wildcard characters do not match newline characters

[Thomas D'Silva (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-2645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thomas D'Silva resolved PHOENIX-2645.
-
Resolution: Fixed

[~kliew]

I have commited to 4.x and 4.8 branches, thanks for the contribution!

> Wildcard characters do not match newline characters
> ---
>
> Key: PHOENIX-2645
> URL: https://issues.apache.org/jira/browse/PHOENIX-2645
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.7.0
> Environment: Phoenix 4.7.0 on Calcite 1.5
>Reporter: Kevin Liew
>Assignee: Kevin Liew
>  Labels: newbie
> Fix For: 4.9.0, 4.8.1
>
>
> Wildcard characters do not match newline characters
> {code:sql}
> 0: jdbc:phoenix:thin:url=http://localhost:876> create table testnewline (pk 
> varchar(10) primary key)
> . . . . . . . . . . . . . . . . . . . . . . .> ;
> No rows affected (2.643 seconds)
> 0: jdbc:phoenix:thin:url=http://localhost:876> upsert into testnewline values 
> ('AA\nA');
> 1 row affected (0.079 seconds)
> 0: jdbc:phoenix:thin:url=http://localhost:876> select * from testnewline 
> where pk like 'AA%'
> . . . . . . . . . . . . . . . . . . . . . . .> ;
> ++
> | PK |
> ++
> | AA
> A   |
> ++
> 1 row selected (0.086 seconds)
> 0: jdbc:phoenix:thin:url=http://localhost:876> select * from testnewline 
> where pk like 'AA_A';
> ++
> | PK |
> ++
> ++
> No rows selected (0.053 seconds)
> 0: jdbc:phoenix:thin:url=http://localhost:876> select * from testnewline 
> where pk like 'AA%A';
> ++
> | PK |
> ++
> ++
> No rows selected (0.032 seconds)
> 0: jdbc:phoenix:thin:url=http://localhost:876> 
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3218) First draft of Phoenix Tuning Guide

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449874#comment-15449874
 ] 

James Taylor commented on PHOENIX-3218:
---

You can start a new thread on the dev and user list soliciting feedback on it.

> First draft of Phoenix Tuning Guide
> ---
>
> Key: PHOENIX-3218
> URL: https://issues.apache.org/jira/browse/PHOENIX-3218
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Peter Conrad
> Attachments: Phoenix-Tuning-Guide.md
>
>
> Here's a first draft of a Tuning Guide for Phoenix performance. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3116) Support incompatible HBase 1.1.5 and HBase 1.2.2

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449867#comment-15449867
 ] 

James Taylor commented on PHOENIX-3116:
---

Not sure, [~poornachandra]. Hard to know what the vendors will do. There's also 
folks who don't rely on the vendors, like us at SFDC - we won't use those patch 
versions because we know about the issue, but others may pick it up and expect 
it to work.

> Support incompatible HBase 1.1.5 and HBase 1.2.2
> 
>
> Key: PHOENIX-3116
> URL: https://issues.apache.org/jira/browse/PHOENIX-3116
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: Rob Leidle
>Assignee: James Taylor
>Priority: Minor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3116_v2.patch, upgrade-hbase-to-1.2.2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> HBase 1.2.2 made a backwards incompatible change in HTableInterface that 
> requires new overrides.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3218) First draft of Phoenix Tuning Guide

[Peter Conrad (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449866#comment-15449866
 ] 

Peter Conrad commented on PHOENIX-3218:
---

What are the steps to get this reviewed and added to the website?

> First draft of Phoenix Tuning Guide
> ---
>
> Key: PHOENIX-3218
> URL: https://issues.apache.org/jira/browse/PHOENIX-3218
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Peter Conrad
> Attachments: Phoenix-Tuning-Guide.md
>
>
> Here's a first draft of a Tuning Guide for Phoenix performance. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PHOENIX-3223) Add hadoop classpath to PQS classpath

[Josh Elser (JIRA)]

Josh Elser created PHOENIX-3223:
---

 Summary: Add hadoop classpath to PQS classpath
 Key: PHOENIX-3223
 URL: https://issues.apache.org/jira/browse/PHOENIX-3223
 Project: Phoenix
  Issue Type: Bug
Reporter: Josh Elser
Assignee: Josh Elser
 Fix For: 4.9.0, 4.8.1


Follow-on from PHOENIX-2369: running on Microsoft's Azure Data Lake Store nets 
some ClassNotFoundExceptions on the corresponding classes. The library is 
available via the Hadoop installation, but we miss it because PQS isn't adding 
the {{hadoop classpath}} like {{sqlline.py}} is (only {{$HADOOP_CONF_DIR}}, 
oops).

Alicia's patch on PHOENIX-2369 already did the calculation work to get the 
classpath; just need to simply add it to PQS' classpath now.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: [DISCUSS] which 4.8 branches will be active?

[Josh Elser]

I have to concur with Enis here; I wouldn't feel great about dropping 
1.{0,1} support in 4.8 since it went out the door like that.


Granted, I don't recall reading any official Phoenix compatibility 
guidelines as to what *will* be supported between versions, so, the 
ability to do it is there.


Obligatory: I do fall rather far to the edge of the compatibility 
spectrum when it comes to "never break users on patch releases".


(4.9 is an entirely different story though)

James Taylor wrote:

The "why" would because no one needs. If there are users consuming it, then
I agree.

On Mon, Aug 29, 2016 at 2:55 PM, Enis Söztutar  wrote:


We have explicitly decided to do the 4.8 release supporting all 4 of 0.98,
1.0, 1.1 and 1.2.

Why would we even consider dropping support for patch releases? Having a
policy for keeping the supported hbase branches supported throughout the
patch releases makes the process simpler. Preventing upgrade to 4.8.1 for
HBase-1.0 and HBase-1.1 users will not look good at all.

I think we can decide the base hbase versions on major version boundaries.

Enis

On Thu, Aug 25, 2016 at 1:39 AM, Andrew Purtell
wrote:


According to the responses received so far on our version usage survey,
HBase 1.1 is the most popular, followed by 1.2, then 0.98.


On Aug 25, 2016, at 12:43 AM, James Taylor

wrote:

I propose we only continue releases for HBase 0.98 and 1.2, both for

4.8

patch releases as well as 4.9 minor releases.

Thoughts?

[jira] [Commented] (PHOENIX-3116) Support incompatible HBase 1.1.5 and HBase 1.2.2

[Poorna Chandra (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449851#comment-15449851
 ] 

Poorna Chandra commented on PHOENIX-3116:
-

[~jamestaylor] Do you think those two versions of HBase will have widespread 
adoption? If none of the major distros ship them, then it should be okay not to 
support those two HBase versions. Even if one of the major distros ship them 
then we'll have to support it.

> Support incompatible HBase 1.1.5 and HBase 1.2.2
> 
>
> Key: PHOENIX-3116
> URL: https://issues.apache.org/jira/browse/PHOENIX-3116
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: Rob Leidle
>Assignee: James Taylor
>Priority: Minor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3116_v2.patch, upgrade-hbase-to-1.2.2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> HBase 1.2.2 made a backwards incompatible change in HTableInterface that 
> requires new overrides.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3116) Support incompatible HBase 1.1.5 and HBase 1.2.2

[Hadoop QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449785#comment-15449785
 ] 

Hadoop QA commented on PHOENIX-3116:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12826212/PHOENIX-3116_v2.patch
  against master branch at commit d873c2ffd1e539ecd56858c82f0ba2d23e877cf9.
  ATTACHMENT ID: 12826212

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
34 warning messages.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:red}-1 lineLengths{color}.  The patch introduces the following lines 
longer than 100:
+// No @Override tag because it's only declared in HBase 1.2.2 and 
subsequently removed (see HBASE-16420).
+// No @Override tag because it's only declared in HBase 1.2.2 and 
subsequently removed (see HBASE-16420).
+// No @Override tag because it's only declared in HBase 1.2.2 and 
subsequently removed (see HBASE-16420).
+// No @Override tag because it's only declared in HBase 1.2.2 and 
subsequently removed (see HBASE-16420).

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 

Test results: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/546//testReport/
Javadoc warnings: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/546//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/546//console

This message is automatically generated.

> Support incompatible HBase 1.1.5 and HBase 1.2.2
> 
>
> Key: PHOENIX-3116
> URL: https://issues.apache.org/jira/browse/PHOENIX-3116
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: Rob Leidle
>Assignee: James Taylor
>Priority: Minor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3116_v2.patch, upgrade-hbase-to-1.2.2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> HBase 1.2.2 made a backwards incompatible change in HTableInterface that 
> requires new overrides.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] phoenix pull request #:

[dbahir]

Github user dbahir commented on the pull request:


https://github.com/apache/phoenix/commit/b5be8d85bfd05aeb3ea17b1c734eda06d914f516#commitcomment-18834022
  
My bad regarding the synchronization, you are correct.

Did you get to look at the comment regarding the user login ?

Can you allow another to login with a different principal? Would taht cause 
a security issue?
If we create one driver(One) with user A and then create another 
driver(Two) with user B the info in the UGI now is that of user B. So there can 
be a situation where driver One will be using credentials of user B.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (PHOENIX-3200) Error trying to remove Hash Cache

[Maryann Xue (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3200?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449729#comment-15449729
 ] 

Maryann Xue commented on PHOENIX-3200:
--

Hi [~nvommi], thank you for reporting the issue. Could you please attach the 
DDL and sample data here? and also for PHOENIX-3202?

> Error trying to remove Hash Cache
> -
>
> Key: PHOENIX-3200
> URL: https://issues.apache.org/jira/browse/PHOENIX-3200
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.7.0
> Environment: Amazon EMR 4.7.2
>Reporter: Nithin
>Priority: Critical
>  Labels: Hbase, Phoenix
> Fix For: 4.7.0
>
>
> Data Set Consists of 500 Million Rows with each row consisting of 132 Columns 
> (With various data types including Big-Int). Data Load was unsuccessful with 
> the following error after loading 344 Million Rows. Please Note that 2 of the 
> salted table's regions are split at this moment.
> 13:03:48.645 [pool-2-thread-2] ERROR 
> org.apache.phoenix.cache.ServerCacheClient - Error trying to remove hash 
> cache for region=EPOEVENT,,1471683321681.6d9e058365a24e63e5802ca783c09660., 
> hostname=ip-172-31-8-132.us-west-2.compute.internal,60020,1471683102656, 
> seqNum=1935501 org.apache.hadoop.hbase.client.RetriesExhaustedException: 
> Failed after attempts=35, exceptions:
> Tue Aug 23 12:54:36 UTC 2016, 
> RpcRetryingCaller{globalStartTime=1471956876660, pause=100, retries=35}, 
> org.apache.hadoop.hbase.regionserver.RegionServerAbortedException: 
> org.apache.hadoop.hbase.regionserver.RegionServerAbortedException: Server 
> ip-172-31-8-129.us-west-2.compute.internal,60020,1471683105586 aborting
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.checkOpen(RSRpcServices.java:1087)
> at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.execService(RSRpcServices.java:1967)
> at 
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33652)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2180)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at 
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:745)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3222) Document namespace feature

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15449651#comment-15449651
 ] 

James Taylor commented on PHOENIX-3222:
---

It'd be great to promote this feature a bit, but there are no docs I can point 
to currently. Is this something you can write up, [~an...@apache.org]?

> Document namespace feature
> --
>
> Key: PHOENIX-3222
> URL: https://issues.apache.org/jira/browse/PHOENIX-3222
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: Ankit Singhal
>
> I think we need a top level markdown page describing the namespace feature. 
> It'd cover things like:
> - how to turn the feature on
> - how to convert existing tables to be in a namespace
> - how the system tables are converted
> - how a schema corresponds to an HBase namespace
> - when it doesn't (for example views still reside in their physical tables 
> namespace)
> - some general information about what an HBase namespace gives you with some 
> links to more detailed HBase docs



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PHOENIX-3222) Document namespace feature

[James Taylor (JIRA)]

James Taylor created PHOENIX-3222:
-

 Summary: Document namespace feature
 Key: PHOENIX-3222
 URL: https://issues.apache.org/jira/browse/PHOENIX-3222
 Project: Phoenix
  Issue Type: Bug
Reporter: James Taylor
Assignee: Ankit Singhal


I think we need a top level markdown page describing the namespace feature. 
It'd cover things like:
- how to turn the feature on
- how to convert existing tables to be in a namespace
- how the system tables are converted
- how a schema corresponds to an HBase namespace
- when it doesn't (for example views still reside in their physical tables 
namespace)
- some general information about what an HBase namespace gives you with some 
links to more detailed HBase docs




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3116) Support incompatible HBase 1.1.5 and HBase 1.2.2

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-3116:
--
Fix Version/s: 4.9.0

> Support incompatible HBase 1.1.5 and HBase 1.2.2
> 
>
> Key: PHOENIX-3116
> URL: https://issues.apache.org/jira/browse/PHOENIX-3116
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: Rob Leidle
>Assignee: James Taylor
>Priority: Minor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3116_v2.patch, upgrade-hbase-to-1.2.2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> HBase 1.2.2 made a backwards incompatible change in HTableInterface that 
> requires new overrides.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3116) Support incompatible HBase 1.1.5 and HBase 1.2.2

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-3116:
--
Summary: Support incompatible HBase 1.1.5 and HBase 1.2.2  (was: Support 
HBase 1.2.2)

> Support incompatible HBase 1.1.5 and HBase 1.2.2
> 
>
> Key: PHOENIX-3116
> URL: https://issues.apache.org/jira/browse/PHOENIX-3116
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: Rob Leidle
>Assignee: James Taylor
>Priority: Minor
> Fix For: 4.9.0, 4.8.1
>
> Attachments: PHOENIX-3116_v2.patch, upgrade-hbase-to-1.2.2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> HBase 1.2.2 made a backwards incompatible change in HTableInterface that 
> requires new overrides.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3116) Support HBase 1.2.2

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-3116:
--
Attachment: PHOENIX-3116_v2.patch

Adjusting patch to remove @Override tags and use reflection to call delegate 
method since these methods will only exist in HBase 1.1.5 and HBase 1.2.2. Will 
need a similar change in Tephra if we want to support these HBase versions. 
WDYT, [~poornachandra]?

The alternative would be to just say we don't support HBase 1.1.5 and HBase 
1.2.2.

> Support HBase 1.2.2
> ---
>
> Key: PHOENIX-3116
> URL: https://issues.apache.org/jira/browse/PHOENIX-3116
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: Rob Leidle
>Assignee: James Taylor
>Priority: Minor
> Fix For: 4.8.1
>
> Attachments: PHOENIX-3116_v2.patch, upgrade-hbase-to-1.2.2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> HBase 1.2.2 made a backwards incompatible change in HTableInterface that 
> requires new overrides.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3116) Support HBase 1.2.2

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-3116:
--
Summary: Support HBase 1.2.2  (was: Upgrade HBase dependency to 1.2.2)

> Support HBase 1.2.2
> ---
>
> Key: PHOENIX-3116
> URL: https://issues.apache.org/jira/browse/PHOENIX-3116
> Project: Phoenix
>  Issue Type: Improvement
>Affects Versions: 4.8.0
>Reporter: Rob Leidle
>Assignee: James Taylor
>Priority: Minor
> Fix For: 4.8.1
>
> Attachments: upgrade-hbase-to-1.2.2.patch
>
>   Original Estimate: 24h
>  Remaining Estimate: 24h
>
> HBase 1.2.2 made a backwards incompatible change in HTableInterface that 
> requires new overrides.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[GitHub] phoenix pull request #:

[joshelser]

Github user joshelser commented on the pull request:


https://github.com/apache/phoenix/commit/b5be8d85bfd05aeb3ea17b1c734eda06d914f516#commitcomment-18831881
  
> UserGroupInformation.getCurrentUser() will not be thread safe.

```
  @InterfaceAudience.Public
  @InterfaceStability.Evolving
  public synchronized
  static UserGroupInformation getCurrentUser() throws IOException {
AccessControlContext context = AccessController.getContext();
Subject subject = Subject.getSubject(context);
if (subject == null || subject.getPrincipals(User.class).isEmpty()) {
  return getLoginUser();
} else {
  return new UserGroupInformation(subject);
}
  }
```

Am I missing something, @dbahir?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

Phoeinix tracing server Issue

[Onkar Kadam]

Hi Phoenix developers/users ,

We are trying to use the phoenix tracing server in our environment but we are 
facing issue while accessing the web application.  We get

HTTP ERROR 404
Problem accessing /webapp/. Reason:
Not Found

While accessing  
http://dev02-slv-01.neura-local.com:8864/webapp/#


Please find attached the log file with this email.

ENVIRONMENT Details:
Cloudera 5.7.0
Hadoop 2.6.0-cdh5.7.0
HBase 1.2.0-cdh5.7.0
Phoenix 4.7.0 with PHOENIX-2834 patch 
(https://issues.apache.org/jira/browse/PHOENIX-2834)
java version 1.8.0_45


Best Regards,
Onkar Kadam

[GitHub] phoenix pull request #:

[dbahir]

Github user dbahir commented on the pull request:


https://github.com/apache/phoenix/commit/b5be8d85bfd05aeb3ea17b1c734eda06d914f516#commitcomment-18831582
  
UserGroupInformation.getCurrentUser() will not be thread safe, uploaded a 
patch of how to fix it.

[phoenix.txt](https://github.com/apache/phoenix/files/445294/phoenix.txt)



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

ApacheCon Seville CFP closes September 9th

[Rich Bowen]

It's traditional. We wait for the last minute to get our talk proposals
in for conferences.

Well, the last minute has arrived. The CFP for ApacheCon Seville closes
on September 9th, which is less than 2 weeks away. It's time to get your
talks in, so that we can make this the best ApacheCon yet.

It's also time to discuss with your developer and user community whether
there's a track of talks that you might want to propose, so that you
have more complete coverage of your project than a talk or two.

For Apache Big Data, the relevant URLs are:
Event details:
http://events.linuxfoundation.org/events/apache-big-data-europe
CFP:
http://events.linuxfoundation.org/events/apache-big-data-europe/program/cfp

For ApacheCon Europe, the relevant URLs are:
Event details: http://events.linuxfoundation.org/events/apachecon-europe
CFP: http://events.linuxfoundation.org/events/apachecon-europe/program/cfp

This year, we'll be reviewing papers "blind" - that is, looking at the
abstracts without knowing who the speaker is. This has been shown to
eliminate the "me and my buddies" nature of many tech conferences,
producing more diversity, and more new speakers. So make sure your
abstracts clearly explain what you'll be talking about.

For further updated about ApacheCon, follow us on Twitter, @ApacheCon,
or drop by our IRC channel, #apachecon on the Freenode IRC network.

-- 
Rich Bowen
WWW: http://apachecon.com/
Twitter: @ApacheCon

[jira] [Commented] (PHOENIX-3220) DAYOFMONTH function not giving correct result for 24hr clock time format

[Ankit Singhal (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3220?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448929#comment-15448929
 ] 

Ankit Singhal commented on PHOENIX-3220:


Yes, it's a bug, as during TO_DATE evaluation , date literal is considered in 
GMT(default) whereas DAYOFMONTH uses a local timezone.

As a workaround until we fix it , you can set phoenix.query.dateFormatTimeZone 
in the client configuration equal to local timezone.(set explicitly , don't use 
local keyward) 


Note:- sqlline may not display date properly when you run TO_DATE after the 
change(because formatter will still use GMT) but you can try using getDate() 
api .


> DAYOFMONTH function not giving correct result for 24hr clock time format
> 
>
> Key: PHOENIX-3220
> URL: https://issues.apache.org/jira/browse/PHOENIX-3220
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.6.0
> Environment: CentOS
>Reporter: Neeru Jaroliya
>
> Query :
> 0: jdbc:phoenix:localhost> select DAYOFMONTH(TO_DATE('2016-04-01 23:45:00'));
> ++
> | 2  |
> ++
> | 2  |
> ++
> Expected Output : 1
> Issue : DAYOFMONTH function on 24hr clock time format increase the result by 
> one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3221) "LOCAL" keyword is not adhered for "phoenix.query.dateFormatTimeZone"

[Ankit Singhal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Singhal updated PHOENIX-3221:
---
Attachment: PHOENIX-3221.patch

> "LOCAL" keyword is not adhered for "phoenix.query.dateFormatTimeZone"
> -
>
> Key: PHOENIX-3221
> URL: https://issues.apache.org/jira/browse/PHOENIX-3221
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Attachments: PHOENIX-3221.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (PHOENIX-3221) "LOCAL" keyword is not adhered for "phoenix.query.dateFormatTimeZone"

[Ankit Singhal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Singhal updated PHOENIX-3221:
---
Description: 
Currently, it is falling back to GMT.


> "LOCAL" keyword is not adhered for "phoenix.query.dateFormatTimeZone"
> -
>
> Key: PHOENIX-3221
> URL: https://issues.apache.org/jira/browse/PHOENIX-3221
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
> Attachments: PHOENIX-3221.patch
>
>
> Currently, it is falling back to GMT.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PHOENIX-3221) "LOCAL" keyword is not adhered for "phoenix.query.dateFormatTimeZone"

[Ankit Singhal (JIRA)]

Ankit Singhal created PHOENIX-3221:
--

 Summary: "LOCAL" keyword is not adhered for 
"phoenix.query.dateFormatTimeZone"
 Key: PHOENIX-3221
 URL: https://issues.apache.org/jira/browse/PHOENIX-3221
 Project: Phoenix
  Issue Type: Bug
Reporter: Ankit Singhal
Assignee: Ankit Singhal






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-3214) Kafka Phoenix Consumer

[Kalyan (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-3214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448913#comment-15448913
 ] 

Kalyan commented on PHOENIX-3214:
-

Providing a new feature to Phoenix.

Directly ingest Kafka messages to Phoenix.

Similar to flume phoenix integration.

Implementation is done using phoenix-flume solution for inserting the kafka 
messages.

Please review the patch:

https://github.com/kalyanhadooptraining/phoenix/commit/b35a7fb8ffb1fd7745a0467a4cea3b761459e463


> Kafka Phoenix Consumer
> --
>
> Key: PHOENIX-3214
> URL: https://issues.apache.org/jira/browse/PHOENIX-3214
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Kalyan
>Assignee: Kalyan
>
> Providing a new feature to Phoenix.
> Directly ingest Kafka messages to Phoenix.
> Similar to flume phoenix integration.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (PHOENIX-2890) Extend IndexTool to allow incremental index rebuilds

[Ankit Singhal (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-2890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15448769#comment-15448769
 ] 

Ankit Singhal commented on PHOENIX-2890:


Yes [~jamestaylor], it was off my plate for sometime but will resume and try to 
get the patch in 2-3 days.

> Extend IndexTool to allow incremental index rebuilds
> 
>
> Key: PHOENIX-2890
> URL: https://issues.apache.org/jira/browse/PHOENIX-2890
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Ankit Singhal
>Assignee: Ankit Singhal
>Priority: Minor
> Fix For: 4.9.0
>
> Attachments: PHOENIX-2890_wip.patch
>
>
> Currently , IndexTool is used for initial index rebuild but I think we should 
> extend it to be used for recovering index from last disabled timestamp too. 
> In general terms if we run IndexTool on already existing/new index, then it 
> should follow the same semantics as followed by background Index rebuilding 
> thread.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (PHOENIX-3220) DAYOFMONTH function not giving correct result for 24hr clock time format

[Neeru Jaroliya (JIRA)]

Neeru Jaroliya created PHOENIX-3220:
---

 Summary: DAYOFMONTH function not giving correct result for 24hr 
clock time format
 Key: PHOENIX-3220
 URL: https://issues.apache.org/jira/browse/PHOENIX-3220
 Project: Phoenix
  Issue Type: Bug
Affects Versions: 4.6.0
 Environment: CentOS
Reporter: Neeru Jaroliya


Query :
0: jdbc:phoenix:localhost> select DAYOFMONTH(TO_DATE('2016-04-01 23:45:00'));
++
| 2  |
++
| 2  |
++

Expected Output : 1
Issue : DAYOFMONTH function on 24hr clock time format increase the result by 
one.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)