Phoenix 5.x and CDH6.x
Hi Do you plan to issue Phoenix 5.x parcel based on CDH6 like there was phoenix 4.x parcels based on CDH 5.x? Regards, Noam PRIVILEGED AND CONFIDENTIAL PLEASE NOTE: The information contained in this message is privileged and confidential, and is intended only for the use of the individual to whom it is addressed and others who have been specifically authorized to receive it. If you are not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, or if any problems occur with transmission, please contact sender. Thank you.
Re: Phoenix 5.x and CDH6.x
Yes, but the first release supporting CDH will be delayed to some point in the next couple of months. On Sun, 21 Oct 2018, 09:48 Bulvik, Noam, wrote: > Hi > > > > Do you plan to issue Phoenix 5.x parcel based on CDH6 like there was > phoenix 4.x parcels based on CDH 5.x? > > > > Regards, > > > > *Noam * > > -- > > PRIVILEGED AND CONFIDENTIAL > PLEASE NOTE: The information contained in this message is privileged and > confidential, and is intended only for the use of the individual to whom it > is addressed and others who have been specifically authorized to receive > it. If you are not the intended recipient, you are hereby notified that any > dissemination, distribution or copying of this communication is strictly > prohibited. If you have received this communication in error, or if any > problems occur with transmission, please contact sender. Thank you. >
Phoenix Metrics - How to understand
Hello Team,
Phoenix provides metrics to understand insights during execution of a
SQL query both at individual SQL and client JVM level, I want to check if
there any document explaining what a each metric mean and what indicates a
query execution went fine or had issues. for eg:
below are the Queries in sequence that were fired using a standalone
program along with metrics response in a hbase cluster with phoenix (hbase
1.1.2 + phoenix 4.8.1)
Query Start Time: 2018-09-04 09:10:58,536
Query End Time: 2018-09-04 09:12:52,932
Time taken to query: ~ 2minutes
Below are the phoenix metrics
*Mutation Write Metrings*
*PhoenixRuntime.getWriteMetricsForMutationsSinceLastReset(connection)*
>>>
{UPSRT_TABLE={MUTATION_COMMIT_TIME=9325278, MUTATION_BATCH_SIZE=8097788,
MUTATION_BYTES=23708733912}}
*Mutation Read Metrings*
*PhoenixRuntime.getReadMetricsForMutationsSinceLastReset(connection)*
>>>
{SLET_TABLE={TASK_QUEUE_WAIT_TIME=29384207, SCAN_BYTES=17973054976,
TASK_END_TO_END_TIME=41399954, TASK_EXECUTED_COUNTER=712,
TASK_EXECUTION_TIME=12015738, TASK_REJECTED_COUNTER=0}}
*Global Metrics*
*PhoenixRuntime.getGlobalPhoenixClientMetrics()*
>>>
MUTATION_BATCH_SIZE :: 8097788
MUTATION_BYTES :: 23708733912
MUTATION_COMMIT_TIME :: 9325278
QUERY_TIME :: 0
NUM_PARALLEL_SCANS :: 712
SCAN_BYTES :: 17973054976
SPOOL_FILE_SIZE :: 0
MEMORY_CHUNK_BYTES :: 0
MEMORY_WAIT_TIME :: 0
TASK_QUEUE_WAIT_TIME :: 29384207
TASK_END_TO_END_TIME :: 41399954
TASK_EXECUTION_TIME :: 12015738
MUTATION_SQL_COUNTER :: 95
SELECT_SQL_COUNTER :: 0
TASK_EXECUTED_COUNTER :: 712
TASK_REJECTED_COUNTER :: 0
QUERY_TIMEOUT_COUNTER :: 0
QUERY_FAILED_COUNTER :: 0
SPOOL_FILE_COUNTER :: 0
OPEN_PHOENIX_CONNECTIONS_COUNTER :: 1
*Q:* *How to understand below metrics *
TASK_QUEUE_WAIT_TIME = 29384207 milliseconds (~490 minutes)
TASK_EXECUTION_TIME = 12015738 milliseconds (~200 minutes)
But query itself completed in ~ 2 minutes
It is also noticed that OPEN_PHOENIX_CONNECTIONS_COUNTER kept on increasing
by 10 for each of the Query that is fired subsequently i.e.
"OPEN_PHOENIX_CONNECTIONS_COUNTER = 11" , "OPEN_PHOENIX_CONNECTIONS_COUNTER
= 21 " etc.,though there were no other noticeable logs indicating some
error, what does this mean, is it like multiple attempts tried internally,
and all the open connections by this are not cleared ?
--
*Venkata Subbarayudu Amanchi.*
Phoenix Delete Query - Hbase rpc timeout relation
Hello Team, How does *Phoenix delete query uses hbase-rpc-timeout* , and can you please point any document illustrating the difference between a delete-query execution Vs Upsert query execution, since I've noticed a different behaviour for this during an *UPSERT *query Vs *DELETE* query. Please find below details on the same hbase.rpc.timeout = 12 ( 2 min ) phoenix.query.keepAliveMs = 12 ( 2 min ) phoenix.query.timeoutMs = 120 ( 20 min ) phoenix-auto-commit = true An upsert query that runs for more than 2 mins and below 20 mins, goes fine (test-query ran for about 12 mins), but a delete query that runs for more than 2 min throws rpc-timeout errors, and when the table is checked few records got deleted, when rpctimeout increased to same as phoenix-query timeout , delete query ran fine, hence wants an understanding on how to set rpc-timeout + phoenix-timeout values for a large table delete and if the delete is expected to take increasing time as the table grows (trying with secondary indexing is one option to keep the timeouts lower, but assuming the delete-time grows even with secondary indexes) Thank you for your support -- *Venkata Subbarayudu Amanchi.*
