[ 
https://issues.apache.org/jira/browse/PHOENIX-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421298#comment-15421298
 ] 

James Taylor commented on PHOENIX-2995:
---------------------------------------

bq. As far as the synchronization is concerned, I think it should now be at the 
PMetadataImpl level.
One thing I did to evaluate the synchronization change (maybe you did too?) was 
to apply the patch and look at the synchronization being done in 
ConnectionQueryServicesImpl. It's actually not too complex. The reason to keep 
it done this way is that each connection goes through the PMetadataImpl level 
and they don't need synchronization. The ConnectionQueryServicesImpl methods 
do, though, as these are the calls that get bubbled up to keep the JVM wide 
cache up to date.

> Write performance severely degrades with large number of views 
> ---------------------------------------------------------------
>
>                 Key: PHOENIX-2995
>                 URL: https://issues.apache.org/jira/browse/PHOENIX-2995
>             Project: Phoenix
>          Issue Type: Bug
>            Reporter: Mujtaba Chohan
>            Assignee: Thomas D'Silva
>              Labels: Argus
>             Fix For: 4.8.1
>
>         Attachments: PHOENIX-2995-v2.patch, PHOENIX-2995-v3.patch, 
> PHOENIX-2995.patch, create_view_and_upsert.png, image.png, image2.png, 
> image3.png, upsert_rate.png
>
>
> Write performance for each 1K batch degrades significantly when there are 
> *10K* views being written in random with default 
> {{phoenix.client.maxMetaDataCacheSize}}. With all views created, upsert rate 
> remains around 25 seconds per 1K batch i.e. ~2K rows/min upsert rate. 
> When {{phoenix.client.maxMetaDataCacheSize}} is increased to 100MB+ then view 
> does not need to get re-resolved and upsert rate gets back to normal ~60K 
> rows/min.
> With *100K* views and {{phoenix.client.maxMetaDataCacheSize}} set to 1GB, I 
> wasn't able create all 100K views as upsert time for each 1K batch keeps on 
> steadily increasing. 
> Following graph shows 1K batch upsert rate over time with variation of number 
> of views. Rows are upserted to random views {{CREATE VIEW IF NOT EXISTS ... 
> APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY=900000}} is executed before 
> upsert statement.
> !upsert_rate.png!
> Base table is also created with {{APPEND_ONLY_SCHEMA = true, 
> UPDATE_CACHE_FREQUENCY = 900000, AUTO_PARTITION_SEQ}}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Reply via email to