[ https://issues.apache.org/jira/browse/PHOENIX-2995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15408509#comment-15408509 ]
James Taylor commented on PHOENIX-2995: --------------------------------------- Nice work, [~tdsilva]. Can you also get an estimate of how many views we can cache per GB of cache? > Write performance severely degrades with large number of views > --------------------------------------------------------------- > > Key: PHOENIX-2995 > URL: https://issues.apache.org/jira/browse/PHOENIX-2995 > Project: Phoenix > Issue Type: Bug > Reporter: Mujtaba Chohan > Assignee: Thomas D'Silva > Labels: Argus > Fix For: 4.8.1 > > Attachments: PHOENIX-2995-v2.patch, PHOENIX-2995.patch, > create_view_and_upsert.png, image.png, image2.png, upsert_rate.png > > > Write performance for each 1K batch degrades significantly when there are > *10K* views being written in random with default > {{phoenix.client.maxMetaDataCacheSize}}. With all views created, upsert rate > remains around 25 seconds per 1K batch i.e. ~2K rows/min upsert rate. > When {{phoenix.client.maxMetaDataCacheSize}} is increased to 100MB+ then view > does not need to get re-resolved and upsert rate gets back to normal ~60K > rows/min. > With *100K* views and {{phoenix.client.maxMetaDataCacheSize}} set to 1GB, I > wasn't able create all 100K views as upsert time for each 1K batch keeps on > steadily increasing. > Following graph shows 1K batch upsert rate over time with variation of number > of views. Rows are upserted to random views {{CREATE VIEW IF NOT EXISTS ... > APPEND_ONLY_SCHEMA = true, UPDATE_CACHE_FREQUENCY=900000}} is executed before > upsert statement. > !upsert_rate.png! > Base table is also created with {{APPEND_ONLY_SCHEMA = true, > UPDATE_CACHE_FREQUENCY = 900000, AUTO_PARTITION_SEQ}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)