[jira] [Commented] (OAK-2679) Query engine: cache execution plans
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906421#comment-14906421 ] Thomas Mueller commented on OAK-2679: - I will change the minimum costs per index type slightly: * 0.0 for uuid * 0.0 for traversal * 1.0 for reference * 2.0 for property * 2.0 for nodetype * 2.1 for lucene property * 2.2 for lucene (old) * 2.3 for solr * 3.0 for ordered (deprecated) > Query engine: cache execution plans > --- > > Key: OAK-2679 > URL: https://issues.apache.org/jira/browse/OAK-2679 > Project: Jackrabbit Oak > Issue Type: Improvement > Components: core, query >Reporter: Thomas Mueller >Assignee: Thomas Mueller > Labels: performance > Fix For: 1.3.7 > > Attachments: 0001-OAK-2679-Reduce-execution-plan-overhead_0.2.patch, > OAK-2679.patch, executionplancache.patch > > > If there are many indexes, preparing a query can take a long time, in > relation to executing the query. > The query execution plans can be cached. The cache should be invalidated if > there are new indexes, or indexes are changed; a simple solution might be to > use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2679) Query engine: cache execution plans
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14651519#comment-14651519 ] Joel Richard commented on OAK-2679: --- Should the execution plan for some query index type change more often, we could also add a getCacheStrategy method to the QueryIndex interface which allows to control whether the execution plan can be cached at all and if so for how long/until when. Query engine: cache execution plans --- Key: OAK-2679 URL: https://issues.apache.org/jira/browse/OAK-2679 Project: Jackrabbit Oak Issue Type: Improvement Components: core, query Reporter: Thomas Mueller Assignee: Thomas Mueller Labels: performance Fix For: 1.3.5 Attachments: OAK-2679.patch, executionplancache.patch If there are many indexes, preparing a query can take a long time, in relation to executing the query. The query execution plans can be cached. The cache should be invalidated if there are new indexes, or indexes are changed; a simple solution might be to use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2679) Query engine: cache execution plans
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647632#comment-14647632 ] Joel Richard commented on OAK-2679: --- [~mmarth], here are a few numbers about the time which is lost at the moment with finding the best execution plan: Static files: 21% of the request (query time is 22%) Page 1: 9.5% of the request (query time is 10.5%) Page 2: 5% of the request (query time is 5.5%) Login: 65% of the login time Query engine: cache execution plans --- Key: OAK-2679 URL: https://issues.apache.org/jira/browse/OAK-2679 Project: Jackrabbit Oak Issue Type: Improvement Components: core, query Reporter: Thomas Mueller Assignee: Thomas Mueller Labels: performance Fix For: 1.3.5 Attachments: OAK-2679.patch, executionplancache.patch If there are many indexes, preparing a query can take a long time, in relation to executing the query. The query execution plans can be cached. The cache should be invalidated if there are new indexes, or indexes are changed; a simple solution might be to use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2679) Query engine: cache execution plans
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14647744#comment-14647744 ] Michael Marth commented on OAK-2679: [~joelrich], impressive numbers. [~tmueller], I was still wondering about what would be the best cache flush strategy: would it be correct to assume that the query plans for a given query rarely change? (e.g. that would happen only if the number of items grows significantly for one index compared to another). If so, we could really just use a large TTL for cache flushes. WDYT? Query engine: cache execution plans --- Key: OAK-2679 URL: https://issues.apache.org/jira/browse/OAK-2679 Project: Jackrabbit Oak Issue Type: Improvement Components: core, query Reporter: Thomas Mueller Assignee: Thomas Mueller Labels: performance Fix For: 1.3.5 Attachments: OAK-2679.patch, executionplancache.patch If there are many indexes, preparing a query can take a long time, in relation to executing the query. The query execution plans can be cached. The cache should be invalidated if there are new indexes, or indexes are changed; a simple solution might be to use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2679) Query engine: cache execution plans
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14593091#comment-14593091 ] Joel Richard commented on OAK-2679: --- [~mmarth], I would expect that the cache size stays quite stable. But maybe we could limit the cache entries and once the threshold is reached remove the least used execution plans? Query engine: cache execution plans --- Key: OAK-2679 URL: https://issues.apache.org/jira/browse/OAK-2679 Project: Jackrabbit Oak Issue Type: Improvement Components: core, query Reporter: Thomas Mueller Assignee: Thomas Mueller Labels: performance Fix For: 1.3.1, 1.2.4 Attachments: OAK-2679.patch, executionplancache.patch If there are many indexes, preparing a query can take a long time, in relation to executing the query. The query execution plans can be cached. The cache should be invalidated if there are new indexes, or indexes are changed; a simple solution might be to use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2679) Query engine: cache execution plans
[
https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14592321#comment-14592321
]
Michael Marth commented on OAK-2679:
re
{quote}
a simple solution might be to use a timeout, and / or a manual cache clean via
JMX or so
{quote}
we could also consider the approach taken by MongoDB: flush the cache by every
1000 writes (overall)
Query engine: cache execution plans
---
Key: OAK-2679
URL: https://issues.apache.org/jira/browse/OAK-2679
Project: Jackrabbit Oak
Issue Type: Improvement
Components: core, query
Reporter: Thomas Mueller
Assignee: Thomas Mueller
Labels: performance
Fix For: 1.3.1, 1.2.4
Attachments: OAK-2679.patch, executionplancache.patch
If there are many indexes, preparing a query can take a long time, in
relation to executing the query.
The query execution plans can be cached. The cache should be invalidated if
there are new indexes, or indexes are changed; a simple solution might be to
use a timeout, and / or a manual cache clean via JMX or so.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (OAK-2679) Query engine: cache execution plans
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14580458#comment-14580458 ] Thomas Mueller commented on OAK-2679: - It would make sense also for 1.2 branch (and possibly also 1.0), however there it should not be enabled by default right now, and only enabled if configured. Query engine: cache execution plans --- Key: OAK-2679 URL: https://issues.apache.org/jira/browse/OAK-2679 Project: Jackrabbit Oak Issue Type: Improvement Components: core, query Reporter: Thomas Mueller Assignee: Thomas Mueller Labels: performance Fix For: 1.3.0, 1.2.4 Attachments: OAK-2679.patch, executionplancache.patch If there are many indexes, preparing a query can take a long time, in relation to executing the query. The query execution plans can be cached. The cache should be invalidated if there are new indexes, or indexes are changed; a simple solution might be to use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (OAK-2679) Query engine: cache execution plans
[ https://issues.apache.org/jira/browse/OAK-2679?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14379993#comment-14379993 ] Thomas Mueller commented on OAK-2679: - The property index currently only returns the cost, and executing the query will _again_ iterate over all property index nodes. This can be improved as well, if the property index returns the plan. Query engine: cache execution plans --- Key: OAK-2679 URL: https://issues.apache.org/jira/browse/OAK-2679 Project: Jackrabbit Oak Issue Type: Improvement Components: core, query Reporter: Thomas Mueller Assignee: Thomas Mueller Fix For: 1.3.0 If there are many indexes, preparing a query can take a long time, in relation to executing the query. The query execution plans can be cached. The cache should be invalidated if there are new indexes, or indexes are changed; a simple solution might be to use a timeout, and / or a manual cache clean via JMX or so. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
