[
https://issues.apache.org/jira/browse/DRILL-5287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15888321#comment-15888321
]
ASF GitHub Bot commented on DRILL-5287:
---------------------------------------
Github user ppadma commented on a diff in the pull request:
https://github.com/apache/drill/pull/758#discussion_r103487885
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/work/foreman/QueryManager.java
---
@@ -280,8 +281,15 @@ public void interrupted(final InterruptedException ex)
{
}
}
- QueryState updateEphemeralState(final QueryState queryState) {
- switch (queryState) {
+ void updateEphemeralState(final QueryState queryState) {
+ // If query is already in zk transient store, ignore the transient
state update option.
+ // Else, they will not be removed from transient store upon
completion.
+ if (transientProfiles.get(stringQueryId) == null &&
--- End diff --
I want to bypass the option for the queries which are already in transient
store when option is enabled. Otherwise, their state will never get updated
and/or will never be removed from transient store. web UI will show these
queries as running forever :-)
Thanks for raising a good point regarding using transientProfiles.get. I
made the change to update and use in memory state instead.
Please review the new diffs.
> Provide option to skip updates of ephemeral state changes in Zookeeper
> ----------------------------------------------------------------------
>
> Key: DRILL-5287
> URL: https://issues.apache.org/jira/browse/DRILL-5287
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.9.0
> Reporter: Padma Penumarthy
> Assignee: Padma Penumarthy
> Fix For: 1.10.0
>
>
> We put transient profiles in zookeeper and update state as query progresses
> and changes states. It is observed that this adds latency of ~45msec for each
> update in the query execution path. This gets even worse when high number of
> concurrent queries are in progress. For concurrency=100, the average query
> response time even for short queries is 8 sec vs 0.2 sec with these updates
> disabled. For short lived queries in a high-throughput scenario, it is of no
> value to update state changes in zookeeper. We need an option to disable
> these updates for short running operational queries.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
