[
https://issues.apache.org/jira/browse/KYLIN-1872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ma Gang updated KYLIN-1872:
---------------------------
Attachment: query_visible_interruptable-master.patch
patch for master
> Make query visible and interruptible, improve server's stablility
> -----------------------------------------------------------------
>
> Key: KYLIN-1872
> URL: https://issues.apache.org/jira/browse/KYLIN-1872
> Project: Kylin
> Issue Type: Improvement
> Components: Query Engine
> Reporter: Ma Gang
> Assignee: Ma Gang
> Attachments: query_visible_interruptable-1.4rc.patch,
> query_visible_interruptable-master.patch
>
>
> Problem:
> 1. Large query result will break kylin server, for example: select * from
> fact_table. Even when properties "kylin.query.scan.threshold" and
> "kylin.query.mem.budget" are set properly, OOM still happens, because the
> hbase rpc thread is not interrupted, the result will continually go to kylin
> server. And server will run OOM quickly when there are multiple such queries.
> 2. Tow many slow queries will occupy all tomcat threads, and make server
> unresponsed.
> 3. There's no corelation id for a specified query, so it is hard to find the
> rpc log for a specified query, if there are too many queries running
> concurrently.
> Solution:
> 1. Interrupt the rpc thread and main query thread when return result size
> larger than the config limit size.
> 2. Make query visible. Admin can view all running queries, and detail of each
> query.
> Split the query into following steps:
> 1) sql parse
> 2) cube plan
> 3) query cache
> 4) multiple cube segment query
> a. for each segment request, have muliple endpoint range request.
> b. for each endpoint range request, have multiple coprocessor request.
> c. for each coprocessor request, have multiple region server rpc.
> Admin can view the startTime/endTime of each step, and the thread stack
> trace if the step is running.
> 3. Add query id as corelation id in the rpc log.
> 4. Admin can interrupt a running query, to release the thread, memory, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
