Ruixuan Zhang created KYLIN-5454:
------------------------------------
Summary: Downloading the async query result may cause OOM error
Key: KYLIN-5454
URL: https://issues.apache.org/jira/browse/KYLIN-5454
Project: Kylin
Issue Type: Bug
Affects Versions: 5.0-alpha
Reporter: Ruixuan Zhang
Assignee: Ruixuan Zhang
Fix For: 5.0-alpha
Attachments: 异步查询OOM.pdf
When using
{{http://host:port/kylin/api/async_query/\{query_id}/result_download?}} to get
asynchronous query result, out-of-memory error might occur. The reason is the
formatting process while downloading the result needs read & write, and there
is batch processing rather than streaming processing. Therefore, a refactor is
needed for async query. The main idea of revision is to move the formatting
from downloading result from HDFS to processing & saving query result on HDFS,
and use streaming read & write instead.
The details of design are in the attached file.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
