Li Can created KYLIN-5595:
-----------------------------
Summary: [kylin 5.0] Launch Job Node not initialize spark session
issue
Key: KYLIN-5595
URL: https://issues.apache.org/jira/browse/KYLIN-5595
Project: Kylin
Issue Type: Improvement
Components: Job Engine
Affects Versions: 5.0-alpha
Reporter: Li Can
Assignee: Li Can
Fix For: 5.0-alpha
Attachments: image (87).png, image (88).png
Saving model will execute 'checkFlatTableSql' method at job node, and it
will not skip the step by default. When execute 'checkFlatTableSql' method, it
will initialize spark session if the job node just started, the process of
getting spark session costs too much time.
The pic 87 shows that get spark session costs more than 63s, and the
execution of checking sql costs more than 2s, it is not friendly for saving
model first time after node launched, and it is also unreasonable.
So I suggest that the job node's process of initialization spark session
should be consistent with the query node, it means that the spark session
should be initialized as the node just started. And the spark session is a
singleton model, just need once initialization, as the pic 88 display.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
