[
https://issues.apache.org/jira/browse/KYLIN-6048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17928364#comment-17928364
]
Guoliang Sun commented on KYLIN-6048:
-------------------------------------
h3. Dev Design
Add a configuration item `kylin.command.max-output-bytes` to control the
maximum length of output logs (default value: 10485760, i.e., 10 MB). Based on
this configuration, truncate the output of local command-line and remote SSH
command-line executions in `CliCommandExecutor`.
If the output exceeds the configured limit, the system will capture the first
half of the configured length and the last half of the configured length of
characters. This ensures that excessively large log outputs do not cause memory
allocation issues, while still preserving both the beginning and end of the log
for debugging purposes.
> OOM Likely to Occur When Build Job Logs Are Too Large
> -----------------------------------------------------
>
> Key: KYLIN-6048
> URL: https://issues.apache.org/jira/browse/KYLIN-6048
> Project: Kylin
> Issue Type: Bug
> Affects Versions: 5.0.0
> Reporter: Guoliang Sun
> Priority: Major
>
> After a build task fails on a build node, the returned log information
> reaches approximately 600MB. When the build retry task attempts to write the
> task record, the excessive memory allocation required for processing the
> large log causes an Out-of-Memory (OOM) error, leading to service downtime.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
