[
https://issues.apache.org/jira/browse/SPARK-27719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16859558#comment-16859558
]
Dongjoon Hyun commented on SPARK-27719:
---------------------------------------
Hi, [~hao.li]. Do you have big log files over 200GB?
In reality, the logs grows. In other words, it started with small sizes and
will grow and exceed the limit.
Are you expecting some entries disappear when you refresh Spark History Server
page?
cc [~vanzin]. How do you think about this suggestion?
> Set maxDisplayLogSize for spark history server
> ----------------------------------------------
>
> Key: SPARK-27719
> URL: https://issues.apache.org/jira/browse/SPARK-27719
> Project: Spark
> Issue Type: Improvement
> Components: Web UI
> Affects Versions: 3.0.0
> Reporter: hao.li
> Priority: Minor
>
> Sometimes a very large eventllog may be useless, and parses it may waste many
> resources.
> It may be useful to avoid parse large enventlogs by setting a configuration
> spark.history.fs.maxDisplayLogSize.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
