[
https://issues.apache.org/jira/browse/HIVE-28954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17985278#comment-17985278
]
László Bodor commented on HIVE-28954:
-------------------------------------
[~zabetak]: this investigation along with the detailed documentation is huge!
> CI fails intermittently due to ephemeral-storage exhaustion
> -----------------------------------------------------------
>
> Key: HIVE-28954
> URL: https://issues.apache.org/jira/browse/HIVE-28954
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
> Reporter: Stamatis Zampetakis
> Priority: Major
> Labels: pull-request-available
> Attachments: ephemeral-storage-usage-feb2025-may2025.png
>
>
> The CI fails intermittently due to ephemeral-storage exhaustion. The failures
> on master have been quite often recently:
> *
> https://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/2512/pipeline
> *
> https://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/2511/pipeline
> *
> https://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/2510/pipeline
> *
> https://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/2507/pipeline
> *
> https://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/2505/pipeline
> *
> https://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/2501/pipeline
> *
> https://ci.hive.apache.org/blue/organizations/jenkins/hive-precommit/detail/master/2474/pipeline
> The error message is shown below.
> {noformat}
> Unable to create live FilePath for
> hive-precommit-master-2512-g4v1j-l5dbf-5l773;
> hive-precommit-master-2512-g4v1j-l5dbf-5l773 was marked offline: Pod failed
> (Reason: Evicted, Message: The node was low on resource: ephemeral-storage.
> Threshold quantity: 10120387530, available: 9866372Ki. Container jnlp was
> using 88Ki, request is 0, has larger consumption of ephemeral-storage.
> Container dind was using 188Ki, request is 0, has larger consumption of
> ephemeral-storage. Container hdb was using 453252Ki, request is 0, has larger
> consumption of ephemeral-storage. )
> {noformat}
> As the message indicates the node where the pod is running is low on
> ephemeral-storage leading to the eviction of some pods from the Kubernetes
> scheduler.
> The ephemeral-storage among other things stores all the jars, logs, and
> output from the test execution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
