[jira] [Commented] (SPARK-21855) When submit job to yarn and add file multiple times,we should log error instead of warning
[
https://issues.apache.org/jira/browse/SPARK-21855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145297#comment-16145297
]
zhoukang commented on SPARK-21855:
--
Yes, i realize that there is an other follow-up jira spark-18099 after
spark-14423.
And my start point is that:if threw exception then we should print error.
Actually,it is trivial.
I will think more carefully before open new issue.Since during last few days,i
got some trivial idea and create some issues bothered you.
Thanks for your time sincerely![~srowen]
> When submit job to yarn and add file multiple times,we should log error
> instead of warning
> --
>
> Key: SPARK-21855
> URL: https://issues.apache.org/jira/browse/SPARK-21855
> Project: Spark
> Issue Type: Improvement
> Components: YARN
>Affects Versions: 2.2.0
>Reporter: zhoukang
>Priority: Trivial
>
> Now when submit job with yarn,and upload same file multiple times.We will
> throw exception but logging level is warn.
> {code:java}
> 17/08/29 11:17:37 WARN yarn.Client: Resource
> hdfs://tjwqtst-galaxy/spark/tjwqtst-transfer/scripts/oom/oom_script.sh added
> multiple times to distributed cache.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21855) When submit job to yarn and add file multiple times,we should log error instead of warning
[
https://issues.apache.org/jira/browse/SPARK-21855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145062#comment-16145062
]
Sean Owen commented on SPARK-21855:
---
That change has been merged to master.
Changing a log level doesn't help; if the situation must fail, then it should
fail fast. If the situation shouldn't fail, then there's a bug elsewhere if it
does.
> When submit job to yarn and add file multiple times,we should log error
> instead of warning
> --
>
> Key: SPARK-21855
> URL: https://issues.apache.org/jira/browse/SPARK-21855
> Project: Spark
> Issue Type: Improvement
> Components: YARN
>Affects Versions: 2.2.0
>Reporter: zhoukang
>Priority: Trivial
>
> Now when submit job with yarn,and upload same file multiple times.We will
> throw exception but logging level is warn.
> {code:java}
> 17/08/29 11:17:37 WARN yarn.Client: Resource
> hdfs://tjwqtst-galaxy/spark/tjwqtst-transfer/scripts/oom/oom_script.sh added
> multiple times to distributed cache.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21855) When submit job to yarn and add file multiple times,we should log error instead of warning
[
https://issues.apache.org/jira/browse/SPARK-21855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144897#comment-16144897
]
zhoukang commented on SPARK-21855:
--
https://github.com/apache/spark/pull/12203 actually，this is the real problem
but have not been merged into master branch.
I do not think this is an trivial issue since this will cause job submitting
fail.But https://github.com/apache/spark/pull/12203 is a better fix.
Sorry for bothering you [~srowen].I have closed the issues reported by me which
won' fix.And will consider more carefully when create new issue.
> When submit job to yarn and add file multiple times,we should log error
> instead of warning
> --
>
> Key: SPARK-21855
> URL: https://issues.apache.org/jira/browse/SPARK-21855
> Project: Spark
> Issue Type: Improvement
> Components: YARN
>Affects Versions: 2.2.0
>Reporter: zhoukang
>Priority: Trivial
>
> Now when submit job with yarn,and upload same file multiple times.We will
> throw exception but logging level is warn.
> {code:java}
> 17/08/29 11:17:37 WARN yarn.Client: Resource
> hdfs://tjwqtst-galaxy/spark/tjwqtst-transfer/scripts/oom/oom_script.sh added
> multiple times to distributed cache.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-21855) When submit job to yarn and add file multiple times,we should log error instead of warning
[
https://issues.apache.org/jira/browse/SPARK-21855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16144710#comment-16144710
]
Apache Spark commented on SPARK-21855:
--
User 'caneGuy' has created a pull request for this issue:
https://github.com/apache/spark/pull/19073
> When submit job to yarn and add file multiple times,we should log error
> instead of warning
> --
>
> Key: SPARK-21855
> URL: https://issues.apache.org/jira/browse/SPARK-21855
> Project: Spark
> Issue Type: Improvement
> Components: Deploy
>Affects Versions: 2.2.0
>Reporter: zhoukang
>Priority: Trivial
>
> Now when submit job with yarn,and upload same file multiple times.We will
> throw exception but logging level is warn.
> {code:java}
> 17/08/29 11:17:37 WARN yarn.Client: Resource
> hdfs://tjwqtst-galaxy/spark/tjwqtst-transfer/scripts/oom/oom_script.sh added
> multiple times to distributed cache.
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
