[jira] [Commented] (SPARK-4181) Create separate options to control the client-mode AM resource allocation request
[ https://issues.apache.org/jira/browse/SPARK-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237794#comment-14237794 ] WangTaoTheTonic commented on SPARK-4181: I see an related issue SPARK-4696 opened by [~tgraves], so close this. > Create separate options to control the client-mode AM resource allocation > request > - > > Key: SPARK-4181 > URL: https://issues.apache.org/jira/browse/SPARK-4181 > Project: Spark > Issue Type: Improvement > Components: YARN >Reporter: WangTaoTheTonic >Priority: Minor > > I found related discussion in https://github.com/apache/spark/pull/2115, > SPARK-1953 and SPARK-1507. And recently I found some inconvenience in > configuring properties like logging while we use yarn-client mode. So if no > one else do the work, I will try it. Maybe start in few days, and complete in > next 1 or 2 weeks. > As not very familiar with spark on yarn, any discussion and feedback is > welcome! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4181) Create separate options to control the client-mode AM resource allocation request
[ https://issues.apache.org/jira/browse/SPARK-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14237163#comment-14237163 ] WangTaoTheTonic commented on SPARK-4181: I couldn't think out one case for extraLibraryPath either. But we should honor extraClasspath as some function, for instance, log system, would load resources from self-defined classpath. > Create separate options to control the client-mode AM resource allocation > request > - > > Key: SPARK-4181 > URL: https://issues.apache.org/jira/browse/SPARK-4181 > Project: Spark > Issue Type: Improvement > Components: YARN >Reporter: WangTaoTheTonic >Priority: Minor > > I found related discussion in https://github.com/apache/spark/pull/2115, > SPARK-1953 and SPARK-1507. And recently I found some inconvenience in > configuring properties like logging while we use yarn-client mode. So if no > one else do the work, I will try it. Maybe start in few days, and complete in > next 1 or 2 weeks. > As not very familiar with spark on yarn, any discussion and feedback is > welcome! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4181) Create separate options to control the client-mode AM resource allocation request
[ https://issues.apache.org/jira/browse/SPARK-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234333#comment-14234333 ] Thomas Graves commented on SPARK-4181: -- ok. as you discovered extraJavaOptions and possibly the others is being discussed in https://github.com/apache/spark/pull/3409. What is your use case for the extraLibraryPath as I couldn't think of one? > Create separate options to control the client-mode AM resource allocation > request > - > > Key: SPARK-4181 > URL: https://issues.apache.org/jira/browse/SPARK-4181 > Project: Spark > Issue Type: Improvement > Components: YARN >Reporter: WangTaoTheTonic >Priority: Minor > > I found related discussion in https://github.com/apache/spark/pull/2115, > SPARK-1953 and SPARK-1507. And recently I found some inconvenience in > configuring properties like logging while we use yarn-client mode. So if no > one else do the work, I will try it. Maybe start in few days, and complete in > next 1 or 2 weeks. > As not very familiar with spark on yarn, any discussion and feedback is > welcome! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4181) Create separate options to control the client-mode AM resource allocation request
[ https://issues.apache.org/jira/browse/SPARK-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234305#comment-14234305 ] WangTaoTheTonic commented on SPARK-4181: Maybe I didn't describe exactly here. What I wanna do is to pass extraJavaOption and extraLibraryPath to AM in yarn-client mode. > Create separate options to control the client-mode AM resource allocation > request > - > > Key: SPARK-4181 > URL: https://issues.apache.org/jira/browse/SPARK-4181 > Project: Spark > Issue Type: Improvement > Components: YARN >Reporter: WangTaoTheTonic >Priority: Minor > > I found related discussion in https://github.com/apache/spark/pull/2115, > SPARK-1953 and SPARK-1507. And recently I found some inconvenience in > configuring properties like logging while we use yarn-client mode. So if no > one else do the work, I will try it. Maybe start in few days, and complete in > next 1 or 2 weeks. > As not very familiar with spark on yarn, any discussion and feedback is > welcome! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-4181) Create separate options to control the client-mode AM resource allocation request
[ https://issues.apache.org/jira/browse/SPARK-4181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14234289#comment-14234289 ] Thomas Graves commented on SPARK-4181: -- What exactly is the change you are proposing here? You reference other jiras that all have specific things to fix. Is this above and beyond those? > Create separate options to control the client-mode AM resource allocation > request > - > > Key: SPARK-4181 > URL: https://issues.apache.org/jira/browse/SPARK-4181 > Project: Spark > Issue Type: Improvement > Components: YARN >Reporter: WangTaoTheTonic >Priority: Minor > > I found related discussion in https://github.com/apache/spark/pull/2115, > SPARK-1953 and SPARK-1507. And recently I found some inconvenience in > configuring properties like logging while we use yarn-client mode. So if no > one else do the work, I will try it. Maybe start in few days, and complete in > next 1 or 2 weeks. > As not very familiar with spark on yarn, any discussion and feedback is > welcome! -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
