[ https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14192538#comment-14192538 ]
Zhijie Shen commented on SPARK-1537: ------------------------------------ bq. Spark needs both to put and read data It's again a vague statement. Can you share your design detail, such that we can evaluate it is really necessary? And what is the actual way of visualizing data? And integration work is not just single bug fix patch, we can divide work into a sequent of sub tasks, and the first step is to enable Spark job to be able to putting the data into the timeline server. By doing this, not only Spark's only web front can visualize job history, it also enable the third-party tools to do Spark job analysis too. bq. I'm not sure why you say it's security-related since there nothing security-related in the example code I posted. I said "According to the exception, the user doesn't pass the authorization for some reason." If you don't agree on it, please post your investigation on YARN-2444, YARN folks will help you on this issue. bq. if something doesn't work in 2.5 but works in 2.6, No matter the integration with timeline service, Spark on YARN is picking Hadoop versions now. It doesn't make sense to ask for a feature by using an early version that hasn't it. > Add integration with Yarn's Application Timeline Server > ------------------------------------------------------- > > Key: SPARK-1537 > URL: https://issues.apache.org/jira/browse/SPARK-1537 > Project: Spark > Issue Type: New Feature > Components: YARN > Reporter: Marcelo Vanzin > Assignee: Marcelo Vanzin > > It would be nice to have Spark integrate with Yarn's Application Timeline > Server (see YARN-321, YARN-1530). This would allow users running Spark on > Yarn to have a single place to go for all their history needs, and avoid > having to manage a separate service (Spark's built-in server). > At the moment, there's a working version of the ATS in the Hadoop 2.4 branch, > although there is still some ongoing work. But the basics are there, and I > wouldn't expect them to change (much) at this point. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org