[
https://issues.apache.org/jira/browse/SPARK-1537?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14329739#comment-14329739
]
Sean Owen commented on SPARK-1537:
----------------------------------
[~zzhan] You have provided a patch as a PR right? anyone can try it. Request
granted.
Given the YARN JIRAs already referenced here, some of which have patches ready
to go too, I think it has been discussed in YARN too? What isn't happening with
YARN that should be, and, can you help with it? I'm not sure if that's where
you are saying the waiting is. That is: hasn't this been blocked on YARN
changes for a long time?
I get it, one person's 'outstanding bug' is another's 'will not fix' but that's
the give and take of OSS. If you want this feature in Spark, and people are
asking that it should depend on some YARN changes -- then what do you think
about lobbying for those YARN changes? or do you disagree that they're
necessary, and can you argue that here please?
I don't understand your second reply. Yes, it sounds like two people have a
similar solution with a similar problem with YARN APIs. You say you're not
waiting on code now, but have repeatedly asked Marcelo to share some (other?)
code. It's odd since, yes, it's very clear you acknowledge you've already seen
his code and reused a bit, which is entirely fine. I hope we're done with that
exchange.
I sense some insinuation that code is being 'hidden' in bad faith, but I can't
figure out the conspiracy. I see every willingness to make *your* change alone
here, if you propose something that addresses the YARN issues raised here. You
are *not* blocked on anyone else's patch. However all of us are 'blocked' on
the consensus of community / committers that care about this issue, and it
looks like the response is clear so far: not until YARN API stuff is sorted out
one way or the other.
Are you suggesting this patch should be committed without the YARN changes? or
that you're working on the YARN changes? what do you want to take over and do
next?
> Add integration with Yarn's Application Timeline Server
> -------------------------------------------------------
>
> Key: SPARK-1537
> URL: https://issues.apache.org/jira/browse/SPARK-1537
> Project: Spark
> Issue Type: New Feature
> Components: YARN
> Reporter: Marcelo Vanzin
> Assignee: Marcelo Vanzin
> Attachments: SPARK-1537.txt, spark-1573.patch
>
>
> It would be nice to have Spark integrate with Yarn's Application Timeline
> Server (see YARN-321, YARN-1530). This would allow users running Spark on
> Yarn to have a single place to go for all their history needs, and avoid
> having to manage a separate service (Spark's built-in server).
> At the moment, there's a working version of the ATS in the Hadoop 2.4 branch,
> although there is still some ongoing work. But the basics are there, and I
> wouldn't expect them to change (much) at this point.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
