Hi folks, Thanks for replying. Please see the response bellow.
bq. 2* The last update of YARN-321 was done in NOV10, this was done from branch-2 (that seems a NIT as it should be against trunk) Basically it's a discussion thread. I'm already in the process of updating the branch. As I mentioned before, one of the motivation of merge branch YARN-321 is to prevent it from further falling behind. bq. 1* The merge must be done in trunk first (then, ideally, from trunk into branch-2) bq. 3* IMO, until we don’t have security, we should merge into trunk only Yes, we plan to merge to trunk, then to branch-2, but I agree, let's have security ready before going to branch-2. We can continue with security on trunk. bq. Regarding doc, while we don't necessarily need full documentation before merging, my feeling is that we should at least have a page on it that will allow give cluster operators a sketch of its role, APIs, and implications. Agree, let's prepare a doc as well. bq. My opinion is that we should mark each API @Stable unless there is a strong reason for it not to be. When I say APIs, I am thinking of the REST APIs, RPC interface, and RM<->AHS shared-bus. I'm afraid it's still too early to confirm which APIs can be marked @Stable. How about not doing this until security is ready and refactoring duplicate code is done? At that time, we should have a clearer picture about it. Thanks, Zhijie On Mon, Jan 6, 2014 at 3:02 PM, Alejandro Abdelnur <t...@cloudera.com>wrote: > This is great news. A few things to consider before doing the merge: > > 1* The merge must be done in trunk first (then, ideally, from trunk into > branch-2) > 2* The last update of YARN-321 was done in NOV10, this was done from > branch-2 (that seems a NIT as it should be against trunk) > 3* IMO, until we don’t have security, we should merge into trunk only > > I would like to see #1 and #2 taken care before making a decision. The > reason for this is that if the source changes outside of the AHS are too > pervasive, then we may end up be making difficult backports from trunk to > branch-2 and release branches because of the delta. > > Can we get a rebase of the YARN-321 to the head of trunk to see if my > concerns are valid or not? > > Thanks. > > Alejandro > > > > On Mon, Jan 6, 2014 at 1:11 AM, Sandy Ryza <sandy.r...@cloudera.com> > wrote: > > > Very excited for this feature and appreciative of all the work put into > > this. Reviewed the JIRA and my only two remaining concerns are about > > documentation and API stability. > > > > Regarding doc, while we don't necessarily need full documentation before > > merging, my feeling is that we should at least have a page on it that > will > > allow give cluster operators a sketch of its role, APIs, and > implications. > > If this doesn't exist yet, would be happy to help with reviewing. > > > > Regarding APIs, I imagine some will jump on this as soon as it becomes > part > > of a release as it's a fairly essential feature for non-MR apps. My > > opinion is that we should mark each API @Stable unless there is a strong > > reason for it not to be. When I say APIs, I am thinking of the REST > APIs, > > RPC interface, and RM<->AHS shared-bus. > > > > -Sandy > > > > > > On Sun, Jan 5, 2014 at 8:33 PM, lohit <lohit.vijayar...@gmail.com> > wrote: > > > > > +1 to merge into branch2 now. > > > On Jan 5, 2014 6:22 PM, "Zhijie Shen" <zs...@hortonworks.com> wrote: > > > > > > > Hi folks, > > > > > > > > Majority of the functionality of Application History Server has been > > > > completed on branch YARN-321. AHS can now work end-to-end. > > > ResourceManager > > > > records the historical information of the application, the > application > > > > attempt and the container in terms of events via a history writer on > a > > > > separate thread. The historical information is going to be persisted > on > > > > HDFS. On the other side, an application history server runs as a > > separate > > > > process, and it allows users to access the historical information via > > RPC > > > > interface, web UI and REST APIs. > > > > > > > > According to the proposal, the only major missing piece is security. > > > Except > > > > it, YARN-321 should be pretty much ready to be merged to Branch-2. We > > > > propose to merge YARN-321 to Branch-2 now, such that we can prevent > > > > YARN-321 from falling behind too much, and reduce the effort of > editing > > > > merge conflicts. After merge, we can continue on security, > refactoring > > > > duplicate code, fixing bugs and other improvements. > > > > > > > > Anyone who is interested in looking at AHS can review/play with > > YARN-321 > > > > branch: > http://svn.apache.org/viewvc/hadoop/common/branches/YARN-321/ > > > > > > > > You can also have a look at the latest design doc: > > > > > > > > > > > > > > https://issues.apache.org/jira/secure/attachment/12619638/Generic%20Application%20History%20-%20Design-20131219.pdf > > > > > > > > If there are no objections, we'll push towards updating the branch, > > > running > > > > the patch through Jenkins and getting ready for merge vote by the end > > of > > > > this week. > > > > > > > > Thanks, > > > > Zhijie > > > > > > > > -- > > > > Zhijie Shen > > > > Hortonworks Inc. > > > > http://hortonworks.com/ > > > > > > > > -- > > > > CONFIDENTIALITY NOTICE > > > > NOTICE: This message is intended for the use of the individual or > > entity > > > to > > > > which it is addressed and may contain information that is > confidential, > > > > privileged and exempt from disclosure under applicable law. If the > > reader > > > > of this message is not the intended recipient, you are hereby > notified > > > that > > > > any printing, copying, dissemination, distribution, disclosure or > > > > forwarding of this communication is strictly prohibited. If you have > > > > received this communication in error, please contact the sender > > > immediately > > > > and delete it from your system. Thank You. > > > > > > > > > > > > > -- > Alejandro > -- Zhijie Shen Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
