On Thu, Oct 24, 2013 at 1:45 PM, Chris Nauroth <cnaur...@hortonworks.com> wrote: > Hi Andrew, > > I've come to the conclusion that I'm very confused about merge votes. :-) > It's not just about HDFS-4949. I'm confused about all merge votes. > Rather than muddy the waters here, I've started a separate discussion on > common-dev. > > I do agree with the general plan outlined here, and I will comment directly > on the HDFS-4949 jira with a binding +1 when I see that we've completed > that plan.
Thanks, Chris. Andrew posted a merge patch to HDFS-4949. We're happy that this code is getting closer to getting into trunk, since it will make it easier to integrate with the other features in trunk (like HDFS-2832). There are still some follow-up tasks, but we feel that it's easier to do those in trunk. I'm going to update the design doc in just a moment so be sure to check it out. Are there any other things we should do today prior to merging? Colin > > Chris Nauroth > Hortonworks > http://hortonworks.com/ > > > > On Wed, Oct 23, 2013 at 2:18 PM, Andrew Wang <andrew.w...@cloudera.com>wrote: > >> Hey Chris, >> >> Right now we're on track to have all of those things done by tomorrow. >> Since the remaining issues are either not technical or do not involve major >> changes, I was hoping we could +1 this merge vote in the spirit of "+1 >> pending jenkins". We've gotten clean unit test runs on upstream Jenkins as >> well, so the only fixups we should need for test-patch.sh are findbugs and >> javac (which are normally pretty trivial to clean up). Of course, all of >> your listed prereqs and test-patch would be taken care of before actually >> merging to trunk. >> >> So, we can reset the vote if you feel strongly about this, but it seems >> like the only real result will be delaying the merge by a week. >> >> Thanks, >> Andrew >> >> >> On Wed, Oct 23, 2013 at 1:03 PM, Chris Nauroth <cnaur...@hortonworks.com >> >wrote: >> >> > I've received some feedback that we haven't handled this merge vote the >> > same as other comparable merge votes, and that the vote should be reset >> > because of this. >> > >> > The recent custom is that we only call for the merge vote after all >> > pre-requisites have been satisfied. This would include committing to the >> > feature branch all patches that the devs deem necessary before the code >> > lands in trunk, posting a test plan, posting an updated design doc in >> case >> > implementation choices diverged from the original design doc, and >> getting a >> > good test-patch run from Jenkins on the merge patch. This was the >> process >> > followed for other recent major features like HDFS-2802 (snapshots), >> > HDFS-347 (short-circuit reads via sharing file descriptors), and >> > HADOOP-8562 (Windows compatibility). In this thread, we've diverged from >> > that process by calling for a vote on a branch that hasn't yet completed >> > the pre-requisites and stating a plan for work to be done before the >> merge. >> > >> > I still support this work, but can we please restart the vote after the >> > pre-requisites have landed in the branch? >> > >> > Chris Nauroth >> > Hortonworks >> > http://hortonworks.com/ >> > >> > >> > >> > On Fri, Oct 18, 2013 at 1:37 PM, Chris Nauroth <cnaur...@hortonworks.com >> > >wrote: >> > >> > > +1 >> > > >> > > Sounds great! >> > > >> > > Regarding testing caching+federation, this is another thing that I had >> > > intended to pick up as part of HDFS-5149. I'm not sure if I can get >> this >> > > done in the next 7 days, so I'll keep you posted. >> > > >> > > Chris Nauroth >> > > Hortonworks >> > > http://hortonworks.com/ >> > > >> > > >> > > >> > > On Fri, Oct 18, 2013 at 11:15 AM, Colin McCabe <cmcc...@alumni.cmu.edu >> > >wrote: >> > > >> > >> Hi Chris, >> > >> >> > >> I think it's feasible to complete those tasks in the next 7 days. >> > >> Andrew is on HDFS-5386. >> > >> >> > >> The test plan document is a great idea. We'll try to get that up >> > >> early next week. We have a lot of unit tests now, clearly, but some >> > >> manual testing is important too. >> > >> >> > >> If we discover any issues during testing, then we can push out the >> > >> merge timeframe. For example, one area that probably needs more >> > >> testing is caching+federation. >> > >> >> > >> I would like to get HDFS-5378 and HDFS-5366 in as well. >> > >> >> > >> The other subtasks are "nice to have" but not really critical, and I >> > >> think it would be just as easy to do them in trunk. We're hoping that >> > >> having this in trunk will make it easier for us to collaborate on >> > >> HDFS-2832 and other ongoing work. >> > >> >> > >> > Also, I want to confirm that this vote only covers trunk. >> > >> > I don't see branch-2 mentioned, so I assume that we're >> > >> > not voting on merge to branch-2 yet. >> > >> >> > >> Yeah, this vote is only to merge to trunk. >> > >> >> > >> cheers. >> > >> Colin >> > >> >> > >> On Fri, Oct 18, 2013 at 10:48 AM, Chris Nauroth >> > >> <cnaur...@hortonworks.com> wrote: >> > >> > I agree that the code has reached a stable point. Colin and Andrew, >> > >> thank >> > >> > you for your contributions and collaboration. >> > >> > >> > >> > Throughout development, I've watched the feature grow by running >> daily >> > >> > builds in a pseudo-distributed deployment. As of this week, the >> full >> > >> > feature set is working end-to-end. I also think we've reached a >> point >> > >> of >> > >> > API stability for clients who want to control caching >> > programmatically. >> > >> > >> > >> > There are several things that I'd like to see completed before the >> > >> merge as >> > >> > pre-requisites: >> > >> > >> > >> > - HDFS-5203: Concurrent clients that add a cache directive on the >> same >> > >> path >> > >> > may prematurely uncache from each other. >> > >> > - HDFS-5385: Caching RPCs are AtMostOnce, but do not persist client >> ID >> > >> and >> > >> > call ID to edit log. >> > >> > - HDFS-5386: Add feature documentation for datanode caching. >> > >> > - Standard clean-ups to satisfy Jenkins pre-commit on the merge >> patch. >> > >> > (For example, I know we've introduced some Javadoc warnings.) >> > >> > - Full test suite run on Windows. (The feature is not yet >> implemented >> > >> on >> > >> > Windows. This is just intended to catch regressions.) >> > >> > - Test plan posted to HDFS-4949, similar in scope to the snapshot >> test >> > >> plan >> > >> > that was posted to HDFS-2802. For my own part, I've run the new >> unit >> > >> > tests, and I've tested end-to-end in a pseudo-distributed >> deployment. >> > >> It's >> > >> > unlikely that I'll get a chance to test fully distributed before the >> > >> vote >> > >> > closes, so I'm curious to hear if you've done this on your side yet. >> > >> > >> > >> > Also, I want to confirm that this vote only covers trunk. I don't >> see >> > >> > branch-2 mentioned, so I assume that we're not voting on merge to >> > >> branch-2 >> > >> > yet. >> > >> > >> > >> > Before I cast my vote, can you please discuss whether or not it's >> > >> feasible >> > >> > to complete all of the above in the next 7 days? For the issues >> > >> assigned >> > >> > to me, I do expect to complete them. >> > >> > >> > >> > Thanks again for all of your hard work! >> > >> > >> > >> > Chris Nauroth >> > >> > Hortonworks >> > >> > http://hortonworks.com/ >> > >> > >> > >> > >> > >> > >> > >> > On Thu, Oct 17, 2013 at 3:07 PM, Colin McCabe < >> cmcc...@alumni.cmu.edu >> > >> >wrote: >> > >> > >> > >> >> +1. Thanks, guys. >> > >> >> >> > >> >> best, >> > >> >> Colin >> > >> >> >> > >> >> On Thu, Oct 17, 2013 at 3:01 PM, Andrew Wang < >> > andrew.w...@cloudera.com >> > >> > >> > >> >> wrote: >> > >> >> > Hello all, >> > >> >> > >> > >> >> > I'd like to call a vote to merge the HDFS-4949 branch (in-memory >> > >> caching) >> > >> >> > to trunk. Colin McCabe and I have been hard at work the last 3.5 >> > >> months >> > >> >> > implementing this feature, and feel that it's reached a level of >> > >> >> stability >> > >> >> > and utility where it's ready for broader testing and integration. >> > >> >> > >> > >> >> > I'd also like to thank Chris Nauroth at Hortonworks for code >> > reviews >> > >> and >> > >> >> > bug fixes, and everyone who's reviewed the HDFS-4949 design doc >> and >> > >> left >> > >> >> > comments. >> > >> >> > >> > >> >> > Obviously, I am +1 for the merge. The vote will run the standard >> 7 >> > >> days, >> > >> >> > closing on October 24 at 11:59PM. >> > >> >> > >> > >> >> > Thanks, >> > >> >> > Andrew >> > >> >> >> > >> > >> > >> > -- >> > >> > CONFIDENTIALITY NOTICE >> > >> > NOTICE: This message is intended for the use of the individual or >> > >> entity to >> > >> > which it is addressed and may contain information that is >> > confidential, >> > >> > privileged and exempt from disclosure under applicable law. If the >> > >> reader >> > >> > of this message is not the intended recipient, you are hereby >> notified >> > >> that >> > >> > any printing, copying, dissemination, distribution, disclosure or >> > >> > forwarding of this communication is strictly prohibited. If you have >> > >> > received this communication in error, please contact the sender >> > >> immediately >> > >> > and delete it from your system. Thank You. >> > >> >> > > >> > > >> > >> > -- >> > CONFIDENTIALITY NOTICE >> > NOTICE: This message is intended for the use of the individual or entity >> to >> > which it is addressed and may contain information that is confidential, >> > privileged and exempt from disclosure under applicable law. If the reader >> > of this message is not the intended recipient, you are hereby notified >> that >> > any printing, copying, dissemination, distribution, disclosure or >> > forwarding of this communication is strictly prohibited. If you have >> > received this communication in error, please contact the sender >> immediately >> > and delete it from your system. Thank You. >> > >> > > -- > CONFIDENTIALITY NOTICE > NOTICE: This message is intended for the use of the individual or entity to > which it is addressed and may contain information that is confidential, > privileged and exempt from disclosure under applicable law. If the reader > of this message is not the intended recipient, you are hereby notified that > any printing, copying, dissemination, distribution, disclosure or > forwarding of this communication is strictly prohibited. If you have > received this communication in error, please contact the sender immediately > and delete it from your system. Thank You.
