Hi folks, Just to follow up here: benchmark and cluster validation results are available on the MR-2841 JIRA. Over the next few days we're closing out a couple small remaining issues and I'll start posting candidate merge patches for Jenkins to test.
I anticipate calling the merge vote in parallel with the Jenkins precommit checks and any small follow-ons that that might expose. If there's any specific testing items you feel you need to see to support this merge, please let me know now. Thanks -Todd On Tue, Aug 26, 2014 at 11:50 AM, Todd Lipcon <[email protected]> wrote: > Hi all, > > I wanted to give a quick status update on the MAPREDUCE-2841 feature > branch which was started earlier this summer. > > Since establishing the branch, we've accomplished a number of key items: > > - Imported the initial code from github > - Made integrations with other projects more pluggable using a > ServiceLoader mechanism > - Integrated the project build into our normal maven structure and release > tarballs > - Fixed several unit test failures that occurred in different developer > environments > - Fixed a couple of licensing issues with code > - Various code cleanup for style, etc. > - Fixed all of the compiler warnings > > I've also spent some time doing comparative microbenchmarks of the mapside > sort between this implementation, the existing MR implementation, and > Facebook's implementation (available on github). I haven't posted those > yet, but I hope to do so later this week. The summary is that the native > implementation compares quite favorably to the other options. > > This week, I'm hoping to close out a couple remaining items: > > - A few small code cleanup items I have pending locally (eg improving > javadocs, etc) > - Some basic cluster verification and performance tests > - Post benchmark results mentioned above > > After these items are done, I think we are ready to merge this feature to > trunk. After it is in trunk, we can continue to make improvements, add > documentation, etc. Because the patch is structured as completely new code > (no changes to existing functionality, manually enabled on a per-job > basis), this should be a "zero-risk" merge. > > I'm anticipating calling this merge vote next week, and would appreciate > if other committers can plan to be ready to vote at that time. That is to > say, if you anticipate wanting to review the branch merge, this is your > "heads up" that the vote is coming. Please start your reviews this week. > > Thanks > -Todd > -- Todd Lipcon Software Engineer, Cloudera
