Hi All, The Drill-on-YARN work (DRILL-1170) is drawing to a close and is ready to be merged into master. The project resulted in about 80+ new files, 10K+ lines of code, so this will be a bit of an effort to merge.
Drill-on-YARN is actually independent of the rest of Drill. Except for changes to the Drill launch scripts (DRILL-4581), no Drill code was harmed in the making of this feature; all code is new. The best approach would be if Drill were set up for semi-independent subprojects. But, since it is not, the DoY build is integrated into Drill’s own build as a Maven module. To make the task more manageable I’ll divide the work into about six requests, split mostly along subsystem boundaries. Each will build. But, (somewhat obviously), the feature won’t run until all subsystems are merged. Testing consists of unit tests for those items that lend themselves to such tests. YARN is a tool for launching distributed apps via generated shell scripts, so it is a bit difficult to test full YARN integration in a unit test. Instead, we’ll rely on system integration tests for that aspect. If you need more background, please consult the documents attached to DRILL-1170. These include an overview and a draft user guide. Thanks ahead of time for reviewing the changes. Please let me know if I can provide other materials to make the task easier. - Paul
