There have been several meat-space developments for Drill lately. Here is a quick summary:
- There have been several proposals for syntax conventions for the logical plan language. Three versions have been built. I built a version along the lines of LLVM and a JSON based version that used a constructor convention along with tight binding of classes to operator types. Jacques built a version on Jackson which used much looser binding and supported DAG structuring much more easily. Overall, I feel that Jacques implementation dominates the ones that I have done in essentially every way. I think we should go with his style. - Jacques has been following up his logical syntax work with a reference interpreter. I have been encouraging him to move to a commit-early/commit-often style, but I think he is going to have a working implementation very shortly. This should give Drill the ability to take a logical plan in JSON format and execute that plan directly, without optimization on a single machine. It will make it possible to query real data sets and start experimenting with the logical plan language. It will also allow people to start developing translators from query syntax to logical plan and to start developing scanners. It will also provide a functional reference that will guide the development of high performance execution systems. - Michael Hausenblas now has employer support for focusing on Drill at least part-time. He has been visiting from Ireland at MapR headquarters to talk with Jacques and myself and has committed to building out use cases with sample data sets and queries. - Timothy Chen has been working with Jacques to develop scanners, particularly with a view towards dealing with schema-less data in an efficient fashion. Timothy seems a bit shy about participating on-list, but encouragement for doing that is on-going.
