We are just about to the point of having a working pure SQL workflow for Beam! One of the last things that remains is how to configure Pipeline Options via a SQL shell. I have written up a proposal to use the set statement, for example "SET runner=DataflowRunner". I'm looking for feedback, particularly on what will make for the best user experience. Please take a look and comment:
https://docs.google.com/document/d/1UTsSBuruJRfGnVOS9eXbQI6NauCD4WnSAPgA_Y0zjdk/edit?usp=sharing Andrew