On Wed, Dec 16, 2015 at 6:18 AM, Jacques Nadeau <jacq...@dremio.com> wrote:
> With regards to Ted's concern: I agree that applying a filter shouldn't > fail a query. That means we will either have to consider the complete union > Schema before pruning files or consider all fields as either known or > possible after pruning files. > I think that is exactly the choice. And I think that Drill can also use "possible" as an approximation of the truth and that truth can be refined as the result of querying. For instance, if querying over the entire set of files in a directory were to cause Drill to emit a statistical summary of all of those files, then on later queries, Drill might be able to quickly say that none of the files have a particular column and thus fail the query.
