I will see what I can do to set up a test.
On Mon, Apr 30, 2018, 08:10 Vitalii Diravka
wrote:
> Ted,
>
> The rules are enabled and DRILL-3855 [1] is resolved.
> Please try your queries with latest Drill master version.
>
> [1] https://issues.apache.org/jira/browse/DRILL-3855
>
> Kind regards
> V
Ted,
The rules are enabled and DRILL-3855 [1] is resolved.
Please try your queries with latest Drill master version.
[1] https://issues.apache.org/jira/browse/DRILL-3855
Kind regards
Vitalii
On Mon, Apr 30, 2018 at 4:31 PM Nicolas Paris wrote:
> Hi
>
> This looks an interesting design.
>
> A
Hi
This looks an interesting design.
Am I correct such view
would hit the RDBMS for every query ?
However such view would hit the parquet file only when
the timestamp predicate would match a partition ?
Any news on a recent test to confirm the design ?
Thanks
2018-03-20 6:49 GMT+01:00 Ted Dunn
Aman,
That is exactly the clarification that I needed. I had a hazy memory of a
problem in this area, but not enough to actually figure out the current
state.
In case anybody cares, being able to do this is really handy. The basic
idea is to keep long history in files and recent history in a DB.
Due to an infinite loop occurring in Calcite planning, we had to disable
the filter pushdown past the union (SetOps). See
https://issues.apache.org/jira/browse/DRILL-3855.
Now that we have rebased on Calcite 1.15.0, we should re-enable this and
test and if the pushdown works then the partition pru
I think Ted's question is 2 fold, with the former being more important.
1. Can we push filters past a union.
2. Will Drill push filters down to the source.
For the latter, it depends on the source.
For the former, it depends primarily on whether Calcite supports this. I
haven't tried it, so I c
First I would suggest to ignore the view and try out a query which has the
required filters as part of the subqueries on both sides of the union (for
both the database and partitioned parquet data). The plan for such a query
should have the answers to your question. If both the subqueries
independe
IF I create a view that is a union of partitioned parquet files and a
database that has secondary indexes, will Drill be able to properly push
down query limits into both parts of the union?
In particular, if I have lots of archival data and parquet partitioned by
time but my query only asks for r
