Hi Paul,
That would be great even if you can just copy the discussion here. Being able to do that would greatly optimize the performance of our product. On Mon, Sep 21, 2020 at 5:57 PM Paul Rogers <par0...@gmail.com> wrote: > Hi Carolina, > > This issue came up recently in one of the Drill Slack channels. I wonder, > can anyone here summarize the findings from that Slack discussion? > > Thanks, > > - Paul > > > On Mon, Sep 21, 2020 at 7:35 AM Carolina Gomes <ca...@afterdata.ai> wrote: > > > Also if it helps, I’m using Drill 1.16 in single-node mode. > > > > On Mon, Sep 21, 2020 at 10:32 AM Carolina Gomes <ca...@afterdata.ai> > > wrote: > > > > > Hi all, > > > > > > I have a question about push down of limit and offset clauses on Drill. > > > For my use case, I’d always like for limit and offset clauses to be > > pushed > > > down to the data sources, which are always RDBMS databases like SQL > > Server, > > > Oracle etc. > > > > > > > > > However, I have noticed the decision to push down seems to happen > > > depending on the size of the limit clause, and on the number of columns > > > being projected. > > > > > > > > > As an example, I have a table of about 250 columns with about 50 > million > > > rows. If I do: > > > > > > > > > select * from table limit 1000 —-> limit push down does not happen, > query > > > takes 30s while if I change the physical plan to push down the limit > > > clause, it takes less than 1s. > > > > > > select * from table limit 1000000 —-> limit push down does happen, > query > > > takes roughly same time as if I queried directly on the source DB. > > > > > > Is there a way of easily telling Drill to always pushdown? > > > -- > > > [Carolina Gomes] > > > CEO, AfterData.ai <https://www.afterdata.ai/> > > > +1 (416) 931 4774 > > > > > > > > > > > > -- > > [Carolina Gomes] > > CEO, AfterData.ai <https://www.afterdata.ai/> > > +1 (416) 931 4774 > > > -- [Carolina Gomes] CEO, AfterData.ai <https://www.afterdata.ai/> +1 (416) 931 4774
