Re: Druid + Presto?

2020-07-09 Thread Samarth Jain
Gian, For the presto-sql version of Druid connector, for V1, we decided to pursue the JDBC route. You can follow along on the progress here - https://github.com/prestosql/presto/issues/1855 My colleague, Parth (cc'ed as well) is working on implementing Druid aggregation push down including

[CRON] Broken: apache/druid#27802 (master - 54a8fb8)

2020-07-09 Thread Travis CI
Build Update for apache/druid - Build: #27802 Status: Broken Duration: 2 mins and 25 secs Commit: 54a8fb8 (master) Author: Maytas Monsereenusorn Message: Fix flaky tests in DruidCoordinatorTest (#10157) * Fix flaky tests in DruidCoordinatorTest * Imporve

Re: Druid + Presto?

2020-07-09 Thread Zhenxiao Luo
Thank you, Mainak. Hi Gian, Glad to see you are interested in Presto Druid connector. My colleague, @Hao Luo @Beinan Wang and me, together, implemented the Presto Druid connector in PrestoDB: https://prestodb.io/docs/current/connector/druid.html Our implementation includes: 1. Presto could

Re: Druid + Presto?

2020-07-09 Thread Mainak Ghosh
Hello Gian, We are currently testing the (other) Presto Druid connector at our end. It has aggregation push down support. Adding Zhenxiao to this thread since he is the primary developer of the connector. He can provide the kind of details you are looking for. Thanks, Mainak > On Jul 9,

Re: Druid + Presto?

2020-07-09 Thread Hao Luo
I wrote the Druid connector that reads directly from deep storage(the one you found in prestodb). It understands the segment file format and read only the data needed for the query. I think its possible to have aggregation pushdown(and even more possibilities with other complex operation

Re: Druid + Presto?

2020-07-09 Thread Gian Merlino
By the way, I see that the other Presto has a Druid connector too: https://prestodb.io/docs/current/connector/druid.html. From the docs it looks like it has different lineage and might even work differently. On Thu, Jul 9, 2020 at 12:22 PM Gian Merlino wrote: > I was thinking of exploring ideas

Re: Druid + Presto?

2020-07-09 Thread Gian Merlino
I was thinking of exploring ideas like pushing down aggregations, enabling Presto to query directly from deep storage (in cases where there aren't any interesting things to push down, this may be more efficient than querying Druid servers), enabling translation from Druid's SQL dialect to Presto's

Re: Druid + Presto?

2020-07-09 Thread Samarth Jain
Hi Gian, I contributed the jdbc based presto-druid connector in prestosql which went out in release 337 https://prestosql.io/docs/current/release/release-337.html. The v1 version of the connector doesn’t support aggregate push down yet. It is being actively worked on and we expect it to be

Druid + Presto?

2020-07-09 Thread Gian Merlino
Hey Druids, I was wondering, is anyone on this list using Druid + Presto together? If so, what does your architecture look like and which edition / flavor of Presto and Druid connector are you using? What's your experience been like? I'm asking since I'm starting to think about whether it makes