I think it's fine to release new query features that don't have Druid SQL support, although I would certainly encourage people to add SQL support even if it's not required. In the long run I wish that SQL could become the primary query language for Druid, because in Druid's space (analytical databases) SQL is what users generally expect to see.
On Mon, Apr 22, 2019 at 6:43 PM Eyal Yurman <eyurma...@verizonmedia.com.invalid> wrote: > What does this mean for release of new query features without Druid SQL > support? > > On Mon, Apr 22, 2019 at 2:07 PM Jihoon Son <jihoon...@apache.org> wrote: > > > +1 > > I'm mostly using only SQL. > > > > Jihoon > > > > On Mon, Apr 22, 2019 at 12:24 PM Jonathan Wei <jon...@apache.org> wrote: > > > > > +1, I think it has enough feature parity with the native JSON queries, > > and > > > it's stable enough to be moved out of experimental. > > > > > > On Thu, Apr 18, 2019 at 6:23 PM Fangjin Yang <fang...@imply.io> wrote: > > > > > > > Strong +1 > > > > > > > > I think there's been enough production usage of Druid SQL, it matches > > > what > > > > native JSON-over-http can do, and it should no longer be labeled as > > > > experimental. > > > > > > > > On Thu, Apr 18, 2019 at 6:06 PM Gian Merlino <g...@apache.org> > wrote: > > > > > > > > > Hey all, > > > > > > > > > > Reviving this thread. Now that > > > > > https://github.com/apache/incubator-druid/pull/6742 and > > > > > https://github.com/apache/incubator-druid/pull/6862 have been > > released > > > > in > > > > > 0.14, I'm re-proposing graduating Druid SQL from experimental > status > > in > > > > the > > > > > next release, 0.15. I don't think we need a formal vote on this, > but > > if > > > > > there seems to be general consensus, I'll do a PR before the next > > > > 3-monthly > > > > > 0.15 code freeze (which is in about 2 weeks). > > > > > > > > > > On Thu, Jan 31, 2019 at 9:20 AM Gian Merlino <g...@apache.org> > > wrote: > > > > > > > > > > > It sounds like the general feeling is +1 on Kafka and maybe wait > > > > another > > > > > > release for SQL. I will do a PR to mark Kafka ingest as > > > > non-experimental, > > > > > > then, and on SQL we'll see whether #6742 and #6862 look solid in > > > 0.14. > > > > > > > > > > > > On Tue, Jan 15, 2019 at 8:39 AM Gian Merlino <g...@apache.org> > > > wrote: > > > > > > > > > > > >> Hi Mat, > > > > > >> > > > > > >> Ah, right. IMO > > https://github.com/apache/incubator-druid/pull/6742 > > > > is a > > > > > >> decent workaround towards making #6176 less of a problem. It > would > > > > > prevent > > > > > >> incorrect results from happening (the broker will not start up > its > > > > http > > > > > >> server & announce itself, and so it won't get picked up by > > clients, > > > if > > > > > it > > > > > >> never got the initialization event). If paired with monitoring > > that > > > > > >> restarts unhealthy brokers, the issue should be fully > > worked-around > > > in > > > > > >> practice. > > > > > >> > > > > > >> Even though there's an (imo) viable workaround, it would still > be > > > good > > > > > to > > > > > >> fix the root cause of #6176. I just raised > > > > > >> https://github.com/apache/incubator-druid/pull/6862 to update > > > Curator > > > > > >> and see if that helps -- there is a bug fixed in the latest > > release > > > > that > > > > > >> looks like it could cause the behavior we're seeing ( > > > > > >> https://issues.apache.org/jira/browse/CURATOR-476). > > > > > >> > > > > > >> My feeling is that it's still reasonable to remove the > > experimental > > > > > label > > > > > >> from Druid SQL in 0.14, especially since #6742 will make SQL and > > > > native > > > > > >> queries behave at parity (initialization getting missed will > delay > > > > > broker > > > > > >> startup for _both_ cases). So in that sense they are at least on > > the > > > > > same > > > > > >> footing. And hopefully #6862 will fix them both, together. > > > > > >> > > > > > >> On Tue, Jan 15, 2019 at 7:56 AM Pierre-Emile Ferron < > > > > > pe.fer...@gmail.com> > > > > > >> wrote: > > > > > >> > > > > > >>> A remaining issue with SQL is > > > > > >>> https://github.com/apache/incubator-druid/issues/6176 > > > > > >>> > > > > > >>> We've seen it happen several times in production on 0.12, where > > > > > >>> thankfully > > > > > >>> SQL doesn't power anything critical. The current workarounds > are: > > > > > >>> 1. Restart the broker. Obviously not a good solution. > > > > > >>> 2. Migrate to HTTP segment discovery. I'm fine with that, and > we > > > are > > > > > >>> actually planning to do it soon in our clusters, but I'm still > > > > > concerned > > > > > >>> about other Druid users—the default setting is still ZK, which > > > means > > > > > that > > > > > >>> SQL would still have this issue by default. > > > > > >>> > > > > > >>> Before marking SQL as non-experimental, I'd suggest either > fixing > > > the > > > > > >>> root > > > > > >>> cause, or making HTTP segment discovery the default and then > > > > explicitly > > > > > >>> deprecating ZK segment discovery. > > > > > >>> > > > > > >>> > > > > > >>> On Mon, Jan 14, 2019 at 2:18 PM Gian Merlino <g...@apache.org> > > > > wrote: > > > > > >>> > > > > > >>> > I'd like to propose graduating a couple of features out of > > > > > >>> 'experimental' > > > > > >>> > status in 0.14. Both are popular features (judging by mailing > > > list > > > > & > > > > > >>> github > > > > > >>> > issue/PR activity). Both have been around for a while and > have > > > > > >>> attained a > > > > > >>> > good level of quality and stability of API & behavior. I > > believe > > > > > >>> removing > > > > > >>> > the 'experimental' banner from these features would more > > > accurately > > > > > >>> reflect > > > > > >>> > reality, and be a good signal to the user community. > > > > > >>> > > > > > > >>> > 1) Kafka indexing service. First introduced in Druid 0.9.1, > it > > > went > > > > > >>> through > > > > > >>> > a major protocol change in Druid 0.12.0 that added > incremental > > > > > >>> publishing, > > > > > >>> > & 'mixing' of data from different partitions. Subjectively, > > > quality > > > > > >>> appears > > > > > >>> > to be getting more solid, based on frequency of bug reports > and > > > > also > > > > > >>> based > > > > > >>> > on our own experiences running this in production. Finally- I > > > > believe > > > > > >>> it is > > > > > >>> > already much more robust than Tranquility, the only 'stable' > > > > > >>> alternative. > > > > > >>> > > > > > > >>> > 2) Druid SQL. First introduced in Druid 0.10.0. It isn't > > feature > > > > > >>> complete > > > > > >>> > yet (multi-value dimensions, datasketches, etc, remain > > > unsupported) > > > > > >>> but the > > > > > >>> > API and behavior have been generally stable. No major issues > > > around > > > > > >>> memory > > > > > >>> > / performance / etc regressions relative to native Druid > > queries > > > > are > > > > > >>> > outstanding. IMO, it is well on its way to becoming a first > > class > > > > way > > > > > >>> to > > > > > >>> > query Druid, and it is a good time to remove the > 'experimental' > > > > > banner. > > > > > >>> > > > > > > >>> > > > > > >> > > > > > > > > > > > > > > >
