Re: Druid 0.12.1 release vote

2018-05-30 Thread Prashant Deva
+1 to Gian's reply. I think some process overall to release minor versions faster would be very beneficial. Prashant On Wed, May 30, 2018 at 1:26 PM Gian Merlino wrote: > My feeling is that the desire to get out regression fixes like > https://github.com/druid-io/druid/pull/5554 faster (the or

Re: Druid 0.12.1 release vote

2018-05-30 Thread Gian Merlino
My feeling is that the desire to get out regression fixes like https://github.com/druid-io/druid/pull/5554 faster (the original reason for doing 0.12.1) outweighs the desire to fix more bugs (including this one). So I would still vote +1 on 0.12.1 as-is and release this fix in 0.12.2 or 0.13.0. ht

Re: Druid 0.12.1 release vote

2018-05-30 Thread Clint Wylie
-1, I think we should backport this fix first https://github.com/druid-io/druid/pull/5815 once it gets merged, since without it queries can silently produce incorrect results. It's not a regression afaict, but it seems pretty major bug when the conditions that cause it are met. Sorry for being at t

Re: Transactions in Druid?

2018-05-30 Thread Edward Bortnikov
Charles,  All great questions. I have answers only to part of them :)  - People struggled with reasoning about eventual consistency model at least since NoSQL is around. To the best of my knowledge, no "golden standard" emerged. In a system that maintains approximate information anyways, the qu

Re: Transactions in Druid?

2018-05-30 Thread Charles Allen
To throw in my two cents. A lot of items in Druid trade off accuracy for speed and approximations. There are things like aliasing effects from the scatter-gather in high cardinality topN queries as well as hyper-log-log approximations for cardinality estimators that favor speed over accuracy. To

Re: Transactions in Druid?

2018-05-30 Thread Edward Bortnikov
Hi Gian,  Thanks for the explanation.  So, the community does not envision traditional OLTP/RT analytics use cases like "read A-compute-write B", cross-partition consistent scans, or atomic updates of multiple indexes? The reason I'm asking is that we also work on the Omid transaction processo

Re: A question about Druid design

2018-05-30 Thread Gian Merlino
Hi Anastasia, 1) At ingestion time the FactsHolder is sorted. The unsorted code path is used by groupBy v1, which hasn't been common since groupBy v2 was made the default a few releases ago. So I would only worry about the sorted case. 2) PlainFactsHolder is used when the user has disabled rollup

A question about Druid design

2018-05-30 Thread Anastasia Braginsky
Hi, Recall our suggestion to use the new concurrent map named Oak as a base for Incremental Index. Oak stands for Off-heap Allocated Keys, for more details please see issue #5698. We had a great progress with Oak integration and stabilizing OakIndex performance. We have some questions regarding