Some Material Before Publishing the Oak Pull Request

Hi All, [ I apologize if someone got multiple copies of this email! As I got rejections due to initial email size.] In continuation to our great talk yesterday (where we agreed about publishing a PR for Oak Incremental Index for Druid), hereby please find some reading material about Oak. The po

Re: Some Material Before Publishing the Oak Pull Request

Oak Incremental Index Pool Request. Below please find a list o... | | | On Wednesday, August 15, 2018, 6:15:07 PM GMT+3, Anastasia Braginsky wrote: Hi All, [ I apologize if someone got multiple copies of this email! As I got rejections due to initial email size.] In

Oak Incremental Index Pull Request

Hi Again, I am afraid that my previous pull request was hiding behind the previous mail, so not all had a chance to get it :)So here is one more time!We would really appreciate any comments! If the request is too big, or if there is any other problem with the request, we would like to hear!We

Re: Oak Incremental Index Pull Request

at 2:44 AM Anastasia Braginsky wrote: > Hi Again, > I am afraid that my previous pull request was hiding behind the previous > mail, so not all had a chance to get it :)So here is one more time!We would > really appreciate any comments! > > If the request is too big, or if there i

Oak ConcurrentNavigableMap for Druid

Hi Guys, I would like to draw your attention to reactivation of the integration of Oak (ConcurrentNavigableMap) into Druid's incremental index. OakMap is an *off-heap* (direct-allocation) concurrent navigable (ordered) KV-map. OakMap is part of the Oak open source project (https://github.com/yah

Druid community weekly meetings

Hi Everyone, Are there no weekly meetings anymore? I recall there were on Tuesdays... Have it been moved to some other time? Thanks,Anastasia

Re: Druid community weekly meetings

y meetings. The chief driver was that the dev list was providing sufficient forum for sync ups and is way more in sync with the Apache Way. if you have specific concerns about dropping the weekly meeting, can you please call them out? Cheers, Charles Allen On Tue, May 21, 2019 at 10:14 AM

A question about Druid design

Hi, Recall our suggestion to use the new concurrent map named Oak as a base for Incremental Index. Oak stands for Off-heap Allocated Keys, for more details please see issue #5698. We had a great progress with Oak integration and stabilizing OakIndex performance. We have some questions regarding

Re: A question about Druid design

ne indexed fact row). But with the PlainFactsHolder there may be more than one fact row per TimeAndDims (in particular: there will be one fact row per input row). Hope this helps. On Wed, May 30, 2018 at 12:14 AM, Anastasia Braginsky < anas...@oath.com.invalid> wrote: > Hi, > Recall ou

Re: A question about Druid design

some answer? Or should it better be discussed in the meeting? Thanks,Anastasia On Thursday, May 31, 2018, 6:40:12 PM GMT+3, Anastasia Braginsky wrote: Hi Gian, Thanks for the explanations! I have one more question: You say that "...the RollupFactsHolder there will be a _single_

Re: A question about Druid design

in Druid corresponding to one row in the input. IIRC the only reason we have a map in that case is because we want to be able to quickly iterate the rows in time-sorted order (query engines like timeseries depend on this ability). On Wed, Jun 13, 2018 at 6:56 AM Anastasia Braginsky wrote: >  Hi

Multi-threaded Druid Tests/Benchmarks

Hi Everybody, >From last Tuesday Druid's meeting I recall Charles mentioned some Druid's >multi-threaded tests/benchmarks that can be applied end-to-end to check the >performance. Can I get some references/names so I can start investigating this direction from multi-threaded Oak-in-Druid perspe

Re: Multi-threaded Druid Tests/Benchmarks

weak and historically very hart to test. There are some topN benchmarks but they are very limited as they don't scale well (heap gets blasted from incremental index) with a large concurrency level. On Sun, Jul 15, 2018 at 6:35 AM Anastasia Braginsky wrote: > Hi Everybody, > From l

Implementing "no-RollUp" (aka PlainFactsHolder) in Oak Incremental Index

Hi Again, Just to summarize once again the way we are implementing the "no-RollUp" (aka PlainFactsHolder) in Oak Incremental Index. If someone is familiar with this part of incremental index, please see if we are wrong in some assumptions. As a reminder, in Oak Incremental Index we are not using

Re: synchronization question about datasketches aggregator

Hi Guys, Just wanted to pay your attention that once OakIncrementalIndex will be in place there is no need to manage the issue of synchronization between aggregators and ingestions. Part of Oak benefits is the synchronization for the simultaneous writes and reads of the same key in the map.