Re: Multi-threaded Druid Tests/Benchmarks

2018-07-18 Thread Charles Allen
io.druid.benchmark.query.TopNBenchmark is the one that tore up heap when i was trying to test alternate strategies for https://github.com/apache/incubator-druid/pull/5913 and https://github.com/apache/incubator-druid/pull/6014 locally. You can control the number of segments created. On Wed, Jul

Re: Multi-threaded Druid Tests/Benchmarks

2018-07-18 Thread Anastasia Braginsky
So this is probably where we can help with the Oak-based incremental index.Can you please give me any reference to those tests? Any descriptions? Thanks! On Tuesday, July 17, 2018, 8:59:57 PM GMT+3, Charles Allen wrote: Unfortunately I think multi-threaded test coverage is kind of

Re: Multi-threaded Druid Tests/Benchmarks

2018-07-17 Thread Charles Allen
Unfortunately I think multi-threaded test coverage is kind of weak and historically very hart to test. There are some topN benchmarks but they are very limited as they don't scale well (heap gets blasted from incremental index) with a large concurrency level. On Sun, Jul 15, 2018 at 6:35 AM