Shekharrajak commented on PR #19510: URL: https://github.com/apache/druid/pull/19510#issuecomment-4525404254
ingestion time at 100k × 5 #### Read Arrow implementation : 17ms Current implementaiton : 53ms #### Index + Persist (= total − read) ~330ms Similar time on both implementation #### total time Arrow implementation: 347 ms current implementation: 410 ms Index+persist does substantially more work per row than read. So even though **Arrow makes read ~3x faster**, that gain is dwarfed when amortized over the much-larger indexing cost. Benchmark added https://github.com/Shekharrajak/druid/pull/1/changes -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
