[GitHub] [druid] liran-funaro commented on a change in pull request #10001: [Extension] oak-incremental-index: Low resource (RAM and CPU) incremental-index implementation using off-heap key/value map

liran-funaro commented on a change in pull request #10001: URL: https://github.com/apache/druid/pull/10001#discussion_r629896625 ## File path: processing/src/main/java/org/apache/druid/segment/DoubleDimensionIndexer.java ## @@ -132,33 +132,32 @@ public DimensionSelector makeDi

[GitHub] [druid] liran-funaro commented on a change in pull request #10001: [Extension] oak-incremental-index: Low resource (RAM and CPU) incremental-index implementation using off-heap key/value map

liran-funaro commented on a change in pull request #10001: URL: https://github.com/apache/druid/pull/10001#discussion_r629888108 ## File path: extensions-contrib/oak-incremental-index/src/main/java/org/apache/druid/segment/incremental/oak/OakIncrementalIndex.java ## @@ -0,0 +1

[GitHub] [druid] liran-funaro commented on a change in pull request #10001: [Extension] oak-incremental-index: Low resource (RAM and CPU) incremental-index implementation using off-heap key/value map

liran-funaro commented on a change in pull request #10001: URL: https://github.com/apache/druid/pull/10001#discussion_r629883796 ## File path: extensions-contrib/oak-incremental-index/pom.xml ## @@ -0,0 +1,123 @@ + + + +http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www

[GitHub] [druid] maytasm commented on pull request #11123: Avoid memory mapping hydrants after they are persisted & after they are merged for native batch ingestion

maytasm commented on pull request #11123: URL: https://github.com/apache/druid/pull/11123#issuecomment-837908713 Possibly legit test failure in `(openjdk8) indexing modules test`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [druid] maytasm opened a new pull request #11232: Add feature to automatically remove compaction configurations for inactive datasources

maytasm opened a new pull request #11232: URL: https://github.com/apache/druid/pull/11232 Add feature to automatically remove compaction configurations for inactive datasources ### Description We currently already have tasklog auto cleanup (https://github.com/apache/druid/pul

[druid] branch master updated (3a660bc -> fae7ebf)

This is an automated email from the ASF dual-hosted git repository. suneet pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 3a660bc Make sure updating coordinator config is protected against race condition (#11144) add fae7ebf change

[GitHub] [druid] suneet-s merged pull request #11218: change errant 'none' configuration to 'manual':

suneet-s merged pull request #11218: URL: https://github.com/apache/druid/pull/11218 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pleas

[GitHub] [druid] suneet-s commented on pull request #11218: change errant 'none' configuration to 'manual':

suneet-s commented on pull request #11218: URL: https://github.com/apache/druid/pull/11218#issuecomment-837819504 Overruling Travis... This doc change should not impact the integration tests -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [druid] loquisgon opened a new issue #11231: Minimize memory utilization in Sinks/Hydrants for native batch ingestion

loquisgon opened a new issue #11231: URL: https://github.com/apache/druid/issues/11231 ### Motivation Currently native batch ingestion has no guardrails with respect to its memory consumption. Native batch internally keeps creating data structures (such as `Sinks` and `Firehydrants`

[GitHub] [druid] suneet-s commented on pull request #11228: Web console: Fix required field treatment

suneet-s commented on pull request #11228: URL: https://github.com/apache/druid/pull/11228#issuecomment-837613502 @clintropolis fyi I added the milestone 0.21.1 to this bug so we can backport it before making the patch release -- This is an automated message from the Apache Git Service.

[GitHub] [druid] vogievetsky commented on pull request #11228: Web console: Fix required field treatment

vogievetsky commented on pull request #11228: URL: https://github.com/apache/druid/pull/11228#issuecomment-837582521 @suneet-s the fact that the endpoint field was incorrectly marked as `required` was there from the start of the data loader but until https://github.com/apache/druid/pull/10

[GitHub] [druid] suneet-s commented on pull request #11228: Web console: Fix required field treatment

suneet-s commented on pull request #11228: URL: https://github.com/apache/druid/pull/11228#issuecomment-837425156 @vogievetsky Do we know when this bug was introduced? I'm wondering if this is bad enough to justify backporting this to 0.21.1 release. -- This is an automated message from

[GitHub] [druid] maytasm commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

maytasm commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629682360 ## File path: server/src/main/java/org/apache/druid/server/coordinator/duty/KillDatasourceMetadata.java ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apac

[druid] branch master updated (f6662b4 -> 3a660bc)

This is an automated email from the ASF dual-hosted git repository. maytasm pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from f6662b4 fix count and average SQL aggregators on constant virtual columns (#11208) add 3a660bc Make sure upda

[GitHub] [druid] maytasm merged pull request #11144: Make sure updating coordinator config is protected against race condition

maytasm merged pull request #11144: URL: https://github.com/apache/druid/pull/11144 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [druid] maytasm commented on a change in pull request #11144: Make sure updating coordinator config is protected against race condition

maytasm commented on a change in pull request #11144: URL: https://github.com/apache/druid/pull/11144#discussion_r629678850 ## File path: server/src/main/java/org/apache/druid/server/http/CoordinatorCompactionConfigsResource.java ## @@ -162,27 +165,68 @@ public Response delete

[GitHub] [druid] maytasm opened a new issue #11230: Make sleep time for org.apache.druid.java.util.common.RetryUtils configurable

maytasm opened a new issue #11230: URL: https://github.com/apache/druid/issues/11230 ### Description Make sleep time for org.apache.druid.java.util.common.RetryUtils configurable Modify the code so that org.apache.druid.java.util.common.RetryUtils and take in parameters config

[GitHub] [druid] maytasm commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

maytasm commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629671791 ## File path: server/src/main/java/org/apache/druid/server/coordinator/duty/KillDatasourceMetadata.java ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apac

[druid] branch master updated (15de29a -> f6662b4)

This is an automated email from the ASF dual-hosted git repository. cwylie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 15de29a Handle not having default rules better (#11214) add f6662b4 fix count and average SQL aggregators on co

[GitHub] [druid] clintropolis merged pull request #11208: fix count and average SQL aggregators on constant virtual columns

clintropolis merged pull request #11208: URL: https://github.com/apache/druid/pull/11208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[GitHub] [druid] clintropolis commented on a change in pull request #11144: Make sure updating coordinator config is protected against race condition

clintropolis commented on a change in pull request #11144: URL: https://github.com/apache/druid/pull/11144#discussion_r629660293 ## File path: server/src/main/java/org/apache/druid/server/http/CoordinatorCompactionConfigsResource.java ## @@ -162,27 +165,68 @@ public Response d

[GitHub] [druid] maytasm commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

maytasm commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629655853 ## File path: server/src/main/java/org/apache/druid/server/coordinator/duty/KillDatasourceMetadata.java ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apac

[GitHub] [druid] maytasm commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

maytasm commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629652513 ## File path: server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java ## @@ -1423,4 +1424,40 @@ public boolean insertDat

[GitHub] [druid] maytasm commented on a change in pull request #11144: Make sure updating coordinator config is protected against race condition

maytasm commented on a change in pull request #11144: URL: https://github.com/apache/druid/pull/11144#discussion_r629647863 ## File path: server/src/main/java/org/apache/druid/server/http/CoordinatorCompactionConfigsResource.java ## @@ -162,27 +165,68 @@ public Response delete

[GitHub] [druid] maytasm commented on a change in pull request #11144: Make sure updating coordinator config is protected against race condition

maytasm commented on a change in pull request #11144: URL: https://github.com/apache/druid/pull/11144#discussion_r629635984 ## File path: core/src/main/java/org/apache/druid/common/config/ConfigManager.java ## @@ -168,64 +171,99 @@ private void poll() return holder.getRefe

[GitHub] [druid] maytasm commented on a change in pull request #11144: Make sure updating coordinator config is protected against race condition

maytasm commented on a change in pull request #11144: URL: https://github.com/apache/druid/pull/11144#discussion_r629636342 ## File path: server/src/main/java/org/apache/druid/server/http/CoordinatorCompactionConfigsResource.java ## @@ -162,27 +165,68 @@ public Response delete

[GitHub] [druid] maytasm commented on a change in pull request #11144: Make sure updating coordinator config is protected against race condition

maytasm commented on a change in pull request #11144: URL: https://github.com/apache/druid/pull/11144#discussion_r629635984 ## File path: core/src/main/java/org/apache/druid/common/config/ConfigManager.java ## @@ -168,64 +171,99 @@ private void poll() return holder.getRefe

[GitHub] [druid] clintropolis commented on a change in pull request #11144: Make sure updating coordinator config is protected against race condition

clintropolis commented on a change in pull request #11144: URL: https://github.com/apache/druid/pull/11144#discussion_r629623614 ## File path: core/src/main/java/org/apache/druid/common/config/ConfigManager.java ## @@ -168,64 +171,99 @@ private void poll() return holder.ge

[GitHub] [druid] maytasm commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

maytasm commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629620902 ## File path: server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java ## @@ -1423,4 +1424,40 @@ public boolean insertDat

[GitHub] [druid] maytasm commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

maytasm commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629618963 ## File path: server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java ## @@ -1423,4 +1424,40 @@ public boolean insertDat

[druid] branch master updated (691d7a1 -> 15de29a)

This is an automated email from the ASF dual-hosted git repository. zachjsh pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 691d7a1 SQL timeseries no longer skip empty buckets with all granularity (#11188) add 15de29a Handle not havi

[GitHub] [druid] zachjsh merged pull request #11214: Web console: Handle not having default rules better

zachjsh merged pull request #11214: URL: https://github.com/apache/druid/pull/11214 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [druid] maytasm commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

maytasm commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629610611 ## File path: server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java ## @@ -1423,4 +1424,40 @@ public boolean insertDat

[GitHub] [druid] maytasm commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

maytasm commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629610774 ## File path: server/src/main/java/org/apache/druid/indexing/overlord/IndexerMetadataStorageCoordinator.java ## @@ -267,6 +267,15 @@ SegmentPublishResult

[GitHub] [druid] maytasm commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

maytasm commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629610611 ## File path: server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java ## @@ -1423,4 +1424,40 @@ public boolean insertDat

[GitHub] [druid] vogievetsky opened a new issue #11229: Create e2e tests that include datasources with overshadowed and unpunished segments

vogievetsky opened a new issue #11229: URL: https://github.com/apache/druid/issues/11229 This would help test to alleviate issues like what was seen in https://github.com/apache/druid/pull/11204 -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [druid] suneet-s commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

suneet-s commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629580751 ## File path: server/src/main/java/org/apache/druid/server/coordinator/duty/KillDatasourceMetadata.java ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apa

[GitHub] [druid] suneet-s commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

suneet-s commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629576496 ## File path: server/src/main/java/org/apache/druid/server/coordinator/duty/KillDatasourceMetadata.java ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apa

[GitHub] [druid] suneet-s commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

suneet-s commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629573919 ## File path: server/src/main/java/org/apache/druid/server/coordinator/duty/KillDatasourceMetadata.java ## @@ -0,0 +1,110 @@ +/* + * Licensed to the Apa

[GitHub] [druid] suneet-s commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

suneet-s commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629569200 ## File path: server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java ## @@ -1423,4 +1424,40 @@ public boolean insertDa

[GitHub] [druid] suneet-s commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

suneet-s commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629566613 ## File path: server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java ## @@ -1423,4 +1424,40 @@ public boolean insertDa

[GitHub] [druid] suneet-s commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

suneet-s commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629565383 ## File path: server/src/main/java/org/apache/druid/indexing/overlord/IndexerMetadataStorageCoordinator.java ## @@ -267,6 +267,15 @@ SegmentPublishResul

[GitHub] [druid] suneet-s commented on a change in pull request #11227: Add feature to automatically remove datasource metadata based on retention period

suneet-s commented on a change in pull request #11227: URL: https://github.com/apache/druid/pull/11227#discussion_r629564966 ## File path: server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java ## @@ -1423,4 +1424,40 @@ public boolean insertDa

[GitHub] [druid] vogievetsky opened a new pull request #11228: Web console: Fix required field treatment

vogievetsky opened a new pull request #11228: URL: https://github.com/apache/druid/pull/11228 This PR fixes a bug in the data loader flow when connecting Kinesis. The issue is that the `Endpoint` field looks like it is filled in with the default value but it also is `required` so it

[GitHub] [druid] gianm commented on pull request #11201: Add "stringEncoding" parameter to DataSketches HLL.

gianm commented on pull request #11201: URL: https://github.com/apache/druid/pull/11201#issuecomment-836992606 > This is not correct, at least for the HLL in datasketches-java (I'm not sure what the Druid adaptor does). Strings are encoded using UTF-8 and have been for as long as I can rem

[druid] branch master updated (fa113fb -> 691d7a1)

This is an automated email from the ASF dual-hosted git repository. cwylie pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from fa113fb Fix default value (#11220) add 691d7a1 SQL timeseries no longer skip empty buckets with all granularit

[GitHub] [druid] clintropolis merged pull request #11188: SQL timeseries no longer skip empty buckets with all granularity

clintropolis merged pull request #11188: URL: https://github.com/apache/druid/pull/11188 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, p

[GitHub] [druid] clintropolis closed issue #11145: sql count(*) returns no results instead of 0 when no rows meet filter

clintropolis closed issue #11145: URL: https://github.com/apache/druid/issues/11145 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[druid] branch master updated (14f1f2a -> fa113fb)

This is an automated email from the ASF dual-hosted git repository. suneet pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 14f1f2a Fix a broken link in the development doc (#11226) add fa113fb Fix default value (#11220) No new revisi

[GitHub] [druid] suneet-s merged pull request #11220: Fix default value of druid.emitter.http.maxBatchSize in doc

suneet-s merged pull request #11220: URL: https://github.com/apache/druid/pull/11220 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pleas

[GitHub] [druid] jp707049 closed issue #9402: Update the GIFs in README.md

jp707049 closed issue #9402: URL: https://github.com/apache/druid/issues/9402 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please conta

[druid] branch master updated (8647040 -> 14f1f2a)

This is an automated email from the ASF dual-hosted git repository. asdf2014 pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/druid.git. from 8647040 Allow user to set group.id for Kafka ingestion task (#11147) add 14f1f2a Fix a broken link in the dev

[GitHub] [druid] binlijin commented on pull request #11224: PulsarIndexingService and PulsarSchemaRegistryBytesDecoder

binlijin commented on pull request #11224: URL: https://github.com/apache/druid/pull/11224#issuecomment-836345010 I think PartitionIdType/SequenceOffsetType for pulsar should be String/MessageId or String/String like in pr https://github.com/apache/druid/pull/11223 -- This is an automa

[GitHub] [druid] asdf2014 merged pull request #11226: Fix a broken link in the development doc

asdf2014 merged pull request #11226: URL: https://github.com/apache/druid/pull/11226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pleas

[GitHub] [druid] maytasm opened a new pull request #11227: Add feature to automatically remove datasource metadata based on retention period

maytasm opened a new pull request #11227: URL: https://github.com/apache/druid/pull/11227 Add feature to automatically remove datasource metadata based on retention period ### Description We currently already have tasklog auto cleanup (https://github.com/apache/druid/pull/367