[GitHub] [druid] ivan-demchenkov edited a comment on issue #10735: Misleading info about tsColumn in the lookups-cached-global docs

ivan-demchenkov edited a comment on issue #10735: URL: https://github.com/apache/druid/issues/10735#issuecomment-757433050 I thought, and, according to docs, the whole point of having tsColumn is to avoid full reload. If use this to support deletions then it's possible to have a situation

[GitHub] [druid] ivan-demchenkov commented on issue #10735: Misleading info about tsColumn in the lookups-cached-global docs

ivan-demchenkov commented on issue #10735: URL: https://github.com/apache/druid/issues/10735#issuecomment-757433050 I thought, and, according to docs, the whole point of having tsColumn is to avoid full reload. Because then it's possible to have a situation when records were deleted, but n

[GitHub] [druid] vogievetsky merged pull request #10710: fix web-console show json bug

vogievetsky merged pull request #10710: URL: https://github.com/apache/druid/pull/10710 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [druid] gianm commented on pull request #10738: Fix collision between #10689 and #10593.

gianm commented on pull request #10738: URL: https://github.com/apache/druid/pull/10738#issuecomment-756905866 Travis passed. I'll merge this. This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [druid] capistrant commented on pull request #10287: allow the LogUsedSegments duty to be skipped

capistrant commented on pull request #10287: URL: https://github.com/apache/druid/pull/10287#issuecomment-756913585 > I like this log message from LogUsedSegments and have found it very useful: > > ```java > log.info("Found [%,d] used segments.", params.getUsedSegments().size());

[GitHub] [druid] vogievetsky commented on pull request #10710: fix web-console show json bug

vogievetsky commented on pull request #10710: URL: https://github.com/apache/druid/pull/10710#issuecomment-757038204 I think this could be tested by adding a snapshot test to each of the changes to make sure the correct dom is rendered even when there are bigints in the JSON. That would be

[GitHub] [druid] zhangyue19921010 commented on pull request #10737: Add missing unit tests for segment loading in historicals

zhangyue19921010 commented on pull request #10737: URL: https://github.com/apache/druid/pull/10737#issuecomment-757230249 Thanks for adding this UT. LTGM :) This is an automated message from the Apache Git Service. To respond

[GitHub] [druid] zhangyue19921010 commented on pull request #10688: Historical reloads damaged segments automatically when lazy on start.

zhangyue19921010 commented on pull request #10688: URL: https://github.com/apache/druid/pull/10688#issuecomment-757039352 This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [druid] zhangyue19921010 closed pull request #10688: Historical reloads damaged segments automatically when lazy on start.

zhangyue19921010 closed pull request #10688: URL: https://github.com/apache/druid/pull/10688 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [druid] gianm merged pull request #10738: Fix collision between #10689 and #10593.

gianm merged pull request #10738: URL: https://github.com/apache/druid/pull/10738 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th

[GitHub] [druid] gianm commented on pull request #10635: Add SQL functions to format numbers into human readable format

gianm commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757031980 My 2¢ on the naming thing: I like having 3 separate functions, because I think a 1-function model works best if you have a good spec for format strings, which isn't what this patc

[GitHub] [druid] suneet-s commented on pull request #10710: fix web-console show json bug

suneet-s commented on pull request #10710: URL: https://github.com/apache/druid/pull/10710#issuecomment-756933916 > > @bananaaggle Thanks for this fix - can you please add a test that catches parsing these large numbers in the web-console so that future refactorings don't accidentally brea

[GitHub] [druid] zhangyouxun commented on issue #10720: Kafka index task is working, how can not found consumer groups

zhangyouxun commented on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757132078 > How do I set the consumer group name？ > > "spec": { > "ioConfig": { > "type": "kafka", > "consumerProperties": { > "bootstrap.servers": "127.0.0.1:9092", >

[GitHub] [druid] LeeMo2K10 commented on issue #10720: Kafka index task is working, how can not found consumer groups

LeeMo2K10 commented on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757129714 How do I set the consumer group name？ "spec": { "ioConfig": { "type": "kafka", "consumerProperties": { "bootstrap.servers": "127.0.0.1:9092

[GitHub] [druid] gianm commented on pull request #10247: Remove CloseQuietly and migrate its usages to other methods.

gianm commented on pull request #10247: URL: https://github.com/apache/druid/pull/10247#issuecomment-756907541 Fixed up some merge conflicts. This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [druid] gianm commented on a change in pull request #10605: bitwise math function expressions

gianm commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r554224182 ## File path: docs/misc/math-expr.md ## @@ -119,6 +119,13 @@ See javadoc of java.lang.Math for detailed explanation for each function. |acos|acos(x) would

[GitHub] [druid] jihoonson commented on a change in pull request #10677: Add URI based allowPrefexList and denyPrefixList

jihoonson commented on a change in pull request #10677: URL: https://github.com/apache/druid/pull/10677#discussion_r554239833 ## File path: indexing-service/src/main/java/org/apache/druid/indexing/common/task/CompactionTask.java ## @@ -153,6 +154,9 @@ @JsonIgnore private

[GitHub] [druid] gianm commented on pull request #10730: Introduce KafkaRecordEntity to support Kafka headers in InputFormats

gianm commented on pull request #10730: URL: https://github.com/apache/druid/pull/10730#issuecomment-757000160 > looks like tests are failing due to insufficient diff coverage for some lines. Many of them appear to be where we introduced additional type parameters. @gianm what's the guidan

[GitHub] [druid] zhangyue19921010 commented on a change in pull request #10737: Add missing unit tests for segment loading in historicals

zhangyue19921010 commented on a change in pull request #10737: URL: https://github.com/apache/druid/pull/10737#discussion_r554432481 ## File path: server/src/test/java/org/apache/druid/server/coordination/SegmentLoadDropHandlerTest.java ## @@ -29,22 +29,28 @@ import org.apach

[GitHub] [druid] jihoonson merged pull request #10734: add offsetFetchPeriod to kinesis ingestion doc

jihoonson merged pull request #10734: URL: https://github.com/apache/druid/pull/10734 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [druid] xvrl merged pull request #10730: Introduce KafkaRecordEntity to support Kafka headers in InputFormats

xvrl merged pull request #10730: URL: https://github.com/apache/druid/pull/10730 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [druid] jihoonson merged pull request #10670: [Doc-Fix] Remove extra comma in `druid/integration-tests/README.md`

jihoonson merged pull request #10670: URL: https://github.com/apache/druid/pull/10670 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [druid] gianm commented on a change in pull request #10635: Add SQL functions to format numbers into human readable format

gianm commented on a change in pull request #10635: URL: https://github.com/apache/druid/pull/10635#discussion_r554228178 ## File path: docs/querying/sql.md ## @@ -563,6 +563,9 @@ The [DataSketches extension](../development/extensions-core/datasketches-extensi |`COALESCE(valu

[GitHub] [druid] jihoonson commented on a change in pull request #10737: Add missing unit tests for segment loading in historicals

jihoonson commented on a change in pull request #10737: URL: https://github.com/apache/druid/pull/10737#discussion_r554469450 ## File path: server/src/test/java/org/apache/druid/server/coordination/SegmentLoadDropHandlerTest.java ## @@ -29,22 +29,28 @@ import org.apache.druid

[GitHub] [druid] capistrant commented on a change in pull request #10287: allow the LogUsedSegments duty to be skipped

capistrant commented on a change in pull request #10287: URL: https://github.com/apache/druid/pull/10287#discussion_r554117678 ## File path: server/src/main/java/org/apache/druid/server/coordinator/DruidCoordinator.java ## @@ -669,21 +669,31 @@ private void stopBeingLeader()

[GitHub] [druid] FrankChen021 commented on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service

FrankChen021 commented on issue #8715: URL: https://github.com/apache/druid/issues/8715#issuecomment-757150063 @Elvis0607 @kyostyle1 @mjain1994 @LeeMo2K10 Could you tell me how does your kafka platform restrict client access by a pre-assigned group id ? Kafka clients provides two wa

[GitHub] [druid] xvrl commented on pull request #10730: Introduce KafkaRecordEntity to support Kafka headers in InputFormats

xvrl commented on pull request #10730: URL: https://github.com/apache/druid/pull/10730#issuecomment-756995841 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [druid] jihoonson merged pull request #10661: [Minor Doc Fix] Correct the default value of `druid.server.http.gracefulShutdownTimeout`

jihoonson merged pull request #10661: URL: https://github.com/apache/druid/pull/10661 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [druid] zhangyouxun edited a comment on issue #10720: Kafka index task is working, how can not found consumer groups

zhangyouxun edited a comment on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757132078 > How do I set the consumer group name？ > > "spec": { > "ioConfig": { > "type": "kafka", > "consumerProperties": { > "bootstrap.servers": "127.0.0.1:9092"

[GitHub] [druid] FrankChen021 commented on pull request #10635: Add SQL functions to format numbers into human readable format

FrankChen021 commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757192914 > > * IMO the name `decimal_format` is misleading. It sounds like a function that will format a number to a specific amount of decimals. Maybe instead we could call

[GitHub] [druid] jihoonson commented on pull request #10677: Add URI based allowPrefexList and denyPrefixList

jihoonson commented on pull request #10677: URL: https://github.com/apache/druid/pull/10677#issuecomment-757043116 @nishantmonu51 BTW, since we are about to cut a branch for 0.21.0 release (scheduled on the next Monday), do you think we can get this PR merged before the branch cut? I'm ask

[GitHub] [druid] abhishekagarwal87 commented on pull request #10635: Add SQL functions to format numbers into human readable format

abhishekagarwal87 commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757354535 I, for one, don't have a strong opinion. `decimal_format` here is equivalent to https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/text/CompactNumberFor

[GitHub] [druid] jihoonson closed issue #10719: Javascript filter return true when the expression result is false

jihoonson closed issue #10719: URL: https://github.com/apache/druid/issues/10719 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [druid] jihoonson commented on pull request #10661: [Minor Doc Fix] Correct the default value of `druid.server.http.gracefulShutdownTimeout`

jihoonson commented on pull request #10661: URL: https://github.com/apache/druid/pull/10661#issuecomment-757045393 > I guess if we indeed do want to support shutdown without waiting, we might need another config property to toggle [setStopAtShutdown](https://www.eclipse.org/jetty/javadoc/c

[GitHub] [druid] LeeMo2K10 commented on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service

LeeMo2K10 commented on issue #8715: URL: https://github.com/apache/druid/issues/8715#issuecomment-757130246 I have this issue too. This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [druid] jihoonson merged pull request #10721: javascript filter result convert to java boolean

jihoonson merged pull request #10721: URL: https://github.com/apache/druid/pull/10721 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [druid] gianm commented on pull request #10267: DruidInputSource: Fix issues in column projection, timestamp handling.

gianm commented on pull request #10267: URL: https://github.com/apache/druid/pull/10267#issuecomment-756633682 Fixed up some merge conflicts. This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [druid] ayushkul2910 commented on a change in pull request #10448: Added CronScheduler support as a proof to clock drift while emitting metrics

ayushkul2910 commented on a change in pull request #10448: URL: https://github.com/apache/druid/pull/10448#discussion_r553893369 ## File path: server/src/main/java/org/apache/druid/server/metrics/MetricsModule.java ## @@ -108,10 +108,10 @@ public MonitorScheduler getMonitorSch

[GitHub] [druid] gianm merged pull request #9667: Thread-safety for ResponseContext.REGISTERED_KEYS

gianm merged pull request #9667: URL: https://github.com/apache/druid/pull/9667 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [druid] gianm commented on pull request #9667: Thread-safety for ResponseContext.REGISTERED_KEYS

gianm commented on pull request #9667: URL: https://github.com/apache/druid/pull/9667#issuecomment-756625948 Thanks @alex-plekhanov! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [druid] clintropolis commented on a change in pull request #10605: bitwise math function expressions

clintropolis commented on a change in pull request #10605: URL: https://github.com/apache/druid/pull/10605#discussion_r553810003 ## File path: core/src/main/java/org/apache/druid/math/expr/Evals.java ## @@ -71,4 +71,14 @@ public static boolean asBoolean(@Nullable String x) {

[GitHub] [druid] clintropolis commented on a change in pull request #10613: expression filter support for vectorized query engines

clintropolis commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553814648 ## File path: processing/src/main/java/org/apache/druid/segment/DimensionHandlerUtils.java ## @@ -305,17 +305,29 @@ private static ColumnCapabilitie

[GitHub] [druid] FrankChen021 edited a comment on issue #10735: Misleading info about tsColumn in the lookups-cached-global docs

FrankChen021 edited a comment on issue #10735: URL: https://github.com/apache/druid/issues/10735#issuecomment-756784327 I guess why `getLookupPairs` currently reloads the whole table is that if there're records deletions in database, such a full reload could drop those deleted records from

[GitHub] [druid] gianm commented on a change in pull request #10613: expression filter support for vectorized query engines

gianm commented on a change in pull request #10613: URL: https://github.com/apache/druid/pull/10613#discussion_r553782719 ## File path: core/src/main/java/org/apache/druid/math/expr/BinaryLogicalOperatorExpr.java ## @@ -74,7 +74,7 @@ public ExprType getOutputType(InputBindingI

[GitHub] [druid] gianm commented on pull request #10043: Add support for REGEXP_REPLACE

gianm commented on pull request #10043: URL: https://github.com/apache/druid/pull/10043#issuecomment-756624149 > one more time please stalebot 🙏 two more times? 🙂 This is an automated message from the Apache Git Servic

[GitHub] [druid] gianm closed issue #9106: Unbalanced synchronization of ResponseContext.REGISTERED_KEYS

gianm closed issue #9106: URL: https://github.com/apache/druid/issues/9106 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [druid] liran-funaro commented on pull request #10593: IncrementalIndex Tests and Benchmarks Parametrization

liran-funaro commented on pull request #10593: URL: https://github.com/apache/druid/pull/10593#issuecomment-756795145 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [druid] stale[bot] commented on pull request #10043: Add support for REGEXP_REPLACE

stale[bot] commented on pull request #10043: URL: https://github.com/apache/druid/pull/10043#issuecomment-756624168 This issue is no longer marked as stale. This is an automated message from the Apache Git Service. To res

[GitHub] [druid] gianm commented on pull request #10287: allow the LogUsedSegments duty to be skipped

gianm commented on pull request #10287: URL: https://github.com/apache/druid/pull/10287#issuecomment-756630739 > circling back on this. my cluster with 161k segments takes ~20ms to execute `LogUsedSegments`. This duty should scale fairly linearly with number of segments so you'd have to 50

[GitHub] [druid] FrankChen021 commented on issue #10735: Misleading info about tsColumn in the lookups-cached-global docs

FrankChen021 commented on issue #10735: URL: https://github.com/apache/druid/issues/10735#issuecomment-756784327 I guess why `getLookupPairs` currently reload the whole table is that if there're records deletions in database, such a full reload could drop those deleted records from cache.

[GitHub] [druid] jihoonson commented on a change in pull request #10737: Add missing unit tests for segment loading in historicals

jihoonson commented on a change in pull request #10737: URL: https://github.com/apache/druid/pull/10737#discussion_r554469450 ## File path: server/src/test/java/org/apache/druid/server/coordination/SegmentLoadDropHandlerTest.java ## @@ -29,22 +29,28 @@ import org.apache.druid

[GitHub] [druid] abhishekagarwal87 commented on pull request #10635: Add SQL functions to format numbers into human readable format

abhishekagarwal87 commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757354535 I, for one, don't have a strong opinion. `decimal_format` here is equivalent to https://docs.oracle.com/en/java/javase/14/docs/api/java.base/java/text/CompactNumberFor

[GitHub] [druid] zhangyue19921010 commented on a change in pull request #10737: Add missing unit tests for segment loading in historicals

zhangyue19921010 commented on a change in pull request #10737: URL: https://github.com/apache/druid/pull/10737#discussion_r554432481 ## File path: server/src/test/java/org/apache/druid/server/coordination/SegmentLoadDropHandlerTest.java ## @@ -29,22 +29,28 @@ import org.apach

[GitHub] [druid] zhangyue19921010 commented on a change in pull request #10737: Add missing unit tests for segment loading in historicals

zhangyue19921010 commented on a change in pull request #10737: URL: https://github.com/apache/druid/pull/10737#discussion_r554432481 ## File path: server/src/test/java/org/apache/druid/server/coordination/SegmentLoadDropHandlerTest.java ## @@ -29,22 +29,28 @@ import org.apach

[GitHub] [druid] zhangyue19921010 commented on pull request #10737: Add missing unit tests for segment loading in historicals

zhangyue19921010 commented on pull request #10737: URL: https://github.com/apache/druid/pull/10737#issuecomment-757230249 Thanks for adding this UT. LTGM :) This is an automated message from the Apache Git Service. To respond

[GitHub] [druid] zhangyue19921010 commented on pull request #10688: Historical reloads damaged segments automatically when lazy on start.

zhangyue19921010 commented on pull request #10688: URL: https://github.com/apache/druid/pull/10688#issuecomment-757223194 CI passed. PTAL :) This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [druid] FrankChen021 commented on pull request #10635: Add SQL functions to format numbers into human readable format

FrankChen021 commented on pull request #10635: URL: https://github.com/apache/druid/pull/10635#issuecomment-757192914 > > * IMO the name `decimal_format` is misleading. It sounds like a function that will format a number to a specific amount of decimals. Maybe instead we could call

[GitHub] [druid] FrankChen021 commented on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service

FrankChen021 commented on issue #8715: URL: https://github.com/apache/druid/issues/8715#issuecomment-757150063 @Elvis0607 @kyostyle1 @mjain1994 @LeeMo2K10 Could you tell me how does your kafka platform restrict client access by a pre-assigned group id ? Kafka clients provides two wa

[GitHub] [druid] zhangyouxun edited a comment on issue #10720: Kafka index task is working, how can not found consumer groups

zhangyouxun edited a comment on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757132078 > How do I set the consumer group name？ > > "spec": { > "ioConfig": { > "type": "kafka", > "consumerProperties": { > "bootstrap.servers": "127.0.0.1:9092"

[GitHub] [druid] zhangyouxun commented on issue #10720: Kafka index task is working, how can not found consumer groups

zhangyouxun commented on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757132078 > How do I set the consumer group name？ > > "spec": { > "ioConfig": { > "type": "kafka", > "consumerProperties": { > "bootstrap.servers": "127.0.0.1:9092", >

[GitHub] [druid] LeeMo2K10 commented on issue #8715: "group.id" of ConsumerPropertie be replaced by "kafka-supervisor-%s" in kafka-indexing-service

LeeMo2K10 commented on issue #8715: URL: https://github.com/apache/druid/issues/8715#issuecomment-757130246 I have this issue too. This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [druid] LeeMo2K10 commented on issue #10720: Kafka index task is working, how can not found consumer groups

LeeMo2K10 commented on issue #10720: URL: https://github.com/apache/druid/issues/10720#issuecomment-757129714 How do I set the consumer group name？ "spec": { "ioConfig": { "type": "kafka", "consumerProperties": { "bootstrap.servers": "127.0.0.1:9092

[GitHub] [druid] zhangyue19921010 commented on pull request #10688: Historical reloads damaged segments automatically when lazy on start.

zhangyue19921010 commented on pull request #10688: URL: https://github.com/apache/druid/pull/10688#issuecomment-757117820 Hi @kaijianding and @clintropolis. Sorry to bother you guys. Code coverage(job 14 to job 16) is passed in CI which means all the change requested is finished. Please ta