Re: [PR] Do not kill segments with referenced load specs from deep storage (druid)

AmatyaAvadhanula commented on code in PR #16667: URL: https://github.com/apache/druid/pull/16667#discussion_r1671717185 ## server/src/main/java/org/apache/druid/metadata/IndexerSQLMetadataStorageCoordinator.java: ## @@ -2217,14 +2230,20 @@ private Set createNewIdsOfAppendSegmen

Re: [PR] Do not kill segments with referenced load specs from deep storage (druid)

AmatyaAvadhanula commented on code in PR #16667: URL: https://github.com/apache/druid/pull/16667#discussion_r1671716820 ## indexing-service/src/main/java/org/apache/druid/indexing/common/task/KillUnusedSegmentsTask.java: ## @@ -231,16 +234,57 @@ public TaskStatus runTask(TaskToo

(druid) branch master updated: Fix NPE in CompactSegments (#16713)

This is an automated email from the ASF dual-hosted git repository. asdf2014 pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new 616ae631c62 Fix NPE in CompactSegments (#16713)

Re: [PR] Fix NPE in CompactSegments (druid)

asdf2014 merged PR #16713: URL: https://github.com/apache/druid/pull/16713 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.apa

Re: [PR] Deserialize complex dimensions in group by queries to their respective types when reading from spilled files and cached results (druid)

LakshSingla commented on code in PR #16620: URL: https://github.com/apache/druid/pull/16620#discussion_r1671700888 ## processing/src/test/java/org/apache/druid/query/groupby/GroupByQueryQueryToolChestTest.java: ## @@ -97,6 +98,8 @@ public class GroupByQueryQueryToolChestTest ext

Re: [PR] Deserialize complex dimensions in group by queries to their respective types when reading from spilled files and cached results (druid)

LakshSingla commented on code in PR #16620: URL: https://github.com/apache/druid/pull/16620#discussion_r1671693644 ## processing/src/main/java/org/apache/druid/query/groupby/GroupByQueryQueryToolChest.java: ## @@ -727,13 +716,33 @@ public ResultRow apply(Object input)

Re: [PR] Deserialize complex dimensions in group by queries to their respective types when reading from spilled files and cached results (druid)

clintropolis commented on code in PR #16620: URL: https://github.com/apache/druid/pull/16620#discussion_r1671675273 ## processing/src/main/java/org/apache/druid/query/groupby/GroupByQueryQueryToolChest.java: ## @@ -727,13 +716,33 @@ public ResultRow apply(Object input)

Re: [PR] Track and emit segment loading rate for HttpLoadQueuePeon on Coordinator (druid)

kfaraz commented on PR #16691: URL: https://github.com/apache/druid/pull/16691#issuecomment-2219630727 @abhishekrb19 , I have updated the docs, description in the PR. Please let me know what you think. -- This is an automated message from the Apache Git Service. To respond to the messag

Re: [PR] Preemptive restriction for queries with approximate count distinct on complex columns of unsupported type (druid)

LakshSingla commented on code in PR #16682: URL: https://github.com/apache/druid/pull/16682#discussion_r1671620773 ## extensions-core/datasketches/src/main/java/org/apache/druid/query/aggregation/datasketches/theta/SketchAggregatorFactory.java: ## @@ -104,20 +99,40 @@ public Agg

Re: [PR] Preemptive restriction for queries with approximate count distinct on complex columns of unsupported type (druid)

LakshSingla commented on code in PR #16682: URL: https://github.com/apache/druid/pull/16682#discussion_r1671612578 ## extensions-core/datasketches/src/main/java/org/apache/druid/query/aggregation/datasketches/hll/HllSketchMergeAggregatorFactory.java: ## @@ -142,6 +150,26 @@ publ

Re: [PR] Fix NPE in CompactSegments (druid)

abhishekrb19 commented on code in PR #16713: URL: https://github.com/apache/druid/pull/16713#discussion_r1671621189 ## server/src/main/java/org/apache/druid/server/coordinator/duty/CompactSegments.java: ## @@ -234,12 +234,13 @@ private boolean cancelTaskIfGranularityChanged(

Re: [PR] Fix NPE in CompactSegments (druid)

kfaraz commented on code in PR #16713: URL: https://github.com/apache/druid/pull/16713#discussion_r1671618431 ## server/src/main/java/org/apache/druid/server/coordinator/duty/CompactSegments.java: ## @@ -234,12 +234,13 @@ private boolean cancelTaskIfGranularityChanged( Gran

Re: [PR] Fix NPE in CompactSegments (druid)

abhishekrb19 commented on code in PR #16713: URL: https://github.com/apache/druid/pull/16713#discussion_r1671612135 ## server/src/main/java/org/apache/druid/server/coordinator/duty/CompactSegments.java: ## @@ -234,12 +234,13 @@ private boolean cancelTaskIfGranularityChanged(

Re: [PR] Fix NPE in CompactSegments (druid)

abhishekrb19 commented on code in PR #16713: URL: https://github.com/apache/druid/pull/16713#discussion_r1671612135 ## server/src/main/java/org/apache/druid/server/coordinator/duty/CompactSegments.java: ## @@ -234,12 +234,13 @@ private boolean cancelTaskIfGranularityChanged(

Re: [PR] Preemptive restriction for queries with approximate count distinct on complex columns of unsupported type (druid)

LakshSingla commented on code in PR #16682: URL: https://github.com/apache/druid/pull/16682#discussion_r1671608608 ## extensions-core/datasketches/src/main/java/org/apache/druid/query/aggregation/datasketches/hll/HllSketchMergeAggregatorFactory.java: ## @@ -142,6 +151,22 @@ publ

Re: [I] Coordinator is not calulating compaction taskSlots correctly (druid)

kfaraz commented on issue #16694: URL: https://github.com/apache/druid/issues/16694#issuecomment-2219472740 Possible duplicate of #15847 . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[PR] Fix NPE in CompactSegments (druid)

kfaraz opened a new pull request, #16713: URL: https://github.com/apache/druid/pull/16713 The NPE causes `CompactSegments` duty to fail and thus not submit any compaction tasks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [PR] Specify node type so that the log filename can get resolved (druid)

Subhashini2610 commented on PR #16282: URL: https://github.com/apache/druid/pull/16282#issuecomment-2219395410 @fectrain @asdf2014 Thanks for this fix!!! Please let me know in which Druid release would this be present? I somehow do not see it in 30.0.0 -- This is an automated message from

Re: [PR] docs: Adding admonition box to warn about MVD (druid)

asdf2014 commented on code in PR #16712: URL: https://github.com/apache/druid/pull/16712#discussion_r1671460501 ## docs/querying/lookups.md: ## @@ -50,6 +50,10 @@ Other lookup types are available as extensions, including: - Globally cached lookups from local files, remote URIs,

Re: [PR] docs: Adding admonition box to warn about MVD (druid)

vtlim commented on code in PR #16712: URL: https://github.com/apache/druid/pull/16712#discussion_r1671395525 ## docs/querying/lookups.md: ## @@ -50,6 +50,10 @@ Other lookup types are available as extensions, including: - Globally cached lookups from local files, remote URIs, or

[PR] docs: Adding admonition box to warn about MVD (druid)

edgar2020 opened a new pull request, #16712: URL: https://github.com/apache/druid/pull/16712 ### Description Adding an admonition box to warn the User that MVDs are not supported as keys in lookups This PR has: - [X] been self-reviewed. -- This is an automated message fro

Re: [PR] Bump commons-io:commons-io from 2.11.0 to 2.16.1 (druid)

github-actions[bot] commented on PR #16255: URL: https://github.com/apache/druid/pull/16255#issuecomment-2219123106 This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this

Re: [I] Explore UI should have a progress bar and ability to cancel background query (druid)

github-actions[bot] closed issue #14941: Explore UI should have a progress bar and ability to cancel background query URL: https://github.com/apache/druid/issues/14941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

Re: [PR] Fixing corner case when only one of upper or lower range is unbounded… (druid)

github-actions[bot] closed pull request #16154: Fixing corner case when only one of upper or lower range is unbounded… URL: https://github.com/apache/druid/pull/16154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Fixing corner case when only one of upper or lower range is unbounded… (druid)

github-actions[bot] commented on PR #16154: URL: https://github.com/apache/druid/pull/16154#issuecomment-2219122922 This pull request/issue has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. -

Re: [I] Explore UI should have a progress bar and ability to cancel background query (druid)

github-actions[bot] commented on issue #14941: URL: https://github.com/apache/druid/issues/14941#issuecomment-2219122444 This issue has been closed due to lack of activity. If you think that is incorrect, or the issue requires additional review, you can revive the issue at any time.

Re: [PR] kubernetes-overlord-extension: Fix tasks not being shutdown (druid)

github-advanced-security[bot] commented on code in PR #16711: URL: https://github.com/apache/druid/pull/16711#discussion_r1671299418 ## extensions-contrib/kubernetes-overlord-extensions/src/main/java/org/apache/druid/k8s/overlord/KubernetesPeonLifecycle.java: ## @@ -254,7 +254,1

[PR] kubernetes-overlord-extension: Fix tasks not being shutdown (druid)

adithyachakilam opened a new pull request, #16711: URL: https://github.com/apache/druid/pull/16711 ### Description In rare cases when fabric client is unable to talk to the k8s server, TaskQueue is skipping the complete shutdown logic because of the exception thrown during figuring t

Re: [PR] Auto-Compaction using Multi-Stage Query Engine (druid)

github-advanced-security[bot] commented on code in PR #16291: URL: https://github.com/apache/druid/pull/16291#discussion_r1670986384 ## server/src/test/java/org/apache/druid/server/http/CoordinatorCompactionConfigsResourceTest.java: ## @@ -401,6 +412,115 @@ Assert.assertNot

[I] Provide/publish docker images based on debian:trixie? (druid)

frankgrimes97 opened a new issue, #16710: URL: https://github.com/apache/druid/issues/16710 We've noticed that there are many HIGH and CRITICAL vulnerabilities reported (as per our Trivy scans) in the latest base Druid image and were wondering if it might make sense to consider moving up fr

Re: [PR] Auto-Compaction using Multi-Stage Query Engine (druid)

gargvishesh commented on PR #16291: URL: https://github.com/apache/druid/pull/16291#issuecomment-2218356161 @kfaraz @LakshSingla Thanks for your comments -- I do see the usefulness of the UTs for the entire MSQ-based compaction flow. The existing tests in `CompactionTaskRunTest` however

Re: [PR] Auto-Compaction using Multi-Stage Query Engine (druid)

gargvishesh commented on code in PR #16291: URL: https://github.com/apache/druid/pull/16291#discussion_r1670952975 ## server/src/test/java/org/apache/druid/server/coordinator/DataSourceCompactionConfigTest.java: ## @@ -69,6 +69,7 @@ public void testSerdeBasic() throws IOExceptio

Re: [PR] Auto-Compaction using Multi-Stage Query Engine (druid)

gargvishesh commented on code in PR #16291: URL: https://github.com/apache/druid/pull/16291#discussion_r1670952343 ## server/src/test/java/org/apache/druid/server/coordinator/duty/CompactSegmentsTest.java: ## @@ -2181,7 +2199,7 @@ public void testFindMaxNumTaskSlotsUsedByOneCom

Re: [PR] Auto-Compaction using Multi-Stage Query Engine (druid)

gargvishesh commented on code in PR #16291: URL: https://github.com/apache/druid/pull/16291#discussion_r1670951215 ## server/src/test/java/org/apache/druid/server/http/CoordinatorCompactionConfigsResourceTest.java: ## @@ -24,6 +24,7 @@ import org.apache.druid.audit.AuditManager

Re: [PR] Auto-Compaction using Multi-Stage Query Engine (druid)

gargvishesh commented on code in PR #16291: URL: https://github.com/apache/druid/pull/16291#discussion_r1670950223 ## server/src/main/java/org/apache/druid/server/coordinator/compact/CompactionStatus.java: ## @@ -305,17 +305,30 @@ private CompactionStatus metricsSpecIsUpToDate()

Re: [PR] Auto-Compaction using Multi-Stage Query Engine (druid)

gargvishesh commented on code in PR #16291: URL: https://github.com/apache/druid/pull/16291#discussion_r1670949647 ## server/src/main/java/org/apache/druid/server/coordinator/duty/CompactSegments.java: ## @@ -296,6 +314,16 @@ static int findMaxNumTaskSlotsUsedByOneCompactionTas

Re: [PR] Refactor SemanticCreator (druid)

github-advanced-security[bot] commented on code in PR #16700: URL: https://github.com/apache/druid/pull/16700#discussion_r1670887098 ## processing/src/test/java/org/apache/druid/common/semantic/SemanticUtilsTest.java: ## @@ -0,0 +1,134 @@ +/* + * Licensed to the Apache Software

Re: [PR] Add a segmentMorphFactory to MSQ Datasource Destination (druid)

cryptoe commented on code in PR #16699: URL: https://github.com/apache/druid/pull/16699#discussion_r1670654147 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/sql/MSQTaskQueryMaker.java: ## @@ -246,7 +246,8 @@ public QueryResponse runQuery(final DruidQuer

Re: [PR] Add a segmentMorphFactory to MSQ Datasource Destination (druid)

cryptoe commented on code in PR #16699: URL: https://github.com/apache/druid/pull/16699#discussion_r1670654611 ## extensions-core/multi-stage-query/src/main/java/org/apache/druid/msq/sql/MSQTaskQueryMaker.java: ## @@ -246,7 +246,8 @@ public QueryResponse runQuery(final DruidQuer

[I] Ingestion using systemFields results in exception (druid)

petermarshallio opened a new issue, #16709: URL: https://github.com/apache/druid/issues/16709 Running the following SQL-based ingestion, which includes systemFields, returns an exception `org.apache.druid.segment.transform.TransformedInputRow and org.apache.druid.data.input.ListBasedInputRo

Re: [PR] Correct aggregators violating names (druid)

kgyrtkirk commented on code in PR #16615: URL: https://github.com/apache/druid/pull/16615#discussion_r1644026933 ## sql/src/main/java/org/apache/druid/sql/calcite/rel/Windowing.java: ## @@ -172,7 +172,9 @@ public static Windowing fromCalciteStuff( throw new CannotBu

Re: [PR] Track and emit segment loading rate for HttpLoadQueuePeon on Coordinator (druid)

kfaraz commented on PR #16691: URL: https://github.com/apache/druid/pull/16691#issuecomment-2217317232 Update: Logic is fixed, need to test it out a little more. Once that is done, I will update the description and documentation here. -- This is an automated message from the Apache

Re: [PR] fix issues with join filter pushdown and virtual column resolution (druid)

asdf2014 commented on code in PR #16702: URL: https://github.com/apache/druid/pull/16702#discussion_r1670196490 ## processing/src/main/java/org/apache/druid/segment/VirtualColumns.java: ## @@ -134,6 +138,16 @@ public static boolean shouldVectorize(Query query, VirtualColumns vi

Re: [PR] fix issues with join filter pushdown and virtual column resolution (druid)

clintropolis commented on code in PR #16702: URL: https://github.com/apache/druid/pull/16702#discussion_r1670100506 ## processing/src/main/java/org/apache/druid/segment/join/JoinVirtualColumnSplit.java: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation (ASF

Re: [PR] Add logging for sketches on workers (druid)

LakshSingla commented on PR #16697: URL: https://github.com/apache/druid/pull/16697#issuecomment-2217090316 Thanks for the patch! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific com

(druid) branch master updated: Add logging for sketches on workers (#16697)

This is an automated email from the ASF dual-hosted git repository. lakshsingla pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new 7c625356c52 Add logging for sketches on work

Re: [PR] Add logging for sketches on workers (druid)

LakshSingla merged PR #16697: URL: https://github.com/apache/druid/pull/16697 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.

Re: [PR] fix issues with join filter pushdown and virtual column resolution (druid)

abhishekagarwal87 commented on code in PR #16702: URL: https://github.com/apache/druid/pull/16702#discussion_r1670076052 ## processing/src/main/java/org/apache/druid/segment/join/JoinVirtualColumnSplit.java: ## @@ -0,0 +1,148 @@ +/* + * Licensed to the Apache Software Foundation

(druid) branch master updated: Fixes a bug when running queries with a limit clause (#16643)

This is an automated email from the ASF dual-hosted git repository. lakshsingla pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/druid.git The following commit(s) were added to refs/heads/master by this push: new af5399cd9db Fixes a bug when running queries

Re: [PR] Fixes a bug when running queries with a limit clause (druid)

LakshSingla merged PR #16643: URL: https://github.com/apache/druid/pull/16643 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@druid.

Re: [PR] Add a segmentMorphFactory to MSQ Datasource Destination (druid)

adarshsanjeev commented on PR #16692: URL: https://github.com/apache/druid/pull/16692#issuecomment-2216872757 Created these PRs: https://github.com/apache/druid/pull/16698 https://github.com/apache/druid/pull/16699 https://github.com/apache/druid/pull/16700 https://github.com/apac

Re: [PR] WriteOutBytes improvements (druid)

clintropolis commented on code in PR #16698: URL: https://github.com/apache/druid/pull/16698#discussion_r1669873534 ## processing/src/main/java/org/apache/druid/segment/writeout/TmpFileSegmentWriteOutMedium.java: ## @@ -43,21 +69,41 @@ public final class TmpFileSegmentWriteOutMe

Re: [PR] WriteOutBytes improvements (druid)

clintropolis commented on code in PR #16698: URL: https://github.com/apache/druid/pull/16698#discussion_r1669868972 ## processing/src/main/java/org/apache/druid/segment/writeout/FileWriteOutBytes.java: ## @@ -33,22 +33,25 @@ import java.nio.channels.FileChannel; import java.ni

Re: [PR] Bug fix for array type selector causing array aggregation over window frame fail (druid)

clintropolis commented on code in PR #16653: URL: https://github.com/apache/druid/pull/16653#discussion_r1669861537 ## processing/src/main/java/org/apache/druid/segment/virtual/ExpressionSelectors.java: ## @@ -492,11 +492,26 @@ static Supplier supplierFromObjectSelector(