techdocsmith commented on code in PR #19945: URL: https://github.com/apache/druid/pull/19945#discussion_r3787327874
########## docs/release-info/release-notes.md: ########## @@ -57,6 +57,80 @@ For tips about how to write a good release note, see [Release notes](https://git This section contains important information about new and existing features. +#### Java + +Druid now supports Java 25. While Druid 21 is still supported, we recommend you upgrade to Java 25. Review Comment: ```suggestion Druid now supports Java 25. While Java 21 is still supported, we recommend you upgrade to Java 25. ``` Should be Java? See line 435 ########## docs/release-info/release-notes.md: ########## @@ -57,6 +57,80 @@ For tips about how to write a good release note, see [Release notes](https://git This section contains important information about new and existing features. +#### Java + +Druid now supports Java 25. While Druid 21 is still supported, we recommend you upgrade to Java 25. + +Support for Java 17 has been dropped. + +[#19304](https://github.com/apache/druid/pull/19304) [#19336](https://github.com/apache/druid/pull/19336) + +#### Historical tier aliases + +You can now use the `historicalTierAliases` Coordinator dynamic configuration to map a virtual tier name to a set of real Historical tiers so that a group of Historical tiers has a single identifier. When a load/drop rule references the alias, the Coordinator replaces it with the actual tiers. For example, if you map the Historical tiers `hot_1` and `hot_2` to the alias `hot`, the rule `{"hot": 2}` loads 2 replicas of each onto `hot_1` and `hot_2`. + +[#19204](https://github.com/apache/druid/pull/19204) [#19667](https://github.com/apache/druid/pull/19667) + +#### New load rule types + +Adds a new family of retention rules, `loadPartialByPeriod`, `loadPartialByInterval`, `loadPartialForever`, laying the groundwork for partial loading of version 10 segment projections on Historicals. + +[#19374](https://github.com/apache/druid/pull/19374) + +#### Realtime segments query context Review Comment: ```suggestion #### Realtime segments mode query context ``` ########## docs/release-info/release-notes.md: ########## @@ -65,42 +139,389 @@ This section contains detailed release notes separated by areas. #### Other web console improvements +- Added the following status details to the **Services** view for Historical services: cloning from another Historical, in turbo loading mode, in decommissioning mode [#19253](https://github.com/apache/druid/pull/19253) +- Added support for resetting a supervisor to the latest offsets and backfilling [#19533](https://github.com/apache/druid/pull/19533) +- Improved how new tabs are handled [#19483](https://github.com/apache/druid/pull/19483) Review Comment: ```suggestion - Improved handling for new tabs [#19483](https://github.com/apache/druid/pull/19483) ``` ########## docs/release-info/release-notes.md: ########## @@ -65,42 +139,389 @@ This section contains detailed release notes separated by areas. #### Other web console improvements +- Added the following status details to the **Services** view for Historical services: cloning from another Historical, in turbo loading mode, in decommissioning mode [#19253](https://github.com/apache/druid/pull/19253) +- Added support for resetting a supervisor to the latest offsets and backfilling [#19533](https://github.com/apache/druid/pull/19533) +- Improved how new tabs are handled [#19483](https://github.com/apache/druid/pull/19483) +- Improved the Home view's **Services** card. It now reports Overlord, Coordinator, Router, Broker, and Indexer counts on clusters where the web console talks to the Coordinator without SQL access [#19481](https://github.com/apache/druid/pull/19481) + ### Ingestion +* You can now use the expression aggregator at ingestion time for expressions that produce a LONG or DOUBLE for both fold and combine expressions [#19508](https://github.com/apache/druid/pull/19508) +* Added `now()` expression function that returns the current system timestamp in milliseconds since epoch. Useful at ingestion time for troubleshooting pipeline delays (e.g., `now() - __time`). Note: `now()` is non-deterministic as it evaluates for every row, so it can break idempotency. This can be added to any besides `__time` [#19386](https://github.com/apache/druid/pull/19386) +* Improved resiliency when ingesting from S3. Druid now retries on `SSLException` and transient credential errors instead of failing [#19617](https://github.com/apache/druid/pull/19617) [#19558](https://github.com/apache/druid/pull/19558) +* Improved S3 performance [#19394](https://github.com/apache/druid/pull/19394) +* Updated the default S3 connection pool size so that it's computed based on the number of available processors [#19536](https://github.com/apache/druid/pull/19536) + #### SQL-based ingestion +##### EXTERN for S3 now supports role ARN + +You can now include the role ARN when running an INSERT INTO EXTERN query. For example: + +``` +INSERT INTO +EXTERN( + s3(bucket => 'test2', prefix => 'export', assumeRoleArn => 'arn:aws:iam::00000:role/test-20260520')) +AS CSV +SELECT ... +``` + +[#19317](https://github.com/apache/druid/pull/19317) + ##### Other SQL-based ingestion improvements +- Added storage counters for the amount of bytes and files written to local and durable storage as well as the state of the local `ByteTracker`. Per-worker storage counters have also been added to the web console [#19316](https://github.com/apache/druid/pull/19316) + #### Streaming ingestion +##### Scaling cool down + +You can now configure different cool downs for scaling up and scaling down streaming task autoscalers. Review Comment: ```suggestion You can now configure different cool down durations for scaling up and scaling down streaming task autoscalers. ``` ########## docs/release-info/release-notes.md: ########## @@ -57,6 +57,80 @@ For tips about how to write a good release note, see [Release notes](https://git This section contains important information about new and existing features. +#### Java + +Druid now supports Java 25. While Druid 21 is still supported, we recommend you upgrade to Java 25. + +Support for Java 17 has been dropped. + +[#19304](https://github.com/apache/druid/pull/19304) [#19336](https://github.com/apache/druid/pull/19336) + +#### Historical tier aliases + +You can now use the `historicalTierAliases` Coordinator dynamic configuration to map a virtual tier name to a set of real Historical tiers so that a group of Historical tiers has a single identifier. When a load/drop rule references the alias, the Coordinator replaces it with the actual tiers. For example, if you map the Historical tiers `hot_1` and `hot_2` to the alias `hot`, the rule `{"hot": 2}` loads 2 replicas of each onto `hot_1` and `hot_2`. Review Comment: ```suggestion You can map a virtual tier name to a number of real Historical tiers with the `historicalTierAliases` Coordinator dynamic configuration. This creates a single identifier for a group of Historical tiers. When a load/drop rule references the alias, the Coordinator replaces it with the actual tiers. For example, if you map the Historical tiers `hot_1` and `hot_2` to the alias `hot`, the rule `{"hot": 2}` loads 2 replicas of each onto `hot_1` and `hot_2`. ``` ########## docs/release-info/release-notes.md: ########## @@ -65,42 +139,389 @@ This section contains detailed release notes separated by areas. #### Other web console improvements +- Added the following status details to the **Services** view for Historical services: cloning from another Historical, in turbo loading mode, in decommissioning mode [#19253](https://github.com/apache/druid/pull/19253) +- Added support for resetting a supervisor to the latest offsets and backfilling [#19533](https://github.com/apache/druid/pull/19533) +- Improved how new tabs are handled [#19483](https://github.com/apache/druid/pull/19483) +- Improved the Home view's **Services** card. It now reports Overlord, Coordinator, Router, Broker, and Indexer counts on clusters where the web console talks to the Coordinator without SQL access [#19481](https://github.com/apache/druid/pull/19481) + ### Ingestion +* You can now use the expression aggregator at ingestion time for expressions that produce a LONG or DOUBLE for both fold and combine expressions [#19508](https://github.com/apache/druid/pull/19508) +* Added `now()` expression function that returns the current system timestamp in milliseconds since epoch. Useful at ingestion time for troubleshooting pipeline delays (e.g., `now() - __time`). Note: `now()` is non-deterministic as it evaluates for every row, so it can break idempotency. This can be added to any besides `__time` [#19386](https://github.com/apache/druid/pull/19386) +* Improved resiliency when ingesting from S3. Druid now retries on `SSLException` and transient credential errors instead of failing [#19617](https://github.com/apache/druid/pull/19617) [#19558](https://github.com/apache/druid/pull/19558) +* Improved S3 performance [#19394](https://github.com/apache/druid/pull/19394) +* Updated the default S3 connection pool size so that it's computed based on the number of available processors [#19536](https://github.com/apache/druid/pull/19536) + #### SQL-based ingestion +##### EXTERN for S3 now supports role ARN + +You can now include the role ARN when running an INSERT INTO EXTERN query. For example: + +``` +INSERT INTO +EXTERN( + s3(bucket => 'test2', prefix => 'export', assumeRoleArn => 'arn:aws:iam::00000:role/test-20260520')) +AS CSV +SELECT ... +``` + +[#19317](https://github.com/apache/druid/pull/19317) + ##### Other SQL-based ingestion improvements +- Added storage counters for the amount of bytes and files written to local and durable storage as well as the state of the local `ByteTracker`. Per-worker storage counters have also been added to the web console [#19316](https://github.com/apache/druid/pull/19316) + #### Streaming ingestion +##### Scaling cool down + +You can now configure different cool downs for scaling up and scaling down streaming task autoscalers. + +[#19286](https://github.com/apache/druid/pull/19286) + +##### Improved supervisor restarts + +Supervisors no longer restart for all changes. Based on the type of change, one of the following can occur: + +- The updated spec is persisted without a restart +- The supervisor is restarted but running tasks aren't impacted +- The supervisor is restarted and its tasks are terminated (the default behavior prior to this change) + +For example, cosmetic changes to a supervisor spec no longer trigger a restart. + +[#19700](https://github.com/apache/druid/pull/19700) [#19720](https://github.com/apache/druid/pull/19720) + +Additionally, the algorithm for determining a change in the spec has been improved. For example, changes to `ioConfig.taskCount` don't trigger a supervisor restart if auto-scaling is enabled. + +[#19541](https://github.com/apache/druid/pull/19541) + +###### Latest offset and backfill + +For Kafka and RabbitMQ, you can now reset a supervisor to the latest offset and start a new bounded backfill supervisor to ingest data from the skipped range. This is a useful feature for operating Druid clusters where the most recent data is the most important, such as for alerting. + +Note the following requirements: + +- The supervisor's `useEarliestSequenceNumber` property must be `false`. +- The supervisor context must have `useConcurrentLocks` set to `true` to allow the backfill supervisor's tasks to write concurrently with the main supervisor's tasks. +- The supervisor must be in a `RUNNING` state. + +Use `POST` `/druid/indexer/v1/supervisor/{supervisorId}/resetToLatestAndBackfill` or the web console to perform this action. + +[#19477](https://github.com/apache/druid/pull/19477) + +##### Prunable shard specs for streaming published segments + +Kafka ingestion can now publish segments that the Broker prunes at query time without waiting for compaction. Set `tuningConfig.streamingPartitionsSpec.partitionDimensions` to a list of low-to-medium cardinality dimensions; each task records +the distinct values it observes per dimension and stamps them onto a new `dim_value_set` shard spec. Queries that filter on a declared dimension then skip segments whose values can't match. Review Comment: ```suggestion Kafka ingestion can now publish segments that the Broker prunes at query time without waiting for compaction. Set `tuningConfig.streamingPartitionsSpec.partitionDimensions` to a list of low-to-medium cardinality dimensions; each task records the distinct values it observes per dimension and stamps them onto a new `dim_value_set` shard spec. Queries that filter on a declared dimension then skip segments whose values don't match. ``` odd line break. "values can't match" reads awkwardly ########## docs/release-info/release-notes.md: ########## @@ -65,42 +139,389 @@ This section contains detailed release notes separated by areas. #### Other web console improvements +- Added the following status details to the **Services** view for Historical services: cloning from another Historical, in turbo loading mode, in decommissioning mode [#19253](https://github.com/apache/druid/pull/19253) +- Added support for resetting a supervisor to the latest offsets and backfilling [#19533](https://github.com/apache/druid/pull/19533) +- Improved how new tabs are handled [#19483](https://github.com/apache/druid/pull/19483) +- Improved the Home view's **Services** card. It now reports Overlord, Coordinator, Router, Broker, and Indexer counts on clusters where the web console talks to the Coordinator without SQL access [#19481](https://github.com/apache/druid/pull/19481) + ### Ingestion +* You can now use the expression aggregator at ingestion time for expressions that produce a LONG or DOUBLE for both fold and combine expressions [#19508](https://github.com/apache/druid/pull/19508) +* Added `now()` expression function that returns the current system timestamp in milliseconds since epoch. Useful at ingestion time for troubleshooting pipeline delays (e.g., `now() - __time`). Note: `now()` is non-deterministic as it evaluates for every row, so it can break idempotency. This can be added to any besides `__time` [#19386](https://github.com/apache/druid/pull/19386) +* Improved resiliency when ingesting from S3. Druid now retries on `SSLException` and transient credential errors instead of failing [#19617](https://github.com/apache/druid/pull/19617) [#19558](https://github.com/apache/druid/pull/19558) +* Improved S3 performance [#19394](https://github.com/apache/druid/pull/19394) +* Updated the default S3 connection pool size so that it's computed based on the number of available processors [#19536](https://github.com/apache/druid/pull/19536) + #### SQL-based ingestion +##### EXTERN for S3 now supports role ARN + +You can now include the role ARN when running an INSERT INTO EXTERN query. For example: + +``` +INSERT INTO +EXTERN( + s3(bucket => 'test2', prefix => 'export', assumeRoleArn => 'arn:aws:iam::00000:role/test-20260520')) +AS CSV +SELECT ... +``` + +[#19317](https://github.com/apache/druid/pull/19317) + ##### Other SQL-based ingestion improvements +- Added storage counters for the amount of bytes and files written to local and durable storage as well as the state of the local `ByteTracker`. Per-worker storage counters have also been added to the web console [#19316](https://github.com/apache/druid/pull/19316) + #### Streaming ingestion +##### Scaling cool down + +You can now configure different cool downs for scaling up and scaling down streaming task autoscalers. + +[#19286](https://github.com/apache/druid/pull/19286) + +##### Improved supervisor restarts + +Supervisors no longer restart for all changes. Based on the type of change, one of the following can occur: + +- The updated spec is persisted without a restart +- The supervisor is restarted but running tasks aren't impacted +- The supervisor is restarted and its tasks are terminated (the default behavior prior to this change) + +For example, cosmetic changes to a supervisor spec no longer trigger a restart. + +[#19700](https://github.com/apache/druid/pull/19700) [#19720](https://github.com/apache/druid/pull/19720) + +Additionally, the algorithm for determining a change in the spec has been improved. For example, changes to `ioConfig.taskCount` don't trigger a supervisor restart if auto-scaling is enabled. + +[#19541](https://github.com/apache/druid/pull/19541) + +###### Latest offset and backfill + +For Kafka and RabbitMQ, you can now reset a supervisor to the latest offset and start a new bounded backfill supervisor to ingest data from the skipped range. This is a useful feature for operating Druid clusters where the most recent data is the most important, such as for alerting. + +Note the following requirements: + +- The supervisor's `useEarliestSequenceNumber` property must be `false`. +- The supervisor context must have `useConcurrentLocks` set to `true` to allow the backfill supervisor's tasks to write concurrently with the main supervisor's tasks. +- The supervisor must be in a `RUNNING` state. + +Use `POST` `/druid/indexer/v1/supervisor/{supervisorId}/resetToLatestAndBackfill` or the web console to perform this action. + +[#19477](https://github.com/apache/druid/pull/19477) + +##### Prunable shard specs for streaming published segments + +Kafka ingestion can now publish segments that the Broker prunes at query time without waiting for compaction. Set `tuningConfig.streamingPartitionsSpec.partitionDimensions` to a list of low-to-medium cardinality dimensions; each task records +the distinct values it observes per dimension and stamps them onto a new `dim_value_set` shard spec. Queries that filter on a declared dimension then skip segments whose values can't match. + +The feature is opt-in, Kafka-only, and disabled by +default; when unset, behavior is unchanged. + +`dim_value_set` is a new core shard spec type with no fallback, so it is not forward-compatible. Upgrade all services before enabling `streamingPartitionsSpec`. Once `dim_value_set` segments are published, downgrade is unsupported +until they are compacted away or `streamingPartitionsSpec` is removed. + +Added `maxValuesPerDimension` (optional) to `streamingPartitionsSpec`. + +[#19571](https://github.com/apache/druid/pull/19571) [#19596](https://github.com/apache/druid/pull/19596) + ##### Other streaming ingestion improvements +- Added a property called `boundedStreamConfig` to the `SeekableStreamSupervisorIOConfig`, which allows operators to spin up a Supervisor that consumes only a specified offset range [#19372](https://github.com/apache/druid/pull/19372) +- Improved the cost-based autoscaler for better throughput [#19646](https://github.com/apache/druid/pull/19646) + ### Querying +#### New query laning strategy + +The `weighted` query laning strategy scores queries by how many thresholds they breach (segment count, interval duration, data age, segment range) and assigns them to configurable graduated lanes with different capacity limits, providing more nuanced lane assignment than the existing binary high/low strategy. + +The weighted query laning strategy supports optional per-threshold cost weights: + +- `periodWeight` +- `durationWeight` +- `segmentCountWeight` +- `segmentRangeWeight` + +These threshold weights default to 1. + +[#19225](https://github.com/apache/druid/pull/19225) [#19665](https://github.com/apache/druid/pull/19665) [#19696](https://github.com/apache/druid/pull/19696) + #### Other querying improvements +- Added nullable `minTime/maxTime` Long fields to `ProjectionMetadata` [#19398](https://github.com/apache/druid/pull/19398) +- Added `getDimensionRangeSet` support to `LikeDimFilter` for equality and prefix cases [#19524](https://github.com/apache/druid/pull/19524) +- Added support for aggregate projections with clustered segments [#19599](https://github.com/apache/druid/pull/19599) +- Optimized performance of aggregators for groupBy queries [#19423](https://github.com/apache/druid/pull/19423) + ### Cluster management +#### Improved `diskNormalized` balancer strategy + +The `diskNormalized` strategy is now more tunable. The primary changes are making the `utilizationThreshold` more intuitive: increasing the threshold increases the "tolerance" of the strategy while decreasing pushes nodes' disk utilization closer together. Review Comment: ```suggestion The `diskNormalized` strategy is now more tunable. The `utilizationThreshold` configuration works more intuitively: increasing the threshold increases the "tolerance" of the strategy while decreasing the threshold pushes nodes' disk utilization closer together. ``` ########## docs/release-info/release-notes.md: ########## @@ -65,42 +139,389 @@ This section contains detailed release notes separated by areas. #### Other web console improvements +- Added the following status details to the **Services** view for Historical services: cloning from another Historical, in turbo loading mode, in decommissioning mode [#19253](https://github.com/apache/druid/pull/19253) +- Added support for resetting a supervisor to the latest offsets and backfilling [#19533](https://github.com/apache/druid/pull/19533) +- Improved how new tabs are handled [#19483](https://github.com/apache/druid/pull/19483) +- Improved the Home view's **Services** card. It now reports Overlord, Coordinator, Router, Broker, and Indexer counts on clusters where the web console talks to the Coordinator without SQL access [#19481](https://github.com/apache/druid/pull/19481) + ### Ingestion +* You can now use the expression aggregator at ingestion time for expressions that produce a LONG or DOUBLE for both fold and combine expressions [#19508](https://github.com/apache/druid/pull/19508) +* Added `now()` expression function that returns the current system timestamp in milliseconds since epoch. Useful at ingestion time for troubleshooting pipeline delays (e.g., `now() - __time`). Note: `now()` is non-deterministic as it evaluates for every row, so it can break idempotency. This can be added to any besides `__time` [#19386](https://github.com/apache/druid/pull/19386) +* Improved resiliency when ingesting from S3. Druid now retries on `SSLException` and transient credential errors instead of failing [#19617](https://github.com/apache/druid/pull/19617) [#19558](https://github.com/apache/druid/pull/19558) +* Improved S3 performance [#19394](https://github.com/apache/druid/pull/19394) +* Updated the default S3 connection pool size so that it's computed based on the number of available processors [#19536](https://github.com/apache/druid/pull/19536) + #### SQL-based ingestion +##### EXTERN for S3 now supports role ARN + +You can now include the role ARN when running an INSERT INTO EXTERN query. For example: + +``` +INSERT INTO +EXTERN( + s3(bucket => 'test2', prefix => 'export', assumeRoleArn => 'arn:aws:iam::00000:role/test-20260520')) +AS CSV +SELECT ... +``` + +[#19317](https://github.com/apache/druid/pull/19317) + ##### Other SQL-based ingestion improvements +- Added storage counters for the amount of bytes and files written to local and durable storage as well as the state of the local `ByteTracker`. Per-worker storage counters have also been added to the web console [#19316](https://github.com/apache/druid/pull/19316) + #### Streaming ingestion +##### Scaling cool down + +You can now configure different cool downs for scaling up and scaling down streaming task autoscalers. + +[#19286](https://github.com/apache/druid/pull/19286) + +##### Improved supervisor restarts + +Supervisors no longer restart for all changes. Based on the type of change, one of the following can occur: + +- The updated spec is persisted without a restart +- The supervisor is restarted but running tasks aren't impacted +- The supervisor is restarted and its tasks are terminated (the default behavior prior to this change) + +For example, cosmetic changes to a supervisor spec no longer trigger a restart. + +[#19700](https://github.com/apache/druid/pull/19700) [#19720](https://github.com/apache/druid/pull/19720) + +Additionally, the algorithm for determining a change in the spec has been improved. For example, changes to `ioConfig.taskCount` don't trigger a supervisor restart if auto-scaling is enabled. + +[#19541](https://github.com/apache/druid/pull/19541) + +###### Latest offset and backfill + +For Kafka and RabbitMQ, you can now reset a supervisor to the latest offset and start a new bounded backfill supervisor to ingest data from the skipped range. This is a useful feature for operating Druid clusters where the most recent data is the most important, such as for alerting. + +Note the following requirements: + +- The supervisor's `useEarliestSequenceNumber` property must be `false`. +- The supervisor context must have `useConcurrentLocks` set to `true` to allow the backfill supervisor's tasks to write concurrently with the main supervisor's tasks. +- The supervisor must be in a `RUNNING` state. + +Use `POST` `/druid/indexer/v1/supervisor/{supervisorId}/resetToLatestAndBackfill` or the web console to perform this action. + +[#19477](https://github.com/apache/druid/pull/19477) + +##### Prunable shard specs for streaming published segments + +Kafka ingestion can now publish segments that the Broker prunes at query time without waiting for compaction. Set `tuningConfig.streamingPartitionsSpec.partitionDimensions` to a list of low-to-medium cardinality dimensions; each task records +the distinct values it observes per dimension and stamps them onto a new `dim_value_set` shard spec. Queries that filter on a declared dimension then skip segments whose values can't match. + +The feature is opt-in, Kafka-only, and disabled by +default; when unset, behavior is unchanged. + +`dim_value_set` is a new core shard spec type with no fallback, so it is not forward-compatible. Upgrade all services before enabling `streamingPartitionsSpec`. Once `dim_value_set` segments are published, downgrade is unsupported +until they are compacted away or `streamingPartitionsSpec` is removed. + +Added `maxValuesPerDimension` (optional) to `streamingPartitionsSpec`. + +[#19571](https://github.com/apache/druid/pull/19571) [#19596](https://github.com/apache/druid/pull/19596) + ##### Other streaming ingestion improvements +- Added a property called `boundedStreamConfig` to the `SeekableStreamSupervisorIOConfig`, which allows operators to spin up a Supervisor that consumes only a specified offset range [#19372](https://github.com/apache/druid/pull/19372) +- Improved the cost-based autoscaler for better throughput [#19646](https://github.com/apache/druid/pull/19646) + ### Querying +#### New query laning strategy + +The `weighted` query laning strategy scores queries by how many thresholds they breach (segment count, interval duration, data age, segment range) and assigns them to configurable graduated lanes with different capacity limits, providing more nuanced lane assignment than the existing binary high/low strategy. + +The weighted query laning strategy supports optional per-threshold cost weights: + +- `periodWeight` +- `durationWeight` +- `segmentCountWeight` +- `segmentRangeWeight` + +These threshold weights default to 1. + +[#19225](https://github.com/apache/druid/pull/19225) [#19665](https://github.com/apache/druid/pull/19665) [#19696](https://github.com/apache/druid/pull/19696) + #### Other querying improvements +- Added nullable `minTime/maxTime` Long fields to `ProjectionMetadata` [#19398](https://github.com/apache/druid/pull/19398) +- Added `getDimensionRangeSet` support to `LikeDimFilter` for equality and prefix cases [#19524](https://github.com/apache/druid/pull/19524) +- Added support for aggregate projections with clustered segments [#19599](https://github.com/apache/druid/pull/19599) +- Optimized performance of aggregators for groupBy queries [#19423](https://github.com/apache/druid/pull/19423) + ### Cluster management +#### Improved `diskNormalized` balancer strategy + +The `diskNormalized` strategy is now more tunable. The primary changes are making the `utilizationThreshold` more intuitive: increasing the threshold increases the "tolerance" of the strategy while decreasing pushes nodes' disk utilization closer together. + +[#19663](https://github.com/apache/druid/pull/19663) + +#### MiddleManager and Indexer restarts + +MiddleManagers and Indexers now persist their enabled or disabled state across restarts. If you prefer the old behavior, where the server re-enables itself after a restart, set `druid.worker.startAlwaysEnabled = true`. + +[#19373](https://github.com/apache/druid/pull/19373) + +#### Kafka idle signal + +Improved the cost-based auto scaler for Kafka. The `poll-idle ratio` only reflected the time spent polling, whether there is spare processing capacity. You can now configure the autoscaler to use a utilization ratio instead: + +``` +1 - (avgProcessingRate / maxObservedRate) +``` + +Set `useUtilizationRatio` to `true` to use this new ratio for autoscaling. + +[#19622](https://github.com/apache/druid/pull/19622) + + #### Other cluster management improvements +- Added `datasource` filter pushdown to `sys.segments` table [#19718](https://github.com/apache/druid/pull/19718) [#19731](https://github.com/apache/druid/pull/19731) +- Added a `restarted` boolean field to the supervisor POST endpoint response to indicate whether the supervisor was actually restarted [#19349](https://github.com/apache/druid/pull/19349) +- Added `error_message` column to `sys.server_properties` table and made the table resilient to unreachable servers. Previously, the entire query would fail if any server was unreachable; now a row is returned with `error_message` populated. The table also now supports filter and projection pushdown [#19459](https://github.com/apache/druid/pull/19459) +- Added `druid.expressions.useVectorApi` config to support the incubating JDK Vector API. To use the API, set the config to `true` and start Druid with the `--add-modules=jdk.incubator.vector` flag [#19512](https://github.com/apache/druid/pull/19512) +- Added debug logging at the INFO level for projections if the debug flag is set [#19613](https://github.com/apache/druid/pull/19613) +- Changed MSQ task engine logging. It now logs the full stack trace when `debug` is set in the context [#19361](https://github.com/apache/druid/pull/19361) +- Improved the cost-based autoscaler so that it scales down over-provisioned supervisors running above the ideal idle ratio with low lag [#19562](https://github.com/apache/druid/pull/19562) +- Improved how Druid handles Java. MiddleManagers now honor `JAVA_HOME` [#19709](https://github.com/apache/druid/pull/19709) + ### Data management #### Other data management improvements +* Improved how compaction supervisor specs start up. They no longer attempt to create tasks with invalid configs [#19223](https://github.com/apache/druid/pull/19223) +* Changed Historical tiers so that they can only be associated with one tier alias [#19595](https://github.com/apache/druid/pull/19595) +* Sped up segment metadata cache syncs [#19672](https://github.com/apache/druid/pull/19672) + ### Metrics and monitoring +#### New metadata cache metrics + +Added the following metrics for the segment metadata cache: + +- `segment/metadataCache/unused/count` +- `segment/metadataCache/fetchIds/time` +- `segment/metadataCache/fetchPayloads/time` +- `segment/metadataCache/fetchPending/time` +- `segment/metadataCache/fetchSchemas/time` +- `segment/metadataCache/fetchIndexingStates/time` +- `segment/metadataCache/updateIds/time` +- `segment/metadataCache/updateSnapshot/time` +- `segment/metadataCache/schema/skipped` +- `segment/metadataCache/indexingState/added` +- `segment/metadataCache/indexingState/deleted` + +[#19672](https://github.com/apache/druid/pull/19672) + +#### Changed metrics for cost-based autoscaling + +Removed the following metrics: + +- `task/autoScaler/costBased/lagCost` +- `task/autoScaler/costBased/idleCost` + +Added the following metrics: + +- `task/autoScaler/costBased/avgProcessingRate` +- `task/autoScaler/costBased/avgPollIdleRatio` +- `task/autoScaler/costBased/lagWeight` +- `task/autoScaler/costBased/costWeight` + +[#19631](https://github.com/apache/druid/pull/19631) + +#### Auth metrics + +You can now configure Druid to emit metrics for authorization events. + +The `auth/forbidden` and `auth/exception` metrics have the following dimensions to support precise alerting on security events: + +- `identity` +- `authorizerName` +- `resourceName` +- `resourceType` +- `action` +- `errorMessage` where applicable + + +To enable this functionality, set `druid.auth.emitAuthMetrics` to `true`. + +[#19552](https://github.com/apache/druid/pull/19552) + +#### Storage metrics + +The `storage/load/bytes` and `storage/virtual/load/bytes` metrics now measure once the load is complete. Previously, they measured when the load starts. Review Comment: "they measured when the load starts" mix of past/present reads awkwardly. Consider revising ########## docs/release-info/release-notes.md: ########## @@ -57,6 +57,80 @@ For tips about how to write a good release note, see [Release notes](https://git This section contains important information about new and existing features. +#### Java + +Druid now supports Java 25. While Druid 21 is still supported, we recommend you upgrade to Java 25. + +Support for Java 17 has been dropped. + +[#19304](https://github.com/apache/druid/pull/19304) [#19336](https://github.com/apache/druid/pull/19336) + +#### Historical tier aliases + +You can now use the `historicalTierAliases` Coordinator dynamic configuration to map a virtual tier name to a set of real Historical tiers so that a group of Historical tiers has a single identifier. When a load/drop rule references the alias, the Coordinator replaces it with the actual tiers. For example, if you map the Historical tiers `hot_1` and `hot_2` to the alias `hot`, the rule `{"hot": 2}` loads 2 replicas of each onto `hot_1` and `hot_2`. + +[#19204](https://github.com/apache/druid/pull/19204) [#19667](https://github.com/apache/druid/pull/19667) + +#### New load rule types + +Adds a new family of retention rules, `loadPartialByPeriod`, `loadPartialByInterval`, `loadPartialForever`, laying the groundwork for partial loading of version 10 segment projections on Historicals. + +[#19374](https://github.com/apache/druid/pull/19374) + +#### Realtime segments query context + +The `realtimeSegmentsOnly` query context parameter has been deprecated and replaced with `realtimeSegmentsMode`. + +You can set `realtimeSegmentsMode` to one of the following: + +- `include` (default): query all segments, including realtime segments +- `exclude`: skip realtime segments for the query +- `exclusive`: query only realtime segments + +This is useful when performing things like blue/green deployments and you only want to query new Historical replica autoscaling groups and not touch any "live" nodes (neither realtime nor historical). + +[#19486](https://github.com/apache/druid/pull/19486) + +#### Faster segment metadata search + +You can set `druid.segment.timeline.fastIntervalSearch` to `true` so that Druid uses an index based on interval trees to store that metadata in memory for faster identification and retrieval. + +This feature is off by default. + +[#19138](https://github.com/apache/druid/issues/19138) [#19850](https://github.com/apache/druid/issues/19850) + +#### Fetching files in the background + +The MSQ task engine now supports workers fetching input files from cloud storage asynchronously. This overlaps downloading with processing, which generally improves throughput when reading larger numbers of files. Otherwise, workers stream each file directly from cloud storage while processing it. + +This feature is on by default and is controlled by the `backgroundFetchExternalFiles` setting. + +[#19539](https://github.com/apache/druid/pull/19539) + +#### Segment prefetching for Dart + +Dart now supports the runtime property `druid.msq.dart.worker.segmentLoadAheadCount`, which controls the number of segments that Dart prefetches. If configured to be greater than 0 for a worker, this becomes the default `segmentLoadAheadCount` value for the worker. If a query includes the `segmentLoadAheadCount` query context parameter, the query context takes precedence. Review Comment: ```suggestion Dart now supports the runtime property `druid.msq.dart.worker.segmentLoadAheadCount`, which controls the number of segments that Dart prefetches. If set greater than 0 for a worker, this setting becomes the default `segmentLoadAheadCount` value for the worker. If a query includes the `segmentLoadAheadCount` query context parameter, the query context takes precedence. ``` ########## docs/release-info/release-notes.md: ########## @@ -65,42 +139,389 @@ This section contains detailed release notes separated by areas. #### Other web console improvements +- Added the following status details to the **Services** view for Historical services: cloning from another Historical, in turbo loading mode, in decommissioning mode [#19253](https://github.com/apache/druid/pull/19253) +- Added support for resetting a supervisor to the latest offsets and backfilling [#19533](https://github.com/apache/druid/pull/19533) +- Improved how new tabs are handled [#19483](https://github.com/apache/druid/pull/19483) +- Improved the Home view's **Services** card. It now reports Overlord, Coordinator, Router, Broker, and Indexer counts on clusters where the web console talks to the Coordinator without SQL access [#19481](https://github.com/apache/druid/pull/19481) + ### Ingestion +* You can now use the expression aggregator at ingestion time for expressions that produce a LONG or DOUBLE for both fold and combine expressions [#19508](https://github.com/apache/druid/pull/19508) +* Added `now()` expression function that returns the current system timestamp in milliseconds since epoch. Useful at ingestion time for troubleshooting pipeline delays (e.g., `now() - __time`). Note: `now()` is non-deterministic as it evaluates for every row, so it can break idempotency. This can be added to any besides `__time` [#19386](https://github.com/apache/druid/pull/19386) Review Comment: missing word ```suggestion * Added `now()` expression function that returns the current system timestamp in milliseconds since epoch. Useful at ingestion time for troubleshooting pipeline delays (e.g., `now() - __time`). Note: `now()` is non-deterministic as it evaluates for every row, so it can break idempotency. This can be added to any column besides `__time` [#19386](https://github.com/apache/druid/pull/19386) ``` ########## docs/release-info/release-notes.md: ########## @@ -65,42 +139,389 @@ This section contains detailed release notes separated by areas. #### Other web console improvements +- Added the following status details to the **Services** view for Historical services: cloning from another Historical, in turbo loading mode, in decommissioning mode [#19253](https://github.com/apache/druid/pull/19253) +- Added support for resetting a supervisor to the latest offsets and backfilling [#19533](https://github.com/apache/druid/pull/19533) +- Improved how new tabs are handled [#19483](https://github.com/apache/druid/pull/19483) +- Improved the Home view's **Services** card. It now reports Overlord, Coordinator, Router, Broker, and Indexer counts on clusters where the web console talks to the Coordinator without SQL access [#19481](https://github.com/apache/druid/pull/19481) + ### Ingestion +* You can now use the expression aggregator at ingestion time for expressions that produce a LONG or DOUBLE for both fold and combine expressions [#19508](https://github.com/apache/druid/pull/19508) +* Added `now()` expression function that returns the current system timestamp in milliseconds since epoch. Useful at ingestion time for troubleshooting pipeline delays (e.g., `now() - __time`). Note: `now()` is non-deterministic as it evaluates for every row, so it can break idempotency. This can be added to any besides `__time` [#19386](https://github.com/apache/druid/pull/19386) +* Improved resiliency when ingesting from S3. Druid now retries on `SSLException` and transient credential errors instead of failing [#19617](https://github.com/apache/druid/pull/19617) [#19558](https://github.com/apache/druid/pull/19558) +* Improved S3 performance [#19394](https://github.com/apache/druid/pull/19394) +* Updated the default S3 connection pool size so that it's computed based on the number of available processors [#19536](https://github.com/apache/druid/pull/19536) + #### SQL-based ingestion +##### EXTERN for S3 now supports role ARN + +You can now include the role ARN when running an INSERT INTO EXTERN query. For example: + +``` +INSERT INTO +EXTERN( + s3(bucket => 'test2', prefix => 'export', assumeRoleArn => 'arn:aws:iam::00000:role/test-20260520')) +AS CSV +SELECT ... +``` + +[#19317](https://github.com/apache/druid/pull/19317) + ##### Other SQL-based ingestion improvements +- Added storage counters for the amount of bytes and files written to local and durable storage as well as the state of the local `ByteTracker`. Per-worker storage counters have also been added to the web console [#19316](https://github.com/apache/druid/pull/19316) + #### Streaming ingestion +##### Scaling cool down + +You can now configure different cool downs for scaling up and scaling down streaming task autoscalers. + +[#19286](https://github.com/apache/druid/pull/19286) + +##### Improved supervisor restarts + +Supervisors no longer restart for all changes. Based on the type of change, one of the following can occur: + +- The updated spec is persisted without a restart +- The supervisor is restarted but running tasks aren't impacted +- The supervisor is restarted and its tasks are terminated (the default behavior prior to this change) Review Comment: ```suggestion - Druid persists the updated spec without a restart. - Druid restarts the supervisor, but running tasks aren't impacted. - Druid restarts the supervisor is restarted and its tasks are terminated, which was the default behavior before this change. ``` ########## docs/release-info/release-notes.md: ########## @@ -65,42 +139,389 @@ This section contains detailed release notes separated by areas. #### Other web console improvements +- Added the following status details to the **Services** view for Historical services: cloning from another Historical, in turbo loading mode, in decommissioning mode [#19253](https://github.com/apache/druid/pull/19253) +- Added support for resetting a supervisor to the latest offsets and backfilling [#19533](https://github.com/apache/druid/pull/19533) +- Improved how new tabs are handled [#19483](https://github.com/apache/druid/pull/19483) +- Improved the Home view's **Services** card. It now reports Overlord, Coordinator, Router, Broker, and Indexer counts on clusters where the web console talks to the Coordinator without SQL access [#19481](https://github.com/apache/druid/pull/19481) + ### Ingestion +* You can now use the expression aggregator at ingestion time for expressions that produce a LONG or DOUBLE for both fold and combine expressions [#19508](https://github.com/apache/druid/pull/19508) +* Added `now()` expression function that returns the current system timestamp in milliseconds since epoch. Useful at ingestion time for troubleshooting pipeline delays (e.g., `now() - __time`). Note: `now()` is non-deterministic as it evaluates for every row, so it can break idempotency. This can be added to any besides `__time` [#19386](https://github.com/apache/druid/pull/19386) +* Improved resiliency when ingesting from S3. Druid now retries on `SSLException` and transient credential errors instead of failing [#19617](https://github.com/apache/druid/pull/19617) [#19558](https://github.com/apache/druid/pull/19558) +* Improved S3 performance [#19394](https://github.com/apache/druid/pull/19394) +* Updated the default S3 connection pool size so that it's computed based on the number of available processors [#19536](https://github.com/apache/druid/pull/19536) + #### SQL-based ingestion +##### EXTERN for S3 now supports role ARN + +You can now include the role ARN when running an INSERT INTO EXTERN query. For example: + +``` +INSERT INTO +EXTERN( + s3(bucket => 'test2', prefix => 'export', assumeRoleArn => 'arn:aws:iam::00000:role/test-20260520')) +AS CSV +SELECT ... +``` + +[#19317](https://github.com/apache/druid/pull/19317) + ##### Other SQL-based ingestion improvements +- Added storage counters for the amount of bytes and files written to local and durable storage as well as the state of the local `ByteTracker`. Per-worker storage counters have also been added to the web console [#19316](https://github.com/apache/druid/pull/19316) + #### Streaming ingestion +##### Scaling cool down + +You can now configure different cool downs for scaling up and scaling down streaming task autoscalers. + +[#19286](https://github.com/apache/druid/pull/19286) + +##### Improved supervisor restarts + +Supervisors no longer restart for all changes. Based on the type of change, one of the following can occur: + +- The updated spec is persisted without a restart +- The supervisor is restarted but running tasks aren't impacted +- The supervisor is restarted and its tasks are terminated (the default behavior prior to this change) + +For example, cosmetic changes to a supervisor spec no longer trigger a restart. + +[#19700](https://github.com/apache/druid/pull/19700) [#19720](https://github.com/apache/druid/pull/19720) + +Additionally, the algorithm for determining a change in the spec has been improved. For example, changes to `ioConfig.taskCount` don't trigger a supervisor restart if auto-scaling is enabled. + +[#19541](https://github.com/apache/druid/pull/19541) + +###### Latest offset and backfill + +For Kafka and RabbitMQ, you can now reset a supervisor to the latest offset and start a new bounded backfill supervisor to ingest data from the skipped range. This is a useful feature for operating Druid clusters where the most recent data is the most important, such as for alerting. Review Comment: ```suggestion For Kafka and RabbitMQ, you can now reset a supervisor to the latest offset and start a new bounded backfill supervisor to ingest data from the skipped range. This is a useful feature for operating Druid clusters where the most recent data is the most important, like alerting scenarios. ``` ########## docs/release-info/release-notes.md: ########## @@ -65,42 +139,389 @@ This section contains detailed release notes separated by areas. #### Other web console improvements +- Added the following status details to the **Services** view for Historical services: cloning from another Historical, in turbo loading mode, in decommissioning mode [#19253](https://github.com/apache/druid/pull/19253) +- Added support for resetting a supervisor to the latest offsets and backfilling [#19533](https://github.com/apache/druid/pull/19533) +- Improved how new tabs are handled [#19483](https://github.com/apache/druid/pull/19483) +- Improved the Home view's **Services** card. It now reports Overlord, Coordinator, Router, Broker, and Indexer counts on clusters where the web console talks to the Coordinator without SQL access [#19481](https://github.com/apache/druid/pull/19481) + ### Ingestion +* You can now use the expression aggregator at ingestion time for expressions that produce a LONG or DOUBLE for both fold and combine expressions [#19508](https://github.com/apache/druid/pull/19508) +* Added `now()` expression function that returns the current system timestamp in milliseconds since epoch. Useful at ingestion time for troubleshooting pipeline delays (e.g., `now() - __time`). Note: `now()` is non-deterministic as it evaluates for every row, so it can break idempotency. This can be added to any besides `__time` [#19386](https://github.com/apache/druid/pull/19386) +* Improved resiliency when ingesting from S3. Druid now retries on `SSLException` and transient credential errors instead of failing [#19617](https://github.com/apache/druid/pull/19617) [#19558](https://github.com/apache/druid/pull/19558) +* Improved S3 performance [#19394](https://github.com/apache/druid/pull/19394) +* Updated the default S3 connection pool size so that it's computed based on the number of available processors [#19536](https://github.com/apache/druid/pull/19536) + #### SQL-based ingestion +##### EXTERN for S3 now supports role ARN + +You can now include the role ARN when running an INSERT INTO EXTERN query. For example: + +``` +INSERT INTO +EXTERN( + s3(bucket => 'test2', prefix => 'export', assumeRoleArn => 'arn:aws:iam::00000:role/test-20260520')) +AS CSV +SELECT ... +``` + +[#19317](https://github.com/apache/druid/pull/19317) + ##### Other SQL-based ingestion improvements +- Added storage counters for the amount of bytes and files written to local and durable storage as well as the state of the local `ByteTracker`. Per-worker storage counters have also been added to the web console [#19316](https://github.com/apache/druid/pull/19316) + #### Streaming ingestion +##### Scaling cool down + +You can now configure different cool downs for scaling up and scaling down streaming task autoscalers. + +[#19286](https://github.com/apache/druid/pull/19286) + +##### Improved supervisor restarts + +Supervisors no longer restart for all changes. Based on the type of change, one of the following can occur: + +- The updated spec is persisted without a restart +- The supervisor is restarted but running tasks aren't impacted +- The supervisor is restarted and its tasks are terminated (the default behavior prior to this change) + +For example, cosmetic changes to a supervisor spec no longer trigger a restart. + +[#19700](https://github.com/apache/druid/pull/19700) [#19720](https://github.com/apache/druid/pull/19720) + +Additionally, the algorithm for determining a change in the spec has been improved. For example, changes to `ioConfig.taskCount` don't trigger a supervisor restart if auto-scaling is enabled. + +[#19541](https://github.com/apache/druid/pull/19541) + +###### Latest offset and backfill + +For Kafka and RabbitMQ, you can now reset a supervisor to the latest offset and start a new bounded backfill supervisor to ingest data from the skipped range. This is a useful feature for operating Druid clusters where the most recent data is the most important, such as for alerting. + +Note the following requirements: + +- The supervisor's `useEarliestSequenceNumber` property must be `false`. +- The supervisor context must have `useConcurrentLocks` set to `true` to allow the backfill supervisor's tasks to write concurrently with the main supervisor's tasks. +- The supervisor must be in a `RUNNING` state. + +Use `POST` `/druid/indexer/v1/supervisor/{supervisorId}/resetToLatestAndBackfill` or the web console to perform this action. + +[#19477](https://github.com/apache/druid/pull/19477) + +##### Prunable shard specs for streaming published segments + +Kafka ingestion can now publish segments that the Broker prunes at query time without waiting for compaction. Set `tuningConfig.streamingPartitionsSpec.partitionDimensions` to a list of low-to-medium cardinality dimensions; each task records +the distinct values it observes per dimension and stamps them onto a new `dim_value_set` shard spec. Queries that filter on a declared dimension then skip segments whose values can't match. + +The feature is opt-in, Kafka-only, and disabled by +default; when unset, behavior is unchanged. + +`dim_value_set` is a new core shard spec type with no fallback, so it is not forward-compatible. Upgrade all services before enabling `streamingPartitionsSpec`. Once `dim_value_set` segments are published, downgrade is unsupported +until they are compacted away or `streamingPartitionsSpec` is removed. + +Added `maxValuesPerDimension` (optional) to `streamingPartitionsSpec`. + +[#19571](https://github.com/apache/druid/pull/19571) [#19596](https://github.com/apache/druid/pull/19596) + ##### Other streaming ingestion improvements +- Added a property called `boundedStreamConfig` to the `SeekableStreamSupervisorIOConfig`, which allows operators to spin up a Supervisor that consumes only a specified offset range [#19372](https://github.com/apache/druid/pull/19372) +- Improved the cost-based autoscaler for better throughput [#19646](https://github.com/apache/druid/pull/19646) + ### Querying +#### New query laning strategy + +The `weighted` query laning strategy scores queries by how many thresholds they breach (segment count, interval duration, data age, segment range) and assigns them to configurable graduated lanes with different capacity limits, providing more nuanced lane assignment than the existing binary high/low strategy. + +The weighted query laning strategy supports optional per-threshold cost weights: + +- `periodWeight` +- `durationWeight` +- `segmentCountWeight` +- `segmentRangeWeight` + +These threshold weights default to 1. + +[#19225](https://github.com/apache/druid/pull/19225) [#19665](https://github.com/apache/druid/pull/19665) [#19696](https://github.com/apache/druid/pull/19696) + #### Other querying improvements +- Added nullable `minTime/maxTime` Long fields to `ProjectionMetadata` [#19398](https://github.com/apache/druid/pull/19398) +- Added `getDimensionRangeSet` support to `LikeDimFilter` for equality and prefix cases [#19524](https://github.com/apache/druid/pull/19524) +- Added support for aggregate projections with clustered segments [#19599](https://github.com/apache/druid/pull/19599) +- Optimized performance of aggregators for groupBy queries [#19423](https://github.com/apache/druid/pull/19423) + ### Cluster management +#### Improved `diskNormalized` balancer strategy + +The `diskNormalized` strategy is now more tunable. The primary changes are making the `utilizationThreshold` more intuitive: increasing the threshold increases the "tolerance" of the strategy while decreasing pushes nodes' disk utilization closer together. + +[#19663](https://github.com/apache/druid/pull/19663) + +#### MiddleManager and Indexer restarts + +MiddleManagers and Indexers now persist their enabled or disabled state across restarts. If you prefer the old behavior, where the server re-enables itself after a restart, set `druid.worker.startAlwaysEnabled = true`. + +[#19373](https://github.com/apache/druid/pull/19373) + +#### Kafka idle signal + +Improved the cost-based auto scaler for Kafka. The `poll-idle ratio` only reflected the time spent polling, whether there is spare processing capacity. You can now configure the autoscaler to use a utilization ratio instead: + +``` +1 - (avgProcessingRate / maxObservedRate) +``` + +Set `useUtilizationRatio` to `true` to use this new ratio for autoscaling. + +[#19622](https://github.com/apache/druid/pull/19622) + + #### Other cluster management improvements +- Added `datasource` filter pushdown to `sys.segments` table [#19718](https://github.com/apache/druid/pull/19718) [#19731](https://github.com/apache/druid/pull/19731) +- Added a `restarted` boolean field to the supervisor POST endpoint response to indicate whether the supervisor was actually restarted [#19349](https://github.com/apache/druid/pull/19349) +- Added `error_message` column to `sys.server_properties` table and made the table resilient to unreachable servers. Previously, the entire query would fail if any server was unreachable; now a row is returned with `error_message` populated. The table also now supports filter and projection pushdown [#19459](https://github.com/apache/druid/pull/19459) +- Added `druid.expressions.useVectorApi` config to support the incubating JDK Vector API. To use the API, set the config to `true` and start Druid with the `--add-modules=jdk.incubator.vector` flag [#19512](https://github.com/apache/druid/pull/19512) +- Added debug logging at the INFO level for projections if the debug flag is set [#19613](https://github.com/apache/druid/pull/19613) +- Changed MSQ task engine logging. It now logs the full stack trace when `debug` is set in the context [#19361](https://github.com/apache/druid/pull/19361) Review Comment: ```suggestion - The MSQ task engine now logs the full stack trace when `debug` is set in the context [#19361](https://github.com/apache/druid/pull/19361) ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
