satwik-pachigolla commented on code in PR #18367:
URL: https://github.com/apache/pinot/pull/18367#discussion_r3169743441
##########
pinot-spi/src/main/java/org/apache/pinot/spi/utils/CommonConstants.java:
##########
@@ -2142,16 +2142,8 @@ public static class MultiStageQueryRunner {
/// - "SAFE": MSE will only send stats if all instances in the cluster are
running 1.4.0 or later.
/// - "ALWAYS": MSE will always send stats, regardless of the version of
the instances in the cluster.
/// - "NEVER": MSE will never send stats.
- ///
- /// The reason for this flag that versions 1.3.0 and lower have two
undesired behaviors:
- /// 1. Some queries using intersection generate incorrect stats
- /// 2. When stats from other nodes are sent but are different from
expected, the query fails.
- ///
- /// In 1.4.0 the first issue is solved and instead of failing when
unexpected stats are received, the query
- /// continues without children stats. But if a query involves servers in
versions 1.3.0 and 1.4.0, the one
- /// running 1.3.0 may fail, which breaks backward compatibility.
public static final String KEY_OF_SEND_STATS_MODE =
"pinot.query.mse.stats.mode";
- public static final String DEFAULT_SEND_STATS_MODE = "SAFE";
+ public static final String DEFAULT_SEND_STATS_MODE = "ALWAYS";
Review Comment:
I don't think there's a safe way do this as an explicit migration boundary.
This requires coordination between nodes on old versions (which we can't
change) and new versions. Otherwise we'd need to use more stable mechanisms of
relying on ZK metadata that would have existed as of <=1.3, none of which I
think are suitable here.
https://github.com/apache/pinot/issues/15890 is a documented case of how
using ZK watchers led to more instability and the partial fix [PR comments also
mention](https://github.com/apache/pinot/pull/15895#issuecomment-2917213249)
that we should go to default `ALWAYS` eventually.
I updated the PR description to make this more clear.
cc @Jackie-Jiang
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
