andygrove opened a new issue, #2744:
URL: https://github.com/apache/datafusion-comet/issues/2744
### What is the problem the feature request solves?
Expressions have consistent configs for enabling, disabling, and allowing
incompatible uses. This is because `CometExpressionSerde` has the following
method:
```scala
def getExprConfigName(expr: T): String = expr.getClass.getSimpleName
```
Config names are built from this:
```scala
def getExprEnabledConfigKey(name: String): String = {
s"${CometConf.COMET_EXPR_CONFIG_PREFIX}.$name.enabled"
}
def getExprAllowIncompatConfigKey(name: String): String = {
s"${CometConf.COMET_EXPR_CONFIG_PREFIX}.$name.allowIncompatible"
}
```
Unfortunately, operators followed a different pattern, where explicit
configs were implemented:
```scala
val COMET_EXEC_GLOBAL_LIMIT_ENABLED: ConfigEntry[Boolean] =
createExecEnabledConfig("globalLimit", defaultValue = true)
val COMET_EXEC_BROADCAST_HASH_JOIN_ENABLED: ConfigEntry[Boolean] =
createExecEnabledConfig("broadcastHashJoin", defaultValue = true)
```
Once https://github.com/apache/datafusion-comet/pull/2741 is merged, there
will be different configs for enabling/disabling versus allowing incompatible.
For example:
```
spark.comet.exec.globalLimit.enabled=true
spark.comet.exec.GlobalLimitExec.allowIncompatible=true
```
We can't simply rename the "enable" configs because that would be a breaking
change for users.
I propose that we update `CometOperatorSerde` to match the
`CometExpressionSerde` approach, but also build in support for respecting the
old / deprecated config as well.
### Describe the potential solution
_No response_
### Additional context
_No response_
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
