GitHub user andygrove closed a discussion: How does Comet compare to Gluten? 
Are there any plans to integrate with Gluten?

Comet and Gluten have similar architectures:

- They both replace Spark's physical plan with a custom plan
- They both use an Intermediate Representation to encode the plan. Gluten uses 
substrait, and Comet uses a simpler protobuf representation that is tightly 
coupled to Spark.
- Gluten typically delegates to Velox when used to accelerate Spark, and Comet 
delegates to DataFusion

The Comet project started before Gluten was an Apache project.

There are no plans to integrate Comet with Gluten as part of the Comet project.

However, there is no reason why Gluten could not add Comet/DataFusion as a 
backend query engine (DataFusion already supports Substrait, and Comet provides 
the Spark-compatible operators and expressions).

GitHub link: https://github.com/apache/datafusion-comet/discussions/994

----
This is an automatically sent email for github@datafusion.apache.org.
To unsubscribe, please send an email to: 
github-unsubscr...@datafusion.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org

Reply via email to