tustvold commented on issue #2175: URL: https://github.com/apache/arrow-datafusion/issues/2175#issuecomment-1092102331
Thank you for raising this, its really good to have these discussions. I would definitely support making the enumeration style consistent, and the expression struct pattern has the non-trivial advantage of being able to name the variant types :+1: With respect to making `ExecutionPlan` an enumeration: * `ExecutionPlan` has a fairly large number of member functions, and having a massive `match` block for each of these seems somewhat sub-optimal * We would still need to have a `dyn Trait` in the topology for extensions, much like we have `UserDefinedLogicalNode` for `LogicalPlan` * The major advantage of an enumeration is being able to enumerate all the possible states, I'm struggling to devise a use-case that wouldn't be equally well served by a member function * Traits provide encapsulation by object, whereas enumerations provide encapsulation by function. Taking the example of a serialization routine, with trait objects each operator would define its serialization routine alongside its other functionality, with an enumeration you would instead define serialization for all operators in one function. It's possibly subjective but I prefer the former * I'm currently working on scheduling of plan execution, with an eventual hope to implement something akin to [morsel-driven parallelism](https://db.in.tum.de/~leis/papers/morsels.pdf). Currently this fudges around the interface of ExecutionPlan, but may eventually warrant non-trivial changes to it (e.g. making it less async), churning this interface multiple times in quick succession is probably something we want to avoid if possible That being said, I don't really feel strongly about any particular design, just putting the information out there :smile: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
