One thing I found missing which was there in original proposal was a class combining both sorting & partitioning traits, SortExchange. It will be useful to have that.
On Tue, Feb 17, 2015 at 2:25 PM, Ashutosh Chauhan <[email protected]> wrote: > Looks good on the first look. Once we have more experience after > integrating in Hive, I may have further feedback. > > On Fri, Feb 13, 2015 at 12:39 AM, Julian Hyde <[email protected]> wrote: > >> I have committed, on my branch >> https://github.com/julianhyde/incubator-calcite/tree/calcite-594, a fix >> for >> https://issues.apache.org/jira/browse/CALCITE-594 that I think meets >> Drill's and Ashutosh's requirements. Please review it before I commit to >> master. >> >> I have 3 questions: >> >> 1. Can we please shorten the distribution type name HASH_DISTRIBUTED to >> HASH (similarly RANDOM_DISTRIBUTED etc.)? >> >> 2. Drill has DrillDistributionTrait.DEFAULT, but I'd rather not create >> RelDistributions.DEFAULT. (The default will be different for different >> systems using Calcite.) >> >> 3. RelDistributionTraitDef.convert does not create specific physical >> operators as it does in Drill, because there are no physical >> implementations of Exchange, just LogicalExchange. Drill will need to >> create rules to convert LogicalExchange to HashToRandomExchangePrel, etc. >> Hope that is OK. >> >> Julian >> >> >> >> On Feb 12, 2015, at 1:57 PM, Jacques Nadeau <[email protected]> wrote: >> >> Got it. >> >> On Thu, Feb 12, 2015 at 11:44 AM, Julian Hyde <[email protected]> >> wrote: >> >> RelMdDistribution (the "Md" that distinguishes it from >> "RelDistribution" stands for "metadata") is just theory, but it will >> look a lot like RelMdCollation. >> >> >> >> https://github.com/apache/incubator-calcite/blob/master/core/src/main/java/org/apache/calcite/rel/metadata/RelMdCollation.java >> >> On Wed, Feb 11, 2015 at 6:47 PM, Ashutosh Chauhan <[email protected]> >> wrote: >> >> I think RelDistribution as a member of Exchange (as in Julian proposal) >> makes more sense as compared to mine at field level. >> >> On Wed, Feb 11, 2015 at 6:42 PM, Jacques Nadeau <[email protected]> >> >> wrote: >> >> >> Does RelMdDistribution exist or as theory? >> >> Definitely inclined to "Exchange" over "Shuffle" as more >> >> generic/consistent >> >> with Volcano. >> >> Note, looking at Ashutosh I wonder a little about the need to have range >> versus hash at the field information level. I'm sure we can come up >> >> with a >> >> use case but it seems to overcomplicate. We already have space >> >> exploration >> >> problems when trying to do distribution subsumption for joins with trait >> propagation so I would be cautious about adding more dimensions to that >> attribute. >> > >
