Github user eyalfa commented on the issue: https://github.com/apache/spark/pull/16043 @hvanhovel, currently in the process of refactoring,luckily I've introduced some tests in the initial sprint so I've cought my mistakes and refreshed my mind around the assumptions this code makes. Regarding nulls,I'm using coalesce to add the first positive match after undetermined ones, when one of the undetermined values is null,coalesce will override it with the positive match's value. This was handled in the normalize method of class ClassifiefKeys,part of the refactoring involves removing this class and simplifying the decision making around matches. Will push later today after running scalastyle on my branch. On Jan 27, 2017 18:40, "Herman van Hovell" <notificati...@github.com> wrote: > *@hvanhovell* commented on this pull request. > ------------------------------ > > In sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ > optimizer/ComplexTypes.scala <https://github.com/apache/spark/pull/16043>: > > > + } > + } > + val res = res1.normalize( requestedKey ) > + res > + } > + > + override def apply(plan: LogicalPlan): LogicalPlan = { > + plan.transformExpressionsUp { > + // attempt to unfold 'constant' key extraction, > + // this enables other optimizations to take place. > + case gmv @ GetMapValue(cm @ CreateMap(elems), key) => > + val kvs = cm.keys.zip(cm.values) > + val classifiedEntries = classifyEntries(kvs, key) > + classifiedEntries match { > + case ClassifiedEntries(Seq(), _, None) => Literal.create(null, gmv.dataType) > + case ClassifiedEntries(`elems`, _, None) => gmv > > It is fine, it just needs a little clarification. > > â > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > <https://github.com/apache/spark/pull/16043>, or mute the thread > <https://github.com/notifications/unsubscribe-auth/ABFFOV2D--1Wea5bmhkZVNwjmAZXxrtLks5rWh3lgaJpZM4K-Obo> > . >
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org