Hi Reynold, That's my point indeed. If non-determinism is a trait/property of a class (objects really at runtime) it's by definition part of a trait and only Nondeterministic expressions are...well...non-deterministic.
I'm yet to review the code regarding the trait, but I suspect that there are places where you check whether an expression is deterministic by the method not the trait. If so, there's this ambiguity I'm talking about. Anyway, I'm glad to learn from you guys! Thanks. Pozdrawiam, Jacek Laskowski ---- https://medium.com/@jaceklaskowski/ Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark Follow me at https://twitter.com/jaceklaskowski On Sat, Sep 24, 2016 at 12:35 AM, Reynold Xin <r...@databricks.com> wrote: > deterministic method describes whether this instance of the expression tree > is deterministic, whereas Nondeterministic trait is about a class. > > > On Fri, Sep 23, 2016 at 10:46 AM, Jacek Laskowski <ja...@japila.pl> wrote: >> >> Hi Herman, >> >> That helps to know that someone can explain why we've got the two >> nondeterministic states. >> >> It's not possible to say...a non-Nondeterministic expression can be >> non-deterministic (the former is the trait while the latter is the >> method) #strange >> >> Pozdrawiam, >> Jacek Laskowski >> ---- >> https://medium.com/@jaceklaskowski/ >> Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark >> Follow me at https://twitter.com/jaceklaskowski >> >> >> On Fri, Sep 23, 2016 at 6:44 PM, Herman van Hövell tot Westerflier >> <hvanhov...@databricks.com> wrote: >> > Jacek, >> > >> > A non-deterministic expression usually holds some state. The >> > Nondeterministic trait makes sure a user can initialize this state >> > properly. >> > Take a look at InterpretedProjection for instance. >> > >> > HTH >> > >> > -Herman >> > >> > On Fri, Sep 23, 2016 at 8:28 AM, Jacek Laskowski <ja...@japila.pl> >> > wrote: >> >> >> >> Hi, >> >> >> >> Just came across the Expression trait [1] that can be check for >> >> determinism by the method deterministic [2] and trait Nondeterministic >> >> [3]. Why both? >> >> >> >> [1] >> >> >> >> https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala#L53 >> >> [2] >> >> >> >> https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala#L80 >> >> [3] >> >> >> >> https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Expression.scala#L271 >> >> >> >> Pozdrawiam, >> >> Jacek Laskowski >> >> ---- >> >> https://medium.com/@jaceklaskowski/ >> >> Mastering Apache Spark 2.0 http://bit.ly/mastering-apache-spark >> >> Follow me at https://twitter.com/jaceklaskowski >> >> >> >> --------------------------------------------------------------------- >> >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> >> >> > >> >> --------------------------------------------------------------------- >> To unsubscribe e-mail: dev-unsubscr...@spark.apache.org >> > --------------------------------------------------------------------- To unsubscribe e-mail: dev-unsubscr...@spark.apache.org
