Kim, Peter Faiman, Matt Cheah, Michael
Armbrust
Subject: Re: And.eval short circuiting
This is "expected" in the sense that DataFrame operations can get re-ordered
under the hood by the optimizer. For example, if the optimizer deems it is
cheaper to apply the 2nd filter first,
re’s one.
>
> Thanks,
> Mingyu
>
> From: Reynold Xin
> Date: Wednesday, September 16, 2015 at 1:17 PM
> To: Zack Sampson
> Cc: "dev@spark.apache.org", Mingyu Kim, Peter Faiman, Matt Cheah, Michael
> Armbrust
>
> Subject: Re: And.eval short circuiting
>
>
I filed SPARK-10703. Thanks!
Mingyu
From: Reynold Xin
Date: Thursday, September 17, 2015 at 11:22 PM
To: Mingyu Kim
Cc: Zack Sampson, "dev@spark.apache.org", Peter Faiman, Matt Cheah, Michael
Armbrust
Subject: Re: And.eval short circuiting
Please file a ticket and cc me. Thanks.
From: Reynold Xin [r...@databricks.com]
Sent: Monday, September 14, 2015 10:14 PM
To: Zack Sampson
Cc: dev@spark.apache.org
Subject: Re: And.eval short circuiting
rxin=# select null and true;
?column?
--
(1 row)
rxin=# select null and false;
?column?
--
f
(1 row)
null
It seems like And.eval can avoid calculating right.eval if left.eval returns
null. Is there a reason it's written like it is?
override def eval(input: Row): Any = {
val l = left.eval(input)
if (l == false) {
false
} else {
val r = right.eval(input)
if (r == false) {
