Re: Why Dataset.hint uses logicalPlan (= analyzed not planWithBarrier)?

2018-01-26 Thread Jacek Laskowski
Thanks Wenchen --> https://github.com/apache/spark/pull/20405 I'd also like to write a new test where broadcast hint could be specified with table identifiers + improve scaladoc for Dataset.hint to note that hint does not have to be used with the Dataset but any Dataset (as long as the table

Re: Why Dataset.hint uses logicalPlan (= analyzed not planWithBarrier)?

2018-01-26 Thread Wenchen Fan
Looks like we missed this one, feel free to submit a patch, thanks for your finding! On Fri, Jan 26, 2018 at 3:39 PM, Jacek Laskowski wrote: > Hi, > > I've just noticed that every time Dataset.hint is used it triggers > execution of logical commands, their unions and hint

Why Dataset.hint uses logicalPlan (= analyzed not planWithBarrier)?

2018-01-25 Thread Jacek Laskowski
Hi, I've just noticed that every time Dataset.hint is used it triggers execution of logical commands, their unions and hint resolution (among other things that analyzer does). Why? Why does hint trigger hint resolution (through QueryExecution.analyzed)? [1] And moreover why not to use