duongcongtoai commented on PR #16186: URL: https://github.com/apache/datafusion/pull/16186#issuecomment-2917657542
> The results are a little inconsistent. __scalar_sq_2."avg(e3.salary)", __scalar_sq_2.dept_id are not valid fields in the above context. Ideally, all the field in e1, e2 and e3 should come up here as they are valid. Beware that this error is thrown after the planning stage has completed, and it is expected because the current limitation of subquery decorrelation. The columns shown in the error is the side-effect of `DecorrelateLateralJoin` optimizor ``` Inner Join: e1.dept_id = __scalar_sq_1.dept_id Filter: CAST(e1.salary AS Decimal128(38, 14)) > __scalar_sq_1.avg(e2.salary) SubqueryAlias: e1 TableScan: employees projection=[employee_name, dept_id, salary] SubqueryAlias: __scalar_sq_1 Projection: avg(e2.salary), e2.dept_id Aggregate: groupBy=[[e2.dept_id]], aggr=[[avg(e2.salary)]] Projection: e2.dept_id, e2.salary Inner Join: Filter: CAST(e2.salary AS Decimal128(38, 14)) > __scalar_sq_2.avg(e3.salary) AND __scalar_sq_2.dept_id = e1.dept_id SubqueryAlias: e2 TableScan: employees projection=[dept_id, salary] SubqueryAlias: __scalar_sq_2 Projection: avg(e3.salary), e3.dept_id Aggregate: groupBy=[[e3.dept_id]], aggr=[[avg(e3.salary)]] SubqueryAlias: e3 TableScan: employees projection=[dept_id, salary] ``` And the error is thrown at this line: ` Inner Join: Filter: CAST(e2.salary AS Decimal128(38, 14)) > __scalar_sq_2.avg(e3.salary) AND __scalar_sq_2.dept_id = e1.dept_id ` Looks like we need a new plan to somehow let this Optimizor back-off during the implementation of new rules :thinking: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org