Github user kyprifog commented on the issue:
https://github.com/apache/spark/pull/11005
@HyukjinKwon Thanks this is what I was looking for. Glad the work is still
being continued.
---
-
To unsubscribe, e-mail:
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/11005
BTW, datasource v2 is in progress too to allow more push downs (see
[SPARK-22386](https://issues.apache.org/jira/browse/SPARK-22386)). You might
want to take a look
---
Github user HyukjinKwon commented on the issue:
https://github.com/apache/spark/pull/11005
Technical reason: It's kind of risky to rely on `CatalystScan` and
completely replace the interface. I think I already see some tests were
disabled here. Also, there look potential better
Github user kyprifog commented on the issue:
https://github.com/apache/spark/pull/11005
@rxin Why was this pull request closed? Can you direct me to a new one
that could have replaced it? I was trying to track down the modification of
push downs for basic operations like "limit".
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/11005
Thanks for the pull request. I'm going through a list of pull requests to
cut them down since the sheer number is breaking some of the tooling we have.
Due to lack of activity on this pull request,
