Github user cloud-fan commented on the issue:
https://github.com/apache/spark/pull/20387
> The reader is to be created and configured by the relation, then the
relation needs to be able to set the table, path, and other properties. This
adds necessary data to the relation that is no longer be passed directly to the
reader from DataFrameReader.
This is an interesting thing that I missed before. Can you give a use case
about this? If we have a requirement for it, I totally I agree we should put
them in `DataSourceV2Relation`.
> I don't see why we wouldn't want to have these options in the immutable
relation node from the start.
My main concern is, this PR itself doesn't show a strong use case for
putting these information into `DataSourceV2Relation`. If we do need it in the
future, we can add it when we really need it. `DataSourceV2Relation` is not a
public API so I think we don't need to rush.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
