[
https://issues.apache.org/jira/browse/SPARK-27751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16843952#comment-16843952
]
Marc-Olivier Andrez commented on SPARK-27751:
---------------------------------------------
Many thanks for the reply.
I perfectly understand that we should migrate the code to use Datasource V2,
our problem is more how to do the migration smoothly, and having the method
`FileFormat.buildReader` marked as protected definitely makes our life harder.
At the moment, I cannot understand why this public method was adding so much
overhead, and it would be great if you could elaborate a bit on that.
Besides, if we keep the method `FileFormat.buildReader` marked as protected,
would it make sense to mark the overrides in `CSVFileFormat`, `JsonFileFormat`,
`TextFileFormat`, `OrcFileFormat`, `SimpleTextSource`, `LibSVMOutputWriter` and
`AvroFileFormat` as protected as well?
Many thanks again for your reply and your time!
> buildReader is now protected
> ----------------------------
>
> Key: SPARK-27751
> URL: https://issues.apache.org/jira/browse/SPARK-27751
> Project: Spark
> Issue Type: Question
> Components: Spark Core
> Affects Versions: 2.4.3
> Reporter: Geet Kumar
> Priority: Major
>
> I have recently upgraded to spark 2.4.0 and was relying on the `buildReader`
> method. It originally was public and now it is protected.
> What was the reason for this change?
> The only workaround I can see is to use `buildReaderWithPartitionValues`
> which remains public. Any plans to revert `buildReader` to be public again?
> The change was made here: [https://github.com/apache/spark/pull/17253/files]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
