rangareddy commented on issue #17129:
URL: https://github.com/apache/hudi/issues/17129#issuecomment-5366099842
This issue was reviewed as part of the JIRA-migrated backlog triage
(HUDI-9677).
**Findings: confirmed on `master`.**
`DataSourceReadOptions.PUSH_DOWN_INCR_FILTERS`
(`hoodie.datasource.read.incr.filters`) is declared at
`hudi-spark-datasource/hudi-spark-common/src/main/scala/org/apache/hudi/DataSourceOptions.scala:151-152`
and read in exactly one place:
```scala
// IncrementalRelationV1.scala:154-155
private val filters =
optParams.getOrElse(DataSourceReadOptions.PUSH_DOWN_INCR_FILTERS.key,
DataSourceReadOptions.PUSH_DOWN_INCR_FILTERS.defaultValue).split(",").filter(!_.isEmpty)
```
That is the legacy V1 relation. Nothing in the file-group-reader path
consults it, so a user who sets it on the current read path gets no error and
no effect. Your other observation holds too - no test references it.
One piece of context that sharpens the decision between your two options:
the legacy read paths are on the way out. PR #14060 (`refactor(spark): Remove
glob paths and deprecate read paths support`, merged 2025-11-25) removed glob
paths and deprecated read paths, and its own description says support will be
removed in a follow-up. So this config's only remaining consumer is scheduled
for deletion.
That makes "deprecate and remove the config" at least as reasonable as
wiring it into the HadoopFsRelation factory - and cheaper, if nobody is using
it. Worth deciding that explicitly rather than porting a config nobody asked
for.
Keeping this open.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
