[ https://issues.apache.org/jira/browse/SPARK-28108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Wenchen Fan reassigned SPARK-28108: ----------------------------------- Assignee: Gengliang Wang > Simplify OrcFilters > ------------------- > > Key: SPARK-28108 > URL: https://issues.apache.org/jira/browse/SPARK-28108 > Project: Spark > Issue Type: Improvement > Components: SQL > Affects Versions: 3.0.0 > Reporter: Gengliang Wang > Assignee: Gengliang Wang > Priority: Minor > > In #24068, @IvanVergiliev reports that OrcFilters.createBuilder has > exponential complexity in the height of the filter tree due to the way the > check-and-build pattern is implemented. > This is because the same method createBuilder is called twice recursively for > any children under And/Or/Not nodes, so that inside the first call, the > second call is called as well(See description in #24068 for details). > Comparing to the approach in #24068, I propose a very simple solution for the > issue. We can rely on the result of convertibleFilters, which can build a > fully convertible tree. With it, we don't need to concern about the children > of a certain node is not convertible in method createBuilder. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org