[jira] [Commented] (FLINK-34214) FLIP-377: Support fine-grained configuration to control filter push down for Table/SQL Sources
[ https://issues.apache.org/jira/browse/FLINK-34214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17811142#comment-17811142 ] Rascal Wu commented on FLINK-34214: --- Hi, may i consulting one question that if we disable the push down for JDBC/MongoDB source, does it mean the source will scan whole data? The performance of scan whole data is worse than filter data via unindexed column? > FLIP-377: Support fine-grained configuration to control filter push down for > Table/SQL Sources > -- > > Key: FLINK-34214 > URL: https://issues.apache.org/jira/browse/FLINK-34214 > Project: Flink > Issue Type: New Feature > Components: Connectors / JDBC, Connectors / MongoDB >Affects Versions: mongodb-1.0.2, jdbc-3.1.2 >Reporter: Jiabao Sun >Priority: Major > Fix For: mongodb-1.1.0, jdbc-3.1.3 > > > This improvement implements [FLIP-377 Support fine-grained configuration to > control filter push down for Table/SQL > Sources|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=276105768] > This FLIP has 2 goals: > * Introduces a new configuration filter.handling.policy to the JDBC and > MongoDB connector. > * Suggests a convention option name if other connectors are going to add an > option for the same purpose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (FLINK-34214) FLIP-377: Support fine-grained configuration to control filter push down for Table/SQL Sources
[ https://issues.apache.org/jira/browse/FLINK-34214?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17816377#comment-17816377 ] Jiabao Sun commented on FLINK-34214: Hi [~stayrascal]. When pushing to the database without filters, we can iterate through the data in batches using primary key indexing or natural order and filter the data using external computing resources. This greatly reduces the computational overhead on the database for filters that do not hit the indexes. > FLIP-377: Support fine-grained configuration to control filter push down for > Table/SQL Sources > -- > > Key: FLINK-34214 > URL: https://issues.apache.org/jira/browse/FLINK-34214 > Project: Flink > Issue Type: New Feature > Components: Connectors / JDBC, Connectors / MongoDB >Affects Versions: mongodb-1.0.2, jdbc-3.1.2 >Reporter: jiabao.sun >Assignee: jiabao.sun >Priority: Major > Fix For: jdbc-3.1.3, mongodb-1.2.0 > > > This improvement implements [FLIP-377 Support fine-grained configuration to > control filter push down for Table/SQL > Sources|https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=276105768] > This FLIP has 2 goals: > * Introduces a new configuration filter.handling.policy to the JDBC and > MongoDB connector. > * Suggests a convention option name if other connectors are going to add an > option for the same purpose. -- This message was sent by Atlassian Jira (v8.20.10#820010)
