Yuming Wang created SPARK-44493: ----------------------------------- Summary: Extract pushable predicates from disjunctive predicates Key: SPARK-44493 URL: https://issues.apache.org/jira/browse/SPARK-44493 Project: Spark Issue Type: Improvement Components: SQL Affects Versions: 4.0.0 Reporter: Yuming Wang Attachments: after.png, before.png
Example: {code:sql} select count(*) from db.very_large_table where session_start_dt between date_sub('2023-07-15', 1) and date_add('2023-07-16', 1) and type = 'event' and date(event_timestamp) between '2023-07-15' and '2023-07-16' and ( ( page_id in (2627, 2835, 2402999) and -- other predicates and rdt = 0 ) or ( page_id in (2616, 3411350) and rdt = 0 ) or ( page_id = 2403006 ) or ( page_id in (2208336, 2356359) and -- other predicates and rdt = 0 ) ) {code} We can push down {{page_id in(2627, 2835, 2402999, 2616, 3411350, 2403006, 2208336, 2356359)}} to datasource. Before: After: -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org