Hi, Shekhar > 1. Optimizer does not use both - ProjectableTableSource and > FilterableTableSource - in a single query even if the source implements both > interfaces. Each interface works correctly if implemented independently.
I didn’t your custom source implementation, but I think the two interface should works together, you can refer the ParquetTableSource and OrcTableSource implementation details. > 2. Implementations of FilterableTableSource fail inside the optimizer for a > few TPC-DS queries in batch mode. IIRC, all TPC-DS tests can passed using Parquet, which means the two interface should works fine? > > 3. And finally, filter expressions containing the current timestamp (& now) > function are not resolved to constant values during predicate pushdown > optimizer. Let's take the following SQL query for example - select count(*) > from T0 where T0.C2 >= current_timestamp. Here, applyPredicate method of > FilterableTableSource receives predicate as a CallExpression of form > greaterThanOrEqual(C2, currentTimestamp()). I'd have expected > currentTimestamp to be resolved to a constant value that is identitcal across > all usages of currentTimestamp in the query. The CURRENT_TIMESTAMP behavior in Flink batch/stream sql have same behavior that calculate the function value for per record, not like hive , which calculate the function value once before the query execution and thus it can be treated as a constant value. To be honest, I tend to use a constant value for the function in batch，which is more intuitive. Could you help create an issue for this, we can have more discussion under the issue. Best, Leonard
