uicosp commented on code in PR #3776:
URL: https://github.com/apache/flink-cdc/pull/3776#discussion_r4323226688
##########
flink-cdc-connect/flink-cdc-source-connectors/flink-connector-mysql-cdc/src/main/java/org/apache/flink/cdc/connectors/mysql/source/utils/StatementUtils.java:
##########
@@ -58,6 +63,29 @@ public static Object[] queryMinMax(JdbcConnection jdbc,
TableId tableId, String
});
}
+ public static Long queryRowCnt(
+ JdbcConnection jdbc, TableId tableId, String columnName, @Nullable
String filter)
+ throws SQLException {
+
+ if (filter == null) {
+ return queryApproximateRowCnt(jdbc, tableId);
+ }
+
+ final String cntQuery =
+ String.format("SELECT COUNT(1) FROM %s WHERE (%s)",
quote(tableId), filter);
Review Comment:
Thanks for raising this. I’d prefer to retain the filtered row count here.
Although it is not required for correctness, it allows us to calculate the
distribution factor and use even chunk splitting when applicable, avoiding
repeated boundary queries. Falling back to uneven splitting still requires
evaluating the filter in each queryNextChunkMax() call, which can also be
expensive without a suitable index. Using the full-table estimate could be
misleading when the filter selects only a small subset of rows.
I agree that the exact count adds initialization cost, but removing it may
shift that cost to repeated boundary queries rather than reduce the overall
snapshot cost. For now, I’d like to keep this behavior and document the
recommendation to use suitable indexes for the filter and chunk key.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
