924060929 commented on code in PR #69991:
URL: https://github.com/apache/doris/pull/69991#discussion_r4448157819
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/hive/source/HiveScanNode.java:
##########
@@ -256,22 +261,20 @@ public void startSplit(int numBackends) {
try {
List<Split> allFiles = Lists.newArrayList();
getFileSplitByPartitions(
- cache,
Collections.singletonList(partition), allFiles, bindBrokerName,
Review Comment:
Fixed in 429442702bd2d8420e1060199e88732a56ae5998. TABLESAMPLE now opts out
of lazy partition batching at the common eligibility gate. The existing eager
Hive path selects files once across all pruned partitions, preserving a single
ROWS byte budget, percentage rounding and seeded shuffle. The metadata-path
tests drive the real planning-mode decision over 32 partitions at batch sizes
16/2/1. Both ROWS and percent reproduced two selected files instead of one
before the fix; all 236 related FE tests and full FE packaging/Checkstyle now
pass locally.
##########
be/src/exec/scan/split_source_connector.cpp:
##########
@@ -191,4 +203,38 @@ Status RemoteSplitSourceConnector::get_next(bool*
has_next, TFileRangeDesc* rang
return Status::OK();
}
+Status RemoteSplitSourceConnector::_fetch_batch(TFetchSplitBatchResult* result,
+ const TFetchSplitBatchRequest&
request) {
+ Status status;
+ FrontendServiceConnection
coord(_state->exec_env()->frontend_client_cache(),
+ _state->get_query_ctx()->coord_addr,
&status);
+ RETURN_IF_ERROR(status);
+ try {
+ coord->fetchSplitBatch(*result, request);
+ } catch (const std::exception& e) {
+ return Status::IOError<false>("Failed to get batch of split source:
{}", e.what());
+ }
+ return Status::OK();
+}
+
+Status RemoteSplitSourceConnector::close() {
+ stop();
+ // Older FEs do not advertise support for the source-close request.
Review Comment:
Fixed in 429442702bd2d8420e1060199e88732a56ae5998. A separate close mutex
serializes cleanup without waiting for the consuming-fetch lock, and the
completed flag is set only after acknowledgement. Close performs a bounded
retry, using the standard ThriftRpcHelper reconnect/retry path only for this
idempotent request; ordinary consuming fetches are not retried. FE already
returns OK when the source was removed, so a lost reply is safe. Exhausted
cleanup errors deliberately still propagate to query cancellation: silently
ignoring them could strand the finite queue and block other BEs. A later close
remains retryable after failure. Tests cover loss before FE execution, a lost
reply after execution, bounded transport/FE failures and recovery; all 63
focused BE ASAN tests, 236 related FE tests and full BE ASAN/FE packaging pass
locally. This is a focused BE test target, not the complete BE UT suite.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
