imay commented on a change in pull request #1323: This commit has brought
contribution to streaming mini load
URL: https://github.com/apache/incubator-doris/pull/1323#discussion_r295651853
##########
File path: be/src/http/action/mini_load.cpp
##########
@@ -318,14 +330,47 @@ int MiniLoadAction::on_header(HttpRequest* req) {
HttpChannel::send_basic_challenge(req, "mini_load");
return -1;
}
- auto st = _on_header(req);
- if (!st.ok()) {
- HttpChannel::send_reply(req, HttpStatus::INTERNAL_SERVER_ERROR,
st.get_error_msg());
+
+ isStreaming = _is_streaming(req);
+ Status status;
+ if (isStreaming) {
+ status = _on_new_header(req);
+ StreamLoadContext* ctx = (StreamLoadContext*)req->handler_ctx();
+ ctx->status = status;
+ } else {
+ status = _on_header(req);
+ }
+ if (!status.ok()) {
+ HttpChannel::send_reply(req, HttpStatus::INTERNAL_SERVER_ERROR,
status.get_error_msg());
return -1;
}
return 0;
}
+bool MiniLoadAction::_is_streaming(HttpRequest* req) {
+ // multi load must be non-streaming
+ if (!req->param(SUB_LABEL_KEY).empty()) {
+ return false;
+ }
+
+
+ const TNetworkAddress& master_address =
_exec_env->master_info()->network_address;
+ TFeResult res;
+ Status status = FrontendHelper::rpc(
+ master_address.hostname, master_address.port,
+ [&res] (FrontendServiceConnection& client) {
+ client->isMiniLoadStreaming(res);
+ });
+ if (!status.ok()) {
+ std::stringstream ss;
+ ss << "This mini load is not streaming because: " <<
status.get_error_msg()
+ << " with address(" << master_address.hostname << ":" <<
master_address.port << ")";
+ LOG(WARNING) << ss.str();
Review comment:
I think this is normal case when system is upgrading. so you should change
WARNING to INFO
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
