hycdong opened a new issue #886: URL: https://github.com/apache/incubator-pegasus/issues/886
In current implementation, meta server control max concurrent ingestion partition count, for example, if max_concurrent_count is 4, meta server will send 4 ingestion request to replica servers, and won't send the fifth request until previous ingestion succeed or failed. Current restriction is precise enough, which can not control concurrent ingesting count of one node, we need to provide another percise plan, a disk-level concurrent ingesting count. Meta server adds `ingestion_context` to store ingesting partitions, including its node address and disk info. The new plan provides the node max ingesting count and disk max ingesting count. Before sending ingestion request, meta server will check if node and disk max count will be exceed after sending the request, if will not exceed, the request will be sent. Besides, when ingestion succeed or failed, the content of ingestion_context will be reset. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
