vagetablechicken opened a new issue #3644: URL: https://github.com/apache/incubator-doris/issues/3644
We do mem limit here. https://github.com/apache/incubator-doris/blob/792307ae54ee9dbe1be8b7e6fe30b4ef90b2cca9/be/src/exec/tablet_sink.cpp#L182-L185 The sink node mem is only reduced by sending batches. add_row() is a sequential process. If we blocked in one channel::add_row(), it's possible that no channel‘s going to reduce the mem, e.g. some channels have no pending batches, some channels are cancelled in the mem check loop. It'll be a endless loop. So we need to consider about cancelled status and pending batches in mem check loop: 1. mem check loop needs to check _cancelled, to break the loop. 1. we allow to add when the node channel has no pending batch, so there will be two situations: 1. if channel already has pending batches, after sent, the mem will reduce. 1. if no pending batches, add_row() can still generate a pending batch, back to situation 1 ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
