freemandealer commented on issue #18092:
URL: https://github.com/apache/doris/issues/18092#issuecomment-1500067677
> Hi @freemandealer, I had exams recently and was unable to work on the
tasks. However, I plan to work on this issue on the upcoming Sunday. If I
encounter any problems, I will be sure to contact you. Thank you for your
understanding.
@AdarshRawat1 Sure. Take your time.
I'll list my proposal briefly below so you can have a reference:
Since the problem is opening tablets from all partitions no matter whether
the data is actually sent to the partition or not, we plan to open the tablets
from the partition only when there is data coming for it.
Original flow:
```
// Before sending data throught SinkNode
PInternalServiceImpl::tablet_writer_open
-> LoadChannel::open
-> TabletsChannel::open
-> _open_all_writers(request)
-> for all tablets open DeltaWriter
// then send data
tablet_writer_add_block
-> LoadChannelMgr
-> LoadChannel
-> TabletsChannel::add_batch
-> DeltaWriter write data
```
New flow:
```
// Before sending data throught SinkNode
PInternalServiceImpl::tablet_writer_open
-> LoadChannel::open
-> TabletsChannel::open
-> no more open writers
// then send data
tablet_writer_add_block
-> LoadChannelMgr
-> LoadChannel
-> TabletsChannel::add_batch
-> if specific DeltaWriter is not opened, open it in place before
write
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
