Hi, > This is a great question. I'm currently working on implementing a solution > for this. > One possible solution is to write the records to a spill file when the flush > pointer > indicates that none have been flushed on the primary. Once they have been > flushed > on the primary, the records can then be copied from the spill file to the WAL > segments. > While this method may lead to increased I/O, if such spills are infrequent, > the overall > performance impact should be minimal. > > Another option would be to notify the sender that there is no more space > available > and to pause sending additional data until records are flushed on the sender > side. > However, this approach could reintroduce some of the replication lag or > network > latency that we are aiming to minimize. > > Kindly let me know your views.
Both options don't strike me as great design choices. A proper solution IMO would be to send and record flushPtr as a usual WAL record (a new resource manager, perhaps). When a replica is promoted to master or restarted it should truncate WAL according to the latest recorded flushPtr. The only thing we will have to worry about is to make sure the latest recorded flushPtr is never truncated, including regular recycling of WAL segments. Everything else will work as it is now, including cascaded replication for instance. -- Best regards, Aleksander Alekseev
