[ https://issues.apache.org/jira/browse/IGNITE-21293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Kirill Sizov updated IGNITE-21293: ----------------------------------- Description: h3. Motivation Open cursors required extra memory on the server side. Hence, resources cannot be stored for a long time. h3. Implementation notes During the recovery procedure, the server receives a cleanup message (the message releases locks). On the message processing, we update the local transaction state, and it should also close all the cursors related to this transaction. h3. Definition of done All cursors should be closed on the RW transaction recovery. h3. Possible solution The reason why the cursors are not being closed during the recovery is that the normal way of closing them is implemented in {{WriteIntentSwitchReplicaRequest}} handler, but for the recovery case we don't have the collection of enlisted partitions, thus no write intent switch is triggered. We could follow the same approach as the lock manager uses, but we need a node-wide access to all the cursors opened in the current transaction. There is another way - instead of closing the cursors directly we can shift the responsibility to the partition listener itself. Each node has an in-memory txnState map, tracking the state of the transactions. If we add listeners to this map, then on registering a new cursor a partition listener will be able to check current transaction state and add a listener for a terminal one. When the tx state is changed to a terminal one, the cursors will be closed. h4. Pitfalls Currently the tx cursors are closed before ensuring the completion of read and update futures. There is a chance that one opens a new cursor after the "close cursors" stage. Checking TX state before registering a cursor should fix this - if the transaction is already in the terminal state - the cursor should be closed immediately. Another one: the tx state is updated from different places - {{PartitionReplicaListener}}, raft's {{PartitionListener}}. Need to make sure the tx cleanup flow is correct. was: h3. Motivation Open cursors required extra memory on the server side. Hence, resources cannot be stored for a long time. h3. Implementation notes During the recovery procedure, the server receives a cleanup message (the message releases locks). On the message processing, we update the local transaction state, and it should also close all the cursors related to this transaction. h3. Definition of done All cursors should be closed on the RW transaction recovery. > Scan cursors do not close on transaction recovery > ------------------------------------------------- > > Key: IGNITE-21293 > URL: https://issues.apache.org/jira/browse/IGNITE-21293 > Project: Ignite > Issue Type: Bug > Reporter: Vladislav Pyatkov > Priority: Major > Labels: ignite-3 > > h3. Motivation > Open cursors required extra memory on the server side. Hence, resources > cannot be stored for a long time. > h3. Implementation notes > During the recovery procedure, the server receives a cleanup message (the > message releases locks). On the message processing, we update the local > transaction state, and it should also close all the cursors related to this > transaction. > h3. Definition of done > All cursors should be closed on the RW transaction recovery. > h3. Possible solution > The reason why the cursors are not being closed during the recovery is that > the normal way of closing them is implemented in > {{WriteIntentSwitchReplicaRequest}} handler, but for the recovery case we > don't have the collection of enlisted partitions, thus no write intent switch > is triggered. > We could follow the same approach as the lock manager uses, but we need a > node-wide access to all the cursors opened in the current transaction. There > is another way - instead of closing the cursors directly we can shift the > responsibility to the partition listener itself. > Each node has an in-memory txnState map, tracking the state of the > transactions. If we add listeners to this map, then on registering a new > cursor a partition listener will be able to check current transaction state > and add a listener for a terminal one. > When the tx state is changed to a terminal one, the cursors will be closed. > h4. Pitfalls > Currently the tx cursors are closed before ensuring the completion of read > and update futures. There is a chance that one opens a new cursor after the > "close cursors" stage. Checking TX state before registering a cursor should > fix this - if the transaction is already in the terminal state - the cursor > should be closed immediately. > Another one: the tx state is updated from different places - > {{PartitionReplicaListener}}, raft's {{PartitionListener}}. Need to make sure > the tx cleanup flow is correct. -- This message was sent by Atlassian Jira (v8.20.10#820010)