[
https://issues.apache.org/jira/browse/FLINK-10267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16598706#comment-16598706
]
ASF GitHub Bot commented on FLINK-10267:
----------------------------------------
azagrebin commented on a change in pull request #6638: [FLINK-10267][State] Fix
arbitrary iterator access on RocksDBMapIterator
URL: https://github.com/apache/flink/pull/6638#discussion_r214344987
##########
File path:
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBMapState.java
##########
@@ -595,6 +595,8 @@ private void loadCache() {
*/
if (lastEntry != null && !lastEntry.deleted) {
iterator.next();
+ cacheEntries.add(lastEntry);
+ cacheIndex = 1;
Review comment:
Could be also just:
```
if (lastEntry != null && !lastEntry.deleted) {
cacheIndex = 1;
}
```
This should work in general. I wonder if we could make it cleaner.
Although, the remove operation is not supposed to be called twice for the
same next,
if it happens and `lastEntry.deleted` is true, the problem will stay.
What if we cache always the previously returned value in `nextEntry()` as a
class field:
```
previousEntry = cacheEntries.get(cacheIndex);
```
then remove could be:
```
@Override
public void remove() {
if (previousEntry == null) {
throw new IllegalStateException("The remove
operation must be called after a valid next operation.");
}
if (!previousEntry.deleted) {
previousEntry.remove();
}
}
```
What do you think?
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> [State] Fix arbitrary iterator access on RocksDBMapIterator
> -----------------------------------------------------------
>
> Key: FLINK-10267
> URL: https://issues.apache.org/jira/browse/FLINK-10267
> Project: Flink
> Issue Type: Bug
> Components: State Backends, Checkpointing
> Affects Versions: 1.5.3, 1.6.0
> Reporter: Yun Tang
> Assignee: Yun Tang
> Priority: Major
> Labels: pull-request-available
> Fix For: 1.6.1, 1.5.4
>
>
> Currently, RocksDBMapIterator would load 128 entries into local cacheEntries
> every time if needed. Both RocksDBMapIterator#next() and
> RocksDBMapIterator#hasNext() action might trigger to load RocksDBEntry into
> cacheEntries.
> However, if the iterator's size larger than 128 and we continue to access the
> iterator with following order: hasNext() -> next() -> hasNext() -> remove(),
> we would meet weird exception when we try to remove the 128th element:
> {code:java}
> java.lang.IllegalStateException: The remove operation must be called after a
> valid next operation.
> {code}
> Since we could not control user's access on iterator, we should fix this bug
> to avoid unexpected exception.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
