[jira] [Commented] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17565065#comment-17565065
]
Alexander Lapin commented on IGNITE-17291:
--
[~Denis Chudov] LGTM
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> List batch = new
> ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> }
> clo.result(new MultipleEntryResponse(batch));{code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
>
> Besides that it's required to update
> org.apache.ignite.internal.metastorage.client.CursorImpl with
> client-side-iteration over batched data and requesting new portion if nothing
> left.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564351#comment-17564351
]
Alexander Lapin commented on IGNITE-17291:
--
[~Denis Chudov] In general LGTM, please check comments in PR.
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> List batch = new
> ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> }
> clo.result(new MultipleEntryResponse(batch));{code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
>
> Besides that it's required to update
> org.apache.ignite.internal.metastorage.client.CursorImpl with
> client-side-iteration over batched data and requesting new portion if nothing
> left.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564177#comment-17564177
]
Denis Chudov commented on IGNITE-17291:
---
[~alapin] could you pls review the code?
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> List batch = new
> ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> }
> clo.result(new MultipleEntryResponse(batch));{code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
>
> Besides that it's required to update
> org.apache.ignite.internal.metastorage.client.CursorImpl with
> client-side-iteration over batched data and requesting new portion if nothing
> left.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
