[jira] [Updated] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17291:
-
Fix Version/s: 3.0.0-alpha6
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-alpha6
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> List batch = new
> ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> }
> clo.result(new MultipleEntryResponse(batch));{code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
>
> Besides that it's required to update
> org.apache.ignite.internal.metastorage.client.CursorImpl with
> client-side-iteration over batched data and requesting new portion if nothing
> left.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17291:
-
Description:
In order to increase metaStorage.range() performance that currently retrieves
entries one by one it's possible to implement simple batching. As an initial
solution we might hardcode the batch-size.
Basically speaking it's required to update CursorNextCommand.
Instead of
{code:java}
Entry e = (Entry) cursorDesc.cursor().next();
clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter())); {code}
we might use something similar to
{code:java}
List batch = new ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
Entry e = (Entry) cursorDesc.cursor().next();
batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter()));
if (! cursorDesc.cursor.hasNext()) {
break;
}
}
clo.result(new MultipleEntryResponse(batch));{code}
It's not trivial to reimplement rocks cursors to also use batching, however
it's not that important because rocks cursors are always local.
Besides that it's required to update
org.apache.ignite.internal.metastorage.client.CursorImpl with
client-side-iteration over batched data and requesting new portion if nothing
left.
was:
In order to increase metaStorage.range() performance that currently retrieves
entries one by one it's possible to implement simple batching. As an initial
solution we might hardcode the batch-size.
Basically speaking it's required to update CursorNextCommand.
Instead of
{code:java}
Entry e = (Entry) cursorDesc.cursor().next();
clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter())); {code}
we might use something similar to
{code:java}
List batch = new ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
Entry e = (Entry) cursorDesc.cursor().next();
batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter()));
if (! cursorDesc.cursor.hasNext()) {
break;
}
}
clo.result(new MultipleEntryResponse(batch));{code}
It's not trivial to reimplement rocks cursors to also use batching, however
it's not that important because rocks cursors are always local.
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> List batch = new
> ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> }
> clo.result(new MultipleEntryResponse(batch));{code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
>
> Besides that it's required to update
> org.apache.ignite.internal.metastorage.client.CursorImpl with
> client-side-iteration over batched data and requesting new portion if nothing
> left.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17291:
-
Description:
In order to increase metaStorage.range() performance that currently retrieves
entries one by one it's possible to implement simple batching. As an initial
solution we might hardcode the batch-size.
Basically speaking it's required to update CursorNextCommand.
Instead of
{code:java}
Entry e = (Entry) cursorDesc.cursor().next();
clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter())); {code}
we might use something similar to
{code:java}
List batch = new ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
Entry e = (Entry) cursorDesc.cursor().next();
batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter()));
if (! cursorDesc.cursor.hasNext()) {
break;
}
}
clo.result(new MultipleEntryResponse(batch));{code}
It's not trivial to reimplement rocks cursors to also use batching, however
it's not that important because rocks cursors are always local.
was:
In order to increase metaStorage.range() performance that currently retrieves
entries one by one it's possible to implement simple batching. As an initial
solution we might hardcode the batch-size.
Basically speaking it's required to update CursorNextCommand.
Instead of
{code:java}
Entry e = (Entry) cursorDesc.cursor().next();
clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter())); {code}
we might use something similar to
{code:java}
for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
Entry e = (Entry) cursorDesc.cursor().next();
batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter()));
if (! cursorDesc.cursor.hasNext()) {
break;
}
} {code}
It's not trivial to reimplement rocks cursors to also use batching, however
it's not that important because rocks cursors are always local.
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> List batch = new
> ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> }
> clo.result(new MultipleEntryResponse(batch));{code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17291:
-
Description:
In order to increase metaStorage.range() performance that currently retrieves
entries one by one it's possible to implement simple batching. As an initial
solution we might hardcode the batch-size.
Basically speaking it's required to update CursorNextCommand.
Instead of
{code:java}
Entry e = (Entry) cursorDesc.cursor().next();
clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter())); {code}
we might use something similar to
{code:java}
for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
Entry e = (Entry) cursorDesc.cursor().next();
batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter()));
if (! cursorDesc.cursor.hasNext()) {
break;
}
} {code}
It's not trivial to reimplement rocks cursors to also use batching, however
it's not that important because rocks cursors are always local.
was:
In order to increase metaStorage.range() performance that currently retrieves
entries one by one it's possible to implement simple batching. As an initial
solution we might hardcode the batch-size.
Basically speaking it's required to update CursorNextCommand.
Instead of
{code:java}
Entry e = (Entry) cursorDesc.cursor().next();
clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter())); {code}
we might use something similar to
{code:java}
for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
Entry e = (Entry) cursorDesc.cursor().next();
batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter()));
if (! cursorDesc.cursor.hasNext()) {
break;
}
} {code}
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
>
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
>
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> } {code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17291:
-
Description:
In order to increase metaStorage.range() performance that currently retrieves
entries one by one it's possible to implement simple batching. As an initial
solution we might hardcode the batch-size.
Basically speaking it's required to update CursorNextCommand.
Instead of
{code:java}
Entry e = (Entry) cursorDesc.cursor().next();
clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter())); {code}
we might use something similar to
{code:java}
for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
Entry e = (Entry) cursorDesc.cursor().next();
batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter()));
if (! cursorDesc.cursor.hasNext()) {
break;
}
} {code}
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
>
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
>
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> } {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
