date:20230724



 [ 
https://issues.apache.org/jira/browse/IGNITE-20037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr Polovtcev updated IGNITE-20037:
-
Summary: Fix issues when building Ignite 3 on Java 17  (was: Fix Gradle 
issues when building Ignite 3 on Java 17)

> Fix issues when building Ignite 3 on Java 17
> 
>
> Key: IGNITE-20037
> URL: https://issues.apache.org/jira/browse/IGNITE-20037
> Project: Ignite
>  Issue Type: Task
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I've discovered the following issues, when trying to build Ignite 3 using 
> Java 17:
> # Stackoverflow error in Gradle when some tests failed. Not related to Java 
> 17, but is simply a bug in Gradle 7.5;
> # Not all required modules were present in "--add-opens" declarations;
> # Some reflection operations were banned in Java 17



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20037) Fix Gradle issues when building Ignite 3 on Java 17



 [ 
https://issues.apache.org/jira/browse/IGNITE-20037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr Polovtcev updated IGNITE-20037:
-
Description: 
I've discovered the following issues, when trying to build Ignite 3 using Java 
17:
# Stackoverflow error in Gradle when some tests failed. Not related to Java 17, 
but is simply a bug in Gradle 7.5;
# Not all required modules were present in "--add-opens" declarations;
# Some reflection operations were banned in Java 17

> Fix Gradle issues when building Ignite 3 on Java 17
> ---
>
> Key: IGNITE-20037
> URL: https://issues.apache.org/jira/browse/IGNITE-20037
> Project: Ignite
>  Issue Type: Task
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> I've discovered the following issues, when trying to build Ignite 3 using 
> Java 17:
> # Stackoverflow error in Gradle when some tests failed. Not related to Java 
> 17, but is simply a bug in Gradle 7.5;
> # Not all required modules were present in "--add-opens" declarations;
> # Some reflection operations were banned in Java 17



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-20037) Fix Gradle issues when building Ignite 3 on Java 17

Aleksandr Polovtcev created IGNITE-20037:


 Summary: Fix Gradle issues when building Ignite 3 on Java 17
 Key: IGNITE-20037
 URL: https://issues.apache.org/jira/browse/IGNITE-20037
 Project: Ignite
  Issue Type: Task
Reporter: Aleksandr Polovtcev
Assignee: Aleksandr Polovtcev






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20036) Fix resources leak in Ignite client tests



 [ 
https://issues.apache.org/jira/browse/IGNITE-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr Polovtcev updated IGNITE-20036:
-
Fix Version/s: 3.0.0-beta2

> Fix resources leak in Ignite client tests
> -
>
> Key: IGNITE-20036
> URL: https://issues.apache.org/jira/browse/IGNITE-20036
> Project: Ignite
>  Issue Type: Task
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When investigating a different issue regarding an OOM in tests, I found out 
> that some threads were not stopped in time (OOM was actually happening in 
> these threads, namely "client-data-streamer-flush" thread and other 
> client-related threads). I added some code in {{AfterEach}} that checked if 
> any of these threads were still running and fixed the code to eliminate all 
> these leaks. Even though that didn't solve the original OOM problem, I still 
> think the changes might be useful, because they help to avoid thread leaks.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20036) Fix resources leak in Ignite client tests



 [ 
https://issues.apache.org/jira/browse/IGNITE-20036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr Polovtcev updated IGNITE-20036:
-
Description: When investigating a different issue regarding an OOM in 
tests, I found out that some threads were not stopped in time (OOM was actually 
happening in these threads, namely "client-data-streamer-flush" thread and 
other client-related threads). I added some code in {{AfterEach}} that checked 
if any of these threads were still running and fixed the code to eliminate all 
these leaks. Even though that didn't solve the original OOM problem, I still 
think the changes might be useful, because they help to avoid thread leaks.

> Fix resources leak in Ignite client tests
> -
>
> Key: IGNITE-20036
> URL: https://issues.apache.org/jira/browse/IGNITE-20036
> Project: Ignite
>  Issue Type: Task
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When investigating a different issue regarding an OOM in tests, I found out 
> that some threads were not stopped in time (OOM was actually happening in 
> these threads, namely "client-data-streamer-flush" thread and other 
> client-related threads). I added some code in {{AfterEach}} that checked if 
> any of these threads were still running and fixed the code to eliminate all 
> these leaks. Even though that didn't solve the original OOM problem, I still 
> think the changes might be useful, because they help to avoid thread leaks.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-20036) Fix resources leak in Ignite client tests

Aleksandr Polovtcev created IGNITE-20036:


 Summary: Fix resources leak in Ignite client tests
 Key: IGNITE-20036
 URL: https://issues.apache.org/jira/browse/IGNITE-20036
 Project: Ignite
  Issue Type: Task
Reporter: Aleksandr Polovtcev
Assignee: Aleksandr Polovtcev






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-19863) Fix Compute API async method names, add sync counterparts

2023-07-24 Thread Igor Sapego (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746541#comment-17746541
 ] 

Igor Sapego commented on IGNITE-19863:
--

Looks good to me.

> Fix Compute API async method names, add sync counterparts
> -
>
> Key: IGNITE-19863
> URL: https://issues.apache.org/jira/browse/IGNITE-19863
> Project: Ignite
>  Issue Type: Improvement
>  Components: compute
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> * Async methods should have "Async" suffix (see 
> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-4.1Allasyncmethodsshouldhaveasyncsuffix)
> * Add sync counterparts to be consistent with all the other APIs



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap

[
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Lapin updated IGNITE-20033:
-
Description:
h3. Motivation

For some purposes, in addition to persistent txnState in commit partition, it's
required to have a txn meta on every transactional actor: txn coordinator,
commit partition (both primary and all backups) and all enlisted partitions
(both primary and backups).
[IEP-91|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol#IEP91:Transactionprotocol-Txnstatemap]
names such meta holder as txn state map and specifies following structure
{code:java}
txId -> (txState, txCoordAddr, commitTs){code}
Such map is originally designed to provide required information for
writeIntentResolution: txState for local write intent resolution and
txCoordAddr for coordinator-path one, but that's not all it's good for, so it
definitely worth to implement it as soon as possible in order to unblock
further activities.

-Why we have this ticket in "commit partition recovery" pack?

-Because in order to implement proper handling of a commit partition primary
replica change (which is definitely a part of the "commit partition pack") it's
required to:
# Support local txnStateMap, in order to
# Implement writeIntentResolution coordinator path, in order to
# Calculate commitTimestamp on txn coordtinator istead of commit partition,
where we do it now.

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

Given map should be adjusted before any further transactional actions within
node in a following way:
* Transaction coordinator.
** On transaction start with state PENDING.
** On transaction commit, right after commitTimestamp calucalttion with state
FINISHING.
** On txFinishReplicaResponse with either COMMITED or ABORTED.
* Commit partition.
** On replica touch with state PENDING.
** After succefull finishTxCommand application on majoirty with either
COMMITED or ABORTED.
** Seems that we don't need FINISHING state here, so let's skip it for now.
* Enlisted replica.
** Primary
*** On replica touch with state PENDING.
*** On TxCleanupCommand COMMITED or ABORTED.
** Backup
*** On touch replication PENDING.
*** On TxCleanupCommand COMMITED or ABORTED, same as for primary.

h3. Implementation Notes

There are some open questions:
* Where to put this txnStateMap? TxManager?
* Properly handle same map multiple updates, based on the fact that Commit
Partition is also an enlisted partition. To be honest I believe it's not that
important. We may extent possible state change application rules to assume that
null > PENDING, PENDING > FINISHING, PENDING > COMMITED, PENDING > ABORTED,
PENIGN > PENDING, COMMITED > COMMITED and ABORTED > ABORTED are all possible.
* Eliminate excessive map updates in case of "one-phase commit".
https://issues.apache.org/jira/browse/IGNITE-15927
* Substitute org.apache.ignite.internal.tx.impl.TxManagerImpl#states with
txnStateMap.
* Definte "touch".
* It's reasonbale to use non-consistent node id as txCoordAddr because
currently there's no sense in sending TxStateReq to the node that lost it's
local txnStateMap because of node restart.

was:
h3. Motivation

-Why we have this ticket in "commit partition recovery" pack?

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap

[
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Lapin updated IGNITE-20033:
-
Description:
h3. Motivation

-Why we have this ticket in "commit partition recovery" pack?

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

h3. Implementation Notes

There are some open questions:
* Where to put this txnStateMap? TxManager?
* Properly handle same map multiple updates, based on the fact that Commit
Partition is also an enlisted partition. To be honest I believe it's not that
important. We may extent possible state change application rules to assume that
null > PENDING, PENDING > FINISHING, PENDING > COMMITED, PENDING > ABORTED,
PENIGN > PENDING, COMMITED > COMMITED and ABORTED > ABORTED are all possible.
* Eliminate excessive map updates in case of "one-phase commit".
https://issues.apache.org/jira/browse/IGNITE-15927
* Substitute org.apache.ignite.internal.tx.impl.TxManagerImpl#states with
txnStateMap.
* Definte "touch".
* It's reasonbale to use non-consitent node id as txCoordAddr because
currently there's no sense in sending TxStateReq to the node that lost it's
local txnStateMap because of node restart.

was:
h3. Motivation

-Why we have this ticket in "commit partition recovery" pack?

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary a

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap

[
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Lapin updated IGNITE-20033:
-
Description:
h3. Motivation

-Why we have this ticket in "commit partition recovery" pack?

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

h3. Implementation Notes

There are some open questions:
* Where to put this txnStateMap? TxManager?
* Properly handle same map multiple updates, based on the fact that Commit
Partition is also an enlisted partition. To be honest I believe it's not that
important. We may extent possible state change application rules to assume that
null -> PENDING, PENDING > FINISHING, PENDING > COMMITED, PENDING- > ABORTED,
PENIGN > PENDING, COMMITED -> COMMITED and ABORTED -> ABORTED are all possible.
* Eliminate excessive map updates in case of "one-phase commit".
https://issues.apache.org/jira/browse/IGNITE-15927
* Substitute org.apache.ignite.internal.tx.impl.TxManagerImpl#states with
txnStateMap.
* Definte "touch".
* It's reasonbale to use non-consitent node id as txCoordAddr because
currently there's no sense in sending TxStateReq to the node that lost it's
local txnStateMap because of node restart.

was:
h3. Motivation

-Why we have this ticket in "commit partition recovery" pack?

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both prima

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap

[
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Lapin updated IGNITE-20033:
-
Description:
h3. Motivation

-Why we have this ticket in "commit partition recovery" pack?

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

h3. Implementation Notes

There are some open questions:
* Where to put this txnStateMap? TxManager?
* Properly handle same map multiple updates, based on the fact that Commit
Partition is also an enlisted partition. To be honest I believe it's not that
important. We may extent possible state change application rules to assume that
null {-}> PENDING, PENDING -> FINISHING, PENDING -> COMMITED, PENDING {{-}}>
ABORTED, PENIGN\{-}> PENDING, COMMITED -> COMMITED and ABORTED -> ABORTED are
all possible.
* Eliminate excessive map updates in case of "one-phase commit".
https://issues.apache.org/jira/browse/IGNITE-15927
* Substitute org.apache.ignite.internal.tx.impl.TxManagerImpl#states with
txnStateMap.
* Definte "touch".
* It's reasonbale to use non-consitent node id as txCoordAddr because
currently there's no sense in sending TxStateReq to the node that lost it's
local txnStateMap because of node restart.

was:
h3. Motivation

-Why we have this ticket in "commit partition recovery" pack?

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions,

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap

[
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Lapin updated IGNITE-20033:
-
Description:
h3. Motivation

-Why we have this ticket in "commit partition recovery" pack?

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

h3. Implementation Notes

There are some open questions:
* Where to put this txnStateMap? TxManager?
* Properly handle same map multiple updates, based on the fact that Commit
Partition is also an enlisted partition. To be honest I believe it's not that
important. We may extent possible state change application rules to assume that
null -> PENDING, PENDING -> FINISHING, PENDING -> COMMITED, PENDING {-}>
ABORTED, PENIGN{-}> PENDING, COMMITED -> COMMITED and ABORTED -> ABORTED are
all possible.
* Eliminate excessive map updates in case of "one-phase commit".
https://issues.apache.org/jira/browse/IGNITE-15927
* Substitute org.apache.ignite.internal.tx.impl.TxManagerImpl#states with
txnStateMap.
* Definte "touch".

was:
h3. Motivation

-Why we have this ticket in "commit partition recovery" pack?

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

Given map should be adjusted before any further

[jira] [Updated] (IGNITE-20035) IndexOutOfBoundsException when statement.SetMaxRows is set



 [ 
https://issues.apache.org/jira/browse/IGNITE-20035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Belyak updated IGNITE-20035:
--
Description: 
If setMaxRows > count(*) - query fail with IndexOutOfBound exception.

Reproducer:

 
{noformat}
try (Connection connection = connect(); Statement statement = 
connection.createStatement()) {
JdbcSteps steps = new JdbcSteps(statement);

steps.executeUpdateQuery("CREATE TABLE Person (id INT PRIMARY KEY, name 
VARCHAR)", "Creating a table with two columns.");
steps.executeUpdateQuery("INSERT INTO Person (id, name) VALUES (1, 
'John')", "Inserting a single record");

statement.setMaxRows(25);
ResultSet res = steps.executeQuery("SELECT * FROM Person", "Selecting all 
the records from the table");
while (res.next()) {
log.info("{}, {}", res.getInt(1), res.getString(2));
assertEquals(1, res.getInt(1));
assertEquals("John", res.getString(2));
}
}{noformat}
Returns:

 

 
{noformat}
Exception while executing query [query=SELECT * FROM Person]. Error 
message:toIndex = 25
java.sql.SQLException: Exception while executing query [query=SELECT * FROM 
Person]. Error message:toIndex = 25
    at 
org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
    at 
org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:149)
    at 
org.apache.ignite.internal.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:108)
    at 
org.gridgain.ai3tests.tests.teststeps.JdbcSteps.executeQuery(JdbcSteps.java:50)
    at 
org.gridgain.ai3tests.tests.BasicOperationsTest.testSaveAndGetFromCache(BasicOperationsTest.java:41)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
    at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at 
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
    at 
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
    at 
org.junit.jupiter.api.extension.InvocationInterceptor.interceptTestMethod(InvocationInterceptor.java:118)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    at 
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
    at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
    at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
    at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
    at 
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
    at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
    at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:68)
    at 
org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursiv

[jira] [Updated] (IGNITE-20035) IndexOutOfBoundsException when statement.SetMaxRows is set



 [ 
https://issues.apache.org/jira/browse/IGNITE-20035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Belyak updated IGNITE-20035:
--
Component/s: sql

> IndexOutOfBoundsException when statement.SetMaxRows is set
> --
>
> Key: IGNITE-20035
> URL: https://issues.apache.org/jira/browse/IGNITE-20035
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Alexander Belyak
>Priority: Major
>  Labels: ignite-3
>
> If setMaxRows > count(*) - query fail with IndexOutOfBound exception.
> Reproducer:
>  
> {noformat}
> try (Connection connection = connect(); Statement statement = 
> connection.createStatement()) {
> JdbcSteps steps = new JdbcSteps(statement);
> steps.executeUpdateQuery("CREATE TABLE Person (id INT PRIMARY KEY, name 
> VARCHAR)", "Creating a table with two columns.");
> steps.executeUpdateQuery("INSERT INTO Person (id, name) VALUES (1, 
> 'John')", "Inserting a single record");
> statement.setMaxRows(25);
> ResultSet res = steps.executeQuery("SELECT * FROM Person", "Selecting all 
> the records from the table");
> while (res.next()) {
> log.info("{}, {}", res.getInt(1), res.getString(2));
> assertEquals(1, res.getInt(1));
> assertEquals("John", res.getString(2));
> }
> }{noformat}
> Returns:
>  
>  
> {noformat}
> Exception while executing query [query=SELECT * FROM Person]. Error 
> message:toIndex = 25
> java.sql.SQLException: Exception while executing query [query=SELECT * FROM 
> Person]. Error message:toIndex = 25
>     at 
> org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
>     at 
> org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:149)
>     at 
> org.apache.ignite.internal.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:108)
>     at 
> org.gridgain.ai3tests.tests.teststeps.JdbcSteps.executeQuery(JdbcSteps.java:50)
>     at 
> org.gridgain.ai3tests.tests.BasicOperationsTest.testSaveAndGetFromCache(BasicOperationsTest.java:41)
>     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
> Method)
>     at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>     at 
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
>     at 
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
>     at 
> org.junit.jupiter.api.extension.InvocationInterceptor.interceptTestMethod(InvocationInterceptor.java:118)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
>     at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMet

[jira] [Created] (IGNITE-20035) IndexOutOfBoundsException when statement.SetMaxRows is set

Alexander Belyak created IGNITE-20035:
-

 Summary: IndexOutOfBoundsException when statement.SetMaxRows is set
 Key: IGNITE-20035
 URL: https://issues.apache.org/jira/browse/IGNITE-20035
 Project: Ignite
  Issue Type: Bug
Reporter: Alexander Belyak


If setMaxRows > count(*) - query fail with IndexOutOfBound exception.

Reproducer:

 
{noformat}
try (Connection connection = connect(); Statement statement = 
connection.createStatement()) {
JdbcSteps steps = new JdbcSteps(statement);

steps.executeUpdateQuery("CREATE TABLE Person (id INT PRIMARY KEY, name 
VARCHAR)", "Creating a table with two columns.");
steps.executeUpdateQuery("INSERT INTO Person (id, name) VALUES (1, 
'John')", "Inserting a single record");

statement.setMaxRows(25);
ResultSet res = steps.executeQuery("SELECT * FROM Person", "Selecting all 
the records from the table");
while (res.next()) {
log.info("{}, {}", res.getInt(1), res.getString(2));
assertEquals(1, res.getInt(1));
assertEquals("John", res.getString(2));
}
}{noformat}
Returns:

 

 
{noformat}
Exception while executing query [query=SELECT * FROM Person]. Error 
message:toIndex = 25
java.sql.SQLException: Exception while executing query [query=SELECT * FROM 
Person]. Error message:toIndex = 25
    at 
org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
    at 
org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:149)
    at 
org.apache.ignite.internal.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:108)
    at 
org.gridgain.ai3tests.tests.teststeps.JdbcSteps.executeQuery(JdbcSteps.java:50)
    at 
org.gridgain.ai3tests.tests.BasicOperationsTest.testSaveAndGetFromCache(BasicOperationsTest.java:41)
    at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
Method)
    at 
java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.base/java.lang.reflect.Method.invoke(Method.java:566)
    at 
org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
    at 
org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
    at 
org.junit.jupiter.api.extension.InvocationInterceptor.interceptTestMethod(InvocationInterceptor.java:118)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    at 
org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
    at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
    at 
org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
    at 
org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
    at 
org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
    at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:217)
    at 
org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
    at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:213)
    at 
org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:138)
    at 
org.junit.jupiter.engine.descriptor.TestMethodTestDes

[jira] [Updated] (IGNITE-20035) IndexOutOfBoundsException when statement.SetMaxRows is set



 [ 
https://issues.apache.org/jira/browse/IGNITE-20035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Belyak updated IGNITE-20035:
--
Labels: ignite-3  (was: )

> IndexOutOfBoundsException when statement.SetMaxRows is set
> --
>
> Key: IGNITE-20035
> URL: https://issues.apache.org/jira/browse/IGNITE-20035
> Project: Ignite
>  Issue Type: Bug
>Reporter: Alexander Belyak
>Priority: Major
>  Labels: ignite-3
>
> If setMaxRows > count(*) - query fail with IndexOutOfBound exception.
> Reproducer:
>  
> {noformat}
> try (Connection connection = connect(); Statement statement = 
> connection.createStatement()) {
> JdbcSteps steps = new JdbcSteps(statement);
> steps.executeUpdateQuery("CREATE TABLE Person (id INT PRIMARY KEY, name 
> VARCHAR)", "Creating a table with two columns.");
> steps.executeUpdateQuery("INSERT INTO Person (id, name) VALUES (1, 
> 'John')", "Inserting a single record");
> statement.setMaxRows(25);
> ResultSet res = steps.executeQuery("SELECT * FROM Person", "Selecting all 
> the records from the table");
> while (res.next()) {
> log.info("{}, {}", res.getInt(1), res.getString(2));
> assertEquals(1, res.getInt(1));
> assertEquals("John", res.getString(2));
> }
> }{noformat}
> Returns:
>  
>  
> {noformat}
> Exception while executing query [query=SELECT * FROM Person]. Error 
> message:toIndex = 25
> java.sql.SQLException: Exception while executing query [query=SELECT * FROM 
> Person]. Error message:toIndex = 25
>     at 
> org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
>     at 
> org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:149)
>     at 
> org.apache.ignite.internal.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:108)
>     at 
> org.gridgain.ai3tests.tests.teststeps.JdbcSteps.executeQuery(JdbcSteps.java:50)
>     at 
> org.gridgain.ai3tests.tests.BasicOperationsTest.testSaveAndGetFromCache(BasicOperationsTest.java:41)
>     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
> Method)
>     at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>     at 
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
>     at 
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
>     at 
> org.junit.jupiter.api.extension.InvocationInterceptor.interceptTestMethod(InvocationInterceptor.java:118)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
>     at 
> org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTe

[jira] [Updated] (IGNITE-20035) IndexOutOfBoundsException when statement.SetMaxRows is set



 [ 
https://issues.apache.org/jira/browse/IGNITE-20035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Belyak updated IGNITE-20035:
--
Affects Version/s: 3.0

> IndexOutOfBoundsException when statement.SetMaxRows is set
> --
>
> Key: IGNITE-20035
> URL: https://issues.apache.org/jira/browse/IGNITE-20035
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 3.0
>Reporter: Alexander Belyak
>Priority: Major
>  Labels: ignite-3
>
> If setMaxRows > count(*) - query fail with IndexOutOfBound exception.
> Reproducer:
>  
> {noformat}
> try (Connection connection = connect(); Statement statement = 
> connection.createStatement()) {
> JdbcSteps steps = new JdbcSteps(statement);
> steps.executeUpdateQuery("CREATE TABLE Person (id INT PRIMARY KEY, name 
> VARCHAR)", "Creating a table with two columns.");
> steps.executeUpdateQuery("INSERT INTO Person (id, name) VALUES (1, 
> 'John')", "Inserting a single record");
> statement.setMaxRows(25);
> ResultSet res = steps.executeQuery("SELECT * FROM Person", "Selecting all 
> the records from the table");
> while (res.next()) {
> log.info("{}, {}", res.getInt(1), res.getString(2));
> assertEquals(1, res.getInt(1));
> assertEquals("John", res.getString(2));
> }
> }{noformat}
> Returns:
>  
>  
> {noformat}
> Exception while executing query [query=SELECT * FROM Person]. Error 
> message:toIndex = 25
> java.sql.SQLException: Exception while executing query [query=SELECT * FROM 
> Person]. Error message:toIndex = 25
>     at 
> org.apache.ignite.internal.jdbc.proto.IgniteQueryErrorCode.createJdbcSqlException(IgniteQueryErrorCode.java:57)
>     at 
> org.apache.ignite.internal.jdbc.JdbcStatement.execute0(JdbcStatement.java:149)
>     at 
> org.apache.ignite.internal.jdbc.JdbcStatement.executeQuery(JdbcStatement.java:108)
>     at 
> org.gridgain.ai3tests.tests.teststeps.JdbcSteps.executeQuery(JdbcSteps.java:50)
>     at 
> org.gridgain.ai3tests.tests.BasicOperationsTest.testSaveAndGetFromCache(BasicOperationsTest.java:41)
>     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native 
> Method)
>     at 
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>     at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>     at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>     at 
> org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:727)
>     at 
> org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
>     at 
> org.junit.jupiter.api.extension.InvocationInterceptor.interceptTestMethod(InvocationInterceptor.java:118)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
>     at 
> org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:86)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
>     at 
> org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
>     at 
> org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
>     at 
> org.junit.jupiter.engine.descriptor.TestMethodTe

[jira] [Resolved] (IGNITE-17261) Implement a coordinator path write intent resolution logic for RO reads



 [ 
https://issues.apache.org/jira/browse/IGNITE-17261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin resolved IGNITE-17261.
--
Resolution: Duplicate

> Implement a coordinator path write intent resolution logic for RO reads
> ---
>
> Key: IGNITE-17261
> URL: https://issues.apache.org/jira/browse/IGNITE-17261
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_ro
> Attachments: Screenshot from 2022-07-06 17-00-26.png
>
>
> Because of lock-free nature, RO reads might interact with writeIntents, 
> meaning that such intents should be either evaluated as committed, aborted or 
> pending. In order to perform writeIntent resolution it's required to
>  * If PartitionReplicaListener read a write intent then it checks a local txn 
> state map for committed or aborted state - allow read if the state is 
> committed and commitTs <= readTs.
>  * If not possible, PartitionReplicaListener send TxStateReq to coordinator 
> by ReplicaService. - this initiates the {*}coordinator path{*}. Coordinator 
> address is fetched from the [txn state 
> map|https://docs.google.com/document/d/1PndaylEfK7CPUN7Kv9RYPKASN299s2qlbnMA5xIOFXo/edit#heading=h.wx3zf7jlf156].
>  * If a coordinator path was not able to resolve the intent, one of the 
> following has happened - the coordinator is dead or txn state is not 
> available in the cache. Calculate a commit partition and send the TxStateReq 
> to its primary replica - this initiates the {*}commit partition path{*}.
>  * Retry commit partition path until a success or timeout.
> On receiving TxStateReq in ReplicaManager on the coordinator:
>  * ReplicaManager reads txn state map. If the local txn is finished, return 
> the response with the outcome: commit or abort. The txn state is stored in a 
> local cache (https://issues.apache.org/jira/browse/IGNITE-17638)
>  * If the local txn is finishing (txState == Finishing) waiting for finish 
> state replication, wait with timeout for outcome and return response with the 
> outcome: commit or abort. txState become Finishing in TxManager on creating 
> TxFinishReplicaRequest. TxManager has a txn state map. We can use future for 
> concurrency and atomic operations on txn state map.
>  * If the outcome is commit, additional timestamp check is required: a commit 
> timestamp must be <= readTs. If the condition is not held, the outcome is 
> changed to abort.
>  * If local txn is active (txState != [finishing, commit, abort]), adjust the 
> txn coordinator node HLC according to readTs to make sure the txn commit 
> timestamp is above the read timestamp. The read timestamp must be installed 
> before txn is started to commit, so commit timestamp is assigned after the 
> read timestamp.
>  * If txn state is not found in a local cache and txn is not active, return 
> NULL.
>  
> There's an open question about MvPartitionStorage api feature: 
> https://issues.apache.org/jira/browse/IGNITE-17627



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20005) Move commitTimestamp generation to the txn coordinator side.



 [ 
https://issues.apache.org/jira/browse/IGNITE-20005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20005:
-
Description: 
h3. Motivation

Please check the context in https://issues.apache.org/jira/browse/IGNITE-20033 
and https://issues.apache.org/jira/browse/IGNITE-20034

After implemention writeIntentResolution coordinator path it'll be nessessary 
to move commitTimestamp generation to the txn coordinator side.
h3. Definition of Done
 * Easy-breezy, calculate commitTimestamp as coordinator.now().

h3. Implementation Notes

Nothing to update on the "client" side, we already have everything we need.
{code:java}
public CompletableFuture finish() {
...
HybridTimestamp commitTimestamp = commit ? clock.now() : null;

TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
 ...
.commitTimestampLong(hybridTimestampToLong(commitTimestamp)) // 2
   ...
} {code}
On the "server" side, we will adjust and simplify code in 
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener#finishTransaction
{code:java}
private CompletableFuture finishTransaction(List 
aggregatedGroupIds, UUID txId, boolean commit) {
// TODO: IGNITE-17261 Timestamp from request is not using until the issue 
has not been fixed (request.commitTimestamp())
var fut = new CompletableFuture();

txTimestampUpdateMap.put(txId, fut);

HybridTimestamp currentTimestamp = hybridClock.now();
HybridTimestamp commitTimestamp = commit ? currentTimestamp : null; {code}

  was:
h3. Motivation

It's possible that commit partition primary replica lease will expire in the 
following phases:
 * Before txnState change

 * During txnState change. Meaning that it's actually before or after, however 
we don't know an actual state before interaction.
 * After txnState change.

Last part is covered with https://issues.apache.org/jira/browse/IGNITE-20002 
and https://issues.apache.org/jira/browse/IGNITE-20004 So, given ticket is 
about lease expiration during first and second phases. In both cases it's 
required to send rollback that will change txnState to ABORTED if it wasn't 
already setted to terminal state previously and return actual terminal state. 
Meaning that it's possible to send rollback to a previously committed 
transaction, and that rollback won't have any effect, but will return COMMITED 
to the TxCoordinator and thus client.
h3. Definition of Done
 * It's required to verify that we properly handle commit partition primary 
replica expiration and commit partition primary replica failure by sending 
rollback request to the new commit partition primary.
 * It's required to slightly adjust 
org.apache.ignite.internal.tx.impl.TxManagerImpl#finish along with 
corresponding internals that will return actual txnState. Meaning that we 
should complete user's commit successfully if we send inner rollback after 
unnoticeable previous commit.

 


> Move commitTimestamp generation to the txn coordinator side.
> 
>
> Key: IGNITE-20005
> URL: https://issues.apache.org/jira/browse/IGNITE-20005
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction, transaction3_recovery
>
> h3. Motivation
> Please check the context in 
> https://issues.apache.org/jira/browse/IGNITE-20033 and 
> https://issues.apache.org/jira/browse/IGNITE-20034
> After implemention writeIntentResolution coordinator path it'll be nessessary 
> to move commitTimestamp generation to the txn coordinator side.
> h3. Definition of Done
>  * Easy-breezy, calculate commitTimestamp as coordinator.now().
> h3. Implementation Notes
> Nothing to update on the "client" side, we already have everything we need.
> {code:java}
> public CompletableFuture finish() {
> ...
> HybridTimestamp commitTimestamp = commit ? clock.now() : null;
> TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
>  ...
> .commitTimestampLong(hybridTimestampToLong(commitTimestamp)) // 2
>...
> } {code}
> On the "server" side, we will adjust and simplify code in 
> org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener#finishTransaction
> {code:java}
> private CompletableFuture finishTransaction(List 
> aggregatedGroupIds, UUID txId, boolean commit) {
> // TODO: IGNITE-17261 Timestamp from request is not using until the issue 
> has not been fixed (request.commitTimestamp())
> var fut = new CompletableFuture();
> txTimestampUpdateMap.put(txId, fut);
> HybridTimestamp currentTimestamp = hybridClock.now();
> HybridTimestamp commitTimestamp = commit ? currentTimestamp : null; {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20031) NPE on the lease updater

2023-07-24 Thread Vyacheslav Koptilin (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-20031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-20031:
-
Fix Version/s: 3.0.0-beta2

> NPE on the lease updater
> 
>
> Key: IGNITE-20031
> URL: https://issues.apache.org/jira/browse/IGNITE-20031
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> During a deactivation of the active actor of Meta storage, it might reset 
> {{leaseNegotiator}} reference before {{updaterTread}} is stopped.
> {noformat}
> 2023-07-23 03:10:07:490 +0300 
> [WARNING][%irsvt_imtw_0%lease-updater-0][LeaseUpdater] Lease updater is 
> interrupted
>   Exception in thread "%irsvt_imtw_0%lease-updater-0" 
> java.lang.NullPointerException
> at 
> org.apache.ignite.internal.placementdriver.LeaseUpdater$Updater.run(LeaseUpdater.java:301)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20005) Move commitTimestamp generation to the txn coordinator side.



 [ 
https://issues.apache.org/jira/browse/IGNITE-20005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20005:
-
Summary: Move commitTimestamp generation to the txn coordinator side.  
(was: Handle commitPartition primaryReplica lease expiration)

> Move commitTimestamp generation to the txn coordinator side.
> 
>
> Key: IGNITE-20005
> URL: https://issues.apache.org/jira/browse/IGNITE-20005
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction, transaction3_recovery
>
> h3. Motivation
> It's possible that commit partition primary replica lease will expire in the 
> following phases:
>  * Before txnState change
>  * During txnState change. Meaning that it's actually before or after, 
> however we don't know an actual state before interaction.
>  * After txnState change.
> Last part is covered with https://issues.apache.org/jira/browse/IGNITE-20002 
> and https://issues.apache.org/jira/browse/IGNITE-20004 So, given ticket is 
> about lease expiration during first and second phases. In both cases it's 
> required to send rollback that will change txnState to ABORTED if it wasn't 
> already setted to terminal state previously and return actual terminal state. 
> Meaning that it's possible to send rollback to a previously committed 
> transaction, and that rollback won't have any effect, but will return 
> COMMITED to the TxCoordinator and thus client.
> h3. Definition of Done
>  * It's required to verify that we properly handle commit partition primary 
> replica expiration and commit partition primary replica failure by sending 
> rollback request to the new commit partition primary.
>  * It's required to slightly adjust 
> org.apache.ignite.internal.tx.impl.TxManagerImpl#finish along with 
> corresponding internals that will return actual txnState. Meaning that we 
> should complete user's commit successfully if we send inner rollback after 
> unnoticeable previous commit.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20034) Implement writeIntentResolution coordinator path



 [ 
https://issues.apache.org/jira/browse/IGNITE-20034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20034:
-
Description: 
h3. Motivation

In order to implement proper handling of a commit partition primary replica 
change it's required to:
 # Support local txnStateMap, in order to
 # Implement writeIntentResolution coordinator path, in order to
 # Calculate commitTimestamp on txn coordtinator istead of commit partition, 
where we do it now.

First item is covered in https://issues.apache.org/jira/browse/IGNITE-20033 
Given ticket is about second item only. Third one will be covered in a separate 
ticket.
h3. Definition of Done

>From the "client" side, it's required to adjust current writeIntentResolution 
>logic by checking local txnStateLocal map.
 * If there's no corresponding transaction in given map - fallback to the 
commitPartition path.
 * If the state is COMMITED or ABORTED - use given value as 
writeInentResolution base. Actually, by doing this we will implelent 
writeIntentResolution local path.
 * If the state is FINISHING - wait for outcome and return response with the 
result: commit or abort. One of the most nontrivial steps, because of awaiting 
logic.
 * If the state is PENDING, retrieve txCoordAddr and ask tx coordinator.
 ** If tx coordinator is missing or returns null fallback to the commit 
partition path.

>From the "server" side
 * If the local txn is COMMITED or ABORTED, return the response with the 
outcome: commit or abort.
 * If the local txn is FINISHING ** (waiting for finish state replication), 
wait for outcome and return response with the result: commit or abort
 ** If the result is to commit, additional timestamp check is required: a 
commit timestamp must be <= readTs. If the condition is not held, the result is 
changed to abort.
 * If local txn is PENDING, adjust the txn coordinator node HLC according to 
readTs to make sure the txn commit timestamp is above the read timestamp. The 
read timestamp must be installed before txn is started to commit, so commit 
timestamp is assigned after the read timestamp. This must be achieved by some 
sort of concurrency control, preferably non-blocking. In this case we must 
ignore the write intent, so the outcome is to abort.
 * If txn state is not found in a local cache and txn is not active, return 
NULL.

h3. Implementation Notes

Open questions:
 * Which component is responsible for handling TxStateReq?
 * How to await FINISHING state termination?

  was:
h3. Motivation

Long story short


> Implement writeIntentResolution coordinator path
> 
>
> Key: IGNITE-20034
> URL: https://issues.apache.org/jira/browse/IGNITE-20034
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction, transaction3_recovery
>
> h3. Motivation
> In order to implement proper handling of a commit partition primary replica 
> change it's required to:
>  # Support local txnStateMap, in order to
>  # Implement writeIntentResolution coordinator path, in order to
>  # Calculate commitTimestamp on txn coordtinator istead of commit partition, 
> where we do it now.
> First item is covered in https://issues.apache.org/jira/browse/IGNITE-20033 
> Given ticket is about second item only. Third one will be covered in a 
> separate ticket.
> h3. Definition of Done
> From the "client" side, it's required to adjust current writeIntentResolution 
> logic by checking local txnStateLocal map.
>  * If there's no corresponding transaction in given map - fallback to the 
> commitPartition path.
>  * If the state is COMMITED or ABORTED - use given value as 
> writeInentResolution base. Actually, by doing this we will implelent 
> writeIntentResolution local path.
>  * If the state is FINISHING - wait for outcome and return response with the 
> result: commit or abort. One of the most nontrivial steps, because of 
> awaiting logic.
>  * If the state is PENDING, retrieve txCoordAddr and ask tx coordinator.
>  ** If tx coordinator is missing or returns null fallback to the commit 
> partition path.
> From the "server" side
>  * If the local txn is COMMITED or ABORTED, return the response with the 
> outcome: commit or abort.
>  * If the local txn is FINISHING ** (waiting for finish state replication), 
> wait for outcome and return response with the result: commit or abort
>  ** If the result is to commit, additional timestamp check is required: a 
> commit timestamp must be <= readTs. If the condition is not held, the result 
> is changed to abort.
>  * If local txn is PENDING, adjust the txn coordinator node HLC according to 
> readTs to make sure the txn commit timestamp is above the read timestamp. The 
> read timestamp must be installed before txn is started to commit, so commit

[jira] [Commented] (IGNITE-20031) NPE on the lease updater



[ 
https://issues.apache.org/jira/browse/IGNITE-20031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746485#comment-17746485
 ] 

Vladislav Pyatkov commented on IGNITE-20031:


Merged 0e5049f7cc479d6596e224e5a741225990d124f6

> NPE on the lease updater
> 
>
> Key: IGNITE-20031
> URL: https://issues.apache.org/jira/browse/IGNITE-20031
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> During a deactivation of the active actor of Meta storage, it might reset 
> {{leaseNegotiator}} reference before {{updaterTread}} is stopped.
> {noformat}
> 2023-07-23 03:10:07:490 +0300 
> [WARNING][%irsvt_imtw_0%lease-updater-0][LeaseUpdater] Lease updater is 
> interrupted
>   Exception in thread "%irsvt_imtw_0%lease-updater-0" 
> java.lang.NullPointerException
> at 
> org.apache.ignite.internal.placementdriver.LeaseUpdater$Updater.run(LeaseUpdater.java:301)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-20031) NPE on the lease updater

2023-07-24 Thread Denis Chudov (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-20031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746482#comment-17746482
 ] 

Denis Chudov commented on IGNITE-20031:
---

[~v.pyatkov] LGTM

> NPE on the lease updater
> 
>
> Key: IGNITE-20031
> URL: https://issues.apache.org/jira/browse/IGNITE-20031
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> During a deactivation of the active actor of Meta storage, it might reset 
> {{leaseNegotiator}} reference before {{updaterTread}} is stopped.
> {noformat}
> 2023-07-23 03:10:07:490 +0300 
> [WARNING][%irsvt_imtw_0%lease-updater-0][LeaseUpdater] Lease updater is 
> interrupted
>   Exception in thread "%irsvt_imtw_0%lease-updater-0" 
> java.lang.NullPointerException
> at 
> org.apache.ignite.internal.placementdriver.LeaseUpdater$Updater.run(LeaseUpdater.java:301)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap

[
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Lapin updated IGNITE-20033:
-
Description:
h3. Motivation

-Why we have this ticket in "commit partition recovery" pack?

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

h3. Implementation Notes

There are some open questions:
* Where to put this txnStateMap? TxManager?
* Properly handle same map multiple updates, based on the fact that Commit
Partition is also an enlisted partition. To be honest I believe it's not that
important. We may extent possible state change application rules to assume that
null -> PENDING, PENDING -> FINISHING, PENDING -> COMMITED, PENDING -> ABORTED,
COMMITED -> COMMITED and ABORTED -> ABORTED are all possible.
* Eliminate excessive map updates in case of "one-phase commit".
https://issues.apache.org/jira/browse/IGNITE-15927
* Substitute org.apache.ignite.internal.tx.impl.TxManagerImpl#states with
txnStateMap.

was:
h3. Motivation

- Because in order to implement proper handling of a commit partition primary
replica change (which is definitely a part of the "commit partition pack") it's
required to:

# Support local txnStateMap, in order to
# Implement writeIntentResolution coordinator path, in order to
# Calculate commitTimestamp on txn coordtinator istead of commit partition,
where we do it now.

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

Given map should be adjusted before any further transactional actions within
node in a

[jira] [Updated] (IGNITE-20034) Implement writeIntentResolution coordinator path



 [ 
https://issues.apache.org/jira/browse/IGNITE-20034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20034:
-
Description: 
h3. Motivation

Long story short

  was:h3. Motivation


> Implement writeIntentResolution coordinator path
> 
>
> Key: IGNITE-20034
> URL: https://issues.apache.org/jira/browse/IGNITE-20034
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction, transaction3_recovery
>
> h3. Motivation
> Long story short



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20030) Delete from table with composite PK failed



 [ 
https://issues.apache.org/jira/browse/IGNITE-20030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yury Gerzhedovich updated IGNITE-20030:
---
Attachment: IGNITE-20030.txt

> Delete from table with composite PK failed
> --
>
> Key: IGNITE-20030
> URL: https://issues.apache.org/jira/browse/IGNITE-20030
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Alexander Belyak
>Priority: Major
>  Labels: ignite-3
> Attachments: IGNITE-20030.txt, IGNITE-20030.zip
>
>
>  
> {noformat}
> drop table if exists testt;
> create table testt (id int, val int, id2 int not null, val2 varchar(255), 
> primary key(id,id2) );
> insert into testt(id, val, id2, val2) 
> values(1,1,1,'1'),(2,2,2,'2'),(3,3,3,'3');
> delete from testt where id = 2;
> delete from testt where id = 2 and id2 = 2;{noformat}
> Expected result: row deleted without errors
>  
> Actual result:
>  
> {noformat}
> [Code: 0, SQL State: 5]  Exception while executing query [query=delete 
> from testt where id = 2]. Error message:Query remote fragment execution 
> failed: nodeName=SmokeOdbcTest_cluster_0, 
> queryId=d47ef0ae-65dd-4762-8e6b-0b40e0333d27, fragmentId=1793, 
> originalMessage=null{noformat}
> Logs in attachment.
>  
> AI3 commit id: f34beaed



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-18875) Sql. Drop AbstractPlannerTest.TestTable.



 [ 
https://issues.apache.org/jira/browse/IGNITE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yury Gerzhedovich updated IGNITE-18875:
---
Epic Link: IGNITE-19502

> Sql. Drop AbstractPlannerTest.TestTable.
> 
>
> Key: IGNITE-18875
> URL: https://issues.apache.org/jira/browse/IGNITE-18875
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Andrey Mashenkov
>Assignee: Gael Yimen Yimga
>Priority: Major
>  Labels: ignite-3, newbie, tech-debt-test
> Fix For: 3.0.0-beta2
>
> Attachments: Screen Shot 2023-04-03 at 1.04.39 AM.png
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> {{org.apache.ignite.internal.sql.engine.planner.AbstractPlannerTest.TestTable}}
>  uses 
> IgniteTypeFactory.createJavaType() method to create RelDataType from java 
> classes.
> We should create tables in tests in same way we do in product code.
> Let's use test framework for schema configuration in tests and replace 
> {code:java}
> org.apache.ignite.internal.sql.engine.planner.AbstractPlannerTest.TestTable
> {code}
> usage with 
> {code:java}
> org.apache.ignite.internal.sql.engine.framework.TestTable
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-18875) Sql. Drop AbstractPlannerTest.TestTable.



[ 
https://issues.apache.org/jira/browse/IGNITE-18875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746460#comment-17746460
 ] 

Yury Gerzhedovich commented on IGNITE-18875:


[~yimengael]Are you still works on the ticket?

> Sql. Drop AbstractPlannerTest.TestTable.
> 
>
> Key: IGNITE-18875
> URL: https://issues.apache.org/jira/browse/IGNITE-18875
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Andrey Mashenkov
>Assignee: Gael Yimen Yimga
>Priority: Major
>  Labels: ignite-3, newbie, tech-debt-test
> Fix For: 3.0.0-beta2
>
> Attachments: Screen Shot 2023-04-03 at 1.04.39 AM.png
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> {{org.apache.ignite.internal.sql.engine.planner.AbstractPlannerTest.TestTable}}
>  uses 
> IgniteTypeFactory.createJavaType() method to create RelDataType from java 
> classes.
> We should create tables in tests in same way we do in product code.
> Let's use test framework for schema configuration in tests and replace 
> {code:java}
> org.apache.ignite.internal.sql.engine.planner.AbstractPlannerTest.TestTable
> {code}
> usage with 
> {code:java}
> org.apache.ignite.internal.sql.engine.framework.TestTable
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20034) Implement writeIntentResolution coordinator path



 [ 
https://issues.apache.org/jira/browse/IGNITE-20034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20034:
-
Epic Link: IGNITE-20001

> Implement writeIntentResolution coordinator path
> 
>
> Key: IGNITE-20034
> URL: https://issues.apache.org/jira/browse/IGNITE-20034
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction, transaction3_recovery
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20034) Implement writeIntentResolution coordinator path



 [ 
https://issues.apache.org/jira/browse/IGNITE-20034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20034:
-
Description: h3. Motivation

> Implement writeIntentResolution coordinator path
> 
>
> Key: IGNITE-20034
> URL: https://issues.apache.org/jira/browse/IGNITE-20034
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction, transaction3_recovery
>
> h3. Motivation



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-20034) Implement writeIntentResolution coordinator path

Alexander Lapin created IGNITE-20034:


 Summary: Implement writeIntentResolution coordinator path
 Key: IGNITE-20034
 URL: https://issues.apache.org/jira/browse/IGNITE-20034
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexander Lapin






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20034) Implement writeIntentResolution coordinator path



 [ 
https://issues.apache.org/jira/browse/IGNITE-20034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20034:
-
Labels: ignite-3 transaction transaction3_recovery  (was: )

> Implement writeIntentResolution coordinator path
> 
>
> Key: IGNITE-20034
> URL: https://issues.apache.org/jira/browse/IGNITE-20034
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction, transaction3_recovery
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20034) Implement writeIntentResolution coordinator path



 [ 
https://issues.apache.org/jira/browse/IGNITE-20034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20034:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Implement writeIntentResolution coordinator path
> 
>
> Key: IGNITE-20034
> URL: https://issues.apache.org/jira/browse/IGNITE-20034
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction, transaction3_recovery
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap

[
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Lapin updated IGNITE-20033:
-
Description:
h3. Motivation

- Because in order to implement proper handling of a commit partition primary
replica change (which is definitely a part of the "commit partition pack") it's
required to:

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

h3. Implementation Notes

There are some open questions:
* Where to put this txnStateMap? TxManager?
* Properly handle same map multiple updates, based on the fact that Commit
Partition is also an enlisted partition. To be honest I believe it's not that
important. We may extent possible state change application rules to assume that
null -> PENDING, PENDING -> FINISHING, PENDING -> COMMITED, PENDING -> ABORTED,
COMMITED -> COMMITED and ABORTED -> ABORTED are all possible.
* Eliminate excessive map updates in case of "one-phase commit".
https://issues.apache.org/jira/browse/IGNITE-15927
* Substitute org.apache.ignite.internal.tx.impl.TxManagerImpl#states with
txnStateMap.

was:
h3. Motivation

- Because in order to implement proper handling of a commit partition primary
replica change (which is definitely a part of the "commit partition pack") it's
required to:

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

Given map should be adjusted before any further transactional actions within
node

[jira] [Resolved] (IGNITE-17638) Implement Txn state map to optimize write intent resolution



 [ 
https://issues.apache.org/jira/browse/IGNITE-17638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin resolved IGNITE-17638.
--
Resolution: Duplicate

> Implement Txn state map to optimize write intent resolution
> ---
>
> Key: IGNITE-17638
> URL: https://issues.apache.org/jira/browse/IGNITE-17638
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Sergey Uttsel
>Priority: Major
>  Labels: ignite-3, transaction3_ro
>
> Need to implement Txn state map which contains txId -> (txState,  
> txCoordinatorAdress, commitTimestamp).
>  # This map will be located on all nodes in replication group (primary and 
> followers) in PartitionReplicaListener and PartitionListener.
>  # Updated on ReadWriteReplicaRequest receiving in PartitionReplicaListener. 
> Initially, only txCoordinatorAddress is known. The coordinator address is 
> available in ReplicaManager on receiving a ReplicaRequest. Need to pass it to 
> PartitionReplicaListener. 
>  # PartitionReplicaListener use raftClient commands for replication. So need 
> to send coordinator address with raft command and save it in 
> PartitionListener.
>  # On TxCleanupReplicaRequest receiving need to set txState and 
> commitTimestamp (if commited).
>  
> Txn state map optimizes write intent resolution. If commitTs <= readTs then 
> RO transaction treat a write intent as committed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap



 [ 
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20033:
-
Description: 
h3. Motivation

For some purposes, in addition to persistent txnState in commit partition, it's 
required to have a txn meta on every transactional actor: txn coordinator, 
commit partition (both primary and all backups) and all enlisted partitions 
(both primary and backups). 
[IEP-91|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol#IEP91:Transactionprotocol-Txnstatemap]
 names such meta holder as txn state map and specifies following structure

 
{code:java}
txId -> (txState, txCoordAddr, commitTs){code}
Such map is originally designed to provide required information for 
writeIntentResolution: txState for local write intent resolution and 
txCoordAddr for coordinator-path one, but that's not all it's good for, so it 
definitely worth to implement it as soon as possible in order to unblock 
further activities.

- Why we have this ticket in "commit partition recovery" pack?

- Because in order to implement proper handling of a commit partition primary 
replica change (which is definitely a part of the "commit partition pack") it's 
required to:
 # Support local txnStateMap, in order to
 # Implement writeIntentResolution coordinator path, in order to
 # Calculate commitTimestamp on txn coordtinator istead of commit partition, 
where we do it now.

h3. Definition of Done

Every transactional actor
 * Txn Coordinator.
 * CommitPartition, both primary and all backups.
 * All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId -> 
(txState, txCoordAddr, commitTs)'.

 

Given map should be adjusted before any further transactional actions within 
node in a following way:
 * Transaction coordinator.
 ** On transaction start with state PENDING.
 ** On transaction commit, right after commitTimestamp calucalttion with state 
FINISHING.
 ** On txFinishReplicaResponse with either COMMITED or ABORTED.
 * Commit partition.
 ** On replica touch with state PENDING.
 ** After succefull finishTxCommand application on majoirty with either 
COMMITED or ABORTED.
 ** Seems that we don't need FINISHING state here, so let's skip it for now.
 * Enlisted replica.
 ** Primary
 *** On replica touch with state PENDING.
 *** On TxCleanupCommand COMMITED or ABORTED.
 ** Backup
 *** On touch replication PENDING.
 *** On TxCleanupCommand COMMITED or ABORTED, same as for primary.

h3. Implementation Notes

There are some open questions:
 * Where to put this txnStateMap? TxManager?
 * Properly handle same map multiple updates, based on the fact that Commit 
Partition is also an enlisted partition. To be honest I believe it's not that 
important. We may extent possible state change application rules to assume that 
null -> PENDING, PENDING -> FINISHING, PENDING -> COMMITED, PENDING -> ABORTED, 
COMMITED -> COMMITED and ABORTED -> ABORTED are all possible.
 * Eliminate excessive map updates in case of "one-phase commit".
 * Substitute org.apache.ignite.internal.tx.impl.TxManagerImpl#states with 
txnStateMap.

 

> Implement local txnStateMap
> ---
>
> Key: IGNITE-20033
> URL: https://issues.apache.org/jira/browse/IGNITE-20033
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_recovery, transactions
>
> h3. Motivation
> For some purposes, in addition to persistent txnState in commit partition, 
> it's required to have a txn meta on every transactional actor: txn 
> coordinator, commit partition (both primary and all backups) and all enlisted 
> partitions (both primary and backups). 
> [IEP-91|https://cwiki.apache.org/confluence/display/IGNITE/IEP-91%3A+Transaction+protocol#IEP91:Transactionprotocol-Txnstatemap]
>  names such meta holder as txn state map and specifies following structure
>  
> {code:java}
> txId -> (txState, txCoordAddr, commitTs){code}
> Such map is originally designed to provide required information for 
> writeIntentResolution: txState for local write intent resolution and 
> txCoordAddr for coordinator-path one, but that's not all it's good for, so it 
> definitely worth to implement it as soon as possible in order to unblock 
> further activities.
> - Why we have this ticket in "commit partition recovery" pack?
> - Because in order to implement proper handling of a commit partition primary 
> replica change (which is definitely a part of the "commit partition pack") 
> it's required to:
>  # Support local txnStateMap, in order to
>  # Implement writeIntentResolution coordinator path, in order to
>  # Calculate commitTimestamp on txn coordtinator istead of commit partition, 
> where we do it now.
> h3. Definition of

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap

[
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Alexander Lapin updated IGNITE-20033:
-
Description:
h3. Motivation

- Because in order to implement proper handling of a commit partition primary
replica change (which is definitely a part of the "commit partition pack") it's
required to:

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

h3. Implementation Notes

was:
h3. Motivation

{code:java}
txId -> (txState, txCoordAddr, commitTs){code}
Such map is originally designed to provide required information for
writeIntentResolution: txState for local write intent resolution and
txCoordAddr for coordinator-path one, but that's not all it's good for, so it
definitely worth to implement it as soon as possible in order to unblock
further activities.

- Why we have this ticket in "commit partition recovery" pack?

- Because in order to implement proper handling of a commit partition primary
replica change (which is definitely a part of the "commit partition pack") it's
required to:
# Support local txnStateMap, in order to
# Implement writeIntentResolution coordinator path, in order to
# Calculate commitTimestamp on txn coordtinator istead of commit partition,
where we do it now.

h3. Definition of Done

Every transactional actor
* Txn Coordinator.
* CommitPartition, both primary and all backups.
* All enlisted partitions, both primary and all backups.

should have volatile txnStateMap with following suggested structure 'txId ->
(txState, txCoordAddr, commitTs)'.

Given map should be adjusted before any further transactional actions within
node in a following way:
* Transaction coordinator.
**

[jira] [Updated] (IGNITE-19960) Backport 1.3.10 release form the JRaft

2023-07-24 Thread Mirza Aliev (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-19960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mirza Aliev updated IGNITE-19960:
-
Description: 
In this task, we need to backport commits from 1.3.10 release from original 
JRaft repo.

The main driver of this activity is backpressure feature for task applying for 
Node#apply(task) and Node#readIndex(task) which determines submitting tasks to 
node in blocking or non-blocking mode, ApplyTaskMode.NonBlocking by default. In 
blocking mode, it will block the invocation to these two methods when node is 
overloaded, and throws OverloadException immediately in non-blocking mode 
instead. 

Also there are some minor bug fixes.

There is a list of commits to backport: 

||Commit message||Link||
|use atomic move to avoid file corruption (#745)|   
https://github.com/sofastack/sofa-jraft/commit/1a9df327afcba7c1b1a5cc527531ae70812c309f|
|Support a config to make read index read failfast. (#738)| 
https://github.com/sofastack/sofa-jraft/commit/4f027979bffb6fb329934f97647ffee44cb76f96|
|(feat) Adds sliceData and getReadOnlyData methods to LogEntry, #755 (#762)|
https://github.com/sofastack/sofa-jraft/commit/75dad4bb4ad6e76927d9c0db6baf83bfc5368306|
|Feature/backpressure (#764) Adds ApplyTaskMode to apply task in blocking or 
non-blocking mode, and improve disruptor usage|
https://github.com/sofastack/sofa-jraft/commit/5de2fbbcabb70ddbefb06a1d3737821781c3e85c|
|(feat) Use deleteFilesInRange/compactRange to replace deleteRange in … (#769)| 

https://github.com/sofastack/sofa-jraft/commit/e9ae4e477576fd98d9dd539faf7470b6b938e98b|
|(fix) ClassCastException in SnapshotExecutorImpl, #728 (#775)| 
https://github.com/sofastack/sofa-jraft/commit/0eaaf957e42051ae82a504f1d6676b7d02d460f2|
|-(fix) refactor ThreadId and fix #781 (#783)-| 
https://github.com/sofastack/sofa-jraft/commit/530224e398b43d22e60ab444bddb819f1e838241|
|Feature/fix node test (#790)|  
https://github.com/sofastack/sofa-jraft/commit/f69e7e9e9b2d048d73ab40844ae78fd62a559277|


UPD: It was decided to port {{(fix) refactor ThreadId and fix #781 (#783)}}  
https://github.com/sofastack/sofa-jraft/commit/530224e398b43d22e60ab444bddb819f1e838241
 in a separate ticket because we need to investigate why this change brings 
hanging thread after run tests from 
{{org.apache.ignite.raft.jraft.core.ItNodeTest}}

  was:
In this task, we need to backport commits from 1.3.10 release from original 
JRaft repo.

The main driver of this activity is backpressure feature for task applying for 
Node#apply(task) and Node#readIndex(task) which determines submitting tasks to 
node in blocking or non-blocking mode, ApplyTaskMode.NonBlocking by default. In 
blocking mode, it will block the invocation to these two methods when node is 
overloaded, and throws OverloadException immediately in non-blocking mode 
instead. 

Also there are some minor bug fixes.

There is a list of commits to backport: 

||Commit message||Link||
|use atomic move to avoid file corruption (#745)|   
https://github.com/sofastack/sofa-jraft/commit/1a9df327afcba7c1b1a5cc527531ae70812c309f|
|Support a config to make read index read failfast. (#738)| 
https://github.com/sofastack/sofa-jraft/commit/4f027979bffb6fb329934f97647ffee44cb76f96|
|(feat) Adds sliceData and getReadOnlyData methods to LogEntry, #755 (#762)|
https://github.com/sofastack/sofa-jraft/commit/75dad4bb4ad6e76927d9c0db6baf83bfc5368306|
|Feature/backpressure (#764) Adds ApplyTaskMode to apply task in blocking or 
non-blocking mode, and improve disruptor usage|
https://github.com/sofastack/sofa-jraft/commit/5de2fbbcabb70ddbefb06a1d3737821781c3e85c|
|(feat) Use deleteFilesInRange/compactRange to replace deleteRange in … (#769)| 

https://github.com/sofastack/sofa-jraft/commit/e9ae4e477576fd98d9dd539faf7470b6b938e98b|
|(fix) ClassCastException in SnapshotExecutorImpl, #728 (#775)| 
https://github.com/sofastack/sofa-jraft/commit/0eaaf957e42051ae82a504f1d6676b7d02d460f2|
|-(fix) refactor ThreadId and fix #781 (#783)-| 
https://github.com/sofastack/sofa-jraft/commit/530224e398b43d22e60ab444bddb819f1e838241|
|Feature/fix node test (#790)|  
https://github.com/sofastack/sofa-jraft/commit/f69e7e9e9b2d048d73ab40844ae78fd62a559277|



> Backport 1.3.10 release form the JRaft
> --
>
> Key: IGNITE-19960
> URL: https://issues.apache.org/jira/browse/IGNITE-19960
> Project: Ignite
>  Issue Type: Task
>Reporter: Mirza Aliev
>Assignee: Mirza Aliev
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In this task, we need to backport commits from 1.3.10 release from original 
> JRaft repo.
> The main driver of this activity is backpressure feature for task applying 
> for Node#apply(task) and Node#readIndex(task) which determines

[jira] [Updated] (IGNITE-19960) Backport 1.3.10 release form the JRaft

2023-07-24 Thread Mirza Aliev (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-19960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mirza Aliev updated IGNITE-19960:
-
Description: 
In this task, we need to backport commits from 1.3.10 release from original 
JRaft repo.

The main driver of this activity is backpressure feature for task applying for 
Node#apply(task) and Node#readIndex(task) which determines submitting tasks to 
node in blocking or non-blocking mode, ApplyTaskMode.NonBlocking by default. In 
blocking mode, it will block the invocation to these two methods when node is 
overloaded, and throws OverloadException immediately in non-blocking mode 
instead. 

Also there are some minor bug fixes.

There is a list of commits to backport: 

||Commit message||Link||
|use atomic move to avoid file corruption (#745)|   
https://github.com/sofastack/sofa-jraft/commit/1a9df327afcba7c1b1a5cc527531ae70812c309f|
|Support a config to make read index read failfast. (#738)| 
https://github.com/sofastack/sofa-jraft/commit/4f027979bffb6fb329934f97647ffee44cb76f96|
|(feat) Adds sliceData and getReadOnlyData methods to LogEntry, #755 (#762)|
https://github.com/sofastack/sofa-jraft/commit/75dad4bb4ad6e76927d9c0db6baf83bfc5368306|
|Feature/backpressure (#764) Adds ApplyTaskMode to apply task in blocking or 
non-blocking mode, and improve disruptor usage|
https://github.com/sofastack/sofa-jraft/commit/5de2fbbcabb70ddbefb06a1d3737821781c3e85c|
|(feat) Use deleteFilesInRange/compactRange to replace deleteRange in … (#769)| 

https://github.com/sofastack/sofa-jraft/commit/e9ae4e477576fd98d9dd539faf7470b6b938e98b|
|(fix) ClassCastException in SnapshotExecutorImpl, #728 (#775)| 
https://github.com/sofastack/sofa-jraft/commit/0eaaf957e42051ae82a504f1d6676b7d02d460f2|
|-(fix) refactor ThreadId and fix #781 (#783)-| 
https://github.com/sofastack/sofa-jraft/commit/530224e398b43d22e60ab444bddb819f1e838241|
|Feature/fix node test (#790)|  
https://github.com/sofastack/sofa-jraft/commit/f69e7e9e9b2d048d73ab40844ae78fd62a559277|


  was:
In this task, we need to backport commits from 1.3.10 release from original 
JRaft repo.

The main driver of this activity is backpressure feature for task applying for 
Node#apply(task) and Node#readIndex(task) which determines submitting tasks to 
node in blocking or non-blocking mode, ApplyTaskMode.NonBlocking by default. In 
blocking mode, it will block the invocation to these two methods when node is 
overloaded, and throws OverloadException immediately in non-blocking mode 
instead. 

Also there are some minor bug fixes.

There is a list of commits to backport: 

||Commit message||Link||
|use atomic move to avoid file corruption (#745)|   
https://github.com/sofastack/sofa-jraft/commit/1a9df327afcba7c1b1a5cc527531ae70812c309f|
|Support a config to make read index read failfast. (#738)| 
https://github.com/sofastack/sofa-jraft/commit/4f027979bffb6fb329934f97647ffee44cb76f96|
|(feat) Adds sliceData and getReadOnlyData methods to LogEntry, #755 (#762)|
https://github.com/sofastack/sofa-jraft/commit/75dad4bb4ad6e76927d9c0db6baf83bfc5368306|
|Feature/backpressure (#764) Adds ApplyTaskMode to apply task in blocking or 
non-blocking mode, and improve disruptor usage|
https://github.com/sofastack/sofa-jraft/commit/5de2fbbcabb70ddbefb06a1d3737821781c3e85c|
|(feat) Use deleteFilesInRange/compactRange to replace deleteRange in … (#769)| 

https://github.com/sofastack/sofa-jraft/commit/e9ae4e477576fd98d9dd539faf7470b6b938e98b|
|(fix) ClassCastException in SnapshotExecutorImpl, #728 (#775)| 
https://github.com/sofastack/sofa-jraft/commit/0eaaf957e42051ae82a504f1d6676b7d02d460f2|
|(fix) refactor ThreadId and fix #781 (#783)|   
https://github.com/sofastack/sofa-jraft/commit/530224e398b43d22e60ab444bddb819f1e838241|
|Feature/fix node test (#790)|  
https://github.com/sofastack/sofa-jraft/commit/f69e7e9e9b2d048d73ab40844ae78fd62a559277|



> Backport 1.3.10 release form the JRaft
> --
>
> Key: IGNITE-19960
> URL: https://issues.apache.org/jira/browse/IGNITE-19960
> Project: Ignite
>  Issue Type: Task
>Reporter: Mirza Aliev
>Assignee: Mirza Aliev
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> In this task, we need to backport commits from 1.3.10 release from original 
> JRaft repo.
> The main driver of this activity is backpressure feature for task applying 
> for Node#apply(task) and Node#readIndex(task) which determines submitting 
> tasks to node in blocking or non-blocking mode, ApplyTaskMode.NonBlocking by 
> default. In blocking mode, it will block the invocation to these two methods 
> when node is overloaded, and throws OverloadException immediately in 
> non-blocking mode instead. 
> Also there are some minor bug fixes.
> There is a list

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap



 [ 
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20033:
-
Labels: ignite-3 transaction3_recovery transactions  (was: )

> Implement local txnStateMap
> ---
>
> Key: IGNITE-20033
> URL: https://issues.apache.org/jira/browse/IGNITE-20033
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_recovery, transactions
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap



 [ 
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20033:
-
Epic Link: IGNITE-20001

> Implement local txnStateMap
> ---
>
> Key: IGNITE-20033
> URL: https://issues.apache.org/jira/browse/IGNITE-20033
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3, transaction3_recovery, transactions
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-20033) Implement local txnStateMap

Alexander Lapin created IGNITE-20033:


 Summary: Implement local txnStateMap
 Key: IGNITE-20033
 URL: https://issues.apache.org/jira/browse/IGNITE-20033
 Project: Ignite
  Issue Type: Improvement
Reporter: Alexander Lapin






--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20033) Implement local txnStateMap



 [ 
https://issues.apache.org/jira/browse/IGNITE-20033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-20033:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Implement local txnStateMap
> ---
>
> Key: IGNITE-20033
> URL: https://issues.apache.org/jira/browse/IGNITE-20033
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-19966) .NET: Thin 3.0: SIMD-accelerated hashing

2023-07-24 Thread Pavel Tupitsyn (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-19966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-19966:

Epic Link: IGNITE-19479

> .NET: Thin 3.0: SIMD-accelerated hashing
> 
>
> Key: IGNITE-19966
> URL: https://issues.apache.org/jira/browse/IGNITE-19966
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Minor
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-beta2
>
>
> [HashUtils.Hash64Internal|https://github.com/apache/ignite-3/blob/7e1e0cf74c741a8d9c7c9e678535a4ca04fb8dc9/modules/platforms/dotnet/Apache.Ignite/Internal/Proto/HashUtils.cs#L187]
>  can benefit from [SIMD 
> acceleration|https://learn.microsoft.com/en-us/dotnet/standard/simd] on large 
> values.
> Currently it splits the data into blocks of 16 bytes (2 longs), but we can 
> process larger blocks with *Vector* (4 pairs at once with AVX2, 8 pairs 
> at once with AVX512).
> Example: https://gist.github.com/dnbaker/0fc1d4edbbdb24069eb063dc2559e4f5



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (IGNITE-19301) Add --all flag to undeploy command



 [ 
https://issues.apache.org/jira/browse/IGNITE-19301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Pochatkin resolved IGNITE-19301.

Resolution: Won't Fix

> Add --all flag to undeploy command
> --
>
> Key: IGNITE-19301
> URL: https://issues.apache.org/jira/browse/IGNITE-19301
> Project: Ignite
>  Issue Type: New Feature
>  Components: cli
>Reporter: Mikhail Pochatkin
>Priority: Major
>  Labels: iep-103, ignite-3
>
> Intorduce --all flag to undeploy command which mutually exclusive to 
> --version. This flag should undeploy all version of provided unit.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Resolved] (IGNITE-19825) Warning during rest integration test build



 [ 
https://issues.apache.org/jira/browse/IGNITE-19825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Pochatkin resolved IGNITE-19825.

Resolution: Won't Fix

> Warning during rest integration test build
> --
>
> Key: IGNITE-19825
> URL: https://issues.apache.org/jira/browse/IGNITE-19825
> Project: Ignite
>  Issue Type: Bug
>  Components: rest
>Reporter: Vadim Pakhnushev
>Assignee: Vadim Pakhnushev
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
>  
> The following warnings are printed during 
> {{:ignite-rest:integrationTestClasses}} build
> {noformat}
> warning: unknown enum constant RequiredMode.REQUIRED
> reason: class file for 
> io.swagger.v3.oas.annotations.media.Schema$RequiredMode not found 
> {noformat}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-19647) Our Cache Templates introductory documentation is misleading -- implies SQL Create Table only

2023-07-24 Thread Igor Gusev (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-19647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Gusev reassigned IGNITE-19647:
---

Assignee: Igor Gusev  (was: Alex Levitski)

> Our Cache Templates introductory documentation is misleading -- implies SQL 
> Create Table only
> -
>
> Key: IGNITE-19647
> URL: https://issues.apache.org/jira/browse/IGNITE-19647
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Alex Levitski
>Assignee: Igor Gusev
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Our introductory documentation (below) on [Cache 
> Templates|https://ignite.apache.org/docs/latest/configuring-caches/configuration-overview#cache-templates]
>  is misleading in that it gives the strong impression that Cache Templates 
> are only useable with the SQL API, whereas it is my understanding that it can 
> also be used with CreateCache and GetOrCreateCache commands.
> It would be helpful to add this clarification and also add links to where we 
> describe how to use CacheTemplates with CreateCache/GetOrCreateCache commands 
> (or add such documentation and examples if this is not documented at all).
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-19717) Get rid of creating indexes at the start of a MvTableStorage

2023-07-24 Thread Sergey Chugunov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-19717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Chugunov updated IGNITE-19717:
-
Epic Link: IGNITE-17766  (was: IGNITE-19502)

> Get rid of creating indexes at the start of a MvTableStorage
> 
>
> Key: IGNITE-19717
> URL: https://issues.apache.org/jira/browse/IGNITE-19717
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Now, at the start of the *MvTableStorage*, we create indexes, which is not 
> correct, this should be done by external code that should know about all 
> indexes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-19112) Get rid of MvTableStorage#getOrCreateIndex

2023-07-24 Thread Sergey Chugunov (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-19112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Chugunov updated IGNITE-19112:
-
Epic Link: IGNITE-17766  (was: IGNITE-19502)

> Get rid of MvTableStorage#getOrCreateIndex
> --
>
> Key: IGNITE-19112
> URL: https://issues.apache.org/jira/browse/IGNITE-19112
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Get rid of methods:
> * org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateIndex
> * 
> org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateHashIndex
> * 
> org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateSortedIndex
> and make new ones:
> * org.apache.ignite.internal.storage.engine.MvTableStorage#createIndex
> * org.apache.ignite.internal.storage.engine.MvTableStorage#getOrCreateIndex
> * org.apache.ignite.internal.storage.engine.MvTableStorage#createHashIndex
> * org.apache.ignite.internal.storage.engine.MvTableStorage#getHashIndex
> * org.apache.ignite.internal.storage.engine.MvTableStorage#createSortedIndex
> * org.apache.ignite.internal.storage.engine.MvTableStorage#getSortedIndex



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-20029) Reduce heap size for integration tests



[ 
https://issues.apache.org/jira/browse/IGNITE-20029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746412#comment-17746412
 ] 

Vladislav Pyatkov commented on IGNITE-20029:


Merged 43e5b004c522cb970366adc39d42a1f8233e15b1

> Reduce heap size for integration tests
> --
>
> Key: IGNITE-20029
> URL: https://issues.apache.org/jira/browse/IGNITE-20029
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Mikhail Pochatkin
>Assignee: Mikhail Pochatkin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently, our integration test set 16g xmx. This is so much for us, because 
> we can lose some problems with memory leaks. Also, currently our TC agents 
> docker has only 18g RAM. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-20029) Reduce heap size for integration tests



[ 
https://issues.apache.org/jira/browse/IGNITE-20029?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746411#comment-17746411
 ] 

Vladislav Pyatkov commented on IGNITE-20029:


LGTM

> Reduce heap size for integration tests
> --
>
> Key: IGNITE-20029
> URL: https://issues.apache.org/jira/browse/IGNITE-20029
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Mikhail Pochatkin
>Assignee: Mikhail Pochatkin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently, our integration test set 16g xmx. This is so much for us, because 
> we can lose some problems with memory leaks. Also, currently our TC agents 
> docker has only 18g RAM. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-20031) NPE on the lease updater



 [ 
https://issues.apache.org/jira/browse/IGNITE-20031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladislav Pyatkov reassigned IGNITE-20031:
--

Assignee: Vladislav Pyatkov

> NPE on the lease updater
> 
>
> Key: IGNITE-20031
> URL: https://issues.apache.org/jira/browse/IGNITE-20031
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> During a deactivation of the active actor of Meta storage, it might reset 
> {{leaseNegotiator}} reference before {{updaterTread}} is stopped.
> {noformat}
> 2023-07-23 03:10:07:490 +0300 
> [WARNING][%irsvt_imtw_0%lease-updater-0][LeaseUpdater] Lease updater is 
> interrupted
>   Exception in thread "%irsvt_imtw_0%lease-updater-0" 
> java.lang.NullPointerException
> at 
> org.apache.ignite.internal.placementdriver.LeaseUpdater$Updater.run(LeaseUpdater.java:301)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-19861) Introduce SQL metrics



 [ 
https://issues.apache.org/jira/browse/IGNITE-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yury Gerzhedovich updated IGNITE-19861:
---
Description: 
AI3 have metrics framework, but SQL engine don't expose any metrics, which 
could help to user.
Let's introduce the first five metrics:
1) hit in plan cache - counter
2) miss in plan cache - counter
3) number of open cursors - gauge

  was:
AI3 have metrics framework, but SQL engine don't expose any metrics, which 
could help to user.
Let's introduce the first five metrics:
1) hit in plan cache - counter
2) miss in plan cache - counter
3) size of queue in query pool - gauge
4) number of open cursors - gauge


> Introduce SQL metrics
> -
>
> Key: IGNITE-19861
> URL: https://issues.apache.org/jira/browse/IGNITE-19861
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Reporter: Yury Gerzhedovich
>Assignee: Yury Gerzhedovich
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> AI3 have metrics framework, but SQL engine don't expose any metrics, which 
> could help to user.
> Let's introduce the first five metrics:
> 1) hit in plan cache - counter
> 2) miss in plan cache - counter
> 3) number of open cursors - gauge



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-20032) NPE on a try to get a RAFT node's striped executor pool

Vladislav Pyatkov created IGNITE-20032:
--

 Summary: NPE on a try to get a RAFT node's striped executor pool
 Key: IGNITE-20032
 URL: https://issues.apache.org/jira/browse/IGNITE-20032
 Project: Ignite
  Issue Type: Bug
Reporter: Vladislav Pyatkov


Stopping the striped pool can happen earlier than the last RPC messafe is 
processed.
Just after the pool is stopped, its reference is reset to {{null}}:
{code:java|title=org.apache.ignite.raft.jraft.core.NodeImpl#join|borderStyle=solid}
if (opts.getStripedExecutor() != null && !opts.isSharedPools()) {
opts.getStripedExecutor().shutdownGracefully();
opts.setStripedExecutor(null);
 }
{code}
{noformat}
2023-07-21 18:45:10:571 +0300 
[ERROR][%int_tcpcat_5004%MessagingService-inbound--0][DefaultMessagingService] 
onMessage() failed while processing InvokeRequestImpl [correlationId=30, 
message=AppendEntriesRequestImpl [committedIndex=18558, 
data=org.apache.ignite.raft.jraft.util.ByteString@1, entriesList=null, 
groupId=unittest, peerId=int_tcpcat_5004, prevLogIndex=18559, prevLogTerm=1, 
serverId=int_tcpcat_5007, term=2, timestampLong=110752845696729088]] from 
int_tcpcat_5007
java.lang.NullPointerException
  at 
org.apache.ignite.raft.jraft.rpc.impl.core.AppendEntriesRequestProcessor.getOrCreatePeerRequestContext(AppendEntriesRequestProcessor.java:351)
  at 
org.apache.ignite.raft.jraft.rpc.impl.core.AppendEntriesRequestProcessor$PeerExecutorSelector.select(AppendEntriesRequestProcessor.java:72)
  at 
org.apache.ignite.raft.jraft.rpc.impl.IgniteRpcServer$RpcMessageHandler.onReceived(IgniteRpcServer.java:182)
  at 
org.apache.ignite.network.DefaultMessagingService.onMessage(DefaultMessagingService.java:375)
  at 
org.apache.ignite.network.DefaultMessagingService.lambda$onMessage$4(DefaultMessagingService.java:335)
  at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
  at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
  at java.base/java.lang.Thread.run(Thread.java:834)
{noformat}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20031) NPE on the lease updater



 [ 
https://issues.apache.org/jira/browse/IGNITE-20031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladislav Pyatkov updated IGNITE-20031:
---
Labels: ignite-3  (was: )

> NPE on the lease updater
> 
>
> Key: IGNITE-20031
> URL: https://issues.apache.org/jira/browse/IGNITE-20031
> Project: Ignite
>  Issue Type: Bug
>Reporter: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
>
> During a deactivation of the active actor of Meta storage, it might reset 
> {{leaseNegotiator}} reference before {{updaterTread}} is stopped.
> {noformat}
> 2023-07-23 03:10:07:490 +0300 
> [WARNING][%irsvt_imtw_0%lease-updater-0][LeaseUpdater] Lease updater is 
> interrupted
>   Exception in thread "%irsvt_imtw_0%lease-updater-0" 
> java.lang.NullPointerException
> at 
> org.apache.ignite.internal.placementdriver.LeaseUpdater$Updater.run(LeaseUpdater.java:301)
> {noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-20031) NPE on the lease updater

Vladislav Pyatkov created IGNITE-20031:
--

 Summary: NPE on the lease updater
 Key: IGNITE-20031
 URL: https://issues.apache.org/jira/browse/IGNITE-20031
 Project: Ignite
  Issue Type: Bug
Reporter: Vladislav Pyatkov


During a deactivation of the active actor of Meta storage, it might reset 
{{leaseNegotiator}} reference before {{updaterTread}} is stopped.

{noformat}
2023-07-23 03:10:07:490 +0300 
[WARNING][%irsvt_imtw_0%lease-updater-0][LeaseUpdater] Lease updater is 
interrupted
  Exception in thread "%irsvt_imtw_0%lease-updater-0" 
java.lang.NullPointerException
at 
org.apache.ignite.internal.placementdriver.LeaseUpdater$Updater.run(LeaseUpdater.java:301)
{noformat}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20030) Delete from table with composite PK failed

2023-07-24 Thread Evgeny Stanilovsky (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-20030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Evgeny Stanilovsky updated IGNITE-20030:

Component/s: sql

> Delete from table with composite PK failed
> --
>
> Key: IGNITE-20030
> URL: https://issues.apache.org/jira/browse/IGNITE-20030
> Project: Ignite
>  Issue Type: Task
>  Components: sql
>Reporter: Alexander Belyak
>Priority: Major
>  Labels: ignite-3
> Attachments: IGNITE-20030.zip
>
>
>  
> {noformat}
> drop table if exists testt;
> create table testt (id int, val int, id2 int not null, val2 varchar(255), 
> primary key(id,id2) );
> insert into testt(id, val, id2, val2) 
> values(1,1,1,'1'),(2,2,2,'2'),(3,3,3,'3');
> delete from testt where id = 2;
> delete from testt where id = 2 and id2 = 2;{noformat}
> Expected result: row deleted without errors
>  
> Actual result:
>  
> {noformat}
> [Code: 0, SQL State: 5]  Exception while executing query [query=delete 
> from testt where id = 2]. Error message:Query remote fragment execution 
> failed: nodeName=SmokeOdbcTest_cluster_0, 
> queryId=d47ef0ae-65dd-4762-8e6b-0b40e0333d27, fragmentId=1793, 
> originalMessage=null{noformat}
> Logs in attachment.
>  
> AI3 commit id: f34beaed



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20030) Delete from table with composite PK failed

2023-07-24 Thread Evgeny Stanilovsky (Jira)



 [ 
https://issues.apache.org/jira/browse/IGNITE-20030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Evgeny Stanilovsky updated IGNITE-20030:

Labels: ignite-3  (was: )

> Delete from table with composite PK failed
> --
>
> Key: IGNITE-20030
> URL: https://issues.apache.org/jira/browse/IGNITE-20030
> Project: Ignite
>  Issue Type: Task
>Reporter: Alexander Belyak
>Priority: Major
>  Labels: ignite-3
> Attachments: IGNITE-20030.zip
>
>
>  
> {noformat}
> drop table if exists testt;
> create table testt (id int, val int, id2 int not null, val2 varchar(255), 
> primary key(id,id2) );
> insert into testt(id, val, id2, val2) 
> values(1,1,1,'1'),(2,2,2,'2'),(3,3,3,'3');
> delete from testt where id = 2;
> delete from testt where id = 2 and id2 = 2;{noformat}
> Expected result: row deleted without errors
>  
> Actual result:
>  
> {noformat}
> [Code: 0, SQL State: 5]  Exception while executing query [query=delete 
> from testt where id = 2]. Error message:Query remote fragment execution 
> failed: nodeName=SmokeOdbcTest_cluster_0, 
> queryId=d47ef0ae-65dd-4762-8e6b-0b40e0333d27, fragmentId=1793, 
> originalMessage=null{noformat}
> Logs in attachment.
>  
> AI3 commit id: f34beaed



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-20011) Shutdown NodeStatusWatchListener thread pool



[ 
https://issues.apache.org/jira/browse/IGNITE-20011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746285#comment-17746285
 ] 

Vladislav Pyatkov commented on IGNITE-20011:


[~Mikhail Pochatkin] Thank you for your contribution.
Merged 16aaee407c2463929eac0d8dd208221452dfdc06

> Shutdown NodeStatusWatchListener thread pool 
> -
>
> Key: IGNITE-20011
> URL: https://issues.apache.org/jira/browse/IGNITE-20011
> Project: Ignite
>  Issue Type: Bug
>Reporter: Mikhail Pochatkin
>Assignee: Mikhail Pochatkin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Currently, 
> org.apache.ignite.internal.deployunit.metastore.NodeStatusWatchListener#executor
>  didn't shutdown after 
> org.apache.ignite.internal.deployunit.DeploymentManagerImpl stop. Need to fix 
> it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-20011) Shutdown NodeStatusWatchListener thread pool



[ 
https://issues.apache.org/jira/browse/IGNITE-20011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746281#comment-17746281
 ] 

Vladislav Pyatkov commented on IGNITE-20011:


LGTM

> Shutdown NodeStatusWatchListener thread pool 
> -
>
> Key: IGNITE-20011
> URL: https://issues.apache.org/jira/browse/IGNITE-20011
> Project: Ignite
>  Issue Type: Bug
>Reporter: Mikhail Pochatkin
>Assignee: Mikhail Pochatkin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently, 
> org.apache.ignite.internal.deployunit.metastore.NodeStatusWatchListener#executor
>  didn't shutdown after 
> org.apache.ignite.internal.deployunit.DeploymentManagerImpl stop. Need to fix 
> it.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-20021) Too many Raft-FSMCaller-Disruptor threads



[ 
https://issues.apache.org/jira/browse/IGNITE-20021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746280#comment-17746280
 ] 

Vladislav Pyatkov commented on IGNITE-20021:


Merged f380ff60c9873a3a497893ae3dd59e95f9965e8c

> Too many Raft-FSMCaller-Disruptor threads
> -
>
> Key: IGNITE-20021
> URL: https://issues.apache.org/jira/browse/IGNITE-20021
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
> Attachments: threads.txt
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> There are too many threads in the attached dump. The majority of them related 
> to stopped nodes:
> {noformat}
> "%irclilurt_n_1%JRaft-FSMCaller-Disruptor-_stripe_5-0" #147152 daemon prio=5 
> os_prio=0 cpu=0.11ms elapsed=184.09s tid=0x7f8a95578000 nid=0xfe0d3 
> waiting on condition  [0x7f85a94e4000]
>java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11.0.17/Native Method)
> - parking to wait for  <0x00040aee3db8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@11.0.17/LockSupport.java:194)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.17/AbstractQueuedSynchronizer.java:2081)
> at 
> com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:45)
> at 
> com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
> at 
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at java.lang.Thread.run(java.base@11.0.17/Thread.java:834)
> {noformat}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-20021) Too many Raft-FSMCaller-Disruptor threads

2023-07-24 Thread Denis Chudov (Jira)



[ 
https://issues.apache.org/jira/browse/IGNITE-20021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746279#comment-17746279
 ] 

Denis Chudov commented on IGNITE-20021:
---

[~v.pyatkov] LGTM

> Too many Raft-FSMCaller-Disruptor threads
> -
>
> Key: IGNITE-20021
> URL: https://issues.apache.org/jira/browse/IGNITE-20021
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Vladislav Pyatkov
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
> Attachments: threads.txt
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> There are too many threads in the attached dump. The majority of them related 
> to stopped nodes:
> {noformat}
> "%irclilurt_n_1%JRaft-FSMCaller-Disruptor-_stripe_5-0" #147152 daemon prio=5 
> os_prio=0 cpu=0.11ms elapsed=184.09s tid=0x7f8a95578000 nid=0xfe0d3 
> waiting on condition  [0x7f85a94e4000]
>java.lang.Thread.State: WAITING (parking)
> at jdk.internal.misc.Unsafe.park(java.base@11.0.17/Native Method)
> - parking to wait for  <0x00040aee3db8> (a 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
> at 
> java.util.concurrent.locks.LockSupport.park(java.base@11.0.17/LockSupport.java:194)
> at 
> java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(java.base@11.0.17/AbstractQueuedSynchronizer.java:2081)
> at 
> com.lmax.disruptor.BlockingWaitStrategy.waitFor(BlockingWaitStrategy.java:45)
> at 
> com.lmax.disruptor.ProcessingSequenceBarrier.waitFor(ProcessingSequenceBarrier.java:56)
> at 
> com.lmax.disruptor.BatchEventProcessor.run(BatchEventProcessor.java:128)
> at java.lang.Thread.run(java.base@11.0.17/Thread.java:834)
> {noformat}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20030) Delete from table with composite PK failed



 [ 
https://issues.apache.org/jira/browse/IGNITE-20030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Belyak updated IGNITE-20030:
--
Attachment: IGNITE-20030.zip

> Delete from table with composite PK failed
> --
>
> Key: IGNITE-20030
> URL: https://issues.apache.org/jira/browse/IGNITE-20030
> Project: Ignite
>  Issue Type: Task
>Reporter: Alexander Belyak
>Priority: Major
> Attachments: IGNITE-20030.zip
>
>
>  
> {noformat}
> drop table if exists testt;
> create table testt (id int, val int, id2 int not null, val2 varchar(255), 
> primary key(id,id2) );
> insert into testt(id, val, id2, val2) 
> values(1,1,1,'1'),(2,2,2,'2'),(3,3,3,'3');
> delete from testt where id = 2;
> delete from testt where id = 2 and id2 = 2;{noformat}
> Expected result: row deleted without errors
>  
> Actual result:
>  
> {noformat}
> [Code: 0, SQL State: 5]  Exception while executing query [query=delete 
> from testt where id = 2]. Error message:Query remote fragment execution 
> failed: nodeName=SmokeOdbcTest_cluster_0, 
> queryId=d47ef0ae-65dd-4762-8e6b-0b40e0333d27, fragmentId=1793, 
> originalMessage=null{noformat}
> Logs in attachment.
>  
> AI3 commit id: f34beaed



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (IGNITE-20030) Delete from table with composite PK failed



 [ 
https://issues.apache.org/jira/browse/IGNITE-20030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Belyak updated IGNITE-20030:
--
Summary: Delete from table with composite PK failed  (was: Delete from 
table failed if )

> Delete from table with composite PK failed
> --
>
> Key: IGNITE-20030
> URL: https://issues.apache.org/jira/browse/IGNITE-20030
> Project: Ignite
>  Issue Type: Task
>Reporter: Alexander Belyak
>Priority: Major
> Attachments: IGNITE-20030.zip
>
>
>  
> {noformat}
> drop table if exists testt;
> create table testt (id int, val int, id2 int not null, val2 varchar(255), 
> primary key(id,id2) );
> insert into testt(id, val, id2, val2) 
> values(1,1,1,'1'),(2,2,2,'2'),(3,3,3,'3');
> delete from testt where id = 2;
> delete from testt where id = 2 and id2 = 2;{noformat}
> Expected result: row deleted without errors
>  
> Actual result:
>  
> {noformat}
> [Code: 0, SQL State: 5]  Exception while executing query [query=delete 
> from testt where id = 2]. Error message:Query remote fragment execution 
> failed: nodeName=SmokeOdbcTest_cluster_0, 
> queryId=d47ef0ae-65dd-4762-8e6b-0b40e0333d27, fragmentId=1793, 
> originalMessage=null{noformat}
> Logs in attachment.
>  
> AI3 commit id: f34beaed



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-20030) Delete from table failed if

Alexander Belyak created IGNITE-20030:
-

 Summary: Delete from table failed if 
 Key: IGNITE-20030
 URL: https://issues.apache.org/jira/browse/IGNITE-20030
 Project: Ignite
  Issue Type: Task
Reporter: Alexander Belyak


 
{noformat}
drop table if exists testt;
create table testt (id int, val int, id2 int not null, val2 varchar(255), 
primary key(id,id2) );
insert into testt(id, val, id2, val2) values(1,1,1,'1'),(2,2,2,'2'),(3,3,3,'3');
delete from testt where id = 2;
delete from testt where id = 2 and id2 = 2;{noformat}
Expected result: row deleted without errors

 

Actual result:

 
{noformat}
[Code: 0, SQL State: 5]  Exception while executing query [query=delete from 
testt where id = 2]. Error message:Query remote fragment execution failed: 
nodeName=SmokeOdbcTest_cluster_0, queryId=d47ef0ae-65dd-4762-8e6b-0b40e0333d27, 
fragmentId=1793, originalMessage=null{noformat}
Logs in attachment.

 

AI3 commit id: f34beaed



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-19971) Flaky ItNodeTest#testLeaseReadAfterSegmentation



[ 
https://issues.apache.org/jira/browse/IGNITE-19971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746259#comment-17746259
 ] 

Vladislav Pyatkov commented on IGNITE-19971:


Merged ddb5a7c1dfb18963e1632d214cd780627bbb2376

> Flaky ItNodeTest#testLeaseReadAfterSegmentation
> ---
>
> Key: IGNITE-19971
> URL: https://issues.apache.org/jira/browse/IGNITE-19971
> Project: Ignite
>  Issue Type: Bug
>Reporter: Mirza Aliev
>Assignee: Mirza Aliev
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> ItNodeTest#testLeaseReadAfterSegmentation is flaky
> {code:java}
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.ignite.raft.jraft.Node.getNodeId()" because the return value of 
> "org.apache.ignite.raft.jraft.core.TestCluster.getLeader()" is null
>   at 
> org.apache.ignite.raft.jraft.core.ItNodeTest.lambda$testLeaseReadAfterSegmentation$43(ItNodeTest.java:3652)
> {code}
> The problem was in the race in 
> {code:java}
> assertTrue(waitForCondition(() -> cluster.getLeader() != null &&
> !leader.getNodeId().equals(cluster.getLeader().getNodeId()), 
> 10_000));
> {code}
> so leader was stepped done in the middle of this assertion, so the first part 
> of this expression {{cluster.getLeader() != null}} was true.
> The fix is to rewrite condition to defence against NPE. 
> After 300 local runs there is no failures, but 3 failures in 100 runs were 
> before fix



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-19971) Flaky ItNodeTest#testLeaseReadAfterSegmentation



[ 
https://issues.apache.org/jira/browse/IGNITE-19971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17746256#comment-17746256
 ] 

Vladislav Pyatkov commented on IGNITE-19971:


LGTM

> Flaky ItNodeTest#testLeaseReadAfterSegmentation
> ---
>
> Key: IGNITE-19971
> URL: https://issues.apache.org/jira/browse/IGNITE-19971
> Project: Ignite
>  Issue Type: Bug
>Reporter: Mirza Aliev
>Assignee: Mirza Aliev
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> ItNodeTest#testLeaseReadAfterSegmentation is flaky
> {code:java}
> java.lang.NullPointerException: Cannot invoke 
> "org.apache.ignite.raft.jraft.Node.getNodeId()" because the return value of 
> "org.apache.ignite.raft.jraft.core.TestCluster.getLeader()" is null
>   at 
> org.apache.ignite.raft.jraft.core.ItNodeTest.lambda$testLeaseReadAfterSegmentation$43(ItNodeTest.java:3652)
> {code}
> The problem was in the race in 
> {code:java}
> assertTrue(waitForCondition(() -> cluster.getLeader() != null &&
> !leader.getNodeId().equals(cluster.getLeader().getNodeId()), 
> 10_000));
> {code}
> so leader was stepped done in the middle of this assertion, so the first part 
> of this expression {{cluster.getLeader() != null}} was true.
> The fix is to rewrite condition to defence against NPE. 
> After 300 local runs there is no failures, but 3 failures in 100 runs were 
> before fix



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Assigned] (IGNITE-20029) Reduce heap size for integration tests



 [ 
https://issues.apache.org/jira/browse/IGNITE-20029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mikhail Pochatkin reassigned IGNITE-20029:
--

Assignee: Mikhail Pochatkin

> Reduce heap size for integration tests
> --
>
> Key: IGNITE-20029
> URL: https://issues.apache.org/jira/browse/IGNITE-20029
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Mikhail Pochatkin
>Assignee: Mikhail Pochatkin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Currently, our integration test set 16g xmx. This is so much for us, because 
> we can lose some problems with memory leaks. Also, currently our TC agents 
> docker has only 18g RAM. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (IGNITE-20029) Reduce heap size for integration tests

Mikhail Pochatkin created IGNITE-20029:
--

 Summary: Reduce heap size for integration tests
 Key: IGNITE-20029
 URL: https://issues.apache.org/jira/browse/IGNITE-20029
 Project: Ignite
  Issue Type: Improvement
Reporter: Mikhail Pochatkin


Currently, our integration test set 16g xmx. This is so much for us, because we 
can lose some problems with memory leaks. Also, currently our TC agents docker 
has only 18g RAM. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Commented] (IGNITE-20021) Too many Raft-FSMCaller-Disruptor threads