[jira] [Created] (IGNITE-20905) Make it possible to add an explicitly NULL column via ADD COLUMN

2023-11-20 Thread Roman Puchkovskiy (Jira)
Roman Puchkovskiy created IGNITE-20905:
--

 Summary: Make it possible to add an explicitly NULL column via ADD 
COLUMN
 Key: IGNITE-20905
 URL: https://issues.apache.org/jira/browse/IGNITE-20905
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Reporter: Roman Puchkovskiy


When creating a table, it's possible to specify that a column is nullable by 
explicitly using NULL:

CREATE TABLE t(id INT PRIMARY KEY, col1 INT NULL)

But, if we add a column to an existing table, this does not work:

ALTER TABLE t ADD COLUMN col2 INT NULL

-> Failed to parse query: Encountered "NULL" at line 1, column X

It seems that for consistency ADD COLUMN should support same syntax as CREATE 
TABLE does.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20737) Idle Verify fails with "Cluster not idle" error when log level is set to DEBUG

2023-11-20 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788269#comment-17788269
 ] 

Ignite TC Bot commented on IGNITE-20737:


{panel:title=Branch: [pull/11052/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/11052/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7617485buildTypeId=IgniteTests24Java8_RunAll]

> Idle Verify fails with "Cluster not idle" error when log level is set to DEBUG
> --
>
> Key: IGNITE-20737
> URL: https://issues.apache.org/jira/browse/IGNITE-20737
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.15
>Reporter: Valery Shorin
>Assignee: Egor Fomin
>Priority: Major
>  Labels: ise
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> In case if idle verify executed with DEBUG log level it fails with error:
>  
> {code:java}
> Exception: org.apache.ignite.IgniteException Cluster not idle. Modifications 
> found in caches or groups: [grpName=default, grpId=1544803905, partId=30] 
> changed during size calculation [updCntrBefore=Counter [init=0, val=1], 
> updCntrAfter=Counter [init=0, val=1]]
>  {code}
>  
> This issue can be reproduced by the following test (shoud be added to 
> \{{GridCommandHandlerTest}}:
>  
> {code:java}
> @Test
> public void testCacheIdleVerifyLogLevelDebug() throws Exception {
> IgniteEx ignite = startGrids(3);
> ignite.cluster().state(ACTIVE);
> IgniteCache cache = ignite.createCache(new 
> CacheConfiguration<>(DEFAULT_CACHE_NAME)
> .setAffinity(new RendezvousAffinityFunction(false, 32))
> .setBackups(1));
> cache.put("key", "value");
> injectTestSystemOut();
> setLoggerDebugLevel();
> assertEquals(EXIT_CODE_OK, execute("--cache", "idle_verify"));
> assertContains(log, testOut.toString(), "no conflicts have been found");
> } {code}
>  
>  
> The reason of this failure - {{equals(}} method is not defined for 
> {{PartitionUpdateCounterDebugWrapper}} class



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20737) Idle Verify fails with "Cluster not idle" error when log level is set to DEBUG

2023-11-20 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788268#comment-17788268
 ] 

Ignite TC Bot commented on IGNITE-20737:


{panel:title=Branch: [pull/11052/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/11052/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7617485buildTypeId=IgniteTests24Java8_RunAll]

> Idle Verify fails with "Cluster not idle" error when log level is set to DEBUG
> --
>
> Key: IGNITE-20737
> URL: https://issues.apache.org/jira/browse/IGNITE-20737
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.15
>Reporter: Valery Shorin
>Assignee: Egor Fomin
>Priority: Major
>  Labels: ise
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> In case if idle verify executed with DEBUG log level it fails with error:
>  
> {code:java}
> Exception: org.apache.ignite.IgniteException Cluster not idle. Modifications 
> found in caches or groups: [grpName=default, grpId=1544803905, partId=30] 
> changed during size calculation [updCntrBefore=Counter [init=0, val=1], 
> updCntrAfter=Counter [init=0, val=1]]
>  {code}
>  
> This issue can be reproduced by the following test (shoud be added to 
> \{{GridCommandHandlerTest}}:
>  
> {code:java}
> @Test
> public void testCacheIdleVerifyLogLevelDebug() throws Exception {
> IgniteEx ignite = startGrids(3);
> ignite.cluster().state(ACTIVE);
> IgniteCache cache = ignite.createCache(new 
> CacheConfiguration<>(DEFAULT_CACHE_NAME)
> .setAffinity(new RendezvousAffinityFunction(false, 32))
> .setBackups(1));
> cache.put("key", "value");
> injectTestSystemOut();
> setLoggerDebugLevel();
> assertEquals(EXIT_CODE_OK, execute("--cache", "idle_verify"));
> assertContains(log, testOut.toString(), "no conflicts have been found");
> } {code}
>  
>  
> The reason of this failure - {{equals(}} method is not defined for 
> {{PartitionUpdateCounterDebugWrapper}} class



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20904) MQ API caused the systemview to be unavailable.

2023-11-20 Thread yafengshi (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

yafengshi updated IGNITE-20904:
---
Description: 
 

After I execute the following code, I want to query system views.
{code:java}
ignite.message(ignite.cluster().forServers()).remoteListen("A1", (nodeId, msg) 
-> {
             System.out.println(msg);
             return true;
});
  
 for (int i = 0; i < 10; i++)
   ignite.message().sendOrdered("A1", Integer.toString(i),0); {code}
 

then it threw an IllegalStateException.
{code:java}
jdbc:ignite:thin://127.0.0.1/sys> select count(1) from CONTINUOUS_QUERIES; 
Error: General error: "java.lang.IllegalStateException";   {code}
 

  was:
 

After I execute the following code, I want to query system views.
{code:java}
ignite.message(ignite.cluster().forServers()).remoteListen("A1", (nodeId, msg) 
-> {
             System.out.println(msg);
             return true;
});
  
 for (int i = 0; i < 10; i++)
   ignite.message().sendOrdered("A1", Integer.toString(i),0); {code}
 

then it threw an IllegalStateException.
{code:java}
jdbc:ignite:thin://127.0.0.1/sys> select count(1) from CONTINUOUS_QUERIES; 
Error: General error: "java.lang.IllegalStateException";   {code}
 


> MQ API caused the systemview to be unavailable.
> ---
>
> Key: IGNITE-20904
> URL: https://issues.apache.org/jira/browse/IGNITE-20904
> Project: Ignite
>  Issue Type: Bug
>  Components: messaging
>Affects Versions: 2.15
>Reporter: yafengshi
>Assignee: yafengshi
>Priority: Critical
>  Labels: newbie
> Fix For: 3.0, 2.16, 2.17
>
>   Original Estimate: 1h
>  Time Spent: 10m
>  Remaining Estimate: 50m
>
>  
> After I execute the following code, I want to query system views.
> {code:java}
> ignite.message(ignite.cluster().forServers()).remoteListen("A1", (nodeId, 
> msg) -> {
>              System.out.println(msg);
>              return true;
> });
>   
>  for (int i = 0; i < 10; i++)
>    ignite.message().sendOrdered("A1", Integer.toString(i),0); {code}
>  
> then it threw an IllegalStateException.
> {code:java}
> jdbc:ignite:thin://127.0.0.1/sys> select count(1) from CONTINUOUS_QUERIES; 
> Error: General error: "java.lang.IllegalStateException";   {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20904) MQ API caused the systemview to be unavailable.

2023-11-20 Thread yafengshi (Jira)
yafengshi created IGNITE-20904:
--

 Summary: MQ API caused the systemview to be unavailable.
 Key: IGNITE-20904
 URL: https://issues.apache.org/jira/browse/IGNITE-20904
 Project: Ignite
  Issue Type: Bug
  Components: messaging
Affects Versions: 2.15
Reporter: yafengshi
Assignee: yafengshi
 Fix For: 3.0, 2.16, 2.17


 

After I execute the following code, I want to query system views.
{code:java}
ignite.message(ignite.cluster().forServers()).remoteListen("A1", (nodeId, msg) 
-> {
             System.out.println(msg);
             return true;
});
  
 for (int i = 0; i < 10; i++)
   ignite.message().sendOrdered("A1", Integer.toString(i),0); {code}
 

then it threw an IllegalStateException.
{code:java}
jdbc:ignite:thin://127.0.0.1/sys> select count(1) from CONTINUOUS_QUERIES; 
Error: General error: "java.lang.IllegalStateException";   {code}
 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20903) [ducktests] Support running on JDK 17

2023-11-20 Thread Ivan Daschinsky (Jira)
Ivan Daschinsky created IGNITE-20903:


 Summary: [ducktests] Support running on JDK 17
 Key: IGNITE-20903
 URL: https://issues.apache.org/jira/browse/IGNITE-20903
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Daschinsky
Assignee: Ivan Daschinsky






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20759) Tcp handshake in communication spi hangs on JDK 12 and later

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20759:
-
Release Note: Fixed writing post-handshake message in NIO server

> Tcp handshake in communication spi hangs on JDK 12 and later
> 
>
> Key: IGNITE-20759
> URL: https://issues.apache.org/jira/browse/IGNITE-20759
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.15
>Reporter: Ivan Daschinsky
>Assignee: Ivan Daschinsky
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> This issue is reproduced in {{GridTcpCommunicationSpiSslSelfTest}}. All tests 
> hangs during tcp handshake process in infinite loop in 
> {{org.apache.ignite.spi.communication.tcp.internal.TcpHandshakeExecutor#tcpHandshake}}.
>  Seems that fixes in IGNITE-16622 are not enough
> The main reason of this behaviour -- 
> https://bugs.openjdk.org/browse/JDK-8277307. Post handshake message seems to 
> be too large for being written in one step, but we don't try to resend the 
> rest in 
> {{org.apache.ignite.internal.util.nio.GridNioServer.DirectNioClientWorker#processWriteSsl}}
>  since the ssl handshake is considered to be finished.
> WA -- set {{-Djdk.tls.server.enableSessionTicketExtension=false}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19878) Restrict possibility to create two caches with the same schema and index name

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-19878:
-
Ignite Flags: Release Notes Required

> Restrict possibility to create two caches with the same schema and index name
> -
>
> Key: IGNITE-19878
> URL: https://issues.apache.org/jira/browse/IGNITE-19878
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.15
>Reporter: Evgeny Stanilovsky
>Assignee: Evgeny Stanilovsky
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> It’s possible to create two different caches with the same schema and index 
> name inside it from a client and after that get a failing of the server node 
> during start-up with the error "Duplicate index name... " during start caches.
> scenario:
> start at least two server nodes
> start client node which creates two different caches during start-up (via 
> setting cache configs to config of client node) with the same schema name and 
> index name, it leads to crushing the cluster
> then try to start the cluster and get the error "Duplicate index name..."



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20759) Tcp handshake in communication spi hangs on JDK 12 and later

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20759:
-
Ignite Flags: Release Notes Required  (was: Docs Required,Release Notes 
Required)

> Tcp handshake in communication spi hangs on JDK 12 and later
> 
>
> Key: IGNITE-20759
> URL: https://issues.apache.org/jira/browse/IGNITE-20759
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.15
>Reporter: Ivan Daschinsky
>Assignee: Ivan Daschinsky
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> This issue is reproduced in {{GridTcpCommunicationSpiSslSelfTest}}. All tests 
> hangs during tcp handshake process in infinite loop in 
> {{org.apache.ignite.spi.communication.tcp.internal.TcpHandshakeExecutor#tcpHandshake}}.
>  Seems that fixes in IGNITE-16622 are not enough
> The main reason of this behaviour -- 
> https://bugs.openjdk.org/browse/JDK-8277307. Post handshake message seems to 
> be too large for being written in one step, but we don't try to resend the 
> rest in 
> {{org.apache.ignite.internal.util.nio.GridNioServer.DirectNioClientWorker#processWriteSsl}}
>  since the ssl handshake is considered to be finished.
> WA -- set {{-Djdk.tls.server.enableSessionTicketExtension=false}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20583) add test scope ignite dependency: opencensus

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20583:
-
Ignite Flags: Release Notes Required

> add test scope ignite dependency: opencensus
> 
>
> Key: IGNITE-20583
> URL: https://issues.apache.org/jira/browse/IGNITE-20583
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Aleksandr Nikolaev
>Assignee: Aleksandr Nikolaev
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> add test scope ignite dependency: opencensus



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20752) Bump opencensus version to latest 0.31.1

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20752?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20752:
-
Fix Version/s: (was: 2.16)

> Bump opencensus version to latest 0.31.1
> 
>
> Key: IGNITE-20752
> URL: https://issues.apache.org/jira/browse/IGNITE-20752
> Project: Ignite
>  Issue Type: Task
>Affects Versions: 2.15
>Reporter: ZhangJian He
>Assignee: Aleksandr Nikolaev
>Priority: Minor
>  Labels: ise
>  Time Spent: 50m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19239) Checkpoint read lock acquisition timeouts during snapshot restore

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-19239:
-
Fix Version/s: (was: 2.16)

> Checkpoint read lock acquisition timeouts during snapshot restore
> -
>
> Key: IGNITE-19239
> URL: https://issues.apache.org/jira/browse/IGNITE-19239
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ilya Shishkov
>Assignee: Maksim Timonin
>Priority: Critical
>  Labels: iep-43, ise
> Attachments: BlockingThreadsOnSnapshotRestoreReproducerTest.patch
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> There may be possible error messages about checkpoint read lock acquisition 
> timeouts and critical threads blocking during snapshot restore process (just 
> after caches start):
> {quote} 
> [2023-04-06T10:55:46,561][ERROR]\[ttl-cleanup-worker-#475%node%][CheckpointTimeoutLock]
>  Checkpoint read lock acquisition has been timed out. 
> {quote} 
> {quote} 
> [2023-04-06T10:55:47,487][ERROR]\[tcp-disco-msg-worker-[crd]\-#23%node%\-#446%node%][G]
>  Blocked system-critical thread has been detected. This can lead to 
> cluster-wide undefined behaviour \[workerName=db-checkpoint-thread, 
> threadName=db-checkpoint-thread-#457%snapshot.BlockingThreadsOnSnapshotRestoreReproducerTest0%,
>  {color:red}blockedFor=100s{color}] 
> {quote} 
> Also there are active exchange process, which finishes with such timings 
> (timing will be approximatelly equal to blocking time of threads): 
> {quote} 
> [2023-04-06T10:55:52,211][INFO 
> ]\[exchange-worker-#450%node%][GridDhtPartitionsExchangeFuture] Exchange 
> timings [startVer=AffinityTopologyVersion [topVer=1, minorTopVer=5], 
> resVer=AffinityTopologyVersion [topVer=1, minorTopVer=5], stage="Waiting in 
> exchange queue" (0 ms), ..., stage="Restore partition states" 
> ({color:red}100163 ms{color}), ..., stage="Total time" ({color:red}100334 
> ms{color})] 
> {quote} 
>  
> Most of time such errors and long lasting threads blocking tells that cluster 
> is in emergency state or will crash very soon.
> So, there are two possible ways to solve problem:
> # If these errors do not affect restoring from snapshot and are false 
> positive ones, they can confuse, so we should remove them from logs.
> # If these errors are not false positive, root cause of them have to be 
> investigated and solved.
>  
> How to reproduce:
>  # Set checkpoint frequency less than failure detection timeout.
>  # Ensure, that cache groups partitions states restoring lasts more than 
> failure detection timeout, i.e. it is actual to sufficiently large caches.
> Reproducer: [^BlockingThreadsOnSnapshotRestoreReproducerTest.patch]



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20737) Idle Verify fails with "Cluster not idle" error when log level is set to DEBUG

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20737:
-
Fix Version/s: (was: 2.16)

> Idle Verify fails with "Cluster not idle" error when log level is set to DEBUG
> --
>
> Key: IGNITE-20737
> URL: https://issues.apache.org/jira/browse/IGNITE-20737
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.15
>Reporter: Valery Shorin
>Assignee: Egor Fomin
>Priority: Major
>  Labels: ise
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> In case if idle verify executed with DEBUG log level it fails with error:
>  
> {code:java}
> Exception: org.apache.ignite.IgniteException Cluster not idle. Modifications 
> found in caches or groups: [grpName=default, grpId=1544803905, partId=30] 
> changed during size calculation [updCntrBefore=Counter [init=0, val=1], 
> updCntrAfter=Counter [init=0, val=1]]
>  {code}
>  
> This issue can be reproduced by the following test (shoud be added to 
> \{{GridCommandHandlerTest}}:
>  
> {code:java}
> @Test
> public void testCacheIdleVerifyLogLevelDebug() throws Exception {
> IgniteEx ignite = startGrids(3);
> ignite.cluster().state(ACTIVE);
> IgniteCache cache = ignite.createCache(new 
> CacheConfiguration<>(DEFAULT_CACHE_NAME)
> .setAffinity(new RendezvousAffinityFunction(false, 32))
> .setBackups(1));
> cache.put("key", "value");
> injectTestSystemOut();
> setLoggerDebugLevel();
> assertEquals(EXIT_CODE_OK, execute("--cache", "idle_verify"));
> assertContains(log, testOut.toString(), "no conflicts have been found");
> } {code}
>  
>  
> The reason of this failure - {{equals(}} method is not defined for 
> {{PartitionUpdateCounterDebugWrapper}} class



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18038) Error warning:“Unordered map java.util.LinkedHashMap is used for putAll operation on cache”

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-18038:
-
Fix Version/s: (was: 2.16)

> Error warning:“Unordered map java.util.LinkedHashMap is used for putAll 
> operation on cache”
> ---
>
> Key: IGNITE-18038
> URL: https://issues.apache.org/jira/browse/IGNITE-18038
> Project: Ignite
>  Issue Type: Improvement
>  Components: cache
>Affects Versions: 2.14
>Reporter: YuJue Li
>Assignee: YuJue Li
>Priority: Minor
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> LinkedHashMap is an ordered Map type.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19988) Add index creation (population) status to index view

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-19988:
-
Fix Version/s: (was: 2.16)

> Add index creation (population) status to index view
> 
>
> Key: IGNITE-19988
> URL: https://issues.apache.org/jira/browse/IGNITE-19988
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.15
>Reporter: Ivan Daschinsky
>Priority: Major
>  Labels: ise
>
> Sometimes index creation can be quite long. A user might start queries 
> without waiting for finish of the index creation process and see slow 
> queries. It is necessary to provide index status information to users by 
> exposing it in the index system view.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20688) Java Thin Client - Error while deserializing Collection

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20688:
-
Fix Version/s: (was: 2.16)

> Java Thin Client - Error while deserializing Collection
> ---
>
> Key: IGNITE-20688
> URL: https://issues.apache.org/jira/browse/IGNITE-20688
> Project: Ignite
>  Issue Type: Bug
>  Components: binary, thin client
>Affects Versions: 2.9, 2.10, 2.12, 2.13, 2.14, 2.15
>Reporter: Rahul Mohan
>Assignee: Mikhail Petrov
>Priority: Critical
>  Labels: ise
> Attachments: image001.png
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> I have encountered an issue in deserializing cache values which are of 
> Collection type.
> The issue occurs if a field in different objects  within the  collection 
> points  to the same reference.
> *Versions:*
> org.apache.ignite:ignite-core:2.9.0 to org.apache.ignite:ignite-core:2.15.0
>  
> {code:java}
> Person.java
> public class Person implements Serializable {
> private String id;
> private String firstName;
> private String lastName;
> private double salary;
> private String country;
> private String deleted;
> private Set accounts;
> }
> Client
>     ClientCacheConfiguration cacheCfg = new 
> ClientCacheConfiguration().setName(cacheName).
>     setCacheMode(CacheMode.REPLICATED).
>     
> setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
>  
>     cache = client.getOrCreateCache(cacheCfg);
>  
>     Set set = new HashSet<>();
> set.add("1");
>  
>     List persons = new ArrayList<>();
>     persons.add(new Person("105286a4","Jack","Smith",1f, 
> "USA","false", set));
>     persons.add(new Person("98545b0fd3af","John", "Doe", 50f, 
> "Australia","false", null));
>     persons.add(new Person("98545b0fd3afd","Hari","M",40f, 
> "India", null, null));
>     persons.add(new 
> Person("985488b0fd3ae","Bugs","Bunny",30f,"Wabbit Land ", null, set));
>     cache.put("group1", value) // Write collection to cache
> 
> List persons = (List) cache.get("group1"); // Get 
> from cache, Exception here {code}
> 
> *Exception:*
> {code:java}
> class org.apache.ignite.binary.BinaryObjectException: Failed to deserialize 
> object [typeName=com.ignite.example.model.Person]
>     at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.read(BinaryClassDescriptor.java:927)
>     at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize0(BinaryReaderExImpl.java:1764)
>     at 
> org.apache.ignite.internal.binary.BinaryReaderExImpl.deserialize(BinaryReaderExImpl.java:1716)
>     at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.deserialize(GridBinaryMarshaller.java:316)
>     at 
> org.apache.ignite.internal.client.thin.ClientBinaryMarshaller.deserialize(ClientBinaryMarshaller.java:74)
>     at 
> org.apache.ignite.internal.client.thin.ClientUtils.unwrapBinary(ClientUtils.java:557)
>     at 
> org.apache.ignite.internal.client.thin.ClientUtils.unwrapCollection(ClientUtils.java:578)
>     at 
> org.apache.ignite.internal.client.thin.ClientUtils.unwrapBinary(ClientUtils.java:562)
>     at 
> org.apache.ignite.internal.client.thin.ClientUtils.readObject(ClientUtils.java:546)
>     at 
> org.apache.ignite.internal.client.thin.TcpClientCache.readObject(TcpClientCache.java:556)
>     at 
> org.apache.ignite.internal.client.thin.TcpClientCache.readObject(TcpClientCache.java:561)
>     at 
> org.apache.ignite.internal.client.thin.TcpClientCache$$Lambda$395/1950117092.apply(Unknown
>  Source)
>     at 
> org.apache.ignite.internal.client.thin.TcpClientChannel.receive(TcpClientChannel.java:284)
>     at 
> org.apache.ignite.internal.client.thin.TcpClientChannel.service(TcpClientChannel.java:219)
>     at 
> org.apache.ignite.internal.client.thin.ReliableChannel.service(ReliableChannel.java:198)
>     at 
> org.apache.ignite.internal.client.thin.ReliableChannel.affinityService(ReliableChannel.java:261)
>     at 
> org.apache.ignite.internal.client.thin.TcpClientCache.cacheSingleKeyOperation(TcpClientCache.java:508)
>     at 
> org.apache.ignite.internal.client.thin.TcpClientCache.get(TcpClientCache.java:111)
>     at 
> com.ignite.example.service.ApacheIgniteService.printAllKeys(ApacheIgniteService.java:117)
>     at 
> com.ignite.example.service.ApacheIgniteService.init(ApacheIgniteService.java:103)
>     at 
> 

[jira] [Updated] (IGNITE-18572) Add query thread pool metrics in the log file

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-18572:
-
Fix Version/s: (was: 2.16)

> Add query thread pool metrics in the log file
> -
>
> Key: IGNITE-18572
> URL: https://issues.apache.org/jira/browse/IGNITE-18572
> Project: Ignite
>  Issue Type: Improvement
>  Components: general
>Affects Versions: 2.14
>Reporter: YuJue Li
>Assignee: YuJue Li
>Priority: Minor
>
> Add query thread pool metrics in the log file.
> The current metric output includes Public, System and Striped thread pool. It 
> is necessary to add the metric output of Query thread pool,



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19436) Add "--add-opens=java.base/java.time=ALL-UNNAMED" to jvm args

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-19436:
-
Fix Version/s: (was: 2.16)

> Add "--add-opens=java.base/java.time=ALL-UNNAMED" to jvm args
> -
>
> Key: IGNITE-19436
> URL: https://issues.apache.org/jira/browse/IGNITE-19436
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Affects Versions: 2.15
> Environment: jdk17 env
>Reporter: YuJue Li
>Assignee: YuJue Li
>Priority: Minor
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 1.start a node with calcite-based sql engine is enabled;
> 2.execute the following SQL:
>   select /*{*}+ QUERY_ENGINE('calcite') *{*}/ count(*) from T;
> then:
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make 
> field private final int java.time.LocalDate.year accessible: module java.base 
> does not "opens java.time" to unnamed module @587d1d39



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-20898) Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)

2023-11-20 Thread Nikita Amelchev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788138#comment-17788138
 ] 

Nikita Amelchev edited comment on IGNITE-20898 at 11/20/23 7:20 PM:


Merged into the master and 2.16.

[~annyakimova], Thank you for the contribution.


was (Author: nsamelchev):
Merged into the master and 2.16.

[~annyakimova], Thank you for the review.

> Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)
> --
>
> Key: IGNITE-20898
> URL: https://issues.apache.org/jira/browse/IGNITE-20898
> Project: Ignite
>  Issue Type: Bug
>Reporter: Nikita Amelchev
>Assignee: Anastasia Iakimova
>Priority: Minor
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL). They 
> are deprecated in javadocs after IGNITE-13003, but annotation is not used. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20898) Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)

2023-11-20 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788137#comment-17788137
 ] 

Ignite TC Bot commented on IGNITE-20898:


{panel:title=Branch: [pull/11058/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/11058/head] Base: [master] : No new tests 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7623531buildTypeId=IgniteTests24Java8_RunAll]

> Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)
> --
>
> Key: IGNITE-20898
> URL: https://issues.apache.org/jira/browse/IGNITE-20898
> Project: Ignite
>  Issue Type: Bug
>Reporter: Nikita Amelchev
>Assignee: Anastasia Iakimova
>Priority: Minor
>  Labels: ise
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL). They 
> are deprecated in javadocs after IGNITE-13003, but annotation is not used. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20898) Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20898:
-
Fix Version/s: 2.16

> Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)
> --
>
> Key: IGNITE-20898
> URL: https://issues.apache.org/jira/browse/IGNITE-20898
> Project: Ignite
>  Issue Type: Bug
>Reporter: Nikita Amelchev
>Assignee: Anastasia Iakimova
>Priority: Minor
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL). They 
> are deprecated in javadocs after IGNITE-13003, but annotation is not used. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19878) Restrict possibility to create two caches with the same schema and index name

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-19878:
-
Release Note: Fixed possibility to create two caches with the same schemas 
and index names

> Restrict possibility to create two caches with the same schema and index name
> -
>
> Key: IGNITE-19878
> URL: https://issues.apache.org/jira/browse/IGNITE-19878
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.15
>Reporter: Evgeny Stanilovsky
>Assignee: Evgeny Stanilovsky
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> It’s possible to create two different caches with the same schema and index 
> name inside it from a client and after that get a failing of the server node 
> during start-up with the error "Duplicate index name... " during start caches.
> scenario:
> start at least two server nodes
> start client node which creates two different caches during start-up (via 
> setting cache configs to config of client node) with the same schema name and 
> index name, it leads to crushing the cluster
> then try to start the cluster and get the error "Duplicate index name..."



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19878) Restrict possibility to create two caches with the same schema and index name

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-19878:
-
Fix Version/s: 2.16

> Restrict possibility to create two caches with the same schema and index name
> -
>
> Key: IGNITE-19878
> URL: https://issues.apache.org/jira/browse/IGNITE-19878
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Affects Versions: 2.15
>Reporter: Evgeny Stanilovsky
>Assignee: Evgeny Stanilovsky
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> It’s possible to create two different caches with the same schema and index 
> name inside it from a client and after that get a failing of the server node 
> during start-up with the error "Duplicate index name... " during start caches.
> scenario:
> start at least two server nodes
> start client node which creates two different caches during start-up (via 
> setting cache configs to config of client node) with the same schema name and 
> index name, it leads to crushing the cluster
> then try to start the cluster and get the error "Duplicate index name..."



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20314) Ignite Java documentation missing required --add-opens

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20314:
-
Fix Version/s: 2.16

> Ignite Java documentation missing required --add-opens
> --
>
> Key: IGNITE-20314
> URL: https://issues.apache.org/jira/browse/IGNITE-20314
> Project: Ignite
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.15
>Reporter: Patrick Peralta
>Assignee: Igor Gusev
>Priority: Minor
> Fix For: 2.16
>
>
> As indicated in IGNITE-17658, 
> {{--add-opens=java.base/java.lang.invoke=ALL-UNNAMED}} is required to 
> serialize lambdas.
> However this flag is not included in the Java 17 section of the Ignite Java 
> Quick Start guide: [https://ignite.apache.org/docs/latest/quick-start/java] 
> Please add this so that Ignite users that require serialization of lambdas 
> (such as scan query filters and entry processors) don't run into this problem:
> {code:java}
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make 
> field private final java.lang.Class 
> java.lang.invoke.SerializedLambda.capturingClass accessible: module java.base 
> does not "opens java.lang.invoke" to unnamed module @41a4555e
>     at 
> java.base/java.lang.reflect.AccessibleObject.throwInaccessibleObjectException(AccessibleObject.java:387)
>     at 
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:363)
>     at 
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:311)
>     at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:181)
>     at java.base/java.lang.reflect.Field.setAccessible(Field.java:175)
>     at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.(BinaryClassDescriptor.java:354)
>     at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.(BinaryClassDescriptor.java:156)
>     at 
> org.apache.ignite.internal.binary.BinaryContext.createDescriptorForClass(BinaryContext.java:675)
>     at 
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:633)
>     at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:182)
>     at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
>     at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:227)
>     at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
>     at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
>     at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
>     at 
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:84)
>     at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:56)
>     at 
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10873)
>     ... 22 more {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20314) Ignite Java documentation missing required --add-opens

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20314:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Ignite Java documentation missing required --add-opens
> --
>
> Key: IGNITE-20314
> URL: https://issues.apache.org/jira/browse/IGNITE-20314
> Project: Ignite
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.15
>Reporter: Patrick Peralta
>Assignee: Igor Gusev
>Priority: Minor
> Fix For: 2.16
>
>
> As indicated in IGNITE-17658, 
> {{--add-opens=java.base/java.lang.invoke=ALL-UNNAMED}} is required to 
> serialize lambdas.
> However this flag is not included in the Java 17 section of the Ignite Java 
> Quick Start guide: [https://ignite.apache.org/docs/latest/quick-start/java] 
> Please add this so that Ignite users that require serialization of lambdas 
> (such as scan query filters and entry processors) don't run into this problem:
> {code:java}
> Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make 
> field private final java.lang.Class 
> java.lang.invoke.SerializedLambda.capturingClass accessible: module java.base 
> does not "opens java.lang.invoke" to unnamed module @41a4555e
>     at 
> java.base/java.lang.reflect.AccessibleObject.throwInaccessibleObjectException(AccessibleObject.java:387)
>     at 
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:363)
>     at 
> java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:311)
>     at java.base/java.lang.reflect.Field.checkCanSetAccessible(Field.java:181)
>     at java.base/java.lang.reflect.Field.setAccessible(Field.java:175)
>     at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.(BinaryClassDescriptor.java:354)
>     at 
> org.apache.ignite.internal.binary.BinaryClassDescriptor.(BinaryClassDescriptor.java:156)
>     at 
> org.apache.ignite.internal.binary.BinaryContext.createDescriptorForClass(BinaryContext.java:675)
>     at 
> org.apache.ignite.internal.binary.BinaryContext.descriptorForClass(BinaryContext.java:633)
>     at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:182)
>     at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
>     at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:227)
>     at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:165)
>     at 
> org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:152)
>     at 
> org.apache.ignite.internal.binary.GridBinaryMarshaller.marshal(GridBinaryMarshaller.java:254)
>     at 
> org.apache.ignite.internal.binary.BinaryMarshaller.marshal0(BinaryMarshaller.java:84)
>     at 
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.marshal(AbstractNodeNameAwareMarshaller.java:56)
>     at 
> org.apache.ignite.internal.util.IgniteUtils.marshal(IgniteUtils.java:10873)
>     ... 22 more {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19647) Our Cache Templates introductory documentation is misleading -- implies SQL Create Table only

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-19647:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Our Cache Templates introductory documentation is misleading -- implies SQL 
> Create Table only
> -
>
> Key: IGNITE-19647
> URL: https://issues.apache.org/jira/browse/IGNITE-19647
> Project: Ignite
>  Issue Type: Task
>  Components: documentation
>Reporter: Alex Levitski
>Assignee: Igor Gusev
>Priority: Minor
> Fix For: 2.16
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Our introductory documentation (below) on [Cache 
> Templates|https://ignite.apache.org/docs/latest/configuring-caches/configuration-overview#cache-templates]
>  is misleading in that it gives the strong impression that Cache Templates 
> are only useable with the SQL API, whereas it is my understanding that it can 
> also be used with CreateCache and GetOrCreateCache commands.
> It would be helpful to add this clarification and also add links to where we 
> describe how to use CacheTemplates with CreateCache/GetOrCreateCache commands 
> (or add such documentation and examples if this is not documented at all).
>  
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20460) Too many files with unapproved license

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20460:
-
Fix Version/s: 2.16

> Too many files with unapproved license
> --
>
> Key: IGNITE-20460
> URL: https://issues.apache.org/jira/browse/IGNITE-20460
> Project: Ignite
>  Issue Type: Bug
>  Components: build, documentation
>Reporter: Artem Egorov
>Assignee: Igor Gusev
>Priority: Critical
> Fix For: 2.16
>
>
> After merging IGNITE-20431, the project build began to fail with an "Too many 
> files with unapproved license" error:
> {quote}1 Unknown Licenses 
> * Files with unapproved 
> licenses: docs/_docs/installation/upgrades.adoc
> {quote}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20431) Document upgrading Ignite version

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20431:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Document upgrading Ignite version
> -
>
> Key: IGNITE-20431
> URL: https://issues.apache.org/jira/browse/IGNITE-20431
> Project: Ignite
>  Issue Type: Task
>Reporter: Igor Gusev
>Assignee: Igor Gusev
>Priority: Major
> Fix For: 2.16
>
>
> There is no information on how to change Ignite version. We should add it to 
> docs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19473) Incorrect schema name in sys view documentation.

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-19473:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Incorrect schema name in sys view documentation.
> 
>
> Key: IGNITE-19473
> URL: https://issues.apache.org/jira/browse/IGNITE-19473
> Project: Ignite
>  Issue Type: Bug
>  Components: documentation
>Affects Versions: 2.15
>Reporter: Evgeny Stanilovsky
>Assignee: Igor Gusev
>Priority: Major
> Fix For: 2.16
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Schema [1] need to be changed:
> _.setSchema("IGNITE")_ need to be changed into: _.setSchema("SYS")_
>  
> [1] [https://ignite.apache.org/docs/latest/monitoring-metrics/system-views]
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20431) Document upgrading Ignite version

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20431?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20431:
-
Fix Version/s: 2.16

> Document upgrading Ignite version
> -
>
> Key: IGNITE-20431
> URL: https://issues.apache.org/jira/browse/IGNITE-20431
> Project: Ignite
>  Issue Type: Task
>Reporter: Igor Gusev
>Assignee: Igor Gusev
>Priority: Major
> Fix For: 2.16
>
>
> There is no information on how to change Ignite version. We should add it to 
> docs.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20440) Document readiness and liveness probe

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20440:
-
Fix Version/s: 2.16

> Document  readiness and liveness probe
> --
>
> Key: IGNITE-20440
> URL: https://issues.apache.org/jira/browse/IGNITE-20440
> Project: Ignite
>  Issue Type: Task
>Reporter: Igor Gusev
>Assignee: Igor Gusev
>Priority: Major
> Fix For: 2.16
>
>
> We need documentation for readiness and liveness probe features.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20440) Document readiness and liveness probe

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20440:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Document  readiness and liveness probe
> --
>
> Key: IGNITE-20440
> URL: https://issues.apache.org/jira/browse/IGNITE-20440
> Project: Ignite
>  Issue Type: Task
>Reporter: Igor Gusev
>Assignee: Igor Gusev
>Priority: Major
> Fix For: 2.16
>
>
> We need documentation for readiness and liveness probe features.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20543) CacheStoreFactory should be invoked within sandbox

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20543:
-
Release Note: Fixed use of sandbox to create CacheStore.

> CacheStoreFactory should be invoked within sandbox
> --
>
> Key: IGNITE-20543
> URL: https://issues.apache.org/jira/browse/IGNITE-20543
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maksim Timonin
>Assignee: Maksim Timonin
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> User can invoke any code while configuring 
> CacheConfiguration#CacheStoreFactory. Factory code is user defined and 
> invoked on Ignite server nodes during cache start. 
>  
> Should wrap this code into sandbox like any other user code (compute jobs, 
> etc).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20543) CacheStoreFactory should be invoked within sandbox

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20543:
-
Ignite Flags: Release Notes Required

> CacheStoreFactory should be invoked within sandbox
> --
>
> Key: IGNITE-20543
> URL: https://issues.apache.org/jira/browse/IGNITE-20543
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maksim Timonin
>Assignee: Maksim Timonin
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> User can invoke any code while configuring 
> CacheConfiguration#CacheStoreFactory. Factory code is user defined and 
> invoked on Ignite server nodes during cache start. 
>  
> Should wrap this code into sandbox like any other user code (compute jobs, 
> etc).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20583) add test scope ignite dependency: opencensus

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20583:
-
Release Note: Moved opencensus-exporter-trace-zipkin dependency to test 
scope

> add test scope ignite dependency: opencensus
> 
>
> Key: IGNITE-20583
> URL: https://issues.apache.org/jira/browse/IGNITE-20583
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Aleksandr Nikolaev
>Assignee: Aleksandr Nikolaev
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> add test scope ignite dependency: opencensus



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20816) in-memory server node is crashed by TcpIgniteClient.putAllConflict() if cache objects transformation applied

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20816:
-
Release Note: Fixed in-memory server node crashing by 
TcpIgniteClient.putAllConflict() if cache objects transformation applied

> in-memory server node is crashed by TcpIgniteClient.putAllConflict() if cache 
> objects transformation applied
> 
>
> Key: IGNITE-20816
> URL: https://issues.apache.org/jira/browse/IGNITE-20816
> Project: Ignite
>  Issue Type: Task
>Reporter: Sergey Korotkov
>Assignee: Anton Vinogradov
>Priority: Major
>  Labels: iep-97, ise
> Fix For: 2.16, 2.17
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> The in-memory server node crashes with the below exception if 
> TcpIgniteClient.putAllConflict() if cache objects transformation is applied.
> {noformat}
> [2023-11-09T09:34:30,401][ERROR][client-connector-#68%target%][] Critical 
> system error detected. Will be handled accordingly to configured handler 
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], 
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class 
> o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is 
> corrupted [groupId=-1407396309, pageIds=[844420635164695], msg=Runtime 
> failure on search row: SearchRow [key=KeyCacheObjectImpl [part=10, val=52234, 
> hasValBytes=true], hash=52234, cacheId=0
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
>  B+Tree is corrupted [groupId=-1407396309, pageIds=[844420635164695], 
> msg=Runtime failure on search row: SearchRow [key=KeyCacheObjectImpl 
> [part=10, val=52234, hasValBytes=true], hash=52234, cacheId=0]]
> at 
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:6534)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:2215)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1692)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1675)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:424)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:1903)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2536)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:1996)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1815)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1688)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:300)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateFuture.map(GridNearAtomicUpdateFuture.java:791)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateFuture.mapOnTopology(GridNearAtomicUpdateFuture.java:645)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:249)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$21.apply(GridDhtAtomicCache.java:1080)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$21.apply(GridDhtAtomicCache.java:1078)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:752)
>  [classes/:?]
> at 
> 

[jira] [Updated] (IGNITE-20816) in-memory server node is crashed by TcpIgniteClient.putAllConflict() if cache objects transformation applied

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20816:
-
Fix Version/s: (was: 2.17)

> in-memory server node is crashed by TcpIgniteClient.putAllConflict() if cache 
> objects transformation applied
> 
>
> Key: IGNITE-20816
> URL: https://issues.apache.org/jira/browse/IGNITE-20816
> Project: Ignite
>  Issue Type: Task
>Reporter: Sergey Korotkov
>Assignee: Anton Vinogradov
>Priority: Major
>  Labels: iep-97, ise
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> The in-memory server node crashes with the below exception if 
> TcpIgniteClient.putAllConflict() if cache objects transformation is applied.
> {noformat}
> [2023-11-09T09:34:30,401][ERROR][client-connector-#68%target%][] Critical 
> system error detected. Will be handled accordingly to configured handler 
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], 
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class 
> o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is 
> corrupted [groupId=-1407396309, pageIds=[844420635164695], msg=Runtime 
> failure on search row: SearchRow [key=KeyCacheObjectImpl [part=10, val=52234, 
> hasValBytes=true], hash=52234, cacheId=0
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
>  B+Tree is corrupted [groupId=-1407396309, pageIds=[844420635164695], 
> msg=Runtime failure on search row: SearchRow [key=KeyCacheObjectImpl 
> [part=10, val=52234, hasValBytes=true], hash=52234, cacheId=0]]
> at 
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:6534)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:2215)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1692)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1675)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:424)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.innerUpdate(GridCacheMapEntry.java:1903)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateSingle(GridDhtAtomicCache.java:2536)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.update(GridDhtAtomicCache.java:1996)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal0(GridDhtAtomicCache.java:1815)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAllAsyncInternal(GridDhtAtomicCache.java:1688)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.sendSingleRequest(GridNearAtomicAbstractUpdateFuture.java:300)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateFuture.map(GridNearAtomicUpdateFuture.java:791)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicUpdateFuture.mapOnTopology(GridNearAtomicUpdateFuture.java:645)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridNearAtomicAbstractUpdateFuture.map(GridNearAtomicAbstractUpdateFuture.java:249)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$21.apply(GridDhtAtomicCache.java:1080)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$21.apply(GridDhtAtomicCache.java:1078)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.asyncOp(GridDhtAtomicCache.java:752)
>  [classes/:?]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.updateAll0(GridDhtAtomicCache.java:1078)
>  [classes/:?]
> at 
> 

[jira] [Updated] (IGNITE-19807) Deprecate legacy authorization approach through Security Context.

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-19807:
-
Release Note:  Deprecated legacy authorization approach via Security Context

> Deprecate legacy authorization approach through Security Context.
> -
>
> Key: IGNITE-19807
> URL: https://issues.apache.org/jira/browse/IGNITE-19807
> Project: Ignite
>  Issue Type: Task
>Reporter: Mikhail Petrov
>Assignee: Mikhail Petrov
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> We currently have several ways to check if a user has permission to perform 
> an operation.
> 1. IgniteSecurity#authorize methods that delegate permission check to 
> security plugin.
> 2. SecurityContext#*OperationAllowed methods. They currently are used just 
> for one check. This approach assumes that granted  permissions set is 
> returned during user authentication and remains immutable.
> Let's deprecate the second authorization approach and migrate completely to 
> the first.
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-19830) Add a description check if the command argument is enum type.

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19830?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-19830:
-
Ignite Flags: Release Notes Required

> Add a description check if the command argument is enum type.
> -
>
> Key: IGNITE-19830
> URL: https://issues.apache.org/jira/browse/IGNITE-19830
> Project: Ignite
>  Issue Type: Task
>Reporter: Nikita Amelchev
>Assignee: Nikita Amelchev
>Priority: Major
>  Labels: IEP-81, ise
> Fix For: 2.16
>
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Add a check for the presence of a enum description ({{@EnumDescription}}) if 
> the parameter ({{@Argument}}) is an enum type.
> Example of an uncovered argument: {{ShutdownPolicyCommandArg#shutdownPolicy}}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18434) Сustom histogram buckets not applied after restart

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-18434:
-
Ignite Flags: Release Notes Required

> Сustom histogram buckets not applied after restart
> --
>
> Key: IGNITE-18434
> URL: https://issues.apache.org/jira/browse/IGNITE-18434
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Stepanov Ilya
>Assignee: Vladimir Steshin
>Priority: Minor
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Setting custom buckets for "TaskExecutionTime" metric in 
> "threadPools.\{thread_pool_name}" register is not applied after node restart.
>  
> Steps to reproduce: 
> 1) Start node
> 2) change buckets for TaskExecutionTime in StripedExecutor (via 
> configureHistogramMetric 
> [https://javadoc.io/static/org.apache.ignite/ignite-core/2.14.0/org/apache/ignite/mxbean/MetricsMxBean.html#configureHistogramMetric-java.lang.String-long:A-])
> 3) Check metrics in JMX "org.apache:group=threadPools,name=StripedExecutor"
> 4) Restart node
> 5) Check metrics in JMX "org.apache:group=threadPools,name=StripedExecutor"



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-20770) Update Ignite dependency zookeeper

2023-11-20 Thread Nikita Amelchev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788072#comment-17788072
 ] 

Nikita Amelchev edited comment on IGNITE-20770 at 11/20/23 5:07 PM:


Merged into the master and 2.16.

[~aonikolaev], Thank you for the contribution.

[~slava.koptilin], Thank you for the initial review.


was (Author: nsamelchev):
Merged into the master.

[~aonikolaev], Thank you for the contribution.

[~slava.koptilin], Thank you for the initial review.

> Update Ignite dependency zookeeper
> --
>
> Key: IGNITE-20770
> URL: https://issues.apache.org/jira/browse/IGNITE-20770
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Aleksandr Nikolaev
>Assignee: Aleksandr Nikolaev
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Update Ignite dependency zookeeper



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20867) Server nodes crash after create persistent cache with /" name

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20867:
-
Fix Version/s: 2.16

> Server nodes crash after create persistent cache with /" name 
> --
>
> Key: IGNITE-20867
> URL: https://issues.apache.org/jira/browse/IGNITE-20867
> Project: Ignite
>  Issue Type: Bug
>Reporter: Anastasia Iakimova
>Assignee: Anastasia Iakimova
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> ignite.log:
> {code:java}
> 2023-11-14 18:12:34.985 [INFO 
> ][exchange-worker-#75][org.apache.ignite.internal.exchange.time] Started 
> exchange init [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=32], 
> crd=false, evt=DISCOVERY_CUSTOM_EVT, evtNode=d91c8f37
> -dc4f-435e-97dc-8ec02b953b6a, customEvt=DynamicCacheChangeBatch 
> [id=3a05698cb81-782afb33-985f-4efc-a47f-2c178ae29301, reqs=ArrayList 
> [DynamicCacheChangeRequest [cacheName=/", hasCfg=true, 
> nodeId=d91c8f37-dc4f-435e-97dc-8ec02b953b6
> a, clientStartOnly=false, stop=false, destroy=false, 
> disabledAfterStart=false]], exchangeActions=ExchangeActions 
> [startCaches=[/"], stopCaches=null, startGrps=[/"], stopGrps=[], 
> resetParts=null, finalizePartitionCounters=false, stateChangeRequest=null], 
> startCaches=false], allowMerge=false, exchangeFreeSwitch=false]
> 2023-11-14 18:12:34.989 [ERROR][exchange-worker-#75][] Critical system error 
> detected. Will be handled accordingly to configured handler 
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], 
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class 
> o.a.i.IgniteCheckedException: Failed to initialize cache working directory 
> (failed to create, make sure the work folder has correct permissions): 
> /u01/ift/storage/server/data/ise_ift_ignite05/cache-/"]]
> org.apache.ignite.IgniteCheckedException: Failed to initialize cache working 
> directory (failed to create, make sure the work folder has correct 
> permissions): /u01/ift/storage/server/data/ise_ift_ignite05/cache-/"
> at 
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.checkAndInitCacheWorkDir(FilePageStoreManager.java:796)
>  ~[ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CachesRegistry.persistCacheConfigurations(CachesRegistry.java:275)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CachesRegistry.registerAllCachesAndGroups(CachesRegistry.java:262)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CachesRegistry.update(CachesRegistry.java:200)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:877)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1462)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:980)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body0(GridCachePartitionExchangeManager.java:3338)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:3172)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125) 
> [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at java.lang.Thread.run(Thread.java:834) [?:?]
> Caused by: java.nio.file.NoSuchFileException: 
> /u01/ift/storage/server/data/ise_ift_ignite05/cache-/"
> at 
> sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) ~[?:?]
> at 
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~[?:?]
> at 
> sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:116) ~[?:?]
> at 
> sun.nio.fs.UnixFileSystemProvider.createDirectory(UnixFileSystemProvider.java:389)
>  ~[?:?]
> at 

[jira] [Commented] (IGNITE-20867) Server nodes crash after create persistent cache with /" name

2023-11-20 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788082#comment-17788082
 ] 

Ignite TC Bot commented on IGNITE-20867:


{panel:title=Branch: [pull/11047/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/11047/head] Base: [master] : New Tests 
(4)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Cache 4{color} [[tests 
4|https://ci2.ignite.apache.org/viewLog.html?buildId=7623101]]
* {color:#013220}IgniteCacheTestSuite4: 
CacheDirectoryNameTest.testCacheDirectoryContainsInvalidFileNameChars[persistenceEnabled=false,
 isGroupName=false] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite4: 
CacheDirectoryNameTest.testCacheDirectoryContainsInvalidFileNameChars[persistenceEnabled=false,
 isGroupName=true] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite4: 
CacheDirectoryNameTest.testCacheDirectoryContainsInvalidFileNameChars[persistenceEnabled=true,
 isGroupName=false] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite4: 
CacheDirectoryNameTest.testCacheDirectoryContainsInvalidFileNameChars[persistenceEnabled=true,
 isGroupName=true] - PASSED{color}

{panel}
[TeamCity *-- Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7623196buildTypeId=IgniteTests24Java8_RunAll]

> Server nodes crash after create persistent cache with /" name 
> --
>
> Key: IGNITE-20867
> URL: https://issues.apache.org/jira/browse/IGNITE-20867
> Project: Ignite
>  Issue Type: Bug
>Reporter: Anastasia Iakimova
>Assignee: Anastasia Iakimova
>Priority: Major
>  Labels: ise
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> ignite.log:
> {code:java}
> 2023-11-14 18:12:34.985 [INFO 
> ][exchange-worker-#75][org.apache.ignite.internal.exchange.time] Started 
> exchange init [topVer=AffinityTopologyVersion [topVer=4, minorTopVer=32], 
> crd=false, evt=DISCOVERY_CUSTOM_EVT, evtNode=d91c8f37
> -dc4f-435e-97dc-8ec02b953b6a, customEvt=DynamicCacheChangeBatch 
> [id=3a05698cb81-782afb33-985f-4efc-a47f-2c178ae29301, reqs=ArrayList 
> [DynamicCacheChangeRequest [cacheName=/", hasCfg=true, 
> nodeId=d91c8f37-dc4f-435e-97dc-8ec02b953b6
> a, clientStartOnly=false, stop=false, destroy=false, 
> disabledAfterStart=false]], exchangeActions=ExchangeActions 
> [startCaches=[/"], stopCaches=null, startGrps=[/"], stopGrps=[], 
> resetParts=null, finalizePartitionCounters=false, stateChangeRequest=null], 
> startCaches=false], allowMerge=false, exchangeFreeSwitch=false]
> 2023-11-14 18:12:34.989 [ERROR][exchange-worker-#75][] Critical system error 
> detected. Will be handled accordingly to configured handler 
> [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, 
> super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet 
> [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], 
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class 
> o.a.i.IgniteCheckedException: Failed to initialize cache working directory 
> (failed to create, make sure the work folder has correct permissions): 
> /u01/ift/storage/server/data/ise_ift_ignite05/cache-/"]]
> org.apache.ignite.IgniteCheckedException: Failed to initialize cache working 
> directory (failed to create, make sure the work folder has correct 
> permissions): /u01/ift/storage/server/data/ise_ift_ignite05/cache-/"
> at 
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStoreManager.checkAndInitCacheWorkDir(FilePageStoreManager.java:796)
>  ~[ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CachesRegistry.persistCacheConfigurations(CachesRegistry.java:275)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CachesRegistry.registerAllCachesAndGroups(CachesRegistry.java:262)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CachesRegistry.update(CachesRegistry.java:200)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:877)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1462)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> at 
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:980)
>  [ignite-core-16.0.0-SNAPSHOT.jar:16.0.0-SNAPSHOT]
> 

[jira] [Updated] (IGNITE-20770) Update Ignite dependency zookeeper

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20770?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20770:
-
Ignite Flags: Release Notes Required  (was: Docs Required,Release Notes 
Required)

> Update Ignite dependency zookeeper
> --
>
> Key: IGNITE-20770
> URL: https://issues.apache.org/jira/browse/IGNITE-20770
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Aleksandr Nikolaev
>Assignee: Aleksandr Nikolaev
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Update Ignite dependency zookeeper



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20770) Update Ignite dependency zookeeper

2023-11-20 Thread Nikita Amelchev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20770?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788072#comment-17788072
 ] 

Nikita Amelchev commented on IGNITE-20770:
--

Merged into the master.

[~aonikolaev], Thank you for the contribution.

[~slava.koptilin], Thank you for the initial review.

> Update Ignite dependency zookeeper
> --
>
> Key: IGNITE-20770
> URL: https://issues.apache.org/jira/browse/IGNITE-20770
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Aleksandr Nikolaev
>Assignee: Aleksandr Nikolaev
>Priority: Major
>  Labels: ise
> Fix For: 2.16
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Update Ignite dependency zookeeper



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787960#comment-17787960
 ] 

Alexey Scherbakov edited comment on IGNITE-17811 at 11/20/23 3:05 PM:
--

This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no size limit, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additional lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce OOMs with a current lock table size.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894
One of possible optimizations is to implement pre-commit waiting (based on 
counters) for reads as well (removing server side wait queues, which are bad 
for heap usage)


was (Author: ascherbakov):
This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additional lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce OOMs with a current lock table size.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894
One of possible optimizations is to implement pre-commit waiting (based on 
counters) for reads as well (removing server side wait queues, which are bad 
for heap usage)

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-17811:
-
Fix Version/s: 3.0.0-beta2

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin reassigned IGNITE-17811:


Assignee: Alexey Scherbakov

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20871) throw sneakyThrow() causes incomplete stack traces

2023-11-20 Thread Pavel Tupitsyn (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17788026#comment-17788026
 ] 

Pavel Tupitsyn commented on IGNITE-20871:
-

[~isapego] looks good to me.

> throw sneakyThrow() causes incomplete stack traces
> --
>
> Key: IGNITE-20871
> URL: https://issues.apache.org/jira/browse/IGNITE-20871
> Project: Ignite
>  Issue Type: Bug
>  Components: thin client
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Igor Sapego
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> *ClientUtils.sync* and some other places use the following code:
> {code}
> throw sneakyThrow(ExceptionUtils.unwrapToPublicException(e))
> {code}
> This causes incomplete stack trace, because we rethrow an existing exception.
> Instead, we should always create a new exception:
> {code}
> throw sneakyThrow(ExceptionUtils.copyExceptionWithCause(e));
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20902) Fix IgniteWalRebalanceTest flakiness

2023-11-20 Thread Ilya Shishkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Shishkov updated IGNITE-20902:
---
Description: 
Some of {{IgniteWalRebalanceTest}} methods are flaky:
# {{#testRebalanceCancelOnSupplyError}} - fails often.
# {{#testWithLocalWalChange}} - fails rarely.

Tests fails with a foolowing error:
{code}
java.lang.AssertionError: WAL rebalance hasn't been invoked.

at 
org.apache.ignite.internal.processors.cache.persistence.db.wal.IgniteWalRebalanceTest.afterTest(IgniteWalRebalanceTest.java:197)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runAfterTest(GridAbstractTest.java:760)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.cleanUpTestEnviroment(GridAbstractTest.java:737)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2550)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$1.evaluate(GridAbstractTest.java:235)
...
{code}

  was:
Some of {{IgniteWalRebalanceTest}} methods are flaky:
{{#testRebalanceCancelOnSupplyError}} - fails often.
{{#testWithLocalWalChange}} - fails rarely.

Tests fails with a foolowing error:
{code}
java.lang.AssertionError: WAL rebalance hasn't been invoked.

at 
org.apache.ignite.internal.processors.cache.persistence.db.wal.IgniteWalRebalanceTest.afterTest(IgniteWalRebalanceTest.java:197)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runAfterTest(GridAbstractTest.java:760)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.cleanUpTestEnviroment(GridAbstractTest.java:737)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2550)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$1.evaluate(GridAbstractTest.java:235)
...
{code}


> Fix IgniteWalRebalanceTest flakiness
> 
>
> Key: IGNITE-20902
> URL: https://issues.apache.org/jira/browse/IGNITE-20902
> Project: Ignite
>  Issue Type: Test
>Reporter: Ilya Shishkov
>Priority: Trivial
>  Labels: ise
>
> Some of {{IgniteWalRebalanceTest}} methods are flaky:
> # {{#testRebalanceCancelOnSupplyError}} - fails often.
> # {{#testWithLocalWalChange}} - fails rarely.
> Tests fails with a foolowing error:
> {code}
> java.lang.AssertionError: WAL rebalance hasn't been invoked.
>   at 
> org.apache.ignite.internal.processors.cache.persistence.db.wal.IgniteWalRebalanceTest.afterTest(IgniteWalRebalanceTest.java:197)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runAfterTest(GridAbstractTest.java:760)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.cleanUpTestEnviroment(GridAbstractTest.java:737)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2550)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$1.evaluate(GridAbstractTest.java:235)
> ...
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20902) Fix IgniteWalRebalanceTest flakiness

2023-11-20 Thread Ilya Shishkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Shishkov updated IGNITE-20902:
---
Description: 
Some of {{IgniteWalRebalanceTest}} methods are flaky:
{{#testRebalanceCancelOnSupplyError}} - fails often.
{{#testWithLocalWalChange}} - fails rarely.

Tests fails with a foolowing error:
{code}
java.lang.AssertionError: WAL rebalance hasn't been invoked.

at 
org.apache.ignite.internal.processors.cache.persistence.db.wal.IgniteWalRebalanceTest.afterTest(IgniteWalRebalanceTest.java:197)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runAfterTest(GridAbstractTest.java:760)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.cleanUpTestEnviroment(GridAbstractTest.java:737)
at 
org.apache.ignite.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2550)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$1.evaluate(GridAbstractTest.java:235)
...
{code}

  was:
Some of {{IgniteWalRebalanceTest}} methods are flaky:
{{#testRebalanceCancelOnSupplyError}} - fails often.
{{#testWithLocalWalChange}} - fails rarely.


> Fix IgniteWalRebalanceTest flakiness
> 
>
> Key: IGNITE-20902
> URL: https://issues.apache.org/jira/browse/IGNITE-20902
> Project: Ignite
>  Issue Type: Test
>Reporter: Ilya Shishkov
>Priority: Trivial
>  Labels: ise
>
> Some of {{IgniteWalRebalanceTest}} methods are flaky:
> {{#testRebalanceCancelOnSupplyError}} - fails often.
> {{#testWithLocalWalChange}} - fails rarely.
> Tests fails with a foolowing error:
> {code}
> java.lang.AssertionError: WAL rebalance hasn't been invoked.
>   at 
> org.apache.ignite.internal.processors.cache.persistence.db.wal.IgniteWalRebalanceTest.afterTest(IgniteWalRebalanceTest.java:197)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runAfterTest(GridAbstractTest.java:760)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.cleanUpTestEnviroment(GridAbstractTest.java:737)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest.runTest(GridAbstractTest.java:2550)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$1.evaluate(GridAbstractTest.java:235)
> ...
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20902) Fix IgniteWalRebalanceTest flakiness

2023-11-20 Thread Ilya Shishkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Shishkov updated IGNITE-20902:
---
Description: 
Some of {{IgniteWalRebalanceTest}} methods are flaky:
{{#testRebalanceCancelOnSupplyError}} - fails often.
{{#testWithLocalWalChange}} - fails rarely.

> Fix IgniteWalRebalanceTest flakiness
> 
>
> Key: IGNITE-20902
> URL: https://issues.apache.org/jira/browse/IGNITE-20902
> Project: Ignite
>  Issue Type: Test
>Reporter: Ilya Shishkov
>Priority: Trivial
>  Labels: ise
>
> Some of {{IgniteWalRebalanceTest}} methods are flaky:
> {{#testRebalanceCancelOnSupplyError}} - fails often.
> {{#testWithLocalWalChange}} - fails rarely.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-19229) Schema validation during tx processing: validators

2023-11-20 Thread Roman Puchkovskiy (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-19229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy reassigned IGNITE-19229:
--

Assignee: Roman Puchkovskiy

> Schema validation during tx processing: validators
> --
>
> Key: IGNITE-19229
> URL: https://issues.apache.org/jira/browse/IGNITE-19229
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: iep-110, ignite-3
>
> As per 
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-110%3A+Schema+synchronization%3A+basic+schema+changes
>  , the following schema changes are forward-compatible:
> # Creating a table
> # Adding a nullable column or a column having a default value
> # Renaming a column
> # Dropping a constraint (including NOT NULL constraint)
> # Changing a column type when any value in the old type has an exact 
> representation in the new type
> Other schema changes are not compatible.
> The supported type changes follow:
> * Widening conversions between integral types (like int->long)
> * Widening conversions between floating-point types (float->double)
> * Widening conversions between date-time types (increasing precision)
> * Widening conversions between decimal types (increasing precision)
> * Increasing allowed length of VARCHAR
> * Increasing allowed length of VARBINARY
> * Conversions of anything to VARCHAR with enough capacity



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20902) Fix IgniteWalRebalanceTest flakiness

2023-11-20 Thread Ilya Shishkov (Jira)
Ilya Shishkov created IGNITE-20902:
--

 Summary: Fix IgniteWalRebalanceTest flakiness
 Key: IGNITE-20902
 URL: https://issues.apache.org/jira/browse/IGNITE-20902
 Project: Ignite
  Issue Type: Test
Reporter: Ilya Shishkov






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20902) Fix IgniteWalRebalanceTest flakiness

2023-11-20 Thread Ilya Shishkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Shishkov updated IGNITE-20902:
---
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Fix IgniteWalRebalanceTest flakiness
> 
>
> Key: IGNITE-20902
> URL: https://issues.apache.org/jira/browse/IGNITE-20902
> Project: Ignite
>  Issue Type: Test
>Reporter: Ilya Shishkov
>Priority: Trivial
>  Labels: ise
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20900) .NET: Thin 3.0: Handle replica assignment timestamp

2023-11-20 Thread Pavel Tupitsyn (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Tupitsyn updated IGNITE-20900:

Description: 
IGNITE-20701 introduces assignment timestamps - in change notification and in 
ClientOp.PartitionAssignmentGet.

Use those timestamps to retrieve correct assignment version and avoid using 
stale assignment - refer to Java client logic.

> .NET: Thin 3.0: Handle replica assignment timestamp
> ---
>
> Key: IGNITE-20900
> URL: https://issues.apache.org/jira/browse/IGNITE-20900
> Project: Ignite
>  Issue Type: Improvement
>  Components: platforms, thin client
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
>  Labels: .NET, ignite-3
> Fix For: 3.0.0-beta2
>
>
> IGNITE-20701 introduces assignment timestamps - in change notification and in 
> ClientOp.PartitionAssignmentGet.
> Use those timestamps to retrieve correct assignment version and avoid using 
> stale assignment - refer to Java client logic.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20900) .NET: Thin 3.0: Handle replica assignment timestamp

2023-11-20 Thread Pavel Tupitsyn (Jira)
Pavel Tupitsyn created IGNITE-20900:
---

 Summary: .NET: Thin 3.0: Handle replica assignment timestamp
 Key: IGNITE-20900
 URL: https://issues.apache.org/jira/browse/IGNITE-20900
 Project: Ignite
  Issue Type: Improvement
  Components: platforms, thin client
Affects Versions: 3.0.0-beta1
Reporter: Pavel Tupitsyn
Assignee: Pavel Tupitsyn
 Fix For: 3.0.0-beta2






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-20898) Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)

2023-11-20 Thread Anastasia Iakimova (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Anastasia Iakimova reassigned IGNITE-20898:
---

Assignee: Anastasia Iakimova

> Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)
> --
>
> Key: IGNITE-20898
> URL: https://issues.apache.org/jira/browse/IGNITE-20898
> Project: Ignite
>  Issue Type: Bug
>Reporter: Nikita Amelchev
>Assignee: Anastasia Iakimova
>Priority: Minor
>  Labels: ise
>
> Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL). They 
> are deprecated in javadocs after IGNITE-13003, but annotation is not used. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20898) Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)

2023-11-20 Thread Nikita Amelchev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikita Amelchev updated IGNITE-20898:
-
Labels: ise  (was: )

> Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)
> --
>
> Key: IGNITE-20898
> URL: https://issues.apache.org/jira/browse/IGNITE-20898
> Project: Ignite
>  Issue Type: Bug
>Reporter: Nikita Amelchev
>Priority: Minor
>  Labels: ise
>
> Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL). They 
> are deprecated in javadocs after IGNITE-13003, but annotation is not used. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-14578) Bootstrap configuation manager with distributed configuration

2023-11-20 Thread Kirill Gusakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787992#comment-17787992
 ] 

Kirill Gusakov commented on IGNITE-14578:
-

LGTM with PR about test removing

> Bootstrap configuation manager with distributed configuration
> -
>
> Key: IGNITE-14578
> URL: https://issues.apache.org/jira/browse/IGNITE-14578
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: iep-73, ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Update IgntionImpl with code that will
>  * Add distributed root keys.
>  * Bootstrap distributed configuration.
> h3. UPD#1
> The issue is no longer relevant, because tables were moved from configuration 
> to catalog. Thus, it's not possible to bootstrap cluster with tables thought 
> configuration. The corresponding test should be removed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-14578) Bootstrap configuation manager with distributed configuration

2023-11-20 Thread Kirill Gusakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787992#comment-17787992
 ] 

Kirill Gusakov edited comment on IGNITE-14578 at 11/20/23 12:45 PM:


LGTM PR about test removing


was (Author: kgusakov):
LGTM with PR about test removing

> Bootstrap configuation manager with distributed configuration
> -
>
> Key: IGNITE-14578
> URL: https://issues.apache.org/jira/browse/IGNITE-14578
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: iep-73, ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Update IgntionImpl with code that will
>  * Add distributed root keys.
>  * Bootstrap distributed configuration.
> h3. UPD#1
> The issue is no longer relevant, because tables were moved from configuration 
> to catalog. Thus, it's not possible to bootstrap cluster with tables thought 
> configuration. The corresponding test should be removed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20899) Sql. Align duplicate classes usage : o.a.i.internal.utils.PrimaryReplica and o.a.i.internal.sql.engine.exec.NodeWithTerm

2023-11-20 Thread Evgeny Stanilovsky (Jira)
Evgeny Stanilovsky created IGNITE-20899:
---

 Summary: Sql. Align duplicate classes usage : 
o.a.i.internal.utils.PrimaryReplica and 
o.a.i.internal.sql.engine.exec.NodeWithTerm
 Key: IGNITE-20899
 URL: https://issues.apache.org/jira/browse/IGNITE-20899
 Project: Ignite
  Issue Type: Improvement
  Components: sql
Affects Versions: 3.0.0-beta1
Reporter: Evgeny Stanilovsky


Seems:
org.apache.ignite.internal.utils.PrimaryReplica and 
org.apache.ignite.internal.sql.engine.exec.NodeWithTerm
have the same semantic and we need to get rid one of them



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20898) Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL)

2023-11-20 Thread Nikita Amelchev (Jira)
Nikita Amelchev created IGNITE-20898:


 Summary: Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, 
READ_WRITE_ALL)
 Key: IGNITE-20898
 URL: https://issues.apache.org/jira/browse/IGNITE-20898
 Project: Ignite
  Issue Type: Bug
Reporter: Nikita Amelchev


Fix deprecation of PartitionLossPolicy (READ_ONLY_ALL, READ_WRITE_ALL). They 
are deprecated in javadocs after IGNITE-13003, but annotation is not used. 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20897) Ignite should clean tmp binaryMeta files on start

2023-11-20 Thread Maksim Timonin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Timonin updated IGNITE-20897:

Ignite Flags:   (was: Docs Required,Release Notes Required)

> Ignite should clean tmp binaryMeta files on start
> -
>
> Key: IGNITE-20897
> URL: https://issues.apache.org/jira/browse/IGNITE-20897
> Project: Ignite
>  Issue Type: Bug
>Reporter: Maksim Timonin
>Priority: Major
>
> Ignite uses tmp files for BinaryMeta for creation. It's possible to land with 
> the tmp file after Ignite node failed. Then such a file must be remove on 
> Ignite start.
> The problem is to find a place where to clean these files:
>  # BinaryMetadataAsyncWriter is a thread that actually writes to binary meta.
>  # But this thread is also created by StandaloneGridKernalContext instances 
> on start. It means that utils using it might have write access to the 
> binary_meta directory.
> Options are:
>  # Clean files in any callback after Ignite started (#onKernalStart0)
>  # Make BinaryMetadataFileStore read only for utils.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20897) Ignite should clean tmp binaryMeta files on start

2023-11-20 Thread Maksim Timonin (Jira)
Maksim Timonin created IGNITE-20897:
---

 Summary: Ignite should clean tmp binaryMeta files on start
 Key: IGNITE-20897
 URL: https://issues.apache.org/jira/browse/IGNITE-20897
 Project: Ignite
  Issue Type: Bug
Reporter: Maksim Timonin


Ignite uses tmp files for BinaryMeta for creation. It's possible to land with 
the tmp file after Ignite node failed. Then such a file must be remove on 
Ignite start.

The problem is to find a place where to clean these files:
 # BinaryMetadataAsyncWriter is a thread that actually writes to binary meta.
 # But this thread is also created by StandaloneGridKernalContext instances on 
start. It means that utils using it might have write access to the binary_meta 
directory.

Options are:
 # Clean files in any callback after Ignite started (#onKernalStart0)
 # Make BinaryMetadataFileStore read only for utils.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787960#comment-17787960
 ] 

Alexey Scherbakov edited comment on IGNITE-17811 at 11/20/23 12:03 PM:
---

This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additional lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce OOMs with a current lock table size.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894
One of possible optimizations is to implement pre-commit waiting (based on 
counters) for reads as well (removing server side wait queues, which are bad 
for heap usage)


was (Author: ascherbakov):
This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additional lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce OOMs with a current lock table size.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894
One of possible fixes is to implement pre-commit waiting (based on counters) 
for reads as well (removing server side wait queues, which are bad for heap 
usage)

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787960#comment-17787960
 ] 

Alexey Scherbakov edited comment on IGNITE-17811 at 11/20/23 12:03 PM:
---

This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additional lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce OOMs with a current lock table size.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894
One of possible fixes is to implement pre-commit waiting (based on counters) 
for reads as well (removing server side wait queues, which are bad for heap 
usage)


was (Author: ascherbakov):
This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additional lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce OOMs with a current lock table size.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20896) Fix MultifieldIndexQueryTest#testCheckBoundaries flakiness

2023-11-20 Thread Ilya Shishkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Shishkov updated IGNITE-20896:
---
Description: 
Test fails with an error:
{code}
java.lang.AssertionError: 
Expected :2
Actual   :3

at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at 
org.apache.ignite.testframework.junits.JUnitAssertAware.assertEquals(JUnitAssertAware.java:95)
at 
org.apache.ignite.cache.query.MultifieldIndexQueryTest.testCheckBoundaries(MultifieldIndexQueryTest.java:170)
{code}

  was:
Test fails with an error:
{code}
java.lang.AssertionError: 
Expected :2
Actual   :3

at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at 
org.apache.ignite.testframework.junits.JUnitAssertAware.assertEquals(JUnitAssertAware.java:95)
at 
org.apache.ignite.cache.query.MultifieldIndexQueryTest.testCheckBoundaries(MultifieldIndexQueryTest.java:170)
at jdk.internal.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2499)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}


> Fix MultifieldIndexQueryTest#testCheckBoundaries flakiness
> --
>
> Key: IGNITE-20896
> URL: https://issues.apache.org/jira/browse/IGNITE-20896
> Project: Ignite
>  Issue Type: Test
>Reporter: Ilya Shishkov
>Priority: Trivial
>  Labels: ise
>
> Test fails with an error:
> {code}
> java.lang.AssertionError: 
> Expected :2
> Actual   :3
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:834)
>   at org.junit.Assert.assertEquals(Assert.java:645)
>   at org.junit.Assert.assertEquals(Assert.java:631)
>   at 
> org.apache.ignite.testframework.junits.JUnitAssertAware.assertEquals(JUnitAssertAware.java:95)
>   at 
> org.apache.ignite.cache.query.MultifieldIndexQueryTest.testCheckBoundaries(MultifieldIndexQueryTest.java:170)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20896) Fix MultifieldIndexQueryTest#testCheckBoundaries flakiness

2023-11-20 Thread Ilya Shishkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ilya Shishkov updated IGNITE-20896:
---
Description: 
Test fails with an error:
{code}
java.lang.AssertionError: 
Expected :2
Actual   :3

at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at 
org.apache.ignite.testframework.junits.JUnitAssertAware.assertEquals(JUnitAssertAware.java:95)
at 
org.apache.ignite.cache.query.MultifieldIndexQueryTest.testCheckBoundaries(MultifieldIndexQueryTest.java:170)
at jdk.internal.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2499)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}

  was:
Test fails with an error:
{code}
java.lang.AssertionError: 
Expected :2
Actual   :3


at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at 
org.apache.ignite.testframework.junits.JUnitAssertAware.assertEquals(JUnitAssertAware.java:95)
at 
org.apache.ignite.cache.query.MultifieldIndexQueryTest.testCheckBoundaries(MultifieldIndexQueryTest.java:170)
at jdk.internal.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2499)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}


> Fix MultifieldIndexQueryTest#testCheckBoundaries flakiness
> --
>
> Key: IGNITE-20896
> URL: https://issues.apache.org/jira/browse/IGNITE-20896
> Project: Ignite
>  Issue Type: Test
>Reporter: Ilya Shishkov
>Priority: Trivial
>  Labels: ise
>
> Test fails with an error:
> {code}
> java.lang.AssertionError: 
> Expected :2
> Actual   :3
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:834)
>   at org.junit.Assert.assertEquals(Assert.java:645)
>   at org.junit.Assert.assertEquals(Assert.java:631)
>   at 
> org.apache.ignite.testframework.junits.JUnitAssertAware.assertEquals(JUnitAssertAware.java:95)
>   at 
> org.apache.ignite.cache.query.MultifieldIndexQueryTest.testCheckBoundaries(MultifieldIndexQueryTest.java:170)
>   at jdk.internal.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
>   at 
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.base/java.lang.reflect.Method.invoke(Method.java:566)
>   at 
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
>   at 
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
>   at 
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
>   at 
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
>   at 
> org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2499)
>   at java.base/java.lang.Thread.run(Thread.java:829)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20896) Fix MultifieldIndexQueryTest#testCheckBoundaries flakiness

2023-11-20 Thread Ilya Shishkov (Jira)
Ilya Shishkov created IGNITE-20896:
--

 Summary: Fix MultifieldIndexQueryTest#testCheckBoundaries flakiness
 Key: IGNITE-20896
 URL: https://issues.apache.org/jira/browse/IGNITE-20896
 Project: Ignite
  Issue Type: Test
Reporter: Ilya Shishkov


Test fails with an error:
{code}
java.lang.AssertionError: 
Expected :2
Actual   :3


at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at org.junit.Assert.assertEquals(Assert.java:631)
at 
org.apache.ignite.testframework.junits.JUnitAssertAware.assertEquals(JUnitAssertAware.java:95)
at 
org.apache.ignite.cache.query.MultifieldIndexQueryTest.testCheckBoundaries(MultifieldIndexQueryTest.java:170)
at jdk.internal.reflect.GeneratedMethodAccessor23.invoke(Unknown Source)
at 
java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at 
org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
at 
org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
at 
org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
at 
org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
at 
org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2499)
at java.base/java.lang.Thread.run(Thread.java:829)
{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787960#comment-17787960
 ] 

Alexey Scherbakov edited comment on IGNITE-17811 at 11/20/23 11:36 AM:
---

This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additional lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce OOMs with a current lock table size.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894


was (Author: ascherbakov):
This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additional lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce leaks.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787960#comment-17787960
 ] 

Alexey Scherbakov edited comment on IGNITE-17811 at 11/20/23 11:35 AM:
---

This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additional lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce leaks.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894


was (Author: ascherbakov):
This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additinal lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce leaks.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787960#comment-17787960
 ] 

Alexey Scherbakov edited comment on IGNITE-17811 at 11/20/23 11:35 AM:
---

This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to measure single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additinal lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce leaks.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894


was (Author: ascherbakov):
This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to manage single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additinal lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce leaks.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787960#comment-17787960
 ] 

Alexey Scherbakov edited comment on IGNITE-17811 at 11/20/23 11:34 AM:
---

This PR includes some optimizations and fixes. Most important:
 # HeapLockManager now has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to manage single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additinal lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce leaks.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894


was (Author: ascherbakov):
This PR includes some optimizations and fixes. Most important:
 # HeapLockManager has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to manage single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additinal lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce leaks.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Comment Edited] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787960#comment-17787960
 ] 

Alexey Scherbakov edited comment on IGNITE-17811 at 11/20/23 11:33 AM:
---

This PR includes some optimizations and fixes. Most important:
 # HeapLockManager has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to manage single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additinal lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce leaks.
 # A ticket was filed for incorrect behavior of commit in a deadlock scenario 
https://issues.apache.org/jira/browse/IGNITE-20894


was (Author: ascherbakov):
This PR includes some optimizations and fixes. Most important:
 # HeapLockManager has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to manage single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additinal lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce leaks.

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-17811) Implement efficient way to retrieve locks by txId

2023-11-20 Thread Alexey Scherbakov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787960#comment-17787960
 ] 

Alexey Scherbakov commented on IGNITE-17811:


This PR includes some optimizations and fixes. Most important:
 # HeapLockManager has additional index on all enlisted locks into a 
transaction.
 # A new method org.apache.ignite.internal.tx.impl.HeapLockManager#releaseAll 
was introduced to release transaction locks in most efficient way.
 # A lock table now has a size limitation to avoid hash map to grow without 
bounds. If a size limit is extended, keys are mapped directly to raw slots 
using hash function of a key. Still the lock queue has no limit size, it has to 
be addressed as well.
 # A more efficient hash function murmur3 is used for byte array keys. Is has 
better collision resistance than a default hash function.
 # LockManagerBenchmark was introduced to manage single threaded locking speed.
 # A hierarchy (coarse grained) locking was refactored to additinal lock table 
for a further improvement, see 
https://issues.apache.org/jira/browse/IGNITE-20895
 # Some memory leaks in tests related to mockito were fixed. It is recommended 
to have clearInlineMocks for each method instead of a suite, because the suite 
can be quite big and produce leaks.

> Implement efficient way to retrieve locks by txId
> -
>
> Key: IGNITE-17811
> URL: https://issues.apache.org/jira/browse/IGNITE-17811
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> org.apache.ignite.internal.tx.impl.HeapLockManager#locks require better 
> implementation that'll use index or similar instead of full locks set 
> iteration. 
> {code:java}
> public Iterator locks(UUID txId) {
> // TODO: tmp, use index instead.
> List result = new ArrayList<>();
> for (Map.Entry entry : locks.entrySet()) {
> Waiter waiter = entry.getValue().waiter(txId);
> if (waiter != null) {
> result.add(
> new Lock(
> entry.getKey(),
> waiter.lockMode(),
> txId
> )
> );
> }
> }
> return result.iterator();
> }{code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20895) Optimize hierarchy lock manager

2023-11-20 Thread Alexey Scherbakov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Scherbakov updated IGNITE-20895:
---
Description: 
Recently a (not very efficient) parentLockManager  [1] was introduced to handle 
hierarchy (coarse-grained) locks.

Currently it's only purpose is to provide strict serializability during full 
table scans, which happens very rarely.

It comes at the cost of acquiring additional lock on each update op. IS locks 
are never used at all.

This should be improved. Possible optimizations:
 # Get rid of hierarchy locks and instead scan the primary index for table 
scan. If I recall correctly, Mysql never uses hierarchy locking.
 # If we are absolutely sure that we need hierarchy locking, replace lock 
queuing for IS,IX locks with counters. This prevents waiting, but is looks ok 
for this kind of locks. Only stronger locks should be allowed to wait.

[1] org.apache.ignite.internal.tx.impl.HeapLockManager#parentLockManager

  was:
Recently a (not very efficient) parentLockManager  [1]was introduced to handle 
hierarchy (coarse-grained) locks.

Currently it's only purpose is to provide strict serializability during full 
table scans, which happed very rarely.

It comes at the cost of acquiring additional lock on each update op. IS locks 
are never used at all.

It should be improved. Possible optimizations:
 # Get rid of hierarchy locks and instead scan the primary index for table 
scane.
 # Replace lock queuing for IS,IX locks with counters. This prevents waiting, 
but is looks ok for this kind of locks. Only stronger locks should be allowed 
to wait.

[1] org.apache.ignite.internal.tx.impl.HeapLockManager#parentLockManager


> Optimize hierarchy lock manager
> ---
>
> Key: IGNITE-20895
> URL: https://issues.apache.org/jira/browse/IGNITE-20895
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 3.0
>Reporter: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0
>
>
> Recently a (not very efficient) parentLockManager  [1] was introduced to 
> handle hierarchy (coarse-grained) locks.
> Currently it's only purpose is to provide strict serializability during full 
> table scans, which happens very rarely.
> It comes at the cost of acquiring additional lock on each update op. IS locks 
> are never used at all.
> This should be improved. Possible optimizations:
>  # Get rid of hierarchy locks and instead scan the primary index for table 
> scan. If I recall correctly, Mysql never uses hierarchy locking.
>  # If we are absolutely sure that we need hierarchy locking, replace lock 
> queuing for IS,IX locks with counters. This prevents waiting, but is looks ok 
> for this kind of locks. Only stronger locks should be allowed to wait.
> [1] org.apache.ignite.internal.tx.impl.HeapLockManager#parentLockManager



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20895) Optimize hierarchy lock manager

2023-11-20 Thread Alexey Scherbakov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Scherbakov updated IGNITE-20895:
---
Labels: ignite-3  (was: )

> Optimize hierarchy lock manager
> ---
>
> Key: IGNITE-20895
> URL: https://issues.apache.org/jira/browse/IGNITE-20895
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 3.0
>Reporter: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0
>
>
> Recently a (not very efficient) parentLockManager  [1]was introduced to 
> handle hierarchy (coarse-grained) locks.
> Currently it's only purpose is to provide strict serializability during full 
> table scans, which happed very rarely.
> It comes at the cost of acquiring additional lock on each update op. IS locks 
> are never used at all.
> It should be improved. Possible optimizations:
>  # Get rid of hierarchy locks and instead scan the primary index for table 
> scane.
>  # Replace lock queuing for IS,IX locks with counters. This prevents waiting, 
> but is looks ok for this kind of locks. Only stronger locks should be allowed 
> to wait.
> [1] org.apache.ignite.internal.tx.impl.HeapLockManager#parentLockManager



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20895) Optimize hierarchy lock manager

2023-11-20 Thread Alexey Scherbakov (Jira)
Alexey Scherbakov created IGNITE-20895:
--

 Summary: Optimize hierarchy lock manager
 Key: IGNITE-20895
 URL: https://issues.apache.org/jira/browse/IGNITE-20895
 Project: Ignite
  Issue Type: Improvement
Affects Versions: 3.0
Reporter: Alexey Scherbakov
 Fix For: 3.0


Recently a (not very efficient) parentLockManager  [1]was introduced to handle 
hierarchy (coarse-grained) locks.

Currently it's only purpose is to provide strict serializability during full 
table scans, which happed very rarely.

It comes at the cost of acquiring additional lock on each update op. IS locks 
are never used at all.

It should be improved. Possible optimizations:
 # Get rid of hierarchy locks and instead scan the primary index for table 
scane.
 # Replace lock queuing for IS,IX locks with counters. This prevents waiting, 
but is looks ok for this kind of locks. Only stronger locks should be allowed 
to wait.

[1] org.apache.ignite.internal.tx.impl.HeapLockManager#parentLockManager



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20894) Fix deadlock recovery on commit

2023-11-20 Thread Alexey Scherbakov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20894?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexey Scherbakov updated IGNITE-20894:
---
Labels: ignite-3  (was: )

> Fix deadlock recovery on commit
> ---
>
> Key: IGNITE-20894
> URL: https://issues.apache.org/jira/browse/IGNITE-20894
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 3.0
>Reporter: Alexey Scherbakov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0
>
>
> Reproduced by [1]
> In short, two transaction are put into deadlock, using no prevention policy.
> First tx is committed, but can't acquire locks.
> Expected behavior - tx is rolled back on commit, releasing associated locks, 
> tx2 succefully commits.
> Actual behavior - commit hangs.
> [1] org.apache.ignite.distributed.ItLockTableTest#testDeadlockRecovery



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20894) Fix deadlock recovery on commit

2023-11-20 Thread Alexey Scherbakov (Jira)
Alexey Scherbakov created IGNITE-20894:
--

 Summary: Fix deadlock recovery on commit
 Key: IGNITE-20894
 URL: https://issues.apache.org/jira/browse/IGNITE-20894
 Project: Ignite
  Issue Type: Bug
Affects Versions: 3.0
Reporter: Alexey Scherbakov
 Fix For: 3.0


Reproduced by [1]

In short, two transaction are put into deadlock, using no prevention policy.

First tx is committed, but can't acquire locks.

Expected behavior - tx is rolled back on commit, releasing associated locks, 
tx2 succefully commits.

Actual behavior - commit hangs.

[1] org.apache.ignite.distributed.ItLockTableTest#testDeadlockRecovery



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20685) Implement ability to trigger transaction recovery

2023-11-20 Thread Vladislav Pyatkov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787946#comment-17787946
 ] 

Vladislav Pyatkov commented on IGNITE-20685:


Merged 8bdbb88f9751fe50ded7732ab3968ebb0bb219a1

> Implement ability to trigger transaction recovery
> -
>
> Key: IGNITE-20685
> URL: https://issues.apache.org/jira/browse/IGNITE-20685
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Attachments: OrphanTxRecovery-LockConflictHandling.jpg
>
>  Time Spent: 5.5h
>  Remaining Estimate: 0h
>
> h3. Motivation
> Let's assume that the datanode somehow found out that the transaction 
> coordinator is dead, but the products of its activity such as locks and write 
> intents are still present. In that case it’s necessary to check whether 
> corresponding transaction was actually finished and if not finish it.
> h3. Definition of Done
>  * Transactions X that detects (detection logic will be covered in a separate 
> ticket) that coordinator is dead awaits commitPartition primary replica and 
> sends initiateRecoveryReplicaRequest to it in a fully asynchronous manner. 
> Meaning that transaction X should behave itself in a way as it specified in 
> deadlock prevention engine and do not explicitly wait for initiateRecovery 
> result. Actually, we do not expect any direct response from initiate 
> recovery. Initiate recovery failover will be implemented in a different way.
>  * Commit partition somewhere handles given request. No-op handling is 
> expected for now, proper one will be added in IGNITE-20735 Let's consider 
> either TransactionStateResolver or TxManagerImpl as initiateRecovery handler. 
> TransactionStateResolver seems as the best choice here, however it should be 
> refactored a bit, basically because it's won't be only state resolver any 
> longer.
> h3. Implementation Notes
>  * Given ticket is trivial and should be considered as a bridge between 
> durable tx coordinator liveness detection and corresponding 
> initiateRecoveryReplicaRequest handling. Both items will be covered in a 
> separate tickets.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-20893) Update snappy in cdc

2023-11-20 Thread Aleksandr Nikolaev (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787937#comment-17787937
 ] 

Aleksandr Nikolaev commented on IGNITE-20893:
-

https://ci2.ignite.apache.org/buildConfiguration/IgniteExtensions_Tests_RunAllTests/7622813?buildTab=overview=true=false=false=false=true

> Update snappy in cdc
> 
>
> Key: IGNITE-20893
> URL: https://issues.apache.org/jira/browse/IGNITE-20893
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Aleksandr Nikolaev
>Assignee: Aleksandr Nikolaev
>Priority: Major
>  Labels: ise
>
> Update snappy in cdc



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20852) Opposite connection attempts may cause connection failure

2023-11-20 Thread Roman Puchkovskiy (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy updated IGNITE-20852:
---
Summary: Opposite connection attempts may cause connection failure  (was: 
Concurrent connection attempts may cause connection failure)

> Opposite connection attempts may cause connection failure
> -
>
> Key: IGNITE-20852
> URL: https://issues.apache.org/jira/browse/IGNITE-20852
> Project: Ignite
>  Issue Type: Bug
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Sometimes 2 nodes (A and B) try to establish logical connections to each 
> other at the same time. Only one of the connections can remain, so another 
> one is aborted. When such an abortion happens, the client that initiated the 
> aborted physical connection must be transparently switched to the connection 
> opened 'from the other side'.
> A subsituation where each of the competitors has already acquired a lock (at 
> different sides of the connection) is called a clinch. In other cases, one 
> node manages to take both locks.
> In most of such situations, MessagingService gets an exception on one of the 
> sides. We must avoid this.
> Another problem is that the clinch resulution protocol sometimes kills an 
> 'almost established connection' (or even a fully estabished connection) [that 
> has managed to aquire recovery descriptors on both sides] just because 
> another connection (that has just failed to aquire first recovery descriptor) 
> is preferred by the tie-breaker. The tie-breaker must only be used in true 
> clinch situations (each of the competitors has 1 descriptor aquired).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20852) Connection attempt clinches may cause connection failure

2023-11-20 Thread Roman Puchkovskiy (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy updated IGNITE-20852:
---
Description: 
Sometimes 2 nodes (A and B) try to establish logical connections to each other 
at the same time. Only one of the connections can remain, so another one is 
aborted. When such an abortion happens, the client that initiated the aborted 
physical connection must be transparently switched to the connection opened 
'from the other side'.

A subsituation where each of the competitors has already acquired a lock (at 
different sides of the connection) is called a clinch. In other cases, one node 
manages to take both locks.

In most of such situations, MessagingService gets an exception on one of the 
sides. We must avoid this.

Another problem is that the clinch resulution protocol sometimes kills an 
'almost established connection' (or even a fully estabished connection) [that 
has managed to aquire recovery descriptors on both sides] just because another 
connection (that has just failed to aquire first recovery descriptor) is 
preferred by the tie-breaker. The tie-breaker must only be used in true clinch 
situations (each of the competitors has 1 descriptor aquired).

  was:
A clinch is a situation when two nodes (A and B) try to establish logical 
connections to each other at the same time. Only one of the connections can 
remain, so another one is aborted. When such an abortion happens, the client 
that initiated the aborted physical connection must be transparently switched 
to the connection opened 'from the other side'.

Instead, in most of the clinches, MessagingService gets an exception as a 
result of the clinch resolution on one of the sides. We must avoid this.

Another problem is that the clinch resulution protocol sometimes kills an 
'almost established connection' (or even a fully estabished connection) [that 
has managed to aquire recovery descriptors on both sides] just because another 
connection (that has just failed to aquire first recovery descriptor) is 
preferred by the tie-breaker. The tie-breaker must only be used in true clinch 
situations (each of the competitors have 1 descriptor aquired).


> Connection attempt clinches may cause connection failure
> 
>
> Key: IGNITE-20852
> URL: https://issues.apache.org/jira/browse/IGNITE-20852
> Project: Ignite
>  Issue Type: Bug
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Sometimes 2 nodes (A and B) try to establish logical connections to each 
> other at the same time. Only one of the connections can remain, so another 
> one is aborted. When such an abortion happens, the client that initiated the 
> aborted physical connection must be transparently switched to the connection 
> opened 'from the other side'.
> A subsituation where each of the competitors has already acquired a lock (at 
> different sides of the connection) is called a clinch. In other cases, one 
> node manages to take both locks.
> In most of such situations, MessagingService gets an exception on one of the 
> sides. We must avoid this.
> Another problem is that the clinch resulution protocol sometimes kills an 
> 'almost established connection' (or even a fully estabished connection) [that 
> has managed to aquire recovery descriptors on both sides] just because 
> another connection (that has just failed to aquire first recovery descriptor) 
> is preferred by the tie-breaker. The tie-breaker must only be used in true 
> clinch situations (each of the competitors has 1 descriptor aquired).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20852) Concurrent connection attempts may cause connection failure

2023-11-20 Thread Roman Puchkovskiy (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20852?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Roman Puchkovskiy updated IGNITE-20852:
---
Summary: Concurrent connection attempts may cause connection failure  (was: 
Connection attempt clinches may cause connection failure)

> Concurrent connection attempts may cause connection failure
> ---
>
> Key: IGNITE-20852
> URL: https://issues.apache.org/jira/browse/IGNITE-20852
> Project: Ignite
>  Issue Type: Bug
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Sometimes 2 nodes (A and B) try to establish logical connections to each 
> other at the same time. Only one of the connections can remain, so another 
> one is aborted. When such an abortion happens, the client that initiated the 
> aborted physical connection must be transparently switched to the connection 
> opened 'from the other side'.
> A subsituation where each of the competitors has already acquired a lock (at 
> different sides of the connection) is called a clinch. In other cases, one 
> node manages to take both locks.
> In most of such situations, MessagingService gets an exception on one of the 
> sides. We must avoid this.
> Another problem is that the clinch resulution protocol sometimes kills an 
> 'almost established connection' (or even a fully estabished connection) [that 
> has managed to aquire recovery descriptors on both sides] just because 
> another connection (that has just failed to aquire first recovery descriptor) 
> is preferred by the tie-breaker. The tie-breaker must only be used in true 
> clinch situations (each of the competitors has 1 descriptor aquired).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20885) Sql. Bump calcite version to 1.36

2023-11-20 Thread Yury Gerzhedovich (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20885?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yury Gerzhedovich updated IGNITE-20885:
---
Summary: Sql. Bump calcite version to 1.36  (was: SQL. Bump calcite version 
to 1.36)

> Sql. Bump calcite version to 1.36
> -
>
> Key: IGNITE-20885
> URL: https://issues.apache.org/jira/browse/IGNITE-20885
> Project: Ignite
>  Issue Type: Improvement
>  Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Evgeny Stanilovsky
>Priority: Major
>
> New version is resolved, also [3] consists in this version and we can 
> simplify corresponding (SqlKind.DEFAULT keyword) code in 
> IgniteSqlToRelConvertor implemented here [2]
> [1] https://calcite.apache.org/docs/history.html#v1-36-0
> [2] https://issues.apache.org/jira/browse/IGNITE-19096
> [3] https://issues.apache.org/jira/browse/CALCITE-5950



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-20750) ExecutionServiceImpl#stop() may hang forever

2023-11-20 Thread Pavel Pereslegin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pavel Pereslegin reassigned IGNITE-20750:
-

Assignee: Pavel Pereslegin

> ExecutionServiceImpl#stop() may hang forever
> 
>
> Key: IGNITE-20750
> URL: https://issues.apache.org/jira/browse/IGNITE-20750
> Project: Ignite
>  Issue Type: Bug
>  Components: sql
>Reporter: Roman Puchkovskiy
>Assignee: Pavel Pereslegin
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> A build hung on TC: 
> [https://ci.ignite.apache.org/buildConfiguration/ApacheIgnite3xGradle_Test_IntegrationTests_ModuleRunner/7589436?hideProblemsFromDependencies=false=false=true=true]
> In the thread dump the following can be seen:
>  
> "Test worker" #1 prio=5 os_prio=0 cpu=123640.80ms elapsed=3573.05s 
> tid=0x7f8de802e000 nid=0x2110df waiting on condition  [0x7f8decb1d000]
>    java.lang.Thread.State: WAITING (parking)
>     at jdk.internal.misc.Unsafe.park(java.base@11.0.17/Native Method)
>     - parking to wait for  <0x00071962ff08> (a 
> java.util.concurrent.CompletableFuture$Signaller)
>     at 
> java.util.concurrent.locks.LockSupport.park(java.base@11.0.17/LockSupport.java:194)
>     at 
> java.util.concurrent.CompletableFuture$Signaller.block(java.base@11.0.17/CompletableFuture.java:1796)
>     at 
> java.util.concurrent.ForkJoinPool.managedBlock(java.base@11.0.17/ForkJoinPool.java:3128)
>     at 
> java.util.concurrent.CompletableFuture.waitingGet(java.base@11.0.17/CompletableFuture.java:1823)
>     at 
> java.util.concurrent.CompletableFuture.join(java.base@11.0.17/CompletableFuture.java:2043)
>     at 
> org.apache.ignite.internal.sql.engine.exec.ExecutionServiceImpl.stop(ExecutionServiceImpl.java:402)
>     at 
> org.apache.ignite.internal.sql.engine.SqlQueryProcessor$$Lambda$2103/0x000800ba7840.close(Unknown
>  Source)
>     at 
> org.apache.ignite.internal.util.IgniteUtils.lambda$closeAll$0(IgniteUtils.java:534)
>     at 
> org.apache.ignite.internal.util.IgniteUtils$$Lambda$2054/0x000800b8f040.accept(Unknown
>  Source)
>     at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(java.base@11.0.17/ForEachOps.java:183)
>     at 
> java.util.stream.ReferencePipeline$2$1.accept(java.base@11.0.17/ReferencePipeline.java:177)
>     at 
> java.util.stream.ReferencePipeline$3$1.accept(java.base@11.0.17/ReferencePipeline.java:195)
>     at 
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(java.base@11.0.17/ArrayList.java:1655)
>     at 
> java.util.stream.AbstractPipeline.copyInto(java.base@11.0.17/AbstractPipeline.java:484)
>     at 
> java.util.stream.AbstractPipeline.wrapAndCopyInto(java.base@11.0.17/AbstractPipeline.java:474)
>     at 
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(java.base@11.0.17/ForEachOps.java:150)
>     at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(java.base@11.0.17/ForEachOps.java:173)
>     at 
> java.util.stream.AbstractPipeline.evaluate(java.base@11.0.17/AbstractPipeline.java:234)
>     at 
> java.util.stream.ReferencePipeline.forEach(java.base@11.0.17/ReferencePipeline.java:497)
>     at 
> org.apache.ignite.internal.util.IgniteUtils.closeAll(IgniteUtils.java:532)
>     at 
> org.apache.ignite.internal.sql.engine.SqlQueryProcessor.stop(SqlQueryProcessor.java:380)
>     - locked <0x000721d65408> (a 
> org.apache.ignite.internal.sql.engine.SqlQueryProcessor)
>     at 
> org.apache.ignite.internal.app.LifecycleManager.lambda$stopAllComponents$1(LifecycleManager.java:133)
>     at 
> org.apache.ignite.internal.app.LifecycleManager$$Lambda$2100/0x000800ba6c40.accept(Unknown
>  Source)
>     at 
> java.util.Iterator.forEachRemaining(java.base@11.0.17/Iterator.java:133)
>     at 
> org.apache.ignite.internal.app.LifecycleManager.stopAllComponents(LifecycleManager.java:131)
>     - locked <0x00071e1eb730> (a 
> org.apache.ignite.internal.app.LifecycleManager)
>     at 
> org.apache.ignite.internal.app.LifecycleManager.stopNode(LifecycleManager.java:115)
>     at org.apache.ignite.internal.app.IgniteImpl.stop(IgniteImpl.java:903)
>     at 
> org.apache.ignite.internal.app.IgnitionImpl.lambda$stop$0(IgnitionImpl.java:113)
>     at 
> org.apache.ignite.internal.app.IgnitionImpl$$Lambda$2056/0x000800b8f840.apply(Unknown
>  Source)
>     at 
> java.util.concurrent.ConcurrentHashMap.computeIfPresent(java.base@11.0.17/ConcurrentHashMap.java:1822)
>     - locked <0x000736357750> (a 
> java.util.concurrent.ConcurrentHashMap$Node)
>     at org.apache.ignite.internal.app.IgnitionImpl.stop(IgnitionImpl.java:111)
>     at org.apache.ignite.IgnitionManager.stop(IgnitionManager.java:96)
>     at org.apache.ignite.IgnitionManager.stop(IgnitionManager.java:82)
>     at 

[jira] [Assigned] (IGNITE-20837) Thin Client Partition Awareness with a node filter

2023-11-20 Thread Vladimir Steshin (Jira)


[jira] [Commented] (IGNITE-20891) Add thenCompose to OrderingFuture

2023-11-20 Thread Roman Puchkovskiy (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-20891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17787892#comment-17787892
 ] 

Roman Puchkovskiy commented on IGNITE-20891:


Thanks

> Add thenCompose to OrderingFuture
> -
>
> Key: IGNITE-20891
> URL: https://issues.apache.org/jira/browse/IGNITE-20891
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Roman Puchkovskiy
>Assignee: Roman Puchkovskiy
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Needed for IGNITE-20852



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-20893) Update snappy in cdc

2023-11-20 Thread Aleksandr Nikolaev (Jira)
Aleksandr Nikolaev created IGNITE-20893:
---

 Summary: Update snappy in cdc
 Key: IGNITE-20893
 URL: https://issues.apache.org/jira/browse/IGNITE-20893
 Project: Ignite
  Issue Type: Improvement
Reporter: Aleksandr Nikolaev
Assignee: Aleksandr Nikolaev


Update snappy in cdc



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-20818) High periodic latency spikes on long put-get benchmark

2023-11-20 Thread Ivan Bessonov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-20818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov updated IGNITE-20818:
---
Reviewer: Roman Puchkovskiy

> High periodic latency spikes on long put-get benchmark
> --
>
> Key: IGNITE-20818
> URL: https://issues.apache.org/jira/browse/IGNITE-20818
> Project: Ignite
>  Issue Type: Bug
>  Components: persistence
>Reporter: Ivan Artiukhov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3, ignite3_performance
> Attachments: kv-get.png, kv-put.png
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> AI3, rev. c15f14f4ddca7d6e9ab3e02d5a661e68d6e9ee21
> Benchmark parameters:
>  * 1 server node
>  * aipersist storage engine, 25 partitions (default), raft.fsync=false
> Benchmark code: 
> [https://github.com/gridgain/YCSB/blob/ycsb-2023.7/ignite3/src/main/java/site/ycsb/db/ignite3/IgniteClient.java]
>  
> Benchmark parameters:
>  * 100% put of 25 million unique entries, 1 thread
>  * then 100% get of 25 million entries, 1 thread
> "put" throughput:
> !kv-put.png!
> "get" throughput:
> !kv-get.png!
> On the graphs we see the performance drop which happen every 5 minutes which 
> begins after ~50 minutes since the start of the test. Then after ~4 hours and 
> 20 minutes the periodic drops stopped to appear. 
> Timings:
> {noformat}
> 12:11 -- puts start
> 13:01 -- first spike
> 15:46 -- puts end
> 15:47 -- gets start
> 16:31 -- spikes ended
> 18:27 -- gets end{noformat}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-14578) Bootstrap configuation manager with distributed configuration

2023-11-20 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-14578:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Bootstrap configuation manager with distributed configuration
> -
>
> Key: IGNITE-14578
> URL: https://issues.apache.org/jira/browse/IGNITE-14578
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: iep-73, ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Update IgntionImpl with code that will
>  * Add distributed root keys.
>  * Bootstrap distributed configuration.
> h3. UPD#1
> The issue is no longer relevant, because tables were moved from configuration 
> to catalog. Thus, it's not possible to bootstrap cluster with tables thought 
> configuration. The corresponding test should be removed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-14578) Bootstrap configuation manager with distributed configuration

2023-11-20 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-14578:
-
Reviewer: Kirill Gusakov

> Bootstrap configuation manager with distributed configuration
> -
>
> Key: IGNITE-14578
> URL: https://issues.apache.org/jira/browse/IGNITE-14578
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: iep-73, ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Update IgntionImpl with code that will
>  * Add distributed root keys.
>  * Bootstrap distributed configuration.
> h3. UPD#1
> The issue is no longer relevant, because tables were moved from configuration 
> to catalog. Thus, it's not possible to bootstrap cluster with tables thought 
> configuration. The corresponding test should be removed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-14578) Bootstrap configuation manager with distributed configuration

2023-11-20 Thread Alexander Lapin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-14578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Lapin updated IGNITE-14578:
-
Description: 
Update IgntionImpl with code that will
 * Add distributed root keys.
 * Bootstrap distributed configuration.

h3. UPD#1

The issue is no longer relevant, because tables were moved from configuration 
to catalog. Thus, it's not possible to bootstrap cluster with tables thought 
configuration. The corresponding test should be removed.

  was:
Update IgntionImpl with code that will 
* Add distributed root keys.
* Bootstrap distributed configuration.


> Bootstrap configuation manager with distributed configuration
> -
>
> Key: IGNITE-14578
> URL: https://issues.apache.org/jira/browse/IGNITE-14578
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
>  Labels: iep-73, ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Update IgntionImpl with code that will
>  * Add distributed root keys.
>  * Bootstrap distributed configuration.
> h3. UPD#1
> The issue is no longer relevant, because tables were moved from configuration 
> to catalog. Thus, it's not possible to bootstrap cluster with tables thought 
> configuration. The corresponding test should be removed.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)