[jira] [Commented] (IGNITE-9228) Spark SQL Table Schema Specification
[ https://issues.apache.org/jira/browse/IGNITE-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16923059#comment-16923059 ] Manoj G T commented on IGNITE-9228: --- [~NIzhikov] [~stuartmacd] I have gone through the implementation of supporting schema name Ignite Spark SQL and it's only having support to schema name to read data, not for write. Any specific reason for the same? Is it in the future roadmap? > Spark SQL Table Schema Specification > > > Key: IGNITE-9228 > URL: https://issues.apache.org/jira/browse/IGNITE-9228 > Project: Ignite > Issue Type: Improvement > Components: spark >Affects Versions: 2.6 >Reporter: Stuart Macdonald >Assignee: Stuart Macdonald >Priority: Major > Fix For: 2.8 > > > The Ignite Spark SQL interface currently takes just “table name” as a > parameter which it uses to supply a Spark dataset with data from the > underlying Ignite SQL table with that name. > To do this it loops through each cache and finds the first one with the > given table name [1]. This causes issues if there are multiple tables > registered in different schema with the same table name as you can only > access one of those from Spark. We could either: > 1. Pass an extra parameter through the Ignite Spark data source which > optionally specifies the schema name. > 2. Support namespacing in the existing table name parameter, ie > “schemaName.tableName” > [1 > ]https://github.com/apache/ignite/blob/ca973ad99c6112160a305df05be9458e29f88307/modules/spark/src/main/scala/org/apache/ignite/spark/impl/package.scala#L119 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (IGNITE-12133) O(log n) partition exchange
[ https://issues.apache.org/jira/browse/IGNITE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922809#comment-16922809 ] Alexei Scherbakov edited comment on IGNITE-12133 at 9/4/19 8:03 PM: PME protocol itself doesn't leverage ring and uses direct node to node communication for sending partition maps (except for special case), but ring is used by discovery protocol, which "discovers" topology changes and delivers event to grid nodes, which triggers PME due to topology changes, for example "node left" or "node added". Also discovery protocol provides "guaranteed ordered messages delivery" which is extensively used by Ignite internals and cannot be replaced easily. Actually, PME consists of three phases: 1. Discovery phase, having O( n ) complexity for default TcpDiscoverySpi implementation. 2. Topology unlock waiting. 3. PME phase having k * O(m) complextity where m is number of I/O sender threads and k depends on topology size. So total PME complexity is sum of 1 2 and 3. To speed up PME we should improve 1 2 and 3. How to improve 1 ? Initially ring was designed for small topologies and still works very well for such cases with default settings. Specially for large topologies zookeeper based discovery was introduced, which have better complexity. So, for small topologies I suggest to use defaults. For large topologies zookeeper discovery should be used. How to improve 2 ? This is discussed on dev list in lightweigh PME topic. How to improve 3 ? For small topologies same as 1, use defaults. For large topologies we could use [~mnk]'s proposal and use tree-like message propagation pattern to achieve log(N) complexity. I agree with [~ivan.glukos] on increasing failover complexity, but I think it's doable. NOTE: same idea could be used for increasing replicated caches performance on large topologies. We have long time known issue with performance degradation if topology is large. [~Jokser] Gossip idea looks interesting, but looks like complicated change and reinventing the wheel. Why not stick to zookeeper? was (Author: ascherbakov): PME protocol itself doesn't leverage ring and uses direct node to node communication for sending partition maps (except for special case), but ring is used by discovery protocol, which "discovers" topology changes and delivers event to grid nodes, which triggers PME due to topology changes, for example "node left" or "node added". Also discovery protocol provides "guaranteed ordered messages delivery" which is extensively used by Ignite internals and cannot be replaced easily. Actually, PME consists of three phases: 1. Discovery phase, having O( n ) complexity for default TcpDiscoverySpi implementation. 2. Topology unlock waiting (out of this post's scope). 3. PME phase having k * O(m) complextity where m is number of I/O sender threads and k depends on topology size. So total PME complexity is sum of 1 2 and 3. To speed up PME we should improve 1 and 3. How to improve 1 ? Initially ring was designed for small topologies and still works very well for such cases with default settings. Specially for large topologies zookeeper based discovery was introduced, which have better complexity. So, for small topologies I suggest to use defaults. For large topologies zookeeper discovery should be used. How to improve 3 ? For small topologies same as 1, use defaults. For large topologies we could use [~mnk]'s proposal and use tree-like message propagation pattern to achieve log(N) complexity. I agree with [~ivan.glukos] on increasing failover complexity, but I think it's doable. NOTE: same idea could be used for increasing replicated caches performance on large topologies. We have long time known issue with performance degradation if topology is large. [~Jokser] Gossip idea looks interesting, but looks like complicated change and reinventing the wheel. Why not stick to zookeeper? > O(log n) partition exchange > --- > > Key: IGNITE-12133 > URL: https://issues.apache.org/jira/browse/IGNITE-12133 > Project: Ignite > Issue Type: Improvement >Reporter: Moti Nisenson-Ken >Priority: Major > > Currently, partition exchange leverages a ring. This means that > communications is O\(n) in number of nodes. It also means that if > non-coordinator nodes hang it can take much longer to successfully resolve > the topology. > Instead, why not use something like a skip-list where the coordinator is > first. The coordinator can notify the first node at each level of the > skip-list. Each node then notifies all of its "near-neighbours" in the > skip-list, where node B is a near-neighbour of node-A, if max-level(nodeB) <= > max-level(nodeA), and nodeB is the first node at its level when traversing > from nodeA in the direction of
[jira] [Comment Edited] (IGNITE-12133) O(log n) partition exchange
[ https://issues.apache.org/jira/browse/IGNITE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922809#comment-16922809 ] Alexei Scherbakov edited comment on IGNITE-12133 at 9/4/19 8:02 PM: PME protocol itself doesn't leverage ring and uses direct node to node communication for sending partition maps (except for special case), but ring is used by discovery protocol, which "discovers" topology changes and delivers event to grid nodes, which triggers PME due to topology changes, for example "node left" or "node added". Also discovery protocol provides "guaranteed ordered messages delivery" which is extensively used by Ignite internals and cannot be replaced easily. Actually, PME consists of three phases: 1. Discovery phase, having O( n ) complexity for default TcpDiscoverySpi implementation. 2. Topology unlock waiting (out of this post's scope). 3. PME phase having k * O(m) complextity where m is number of I/O sender threads and k depends on topology size. So total PME complexity is sum of 1 2 and 3. To speed up PME we should improve 1 and 3. How to improve 1 ? Initially ring was designed for small topologies and still works very well for such cases with default settings. Specially for large topologies zookeeper based discovery was introduced, which have better complexity. So, for small topologies I suggest to use defaults. For large topologies zookeeper discovery should be used. How to improve 3 ? For small topologies same as 1, use defaults. For large topologies we could use [~mnk]'s proposal and use tree-like message propagation pattern to achieve log(N) complexity. I agree with [~ivan.glukos] on increasing failover complexity, but I think it's doable. NOTE: same idea could be used for increasing replicated caches performance on large topologies. We have long time known issue with performance degradation if topology is large. [~Jokser] Gossip idea looks interesting, but looks like complicated change and reinventing the wheel. Why not stick to zookeeper? was (Author: ascherbakov): PME protocol itself doesn't leverage ring and uses direct node to node communication for sending partition maps (except for special case), but ring is used by discovery protocol, which "discovers" topology changes and delivers event to grid nodes, which triggers PME due to topology changes, for example "node left" or "node added". Also discovery protocol provides "guaranteed ordered messages delivery" which is extensively used by Ignite internals and cannot be replaced easily. Actually, PME consists of three phases: 1. Discovery phase, having O( n ) complexity for default TcpDiscoverySpi implementation. 2. Topology unlock waiting (out of this post's scope). 3. PME phase having k * O(m) complextity where m is number of I/O sender threads and k depends on topology size. So total PME complexity is sum of 1 and 3. To speed up PME we should improve 1 and 3. How to improve 1 ? Initially ring was designed for small topologies and still works very well for such cases with default settings. Specially for large topologies zookeeper based discovery was introduced, which have better complexity. So, for small topologies I suggest to use defaults. For large topologies zookeeper discovery should be used. How to improve 3 ? For small topologies same as 1, use defaults. For large topologies we could use [~mnk]'s proposal and use tree-like message propagation pattern to achieve log(N) complexity. I agree with [~ivan.glukos] on increasing failover complexity, but I think it's doable. NOTE: same idea could be used for increasing replicated caches performance on large topologies. We have long time known issue with performance degradation if topology is large. [~Jokser] Gossip idea looks interesting, but looks like complicated change and reinventing the wheel. Why not stick to zookeeper? > O(log n) partition exchange > --- > > Key: IGNITE-12133 > URL: https://issues.apache.org/jira/browse/IGNITE-12133 > Project: Ignite > Issue Type: Improvement >Reporter: Moti Nisenson-Ken >Priority: Major > > Currently, partition exchange leverages a ring. This means that > communications is O\(n) in number of nodes. It also means that if > non-coordinator nodes hang it can take much longer to successfully resolve > the topology. > Instead, why not use something like a skip-list where the coordinator is > first. The coordinator can notify the first node at each level of the > skip-list. Each node then notifies all of its "near-neighbours" in the > skip-list, where node B is a near-neighbour of node-A, if max-level(nodeB) <= > max-level(nodeA), and nodeB is the first node at its level when traversing > from nodeA in the direction of nodeB, skipping over nodes C which have > max-leve
[jira] [Commented] (IGNITE-12133) O(log n) partition exchange
[ https://issues.apache.org/jira/browse/IGNITE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922809#comment-16922809 ] Alexei Scherbakov commented on IGNITE-12133: PME protocol itself doesn't leverage ring and uses direct node to node communication for sending partition maps (except for special case), but ring is used by discovery protocol, which "discovers" topology changes and delivers event to grid nodes, which triggers PME due to topology changes, for example "node left" or "node added". Also discovery protocol provides "guaranteed ordered messages delivery" which is extensively used by Ignite internals and cannot be replaced easily. Actually, PME consists of three phases: 1. Discovery phase, having O(n) complexity for default TcpDiscoverySpi implementation. 2. Topology unlock waiting (out of this post's scope). 3. PME phase having k * O(m) complextity where m is number of I/O sender threads and k depends on topology size. So total PME complexity is sum of 1 and 3. To speed up PME we should improve 1 and 3. How to improve 1 ? Initially ring was designed for small topologies and still works very well for such cases with default settings. Specially for large topologies zookeeper based discovery was introduced, which have better complexity. So, for small topologies I suggest to use defaults. For large topologies zookeeper discovery should be used. How to improve 3 ? For small topologies same as 1, use defaults. For large topologies we could use [~mnk]'s proposal and use tree-like message propagation pattern to achieve log(N) complexity. I agree with [~ivan.glukos] on increasing failover complexity, but I think it's doable. NOTE: same idea could be used for increasing replicated caches performance on large topologies. We have long time known issue with performance degradation if topology is large. [~Jokser] Gossip idea looks interesting, but looks like complicated change and reinventing the wheel. Why not stick to zookeeper? > O(log n) partition exchange > --- > > Key: IGNITE-12133 > URL: https://issues.apache.org/jira/browse/IGNITE-12133 > Project: Ignite > Issue Type: Improvement >Reporter: Moti Nisenson-Ken >Priority: Major > > Currently, partition exchange leverages a ring. This means that > communications is O\(n) in number of nodes. It also means that if > non-coordinator nodes hang it can take much longer to successfully resolve > the topology. > Instead, why not use something like a skip-list where the coordinator is > first. The coordinator can notify the first node at each level of the > skip-list. Each node then notifies all of its "near-neighbours" in the > skip-list, where node B is a near-neighbour of node-A, if max-level(nodeB) <= > max-level(nodeA), and nodeB is the first node at its level when traversing > from nodeA in the direction of nodeB, skipping over nodes C which have > max-level(C) > max-level(A). > 1 > 1 . . .3 > 1 3 . . . 5 > 1 . 2 . 3 . 4 . 5 . 6 > In the above 1 would notify 2 and 3, 3 would notify 4 and 5, 2 -> 4, and 4 -> > 6, and 5 -> 6. > One can achieve better redundancy by having each node traverse in both > directions, and having the coordinator also notify the last node in the list > at each level. This way in the above example if 2 and 3 were both down, 4 > would still get notified from 5 and 6 (in the backwards direction). > > The idea is that each individual node has O(log n) nodes to notify - so the > overall time is reduced. Additionally, we can deal well with at least 1 node > failure - if one includes the option of processing backwards, 2 consecutive > node failures can be handled as well. By taking this kind of an approach, > then the coordinator can basically treat any nodes it didn't receive a > message from as not-connected, and update the topology as well (disconnecting > any nodes that it didn't get a notification from). While there are some edge > cases here (e.g. 2 disconnected nodes, then 1 connected node, then 2 > disconnected nodes - the connected node would be wrongly ejected from the > topology), these would generally be too rare to need explicit handling for. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (IGNITE-12133) O(log n) partition exchange
[ https://issues.apache.org/jira/browse/IGNITE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922809#comment-16922809 ] Alexei Scherbakov edited comment on IGNITE-12133 at 9/4/19 8:00 PM: PME protocol itself doesn't leverage ring and uses direct node to node communication for sending partition maps (except for special case), but ring is used by discovery protocol, which "discovers" topology changes and delivers event to grid nodes, which triggers PME due to topology changes, for example "node left" or "node added". Also discovery protocol provides "guaranteed ordered messages delivery" which is extensively used by Ignite internals and cannot be replaced easily. Actually, PME consists of three phases: 1. Discovery phase, having O( n ) complexity for default TcpDiscoverySpi implementation. 2. Topology unlock waiting (out of this post's scope). 3. PME phase having k * O(m) complextity where m is number of I/O sender threads and k depends on topology size. So total PME complexity is sum of 1 and 3. To speed up PME we should improve 1 and 3. How to improve 1 ? Initially ring was designed for small topologies and still works very well for such cases with default settings. Specially for large topologies zookeeper based discovery was introduced, which have better complexity. So, for small topologies I suggest to use defaults. For large topologies zookeeper discovery should be used. How to improve 3 ? For small topologies same as 1, use defaults. For large topologies we could use [~mnk]'s proposal and use tree-like message propagation pattern to achieve log(N) complexity. I agree with [~ivan.glukos] on increasing failover complexity, but I think it's doable. NOTE: same idea could be used for increasing replicated caches performance on large topologies. We have long time known issue with performance degradation if topology is large. [~Jokser] Gossip idea looks interesting, but looks like complicated change and reinventing the wheel. Why not stick to zookeeper? was (Author: ascherbakov): PME protocol itself doesn't leverage ring and uses direct node to node communication for sending partition maps (except for special case), but ring is used by discovery protocol, which "discovers" topology changes and delivers event to grid nodes, which triggers PME due to topology changes, for example "node left" or "node added". Also discovery protocol provides "guaranteed ordered messages delivery" which is extensively used by Ignite internals and cannot be replaced easily. Actually, PME consists of three phases: 1. Discovery phase, having O(n) complexity for default TcpDiscoverySpi implementation. 2. Topology unlock waiting (out of this post's scope). 3. PME phase having k * O(m) complextity where m is number of I/O sender threads and k depends on topology size. So total PME complexity is sum of 1 and 3. To speed up PME we should improve 1 and 3. How to improve 1 ? Initially ring was designed for small topologies and still works very well for such cases with default settings. Specially for large topologies zookeeper based discovery was introduced, which have better complexity. So, for small topologies I suggest to use defaults. For large topologies zookeeper discovery should be used. How to improve 3 ? For small topologies same as 1, use defaults. For large topologies we could use [~mnk]'s proposal and use tree-like message propagation pattern to achieve log(N) complexity. I agree with [~ivan.glukos] on increasing failover complexity, but I think it's doable. NOTE: same idea could be used for increasing replicated caches performance on large topologies. We have long time known issue with performance degradation if topology is large. [~Jokser] Gossip idea looks interesting, but looks like complicated change and reinventing the wheel. Why not stick to zookeeper? > O(log n) partition exchange > --- > > Key: IGNITE-12133 > URL: https://issues.apache.org/jira/browse/IGNITE-12133 > Project: Ignite > Issue Type: Improvement >Reporter: Moti Nisenson-Ken >Priority: Major > > Currently, partition exchange leverages a ring. This means that > communications is O\(n) in number of nodes. It also means that if > non-coordinator nodes hang it can take much longer to successfully resolve > the topology. > Instead, why not use something like a skip-list where the coordinator is > first. The coordinator can notify the first node at each level of the > skip-list. Each node then notifies all of its "near-neighbours" in the > skip-list, where node B is a near-neighbour of node-A, if max-level(nodeB) <= > max-level(nodeA), and nodeB is the first node at its level when traversing > from nodeA in the direction of nodeB, skipping over nodes C which have > max-level(C)
[jira] [Resolved] (IGNITE-12137) ODBC: Application hang when connecting to a starting single node
[
https://issues.apache.org/jira/browse/IGNITE-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov resolved IGNITE-12137.
Fix Version/s: 2.8
Resolution: Duplicate
[~toninlg], ok, thank you!
I've closed the ticket as duplicate.
> ODBC: Application hang when connecting to a starting single node
>
>
> Key: IGNITE-12137
> URL: https://issues.apache.org/jira/browse/IGNITE-12137
> Project: Ignite
> Issue Type: Bug
> Components: odbc
>Affects Versions: 2.7.5
> Environment: Windows 7
>Reporter: AH
>Priority: Major
> Fix For: 2.8
>
>
> If I repeatedly try to connect with ODBC before starting only one node, it
> will at one time connect but the first query causes my application to hang.
> After reading that ODBC driver is not thread safe, I make sure to issue
> statement on one thread at a time and the behaviour is still the same.
> +console output:+
> D:\apache-ignite-2.7.5-bin\platforms\dotnet\bin>Apache.Ignite.exe
> Sep 03, 2019 9:32:21 AM
> org.springframework.beans.factory.xml.XmlBeanDefinitionR
> eader loadBeanDefinitions
> INFO: Loading XML bean definitions from URL
> [file:/D:/apache-ignite-2.7.5-bin/co
> nfig/default-config.xml]
> Sep 03, 2019 9:32:21 AM
> org.springframework.context.support.AbstractApplicationC
> ontext prepareRefresh
> INFO: Refreshing
> org.springframework.context.support.GenericApplicationContext@2
> 7082746: startup date [Tue Sep 03 09:32:21 CEST 2019]; root of context
> hierarchy
> [09:32:22] __
> [09:32:22] / _/ ___/ |/ / _/_ __/ __/
> [09:32:22] _/ // (7 7 // / / / / _/
> [09:32:22] /___/\___/_/|_/___/ /_/ /___/
> [09:32:22]
> [09:32:22] ver. 2.7.5#20190603-sha1:be4f2a15
> [09:32:22] 2018 Copyright(C) Apache Software Foundation
> [09:32:22]
> [09:32:22] Ignite documentation: http://ignite.apache.org
> [09:32:22]
> [09:32:22] Quiet mode.
> [09:32:22] ^-- Logging to file 'D:\apache-ignite-2.7.5-bin\work\log\ignite-413
> 535b7.0.log'
> [09:32:22] ^-- Logging by 'JavaLogger [quiet=true, config=null]'
> [09:32:22] ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-
> v" to ignite.\{sh|bat}
> [09:32:22]
> [09:32:22] OS: Windows 7 6.1 amd64
> [09:32:22] VM information: Java(TM) SE Runtime Environment 1.8.0_191-b12
> Oracle
> Corporation Java HotSpot(TM) 64-Bit Server VM 25.191-b12
> [09:32:22] Please set system property '-Djava.net.preferIPv4Stack=true' to
> avoid
> possible problems in mixed environments.
> [09:32:22] Initial heap size is 192MB (should be no less than 512MB, use
> -Xms512
> m -Xmx512m).
> [09:32:22] Configured plugins:
> [09:32:22] ^-- None
> [09:32:22]
> [09:32:22] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler
> [trySto
> p=false, timeout=0, super=AbstractFailureHandler
> [ignoredFailureTypes=[SYSTEM_WO
> RKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT
> [09:32:23] Message queue limit is set to 0 which may lead to potential OOMEs
> whe
> n running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message
> qu
> eues growth on sender and receiver sides.
> [09:32:23] Security status [authentication=off, tls/ssl=off]
> [09:32:25,782][SEVERE][client-connector-#41][ClientListenerProcessor] Runtime
> er
> ror caught during grid runnable execution: GridWorker
> [name=message-received-not
> ify, igniteInstanceName=null, finished=false, heartbeatTs=1567495945774,
> hashCod
> e=215381183, interrupted=false, runner=client-connector-#41]
> java.lang.NullPointerException
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> nextConnectionId(ClientListenerNioListener.java:306)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> prepareContext(ClientListenerNioListener.java:285)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onHandshake(ClientListenerNioListener.java:223)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onMessage(ClientListenerNioListener.java:129)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onMessage(ClientListenerNioListener.java:45)
> at org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onM
> essageReceived(GridNioFilterChain.java:279)
> at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessa
> geReceived(GridNioFilterAdapter.java:109)
> at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(G
> ridNioAsyncNotifyFilter.java:97)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java
> :120)
> at org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorke
> rPool.java:70)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> java:1149)
> at java.util.concurr
[jira] [Commented] (IGNITE-12137) ODBC: Application hang when connecting to a starting single node
[
https://issues.apache.org/jira/browse/IGNITE-12137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922510#comment-16922510
]
AH commented on IGNITE-12137:
-
I've tested with the master. It is fixed.
> ODBC: Application hang when connecting to a starting single node
>
>
> Key: IGNITE-12137
> URL: https://issues.apache.org/jira/browse/IGNITE-12137
> Project: Ignite
> Issue Type: Bug
> Components: odbc
>Affects Versions: 2.7.5
> Environment: Windows 7
>Reporter: AH
>Priority: Major
>
> If I repeatedly try to connect with ODBC before starting only one node, it
> will at one time connect but the first query causes my application to hang.
> After reading that ODBC driver is not thread safe, I make sure to issue
> statement on one thread at a time and the behaviour is still the same.
> +console output:+
> D:\apache-ignite-2.7.5-bin\platforms\dotnet\bin>Apache.Ignite.exe
> Sep 03, 2019 9:32:21 AM
> org.springframework.beans.factory.xml.XmlBeanDefinitionR
> eader loadBeanDefinitions
> INFO: Loading XML bean definitions from URL
> [file:/D:/apache-ignite-2.7.5-bin/co
> nfig/default-config.xml]
> Sep 03, 2019 9:32:21 AM
> org.springframework.context.support.AbstractApplicationC
> ontext prepareRefresh
> INFO: Refreshing
> org.springframework.context.support.GenericApplicationContext@2
> 7082746: startup date [Tue Sep 03 09:32:21 CEST 2019]; root of context
> hierarchy
> [09:32:22] __
> [09:32:22] / _/ ___/ |/ / _/_ __/ __/
> [09:32:22] _/ // (7 7 // / / / / _/
> [09:32:22] /___/\___/_/|_/___/ /_/ /___/
> [09:32:22]
> [09:32:22] ver. 2.7.5#20190603-sha1:be4f2a15
> [09:32:22] 2018 Copyright(C) Apache Software Foundation
> [09:32:22]
> [09:32:22] Ignite documentation: http://ignite.apache.org
> [09:32:22]
> [09:32:22] Quiet mode.
> [09:32:22] ^-- Logging to file 'D:\apache-ignite-2.7.5-bin\work\log\ignite-413
> 535b7.0.log'
> [09:32:22] ^-- Logging by 'JavaLogger [quiet=true, config=null]'
> [09:32:22] ^-- To see **FULL** console log here add -DIGNITE_QUIET=false or "-
> v" to ignite.\{sh|bat}
> [09:32:22]
> [09:32:22] OS: Windows 7 6.1 amd64
> [09:32:22] VM information: Java(TM) SE Runtime Environment 1.8.0_191-b12
> Oracle
> Corporation Java HotSpot(TM) 64-Bit Server VM 25.191-b12
> [09:32:22] Please set system property '-Djava.net.preferIPv4Stack=true' to
> avoid
> possible problems in mixed environments.
> [09:32:22] Initial heap size is 192MB (should be no less than 512MB, use
> -Xms512
> m -Xmx512m).
> [09:32:22] Configured plugins:
> [09:32:22] ^-- None
> [09:32:22]
> [09:32:22] Configured failure handler: [hnd=StopNodeOrHaltFailureHandler
> [trySto
> p=false, timeout=0, super=AbstractFailureHandler
> [ignoredFailureTypes=[SYSTEM_WO
> RKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT
> [09:32:23] Message queue limit is set to 0 which may lead to potential OOMEs
> whe
> n running cache operations in FULL_ASYNC or PRIMARY_SYNC modes due to message
> qu
> eues growth on sender and receiver sides.
> [09:32:23] Security status [authentication=off, tls/ssl=off]
> [09:32:25,782][SEVERE][client-connector-#41][ClientListenerProcessor] Runtime
> er
> ror caught during grid runnable execution: GridWorker
> [name=message-received-not
> ify, igniteInstanceName=null, finished=false, heartbeatTs=1567495945774,
> hashCod
> e=215381183, interrupted=false, runner=client-connector-#41]
> java.lang.NullPointerException
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> nextConnectionId(ClientListenerNioListener.java:306)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> prepareContext(ClientListenerNioListener.java:285)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onHandshake(ClientListenerNioListener.java:223)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onMessage(ClientListenerNioListener.java:129)
> at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.
> onMessage(ClientListenerNioListener.java:45)
> at org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onM
> essageReceived(GridNioFilterChain.java:279)
> at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessa
> geReceived(GridNioFilterAdapter.java:109)
> at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(G
> ridNioAsyncNotifyFilter.java:97)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java
> :120)
> at org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorke
> rPool.java:70)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.
> java:1149)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor
> .java:624)
> at java.la
[jira] [Updated] (IGNITE-12127) WAL writer may close file IO with unflushed changes when MMAP is disabled
[
https://issues.apache.org/jira/browse/IGNITE-12127?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitriy Govorukhin updated IGNITE-12127:
Ignite Flags: (was: Docs Required,Release Notes Required)
> WAL writer may close file IO with unflushed changes when MMAP is disabled
> -
>
> Key: IGNITE-12127
> URL: https://issues.apache.org/jira/browse/IGNITE-12127
> Project: Ignite
> Issue Type: Bug
>Reporter: Dmitriy Govorukhin
>Assignee: Dmitriy Govorukhin
>Priority: Critical
> Fix For: 2.7.6
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Most likely the issue manifests itself as the following critical error:
> {code}
> 2019-08-27 14:52:31.286 ERROR 26835 --- [wal-write-worker%null-#447] ROOT :
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=class o.a.i.failure.StopNodeOrHaltFailureHandler,
> failureCtx=FailureContext [type=CRITICAL_ERROR, err=class
> o.a.i.i.processors.cache.persistence.StorageException: Failed to write
> buffer.]]
> org.apache.ignite.internal.processors.cache.persistence.StorageException:
> Failed to write buffer.
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$WALWriter.writeBuffer(FileWriteAheadLogManager.java:3444)
> [ignite-core-2.5.7.jar!/:2.5.7]
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$WALWriter.body(FileWriteAheadLogManager.java:3249)
> [ignite-core-2.5.7.jar!/:2.5.7]
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> [ignite-core-2.5.7.jar!/:2.5.7]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_201]
> Caused by: java.nio.channels.ClosedChannelException: null
> at sun.nio.ch.FileChannelImpl.ensureOpen(FileChannelImpl.java:110)
> ~[na:1.8.0_201]
> at sun.nio.ch.FileChannelImpl.position(FileChannelImpl.java:253)
> ~[na:1.8.0_201]
> at
> org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.position(RandomAccessFileIO.java:48)
> ~[ignite-core-2.5.7.jar!/:2.5.7]
> at
> org.apache.ignite.internal.processors.cache.persistence.file.FileIODecorator.position(FileIODecorator.java:41)
> ~[ignite-core-2.5.7.jar!/:2.5.7]
> at
> org.apache.ignite.internal.processors.cache.persistence.file.AbstractFileIO.writeFully(AbstractFileIO.java:111)
> ~[ignite-core-2.5.7.jar!/:2.5.7]
> at
> org.apache.ignite.internal.processors.cache.persistence.wal.FileWriteAheadLogManager$WALWriter.writeBuffer(FileWriteAheadLogManager.java:3437)
> [ignite-core-2.5.7.jar!/:2.5.7]
> ... 3 common frames omitted
> {code}
> It appears that there following sequence is possible:
> * Thread A attempts to log a large record which does not fit segment,
> {{addRecord}} fails and the thread A starts segment rollover. I successfully
> runs {{flushOrWait(null)}} and gets de-scheduled before adding switch segment
> record
> * Thread B attempts to log another record, which fits exactly till the end
> of the current segment. The record is added to the buffer
> * Thread A resumes and fails to add the switch segment record. No flush is
> performed and the thread immediately proceeds for wal-writer close
> * WAL writer thread wakes up, sees that there is a CLOSE request, closes the
> file IO and immediately proceeds to write unflushed changes causing the
> exception.
> Unconditional flush after switch segment record write should fix the issue.
> Besides the bug itself, I suggest the following changes to the
> {{FileWriteHandleImpl}} ({{FileWriteAheadLogManager}} in earlier versions):
> * There is an {{fsync(filePtr)}} call inside {{close()}}; however,
> {{fsync()}} checks the {{stop}} flag (which is set inside {{close}}) and
> returns immediately after {{flushOrWait()}} if the flag is set - this is very
> confusing. After all, the {{close()}} itself explicitly calls {{force}} after
> flush
> * There is an ignored IO exception in mmap mode - this should be propagated
> to the failure handler
> * In WAL writer, we check for file CLOSE and then attemp to write to
> (possibly) the same write handle - write should be always before close
> * In WAL writer, there are racy reads of current handle - it would be better
> if we read the current handle once and then operate on it during the whole
> loop iteration
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (IGNITE-12121) Double checkpoint triggering due to incorrect place of update current checkpoint
[ https://issues.apache.org/jira/browse/IGNITE-12121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Govorukhin updated IGNITE-12121: Ignite Flags: (was: Docs Required,Release Notes Required) > Double checkpoint triggering due to incorrect place of update current > checkpoint > > > Key: IGNITE-12121 > URL: https://issues.apache.org/jira/browse/IGNITE-12121 > Project: Ignite > Issue Type: Bug >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > Double checkpoint triggering due to incorrect place of update current > checkpoint. This can lead to two ckeckpoints one by one if checkpoint trigger > was 'too many dirty pages'. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (IGNITE-12121) Double checkpoint triggering due to incorrect place of update current checkpoint
[ https://issues.apache.org/jira/browse/IGNITE-12121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dmitriy Govorukhin updated IGNITE-12121: Fix Version/s: 2.8 > Double checkpoint triggering due to incorrect place of update current > checkpoint > > > Key: IGNITE-12121 > URL: https://issues.apache.org/jira/browse/IGNITE-12121 > Project: Ignite > Issue Type: Bug >Reporter: Anton Kalashnikov >Assignee: Anton Kalashnikov >Priority: Major > Fix For: 2.8 > > Time Spent: 10m > Remaining Estimate: 0h > > Double checkpoint triggering due to incorrect place of update current > checkpoint. This can lead to two ckeckpoints one by one if checkpoint trigger > was 'too many dirty pages'. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (IGNITE-12112) H2TreeIndex should throw CorruptTreeException with cacheId, cacheName and indexName
[
https://issues.apache.org/jira/browse/IGNITE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Mashenkov updated IGNITE-12112:
--
Component/s: sql
> H2TreeIndex should throw CorruptTreeException with cacheId, cacheName and
> indexName
> ---
>
> Key: IGNITE-12112
> URL: https://issues.apache.org/jira/browse/IGNITE-12112
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Denis Chudov
>Assignee: Denis Chudov
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> At the moment we can't define problem cache, If we have many caches in one
> cache group and CorruptTreeException was thrown CorruptTreeException.
> For example:
> {code:java}
> org.h2.message.DbException: General error: "class
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
> Runtime failure on row: Row@346ba4aa[ key: 29003, val:
> com.sbt.acquiring.processing.entities.dictionaries.TurnoverRange_DPL_PROXY
> [idHash=492721050, hash=-616731792, valueStart=2001, isImmutable=false,
> lastChangeDate=1560917668719, name=Ñâûøå 20, valueEnd=null,
> changeState=null, id=29003, pkey=7c0d0d59-f33e-41d7-a5c9-4d8c7e74b976,
> ownerId=acquiring-processing-replication, base=true], ver: GridCacheVersion
> [topVer=172396060, order=1560917613306, nodeOrder=2] ][ Ñâûøå 20, null,
> 2001, TRUE, null, 29003, 7c0d0d59-f33e-41d7-a5c9-4d8c7e74b976 ]"
> [5-195]
> at org.h2.message.DbException.get(DbException.java:168)
> at org.h2.message.DbException.convert(DbException.java:295)
> at
> org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.putx(H2TreeIndex.java:251)
> at
> org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.addToIndex(GridH2Table.java:548)
> at
> org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.update(GridH2Table.java:480)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.store(IgniteH2Indexing.java:709)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:1863)
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:403)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:1402)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1263)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:1625)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:358)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3629)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:2793)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.preloadEntry(GridDhtPartitionDemander.java:902)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.handleSupplyMessage(GridDhtPartitionDemander.java:772)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleSupplyMessage(GridDhtPreloader.java:344)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:418)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:408)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1061)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:101)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1624)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$GridCommunicationMessageSet.unwind(GridIoManager.ja
[jira] [Updated] (IGNITE-12087) Transactional putAll - significant performance drop on big batches of entries.
[
https://issues.apache.org/jira/browse/IGNITE-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12087:
--
Description:
After IGNITE-5227 have been fixed I found significant performance drop in
putAll operation.
Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
After IGNITE-5227 - 130 seconds (~100x slower).
I checked a different batch size:
10_000 - 10 seconds
20_000 - 48 seconds
30_000 - 130 seconds
and I was not able to wait for the result of 100_000 entries.
Reproducer:
{code:java}
public class CheckPutAll extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
cfg.setCacheConfiguration(ccfg);
return cfg;
}
@Test
public void check() throws Exception {
int cnt = 30_000;
Map data = new HashMap<>(U.capacity(cnt));
for (int i = 0; i < cnt; i++)
data.put(i, i);
Ignite node0 = startGrid(0);
IgniteCache cache0 = node0.cache(DEFAULT_CACHE_NAME);
cache0.putAll(data);
}
}{code}
was:
After IGNITE-5227 have been fixed I found significant performance drop in
putAll operation.
Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
After IGNITE-5227 - 130 seconds (~100x slower).
I checked a different batch size:
10_000 - 10 seconds
20_000 - 48 seconds
30_000 - 130 seconds
and I was not able to wait for the result of 100_000 entries.
Reproducer
{code:java}
public class CheckPutAll extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
cfg.setCacheConfiguration(ccfg);
return cfg;
}
@Test
public void check() throws Exception {
int cnt = 30_000;
Map data = new HashMap<>(U.capacity(cnt));
for (int i = 0; i < cnt; i++)
data.put(i, i);
Ignite node0 = startGrid(0);
IgniteCache cache0 = node0.cache(DEFAULT_CACHE_NAME);
cache0.putAll(data);
}
}{code}
> Transactional putAll - significant performance drop on big batches of entries.
> --
>
> Key: IGNITE-12087
> URL: https://issues.apache.org/jira/browse/IGNITE-12087
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Pavel Pereslegin
>Assignee: Eduard Shangareev
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After IGNITE-5227 have been fixed I found significant performance drop in
> putAll operation.
> Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
> After IGNITE-5227 - 130 seconds (~100x slower).
> I checked a different batch size:
> 10_000 - 10 seconds
> 20_000 - 48 seconds
> 30_000 - 130 seconds
> and I was not able to wait for the result of 100_000 entries.
> Reproducer:
> {code:java}
> public class CheckPutAll extends GridCommonAbstractTest {
> @Override protected IgniteConfiguration getConfiguration(String
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
> ccfg.setAtomicityMode(TRANSACTIONAL);
> cfg.setCacheConfiguration(ccfg);
> return cfg;
> }
> @Test
> public void check() throws Exception {
> int cnt = 30_000;
> Map data = new HashMap<>(U.capacity(cnt));
> for (int i = 0; i < cnt; i++)
> data.put(i, i);
> Ignite node0 = startGrid(0);
> IgniteCache cache0 =
> node0.cache(DEFAULT_CACHE_NAME);
> cache0.putAll(data);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (IGNITE-12087) Transactional putAll - significant performance drop on big batches of entries.
[
https://issues.apache.org/jira/browse/IGNITE-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12087:
--
Description:
After IGNITE-5227 have been fixed I found significant performance drop in
putAll operation.
Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
After IGNITE-5227 - 130 seconds (~100x slower).
I checked a different batch size:
10_000 - 10 seconds
20_000 - 48 seconds
30_000 - 130 seconds
and I was not able to wait for the result of 100_000 entries.
Reproducer
{code:java}
public class CheckPutAll extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
cfg.setCacheConfiguration(ccfg);
return cfg;
}
@Test
public void check() throws Exception {
int cnt = 30_000;
Map data = new HashMap<>(U.capacity(cnt));
for (int i = 0; i < cnt; i++)
data.put(i, i);
Ignite node0 = startGrid(0);
IgniteCache cache0 = node0.cache(DEFAULT_CACHE_NAME);
cache0.putAll(data);
}
}{code}
was:
After IGNITE-5227 have been fixed I found significant performance drop in
putAll operation.
Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
After IGNITE-5227 - 130 seconds (~100x slower).
I checked a different batch size:
10_000 - 10 seconds
20_000 - 48 seconds
30_000 - 130 seconds
and I was not able to wait for the result of 100_000 entries.
Reproducer:
{code:java}
public class CheckPutAll extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
cfg.setCacheConfiguration(ccfg);
return cfg;
}
@Test
public void check() throws Exception {
int cnt = 30_000;
Map data = new HashMap<>(U.capacity(cnt));
for (int i = 0; i < cnt; i++)
data.put(i, i);
Ignite node0 = startGrid(0);
IgniteCache cache0 = node0.cache(DEFAULT_CACHE_NAME);
cache0.putAll(data);
}
}{code}
> Transactional putAll - significant performance drop on big batches of entries.
> --
>
> Key: IGNITE-12087
> URL: https://issues.apache.org/jira/browse/IGNITE-12087
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Pavel Pereslegin
>Assignee: Eduard Shangareev
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After IGNITE-5227 have been fixed I found significant performance drop in
> putAll operation.
> Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
> After IGNITE-5227 - 130 seconds (~100x slower).
> I checked a different batch size:
> 10_000 - 10 seconds
> 20_000 - 48 seconds
> 30_000 - 130 seconds
> and I was not able to wait for the result of 100_000 entries.
> Reproducer
> {code:java}
> public class CheckPutAll extends GridCommonAbstractTest {
> @Override protected IgniteConfiguration getConfiguration(String
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
> ccfg.setAtomicityMode(TRANSACTIONAL);
> cfg.setCacheConfiguration(ccfg);
> return cfg;
> }
> @Test
> public void check() throws Exception {
> int cnt = 30_000;
> Map data = new HashMap<>(U.capacity(cnt));
> for (int i = 0; i < cnt; i++)
> data.put(i, i);
> Ignite node0 = startGrid(0);
> IgniteCache cache0 =
> node0.cache(DEFAULT_CACHE_NAME);
> cache0.putAll(data);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (IGNITE-12133) O(log n) partition exchange
[ https://issues.apache.org/jira/browse/IGNITE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922393#comment-16922393 ] Pavel Kovalenko commented on IGNITE-12133: -- [~ivan.glukos] I guess the general solution of such problem will be in introducing Gossip protocol like in Cassandra. In this case, we shouldn't have a pre-determined skip-list topology for the nodes. The probabilistic nature of Gossip also gives us ~ log(N) rounds for the dissemination of such messages. > O(log n) partition exchange > --- > > Key: IGNITE-12133 > URL: https://issues.apache.org/jira/browse/IGNITE-12133 > Project: Ignite > Issue Type: Improvement >Reporter: Moti Nisenson-Ken >Priority: Major > > Currently, partition exchange leverages a ring. This means that > communications is O\(n) in number of nodes. It also means that if > non-coordinator nodes hang it can take much longer to successfully resolve > the topology. > Instead, why not use something like a skip-list where the coordinator is > first. The coordinator can notify the first node at each level of the > skip-list. Each node then notifies all of its "near-neighbours" in the > skip-list, where node B is a near-neighbour of node-A, if max-level(nodeB) <= > max-level(nodeA), and nodeB is the first node at its level when traversing > from nodeA in the direction of nodeB, skipping over nodes C which have > max-level(C) > max-level(A). > 1 > 1 . . .3 > 1 3 . . . 5 > 1 . 2 . 3 . 4 . 5 . 6 > In the above 1 would notify 2 and 3, 3 would notify 4 and 5, 2 -> 4, and 4 -> > 6, and 5 -> 6. > One can achieve better redundancy by having each node traverse in both > directions, and having the coordinator also notify the last node in the list > at each level. This way in the above example if 2 and 3 were both down, 4 > would still get notified from 5 and 6 (in the backwards direction). > > The idea is that each individual node has O(log n) nodes to notify - so the > overall time is reduced. Additionally, we can deal well with at least 1 node > failure - if one includes the option of processing backwards, 2 consecutive > node failures can be handled as well. By taking this kind of an approach, > then the coordinator can basically treat any nodes it didn't receive a > message from as not-connected, and update the topology as well (disconnecting > any nodes that it didn't get a notification from). While there are some edge > cases here (e.g. 2 disconnected nodes, then 1 connected node, then 2 > disconnected nodes - the connected node would be wrongly ejected from the > topology), these would generally be too rare to need explicit handling for. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (IGNITE-12087) Transactional putAll - significant performance drop on big batches of entries.
[
https://issues.apache.org/jira/browse/IGNITE-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12087:
--
Description:
After IGNITE-5227 have been fixed I found significant performance drop in
putAll operation.
Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
After IGNITE-5227 - 130 seconds (~100x slower).
I checked a different batch size:
10_000 - 10 seconds
20_000 - 48 seconds
30_000 - 130 seconds
and I was not able to wait for the result of 100_000 entries.
Reproducer:
{code:java}
public class CheckPutAll extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
cfg.setCacheConfiguration(ccfg);
return cfg;
}
@Test
public void check() throws Exception {
int cnt = 30_000;
Map data = new HashMap<>(U.capacity(cnt));
for (int i = 0; i < cnt; i++)
data.put(i, i);
Ignite node0 = startGrid(0);
IgniteCache cache0 = node0.cache(DEFAULT_CACHE_NAME);
cache0.putAll(data);
}
}{code}
was:
After IGNITE-5227 have been fixed I found significant performance drop in
putAll operation.
Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
After IGNITE-5227 - 130 seconds (~100x slower).
I checked a different batch size:
10_000 - 10 seconds
20_000 - 48 seconds
30_000 - 130 seconds
and I was not able to wait for the result of 100_000 entries.
Reproducer
{code:java}
public class CheckPutAll extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
cfg.setCacheConfiguration(ccfg);
return cfg;
}
@Test
public void check() throws Exception {
int cnt = 30_000;
Map data = new HashMap<>(U.capacity(cnt));
for (int i = 0; i < cnt; i++)
data.put(i, i);
Ignite node0 = startGrid(0);
IgniteCache cache0 = node0.cache(DEFAULT_CACHE_NAME);
cache0.putAll(data);
}
}{code}
> Transactional putAll - significant performance drop on big batches of entries.
> --
>
> Key: IGNITE-12087
> URL: https://issues.apache.org/jira/browse/IGNITE-12087
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Pavel Pereslegin
>Assignee: Eduard Shangareev
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After IGNITE-5227 have been fixed I found significant performance drop in
> putAll operation.
> Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
> After IGNITE-5227 - 130 seconds (~100x slower).
> I checked a different batch size:
> 10_000 - 10 seconds
> 20_000 - 48 seconds
> 30_000 - 130 seconds
> and I was not able to wait for the result of 100_000 entries.
> Reproducer:
> {code:java}
> public class CheckPutAll extends GridCommonAbstractTest {
> @Override protected IgniteConfiguration getConfiguration(String
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
> ccfg.setAtomicityMode(TRANSACTIONAL);
> cfg.setCacheConfiguration(ccfg);
> return cfg;
> }
> @Test
> public void check() throws Exception {
> int cnt = 30_000;
> Map data = new HashMap<>(U.capacity(cnt));
> for (int i = 0; i < cnt; i++)
> data.put(i, i);
> Ignite node0 = startGrid(0);
> IgniteCache cache0 =
> node0.cache(DEFAULT_CACHE_NAME);
> cache0.putAll(data);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (IGNITE-12087) Transactional putAll - significant performance drop on big batches of entries.
[
https://issues.apache.org/jira/browse/IGNITE-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12087:
--
Description:
After IGNITE-5227 have been fixed I found significant performance drop in
putAll operation.
Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
After IGNITE-5227 - 130 seconds (~100x slower).
I checked a different batch size:
10_000 - 10 seconds
20_000 - 48 seconds
30_000 - 130 seconds
and I was not able to wait for the result of 100_000 entries.
Reproducer
{code:java}
public class CheckPutAll extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
cfg.setCacheConfiguration(ccfg);
return cfg;
}
@Test
public void check() throws Exception {
int cnt = 30_000;
Map data = new HashMap<>(U.capacity(cnt));
for (int i = 0; i < cnt; i++)
data.put(i, i);
Ignite node0 = startGrid(0);
IgniteCache cache0 = node0.cache(DEFAULT_CACHE_NAME);
cache0.putAll(data);
}
}{code}
was:
After IGNITE-5227 have been fixed I found significant performance drop in
putAll operation.
Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
After IGNITE-5227 - 130 seconds (~100x slower).
I checked a different batch size:
10_000 - 10 seconds
20_000 - 48 seconds
30_000 - 130 seconds
and I was not able to wait for the result of 100_000 entries.
Reproducer:
{code:java}
public class CheckPutAll extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
cfg.setCacheConfiguration(ccfg);
return cfg;
}
@Test
public void check() throws Exception {
int cnt = 30_000;
Map data = new HashMap<>(U.capacity(cnt));
for (int i = 0; i < cnt; i++)
data.put(i, i);
Ignite node0 = startGrid(0);
IgniteCache cache0 = node0.cache(DEFAULT_CACHE_NAME);
cache0.putAll(data);
}
}{code}
> Transactional putAll - significant performance drop on big batches of entries.
> --
>
> Key: IGNITE-12087
> URL: https://issues.apache.org/jira/browse/IGNITE-12087
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Pavel Pereslegin
>Assignee: Eduard Shangareev
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After IGNITE-5227 have been fixed I found significant performance drop in
> putAll operation.
> Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
> After IGNITE-5227 - 130 seconds (~100x slower).
> I checked a different batch size:
> 10_000 - 10 seconds
> 20_000 - 48 seconds
> 30_000 - 130 seconds
> and I was not able to wait for the result of 100_000 entries.
> Reproducer
> {code:java}
> public class CheckPutAll extends GridCommonAbstractTest {
> @Override protected IgniteConfiguration getConfiguration(String
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
> ccfg.setAtomicityMode(TRANSACTIONAL);
> cfg.setCacheConfiguration(ccfg);
> return cfg;
> }
> @Test
> public void check() throws Exception {
> int cnt = 30_000;
> Map data = new HashMap<>(U.capacity(cnt));
> for (int i = 0; i < cnt; i++)
> data.put(i, i);
> Ignite node0 = startGrid(0);
> IgniteCache cache0 =
> node0.cache(DEFAULT_CACHE_NAME);
> cache0.putAll(data);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (IGNITE-12112) H2TreeIndex should throw CorruptTreeException with cacheId, cacheName and indexName
[
https://issues.apache.org/jira/browse/IGNITE-12112?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922354#comment-16922354
]
Denis Chudov commented on IGNITE-12112:
---
[~amashenkov] fixed.
Also I added restoring of BPlusTree,pageHndWrapper in test.
> H2TreeIndex should throw CorruptTreeException with cacheId, cacheName and
> indexName
> ---
>
> Key: IGNITE-12112
> URL: https://issues.apache.org/jira/browse/IGNITE-12112
> Project: Ignite
> Issue Type: Improvement
>Reporter: Denis Chudov
>Assignee: Denis Chudov
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> At the moment we can't define problem cache, If we have many caches in one
> cache group and CorruptTreeException was thrown CorruptTreeException.
> For example:
> {code:java}
> org.h2.message.DbException: General error: "class
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
> Runtime failure on row: Row@346ba4aa[ key: 29003, val:
> com.sbt.acquiring.processing.entities.dictionaries.TurnoverRange_DPL_PROXY
> [idHash=492721050, hash=-616731792, valueStart=2001, isImmutable=false,
> lastChangeDate=1560917668719, name=Ñâûøå 20, valueEnd=null,
> changeState=null, id=29003, pkey=7c0d0d59-f33e-41d7-a5c9-4d8c7e74b976,
> ownerId=acquiring-processing-replication, base=true], ver: GridCacheVersion
> [topVer=172396060, order=1560917613306, nodeOrder=2] ][ Ñâûøå 20, null,
> 2001, TRUE, null, 29003, 7c0d0d59-f33e-41d7-a5c9-4d8c7e74b976 ]"
> [5-195]
> at org.h2.message.DbException.get(DbException.java:168)
> at org.h2.message.DbException.convert(DbException.java:295)
> at
> org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.putx(H2TreeIndex.java:251)
> at
> org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.addToIndex(GridH2Table.java:548)
> at
> org.apache.ignite.internal.processors.query.h2.opt.GridH2Table.update(GridH2Table.java:480)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.store(IgniteH2Indexing.java:709)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:1863)
> at
> org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:403)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:1402)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1263)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.invoke(GridCacheOffheapManager.java:1625)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:358)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3629)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:2793)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.preloadEntry(GridDhtPartitionDemander.java:902)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionDemander.handleSupplyMessage(GridDhtPartitionDemander.java:772)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPreloader.handleSupplyMessage(GridDhtPreloader.java:344)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:418)
> at
> org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$5.apply(GridCachePartitionExchangeManager.java:408)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1061)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:586)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$700(GridCacheIoManager.java:101)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$OrderedMessageListener.onMessage(GridCacheIoManager.java:1624)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4100(GridIoManager.java:125)
> at
> org.apache.ignite.internal.mana
[jira] [Commented] (IGNITE-12133) O(log n) partition exchange
[ https://issues.apache.org/jira/browse/IGNITE-12133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922349#comment-16922349 ] Andrew Mashenkov commented on IGNITE-12133: --- [~avinogradov], message ordering is important. What if some nodes will got "create cache" and "drop cache" events in different order? > O(log n) partition exchange > --- > > Key: IGNITE-12133 > URL: https://issues.apache.org/jira/browse/IGNITE-12133 > Project: Ignite > Issue Type: Improvement >Reporter: Moti Nisenson-Ken >Priority: Major > > Currently, partition exchange leverages a ring. This means that > communications is O\(n) in number of nodes. It also means that if > non-coordinator nodes hang it can take much longer to successfully resolve > the topology. > Instead, why not use something like a skip-list where the coordinator is > first. The coordinator can notify the first node at each level of the > skip-list. Each node then notifies all of its "near-neighbours" in the > skip-list, where node B is a near-neighbour of node-A, if max-level(nodeB) <= > max-level(nodeA), and nodeB is the first node at its level when traversing > from nodeA in the direction of nodeB, skipping over nodes C which have > max-level(C) > max-level(A). > 1 > 1 . . .3 > 1 3 . . . 5 > 1 . 2 . 3 . 4 . 5 . 6 > In the above 1 would notify 2 and 3, 3 would notify 4 and 5, 2 -> 4, and 4 -> > 6, and 5 -> 6. > One can achieve better redundancy by having each node traverse in both > directions, and having the coordinator also notify the last node in the list > at each level. This way in the above example if 2 and 3 were both down, 4 > would still get notified from 5 and 6 (in the backwards direction). > > The idea is that each individual node has O(log n) nodes to notify - so the > overall time is reduced. Additionally, we can deal well with at least 1 node > failure - if one includes the option of processing backwards, 2 consecutive > node failures can be handled as well. By taking this kind of an approach, > then the coordinator can basically treat any nodes it didn't receive a > message from as not-connected, and update the topology as well (disconnecting > any nodes that it didn't get a notification from). While there are some edge > cases here (e.g. 2 disconnected nodes, then 1 connected node, then 2 > disconnected nodes - the connected node would be wrongly ejected from the > topology), these would generally be too rare to need explicit handling for. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (IGNITE-11829) Distribute joins fail if number of tables > 7
[
https://issues.apache.org/jira/browse/IGNITE-11829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Diana Iakovleva reassigned IGNITE-11829:
Assignee: Diana Iakovleva
> Distribute joins fail if number of tables > 7
> -
>
> Key: IGNITE-11829
> URL: https://issues.apache.org/jira/browse/IGNITE-11829
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Affects Versions: 2.7
>Reporter: Stanislav Lukyanov
>Assignee: Diana Iakovleva
>Priority: Major
> Labels: newbie
>
> Distributed joins fail with ArrayIndexOutOfBounds when the total number of
> tables is > 7.
> Example:
> {code}
> try (Ignite ignite =
> Ignition.start("examples/config/example-ignite.xml");) {
> IgniteCache cache = ignite.createCache("foo");
> cache.query(new SqlFieldsQuery("CREATE TABLE Person(ID INTEGER
> PRIMARY KEY, NAME VARCHAR(100));"));
> cache.query(new SqlFieldsQuery("INSERT INTO Person(ID, NAME)
> VALUES (1, 'Ed'), (2, 'Ann'), (3, 'Emma');"));
> cache.query(new SqlFieldsQuery("SELECT *\n" +
> "FROM PERSON P1\n" +
> "JOIN PERSON P2 ON P1.ID = P2.ID\n" +
> "JOIN PERSON P3 ON P1.ID = P3.ID\n" +
> "JOIN PERSON P4 ON P1.ID = P4.ID\n" +
> "JOIN PERSON P5 ON P1.ID = P5.ID\n" +
> "JOIN PERSON P6 ON P1.ID = P6.ID\n" +
> "JOIN PERSON P7 ON P1.ID = P7.ID\n" +
> "JOIN PERSON P8 ON P1.ID =
> P8.ID").setDistributedJoins(true).setEnforceJoinOrder(false));
> }
> {code}
> throws
> {code}
> Exception in thread "main" javax.cache.CacheException: General error:
> "java.lang.ArrayIndexOutOfBoundsException" [5-197]
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:832)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:765)
> at
> org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:403)
> at
> org.apache.ignite.examples.ExampleNodeStartup.main(ExampleNodeStartup.java:60)
> Caused by: class
> org.apache.ignite.internal.processors.query.IgniteSQLException: General
> error: "java.lang.ArrayIndexOutOfBoundsException" [5-197]
> at
> org.apache.ignite.internal.processors.query.h2.QueryParser.parseH2(QueryParser.java:454)
> at
> org.apache.ignite.internal.processors.query.h2.QueryParser.parse0(QueryParser.java:156)
> at
> org.apache.ignite.internal.processors.query.h2.QueryParser.parse(QueryParser.java:121)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1191)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2261)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2257)
> at
> org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:53)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2767)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$1(GridQueryProcessor.java:2277)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2297)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2250)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2177)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:817)
> ... 3 more
> Caused by: org.h2.jdbc.JdbcSQLException: General error:
> "java.lang.ArrayIndexOutOfBoundsException" [5-197]
> at org.h2.message.DbException.getJdbcSQLException(DbException.java:357)
> at org.h2.message.DbException.get(DbException.java:168)
> at org.h2.message.DbException.convert(DbException.java:307)
> at org.h2.message.DbException.toSQLException(DbException.java:280)
> at org.h2.message.TraceObject.logAndConvert(TraceObject.java:357)
> at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:308)
> at
> org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.prepare(GridSqlQuerySplitter.java:1770)
> at
> org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.split0(GridSqlQuerySplitter.java:299)
> at
> org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.split(Gri
[jira] [Updated] (IGNITE-12141) Ignite Spark Integration not working with Schema Name
[ https://issues.apache.org/jira/browse/IGNITE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj G T updated IGNITE-12141: --- Ignite Flags: Docs Required (was: Docs Required,Release Notes Required) > Ignite Spark Integration not working with Schema Name > - > > Key: IGNITE-12141 > URL: https://issues.apache.org/jira/browse/IGNITE-12141 > Project: Ignite > Issue Type: Bug > Components: spark >Affects Versions: 2.7.5 >Reporter: Manoj G T >Priority: Critical > > Users can't able to specify schema when trying to persist Spark DF to Ignite. > > [https://stackoverflow.com/questions/57782033/apache-ignite-spark-integration-not-working-with-schema-name] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (IGNITE-12141) Ignite Spark Integration not working with Schema Name
[ https://issues.apache.org/jira/browse/IGNITE-12141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Manoj G T updated IGNITE-12141: --- Fix Version/s: 2.7.6 > Ignite Spark Integration not working with Schema Name > - > > Key: IGNITE-12141 > URL: https://issues.apache.org/jira/browse/IGNITE-12141 > Project: Ignite > Issue Type: Bug > Components: spark >Affects Versions: 2.7.5 >Reporter: Manoj G T >Priority: Critical > Fix For: 2.7.6 > > > Users can't able to specify schema when trying to persist Spark DF to Ignite. > > [https://stackoverflow.com/questions/57782033/apache-ignite-spark-integration-not-working-with-schema-name] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IGNITE-12141) Ignite Spark Integration not working with Schema Name
Manoj G T created IGNITE-12141: -- Summary: Ignite Spark Integration not working with Schema Name Key: IGNITE-12141 URL: https://issues.apache.org/jira/browse/IGNITE-12141 Project: Ignite Issue Type: Bug Components: spark Affects Versions: 2.7.5 Reporter: Manoj G T Users can't able to specify schema when trying to persist Spark DF to Ignite. [https://stackoverflow.com/questions/57782033/apache-ignite-spark-integration-not-working-with-schema-name] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (IGNITE-12087) Transactional putAll - significant performance drop on big batches of entries.
[
https://issues.apache.org/jira/browse/IGNITE-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12087:
--
Description:
After IGNITE-5227 have been fixed I found significant performance drop in
putAll operation.
Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
After IGNITE-5227 - 130 seconds (~100x slower).
I checked a different batch size:
10_000 - 10 seconds
20_000 - 48 seconds
30_000 - 130 seconds
and I was not able to wait for the result of 100_000 entries.
Reproducer:
{code:java}
public class CheckPutAll extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
cfg.setCacheConfiguration(ccfg);
return cfg;
}
@Test
public void check() throws Exception {
int cnt = 30_000;
Map data = new HashMap<>(U.capacity(cnt));
for (int i = 0; i < cnt; i++)
data.put(i, i);
Ignite node0 = startGrid(0);
IgniteCache cache0 = node0.cache(DEFAULT_CACHE_NAME);
cache0.putAll(data);
}
}{code}
was:
After IGNITE-5227 have been fixed I found significant performance drop in
putAll operation.
Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
After IGNITE-5227 - 130 seconds (~100x slower).
I checked a different batch size:
10_000 - 10 seconds
20_000 - 48 seconds
30_000 - 130 seconds
and I was not able to wait for the result of 100_000 entries.
Reproducer
{code:java}
public class CheckPutAll extends GridCommonAbstractTest {
@Override protected IgniteConfiguration getConfiguration(String
igniteInstanceName) throws Exception {
IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
ccfg.setAtomicityMode(TRANSACTIONAL);
cfg.setCacheConfiguration(ccfg);
return cfg;
}
@Test
public void check() throws Exception {
int cnt = 30_000;
Map data = new HashMap<>(U.capacity(cnt));
for (int i = 0; i < cnt; i++)
data.put(i, i);
Ignite node0 = startGrid(0);
IgniteCache cache0 = node0.cache(DEFAULT_CACHE_NAME);
cache0.putAll(data);
}
}{code}
> Transactional putAll - significant performance drop on big batches of entries.
> --
>
> Key: IGNITE-12087
> URL: https://issues.apache.org/jira/browse/IGNITE-12087
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Pavel Pereslegin
>Assignee: Eduard Shangareev
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After IGNITE-5227 have been fixed I found significant performance drop in
> putAll operation.
> Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
> After IGNITE-5227 - 130 seconds (~100x slower).
> I checked a different batch size:
> 10_000 - 10 seconds
> 20_000 - 48 seconds
> 30_000 - 130 seconds
> and I was not able to wait for the result of 100_000 entries.
> Reproducer:
> {code:java}
> public class CheckPutAll extends GridCommonAbstractTest {
> @Override protected IgniteConfiguration getConfiguration(String
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
> ccfg.setAtomicityMode(TRANSACTIONAL);
> cfg.setCacheConfiguration(ccfg);
> return cfg;
> }
> @Test
> public void check() throws Exception {
> int cnt = 30_000;
> Map data = new HashMap<>(U.capacity(cnt));
> for (int i = 0; i < cnt; i++)
> data.put(i, i);
> Ignite node0 = startGrid(0);
> IgniteCache cache0 =
> node0.cache(DEFAULT_CACHE_NAME);
> cache0.putAll(data);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (IGNITE-12087) Transactional putAll - significant performance drop on big batches of entries.
[
https://issues.apache.org/jira/browse/IGNITE-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12087:
--
Fix Version/s: 2.8
> Transactional putAll - significant performance drop on big batches of entries.
> --
>
> Key: IGNITE-12087
> URL: https://issues.apache.org/jira/browse/IGNITE-12087
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Pavel Pereslegin
>Assignee: Eduard Shangareev
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After IGNITE-5227 have been fixed I found significant performance drop in
> putAll operation.
> Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
> After IGNITE-5227 - 130 seconds (~100x slower).
> I checked a different batch size:
> 10_000 - 10 seconds
> 20_000 - 48 seconds
> 30_000 - 130 seconds
> and I was not able to wait for the result of 100_000 entries.
> Reproducer
> {code:java}
> public class CheckPutAll extends GridCommonAbstractTest {
> @Override protected IgniteConfiguration getConfiguration(String
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
> ccfg.setAtomicityMode(TRANSACTIONAL);
> cfg.setCacheConfiguration(ccfg);
> return cfg;
> }
> @Test
> public void check() throws Exception {
> int cnt = 30_000;
> Map data = new HashMap<>(U.capacity(cnt));
> for (int i = 0; i < cnt; i++)
> data.put(i, i);
> Ignite node0 = startGrid(0);
> IgniteCache cache0 =
> node0.cache(DEFAULT_CACHE_NAME);
> cache0.putAll(data);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (IGNITE-12087) Transactional putAll - significant performance drop on big batches of entries.
[
https://issues.apache.org/jira/browse/IGNITE-12087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Goncharuk updated IGNITE-12087:
--
Fix Version/s: (was: 2.8)
> Transactional putAll - significant performance drop on big batches of entries.
> --
>
> Key: IGNITE-12087
> URL: https://issues.apache.org/jira/browse/IGNITE-12087
> Project: Ignite
> Issue Type: Bug
> Components: cache
>Reporter: Pavel Pereslegin
>Assignee: Eduard Shangareev
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> After IGNITE-5227 have been fixed I found significant performance drop in
> putAll operation.
> Insertion of 30_000 entries before IGNITE-5227 took ~1 second.
> After IGNITE-5227 - 130 seconds (~100x slower).
> I checked a different batch size:
> 10_000 - 10 seconds
> 20_000 - 48 seconds
> 30_000 - 130 seconds
> and I was not able to wait for the result of 100_000 entries.
> Reproducer
> {code:java}
> public class CheckPutAll extends GridCommonAbstractTest {
> @Override protected IgniteConfiguration getConfiguration(String
> igniteInstanceName) throws Exception {
> IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName);
> CacheConfiguration ccfg = new CacheConfiguration(DEFAULT_CACHE_NAME);
> ccfg.setAtomicityMode(TRANSACTIONAL);
> cfg.setCacheConfiguration(ccfg);
> return cfg;
> }
> @Test
> public void check() throws Exception {
> int cnt = 30_000;
> Map data = new HashMap<>(U.capacity(cnt));
> for (int i = 0; i < cnt; i++)
> data.put(i, i);
> Ignite node0 = startGrid(0);
> IgniteCache cache0 =
> node0.cache(DEFAULT_CACHE_NAME);
> cache0.putAll(data);
> }
> }{code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (IGNITE-12124) Stopping the cache does not wait for expiration process, which may be started and may lead to errors
[
https://issues.apache.org/jira/browse/IGNITE-12124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-12124:
-
Description:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that unregistering of
{{GridCacheTtlManager}} (see {{GridCacheSharedTtlCleanupManager#unregister}}
does not wait for the finish of expiration (in that particular case,
{{GridCacheContext}} is already cleaned up).
So, unregistering of {{GridCacheTtlManager}}, caused by cache stopping, must
wait for expiration if it is running for the cache that stops. On the other
hand, it does not seem correct to wait for expiration under the
{{checkpointReadLock}} see
{{GridCacheProcessor#processCacheStopRequestOnExchangeDone}}:
{code:java}
private void processCacheStopRequestOnExchangeDone(ExchangeActions exchActions)
{
...
doInParallel(
parallelismLvl,
sharedCtx.kernalContext().getSystemExecutorService(),
cachesToStop.entrySet(),
cachesToStopByGrp -> {
...
for (ExchangeActions.CacheActionData action :
cachesToStopByGrp.getValue()) {
...
sharedCtx.database().checkpointReadLock();
try {
prepareCacheStop(...); <---unregistering of
GridCacheTtlManager is performed here
}
finally {
sharedCtx.database().checkpointReadUnlock();
}
}
...
}
}
{code}
was:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that unregistering of
{{GridCacheTtlManager}} (see {{GridCacheSharedTtlCleanupManager#unregister}}
does not wait for the finish of expiration (in that particular case,
{{GridCacheContext}} is already cleaned up).
So, unregistering of {{GridCacheTtlManager}}, cause
[jira] [Updated] (IGNITE-12124) Stopping the cache does not wait for expiration process, which may be started and may lead to errors
[
https://issues.apache.org/jira/browse/IGNITE-12124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-12124:
-
Description:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that unregistering of
{{GridCacheTtlManager}} (see {{GridCacheSharedTtlCleanupManager#unregister}}
does not wait for the finish of expiration (in that particular case,
{{GridCacheContext}} is already cleaned up).
So, unregistering of {{GridCacheTtlManager}}, caused by cache stopping, must
wait for expiration if it is running for the cache that stops. On the other
hand, it does not seem correct to wait for expiration under the
{{checkpointReadLock}} see
{{GridCacheProcessor#processCacheStopRequestOnExchangeDone}}:
{code:java}
private void processCacheStopRequestOnExchangeDone(ExchangeActions exchActions)
{
...
doInParallel(
parallelismLvl,
sharedCtx.kernalContext().getSystemExecutorService(),
cachesToStop.entrySet(),
cachesToStopByGrp -> {
...
for (ExchangeActions.CacheActionData action :
cachesToStopByGrp.getValue()) {
...
sharedCtx.database().checkpointReadLock();
try {
prepareCacheStop(action.request().cacheName(),
action.request().destroy()); <---unregistering of GridCacheTtlManager is
performed here
}
finally {
sharedCtx.database().checkpointReadUnlock();
}
}
...
}
}
{code}
was:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that unregistering of
{{GridCacheTtlManager}} (see {{GridCacheSharedTtlCleanupManager#unregister}}
does not wait for the finish of expiration (in that particular case,
{{GridCacheContext}} is already cleaned up).
[jira] [Updated] (IGNITE-12124) Stopping the cache does not wait for expiration process, which may be started and may lead to errors
[
https://issues.apache.org/jira/browse/IGNITE-12124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-12124:
-
Description:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that unregistering of
{{GridCacheTtlManager}} (see {{GridCacheSharedTtlCleanupManager#unregister}}
does not wait for the finish of expiration (in that particular case,
{{GridCacheContext}} is already cleaned up).
So, unregistering of {{GridCacheTtlManager}}, caused by cache stopping, must
wait for expiration if it is running for the cache that stops. On the other
hand, it does not seem correct to wait for expiration under the
{{checkpointReadLock}} see
{{GridCacheProcessor#processCacheStopRequestOnExchangeDone}}:
{code:java}
private void processCacheStopRequestOnExchangeDone(ExchangeActions
exchActions) {
...
try {
doInParallel(
parallelismLvl,
sharedCtx.kernalContext().getSystemExecutorService(),
cachesToStop.entrySet(),
cachesToStopByGrp -> {
...
for (ExchangeActions.CacheActionData action:
cachesToStopByGrp.getValue()) {
...
sharedCtx.database().checkpointReadLock();
try {
prepareCacheStop(action.request().cacheName(), action.request().destroy());
<--- unregistering of GridCacheTtlManager is performed here
}
finally {
sharedCtx.database().checkpointReadUnlock();
}
}
...
}
{code}
was:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for th
[jira] [Updated] (IGNITE-12124) Stopping the cache does not wait for expiration process, which may be started and may lead to errors
[
https://issues.apache.org/jira/browse/IGNITE-12124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-12124:
-
Description:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that unregistering of
{{GridCacheTtlManager}} (see {{GridCacheSharedTtlCleanupManager#unregister}}
does not wait for the finish of expiration (in that particular case,
{{GridCacheContext}} is already cleaned up).
So, it seems to me, unregistering of {{GridCacheTtlManager, caused by cache
stopping, must wait for expiration if it is running for the cache to be
stopped.}}
was:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that unregistering of
{{GridCacheTtlManager}} (see {{GridCacheSharedTtlCleanupManager#unregister}}
does not wait for the finish of expiration (in that particular case,
{{GridCacheContext}} is already cleaned up).
So, it seems to me, unregistering of {{GridCacheTtlManager, caused by cache
stopping, must wait for expiration }}
> Stopping the cache does not wait for expiration process, which may be started
> and may lead to errors
>
>
> Key: IGNITE-12124
> URL: https://issues.apache.org/jira/browse/IGNITE-12124
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Stopping a cache with configured TTL may lead to errors. For instance,
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:170
[jira] [Updated] (IGNITE-12124) Stopping the cache does not wait for expiration process, which may be started and may lead to errors
[
https://issues.apache.org/jira/browse/IGNITE-12124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-12124:
-
Description:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that unregistering of
{{GridCacheTtlManager}} (see {{GridCacheSharedTtlCleanupManager#unregister}}
does not wait for the finish of expiration (in that particular case,
{{GridCacheContext}} is already cleaned up).
So, it seems to me, unregistering of {{GridCacheTtlManager, caused by cache
stopping, must wait for expiration }}
was:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that unregistering of
{{GridCacheTtlManager}} (see {{GridCacheSharedTtlCleanupManager#unregister}}
does not wait for the finish of expiration (in that particular case,
{{GridCacheContext}} is already cleaned up).
> Stopping the cache does not wait for expiration process, which may be started
> and may lead to errors
>
>
> Key: IGNITE-12124
> URL: https://issues.apache.org/jira/browse/IGNITE-12124
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Stopping a cache with configured TTL may lead to errors. For instance,
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
> at
> org.apache.ignite.internal.
[jira] [Updated] (IGNITE-12124) Stopping the cache does not wait for expiration process, which may be started and may lead to errors
[
https://issues.apache.org/jira/browse/IGNITE-12124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-12124:
-
Description:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that unregistering of
{{GridCacheTtlManager}} (see {{GridCacheSharedTtlCleanupManager#unregister}}
does not wait for the finish of expiration (in that particular case,
{{GridCacheContext}} is already cleaned up).
was:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that {{
> Stopping the cache does not wait for expiration process, which may be started
> and may lead to errors
>
>
> Key: IGNITE-12124
> URL: https://issues.apache.org/jira/browse/IGNITE-12124
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Stopping a cache with configured TTL may lead to errors. For instance,
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
> at
> org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
> at
> org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
> at
> org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
> at
> org.apache.ignite.in
[jira] [Updated] (IGNITE-12124) Stopping the cache does not wait for expiration process, which may be started and may lead to errors
[
https://issues.apache.org/jira/browse/IGNITE-12124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-12124:
-
Description:
Stopping a cache with configured TTL may lead to errors. For instance,
{noformat}
java.lang.NullPointerException
at
org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
at
org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
at
org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
at
org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at java.lang.Thread.run(Thread.java:748){noformat}
The obvious reason for this {{NullPointerException}} is that {{
> Stopping the cache does not wait for expiration process, which may be started
> and may lead to errors
>
>
> Key: IGNITE-12124
> URL: https://issues.apache.org/jira/browse/IGNITE-12124
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.7
>Reporter: Vyacheslav Koptilin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Fix For: 2.8
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Stopping a cache with configured TTL may lead to errors. For instance,
> {noformat}
> java.lang.NullPointerException
> at
> org.apache.ignite.internal.processors.cache.GridCacheContext.onDeferredDelete(GridCacheContext.java:1702)
> at
> org.apache.ignite.internal.processors.cache.GridCacheMapEntry.onTtlExpired(GridCacheMapEntry.java:4040)
> at
> org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:75)
> at
> org.apache.ignite.internal.processors.cache.GridCacheTtlManager$1.applyx(GridCacheTtlManager.java:66)
> at
> org.apache.ignite.internal.util.lang.IgniteInClosure2X.apply(IgniteInClosure2X.java:37)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2501)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2427)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:989)
> at
> org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:233)
> at
> org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:150)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
> at java.lang.Thread.run(Thread.java:748){noformat}
> The obvious reason for this {{NullPointerException}} is that {{
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (IGNITE-10223) Affinity methods should return List instead of Collection
[
https://issues.apache.org/jira/browse/IGNITE-10223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16922270#comment-16922270
]
Ignite TC Bot commented on IGNITE-10223:
{panel:title=Branch: [pull/6837/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4564401&buildTypeId=IgniteTests24Java8_RunAll]
> Affinity methods should return List instead of Collection
> -
>
> Key: IGNITE-10223
> URL: https://issues.apache.org/jira/browse/IGNITE-10223
> Project: Ignite
> Issue Type: Bug
>Reporter: Stanislav Lukyanov
>Assignee: Diana Iakovleva
>Priority: Minor
> Labels: newbie
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Methods of class Affinity should return List instead of Collection when order
> of nodes matters.
> The following methods are supposed to always return a collection with primary
> node preceding backups:
> Affinity::mapPartitionToPrimaryAndBackups
> Affinity::mapKeyToPrimaryAndBackups
> The return type for both methods is Collection. The Javadocs
> say: "Note that primary node is always first in the returned collection".
> In general, the notion of "order" and "first element" is not defined for a
> Collection (we could play with iterators, but its awkward from both code and
> logic perspective). It makes more sense for this methods to return List
> instead.
> The task is to introduce List equivalents of these methods. It would also be
> nice to check if other APIs have similar issues.
> The tricky part is compatibility.
> Changing return type from Collection to List is source compatible (meaning
> the old client code will compile with the new method), but not binary
> compatible (meaning the already compiled client code will break because the
> signature has changed).
> A compatible solution would be to
> - deprecate the existing methods
> - add methods with different names, like mapPartitionToPrimaryAndBackups*List*
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
