[jira] [Created] (IGNITE-14507) Un-deprecate IGNITE_BINARY_SORT_OBJECT_FIELDS
Stanislav Lukyanov created IGNITE-14507: --- Summary: Un-deprecate IGNITE_BINARY_SORT_OBJECT_FIELDS Key: IGNITE-14507 URL: https://issues.apache.org/jira/browse/IGNITE-14507 Project: Ignite Issue Type: Improvement Components: binary Reporter: Stanislav Lukyanov IGNITE_BINARY_SORT_OBJECT_FIELDS should not be deprecated - on the contrary, every new environment better have this option. We need to advertise it more, so we need to remove the deprecation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13985) C++: Continuous Queries should fail to register if remote filter is in C++ and there are non-C++ affinity nodes
Stanislav Lukyanov created IGNITE-13985: --- Summary: C++: Continuous Queries should fail to register if remote filter is in C++ and there are non-C++ affinity nodes Key: IGNITE-13985 URL: https://issues.apache.org/jira/browse/IGNITE-13985 Project: Ignite Issue Type: Improvement Reporter: Stanislav Lukyanov See https://issues.apache.org/jira/browse/IGNITE-6800. Currently, if there is a pure Java node in the cluster and a CQ is being registered from C++, the C++ remote filter will just be ignored on the pure Java node leading to unexpected CQ notifications. Need to add a validation to prevent incorrect results. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13975) Validate that tables beind joined have the same partition distribution
Stanislav Lukyanov created IGNITE-13975: --- Summary: Validate that tables beind joined have the same partition distribution Key: IGNITE-13975 URL: https://issues.apache.org/jira/browse/IGNITE-13975 Project: Ignite Issue Type: Improvement Components: sql Reporter: Stanislav Lukyanov To be able to join two PARTITIONED tables one needs to perform the JOIN on an affinity key, and also to make sure that the tables have the same partition distribution - otherwise, the result will be incorrect. Validation of whether the affinity key is being used for JOIN is to be done in https://issues.apache.org/jira/browse/IGNITE-13019. The goal of this ticket is to add validation of the partition distribution. Whenever a JOIN of PARTITIONED tables is being done, the SQL engine needs to ensure that * The tables use the same affinity function * The tables have the same number of partitions * The tables have the same node filters Same is for when REPLICATED tables are used and `treatReplicatedAsPartitioned=true` (see https://issues.apache.org/jira/browse/IGNITE-8732). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13777) idle_verify should report real size of the partitions
Stanislav Lukyanov created IGNITE-13777: --- Summary: idle_verify should report real size of the partitions Key: IGNITE-13777 URL: https://issues.apache.org/jira/browse/IGNITE-13777 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Currently, idle_verify checks the content of partitions (through hash) and the partition size that is stored in the partition meta. It will be better if idle_verify also counts the entries inside the partition and returns both the size from the meta AND the real size. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13341) OOME is ignored if thrown in a client connector thread
Stanislav Lukyanov created IGNITE-13341: --- Summary: OOME is ignored if thrown in a client connector thread Key: IGNITE-13341 URL: https://issues.apache.org/jira/browse/IGNITE-13341 Project: Ignite Issue Type: Bug Components: thin client Reporter: Stanislav Lukyanov If a thin client operation causes OOME then the thread will die and an error will be logged, but that's all. The clients are left hanging and waiting for reply until timeout. Correct behavior in this case would be to trigger Failure Handler. Example of the error: {code} Exception in thread "client-connector-#69%xxx%" java.lang.OutOfMemoryError: Java heap space Aug 08, 2020 7:08:43 AM org.apache.ignite.logger.java.JavaLogger error SEVERE: Runtime error caught during grid runnable execution: GridWorker [name=message-received-notify, igniteInstanceName=xxx, finished=false, heartbeatTs=1596859722580, hashCode=1209004925, interrupted=false, runner=client-connector-#66%xxx%] java.lang.OutOfMemoryError: Java heap space at org.apache.ignite.internal.binary.streams.BinaryMemoryAllocatorChunk.reallocate(BinaryMemoryAllocatorChunk.java:68) at org.apache.ignite.internal.binary.streams.BinaryHeapOutputStream.ensureCapacity(BinaryHeapOutputStream.java:64) at org.apache.ignite.internal.binary.streams.BinaryAbstractOutputStream.unsafeEnsure(BinaryAbstractOutputStream.java:261) at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteBinaryObject(BinaryWriterExImpl.java:970) at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:756) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:231) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:164) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:151) at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteObject(BinaryWriterExImpl.java:523) at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObject(BinaryWriterExImpl.java:1502) at org.apache.ignite.internal.processors.platform.client.ClientObjectResponse.encode(ClientObjectResponse.java:43) at org.apache.ignite.internal.processors.platform.client.ClientMessageParser.encode(ClientMessageParser.java:459) at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:203) at org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:49) at org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:278) at org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:108) at org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:135) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119) at org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:69) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13340) copyOnRead=false doesn't work for byte array values
Stanislav Lukyanov created IGNITE-13340: --- Summary: copyOnRead=false doesn't work for byte array values Key: IGNITE-13340 URL: https://issues.apache.org/jira/browse/IGNITE-13340 Project: Ignite Issue Type: Bug Components: cache Reporter: Stanislav Lukyanov If near cache is used and copyOnRead=false is set then the Java object should only be created on the first read, and all subsequent reads must use the same copy. However, when byte array value is used (e.g. `put(42, new byte[100]`) then the copying is done on every read. It seems that the reason is that byte array values have their own implementation of `CacheObject` - `CacheObjectByteArrayImpl`. That implementation doesn't use `CacheObjectValueContext::storeValue` which controls whether copying needs to be done; CacheObjectImpl uses it. Need to correctly implement copyOnRead=false for caches. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-11977) Data streamer pool MXBean is registered as ThreadPoolMXBean instead of StripedExecutorMXBean
Stanislav Lukyanov created IGNITE-11977: --- Summary: Data streamer pool MXBean is registered as ThreadPoolMXBean instead of StripedExecutorMXBean Key: IGNITE-11977 URL: https://issues.apache.org/jira/browse/IGNITE-11977 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Data streamer pool is registered with a ThreadPoolMXBean while it is actually a StripedExecutor and can use a StripedExecutorMXBean. Need to change the registration in the IgniteKernal code. It should be registered the same way as the striped executor pool. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (IGNITE-11976) @SpringResource is silently ignored if no Spring context is provided
Stanislav Lukyanov created IGNITE-11976: --- Summary: @SpringResource is silently ignored if no Spring context is provided Key: IGNITE-11976 URL: https://issues.apache.org/jira/browse/IGNITE-11976 Project: Ignite Issue Type: Improvement Components: spring Affects Versions: 2.7 Reporter: Stanislav Lukyanov @SpringResource annotation is silently ignored and the annotated field is null if Spring context. For @SpringResource to work the node needs to be started with IgniteSpring::start instead of Ignition::start, but the user may not know that. Need to add a warning for this. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (IGNITE-11968) FailureProcessor reports tcp-comm-worker as blocked when resolving communication errors in ZK
Stanislav Lukyanov created IGNITE-11968: --- Summary: FailureProcessor reports tcp-comm-worker as blocked when resolving communication errors in ZK Key: IGNITE-11968 URL: https://issues.apache.org/jira/browse/IGNITE-11968 Project: Ignite Issue Type: Improvement Affects Versions: 2.7 Reporter: Stanislav Lukyanov When multiple clients fail (stopped) at the same time and ZK discovery is used FailureProcessor treats tcp-comm-worker as blocked with the following trace: {code} Thread [name="tcp-comm-worker-#1", id=122, state=WAITING, blockCnt=1465, waitCnt=4557] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304) at o.a.i.i.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178) at o.a.i.i.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141) at o.a.i.spi.discovery.zk.internal.ZookeeperDiscoveryImpl.resolveCommunicationError(ZookeeperDiscoveryImpl.java:359) at o.a.i.spi.discovery.zk.ZookeeperDiscoverySpi.resolveCommunicationFailure(ZookeeperDiscoverySpi.java:259) at o.a.i.i.managers.discovery.GridDiscoveryManager.resolveCommunicationError(GridDiscoveryManager.java:2549) at o.a.i.i.managers.GridManagerAdapter$1.resolveCommunicationFailure(GridManagerAdapter.java:626) at o.a.i.spi.communication.tcp.TcpCommunicationSpi.processClientCreationError(TcpCommunicationSpi.java:3530) at o.a.i.spi.communication.tcp.TcpCommunicationSpi.createTcpClient(TcpCommunicationSpi.java:3483) at o.a.i.spi.communication.tcp.TcpCommunicationSpi.createNioClient(TcpCommunicationSpi.java:2987) at o.a.i.spi.communication.tcp.TcpCommunicationSpi.reserveClient(TcpCommunicationSpi.java:2870) at o.a.i.spi.communication.tcp.TcpCommunicationSpi.access$6000(TcpCommunicationSpi.java:271) at o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.processDisconnect(TcpCommunicationSpi.java:4489) at o.a.i.spi.communication.tcp.TcpCommunicationSpi$CommunicationWorker.body(TcpCommunicationSpi.java:4294) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120) at o.a.i.spi.communication.tcp.TcpCommunicationSpi$5.body(TcpCommunicationSpi.java:2237) at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11894) Add fetchSize to JDBC cache stores
Stanislav Lukyanov created IGNITE-11894: --- Summary: Add fetchSize to JDBC cache stores Key: IGNITE-11894 URL: https://issues.apache.org/jira/browse/IGNITE-11894 Project: Ignite Issue Type: Improvement Components: cache Reporter: Stanislav Lukyanov JDBC's PreparedStatement accepts a fetchSize parameter which defines how many rows will be loaded from the DB at a time. Currently the only way to change that is by specifying it in a customer implementation of the JdbcDialect::fetchSize method (and even then it seems not be be used in some cases). Would be good to have a fetchSize property in all of JDBC-based cache stores. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11836) keyFields must be empty if keyType is an SQL type
Stanislav Lukyanov created IGNITE-11836: --- Summary: keyFields must be empty if keyType is an SQL type Key: IGNITE-11836 URL: https://issues.apache.org/jira/browse/IGNITE-11836 Project: Ignite Issue Type: Improvement Components: sql Reporter: Stanislav Lukyanov Currently if QueryEntity.keyType is an SQL type (e.g. String or Integer) then keyFields property is simply ignored. This creates room for a misconfiguration. Let's add a check that keyFields is null or empty if keyType is an SQL type. Probably the same should be done for valueType and valueFields. Also, fields property should be handled somehow. An obvious case is that fields should be null or empty when both key and value are SQL types. There is probably more invariants to enforce though. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11829) Distribute joins fail if number of tables > 7
Stanislav Lukyanov created IGNITE-11829: --- Summary: Distribute joins fail if number of tables > 7 Key: IGNITE-11829 URL: https://issues.apache.org/jira/browse/IGNITE-11829 Project: Ignite Issue Type: Improvement Components: sql Affects Versions: 2.7 Reporter: Stanislav Lukyanov Distributed joins fail with ArrayIndexOutOfBounds when the total number of tables is > 7. Example: {code} try (Ignite ignite = Ignition.start("examples/config/example-ignite.xml");) { IgniteCache cache = ignite.createCache("foo"); cache.query(new SqlFieldsQuery("CREATE TABLE Person(ID INTEGER PRIMARY KEY, NAME VARCHAR(100));")); cache.query(new SqlFieldsQuery("INSERT INTO Person(ID, NAME) VALUES (1, 'Ed'), (2, 'Ann'), (3, 'Emma');")); cache.query(new SqlFieldsQuery("SELECT *\n" + "FROM PERSON P1\n" + "JOIN PERSON P2 ON P1.ID = P2.ID\n" + "JOIN PERSON P3 ON P1.ID = P3.ID\n" + "JOIN PERSON P4 ON P1.ID = P4.ID\n" + "JOIN PERSON P5 ON P1.ID = P5.ID\n" + "JOIN PERSON P6 ON P1.ID = P6.ID\n" + "JOIN PERSON P7 ON P1.ID = P7.ID\n" + "JOIN PERSON P8 ON P1.ID = P8.ID").setDistributedJoins(true).setEnforceJoinOrder(false)); } {code} throws {code} Exception in thread "main" javax.cache.CacheException: General error: "java.lang.ArrayIndexOutOfBoundsException" [5-197] at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:832) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:765) at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:403) at org.apache.ignite.examples.ExampleNodeStartup.main(ExampleNodeStartup.java:60) Caused by: class org.apache.ignite.internal.processors.query.IgniteSQLException: General error: "java.lang.ArrayIndexOutOfBoundsException" [5-197] at org.apache.ignite.internal.processors.query.h2.QueryParser.parseH2(QueryParser.java:454) at org.apache.ignite.internal.processors.query.h2.QueryParser.parse0(QueryParser.java:156) at org.apache.ignite.internal.processors.query.h2.QueryParser.parse(QueryParser.java:121) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.querySqlFields(IgniteH2Indexing.java:1191) at org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2261) at org.apache.ignite.internal.processors.query.GridQueryProcessor$3.applyx(GridQueryProcessor.java:2257) at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:53) at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2767) at org.apache.ignite.internal.processors.query.GridQueryProcessor.lambda$querySqlFields$1(GridQueryProcessor.java:2277) at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuerySafe(GridQueryProcessor.java:2297) at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2250) at org.apache.ignite.internal.processors.query.GridQueryProcessor.querySqlFields(GridQueryProcessor.java:2177) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:817) ... 3 more Caused by: org.h2.jdbc.JdbcSQLException: General error: "java.lang.ArrayIndexOutOfBoundsException" [5-197] at org.h2.message.DbException.getJdbcSQLException(DbException.java:357) at org.h2.message.DbException.get(DbException.java:168) at org.h2.message.DbException.convert(DbException.java:307) at org.h2.message.DbException.toSQLException(DbException.java:280) at org.h2.message.TraceObject.logAndConvert(TraceObject.java:357) at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:308) at org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.prepare(GridSqlQuerySplitter.java:1770) at org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.split0(GridSqlQuerySplitter.java:299) at org.apache.ignite.internal.processors.query.h2.sql.GridSqlQuerySplitter.split(GridSqlQuerySplitter.java:219) at org.apache.ignite.internal.processors.query.h2.QueryParser.parseH2(QueryParser.java:415) ... 15 more Caused by: java.lang.ArrayIndexOutOfBoundsException at java.lang.System.arraycopy(Native Method) at org.apache.ignite.internal.processors.query.h2.opt.join.CollocationModel.childFilters(CollocationModel.java:240) at org.
[jira] [Created] (IGNITE-11533) Improve memory usage reporting in the logs
Stanislav Lukyanov created IGNITE-11533: --- Summary: Improve memory usage reporting in the logs Key: IGNITE-11533 URL: https://issues.apache.org/jira/browse/IGNITE-11533 Project: Ignite Issue Type: Improvement Reporter: Stanislav Lukyanov Currently node's memory usage is reported in the log like this {code} ^-- PageMemory [pages=66407] ^-- Heap [used=303MB, free=91,6%, comm=849MB] ^-- Off-heap [used=260MB, free=62,73%, comm=580MB] ^-- sysMemPlc region [used=0MB, free=99,21%, comm=40MB] ^-- TxLog region [used=0MB, free=100%, comm=40MB] ^-- Default_Region region [used=260MB, free=47,97%, comm=500MB] {code} And it also used to report "Non heap" metrics which were removed in IGNITE-9305. This has several issues 1. "PageMemory" line alone doesn't help to understand the Ignite off-heap memory usage - page size is needed to calculated the used size, and maxSize of the data regions are needed to calculate the amount of the memory left. 2. "Non heap" could actually be useful. The line used to show "free=-1%" in most setups. This is because metaspace pool of the JVM is not limited by default which makes the total size unlimited. In general, it would be good to see distribution among the non-heap pools (metaspace, direct buffers) to be able to tune JVM settings accordingly. Let's do the following: - Don't print "PageMemory" line - it's cryptic and adds little value to the "Off-heap" section - Bring "Non heap" back in some way -- Don't print "free=-1%" - either print "N/A" or don't print "free" at all -- Add a suggestion to limit metaspace (same as we do for direct memory) - which will enable correct "free" calculation -- Look into possibility of showing distinct metrics for direct memory and metaspace, so that the user can tell which to tune -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11471) JDBC: Thin driver should start with a random server to connect
Stanislav Lukyanov created IGNITE-11471: --- Summary: JDBC: Thin driver should start with a random server to connect Key: IGNITE-11471 URL: https://issues.apache.org/jira/browse/IGNITE-11471 Project: Ignite Issue Type: Improvement Components: jdbc Reporter: Stanislav Lukyanov Thin JDBC driver uses a list of server addresses to connect to. Connections created in a single JVM will use round-robin to choose the server to connect, yielding a uniform distribution. However, the round-robin always starts from the first server in the list. It means that multiple JVMs with the same JDBC connection URL and one connection each will all connect to the same server. Need to choose the first server randomly to have a better distribution. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11365) Partition sizes differ after streaming with allowOverwrite=false with concurrent PME
Stanislav Lukyanov created IGNITE-11365: --- Summary: Partition sizes differ after streaming with allowOverwrite=false with concurrent PME Key: IGNITE-11365 URL: https://issues.apache.org/jira/browse/IGNITE-11365 Project: Ignite Issue Type: Improvement Reporter: Stanislav Lukyanov Attachments: DataStreamerRebalanceMissingDocTest.java Scenario - Start 3 nodes - Run N threads, each thread starts a replicated cache and then streams data to that cache with allowOverwrite=false During the streaming there are "Partition validation failed" warnings. After the streaming in all threads ends, some caches have incorrect size and local SQL queries return less data. Seems that PME can break an ongoing streaming with allowOverwrite=false. allowOverwrite=true is a workaround. Reproducer is attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11355) Mention CacheConfiguration.sqlIndexMaxInlineSize and QueryGroupIndex.inlineSize in inline size suggestions
Stanislav Lukyanov created IGNITE-11355: --- Summary: Mention CacheConfiguration.sqlIndexMaxInlineSize and QueryGroupIndex.inlineSize in inline size suggestions Key: IGNITE-11355 URL: https://issues.apache.org/jira/browse/IGNITE-11355 Project: Ignite Issue Type: Improvement Reporter: Stanislav Lukyanov Currently the suggestion for changing inline size of the PK and AK indexes reads like this {code} recommendation = "set system property " + IgniteSystemProperties.IGNITE_MAX_INDEX_PAYLOAD_SIZE + " with recommended size " + "(be aware it will be used by default for all indexes without explicit inline size)"; {code} However, there is also property CacheConfiguration.sqlIndexMaxInlineSize that can be used. In fact, in most cases it is even better to use the cache configuration property instead of the system property because 1) it is local, not global 2) it is a part of the static configuration and doesn't depend of the correct environment setup The suggestion for other indexes reads {code} recommendation = "use INLINE_SIZE option for CREATE INDEX command, " + "QuerySqlField.inlineSize for annotated classes, or QueryIndex.inlineSize for explicit " + "QueryEntity configuration"; {code} It should also mention QueryGroupIndex.inlineSize for group indexes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11312) JDBC: Thin driver doesn't reports incorrect property names
Stanislav Lukyanov created IGNITE-11312: --- Summary: JDBC: Thin driver doesn't reports incorrect property names Key: IGNITE-11312 URL: https://issues.apache.org/jira/browse/IGNITE-11312 Project: Ignite Issue Type: Improvement Components: jdbc Reporter: Stanislav Lukyanov JDBC driver reports the properties it supports via getPropertyInfo method. It currently reports the property names as simple strings, like "enforceJoinOrder". However, when the properties are processed on connect they are looked up with prefix "ignite.jdbc", e.g. "ignite.jdbc.enforceJoinOrder". Because of this UI tools like DBeaver can't properly pass the properties to Ignite. For example, when "enforceJoinOrder" is set to true in "Connection settings" -> "Driver properties" menu of DBeaver it has no effect. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11230) Fix Javadoc for DataStorageConfiguration
Stanislav Lukyanov created IGNITE-11230: --- Summary: Fix Javadoc for DataStorageConfiguration Key: IGNITE-11230 URL: https://issues.apache.org/jira/browse/IGNITE-11230 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Stanislav Lukyanov Need to polish DataStorageConfiguration Javadoc: - add links to set* methods from get* counterparts - add links to the default values for all properties -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11146) SQL: QueryIndex and DDL generate indexes with different default names
Stanislav Lukyanov created IGNITE-11146: --- Summary: SQL: QueryIndex and DDL generate indexes with different default names Key: IGNITE-11146 URL: https://issues.apache.org/jira/browse/IGNITE-11146 Project: Ignite Issue Type: Task Components: sql Reporter: Stanislav Lukyanov `QueryIndex` with a name set and `CREATE INDEX` without a specified name create indexes with different names, even if all parameters are the same - `CREATE INDEX` produces a lower-case name, and `QueryIndex` produces an upper-case one. For example {code} new QueryIndex("a") {code} in a table "Foo" creates an index with name {code} FOO_A_ASC_IDX {code} while the query {code} CREATE INDEX ON PUBLIC.FOO (a) {code} creates an index {code} foo_a_asc_idx {code} It would be better if DDL generated an upper-case name since that's the default in SQL. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11141) Add Java method to create a cache from a template
Stanislav Lukyanov created IGNITE-11141: --- Summary: Add Java method to create a cache from a template Key: IGNITE-11141 URL: https://issues.apache.org/jira/browse/IGNITE-11141 Project: Ignite Issue Type: Task Reporter: Stanislav Lukyanov Currently cache templates can only be used via REST or SQL: https://apacheignite.readme.io/docs/cache-template. It seems reasonable to add the same functionality to Java. The most flexible way would probably be to add a method like `cacheConfiguration(String)` that would return a configuration of a template with the given name. Then to create a cache from a template one would write code like this {code} CacheConfiguration cfg = ignite.cacheConfiguration("myTemplate"); cfg.setName("myCacheFromTemplate"); ignite.createCache(cfg); {code} As a bonus, the `cacheConfiguration(String)` method may work with regular caches too. For them it would be a shortcut for `ignite.cache("myCache").getConfiguration()`. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10983) Check that persistenceEnabled is consistent on all nodes
Stanislav Lukyanov created IGNITE-10983: --- Summary: Check that persistenceEnabled is consistent on all nodes Key: IGNITE-10983 URL: https://issues.apache.org/jira/browse/IGNITE-10983 Project: Ignite Issue Type: Task Reporter: Stanislav Lukyanov Currently it is possible to have a cluster where the same data region is persistent on some nodes and not persistent on others. This use case doesn't have enough testing, so it's better to deny it for now by adding a check for that and not allowing a node with a different persistenceEnabled value to join the cluster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10952) Document SET STREAMING command options
Stanislav Lukyanov created IGNITE-10952: --- Summary: Document SET STREAMING command options Key: IGNITE-10952 URL: https://issues.apache.org/jira/browse/IGNITE-10952 Project: Ignite Issue Type: Task Components: documentation Reporter: Stanislav Lukyanov SET STREAMING command (https://apacheignite-sql.readme.io/docs/set) has a number of supported options related to the underlying data streamer configuration. However, they are not documented. Need to add documentation (BNFs and examples) for all of the options. The list of options can be found in the SqlSetStreamingCommand source code. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10922) Add a way to override any configuration property via a system property
Stanislav Lukyanov created IGNITE-10922: --- Summary: Add a way to override any configuration property via a system property Key: IGNITE-10922 URL: https://issues.apache.org/jira/browse/IGNITE-10922 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov >From time to time we introduce system properties as a shorthand way to set a >specific configuration parameter, e.g. IGNITE-10921. It would be nice to have a standard way to override any property like that. E.g. instead of -DIGNITE_CONSISTENT_ID=foo we would use something like -Dorg.apache.ignite.IgniteConfiguration.consistentId=foo This mechanism should work for any configuration sections. It seems that it is rather hard to implement for complex types, but perhaps we could start by doing this for simple properties (strings, numbers, etc). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10921) Add IGNITE_CONSISTENT_ID system property
Stanislav Lukyanov created IGNITE-10921: --- Summary: Add IGNITE_CONSISTENT_ID system property Key: IGNITE-10921 URL: https://issues.apache.org/jira/browse/IGNITE-10921 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov IGNITE-10900 explains the benefits of setting consistent ID explicitly and suggests adding a warning it isn't. However, currently changing consistentId for each node is rather cumbersome if the nodes share the configuration. It would be good to add a new system property as a shorthand way to set consistent ID. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10900) Print a warning if native persistence is used without an explicit consistent ID
Stanislav Lukyanov created IGNITE-10900: --- Summary: Print a warning if native persistence is used without an explicit consistent ID Key: IGNITE-10900 URL: https://issues.apache.org/jira/browse/IGNITE-10900 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Experience shows that when Native Persistence is enabled, it is better to explicitly set ConsistentIDs than use the autogenerated ones. First, it simplifies managing the baseline topology. It is much easier to manage it via control.sh when the nodes have stable and meaningful names. Second, it helps to avoid certain shoot-yourself-in-the-foot issues. E.g. if one loses all the data of a baseline node, when that node is restarted it doesn't have its old autogenerated consistent ID - so it is not a part of the baseline anymore. This may be unexpected and confusing. Finally, having explicit consistent IDs improves the general stability of the setup - one knows what the the set of nodes, where they run and what they're called. All in all, it seems beneficial to urge users to explicitly configure consistent IDs. We can do this by introducing a warning that is printed every time a new consistent ID is automatically generated. It should also be printed when a node doesn't have an explicit consistent ID and picks up one from an existing peristence folder. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10887) Docs: Align performance tips for .Net with Java
Stanislav Lukyanov created IGNITE-10887: --- Summary: Docs: Align performance tips for .Net with Java Key: IGNITE-10887 URL: https://issues.apache.org/jira/browse/IGNITE-10887 Project: Ignite Issue Type: Bug Components: documentation Reporter: Stanislav Lukyanov It seems that the .Net Performnace Tips page (https://apacheignite-net.readme.io/docs/performance-tips) is outdated/not aligned with what we have in Java. I think the best way would be to remove most (or all) info from the .Net part, and only leave a link to the main page, since the performance tips should be roughly the same for all platforms. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10856) cassandra-store seems to be broken by incorrect guava version
Stanislav Lukyanov created IGNITE-10856: --- Summary: cassandra-store seems to be broken by incorrect guava version Key: IGNITE-10856 URL: https://issues.apache.org/jira/browse/IGNITE-10856 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov IGNITE-9131 upgrade guava from 18 to 25. However, cassandra-driver-core:3.0 dependency of the Ignite's cassandra-store requires guava 16-19. Need to fix the dependency conflict there. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10808) Discovery message queue may build up with TcpDiscoveryMetricsUpdateMessage
Stanislav Lukyanov created IGNITE-10808: --- Summary: Discovery message queue may build up with TcpDiscoveryMetricsUpdateMessage Key: IGNITE-10808 URL: https://issues.apache.org/jira/browse/IGNITE-10808 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Attachments: IgniteMetricsOverflowTest.java A node receives a new metrics update message every `metricsUpdateFrequency` milliseconds, and the message will be put at the top of the queue (because it is a high priority message). If processing one message takes more than `metricsUpdateFrequency` then multiple `TcpDiscoveryMetricsUpdateMessage` will be in the queue. A long enough delay (e.g. caused by a network glitch or GC) may lead to the queue building up tens of metrics update messages which are essentially useless to be processed. Finally, if processing a message on average takes a little more than `metricsUpdateFrequency` (even for a relatively short period of time, say, for a minute due to network issues) then the message worker will end up processing only the metrics updates and the cluster will essentially hang. Reproducer is attached. In the test, the queue first builds up and then very slowly being teared down, causing "Failed to wait for PME" messages. Need to change ServerImpl's SocketReader not to put another metrics update message to the top of the queue if it already has one (or replace the one on the top with new one). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10563) Allow manual fsync for WAL
Stanislav Lukyanov created IGNITE-10563: --- Summary: Allow manual fsync for WAL Key: IGNITE-10563 URL: https://issues.apache.org/jira/browse/IGNITE-10563 Project: Ignite Issue Type: Improvement Reporter: Stanislav Lukyanov When walMode is set to LOG_ONLY or BACKGROUND there is a gap between successful return of cache write operations and actual of the data to the persistent memory. This gap is, while usually low, generally can be of any length and depends on low-level system parameters (e.g. sysctl memory and IO settings on Linux). On the other hand, there are situations when a user may want to ensure that at certain points all writes have been propagated to the disk. For example, say a large batch of data is being inserted into Ignite from an upstream system. After finishing the batch we want to ensure that all of the data is secure, and after that we remove the batch from the upstream storage. In other words, we want {code} map = upstream.getData(); cache.putAll(map); cache.fsync(); // <--- upstream.removeAll(map); {code} Currently there is no easy way to ensure that certain write (or all writes until a point in time) has been flushed to a device. We can only rely on settings like WAL sync interval, checkpoint timeout, etc. It would be nice to have a way to manually call fsync() for WAL to have a strong guarantee that all previous updates have been fully written on disk. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10516) Storage is corrupted after CREATE INDEX IF NOT EXISTS on different tables
Stanislav Lukyanov created IGNITE-10516: --- Summary: Storage is corrupted after CREATE INDEX IF NOT EXISTS on different tables Key: IGNITE-10516 URL: https://issues.apache.org/jira/browse/IGNITE-10516 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Given two tables in the same schema, we can't create an index with the same name for both tables. In other words, the following code leads to an error - which is good {code} CREATE INDEX IDX on T1 (COL); CREATE INDEX IDX on T2 (COL); {code} If used with `IF NOT EXISTS`, the queries pass. It might be OK or not - one needs to look into SQL spec to check if the second operation should be a no-op (because IDX exists) or fail (because IDX exists for a different table, so the caller is probably doing something wrong) {code} CREATE INDEX IDX on T1 (COL); CREATE INDEX IF NOT EXISTS IDX on T2 (COL); {code} However, if persistence is enabled, the node will fail to restart complaining about duplicate index names. {code} class org.apache.ignite.IgniteCheckedException: Duplicate index name [cache=SQL_PUBLIC_T2, schemaName=PUBLIC, idxName=IDX, existingTable=T, table=T2] at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1183) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2040) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1732) at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656) at org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:959) at org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:900) at org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:888) at org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:854) at org.apache.ignite.IndexWithSameNameTest.test(IndexWithSameNameTest.java:77) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at junit.framework.TestCase.runTest(TestCase.java:176) at org.apache.ignite.testframework.junits.GridAbstractTest.access$001(GridAbstractTest.java:150) at org.apache.ignite.testframework.junits.GridAbstractTest$6.evaluate(GridAbstractTest.java:2104) at org.apache.ignite.testframework.junits.GridAbstractTest$7.run(GridAbstractTest.java:2119) at java.lang.Thread.run(Thread.java:748) Caused by: class org.apache.ignite.IgniteException: Duplicate index name [cache=SQL_PUBLIC_T2, schemaName=PUBLIC, idxName=IDX, existingTable=T, table=T2] at org.apache.ignite.internal.processors.query.GridQueryProcessor.registerCache0(GridQueryProcessor.java:1650) at org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart0(GridQueryProcessor.java:803) at org.apache.ignite.internal.processors.query.GridQueryProcessor.onCacheStart(GridQueryProcessor.java:866) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCacheInRecoveryMode(GridCacheProcessor.java:2595) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.access$1400(GridCacheProcessor.java:204) at org.apache.ignite.internal.processors.cache.GridCacheProcessor$CacheRecoveryLifecycle.afterBinaryMemoryRestore(GridCacheProcessor.java:5481) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.restoreBinaryMemory(GridCacheDatabaseSharedManager.java:947) at org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.startMemoryRestore(GridCacheDatabaseSharedManager.java:1922) at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1050) ... 18 more {code} It looks like the second index (on T2) is partially created after all. Need to either block index creation by `CREATE INDEX IF NOT EXISTS` completely, or just fail that query when the table names don't match (if SQL spec allows it). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10506) Emphasize the need to close the query cursors in docs
Stanislav Lukyanov created IGNITE-10506: --- Summary: Emphasize the need to close the query cursors in docs Key: IGNITE-10506 URL: https://issues.apache.org/jira/browse/IGNITE-10506 Project: Ignite Issue Type: Bug Components: documentation Reporter: Stanislav Lukyanov Currently the need to close query cursors is mentioned only in two places: - Queries docs on readme.io: https://apacheignite.readme.io/docs/cache-queries#section-querycursor - QueryCursor::close javadoc https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/cache/query/QueryCursor.html#close-- A failure to close a cursor may lead to severe resource leaks. It seems reasonable to better emphasize and explain how to approach cursors closing. Most importantly, we need to have a mention of that in the Java API secton of SQL readme.io docs (https://apacheignite-sql.readme.io/docs/java-sql-api). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10225) Striped pool starvation check reports false-positive warnings if a job starts right before the check
Stanislav Lukyanov created IGNITE-10225: --- Summary: Striped pool starvation check reports false-positive warnings if a job starts right before the check Key: IGNITE-10225 URL: https://issues.apache.org/jira/browse/IGNITE-10225 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov StripedExecutor::checkStarvation checks if two conditions are met: a stripe is executing a job and the stripe's job counter hasn't changed since the last check. These conditions will be met if no jobs have been executed for a long time and a job (even a very short one) has just started. This check isn't supposed to be exact and allowed to produce false-positives. However, the check is supposed to detect jobs running for more than starvation check interval (IGNITE_STARVATION_CHECK_INTERVAL), but in this case it may detect even very short ones. To fix this we may require a stripe to have no progress in two consecutive checks - then we will have much less chance for a false-positive, but still detect hanging or very long jobs. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10223) Affinity methods should return List instead of Collection
Stanislav Lukyanov created IGNITE-10223: --- Summary: Affinity methods should return List instead of Collection Key: IGNITE-10223 URL: https://issues.apache.org/jira/browse/IGNITE-10223 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Methods of class Affinity should return List instead of Collection when order of nodes matters. The following methods are supposed to always return a collection with primary node preceding backups: Affinity::mapPartitionToPrimaryAndBackups Affinity::mapKeyToPrimaryAndBackups The return type for both methods is Collection. The Javadocs say: "Note that primary node is always first in the returned collection". In general, the notion of "order" and "first element" is not defined for a Collection (we could play with iterators, but its awkward from both code and logic perspective). It makes more sense for this methods to return List instead. The task is to introduce List equivalents of these methods. It would also be nice to check if other APIs have similar issues. The tricky part is compatibility. Changing return type from Collection to List is source compatible (meaning the old client code will compile with the new method), but not binary compatible (meaning the already compiled client code will break because the signature has changed). A compatible solution would be to - deprecate the existing methods - add methods with different names, like mapPartitionToPrimaryAndBackups*List* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10152) Allow warming up data region before a node has joined
Stanislav Lukyanov created IGNITE-10152: --- Summary: Allow warming up data region before a node has joined Key: IGNITE-10152 URL: https://issues.apache.org/jira/browse/IGNITE-10152 Project: Ignite Issue Type: New Feature Reporter: Stanislav Lukyanov After a node with existing PDS joins the cluster its performance will be low for a while - until it loads all data into RAM. This is a feature of Ignite - cold restart - that allows nodes to start as fast as possible. However, it would be good to also allow "hot restart" - a way to preload as much data into RAM as needed before the node has joined. Then the node will not slow the cluster down during its warm-up. There are several recent improvements with similar goals: - IGNITE-8873 added methods to preload data of a specific partition - IGNITE-7196 and IGNITE-9420 moved data recovery to before node joins the cluster Combining the above approaches we could have allow a user to trigger synchronous preloading right after the data recovery but before discovery starts up. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10150) DisappearedCacheWasNotFoundMessageSelfTest fails with NPE
Stanislav Lukyanov created IGNITE-10150: --- Summary: DisappearedCacheWasNotFoundMessageSelfTest fails with NPE Key: IGNITE-10150 URL: https://issues.apache.org/jira/browse/IGNITE-10150 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov https://ci.ignite.apache.org/viewLog.html?buildId=2226328&tab=buildResultsDiv&buildTypeId=IgniteTests24Java8_Queries2 {code} java.lang.NullPointerException at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.nodesForPartitions(GridReduceQueryExecutor.java:1702) at org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.query(GridReduceQueryExecutor.java:690) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$8.iterator(IgniteH2Indexing.java:1801) at org.apache.ignite.internal.processors.cache.QueryCursorImpl.iterator(QueryCursorImpl.java:95) at org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$9.iterator(IgniteH2Indexing.java:1869) at org.apache.ignite.internal.processors.cache.QueryCursorImpl.iterator(QueryCursorImpl.java:95) at org.apache.ignite.internal.processors.cache.QueryCursorImpl.getAll(QueryCursorImpl.java:114) at org.apache.ignite.internal.processors.query.h2.twostep.DisappearedCacheWasNotFoundMessageSelfTest.testDisappearedCacheWasNotFoundMessage(DisappearedCacheWasNotFoundMessageSelfTest.java:59) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at junit.framework.TestCase.runTest(TestCase.java:176) at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2206) at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:144) at org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2121) at java.lang.Thread.run(Thread.java:748) {code] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10067) Tracking progress of index building
Stanislav Lukyanov created IGNITE-10067: --- Summary: Tracking progress of index building Key: IGNITE-10067 URL: https://issues.apache.org/jira/browse/IGNITE-10067 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov When a new index is being created or indexes are being rebuilt in case a node is restarted without index.bin, the performance of queries is low until the index is ready. It would be good to track the process of indexing to have an approximate completion time. Currently we only have "started" and "finished" messages in the logs for each index. It would be good to also know - All indexes to be built - Number (+ percentage?) of processed entries (or pages, if it's hard to update progress for each entry) - Number (+ percentage?) of entries left It seems reasonable to start collecting this data and just print it in the logs for now. `metricsLogFrequency` could be reused as the frequency of logging. After community has some experience with using this, this metrics could be exposed via Java API/JMX/etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10059) Local scan query against LOST partition fails
Stanislav Lukyanov created IGNITE-10059: --- Summary: Local scan query against LOST partition fails Key: IGNITE-10059 URL: https://issues.apache.org/jira/browse/IGNITE-10059 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Local scan queries executed against a LOST partition always fail. Stack trace: {code} javax.cache.CacheException: No queryable nodes for partition 0 [forced local query=GridCacheQueryAdapter [type=SCAN, clsName=null, clause=null, filter=null, transform=null, part=0, incMeta=false, metrics=GridCacheQueryMetricsAdapter [minTime=9223372036854775807, maxTime=0, sumTime=0, avgTime=0.0, execs=0, completed=0, fails=0], pageSize=1024, timeout=0, incBackups=false, forceLocal=true, dedup=false, prj=org.apache.ignite.internal.cluster.ClusterGroupAdapter@3a806080, keepBinary=false, subjId=null, taskHash=0, mvccSnapshot=null]] at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:697) at org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.query(GatewayProtectedCacheProxy.java:376) at org.apache.ignite.internal.processors.cache.distributed.IgniteCachePartitionLossPolicySelfTest.checkQueryPasses(IgniteCachePartitionLossPolicySelfTest.java:767) at org.apache.ignite.internal.processors.cache.distributed.IgniteCachePartitionLossPolicySelfTest.validateQuery0(IgniteCachePartitionLossPolicySelfTest.java:720) at org.apache.ignite.internal.processors.cache.distributed.IgniteCachePartitionLossPolicySelfTest.validateQuery(IgniteCachePartitionLossPolicySelfTest.java:691) at org.apache.ignite.internal.processors.cache.distributed.IgniteCachePartitionLossPolicySelfTest.checkLostPartition(IgniteCachePartitionLossPolicySelfTest.java:497) at org.apache.ignite.internal.processors.cache.distributed.IgniteCachePartitionLossPolicySelfTest.testReadOnlyAll(IgniteCachePartitionLossPolicySelfTest.java:185) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at junit.framework.TestCase.runTest(TestCase.java:176) at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2209) at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:144) at org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:2124) at java.lang.Thread.run(Thread.java:748) Caused by: class org.apache.ignite.IgniteCheckedException: No queryable nodes for partition 0 [forced local query=GridCacheQueryAdapter [type=SCAN, clsName=null, clause=null, filter=null, transform=null, part=0, incMeta=false, metrics=GridCacheQueryMetricsAdapter [minTime=9223372036854775807, maxTime=0, sumTime=0, avgTime=0.0, execs=0, completed=0, fails=0], pageSize=1024, timeout=0, incBackups=false, forceLocal=true, dedup=false, prj=org.apache.ignite.internal.cluster.ClusterGroupAdapter@3a806080, keepBinary=false, subjId=null, taskHash=0, mvccSnapshot=null]] at org.apache.ignite.internal.processors.cache.query.GridCacheQueryAdapter.executeScanQuery(GridCacheQueryAdapter.java:541) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl$1.applyx(IgniteCacheProxyImpl.java:410) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl$1.applyx(IgniteCacheProxyImpl.java:408) at org.apache.ignite.internal.util.lang.IgniteOutClosureX.apply(IgniteOutClosureX.java:36) at org.apache.ignite.internal.processors.query.GridQueryProcessor.executeQuery(GridQueryProcessor.java:2713) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:407) at org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.query(IgniteCacheProxyImpl.java:689) ... 15 more {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10058) resetLostPartitions() leaves an additional copy of a partition in the cluster
Stanislav Lukyanov created IGNITE-10058: --- Summary: resetLostPartitions() leaves an additional copy of a partition in the cluster Key: IGNITE-10058 URL: https://issues.apache.org/jira/browse/IGNITE-10058 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov If there are several copies of a LOST partition, resetLostPartitions() will leave all of them in the cluster as OWNING. Scenario: 1) Start 4 nodes, a cache with backups=0 and READ_WRITE_SAFE, fill the cache 2) Stop one node - some partitions are recreated on the remaining nodes as LOST 3) Start one node - the LOST partitions are being rebalanced to the new node from the existing ones 4) Wait for rebalance to complete 5) Call resetLostPartitions() After that the partitions that were LOST become OWNING on all nodes that had them. Eviction of these partitions doesn't start. Need to correctly evict additional copies of LOST partitions either after rebalance on step 4 or after resetLostPartitions() call on step 5. Current resetLostPartitions() implementation does call checkEvictions(), but the ready affinity assignment contains several nodes per partition for some reason. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10057) SQL queries hang during rebalance if there are LOST partitions
Stanislav Lukyanov created IGNITE-10057: --- Summary: SQL queries hang during rebalance if there are LOST partitions Key: IGNITE-10057 URL: https://issues.apache.org/jira/browse/IGNITE-10057 Project: Ignite Issue Type: Bug Components: sql Reporter: Stanislav Lukyanov When there are both LOST and MOVING partitions in the cluster, SQL queries will hang until rebalance has completed and there aren't any MOVING partitions. During that time the reducer node keeps printing {code} [2018-10-30 10:26:06,787][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=13, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:06,798][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=14, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:06,818][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=15, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:06,849][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=16, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:06,889][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=17, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:06,940][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=18, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:07,001][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=19, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:07,072][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=20, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:07,152][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=21, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:07,243][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=22, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:07,343][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=23, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:07,454][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but corresponding cache group has data nodes) [qryId=24, cacheIds=[1254100233], cacheName=partitioned, cacheId=1254100233, part=0, cacheGroupId=1254100233] [2018-10-30 10:26:07,575][INFO ][test-runner-#1%cache.IndexingCachePartitionLossPolicySelfTest%][GridReduceQueryExecutor] Failed to calculate nodes for SQL query (partition has no owners, but correspondi
[jira] [Created] (IGNITE-10044) LOST partition is marked as OWNING after the owner rejoins with existing persistent data
Stanislav Lukyanov created IGNITE-10044: --- Summary: LOST partition is marked as OWNING after the owner rejoins with existing persistent data Key: IGNITE-10044 URL: https://issues.apache.org/jira/browse/IGNITE-10044 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov When persistence is enabled LOST partition may become OWNING without a call to resetLostPartitions. If a partition is marked as LOST, and a node with that partition in persistence joins, the partition becomes OWNING. IgniteCache::lostPartitions doesn't return that partition anymore. Apparently, it only affects the lost partitions list, while needsRecovery flag stays set until resetLostPartitions is called. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10043) Lost partitions list is reset when only one server is alive in the cluster
Stanislav Lukyanov created IGNITE-10043: --- Summary: Lost partitions list is reset when only one server is alive in the cluster Key: IGNITE-10043 URL: https://issues.apache.org/jira/browse/IGNITE-10043 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov When server nodes are killed one by one IgniteCache.lostPartitions() returns non-empty list up until there is just one node left. When only one node is alive, the lostPartitions() returns an empty collection even though the partition loss events has been fired and the partitions are missing. This only affects cluster with persistence enabled. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10041) Partition loss policies work incorrectly with BLT
Stanislav Lukyanov created IGNITE-10041: --- Summary: Partition loss policies work incorrectly with BLT Key: IGNITE-10041 URL: https://issues.apache.org/jira/browse/IGNITE-10041 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Current partition loss handling (initially designed before persistence and BLT were introduced) aligns poorly with BLT semantics. The general problem is that partition loss handling assumes that rebalancing always happens on topology change - which is not the case with BLT. The features with semantics conflicting with BLT are: - PartitionLossPolicy.READ_ONLY_ALL and PartitionLossPolicy.READ_WRITE_ALL READ_*_ALL policies assume that there is "some" partition created in the cluster which we could read or write, even though it is marked as LOST. With BLT the LOST partition only exists on an offline node, so there is no place to route the requests to. - PartitionLossPolicy.IGNORE Somewhat similar to the READ_*_ALL, but IGNORE also expects that the lost partition is immediately recreated on another node - which is impossible without changing affinity, i.e. without resetting BLT. - IgniteCache::withPartitionRecover withPartitionRecover is supposed to ignore partition loss policies and allow all operations, but when a partition is lost in a BLT cluster it's actually unavailable, there is straightforward way to ignore it. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9964) SQL: query by affinity key fails when a table is created from a custom template
Stanislav Lukyanov created IGNITE-9964: -- Summary: SQL: query by affinity key fails when a table is created from a custom template Key: IGNITE-9964 URL: https://issues.apache.org/jira/browse/IGNITE-9964 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov SELECT by affinity key doesn't work on a table that was created with a custom cache template if entries were added through JCache (using withKeepBinary()). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9902) ScanQuery doesn't take lost partitions into account when persistence is enabled
Stanislav Lukyanov created IGNITE-9902: -- Summary: ScanQuery doesn't take lost partitions into account when persistence is enabled Key: IGNITE-9902 URL: https://issues.apache.org/jira/browse/IGNITE-9902 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Same as IGNITE-9841, but about scan queries. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9900) Upgrade annotations.jar to a new version
Stanislav Lukyanov created IGNITE-9900: -- Summary: Upgrade annotations.jar to a new version Key: IGNITE-9900 URL: https://issues.apache.org/jira/browse/IGNITE-9900 Project: Ignite Issue Type: Improvement Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov OWASP Dependency Check reports that annotations.jar of version 13.0 is affected by vulnerability CVE-2017-8316, while it obviously isn't (the CVE is about XXE attack, and annotations.jar is, well, annotations). The issue is that NVD database only says that "intellij_idea" is affected, and OWASP doesn't know better than to map annotations.jar to the intellij_idea product. While the problem could be (and perhaps will be, see https://youtrack.jetbrains.com/issue/IDEA-200601) sorted out on the level of OWASP and NVD, the easiest way to workaround this is to upgrade annotations.jar to a new version (currently 16.0.3). It will help Ignite users to avoid annoying false-positive warnings from OWASP. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9841) SQL take lost partitions into account when persistence is enabled
Stanislav Lukyanov created IGNITE-9841: -- Summary: SQL take lost partitions into account when persistence is enabled Key: IGNITE-9841 URL: https://issues.apache.org/jira/browse/IGNITE-9841 Project: Ignite Issue Type: Bug Components: sql Reporter: Stanislav Lukyanov Attachments: IgniteSqlQueryWithLostPartitionsTest.java IGNITE-8927 changed SQL queries to honor lost partitions. However, it doesn't work if persistence is enabled. E.g. if there are lost partitions then `select * from T` fails for in-memory caches (good) and silently ignores the lost partitions for persistent caches (bad). See attached reproducer. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9825) Need test for cache writes on unstable topology
Stanislav Lukyanov created IGNITE-9825: -- Summary: Need test for cache writes on unstable topology Key: IGNITE-9825 URL: https://issues.apache.org/jira/browse/IGNITE-9825 Project: Ignite Issue Type: Test Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov It appears that ignite-core test suites have no tests for a simple case - cache writes on unstable topology. Need to add a simple test covering this. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9651) Binary objects comparison may fail when the same reference occurs more than once
Stanislav Lukyanov created IGNITE-9651: -- Summary: Binary objects comparison may fail when the same reference occurs more than once Key: IGNITE-9651 URL: https://issues.apache.org/jira/browse/IGNITE-9651 Project: Ignite Issue Type: Bug Components: binary Reporter: Stanislav Lukyanov Attachments: RepeatingFieldSqlTest.java, RepeatingFieldTest.java If a composite object contains multiple occurrences of the same POJO that POJO will be properly serialized only the first time, and all other references will refer to that first object. Because of that the same object may have different views in binary format when serialized on its own and when serialized as a part of another object. Having multiple binary views of the same object leads to several issues with binary objects comparison. Examples in pseudocode are below, JUnit tests are attached Example 1 (RepeatingFieldTest.java) {code} class Key { int i, Pojo pojo1, Pojo pojo2 } class Pojo { int i } cache.put(new Key(42, new Pojo(42), new Pojo(42)), 42); // new Pojo is created for both fields - works assertEquals(INT, cache.get(new Key(42, new Pojo(42), new Pojo(42; // same Pojo is used for both fields - fails // this is because the second field is serialized as a reference to the first one, and binary representation differs Pojo obj = new Pojo(42); assertEquals(42, cache.get(new Key(42, obj, obj))); {code} Example 2 (RepeatingFieldSqlTest.java) {code} class Pojo { int i } class Key { int i, Pojo pojo } class Value { Pojo pojo, Key key } Pojo obj = new Pojo(INT); Key key = new Key(INT, obj); Value val = new Value(key, obj); cache.put(key, val); // supposed to return 1 row, but returns 0 SqlFieldsQuery qry = new SqlFieldsQuery("select * from Value where _key = key"); {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9526) Allow explicitly setting index inline size for Java objects
Stanislav Lukyanov created IGNITE-9526: -- Summary: Allow explicitly setting index inline size for Java objects Key: IGNITE-9526 URL: https://issues.apache.org/jira/browse/IGNITE-9526 Project: Ignite Issue Type: Bug Components: sql Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov Currently Ignite never inlines indexed objects when they are POJOs. It makes sense, as there is currently no way of inlining a POJO other than by inlining the binary object, and the headers of the binary objects (i.e. their first 24 bytes) are usually the same for the same indexed field. However, this approach has downsides. Silently ignoring the specified inline size might be surprising for a user willing to trade (a lot of) memory for speed and setting a large inline size for that purpose. On the other hand, inlining headers and just several first bytes of POJOs in binary format has proven to be beneficial. It is especially beneficial in case page replacements are happening, as this greatly reduces the number of data pages required to traverse an index tree. The proposal is to allow POJO inlining in case the inline size is given for the specific POJO field. So, unlike strings and byte arrays, POJOs will not use the max inline size by default - as in most cases inlining POJOs isn't going to help. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9331) Add thin client tests for custom TLS protocols
Stanislav Lukyanov created IGNITE-9331: -- Summary: Add thin client tests for custom TLS protocols Key: IGNITE-9331 URL: https://issues.apache.org/jira/browse/IGNITE-9331 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov IGNITE-6167 added an ability to specify a custom TLS protocol for Ignite connections. It includes tests for regular Ignite node and Java Thin client. Need to add tests for other thin clients: JDBC, ODBC, .Net, Node.js, etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9315) Eviction meta from near cache may appear in DHT entries
Stanislav Lukyanov created IGNITE-9315: -- Summary: Eviction meta from near cache may appear in DHT entries Key: IGNITE-9315 URL: https://issues.apache.org/jira/browse/IGNITE-9315 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov There are cases when eviction metadata from near cache's eviction policy may be added to a DHT cache entry, leading to unpredictable behavior. The bug is caused by coding mistakes when a DHT entry is being updated, but near cache entry is being touched. E.g. in GridDhtPartitionDemander::preloadEntry: {code} GridCacheContext cctx = grp.sharedGroup() ? ctx.cacheContext(entry.cacheId()) : grp.singleCacheContext(); cached = cctx.dhtCache().entryEx(entry.key()); //cctx = cached.context(); if (log.isDebugEnabled()) log.debug("Rebalancing key [key=" + entry.key() + ", part=" + p + ", node=" + from.id() + ']'); if (preloadPred == null || preloadPred.apply(entry)) { if (cached.initialValue( entry.value(), entry.version(), entry.ttl(), entry.expireTime(), true, topVer, cctx.isDrEnabled() ? DR_PRELOAD : DR_NONE, false )) { cctx.evicts().touch(cached, topVer); // Start tracking. {code} Here `cached` is always DHT entry, but `cctx` can be near. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9247) CPP Thin: implement GetAll
Stanislav Lukyanov created IGNITE-9247: -- Summary: CPP Thin: implement GetAll Key: IGNITE-9247 URL: https://issues.apache.org/jira/browse/IGNITE-9247 Project: Ignite Issue Type: New Feature Components: thin client Reporter: Stanislav Lukyanov Need to implement GetAll in C++ Thin client. Currently, there is no way to extract values from cache via C++ Thin client without knowing the keys beforehand. GetAll would be the easiest way to do that. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9182) Propagate lazy flag to synthetic SELECT used by first step of DELETE
Stanislav Lukyanov created IGNITE-9182: -- Summary: Propagate lazy flag to synthetic SELECT used by first step of DELETE Key: IGNITE-9182 URL: https://issues.apache.org/jira/browse/IGNITE-9182 Project: Ignite Issue Type: Bug Components: sql Reporter: Stanislav Lukyanov When DELETE is executed it first executed SELECT with the same WHERE, and then deletes all results via an invoke(). If a large number of entries is to be deleted, it may lead to OOM. The task is to allow to perform the SELECT with the parent query's LAZY flag. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8952) Move CacheStore::loadCache to a separate interface
Stanislav Lukyanov created IGNITE-8952: -- Summary: Move CacheStore::loadCache to a separate interface Key: IGNITE-8952 URL: https://issues.apache.org/jira/browse/IGNITE-8952 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov 3rd party persistence has three way of interacting with the backing storage: - JCache's CacheLoader methods for read-through - JCache's CacheWriter methods for write-through - Ignite's CacheStore::loadCache for bulk cache preloading Usually, one just uses a CacheStore implementation and provides a no-op implementation for the methods that are not needed (e.g. no-op implementation for write() if write-through isn't used). However, this adds certain redundancy. In particular, if a user only wants cache preloading (loadCache()) but no read-through or write-through, they will need to provide a bunch of these no-op methods, which can seem messy. Instead, it would be nice to move loadCache() method to a separate interface, say CachePreloader. This way we'll have 4 interfaces: - JCache's CacheLoader for read-through - JCache's CacheWriter for write-through - Ignite's CachePreloader for loadCache() - Ignite's CacheStore for all three together One can either choose any combination of loader, writer and preloader, OR set the store. The task is to - extract CacheStore::loadCache into a new interface CachePreloader - add CacheConfiguration::setCachePreloader - make sure that setCachePreloader works alone or in combination with loader and writer, and doesn't work with store -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8776) Eviction policy MBeans are never registered if evictionPolicyFactory is used
Stanislav Lukyanov created IGNITE-8776: -- Summary: Eviction policy MBeans are never registered if evictionPolicyFactory is used Key: IGNITE-8776 URL: https://issues.apache.org/jira/browse/IGNITE-8776 Project: Ignite Issue Type: Bug Affects Versions: 2.5 Reporter: Stanislav Lukyanov Eviction policy MBeans, such as LruEvictionPolicyMBean, are never registered if evictionPolicyFactory is set instead of evictionPolicy (the latter is deprecated). This happens because GridCacheProcessor::registerMbean attempts to find either an *MBean interface or IgniteMBeanAware interface on the passed object. It works for LruEvictionPolicy but not for LruEvictionPolicyFactory (which doesn't implement these interfaces). The code needs to be adjusted to handle factories correctly. New tests are needed to make sure that all standard beans are registered (IgniteKernalMbeansTest does that for kernal mbeans - need the same for cache beans). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8774) Daemon moves cluster to compatibility mode when joins
Stanislav Lukyanov created IGNITE-8774: -- Summary: Daemon moves cluster to compatibility mode when joins Key: IGNITE-8774 URL: https://issues.apache.org/jira/browse/IGNITE-8774 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov When a daemon node joins the cluster seems to switch to compatibility mode (allowing nodes without baseline support). It prevents baseline nodes from being restarted. Example: {code} Ignite ignite1 = IgnitionEx.start("examples/config/persistentstore/example-persistent-store.xml", "srv1"); Ignite ignite2 = IgnitionEx.start("examples/config/persistentstore/example-persistent-store.xml", "srv2"); ignite2.cluster().active(true); IgnitionEx.setClientMode(true); IgnitionEx.setDaemon(true); Ignite daemon = IgnitionEx.start("examples/config/persistentstore/example-persistent-store.xml", "daemon"); IgnitionEx.setClientMode(false); IgnitionEx.setDaemon(false); ignite2.close(); IgnitionEx.start("examples/config/persistentstore/example-persistent-store.xml", "srv2"); {code} The attempt to restart ignite2 throws an exception: {code} [2018-06-11 18:45:25,766][ERROR][tcp-disco-msg-worker-#39%srv2%][root] Critical system error detected. Will be handled accordingly to configured handler [hnd=class o.a.i.failure.StopNodeOrHaltFailureHandler, failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=class o.a.i.IgniteException: Node with BaselineTopology cannot join mixed cluster running in compatibility mode]] class org.apache.ignite.IgniteException: Node with BaselineTopology cannot join mixed cluster running in compatibility mode at org.apache.ignite.internal.processors.cluster.GridClusterStateProcessor.onGridDataReceived(GridClusterStateProcessor.java:714) at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$5.onExchange(GridDiscoveryManager.java:883) at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.onExchange(TcpDiscoverySpi.java:1939) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processNodeAddedMessage(ServerImpl.java:4354) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2744) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.processMessage(ServerImpl.java:2536) at org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerAdapter.body(ServerImpl.java:6775) at org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2621) at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62) {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8726) Highlight that memory metrics are local for a node in the docs
Stanislav Lukyanov created IGNITE-8726: -- Summary: Highlight that memory metrics are local for a node in the docs Key: IGNITE-8726 URL: https://issues.apache.org/jira/browse/IGNITE-8726 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Stanislav Lukyanov Memory Metrics (DataRegionMetrics and DataStorageMetrics) in Ignite are local for each node. However, this is not highlighted in the documentation enough. The code snippets suggest to just call `ignite.dataRegionMetrics()` which seems to be a bit at odds with the general use case of Ignite servers being started via ignite.sh. It would be good to have an easily noticeable warning that the metrics will only return data for the local node (and that, for example, on client they would typically always print 0). Also, would be nice to include a couple of practical approaches other than JMX to collect metrics. E.g. a snippet of client code getting metrics from all servers: {code} Collection metricsFromNodes = ignite.compute().broadcast(() -> { Ignite ignite = Ignition.localIgnite(); StringBuilder sb = new StringBuilder(); sb.append("Node: " + ignite.name()); for (DataRegionMetrics metrics : ignite.dataRegionMetrics()) { // append metrics to the builder } return sb.toString(); }); for (String metricsString : metricsFromNodes) System.out.println(metricsString); {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8627) Verify local partition map against global
Stanislav Lukyanov created IGNITE-8627: -- Summary: Verify local partition map against global Key: IGNITE-8627 URL: https://issues.apache.org/jira/browse/IGNITE-8627 Project: Ignite Issue Type: Improvement Reporter: Stanislav Lukyanov Assignee: Alexey Goncharuk IGNITE-7467 introduced verification of partition update counters and sizes between replicas. Similarly, it's possible to verify that local copy of the partition map doesn't differ from the map on the other nodes, detecting issues that may lead to inconsistent reads or data loss. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8611) Binary marshaller documentation should cover how data classes can or can't be changed
Stanislav Lukyanov created IGNITE-8611: -- Summary: Binary marshaller documentation should cover how data classes can or can't be changed Key: IGNITE-8611 URL: https://issues.apache.org/jira/browse/IGNITE-8611 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Stanislav Lukyanov Binary marshaller docs (https://apacheignite.readme.io/docs/binary-marshaller) give an idea that a class structure may be changed (fields can be added or removed) and the marshaller will handle such change. However, not all changes are supported. One corner case is when an enum value is stored in the cache: if the order of the enum constants is changed, or if a new constant is added at the start or at the middle of the constants list, it will lead to an error. This is because the enums are stored as ordinals (integers), and the ordinals of an enum depend on the order of values in the code. The task is to update the documentation with the description of data class changes that are incompatible from binary marshallers point of view. At least the enum case should be covered. If more cases are discovered, they should be documented as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8550) CacheAbstractJdbcStore expects merge to always return 1 but MySQL may also return 2 or 0
Stanislav Lukyanov created IGNITE-8550: -- Summary: CacheAbstractJdbcStore expects merge to always return 1 but MySQL may also return 2 or 0 Key: IGNITE-8550 URL: https://issues.apache.org/jira/browse/IGNITE-8550 Project: Ignite Issue Type: Bug Components: sql Reporter: Stanislav Lukyanov CacheAbstractJdbcStore.write attempts to execute a merge update if it is available, and expects the merge to always return 1 (as the number of updated entries is always 1). However, MySQL's `INSERT ... ON DUPLICATE KEY UPDATE` (https://dev.mysql.com/doc/refman/8.0/en/insert-on-duplicate.html) may return 0 or 2, depending on what was updated: {quote}With ON DUPLICATE KEY UPDATE, the affected-rows value per row is 1 if the row is inserted as a new row, 2 if an existing row is updated, and 0 if an existing row is set to its current values.{quote} Because of that, CacheAbstractJdbcStore may report a false warning. Need to consider either removing the warning or special-case the MySQL dialect to allow to return values other than 1. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8501) Retry on GridServiceNotFoundException in GridServiceProxy needs to be fixed
Stanislav Lukyanov created IGNITE-8501: -- Summary: Retry on GridServiceNotFoundException in GridServiceProxy needs to be fixed Key: IGNITE-8501 URL: https://issues.apache.org/jira/browse/IGNITE-8501 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov `GridServiceProxy::invokeMethod` attempts to invoke a method of an Ignite service and performs retries in case the invocation procedure throws `GridServiceNotFoundException` or `ClusterTopologyCheckedException` (these exceptions may be thrown in case the service assignments were already calculated, but the service instance was not yet created and initialized on the affinity server). After the fix IGNITE-7904 the exception type thrown by the remote invocation code changed to `IgniteCheckedException` (with a cause being `GridServiceNotFoundException` or `ClusterTopologyCheckedException`). Because of that, the retry doesn't work now. Need to fix the code to correctly handle new exception chain. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8486) Update ConcurrentLinkedDeque in Ignite's master repository to the latest JDK version
Stanislav Lukyanov created IGNITE-8486: -- Summary: Update ConcurrentLinkedDeque in Ignite's master repository to the latest JDK version Key: IGNITE-8486 URL: https://issues.apache.org/jira/browse/IGNITE-8486 Project: Ignite Issue Type: Improvement Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov Ignite still uses copies of several JSR 166 (j.u.concurrent) classes in it's sources. Those copies are now outdated compared to the latest versions used in JDK. In particular, `ConcurrentLinkedDeque` has received a couple of correctness fixes recently (https://bugs.openjdk.java.net/browse/JDK-8188900, https://bugs.openjdk.java.net/browse/JDK-8189387). It would be good to have them in Ignite as well to protect ourselves from possible issues. The task is to update Ignite's `ConcurrentLinkedDeque8` to the latest version of `ConcurrentLinkedDeque`, although keeping compatibility with earlier Java version (e.g. JDK's version now uses Java 9's VarHandles which we can't use yet). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8470) NPE when starting LOCAL cache on a client with no data regions
Stanislav Lukyanov created IGNITE-8470: -- Summary: NPE when starting LOCAL cache on a client with no data regions Key: IGNITE-8470 URL: https://issues.apache.org/jira/browse/IGNITE-8470 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov If a LOCAL cache is started on a client and the client has no data regions configured then an NPE is thrown with no error message: {code} class org.apache.ignite.IgniteCheckedException: null at org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7284) at org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:259) at org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:232) at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:159) at org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:151) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager.onKernalStart(GridCachePartitionExchangeManager.java:632) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.onKernalStart(GridCacheProcessor.java:829) at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1043) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1973) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1716) at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1144) at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:642) at org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:849) at org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:798) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapIndexScanTest.beforeTestsStarted(IgniteCacheOffheapIndexScanTest.java:78) at org.apache.ignite.testframework.junits.GridAbstractTest.setUp(GridAbstractTest.java:599) at org.apache.ignite.testframework.junits.common.GridCommonAbstractTest.setUp(GridCommonAbstractTest.java:485) at junit.framework.TestCase.runBare(TestCase.java:139) at junit.framework.TestResult$1.protect(TestResult.java:122) at junit.framework.TestResult.runProtected(TestResult.java:142) at junit.framework.TestResult.run(TestResult.java:125) at junit.framework.TestCase.run(TestCase.java:129) at junit.framework.TestSuite.runTest(TestSuite.java:255) at junit.framework.TestSuite.run(TestSuite.java:250) at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:84) at org.junit.runner.JUnitCore.run(JUnitCore.java:160) at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68) at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47) at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242) at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70) Caused by: java.lang.NullPointerException at org.apache.ignite.internal.processors.cache.CacheGroupContext.(CacheGroupContext.java:207) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCacheGroup(GridCacheProcessor.java:1983) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1881) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCachesOnLocalJoin(GridCacheProcessor.java:1773) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.initCachesOnLocalJoin(GridDhtPartitionsExchangeFuture.java:784) at org.apache.ignite.internal.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartitionsExchangeFuture.java:666) at org.apache.ignite.internal.processors.cache.GridCachePartitionExchangeManager$ExchangeWorker.body(GridCachePartitionExchangeManager.java:2344) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) at java.lang.Thread.run(Thread.java:748) {code} If assertions are enabled (-ea), an AssertionError is thrown instead: {code} java.lang.AssertionError at org.apache.ignite.internal.processors.cache.CacheGroupContext.(CacheGroupContext.java:185) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.startCacheGroup(GridCacheProcessor.java:1983) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1881) at org.apache.ignite.internal.processors.ca
[jira] [Created] (IGNITE-8210) Rebalancing sometimes is not triggered when a node is added to the baseline topology
Stanislav Lukyanov created IGNITE-8210: -- Summary: Rebalancing sometimes is not triggered when a node is added to the baseline topology Key: IGNITE-8210 URL: https://issues.apache.org/jira/browse/IGNITE-8210 Project: Ignite Issue Type: Bug Components: persistence Reporter: Stanislav Lukyanov When a cluster already has baseline topology set and activated adding a new node to the baseline should lead to rebalancing. However, in some cases the new node will not receive any data when joined and added to the baseline; if a cluster is later deactivated and activated again, the rebalancing happens properly and partitions are assigned and transferred to the new node. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8134) Services can't be deployed on servers outside of baseline topology
Stanislav Lukyanov created IGNITE-8134: -- Summary: Services can't be deployed on servers outside of baseline topology Key: IGNITE-8134 URL: https://issues.apache.org/jira/browse/IGNITE-8134 Project: Ignite Issue Type: Bug Components: managed services, persistence Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov If a node is not a part of the baseline topology, the services will never be deployed on it. In particular, if that node calls a synchronous deploy* method, the method will hang. After the node is added to the baseline, all previously initiated deployments succeed (and deploy* methods return). It seems that the issue is with the continuous query started by the GridServiceProcessor on the ignite-sys-cache. Example: = public class BltServicesBug { public static void main(String[] args) { // start one node IgniteConfiguration cfg1 = new IgniteConfiguration() .setIgniteInstanceName("node1") .setDataStorageConfiguration( new DataStorageConfiguration() .setDefaultDataRegionConfiguration( new DataRegionConfiguration() .setPersistenceEnabled(true) ) ); try (Ignite ignite1 = Ignition.start(cfg1)) { // activate and set baseline topology ignite1.cluster().active(true); // start another node IgniteConfiguration cfg2 = new IgniteConfiguration(cfg1) .setIgniteInstanceName("node2"); try (Ignite ignite2 = Ignition.start(cfg2)) { // try to deploy a service; // this call hangs until the second node is added to the BLT (e.g. externally via control.sh) ignite2.services().deployNodeSingleton("myService", new MyServiceImpl()); System.out.println("> Deployed"); } } } private static class MyServiceImpl implements Service { @Override public void cancel(ServiceContext ctx) { } @Override public void init(ServiceContext ctx) { } @Override public void execute(ServiceContext ctx) { } } } = -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8133) Baseline topology documentation improvement
Stanislav Lukyanov created IGNITE-8133: -- Summary: Baseline topology documentation improvement Key: IGNITE-8133 URL: https://issues.apache.org/jira/browse/IGNITE-8133 Project: Ignite Issue Type: Bug Components: documentation Affects Versions: 2.4 Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov Baseline topology concept was added to Ignite in 2.4 by IEP-4. This changed Ignite cluster behavior when persistence is enabled (first of all, activation and rebalancing timings). It seems that the current documentation may be confusing. For example, the sentence {quote}Note that the baseline topology is not set when the cluster is started for the first time; that's the only time when a manual intervention is needed.{quote} may lead one to thinking that baseline topology is not used by default and needs to be enabled only if one wants to use it. Also, the documentation describes the tools and commands that are used to manage the baseline topology and activation, but doesn't give guidelines on which nodes should be in the topology, when should it be changed, etc. The documentation should be enhanced to - give clear understanding that baseline topology always needs to be considered as a part of the cluster architecture when persistence is enabled; - provide overview of the behavioral changes compared to AI 2.3 (use a note/warning block for that to separate it from the main text?); - provide basic guidelines and suggestions of how one can start a new cluster and manage it (when to activate/deactivate, when to change baseline topology, what happens and what needs to be done when a node fails or joins, how to use consistentId) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8023) Processors are incorrectly initialized if during a node's reconnection cluster changes its state
Stanislav Lukyanov created IGNITE-8023: -- Summary: Processors are incorrectly initialized if during a node's reconnection cluster changes its state Key: IGNITE-8023 URL: https://issues.apache.org/jira/browse/IGNITE-8023 Project: Ignite Issue Type: Bug Affects Versions: 2.4, 2.3, 2.5 Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov If a node reconnects during the cluster activation process then some of the GridProcessor instances of that node will be incorrectly initialized. While GridClusterStateProcessor will correctly report the active cluster state, other processors that are sensitive to the cluster state, e.g. GridServiceProcessor, will be not initialized. This is a problem similar to IGNITE-7753, except that the timing (and the code that should handle this) is different. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7908) NULLS LAST works incorrectly for PARTITIONED caches
Stanislav Lukyanov created IGNITE-7908: -- Summary: NULLS LAST works incorrectly for PARTITIONED caches Key: IGNITE-7908 URL: https://issues.apache.org/jira/browse/IGNITE-7908 Project: Ignite Issue Type: Bug Components: sql Reporter: Stanislav Lukyanov When a `SELECT ORDER BY ... NULLS LAST` query is executed on a partitioned cache, the null values at the end of the result are interleaved with non-null values. It seems that for each node participating in the query all null values that are stored on that node are returned immediately after the last non-null value. As a result, non-null values are correctly ordered and null values appear in a few "portions" (the number of "portions" is equal to the number of nodes) among the last values. Example: {code:java} public class NullsLastBug { public static void main(String[] args) throws Exception { try ( Ignite srv1 = IgnitionEx.start("examples/config/example-ignite.xml", "server-1"); Ignite srv2 = IgnitionEx.start("examples/config/example-ignite.xml", "server-2"); Ignite srv3 = IgnitionEx.start("examples/config/example-ignite.xml", "server-3") ) { Ignition.setClientMode(true); try (Ignite client = IgnitionEx.start("examples/config/example-ignite.xml", "client")) { IgniteCache cache = client.getOrCreateCache( new CacheConfiguration("cache") .setCacheMode(CacheMode.PARTITIONED) .setIndexedTypes(Long.class, Box.class) ); for (long i = 0; i < 30; i++) { Long num = (i % 2 != 0) ? i : null; cache.put(i, new Box(num)); } SqlFieldsQuery selectAll = new SqlFieldsQuery("SELECT num FROM Box ORDER BY num NULLS LAST"); for (List list : cache.query(selectAll)) System.out.println(list); } } } } class Box implements Serializable { @QuerySqlField(index = true) private Long num; Box(Long num) { this.num = num; } } {code} Output: {code} [1] [3] [5] [7] [9] [11] [13] [15] [17] [19] [21] [23] [null] [null] [25] [27] [null] [null] [null] [null] [null] [null] [null] [null] [29] [null] [null] [null] [null] [null] {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7904) ComputeTaskFuture.get() throws incorrect exception if ComputeTask.result()
Stanislav Lukyanov created IGNITE-7904: -- Summary: ComputeTaskFuture.get() throws incorrect exception if ComputeTask.result() Key: IGNITE-7904 URL: https://issues.apache.org/jira/browse/IGNITE-7904 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov ComputeTask.result() javadoc says: "Throws: IgniteException - If handling a job result caused an error effectively rejecting a failover. This exception will be thrown out of ComputeTaskFuture.get() method." However, GridFutureAdapter calls IgniteUtils.cast(Throwable) on the exception before throwing it from get(), and the latter method trims the stack trace to the first occurence of an IgniteCheckedException. Because of that, get() throws not the IgniteException thrown from the ComputeTask.result() but one of its causes. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7771) Names of Ignite JMX beans should not be quoted unless required
Stanislav Lukyanov created IGNITE-7771: -- Summary: Names of Ignite JMX beans should not be quoted unless required Key: IGNITE-7771 URL: https://issues.apache.org/jira/browse/IGNITE-7771 Project: Ignite Issue Type: Bug Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov Names of Ignite JMX beans and bean groups are currently quoted if they contain non-alphanumeric characters. This leads to names with spaces, e.g. Thread Pools, appearing as "Thread Pools". Moreover, Thread Pools and "Thread Pools" are recognized by JMX as distinct names, so code accessing that MBean needs to take that into account. It would be better not to quote the bean and group names unless they contain specific control characters. That way users won't need to quote names in their JMX clients. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7753) Processors are incorrectly initialized if a node joins during cluster activation
Stanislav Lukyanov created IGNITE-7753: -- Summary: Processors are incorrectly initialized if a node joins during cluster activation Key: IGNITE-7753 URL: https://issues.apache.org/jira/browse/IGNITE-7753 Project: Ignite Issue Type: Bug Affects Versions: 2.3, 2.4, 2.5 Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov If a node joins during the cluster activation process (while the related exchange operation is in progress), then some of the GridProcessor instances of that node will be incorrectly initialized. While GridClusterStateProcessor will correctly report the active cluster state, other processors that are sensitive to the cluster state, e.g. GridServiceProcessor, will be not initialized. A reproducer is below. === Ignite server = IgnitionEx.start("examples/config/persistentstore/example-persistent-store.xml", "server"); CyclicBarrier barrier = new CyclicBarrier(2); Thread activationThread = new Thread(() -> { try { barrier.await(); server.active(true); } catch (Exception e) { e.printStackTrace(); // TODO implement. } }); activationThread.start(); barrier.await(); IgnitionEx.setClientMode(true); Ignite client = IgnitionEx.start("examples/config/persistentstore/example-persistent-store.xml", "client"); activationThread.join(); client.services().deployClusterSingleton("myClusterSingleton", new SimpleMapServiceImpl<>()); === Here a single server node is started, then simultaneously a client node is being started and the cluster is being activated, then client attempts to deploy a service. As the result, the thread calling the deploy method hangs forever with a stack trace like this: === "main@1" prio=5 tid=0x1 nid=NA waiting java.lang.Thread.State: WAITING at sun.misc.Unsafe.park(Unsafe.java:-1) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304) at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231) at org.apache.ignite.internal.util.IgniteUtils.awaitQuiet(IgniteUtils.java:7505) at org.apache.ignite.internal.processors.service.GridServiceProcessor.serviceCache(GridServiceProcessor.java:290) at org.apache.ignite.internal.processors.service.GridServiceProcessor.writeServiceToCache(GridServiceProcessor.java:728) at org.apache.ignite.internal.processors.service.GridServiceProcessor.deployAll(GridServiceProcessor.java:634) at org.apache.ignite.internal.processors.service.GridServiceProcessor.deployAll(GridServiceProcessor.java:600) at org.apache.ignite.internal.processors.service.GridServiceProcessor.deployMultiple(GridServiceProcessor.java:488) at org.apache.ignite.internal.processors.service.GridServiceProcessor.deployClusterSingleton(GridServiceProcessor.java:469) at org.apache.ignite.internal.IgniteServicesImpl.deployClusterSingleton(IgniteServicesImpl.java:120) === The behavior depends on the timings - the client has to join in the middle of the activation's exchange process. Putting Thread.sleep(4000) into GridDhtPartitionsExchangeFuture.onClusterStateChangeRequest seems to work on a development laptop. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7687) SQL SELECT doesn't update TTL for Touched/AccessedExpiryPolicy
Stanislav Lukyanov created IGNITE-7687: -- Summary: SQL SELECT doesn't update TTL for Touched/AccessedExpiryPolicy Key: IGNITE-7687 URL: https://issues.apache.org/jira/browse/IGNITE-7687 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.5 Reporter: Stanislav Lukyanov SQL SELECT queries don't update TTLs when TouchedExpiryPolicy or AccessedExpiryPolicy is used (unlike IgniteCache::get which does update the TTLs). Example (modified SqlDmlExample): CacheConfiguration orgCacheCfg = new CacheConfiguration(ORG_CACHE) .setIndexedTypes(Long.class, Organization.class) .setExpiryPolicyFactory(TouchedExpiryPolicy.factoryOf(new Duration(TimeUnit.SECONDS, 10))); IgniteCache orgCache = ignite.getOrCreateCache(orgCacheCfg); SqlFieldsQuery qry = new SqlFieldsQuery("insert into Organization (_key, id, name) values (?, ?, ?)"); orgCache.query(qry.setArgs(1L, 1L, "ASF")); orgCache.query(qry.setArgs(2L, 2L, "Eclipse")); SqlFieldsQuery qry1 = new SqlFieldsQuery("select id, name from Organization as o"); for (int i = 0; ;i++) { List> res = orgCache.query(qry1).getAll(); print("i = " + i); for (Object next : res) System.out.println(">>> " + next); U.sleep(5000); } Output: >>> i = 0 >>> [1, ASF] >>> [2, Eclipse] >>> i = 1 >>> [1, ASF] >>> [2, Eclipse] >>> i = 2 >>> i = 3 ... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7611) Document logger configuration options
Stanislav Lukyanov created IGNITE-7611: -- Summary: Document logger configuration options Key: IGNITE-7611 URL: https://issues.apache.org/jira/browse/IGNITE-7611 Project: Ignite Issue Type: Bug Components: documentation Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov Existing documentation on readme.io (https://apacheignite.readme.io/docs/logging) gives details on how to enable each of the supported logging frameworks, and some info on the default configuration (e.g. IGNITE_QUIET). The documentation should also cover some other features of Ignite logging, such as recently added features of automatic reconfiguration of log4j 1.x and 2.x (IGNITE-6946) and DEV_ONLY marker in logging messages (IGNITE-7284), as well as other features like automatic metrics logging (MetricsLogFrequency) and performance suggestions on start (IGNITE_PERFORMANCE_SUGGESTIONS_DISABLED). -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7508) GridKernalContextImpl::isDaemon creates contention on system properties access
Stanislav Lukyanov created IGNITE-7508: -- Summary: GridKernalContextImpl::isDaemon creates contention on system properties access Key: IGNITE-7508 URL: https://issues.apache.org/jira/browse/IGNITE-7508 Project: Ignite Issue Type: Bug Components: general Reporter: Stanislav Lukyanov GridKernalContextImpl::isDaemon reads system property IGNITE_DAEMON on every call, leading to contention on the system properties lock. The lock is shown as contended in the Java Mission Control analysis of a JFR recording of the IgnitePutGetBenchmark. The fix would be to cache IGNITE_DAEMON value (e.g. in IgniteUtils) since it isn't supposed to be changed during the JVM's lifetime anyway. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7398) InstanceNotFoundException appears in the log when a cache is closed and MBeans are disabled
Stanislav Lukyanov created IGNITE-7398: -- Summary: InstanceNotFoundException appears in the log when a cache is closed and MBeans are disabled Key: IGNITE-7398 URL: https://issues.apache.org/jira/browse/IGNITE-7398 Project: Ignite Issue Type: Bug Affects Versions: 2.4 Reporter: Stanislav Lukyanov Priority: Minor If IGNITE_MBEANS_DISABLED is set to true, whenever a cache is started the following error is logged: javax.management.InstanceNotFoundException: org.apache:clsLdr=18b4aac2,igniteInstanceName=mbeans.GridMBeanDisableSelfTest0,group=Cache groups,name=MyCache at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.getMBean(DefaultMBeanServerInterceptor.java:1095) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.exclusiveUnregisterMBean(DefaultMBeanServerInterceptor.java:427) at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.unregisterMBean(DefaultMBeanServerInterceptor.java:415) at com.sun.jmx.mbeanserver.JmxMBeanServer.unregisterMBean(JmxMBeanServer.java:546) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.cleanup(GridCacheProcessor.java:606) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2264) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCacheGroup(GridCacheProcessor.java:2253) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.stopCaches(GridCacheProcessor.java:907) at org.apache.ignite.internal.processors.cache.GridCacheProcessor.stop(GridCacheProcessor.java:872) at org.apache.ignite.internal.IgniteKernal.stop0(IgniteKernal.java:2316) at org.apache.ignite.internal.IgniteKernal.stop(IgniteKernal.java:2182) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop0(IgnitionEx.java:2512) at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.stop(IgnitionEx.java:2475) at org.apache.ignite.internal.IgnitionEx.stop(IgnitionEx.java:362) at org.apache.ignite.Ignition.stop(Ignition.java:224) at org.apache.ignite.internal.IgniteKernal.close(IgniteKernal.java:3509) at org.apache.ignite.util.mbeans.GridMBeanDisableSelfTest.testCorrectMBeanInfo(GridMBeanDisableSelfTest.java:101) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at junit.framework.TestCase.runTest(TestCase.java:176) at org.apache.ignite.testframework.junits.GridAbstractTest.runTestInternal(GridAbstractTest.java:2001) at org.apache.ignite.testframework.junits.GridAbstractTest.access$000(GridAbstractTest.java:133) at org.apache.ignite.testframework.junits.GridAbstractTest$5.run(GridAbstractTest.java:1916) at java.lang.Thread.run(Thread.java:748) That's because GridCacheProcessor::cleanup unconditionally attempts to unregister cache's MBean, not checking whether or not the cache actually has an MBean registered for it. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7334) MBeans fail to be created for caches with special symbols in names
Stanislav Lukyanov created IGNITE-7334: -- Summary: MBeans fail to be created for caches with special symbols in names Key: IGNITE-7334 URL: https://issues.apache.org/jira/browse/IGNITE-7334 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.4 Reporter: Stanislav Lukyanov Assignee: Stanislav Lukyanov If a cache contains a certain symbols, e.g. an asterisk, its creation will fail due to an exception during MBean creation. This is because of insufficient escaping in the shared MBean registration code in IgniteUtils. -- This message was sent by Atlassian JIRA (v6.4.14#64029)