[jira] [Created] (IGNITE-14100) GridCachePartitionedNodeRestartTest fails due to wrong tx mapping
Alexander Lapin created IGNITE-14100: Summary: GridCachePartitionedNodeRestartTest fails due to wrong tx mapping Key: IGNITE-14100 URL: https://issues.apache.org/jira/browse/IGNITE-14100 Project: Ignite Issue Type: Bug Reporter: Alexander Lapin Assignee: Alexander Lapin Sometimes GridCachePartitionedNodeRestartTest#testRestartWithTxEightNodesTwoBackups hangs with the following AssertionError: {code:java} [18:06:54]W: [org.gridgain:ignite-core] java.lang.AssertionError: cacheId=-838655627, localNode = c2156a8f-1ac7-4098-8808-b1f9fad5, dhtNodes = [TcpDiscoveryNode [id=51df7fb1-2cad-4bfc-a02e-8225fbc0, consistentId=127.0.0.1:47500, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47500], discPort=47500, order=1, intOrder=1, lastExchangeTime=1603552013349, loc=false, ver=8.8.127#20201023-sha1:8f62caf2, isClient=false], TcpDiscoveryNode [id=c2156a8f-1ac7-4098-8808-b1f9fad5, consistentId=127.0.0.1:47504, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47504], discPort=47504, order=21, intOrder=13, lastExchangeTime=1603552013259, loc=true, ver=8.8.127#20201023-sha1:8f62caf2, isClient=false], TcpDiscoveryNode [id=f7355089-fdea-4a96-af57-00ebeb94, consistentId=127.0.0.1:47505, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47505], discPort=47505, order=22, intOrder=14, lastExchangeTime=1603552013390, loc=false, ver=8.8.127#20201023-sha1:8f62caf2, isClient=false], TcpDiscoveryNode [id=4e8a8c17-e1c7-43e2-8461-9b5348d3, consistentId=127.0.0.1:47503, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47503], discPort=47503, order=4, intOrder=4, lastExchangeTime=1603552013349, loc=false, ver=8.8.127#20201023-sha1:8f62caf2, isClient=false], TcpDiscoveryNode [id=d4f66e72-4ecc-4aca-a6d1-91242331, consistentId=127.0.0.1:47501, addrs=ArrayList [127.0.0.1], sockAddrs=HashSet [/127.0.0.1:47501], discPort=47501, order=2, intOrder=2, lastExchangeTime=1603552013349, loc=false, ver=8.8.127#20201023-sha1:8f62caf2, isClient=false]] [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.map(GridDhtTxPrepareFuture.java:1669) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare0(GridDhtTxPrepareFuture.java:1355) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.mapIfLocked(GridDhtTxPrepareFuture.java:732) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:1125) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.prepareAsync(GridDhtTxLocal.java:410) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:591) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:388) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest0(IgniteTxHandler.java:197) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest(IgniteTxHandler.java:174) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$000(IgniteTxHandler.java:134) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:219) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:217) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1142) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:591) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:392) [18:06:54]W: [org.gridgain:ignite-core] at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:318)
[jira] [Created] (IGNITE-14099) CPP: Remove 32-bit configs
Igor Sapego created IGNITE-14099: Summary: CPP: Remove 32-bit configs Key: IGNITE-14099 URL: https://issues.apache.org/jira/browse/IGNITE-14099 Project: Ignite Issue Type: Bug Components: platforms Affects Versions: 2.9.1 Reporter: Igor Sapego Assignee: Igor Sapego Fix For: 2.11 32-bit test config files are not needed anymore, as we do not run them anyway. Remove them. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14098) ignite reentrant lock is not in user facing docs
Scott Feldstein created IGNITE-14098: Summary: ignite reentrant lock is not in user facing docs Key: IGNITE-14098 URL: https://issues.apache.org/jira/browse/IGNITE-14098 Project: Ignite Issue Type: Bug Components: documentation Reporter: Scott Feldstein Hi, [Ignite Reentrant lock|https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/Ignite.html#reentrantLock-java.lang.String-boolean-boolean-boolean-] is not part of the user facing documentation in [https://ignite.apache.org/]. It would be extremely useful if this were added and included when to use reentrant lock vs cache lock vs semaphore -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSSION] Apache Ignite Release 2.10 (time, scope, manager)
Hello Nikita, Thank you for sharing the state. I think we can move on to the next stages of the release. There are a few important steps left to do: release_notes, stress testing. This may take some time, so we still have time to complete the rest of the documentation tasks. On Thu, 28 Jan 2021 at 02:51, Никита Сафонов wrote: > > Hi Maxim et al, > > thank you! > > The rest of the documentation tasks for important features are also > completed. > > Nevertheless, there are still some items (the ones on my list that do not > have any doc tasks) that can be done for the 2.10 release. > If you want them to be included, I will try to provide patches ASAP. If > not, I am still ready to create the documentation for them and provide it > later. > > So, I will fully rely on your decision. > > Regards, > > Nikita > > вт, 26 янв. 2021 г. в 23:08, Maxim Muzafarov : > > > Nikita, Folks, > > > > All documentation tasks from the *[list #2]* [1] that you mentioned > > above have been processed and merged to the release branch. > > > > [1] > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Apache-Ignite-Release-2-10-time-scope-manager-tp50037p51061.html > > > > On Mon, 25 Jan 2021 at 18:59, Maxim Muzafarov wrote: > > > > > > Shiva, > > > > > > I've cherry-picked the issue to 2.10 branch. > > > https://issues.apache.org/jira/browse/IGNITE-13912 > > > > > > Ilya, > > > > > > Thank you for preparing the patch. I've cherry-picked to 2.10 it too. > > > https://issues.apache.org/jira/browse/IGNITE-14039 > > > > > > On Mon, 25 Jan 2021 at 17:50, Ilya Kasnacheev > > wrote: > > > > > > > > Hello! > > > > > > > > I have pushed the developer warning, javadoc warning and documentation > > > > update to https://issues.apache.org/jira/browse/IGNITE-14039 about the > > > > dangers of WAL state change. > > > > > > > > Please cherry-pick it to 2.10 branch or mention me if it's OK for me > > to do > > > > this. > > > > > > > > Regards, > > > > -- > > > > Ilya Kasnacheev > > > > > > > > > > > > пн, 25 янв. 2021 г. в 14:44, Никита Сафонов > >: > > > > > > > > > Hi Denis, > > > > > > > > > > Thanks a lot! > > > > > I'll update my PR's accordingly. > > > > > > > > > > Regards, > > > > > Nikita > > > > > > > > > > сб, 23 янв. 2021 г. в 00:59, Denis Magda : > > > > > > > > > > > Nikita, thanks. I reviewed those and shared some suggestions. > > Please take > > > > > > them into consideration. > > > > > > > > > > > > - > > > > > > Denis > > > > > > > > > > > > > > > > > > On Fri, Jan 22, 2021 at 1:45 PM Никита Сафонов < > > > > > vlasovpavel2...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > Hi everyone, > > > > > > > > > > > > > > The following documentation tasks for the important features now > > have > > > > > > PATCH > > > > > > > AVAILABLE status: > > > > > > > > > > > > > >- Caches warming up strategy - > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-13385?src=confmacro > > > > > > >- Update on JMX default configuration removal - > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-13606?src=confmacro > > > > > > >- Control.sh indexes manipulation commands - > > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-13285?src=confmacro > > > > > > > > > > > > > > Please see the PR's attached to the tickets. > > > > > > > > > > > > > > Thank you! > > > > > > > > > > > > > > Regards, > > > > > > > Nikita > > > > > > > > > > > > > > пт, 22 янв. 2021 г. в 18:03, shm : > > > > > > > > > > > > > > > Hi All, > > > > > > > > Can you please also include a critical ticket > > > > > > > > https://issues.apache.org/jira/browse/IGNITE-13912 > > > > > > > > to 2.10 In terms of functionality it is a blocker. Still some > > > > > > debugging > > > > > > > is > > > > > > > > going on related to this issue. > > > > > > > > > > > > > > > > Thanks, > > > > > > > > Shiva > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Sent from: > > http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > > > > > > > > > > > > > > > > > > > > > > >
[jira] [Created] (IGNITE-14097) Fix checkstyle plugin configuration
Semyon Danilov created IGNITE-14097: --- Summary: Fix checkstyle plugin configuration Key: IGNITE-14097 URL: https://issues.apache.org/jira/browse/IGNITE-14097 Project: Ignite Issue Type: Improvement Affects Versions: 3.0.0-alpha2 Reporter: Semyon Danilov Assignee: Semyon Danilov -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Request of TC permissions
It seems that we need full set of tests for extensions. I will refactor that project in couple of days and add suite for your extension too. > On 28 Jan 2021, at 16:34, Mikhail Petrov wrote: > > Petr, I planned to copy the existing ignite-extensions build configuration > and change its parameters [1] to add a new module and test-suite. > > It turned out that build configuration parameters can be overridden for each > particular build without additional permissions or copying something. > > Sorry for bothering you. > > On 28.01.2021 16:17, Petr Ivanov wrote: >> Hi, Mikhail. >> >> >> Can you please describe what exactly do you what to achieve with the build — >> I will help with it. >> >> Seems that currently we have no test suites for extensions at all [1] >> >> >> >> https://ci.ignite.apache.org/project/IgniteExtensions_Tests?mode=builds >> >>> On 28 Jan 2021, at 15:34, Mikhail Petrov wrote: >>> >>> Igniters, >>> >>> I am currently working on the migration of the Spring Transactions >>> integration to a separate ignite-extensions module - [1]. I need to create >>> a debug ignite-extensions build on TC with the migrated test suite >>> included, making sure the new tests are ok. Can anyone help me get the >>> required TC permissions while I work on this issue? >>> >>> Regards, >>> Mikhail. >>> >>> [1] - https://issues.apache.org/jira/browse/IGNITE-13992 >>>
Re: Re[2]: [DISCUSSION] Request for thread unsafe Compute functionality deprecation.
Hello! Please publish it. I don't see why not. Regards, -- Ilya Kasnacheev чт, 28 янв. 2021 г. в 14:28, Zhenya Stanilovsky : > > > Hi Ilya , of course it contains in my PR (i don`t know if it shout be > published before this discussion will be finished). > Little changes from single thread into multiple, for example here [1] will > highlight a problem, or i can just publish my PR. > > [1] > https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/IgniteExplicitImplicitDeploymentSelfTest.java#L221 > > > > >> > >>>Hello! > >>> > >>>Do you have some kind of reproducer which demonstrates the issue? > >>> > >>>Regards, > >>>-- > >>>Ilya Kasnacheev > >>> > >>> > >>>чт, 28 янв. 2021 г. в 10:32, Zhenya Stanilovsky < > arzamas...@mail.ru.invalid > : > >>> > > Hello Igniters ! > In the process of Ignite usage i found that some part of Compute > functionality are thread unsafe and seems was designed with such > limitations initially. > Example : one (client, but it doesn`t matter at all) instance is > shared between numerous of fabric, all of them calls something like : > IgniteCompute#execute(ComputeTask, T) > or > IgniteCompute#execute(java.lang.Class>, T) > and appropriate «async» methods — what kind of instance will be > called is > nondeterministic for now and as a confirmation of my words — i found > no > tests covered multi thread usage of Computing i also found nothing on > documentation page [1]. > We have all necessary info for correct processing of such cases: > from initiator (ignite.compute(...) starter) side we have Class or it > instance and appropriate class loader which will be wired by class > loader > id from execution side. > I create a fix and seems all work perfectly well besides one place, > this > functionality : > /** > * Executes given task within the cluster group. For step-by-step > explanation of task execution process > * refer to {@link ComputeTask} documentation. > * > * If task for given name has not been deployed yet, then {@code > taskName} > will be > * used as task class name to auto-deploy the task (see {@link > #localDeployTask(Class, ClassLoader)} method). > */ > public R execute(String taskName, T arg) throws > IgniteException; > and attendant > /** > * Finds class loader for the given class. > * > * @param rsrcName Class name or class alias to find class loader for. > * @return Deployed class loader, or {@code null} if not deployed. > */ > public DeploymentResource findResource(String rsrcName); > is thread unsafe by default, no guarantee that concurrent call of > localDeployTask and execute will bring expected result. > My proposal is to deprecate (or probably annotate [2], as a minimal > — additionally document it) this methods and to append additional : > public DeploymentResource findResource(String rsrcName, ClassLoader > clsLdr); > Only one problem i can observe here, if someone creates new class > loaders > and appropriate class instances in loop (i don`t know the purpose) and > doesn`t undeploy them then he will get possibly OOM here. > > Such approach will give a possibility to use compute in concurrent > scenario. If there is no objections here i will mark this methods and > publish my PR, of course with additional tests. > > What do you think ? > > > [1] > > https://ignite.apache.org/docs/latest/code-deployment/peer-class-loading > [2] > > https://jcip.net/annotations/doc/net/jcip/annotations/NotThreadSafe.html > > > >> > >> > >> > >>
[jira] [Created] (IGNITE-14096) Try to bring randomization in node waiting with TcpDiscoverySpi.reconnectDelay.
Vladimir Steshin created IGNITE-14096: - Summary: Try to bring randomization in node waiting with TcpDiscoverySpi.reconnectDelay. Key: IGNITE-14096 URL: https://issues.apache.org/jira/browse/IGNITE-14096 Project: Ignite Issue Type: Improvement Reporter: Vladimir Steshin Assignee: Vladimir Steshin To speed up cluster start slyghtly, try to bring randomization in node waiting with TcpDiscoverySpi.reconnectDelay. Check with the ducktape integration tests. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14095) Try fasten cluster start in the ducktests with decreasing 'spi.reconnectDelay'
Vladimir Steshin created IGNITE-14095: - Summary: Try fasten cluster start in the ducktests with decreasing 'spi.reconnectDelay' Key: IGNITE-14095 URL: https://issues.apache.org/jira/browse/IGNITE-14095 Project: Ignite Issue Type: Improvement Reporter: Vladimir Steshin Assignee: Vladimir Steshin -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14094) Configuration storage interface
Semyon Danilov created IGNITE-14094: --- Summary: Configuration storage interface Key: IGNITE-14094 URL: https://issues.apache.org/jira/browse/IGNITE-14094 Project: Ignite Issue Type: Sub-task Reporter: Semyon Danilov Assignee: Semyon Danilov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14093) ttl-cleanup-worker falls with AssertionError and leads to CorruptiedTreeException
Mirza Aliev created IGNITE-14093: Summary: ttl-cleanup-worker falls with AssertionError and leads to CorruptiedTreeException Key: IGNITE-14093 URL: https://issues.apache.org/jira/browse/IGNITE-14093 Project: Ignite Issue Type: Bug Affects Versions: 2.9.1 Reporter: Mirza Aliev Assignee: Mirza Aliev Attachments: IgnitePdsWithTtlDeferredDeleteOnRestartTest (1).java This issue is very rare, it's quite hard to reproduce on mac, some windows users reproduced it a bit often Scenario: # 2 baseline nodes, cache with expiry policy = 60 sec. # Put some entries in the cache, stop one node immediately. # Remove node from baseline. # Wait until expiration. # Start the stopped node — NPE on node start. {code:java} [2020-05-08 16:07:17,925][ERROR][ttl-cleanup-worker-#43][root] Critical system error detected. Will be handled accordingly to configured handler [hnd=StopNodeOrHaltFailureHandler [tryStop=false, timeout=0, super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=SYSTEM_WORKER_TERMINATION, err=java.lang.NullPointerException]] java.lang.NullPointerException at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2765) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2696) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:1073) at org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:242) at org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.lambda$body$0(GridCacheSharedTtlCleanupManager.java:178) at java.util.concurrent.ConcurrentHashMap.computeIfPresent(ConcurrentHashMap.java:1769) at org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:177) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119) at java.lang.Thread.run(Thread.java:748) {code} In some cases, it is possible to get this stacktrace {code:java} [2020-05-25 10:49:29,677][ERROR][ttl-cleanup-worker-#242%db.IgnitePdsWithTtlDeferredDeleteOnRestartTest2%][IgniteTestResources] Critical system error detected. Will be handled accordingly to configured handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler [ignoredFailureTypes=UnmodifiableSet [SYSTEM_WORKER_BLOCKED, SYSTEM_CRITICAL_OPERATION_TIMEOUT]]], failureCtx=FailureContext [type=CRITICAL_ERROR, err=class o.a.i.i.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple [val1=-1237460590, val2=0]], groupName=group1, msg=Runtime failure on bounds: [lower=PendingRow [], upper=PendingRow [] class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple [val1=-1237460590, val2=0]], groupName=group1, msg=Runtime failure on bounds: [lower=PendingRow [], upper=PendingRow []]] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:6110) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1119) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1083) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.find(BPlusTree.java:1078) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpiredInternal(GridCacheOffheapManager.java:2742) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.purgeExpired(GridCacheOffheapManager.java:2696) at org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager.expire(GridCacheOffheapManager.java:1073) at org.apache.ignite.internal.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:242) at org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.lambda$body$0(GridCacheSharedTtlCleanupManager.java:178) at java.util.concurrent.ConcurrentHashMap.computeIfPresent(ConcurrentHashMap.java:1769) at org.apache.ignite.internal.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:177) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119
[jira] [Created] (IGNITE-14092) Design network address resolver
Anton Kalashnikov created IGNITE-14092: -- Summary: Design network address resolver Key: IGNITE-14092 URL: https://issues.apache.org/jira/browse/IGNITE-14092 Project: Ignite Issue Type: Sub-task Reporter: Anton Kalashnikov It needs to design network address resolver/ip finder/discovery which would help to choose the right ip/port for connection. Perhaps we don't need such a service at all but it should be explicitly agreed. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14091) Implement messaging service
Anton Kalashnikov created IGNITE-14091: -- Summary: Implement messaging service Key: IGNITE-14091 URL: https://issues.apache.org/jira/browse/IGNITE-14091 Project: Ignite Issue Type: Sub-task Reporter: Anton Kalashnikov It needs to implement the ability to send/receive messages to/from network members: * there's a requirements of being able to send idempotent messages with very weak guarantees: ** no delivery guarantees required; ** multiple copies of the same message might be sent; ** no need to have any kind of acknowledgement; * there's another requirement for the common use: ** message must be sent exactly once with an acknowledgement that it has actually been received (not necessarily processed); ** messages must be received in the same order they were sent. These types of messages might utilize current recovery protocol with acks every 32 (or so) messages. This setting must be flexible enough so that we won't get OOM in big topologies. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14090) Networking API
Anton Kalashnikov created IGNITE-14090: -- Summary: Networking API Key: IGNITE-14090 URL: https://issues.apache.org/jira/browse/IGNITE-14090 Project: Ignite Issue Type: Sub-task Reporter: Anton Kalashnikov It needs to design convinient public API for networking module which allow to get information about network members and send/receive messages from them. Draft: {noformat} public interface NetworkService \{ static NetworkService create(NetworkConfiguration cfg); void shutdown() throws ???; NetworkMember localMember(); Collection remoteMembers(); void weakSend(NetworkMember member, Message msg); Future guaranteedSend(NetworkMember member, Message msg); void listenMembers(MembershipListener lsnr); void listenMessages(Consumer lsnr); } public interface MembershipListener \{ void onAppeared(NetworkMember member); void onDisappeared(NetworkMember member); void onAcceptedByGroup(List remoteMembers); } public interface NetworkMember \{ UUID id(); } {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14089) Override scalecube internal message by custom one
Anton Kalashnikov created IGNITE-14089: -- Summary: Override scalecube internal message by custom one Key: IGNITE-14089 URL: https://issues.apache.org/jira/browse/IGNITE-14089 Project: Ignite Issue Type: Sub-task Reporter: Anton Kalashnikov There is some custom logic in the networking module like a specific handshake, message recovery etc. which requires to have specific messages but at the same time default scalecube behaviour should be worked correctly. So it needs to implement one logic over another. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14088) Implement scalecube transport API over netty
Anton Kalashnikov created IGNITE-14088: -- Summary: Implement scalecube transport API over netty Key: IGNITE-14088 URL: https://issues.apache.org/jira/browse/IGNITE-14088 Project: Ignite Issue Type: Sub-task Reporter: Anton Kalashnikov scalecube has its own netty inside but it is idea to integrate our expanded netty into it. It will help us to support more features like our own handshake, marshalling etc. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Request of TC permissions
Petr, I planned to copy the existing ignite-extensions build configuration and change its parameters [1] to add a new module and test-suite. It turned out that build configuration parameters can be overridden for each particular build without additional permissions or copying something. Sorry for bothering you. On 28.01.2021 16:17, Petr Ivanov wrote: Hi, Mikhail. Can you please describe what exactly do you what to achieve with the build — I will help with it. Seems that currently we have no test suites for extensions at all [1] https://ci.ignite.apache.org/project/IgniteExtensions_Tests?mode=builds On 28 Jan 2021, at 15:34, Mikhail Petrov wrote: Igniters, I am currently working on the migration of the Spring Transactions integration to a separate ignite-extensions module - [1]. I need to create a debug ignite-extensions build on TC with the migrated test suite included, making sure the new tests are ok. Can anyone help me get the required TC permissions while I work on this issue? Regards, Mikhail. [1] - https://issues.apache.org/jira/browse/IGNITE-13992
Re: Request of TC permissions
Hi, Mikhail. Can you please describe what exactly do you what to achieve with the build — I will help with it. Seems that currently we have no test suites for extensions at all [1] https://ci.ignite.apache.org/project/IgniteExtensions_Tests?mode=builds > On 28 Jan 2021, at 15:34, Mikhail Petrov wrote: > > Igniters, > > I am currently working on the migration of the Spring Transactions > integration to a separate ignite-extensions module - [1]. I need to create a > debug ignite-extensions build on TC with the migrated test suite included, > making sure the new tests are ok. Can anyone help me get the required TC > permissions while I work on this issue? > > Regards, > Mikhail. > > [1] - https://issues.apache.org/jira/browse/IGNITE-13992 >
[jira] [Created] (IGNITE-14087) Implement code generation for interfaces introduced in IGNITE-14062
Ivan Bessonov created IGNITE-14087: -- Summary: Implement code generation for interfaces introduced in IGNITE-14062 Key: IGNITE-14087 URL: https://issues.apache.org/jira/browse/IGNITE-14087 Project: Ignite Issue Type: Sub-task Reporter: Ivan Bessonov Assignee: Ivan Bessonov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14086) Implement retry of establishing connection if it was lost
Anton Kalashnikov created IGNITE-14086: -- Summary: Implement retry of establishing connection if it was lost Key: IGNITE-14086 URL: https://issues.apache.org/jira/browse/IGNITE-14086 Project: Ignite Issue Type: Sub-task Reporter: Anton Kalashnikov It needs to implement a retry of establishing the connection. It is not clear which way is better to implement such idea because the current implementation too difficult to configure(number of retries, several properties of retry time). So it needs to think a better way to configure it. And it needs to be implementeded. Perhaps, scalecube(gossip protocol) do all work already and we should do nothing here. Need to recheck. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14085) Implement message recovery protocol over handshake
Anton Kalashnikov created IGNITE-14085: -- Summary: Implement message recovery protocol over handshake Key: IGNITE-14085 URL: https://issues.apache.org/jira/browse/IGNITE-14085 Project: Ignite Issue Type: Sub-task Reporter: Anton Kalashnikov The central idea of recovery protocol is the same as it is in the current implementation. So it needs to implement a similar idea with the recovery descriptor. This means information about last sending/received messages should be sent during the handshake and according to this information messages which were not received should be sent one more time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14084) Integrate direct marshalling to networking
Anton Kalashnikov created IGNITE-14084: -- Summary: Integrate direct marshalling to networking Key: IGNITE-14084 URL: https://issues.apache.org/jira/browse/IGNITE-14084 Project: Ignite Issue Type: Sub-task Reporter: Anton Kalashnikov Direct marshalling can be extracted from ignite2.x and integrate to ignite3.0. It helps to avoid extra data copy during the sending/receiving messages. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14083) Add SSL support to networking
Anton Kalashnikov created IGNITE-14083: -- Summary: Add SSL support to networking Key: IGNITE-14083 URL: https://issues.apache.org/jira/browse/IGNITE-14083 Project: Ignite Issue Type: Sub-task Reporter: Anton Kalashnikov It needs to add the ability to establish SSL connection. It looks like it should not be a problem. But at least, it needs to design configuration which allow to manage the ssl(path to certificate, password, etc.) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14082) Implementation of handshake for new connection
Anton Kalashnikov created IGNITE-14082: -- Summary: Implementation of handshake for new connection Key: IGNITE-14082 URL: https://issues.apache.org/jira/browse/IGNITE-14082 Project: Ignite Issue Type: Sub-task Reporter: Anton Kalashnikov It needs to implement the handshake after netty establish the connection. Perhaps, It makes sense to use netty handlers. During the handshake, It needs to exchange instanceId from one endpoint to another. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Request of TC permissions
Igniters, I am currently working on the migration of the Spring Transactions integration to a separate ignite-extensions module - [1]. I need to create a debug ignite-extensions build on TC with the migrated test suite included, making sure the new tests are ok. Can anyone help me get the required TC permissions while I work on this issue? Regards, Mikhail. [1] - https://issues.apache.org/jira/browse/IGNITE-13992
[jira] [Created] (IGNITE-14081) Networking module
Anton Kalashnikov created IGNITE-14081: -- Summary: Networking module Key: IGNITE-14081 URL: https://issues.apache.org/jira/browse/IGNITE-14081 Project: Ignite Issue Type: New Feature Reporter: Anton Kalashnikov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14080) binary-metadata-writer hung with no ignite instance
Mirza Aliev created IGNITE-14080: Summary: binary-metadata-writer hung with no ignite instance Key: IGNITE-14080 URL: https://issues.apache.org/jira/browse/IGNITE-14080 Project: Ignite Issue Type: Bug Affects Versions: 2.9.1 Reporter: Mirza Aliev If you start with no ignite instance, {{IgniteWalIteratorFactory}} creates BinaryMetadataFileStore which never stops by itself and hung on {{queue.take()}}. stacktrace: {code:java} "binary-metadata-writer-#1" #22 prio=5 os_prio=0 tid=0x7f2c0885b800 nid=0x2a3ac9 waiting on condition [0x7f2afc55b000] java.lang.Thread.State: WAITING (parking) at sun.misc.Unsafe.park(Native Method) - parking to wait for <0x000580ad7130> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) at java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) at org.apache.ignite.internal.processors.cache.binary.BinaryMetadataFileStore$BinaryMetadataAsyncWriter.body0(BinaryMetadataFileStore.java:362) at org.apache.ignite.internal.processors.cache.binary.BinaryMetadataFileStore$BinaryMetadataAsyncWriter.body(BinaryMetadataFileStore.java:343) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.lang.Thread.run(Thread.java:748) {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re[2]: [DISCUSSION] Request for thread unsafe Compute functionality deprecation.
Hi Ilya , of course it contains in my PR (i don`t know if it shout be published before this discussion will be finished). Little changes from single thread into multiple, for example here [1] will highlight a problem, or i can just publish my PR. [1] https://github.com/apache/ignite/blob/master/modules/core/src/test/java/org/apache/ignite/internal/IgniteExplicitImplicitDeploymentSelfTest.java#L221 > >> >>>Hello! >>> >>>Do you have some kind of reproducer which demonstrates the issue? >>> >>>Regards, >>>-- >>>Ilya Kasnacheev >>> >>> >>>чт, 28 янв. 2021 г. в 10:32, Zhenya Stanilovsky < arzamas...@mail.ru.invalid : >>> Hello Igniters ! In the process of Ignite usage i found that some part of Compute functionality are thread unsafe and seems was designed with such limitations initially. Example : one (client, but it doesn`t matter at all) instance is shared between numerous of fabric, all of them calls something like : IgniteCompute#execute(ComputeTask, T) or IgniteCompute#execute(java.lang.Class>, T) and appropriate «async» methods — what kind of instance will be called is nondeterministic for now and as a confirmation of my words — i found no tests covered multi thread usage of Computing i also found nothing on documentation page [1]. We have all necessary info for correct processing of such cases: from initiator (ignite.compute(...) starter) side we have Class or it instance and appropriate class loader which will be wired by class loader id from execution side. I create a fix and seems all work perfectly well besides one place, this functionality : /** * Executes given task within the cluster group. For step-by-step explanation of task execution process * refer to {@link ComputeTask} documentation. * * If task for given name has not been deployed yet, then {@code taskName} will be * used as task class name to auto-deploy the task (see {@link #localDeployTask(Class, ClassLoader)} method). */ public R execute(String taskName, T arg) throws IgniteException; and attendant /** * Finds class loader for the given class. * * @param rsrcName Class name or class alias to find class loader for. * @return Deployed class loader, or {@code null} if not deployed. */ public DeploymentResource findResource(String rsrcName); is thread unsafe by default, no guarantee that concurrent call of localDeployTask and execute will bring expected result. My proposal is to deprecate (or probably annotate [2], as a minimal — additionally document it) this methods and to append additional : public DeploymentResource findResource(String rsrcName, ClassLoader clsLdr); Only one problem i can observe here, if someone creates new class loaders and appropriate class instances in loop (i don`t know the purpose) and doesn`t undeploy them then he will get possibly OOM here. Such approach will give a possibility to use compute in concurrent scenario. If there is no objections here i will mark this methods and publish my PR, of course with additional tests. What do you think ? [1] https://ignite.apache.org/docs/latest/code-deployment/peer-class-loading [2] https://jcip.net/annotations/doc/net/jcip/annotations/NotThreadSafe.html >> >> >> >>
[jira] [Created] (IGNITE-14079) Add test for checking partition eviction order
Mirza Aliev created IGNITE-14079: Summary: Add test for checking partition eviction order Key: IGNITE-14079 URL: https://issues.apache.org/jira/browse/IGNITE-14079 Project: Ignite Issue Type: Bug Affects Versions: 2.9.1 Reporter: Mirza Aliev Assignee: Mirza Aliev Add test to check that {{CacheRebalanceMode#SYNC}} caches are evicted before {{CacheRebalanceMode#ASYNC}}. It is important because otherwise can lead to significant start-up delays. Ignite can send some fat {{ASYNC}} cache groups for eviction first and after this only add system cache, which is {{SYNC}}, for eviction as result sys cache will wait in queue and node startup will be blocked until fat partitions will be evicted. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSSION] Request for thread unsafe Compute functionality deprecation.
Hello! Do you have some kind of reproducer which demonstrates the issue? Regards, -- Ilya Kasnacheev чт, 28 янв. 2021 г. в 10:32, Zhenya Stanilovsky : > > Hello Igniters ! > In the process of Ignite usage i found that some part of Compute > functionality are thread unsafe and seems was designed with such > limitations initially. > Example : one (client, but it doesn`t matter at all) instance is > shared between numerous of fabric, all of them calls something like : > IgniteCompute#execute(ComputeTask, T) > or > IgniteCompute#execute(java.lang.Class>, T) > and appropriate «async» methods — what kind of instance will be called is > nondeterministic for now and as a confirmation of my words — i found no > tests covered multi thread usage of Computing i also found nothing on > documentation page [1]. > We have all necessary info for correct processing of such cases: > from initiator (ignite.compute(...) starter) side we have Class or it > instance and appropriate class loader which will be wired by class loader > id from execution side. > I create a fix and seems all work perfectly well besides one place, this > functionality : > /** > * Executes given task within the cluster group. For step-by-step > explanation of task execution process > * refer to {@link ComputeTask} documentation. > * > * If task for given name has not been deployed yet, then {@code taskName} > will be > * used as task class name to auto-deploy the task (see {@link > #localDeployTask(Class, ClassLoader)} method). > */ > public R execute(String taskName, T arg) throws IgniteException; > and attendant > /** > * Finds class loader for the given class. > * > * @param rsrcName Class name or class alias to find class loader for. > * @return Deployed class loader, or {@code null} if not deployed. > */ > public DeploymentResource findResource(String rsrcName); > is thread unsafe by default, no guarantee that concurrent call of > localDeployTask and execute will bring expected result. > My proposal is to deprecate (or probably annotate [2], as a minimal > — additionally document it) this methods and to append additional : > public DeploymentResource findResource(String rsrcName, ClassLoader > clsLdr); > Only one problem i can observe here, if someone creates new class loaders > and appropriate class instances in loop (i don`t know the purpose) and > doesn`t undeploy them then he will get possibly OOM here. > > Such approach will give a possibility to use compute in concurrent > scenario. If there is no objections here i will mark this methods and > publish my PR, of course with additional tests. > > What do you think ? > > > [1] > https://ignite.apache.org/docs/latest/code-deployment/peer-class-loading > [2] > https://jcip.net/annotations/doc/net/jcip/annotations/NotThreadSafe.html > >
[jira] [Created] (IGNITE-14078) Deadlock on GridCacheSharedTtlCleanupManager#mgrs if cache is created when ttl cleanup is running
Mirza Aliev created IGNITE-14078: Summary: Deadlock on GridCacheSharedTtlCleanupManager#mgrs if cache is created when ttl cleanup is running Key: IGNITE-14078 URL: https://issues.apache.org/jira/browse/IGNITE-14078 Project: Ignite Issue Type: Bug Affects Versions: 2.9.1 Reporter: Mirza Aliev ttl-cleanup-worker does a block of work in ConcurrentHashMap.compute() and tries to acquire checkpoint read lock: {code:java} Thread [name="ttl-cleanup-worker-#120%1%", id=225, state=WAITING, blockCnt=0, waitCnt=81486] Lock [object=java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync@35608c45, ownerName=null, ownerId=-1] at sun.misc.Unsafe.park(Native Method) at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967) at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283) at java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727) at o.a.i.i.processors.cache.persistence.GridCacheDatabaseSharedManager.checkpointReadLock(GridCacheDatabaseSharedManager.java:1730) at o.a.i.i.processors.cache.IgniteCacheOffheapManagerImpl.expireInternal(IgniteCacheOffheapManagerImpl.java:1346) at o.a.i.i.processors.cache.IgniteCacheOffheapManagerImpl.expire(IgniteCacheOffheapManagerImpl.java:1323) at o.a.i.i.processors.cache.GridCacheTtlManager.expire(GridCacheTtlManager.java:242) at o.a.i.i.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.lambda$body$0(GridCacheSharedTtlCleanupManager.java:178) at o.a.i.i.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker$$Lambda$619/1960552474.apply(Unknown Source) at java.util.concurrent.ConcurrentHashMap.computeIfPresent(ConcurrentHashMap.java:1769) - locked java.util.concurrent.ConcurrentHashMap$Node@4f66c754 at o.a.i.i.processors.cache.GridCacheSharedTtlCleanupManager$CleanupWorker.body(GridCacheSharedTtlCleanupManager.java:177) at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:119) at java.lang.Thread.run(Thread.java:748) {code} Meanwhile, exchange thread is waiting on the same ConcurrentHashMap node: {code:java} Thread [name="exchange-worker-#93%1%", id=193, state=BLOCKED, blockCnt=8, waitCnt=1669] Lock [object=java.util.concurrent.ConcurrentHashMap$Node@4f66c754, ownerName=ttl-cleanup-worker-#120%1%, ownerId=225] at java.util.concurrent.ConcurrentHashMap.transfer(ConcurrentHashMap.java:2426) at java.util.concurrent.ConcurrentHashMap.addCount(ConcurrentHashMap.java:2288) at java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1070) at java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) at o.a.i.i.processors.cache.GridCacheSharedTtlCleanupManager.register(GridCacheSharedTtlCleanupManager.java:68) at o.a.i.i.processors.cache.GridCacheTtlManager.start0(GridCacheTtlManager.java:107) at o.a.i.i.processors.cache.GridCacheManagerAdapter.start(GridCacheManagerAdapter.java:49) at o.a.i.i.processors.cache.GridCacheProcessor.initCacheContext(GridCacheProcessor.java:2176) at o.a.i.i.processors.cache.GridCacheProcessor.prepareCacheContext(GridCacheProcessor.java:1964) at o.a.i.i.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1883) at o.a.i.i.processors.cache.GridCacheProcessor.lambda$prepareStartCaches$55a0e703$1(GridCacheProcessor.java:1758) at o.a.i.i.processors.cache.GridCacheProcessor$$Lambda$527/649205444.apply(Unknown Source) at o.a.i.i.processors.cache.GridCacheProcessor.lambda$prepareStartCachesIfPossible$14(GridCacheProcessor.java:1728) at o.a.i.i.processors.cache.GridCacheProcessor$$Lambda$526/1117407359.handle(Unknown Source) at o.a.i.i.processors.cache.GridCacheProcessor.prepareStartCaches(GridCacheProcessor.java:1755) at o.a.i.i.processors.cache.GridCacheProcessor.prepareStartCachesIfPossible(GridCacheProcessor.java:1726) at o.a.i.i.processors.cache.CacheAffinitySharedManager.processCacheStartRequests(CacheAffinitySharedManager.java:1005) at o.a.i.i.processors.cache.CacheAffinitySharedManager.onCacheChangeRequest(CacheAffinitySharedManager.java:891) at o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.onCacheChangeRequest(GridDhtPartitionsExchangeFuture.java:1459) at o.a.i.i.processors.cache.distributed.dht.preloader.GridDhtPartitionsExchangeFuture.init(GridDhtPartition
[jira] [Created] (IGNITE-14077) Implement SchemaManager.
Andrey Mashenkov created IGNITE-14077: - Summary: Implement SchemaManager. Key: IGNITE-14077 URL: https://issues.apache.org/jira/browse/IGNITE-14077 Project: Ignite Issue Type: Bug Reporter: Andrey Mashenkov Create SchemaManager API and implement schema versioning logic. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-14076) Exponential putAll performance degradation in transactional cache
Pavel Tupitsyn created IGNITE-14076: --- Summary: Exponential putAll performance degradation in transactional cache Key: IGNITE-14076 URL: https://issues.apache.org/jira/browse/IGNITE-14076 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.9.1 Reporter: Pavel Tupitsyn Fix For: 2.11 {{putAll}} execution time grows almost exponentially while the number of keys grows linearly in the following test: {code:java} public class PutAllTxTest extends GridCommonAbstractTest { @Test public void testPutAll() throws Exception { Ignition.start(getConfiguration("server1")); Ignition.start(getConfiguration("server2")); Ignite ignite = Ignition.start(getConfiguration("client").setClientMode(true)); IgniteCache cache = ignite.createCache( new CacheConfiguration("c") .setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL)); int count = 5; Map data = new TreeMap<>(); for (int i = 0; i < count; i++) data.put(i, i); long begin = System.nanoTime(); cache.putAll(data); long dur = System.nanoTime() - begin; System.out.println("> " + dur / 100); } } {code} ||Entries||Seconds|| |1000|0.4| |5000|1.9| |1|3.8| |2|10.7| |4|41| |5|64| -- This message was sent by Atlassian Jira (v8.3.4#803005)