[TeamCIty] Issues with the disc space
Hello, Igniters. Looks like we have an issues with the TC agents disc space [1] Can someone take a look? [1] https://ci.ignite.apache.org/viewLog.html?buildId=4809938=buildResultsDiv=IgniteTests24Java8_PlatformCPPLinuxClang
Re: [DISCUSS] Links to commercial version resources in community wiki
Hi Igniters, It is not good if we'll made a tradition of linking to company websites from the community website / wiki / lists. But if it makes sense to mention something useful for users, why not. And in this particular case it may be ok, if it is relevant. Just remember, community resources are not Nascar cars. See also https://www.apache.org/foundation/marks/linking Sincerely, Dmitriy Pavlov вт, 3 дек. 2019 г. в 19:50, Denis Magda : > Hi Ivan, > > If an author of an article believes that there is a 3rd party resource that > is handy and helpful for his/her work then there is nothing wrong to refer > to it. Based on Alexey's response in another thread, that GridGain page > brings more clarity to the topic, thus, don't see any issue with that. > > > - > Denis > > > On Tue, Dec 3, 2019 at 6:41 AM Ivan Pavlukhin wrote: > > > Folks, > > > > In a thread about a document describing data consistency [1] in Ignite > > it was identified that there is a link to GridGain documentation. > > > > I do not have strong arguments against having such links in community > > wiki. But it interesting to me to discuss the subject with community. > > Might be there is some common practice in ASF communities about the > > subject. > > > > Please, share your thoughts. > > > > [1] https://cwiki.apache.org/confluence/display/IGNITE/Data+consistency > > -- > > Best regards, > > Ivan Pavlukhin > > >
Re: [DISCUSS] Links to commercial version resources in community wiki
Hi Ivan, If an author of an article believes that there is a 3rd party resource that is handy and helpful for his/her work then there is nothing wrong to refer to it. Based on Alexey's response in another thread, that GridGain page brings more clarity to the topic, thus, don't see any issue with that. - Denis On Tue, Dec 3, 2019 at 6:41 AM Ivan Pavlukhin wrote: > Folks, > > In a thread about a document describing data consistency [1] in Ignite > it was identified that there is a link to GridGain documentation. > > I do not have strong arguments against having such links in community > wiki. But it interesting to me to discuss the subject with community. > Might be there is some common practice in ASF communities about the > subject. > > Please, share your thoughts. > > [1] https://cwiki.apache.org/confluence/display/IGNITE/Data+consistency > -- > Best regards, > Ivan Pavlukhin >
Re: Full-text queries in Thin Client protocol
Pavel, I'm against adding this feature, as there were talks recently, that we should stop supporting TextQuery altogether. No sense in adding something, that we will need to depreciate and remove soon. Best Regards, Igor On Tue, Dec 3, 2019 at 5:46 PM Pavel Tupitsyn wrote: > Oh wow, the next thread actually discusses this very issue about > discontinuation: > > http://apache-ignite-developers.2346864.n4.nabble.com/Text-queries-indexes-GridLuceneIndex-QueryTextFiled-td43335.html > > But the question stands - add to Thin Clients or not? > > On Tue, Dec 3, 2019 at 5:43 PM Pavel Tupitsyn > wrote: > > > Igniters, > > > > I'd like to discuss adding full-text queries (TextQuery, our Lucene-based > > engine) to the Thin Client protocol. > > > > In my opinion, this is a good addition, and rather easy to implement. > > However, I vaguely remember in-person discussions with some community > > members along the lines of > > "Ignite full-text search capabilities may be deprecated or reworked soon, > > let's not add to thin clients". > > > > Any insights on this? Thoughts, objections? > > > > Thanks, > > Pavel > > > > > > > > >
Re: Full-text queries in Thin Client protocol
Oh wow, the next thread actually discusses this very issue about discontinuation: http://apache-ignite-developers.2346864.n4.nabble.com/Text-queries-indexes-GridLuceneIndex-QueryTextFiled-td43335.html But the question stands - add to Thin Clients or not? On Tue, Dec 3, 2019 at 5:43 PM Pavel Tupitsyn wrote: > Igniters, > > I'd like to discuss adding full-text queries (TextQuery, our Lucene-based > engine) to the Thin Client protocol. > > In my opinion, this is a good addition, and rather easy to implement. > However, I vaguely remember in-person discussions with some community > members along the lines of > "Ignite full-text search capabilities may be deprecated or reworked soon, > let's not add to thin clients". > > Any insights on this? Thoughts, objections? > > Thanks, > Pavel > > > >
Full-text queries in Thin Client protocol
Igniters, I'd like to discuss adding full-text queries (TextQuery, our Lucene-based engine) to the Thin Client protocol. In my opinion, this is a good addition, and rather easy to implement. However, I vaguely remember in-person discussions with some community members along the lines of "Ignite full-text search capabilities may be deprecated or reworked soon, let's not add to thin clients". Any insights on this? Thoughts, objections? Thanks, Pavel
[DISCUSS] Links to commercial version resources in community wiki
Folks, In a thread about a document describing data consistency [1] in Ignite it was identified that there is a link to GridGain documentation. I do not have strong arguments against having such links in community wiki. But it interesting to me to discuss the subject with community. Might be there is some common practice in ASF communities about the subject. Please, share your thoughts. [1] https://cwiki.apache.org/confluence/display/IGNITE/Data+consistency -- Best regards, Ivan Pavlukhin
Re: Re[4]: Text queries/indexes (GridLuceneIndex, @QueryTextFiled)
*on topologies вт, 3 дек. 2019 г. в 17:15, Ivan Pavlukhin : > > Ilya, Yuriy, > > It seems that text queries can return incorrect results on tologies > with MOVING partitions. I left a comment in JIRA [1]. > > [1] https://issues.apache.org/jira/browse/IGNITE-12401 > > пн, 2 дек. 2019 г. в 15:00, Ivan Pavlukhin : > > > > Ilya, > > > > I checked your test on a revision before "limit" and it fails there as > > well. Could you please double check? > > > > пн, 2 дек. 2019 г. в 13:21, Ilya Kasnacheev : > > > > > > Hello! > > > > > > The problem is NOT specific to range queries. Range queries were broken > > > previously and they are broken now, but now even a simple "token in field > > > with limit" returns duplicates. > > > > > > Before limits were introduced, any tested use cases were unaffected by > > > duplicates, but now they are. > > > > > > Regards, > > > -- > > > Ilya Kasnacheev > > > > > > > > > пн, 2 дек. 2019 г. в 12:23, Ivan Pavlukhin : > > > > > > > And is the problem specific to range queries or not? > > > > > > > > пн, 2 дек. 2019 г. в 11:12, Ivan Pavlukhin : > > > > > > > > > > Yuriy, > > > > > > > > > > Thank you for investigating the problem [1]. Still cannot realize how > > > > > the problem relates to introduced "limit"? Is it right that there were > > > > > no duplicates before "limit" support? After that support is introduced > > > > > are only limited queries contain duplicates, or unlimited, or both? > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12401 > > > > > > > > > > чт, 28 нояб. 2019 г. в 18:30, Ilya Kasnacheev > > > > > > > > >: > > > > > > > > > > > > Hello! > > > > > > > > > > > > I have just found what I consider a major regression in Text > > > > > > Queries: > > > > it > > > > > > seems to me that text queries with limits will return same key-value > > > > > > entries multiple times. > > > > > > > > > > > > Please check the issue > > > > https://issues.apache.org/jira/browse/IGNITE-12401 > > > > > > and corresponding build > > > > > > https://ci.ignite.apache.org/viewQueued.html?itemId=4799634 > > > > > > > > > > > > Regards, > > > > > > -- > > > > > > Ilya Kasnacheev > > > > > > > > > > > > > > > > > > > > -- > > > > > Best regards, > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Ivan Pavlukhin > > > > > > > > > > > > -- > > Best regards, > > Ivan Pavlukhin > > > > -- > Best regards, > Ivan Pavlukhin -- Best regards, Ivan Pavlukhin
Re: Re[4]: Text queries/indexes (GridLuceneIndex, @QueryTextFiled)
Ilya, Yuriy, It seems that text queries can return incorrect results on tologies with MOVING partitions. I left a comment in JIRA [1]. [1] https://issues.apache.org/jira/browse/IGNITE-12401 пн, 2 дек. 2019 г. в 15:00, Ivan Pavlukhin : > > Ilya, > > I checked your test on a revision before "limit" and it fails there as > well. Could you please double check? > > пн, 2 дек. 2019 г. в 13:21, Ilya Kasnacheev : > > > > Hello! > > > > The problem is NOT specific to range queries. Range queries were broken > > previously and they are broken now, but now even a simple "token in field > > with limit" returns duplicates. > > > > Before limits were introduced, any tested use cases were unaffected by > > duplicates, but now they are. > > > > Regards, > > -- > > Ilya Kasnacheev > > > > > > пн, 2 дек. 2019 г. в 12:23, Ivan Pavlukhin : > > > > > And is the problem specific to range queries or not? > > > > > > пн, 2 дек. 2019 г. в 11:12, Ivan Pavlukhin : > > > > > > > > Yuriy, > > > > > > > > Thank you for investigating the problem [1]. Still cannot realize how > > > > the problem relates to introduced "limit"? Is it right that there were > > > > no duplicates before "limit" support? After that support is introduced > > > > are only limited queries contain duplicates, or unlimited, or both? > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12401 > > > > > > > > чт, 28 нояб. 2019 г. в 18:30, Ilya Kasnacheev > > >: > > > > > > > > > > Hello! > > > > > > > > > > I have just found what I consider a major regression in Text Queries: > > > it > > > > > seems to me that text queries with limits will return same key-value > > > > > entries multiple times. > > > > > > > > > > Please check the issue > > > https://issues.apache.org/jira/browse/IGNITE-12401 > > > > > and corresponding build > > > > > https://ci.ignite.apache.org/viewQueued.html?itemId=4799634 > > > > > > > > > > Regards, > > > > > -- > > > > > Ilya Kasnacheev > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Ivan Pavlukhin > > > > > > > > > > > > -- > > > Best regards, > > > Ivan Pavlukhin > > > > > > > -- > Best regards, > Ivan Pavlukhin -- Best regards, Ivan Pavlukhin
[jira] [Created] (IGNITE-12414) .NET: Performance: review CopyOnWriteConcurrentDictionary.GetOrAdd usage and locking
Pavel Tupitsyn created IGNITE-12414: --- Summary: .NET: Performance: review CopyOnWriteConcurrentDictionary.GetOrAdd usage and locking Key: IGNITE-12414 URL: https://issues.apache.org/jira/browse/IGNITE-12414 Project: Ignite Issue Type: Improvement Components: platforms Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 3.0, 2.9 CopyOnWriteConcurrentDictionary.GetOrAdd uses lock right away, while the class assumes frequent reads and infrequent writes. It can be beneficial to check for the key outside of the lock. In particular, this often causes contention because of BinarySystemHandlers.GetWriteHandler call. Review other usages of this method. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
No objections. вт, 3 дек. 2019 г. в 15:02, Maxim Muzafarov : > Alexey, > > > Yet another issue [1] with corrupted B+Tree exception on the master branch. > My suggestion is to revert the IGNITE-11704 [3] issue from the master > branch and rework the patch. > > Any objections? > > Configuration: > [1] persistence = false > [2] persistence = true > > class > org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: > B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple > [val1=1544803905, val2=844420635196573]], msg=Runtime failure on > cursor iteration] > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:5927) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5438) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.next(BPlusTree.java:5661) > at > org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl$1.next(IgniteCacheOffheapManagerImpl.java:3020) > at > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition$2.hasNextX(GridDhtLocalPartition.java:1226) > at > org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) > at > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.doClear(GridDhtLocalPartition.java:1295) > at > org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.clearTombstones(GridDhtLocalPartition.java:1242) > at > org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$ClearTombstonesTask.run0(PartitionsEvictManager.java:673) > at > org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$AbstractEvictionTask.run(PartitionsEvictManager.java:587) > at > org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7061) > at > org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827) > at > org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > at java.lang.Thread.run(Thread.java:748) > > Caused by: java.lang.AssertionError: Key is not ready: > CacheDataRowAdapter [key=null, val=null, expireTime=-1, ver=null, > cacheId=0, link=0001003c77d8] > at > org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.key(CacheDataRowAdapter.java:837) > at > org.apache.ignite.internal.processors.cache.tree.CacheDataTree.compare(CacheDataTree.java:382) > at > org.apache.ignite.internal.processors.cache.tree.CacheDataTree.compare(CacheDataTree.java:63) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.compare(BPlusTree.java:5200) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.findLowerBound(BPlusTree.java:5317) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer0(BPlusTree.java:5588) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.fillFromBuffer(BPlusTree.java:5376) > at > org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5428) > > > [1] > https://ci.ignite.apache.org/viewLog.html?buildId=4807946=buildResultsDiv=IgniteTests24Java8_Cache9#testNameId1910487508546147692 > [2] > https://ci.ignite.apache.org/viewLog.html?buildId=4796925=buildResultsDiv=IgniteTests24Java8_Cache9#testNameId7609674190425495190 > [3] https://issues.apache.org/jira/browse/IGNITE-11704 > > > > > On Tue, 3 Dec 2019 at 13:39, Alexei Scherbakov > wrote: > > > > Maxim, > > > > I'm not sure this is purely a "tombstone" problem, could be a tree > > concurrency issue. > > Looks like the investigation is required. > > For example, tombstone logic can be reverted but test logic is kept. > > > > It seems Ivan Rakov already suggested to revert a commit from master > branch > > in another thread. > > > > пн, 2 дек. 2019 г. в 17:17, Maxim Muzafarov : > > > > > Igniters, > > > > > > > > > I've found a scary error [1] with `CorruptedTreeException: B+Tree is > > > corrupted` on TC in the master branch. It seems that the issue is > > > related to a [2] tombstone implementation (probably not). > > > > > > Can anyone confirm that the problem is still actual? > > > If it is true, do we have time for resolving the problem? > > > Will it be better to revert this commit from the release branch? > > > > > > [1] > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4796925=buildResultsDiv=IgniteTests24Java8_Cache9#testNameId7609674190425495190 > > > [2]
Re: Joining node validation failure event.
Mikhail, Yep, I see IgniteNodeValidationResult in new event in PR [1]. > Discovery events such as (join/left/failed) are connected with a > topology version change. In my case that not happens and may be > misleading. That's why the new event type was chosen. I did not mean that one of those events should be used. I meant that it sounds natural to me to have an additional "unsuccessful node join" event (like is done in PR[1]). https://github.com/apache/ignite/pull/7057/files вт, 3 дек. 2019 г. в 14:32, Mikhail Petrov : > > Nikolay, Ivan, > > I understood the possible problem. It seems that it can be solved using > EventStorageSpi which starts before DiscoveryManager. > > As for me, ClusterNode is enough. It contains all info about joining > node including its attributes. > > Discovery events such as (join/left/failed) are connected with a > topology version change. In my case that not happens and may be > misleading. That's why the new event type was chosen. > > The cause of the failure is also presented in the event. > > Regards, > Mikhail. > > On 03.12.2019 13:19, Николай Ижиков wrote: > > Exception(s) from component(s) that don’t want node joined. > > > >> 3 дек. 2019 г., в 12:39, Ivan Pavlukhin написал(а): > >> > >> How that reason will look like? Actually, I mostly thinking about > >> general API here. What I would like to avoid is exposing something not > >> general but needed only for a particular extension (plugin). > >> > >> вт, 3 дек. 2019 г. в 12:31, Николай Ижиков : > >>> I think we also should provide the reason why join failed. > >>> > 3 дек. 2019 г., в 12:22, Ivan Pavlukhin написал(а): > > Mikhail, > > So, I suppose there should be ordering guarantees that listener is > registered before first validation failure can occur. Hope > GridComponent#onKernalStart is the right place. Is it enough to pass > only problematic node id (or ClusterNode) with an event? Actually such > event seems to fit naturally node join/left/failed events. > > вт, 3 дек. 2019 г. в 10:38, Mikhail Petrov : > > Hi Ivan. > > > > No other lifecycle events are needed in my case. > > > > We can register a listener in the security component's > > GridComponent#onKernalStart method and listen locally to every failed > > joining attempt. Am I missing something? > > > > Regards, > > Mikhail. > > > > On 03.12.2019 8:48, Ivan Pavlukhin wrote: > >> Mikhail, > >> > >> Do you need some ordering guarantees among node lifecycle events and > >> listener notifications. For example, I can imagine that it is good to > >> notify security component about every node failed validation. How can > >> we achieve it with events (I assume dynamic listener registration)? > >> > >> пн, 2 дек. 2019 г. в 18:09, Mikhail Petrov : > >>> Hi, Andrey. > >>> > >>> It doesn't influence on authentication or authorization process. There > >>> is a security requirement that demands to notify some outer security > >>> subsystems in a specific way when joining node validation failed in > >>> any > >>> Ignite component (e.g. GridCacheProcessor) not only in > >>> IgniteSecurityProcessor. So PluginProvider#validateNewNode is not > >>> acceptable for me. > >>> > >>> Regards, > >>> Mikhail. > >>> > >>> On 02.12.2019 16:35, Andrey Gura wrote: > Mikhail, > > I don't understand how node validation on join affects security. But > it seems that you can use > PluginProvider#validateNewNode(org.apache.ignite.cluster.ClusterNode, > java.io.Serializable) method. Does it fit for your needs? > > On Mon, Dec 2, 2019 at 12:54 PM Mikhail Petrov > wrote: > > Hi, Ivan. > > > > > > Unfortunately, we came to no decision yet. As I mentioned above this > > event is disabled by default and no node will receive this event > > without > > an explicit subscription. In my case, that event is assumed to be > > used > > on node locally to share joining node info between security and > > discovery components. I have no idea how to solve this problem > > without > > publishing a new event too. But I'm concerned about the acceptance > > of > > that solution. Maybe it can be solved some other way? I will > > appreciate > > any suggestion or review PR [1] with event implementation. > > > > > > [1] https://github.com/apache/ignite/pull/7057 > > > > > > Regards, > > > > Mikhail. > > > > On 02.12.2019 10:38, Ivan Pavlukhin wrote: > >> Mikhail, Andrey, > >> > >> Have you come to a common decision here? As for me, it sounds > >> useful > >> to expose node join failure details
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Alexey, Yet another issue [1] with corrupted B+Tree exception on the master branch. My suggestion is to revert the IGNITE-11704 [3] issue from the master branch and rework the patch. Any objections? Configuration: [1] persistence = false [2] persistence = true class org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException: B+Tree is corrupted [pages(groupId, pageId)=[IgniteBiTuple [val1=1544803905, val2=844420635196573]], msg=Runtime failure on cursor iteration] at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.corruptedTreeException(BPlusTree.java:5927) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5438) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.next(BPlusTree.java:5661) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl$1.next(IgniteCacheOffheapManagerImpl.java:3020) at org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition$2.hasNextX(GridDhtLocalPartition.java:1226) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.doClear(GridDhtLocalPartition.java:1295) at org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.clearTombstones(GridDhtLocalPartition.java:1242) at org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$ClearTombstonesTask.run0(PartitionsEvictManager.java:673) at org.apache.ignite.internal.processors.cache.distributed.dht.topology.PartitionsEvictManager$AbstractEvictionTask.run(PartitionsEvictManager.java:587) at org.apache.ignite.internal.util.IgniteUtils.wrapThreadLoader(IgniteUtils.java:7061) at org.apache.ignite.internal.processors.closure.GridClosureProcessor$1.body(GridClosureProcessor.java:827) at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.lang.AssertionError: Key is not ready: CacheDataRowAdapter [key=null, val=null, expireTime=-1, ver=null, cacheId=0, link=0001003c77d8] at org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.key(CacheDataRowAdapter.java:837) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.compare(CacheDataTree.java:382) at org.apache.ignite.internal.processors.cache.tree.CacheDataTree.compare(CacheDataTree.java:63) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.compare(BPlusTree.java:5200) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.findLowerBound(BPlusTree.java:5317) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$ForwardCursor.fillFromBuffer0(BPlusTree.java:5588) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.fillFromBuffer(BPlusTree.java:5376) at org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$AbstractForwardCursor.nextPage(BPlusTree.java:5428) [1] https://ci.ignite.apache.org/viewLog.html?buildId=4807946=buildResultsDiv=IgniteTests24Java8_Cache9#testNameId1910487508546147692 [2] https://ci.ignite.apache.org/viewLog.html?buildId=4796925=buildResultsDiv=IgniteTests24Java8_Cache9#testNameId7609674190425495190 [3] https://issues.apache.org/jira/browse/IGNITE-11704 On Tue, 3 Dec 2019 at 13:39, Alexei Scherbakov wrote: > > Maxim, > > I'm not sure this is purely a "tombstone" problem, could be a tree > concurrency issue. > Looks like the investigation is required. > For example, tombstone logic can be reverted but test logic is kept. > > It seems Ivan Rakov already suggested to revert a commit from master branch > in another thread. > > пн, 2 дек. 2019 г. в 17:17, Maxim Muzafarov : > > > Igniters, > > > > > > I've found a scary error [1] with `CorruptedTreeException: B+Tree is > > corrupted` on TC in the master branch. It seems that the issue is > > related to a [2] tombstone implementation (probably not). > > > > Can anyone confirm that the problem is still actual? > > If it is true, do we have time for resolving the problem? > > Will it be better to revert this commit from the release branch? > > > > [1] > > https://ci.ignite.apache.org/viewLog.html?buildId=4796925=buildResultsDiv=IgniteTests24Java8_Cache9#testNameId7609674190425495190 > > [2] https://issues.apache.org/jira/browse/IGNITE-11704 > > > > On Wed, 13 Nov 2019 at 16:50, Maxim Muzafarov wrote: > > > > > > Igniters, > > > > > > > > > I've had conversations with Nikolay and Alexey about working progress > > > [1] on ML, Spark, Monitoring
Re: Joining node validation failure event.
Nikolay, Ivan, I understood the possible problem. It seems that it can be solved using EventStorageSpi which starts before DiscoveryManager. As for me, ClusterNode is enough. It contains all info about joining node including its attributes. Discovery events such as (join/left/failed) are connected with a topology version change. In my case that not happens and may be misleading. That's why the new event type was chosen. The cause of the failure is also presented in the event. Regards, Mikhail. On 03.12.2019 13:19, Николай Ижиков wrote: Exception(s) from component(s) that don’t want node joined. 3 дек. 2019 г., в 12:39, Ivan Pavlukhin написал(а): How that reason will look like? Actually, I mostly thinking about general API here. What I would like to avoid is exposing something not general but needed only for a particular extension (plugin). вт, 3 дек. 2019 г. в 12:31, Николай Ижиков : I think we also should provide the reason why join failed. 3 дек. 2019 г., в 12:22, Ivan Pavlukhin написал(а): Mikhail, So, I suppose there should be ordering guarantees that listener is registered before first validation failure can occur. Hope GridComponent#onKernalStart is the right place. Is it enough to pass only problematic node id (or ClusterNode) with an event? Actually such event seems to fit naturally node join/left/failed events. вт, 3 дек. 2019 г. в 10:38, Mikhail Petrov : Hi Ivan. No other lifecycle events are needed in my case. We can register a listener in the security component's GridComponent#onKernalStart method and listen locally to every failed joining attempt. Am I missing something? Regards, Mikhail. On 03.12.2019 8:48, Ivan Pavlukhin wrote: Mikhail, Do you need some ordering guarantees among node lifecycle events and listener notifications. For example, I can imagine that it is good to notify security component about every node failed validation. How can we achieve it with events (I assume dynamic listener registration)? пн, 2 дек. 2019 г. в 18:09, Mikhail Petrov : Hi, Andrey. It doesn't influence on authentication or authorization process. There is a security requirement that demands to notify some outer security subsystems in a specific way when joining node validation failed in any Ignite component (e.g. GridCacheProcessor) not only in IgniteSecurityProcessor. So PluginProvider#validateNewNode is not acceptable for me. Regards, Mikhail. On 02.12.2019 16:35, Andrey Gura wrote: Mikhail, I don't understand how node validation on join affects security. But it seems that you can use PluginProvider#validateNewNode(org.apache.ignite.cluster.ClusterNode, java.io.Serializable) method. Does it fit for your needs? On Mon, Dec 2, 2019 at 12:54 PM Mikhail Petrov wrote: Hi, Ivan. Unfortunately, we came to no decision yet. As I mentioned above this event is disabled by default and no node will receive this event without an explicit subscription. In my case, that event is assumed to be used on node locally to share joining node info between security and discovery components. I have no idea how to solve this problem without publishing a new event too. But I'm concerned about the acceptance of that solution. Maybe it can be solved some other way? I will appreciate any suggestion or review PR [1] with event implementation. [1] https://github.com/apache/ignite/pull/7057 Regards, Mikhail. On 02.12.2019 10:38, Ivan Pavlukhin wrote: Mikhail, Andrey, Have you come to a common decision here? As for me, it sounds useful to expose node join failure details somehow. The thing to decide on is a mechanism to expose it. Unfortunately, immediately have no idea better than using events. What is purpose of the special cluster wide event? Node is not joined to the topology. Why topology nodes should know something about this node? Was this question answered? I suppose that at least coordinator will receive the event, will not it? чт, 28 нояб. 2019 г. в 10:10, Mikhail Petrov : Hi, Ivan. I'm sorry that the discussion was moved in private channel. The problem I'm trying to solve is described below in the thread. The security plugin in my case stores and analyzesinfo about a node that failed to join. Regards, Mikhail. Forwarded Message Subject:Re: Joining node validation failure event. Date: Thu, 21 Nov 2019 21:43:33 +0300 From: Mikhail Petrov To: Andrey Gura Hi, Andrey. In my task security plugin running on the coordinator must locally handle the situation when some node is trying to join the topology with the invalid configuration. I can't handle the response on a node that failed to connect because it's untrusted. Actually I'm only concerned about one validation -- when all Ignite components validate new node. In my case plugin must be able to obtain general and security subject information from joining TcpDiscoveryNode attributes. But I have no idea how to share information between the security and discovery components without
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Maxim, I'm not sure this is purely a "tombstone" problem, could be a tree concurrency issue. Looks like the investigation is required. For example, tombstone logic can be reverted but test logic is kept. It seems Ivan Rakov already suggested to revert a commit from master branch in another thread. пн, 2 дек. 2019 г. в 17:17, Maxim Muzafarov : > Igniters, > > > I've found a scary error [1] with `CorruptedTreeException: B+Tree is > corrupted` on TC in the master branch. It seems that the issue is > related to a [2] tombstone implementation (probably not). > > Can anyone confirm that the problem is still actual? > If it is true, do we have time for resolving the problem? > Will it be better to revert this commit from the release branch? > > [1] > https://ci.ignite.apache.org/viewLog.html?buildId=4796925=buildResultsDiv=IgniteTests24Java8_Cache9#testNameId7609674190425495190 > [2] https://issues.apache.org/jira/browse/IGNITE-11704 > > On Wed, 13 Nov 2019 at 16:50, Maxim Muzafarov wrote: > > > > Igniters, > > > > > > I've had conversations with Nikolay and Alexey about working progress > > [1] on ML, Spark, Monitoring features which we are waiting for. It > > seems the progress dates is a bit slower than we expected earlier. I'd > > suggest the following dates: > > > > 2 December - the release branch cutting. > > 30 December - code freeze of the release branch (only blocker issues > allowed) > > 27 January 2020 - the voting date (2+ weeks of stabilization period) > > > > Any objections? > > > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8#ApacheIgnite2.8-Awatingfeaturescompletion > > > > On Thu, 7 Nov 2019 at 18:01, Maxim Muzafarov wrote: > > > > > > Alexey, > > > > > > > > > Not yet. I will announce the release branch cutting stage with a > > > separate post, probably, for a one week before the cut. No one will > > > not miss it for sure. > > > > > > Currently, all issues can safely be pinned to 2.8 release, but for > > > huge issues\features (e.g. `baseline topology`, `persistence`) which > > > are going to be included into the release, it's better to have a > > > separate thread discussion. Do you have such issues in mind? > > > > > > I'm trying to estimate new release dates due to not all features which > > > we are waiting for [1] will be completed by the due dates discussed > > > before. > > > I'll back soon. > > > > > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8#ApacheIgnite2.8-Awatingfeaturescompletion > > > > > > On Thu, 7 Nov 2019 at 17:46, Alexey Goncharuk > > > wrote: > > > > > > > > Maxim, > > > > > > > > A side note - we did not cut the 2.8 branch yet, did we? > > > > > > > > This information is not reflected on the release page and I just > realized > > > > that it is hard to choose a fix version for a ticket that is being > merged > > > > to master when release scope is being finalized. This moment is > covered > > > > neither in the release process nor in the development process > (unless I am > > > > missing something). > > > > > > > > ср, 6 нояб. 2019 г. в 13:55, Maxim Muzafarov : > > > > > > > > > Igniters, > > > > > Ivan, > > > > > > > > > > Yes, it's true that currently, all resolved issues with 2.8 > fixVersion > > > > > will get into 2.8 release. > > > > > Moreover, I've marked all issues with empty fixVersion filed closed > > > > > since 2.7 major release with 2.8 labels. > > > > > > > > > > On Tue, 5 Nov 2019 at 15:03, Ivan Pavlukhin > wrote: > > > > > > > > > > > > Maxim, > > > > > > > > > > > > Could you please elaborate about a question from Akash. What > tickets > > > > > > do we consider for release? All with 2.8 fix version in Jira or > there > > > > > > is a different criteria? > > > > > > > > > > > > вт, 5 нояб. 2019 г. в 15:01, Ivan Pavlukhin >: > > > > > > > > > > > > > > Alexey, > > > > > > > > > > > > > > True. It is definitely to early for Calcite. > > > > > > > > > > > > > > вт, 5 нояб. 2019 г. в 14:46, Alexey Zinoviev < > zaleslaw@gmail.com>: > > > > > > > > > > > > > > > > I have no ideas, I think it's enough for this release. > > > > > > > > > > > > > > > > As I understand it's not the proper time for Calcite > integration and > > > > > so on, > > > > > > > > isn't it @Igor Seliverstov? > > > > > > > > > > > > > > > > вт, 5 нояб. 2019 г. в 14:41, Maxim Muzafarov < > mmu...@apache.org>: > > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > Do we want to discuss something? > > > > > > > > > What else features do we want to include? > > > > > > > > > > > > > > > > > > If not, I propose to exclude any other major features > (except > > > > > already > > > > > > > > > discussed - ML, Monitoring, Spark) and focus on bug fixing > and the > > > > > > > > > master branch stabilization. > > > > > > > > > > > > > > > > > > On Wed, 30 Oct 2019 at 17:57, Akash Shinde < > akashshi...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Because I didn't see it in this
Re: Joining node validation failure event.
Exception(s) from component(s) that don’t want node joined. > 3 дек. 2019 г., в 12:39, Ivan Pavlukhin написал(а): > > How that reason will look like? Actually, I mostly thinking about > general API here. What I would like to avoid is exposing something not > general but needed only for a particular extension (plugin). > > вт, 3 дек. 2019 г. в 12:31, Николай Ижиков : >> >> I think we also should provide the reason why join failed. >> >>> 3 дек. 2019 г., в 12:22, Ivan Pavlukhin написал(а): >>> >>> Mikhail, >>> >>> So, I suppose there should be ordering guarantees that listener is >>> registered before first validation failure can occur. Hope >>> GridComponent#onKernalStart is the right place. Is it enough to pass >>> only problematic node id (or ClusterNode) with an event? Actually such >>> event seems to fit naturally node join/left/failed events. >>> >>> вт, 3 дек. 2019 г. в 10:38, Mikhail Petrov : Hi Ivan. No other lifecycle events are needed in my case. We can register a listener in the security component's GridComponent#onKernalStart method and listen locally to every failed joining attempt. Am I missing something? Regards, Mikhail. On 03.12.2019 8:48, Ivan Pavlukhin wrote: > Mikhail, > > Do you need some ordering guarantees among node lifecycle events and > listener notifications. For example, I can imagine that it is good to > notify security component about every node failed validation. How can > we achieve it with events (I assume dynamic listener registration)? > > пн, 2 дек. 2019 г. в 18:09, Mikhail Petrov : >> Hi, Andrey. >> >> It doesn't influence on authentication or authorization process. There >> is a security requirement that demands to notify some outer security >> subsystems in a specific way when joining node validation failed in any >> Ignite component (e.g. GridCacheProcessor) not only in >> IgniteSecurityProcessor. So PluginProvider#validateNewNode is not >> acceptable for me. >> >> Regards, >> Mikhail. >> >> On 02.12.2019 16:35, Andrey Gura wrote: >>> Mikhail, >>> >>> I don't understand how node validation on join affects security. But >>> it seems that you can use >>> PluginProvider#validateNewNode(org.apache.ignite.cluster.ClusterNode, >>> java.io.Serializable) method. Does it fit for your needs? >>> >>> On Mon, Dec 2, 2019 at 12:54 PM Mikhail Petrov >>> wrote: Hi, Ivan. Unfortunately, we came to no decision yet. As I mentioned above this event is disabled by default and no node will receive this event without an explicit subscription. In my case, that event is assumed to be used on node locally to share joining node info between security and discovery components. I have no idea how to solve this problem without publishing a new event too. But I'm concerned about the acceptance of that solution. Maybe it can be solved some other way? I will appreciate any suggestion or review PR [1] with event implementation. [1] https://github.com/apache/ignite/pull/7057 Regards, Mikhail. On 02.12.2019 10:38, Ivan Pavlukhin wrote: > Mikhail, Andrey, > > Have you come to a common decision here? As for me, it sounds useful > to expose node join failure details somehow. The thing to decide on is > a mechanism to expose it. Unfortunately, immediately have no idea > better than using events. > >> What is purpose of the special cluster wide event? Node is not joined >> to the topology. Why topology nodes should know something about this >> node? > Was this question answered? I suppose that at least coordinator will > receive the event, will not it? > > чт, 28 нояб. 2019 г. в 10:10, Mikhail Petrov : >> Hi, Ivan. >> >> I'm sorry that the discussion was moved in private channel. The >> problem >> I'm trying to solve is described below in the thread. The security >> plugin in my case stores and analyzesinfo about a node that failed >> to join. >> >> >> Regards, >> >> Mikhail. >> >> >> >> Forwarded Message >> Subject:Re: Joining node validation failure event. >> Date: Thu, 21 Nov 2019 21:43:33 +0300 >> From: Mikhail Petrov >> To: Andrey Gura >> >> >> >> Hi, Andrey. >> >> In my task security plugin running on the coordinator must locally >> handle the situation when some node is trying to join the topology >> with
Re: Joining node validation failure event.
How that reason will look like? Actually, I mostly thinking about general API here. What I would like to avoid is exposing something not general but needed only for a particular extension (plugin). вт, 3 дек. 2019 г. в 12:31, Николай Ижиков : > > I think we also should provide the reason why join failed. > > > 3 дек. 2019 г., в 12:22, Ivan Pavlukhin написал(а): > > > > Mikhail, > > > > So, I suppose there should be ordering guarantees that listener is > > registered before first validation failure can occur. Hope > > GridComponent#onKernalStart is the right place. Is it enough to pass > > only problematic node id (or ClusterNode) with an event? Actually such > > event seems to fit naturally node join/left/failed events. > > > > вт, 3 дек. 2019 г. в 10:38, Mikhail Petrov : > >> > >> Hi Ivan. > >> > >> No other lifecycle events are needed in my case. > >> > >> We can register a listener in the security component's > >> GridComponent#onKernalStart method and listen locally to every failed > >> joining attempt. Am I missing something? > >> > >> Regards, > >> Mikhail. > >> > >> On 03.12.2019 8:48, Ivan Pavlukhin wrote: > >>> Mikhail, > >>> > >>> Do you need some ordering guarantees among node lifecycle events and > >>> listener notifications. For example, I can imagine that it is good to > >>> notify security component about every node failed validation. How can > >>> we achieve it with events (I assume dynamic listener registration)? > >>> > >>> пн, 2 дек. 2019 г. в 18:09, Mikhail Petrov : > Hi, Andrey. > > It doesn't influence on authentication or authorization process. There > is a security requirement that demands to notify some outer security > subsystems in a specific way when joining node validation failed in any > Ignite component (e.g. GridCacheProcessor) not only in > IgniteSecurityProcessor. So PluginProvider#validateNewNode is not > acceptable for me. > > Regards, > Mikhail. > > On 02.12.2019 16:35, Andrey Gura wrote: > > Mikhail, > > > > I don't understand how node validation on join affects security. But > > it seems that you can use > > PluginProvider#validateNewNode(org.apache.ignite.cluster.ClusterNode, > > java.io.Serializable) method. Does it fit for your needs? > > > > On Mon, Dec 2, 2019 at 12:54 PM Mikhail Petrov > > wrote: > >> Hi, Ivan. > >> > >> > >> Unfortunately, we came to no decision yet. As I mentioned above this > >> event is disabled by default and no node will receive this event > >> without > >> an explicit subscription. In my case, that event is assumed to be used > >> on node locally to share joining node info between security and > >> discovery components. I have no idea how to solve this problem without > >> publishing a new event too. But I'm concerned about the acceptance of > >> that solution. Maybe it can be solved some other way? I will appreciate > >> any suggestion or review PR [1] with event implementation. > >> > >> > >> [1] https://github.com/apache/ignite/pull/7057 > >> > >> > >> Regards, > >> > >> Mikhail. > >> > >> On 02.12.2019 10:38, Ivan Pavlukhin wrote: > >>> Mikhail, Andrey, > >>> > >>> Have you come to a common decision here? As for me, it sounds useful > >>> to expose node join failure details somehow. The thing to decide on is > >>> a mechanism to expose it. Unfortunately, immediately have no idea > >>> better than using events. > >>> > What is purpose of the special cluster wide event? Node is not joined > to the topology. Why topology nodes should know something about this > node? > >>> Was this question answered? I suppose that at least coordinator will > >>> receive the event, will not it? > >>> > >>> чт, 28 нояб. 2019 г. в 10:10, Mikhail Petrov : > Hi, Ivan. > > I'm sorry that the discussion was moved in private channel. The > problem > I'm trying to solve is described below in the thread. The security > plugin in my case stores and analyzesinfo about a node that failed > to join. > > > Regards, > > Mikhail. > > > > Forwarded Message > Subject:Re: Joining node validation failure event. > Date: Thu, 21 Nov 2019 21:43:33 +0300 > From: Mikhail Petrov > To: Andrey Gura > > > > Hi, Andrey. > > In my task security plugin running on the coordinator must locally > handle the situation when some node is trying to join the topology > with > the invalid configuration. I can't handle the response on a node that > failed to connect because it's untrusted. > > Actually I'm only
Re: Data consistency essentials explained
Ivan, for me GG docs have more clear information on the subject. As soon as community will improve docs quality link could be changed back to Apache. сб, 30 нояб. 2019 г. в 17:22, Ivan Pavlukhin : > Alexei, > > > Ivan, the link is intentional. > > Could you please elaborate why is it fine to have a such link? The > referred document is out of community control. Do not we have a > similar one in Ignite documentation? Should we discuss it in another > mail thread? > > сб, 30 нояб. 2019 г. в 15:48, Alexei Scherbakov < > alexey.scherbak...@gmail.com>: > > > > Ivan, the link is intentional. > > > > Nikolai, the formatting looks ok to me. Feel free to change it however > you > > like. > > > > чт, 28 нояб. 2019 г. в 09:53, Николай Ижиков : > > > > > Hello, Alex. > > > > > > I think we should improve article formatting - code highlight, fonts, > etc. > > > > > > For now, it’s very hard to read it. > > > > > > Can you, please, do it? > > > > > > > 28 нояб. 2019 г., в 09:25, Ivan Pavlukhin > > > написал(а): > > > > > > > > Alexei, > > > > > > > > Many thanks for that article! Really nice that now we have a document > > > > describing Ignite approach for data consistency. I think it is super > > > > important because it is not trivial and very Ignite specific. I > > > > believe lots of Ignite developers will learn something new from this > > > > document. > > > > > > > > By the way, I noticed a link to GridGain documentation [1]. Is it > fine? > > > > > > > > [1] > > > > https://www.gridgain.com/docs/latest/developers-guide/key-value-api/transactions#pessimistic-transactions > > > > > > > > чт, 28 нояб. 2019 г. в 02:49, Denis Magda : > > > >> > > > >> Alex, > > > >> > > > >> Thanks a lot for putting your knowledge on paper. That's an > invaluable > > > contribution! > > > >> > > > >> Looping in the user community (will be interesting for those who use > > > native persistence) and has just spread the word via Ignite twitter > handle: > > > >> https://twitter.com/ApacheIgnite/status/1199837055007612928 > > > >> > > > >> - > > > >> Denis > > > >> > > > >> > > > >> On Wed, Nov 27, 2019 at 12:29 AM Alexei Scherbakov < > > > alexey.scherbak...@gmail.com> wrote: > > > >>> > > > >>> Igniters, > > > >>> > > > >>> As a final action in my almost year long effort in repairing data > > > >>> consistency related issues for transactional persistent caches I've > > > >>> prepared an article explaining the principles of achieving the > > > consistency > > > >>> between partition copies for these scenarios [1] > > > >>> Comments and suggestions are welcome. > > > >>> I plan to keep the article in actual state if something will > change in > > > this > > > >>> area in the future. > > > >>> Fixes were mostly done under IGNITE-10780 and several follow-up > fixes. > > > >>> > > > >>> There is still work to be done. Currently I'm working on similar > issue > > > for > > > >>> atomic persistent caches [2]. I hope to give an update on that > soon. > > > >>> > > > >>> [1] > > > https://cwiki.apache.org/confluence/display/IGNITE/Data+consistency > > > >>> [2] https://issues.apache.org/jira/browse/IGNITE-11797 > > > >>> -- > > > >>> > > > >>> Best regards, > > > >>> Alexei Scherbakov > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Ivan Pavlukhin > > > > > > > > > > -- > > > > Best regards, > > Alexei Scherbakov > > > > -- > Best regards, > Ivan Pavlukhin > -- Best regards, Alexei Scherbakov
Re: Joining node validation failure event.
I think we also should provide the reason why join failed. > 3 дек. 2019 г., в 12:22, Ivan Pavlukhin написал(а): > > Mikhail, > > So, I suppose there should be ordering guarantees that listener is > registered before first validation failure can occur. Hope > GridComponent#onKernalStart is the right place. Is it enough to pass > only problematic node id (or ClusterNode) with an event? Actually such > event seems to fit naturally node join/left/failed events. > > вт, 3 дек. 2019 г. в 10:38, Mikhail Petrov : >> >> Hi Ivan. >> >> No other lifecycle events are needed in my case. >> >> We can register a listener in the security component's >> GridComponent#onKernalStart method and listen locally to every failed >> joining attempt. Am I missing something? >> >> Regards, >> Mikhail. >> >> On 03.12.2019 8:48, Ivan Pavlukhin wrote: >>> Mikhail, >>> >>> Do you need some ordering guarantees among node lifecycle events and >>> listener notifications. For example, I can imagine that it is good to >>> notify security component about every node failed validation. How can >>> we achieve it with events (I assume dynamic listener registration)? >>> >>> пн, 2 дек. 2019 г. в 18:09, Mikhail Petrov : Hi, Andrey. It doesn't influence on authentication or authorization process. There is a security requirement that demands to notify some outer security subsystems in a specific way when joining node validation failed in any Ignite component (e.g. GridCacheProcessor) not only in IgniteSecurityProcessor. So PluginProvider#validateNewNode is not acceptable for me. Regards, Mikhail. On 02.12.2019 16:35, Andrey Gura wrote: > Mikhail, > > I don't understand how node validation on join affects security. But > it seems that you can use > PluginProvider#validateNewNode(org.apache.ignite.cluster.ClusterNode, > java.io.Serializable) method. Does it fit for your needs? > > On Mon, Dec 2, 2019 at 12:54 PM Mikhail Petrov > wrote: >> Hi, Ivan. >> >> >> Unfortunately, we came to no decision yet. As I mentioned above this >> event is disabled by default and no node will receive this event without >> an explicit subscription. In my case, that event is assumed to be used >> on node locally to share joining node info between security and >> discovery components. I have no idea how to solve this problem without >> publishing a new event too. But I'm concerned about the acceptance of >> that solution. Maybe it can be solved some other way? I will appreciate >> any suggestion or review PR [1] with event implementation. >> >> >> [1] https://github.com/apache/ignite/pull/7057 >> >> >> Regards, >> >> Mikhail. >> >> On 02.12.2019 10:38, Ivan Pavlukhin wrote: >>> Mikhail, Andrey, >>> >>> Have you come to a common decision here? As for me, it sounds useful >>> to expose node join failure details somehow. The thing to decide on is >>> a mechanism to expose it. Unfortunately, immediately have no idea >>> better than using events. >>> What is purpose of the special cluster wide event? Node is not joined to the topology. Why topology nodes should know something about this node? >>> Was this question answered? I suppose that at least coordinator will >>> receive the event, will not it? >>> >>> чт, 28 нояб. 2019 г. в 10:10, Mikhail Petrov : Hi, Ivan. I'm sorry that the discussion was moved in private channel. The problem I'm trying to solve is described below in the thread. The security plugin in my case stores and analyzesinfo about a node that failed to join. Regards, Mikhail. Forwarded Message Subject:Re: Joining node validation failure event. Date: Thu, 21 Nov 2019 21:43:33 +0300 From: Mikhail Petrov To: Andrey Gura Hi, Andrey. In my task security plugin running on the coordinator must locally handle the situation when some node is trying to join the topology with the invalid configuration. I can't handle the response on a node that failed to connect because it's untrusted. Actually I'm only concerned about one validation -- when all Ignite components validate new node. In my case plugin must be able to obtain general and security subject information from joining TcpDiscoveryNode attributes. But I have no idea how to share information between the security and discovery components without recording event and listening it locally. This event is assumed to be disable by default and in my
Re: Joining node validation failure event.
Mikhail, So, I suppose there should be ordering guarantees that listener is registered before first validation failure can occur. Hope GridComponent#onKernalStart is the right place. Is it enough to pass only problematic node id (or ClusterNode) with an event? Actually such event seems to fit naturally node join/left/failed events. вт, 3 дек. 2019 г. в 10:38, Mikhail Petrov : > > Hi Ivan. > > No other lifecycle events are needed in my case. > > We can register a listener in the security component's > GridComponent#onKernalStart method and listen locally to every failed > joining attempt. Am I missing something? > > Regards, > Mikhail. > > On 03.12.2019 8:48, Ivan Pavlukhin wrote: > > Mikhail, > > > > Do you need some ordering guarantees among node lifecycle events and > > listener notifications. For example, I can imagine that it is good to > > notify security component about every node failed validation. How can > > we achieve it with events (I assume dynamic listener registration)? > > > > пн, 2 дек. 2019 г. в 18:09, Mikhail Petrov : > >> Hi, Andrey. > >> > >> It doesn't influence on authentication or authorization process. There > >> is a security requirement that demands to notify some outer security > >> subsystems in a specific way when joining node validation failed in any > >> Ignite component (e.g. GridCacheProcessor) not only in > >> IgniteSecurityProcessor. So PluginProvider#validateNewNode is not > >> acceptable for me. > >> > >> Regards, > >> Mikhail. > >> > >> On 02.12.2019 16:35, Andrey Gura wrote: > >>> Mikhail, > >>> > >>> I don't understand how node validation on join affects security. But > >>> it seems that you can use > >>> PluginProvider#validateNewNode(org.apache.ignite.cluster.ClusterNode, > >>> java.io.Serializable) method. Does it fit for your needs? > >>> > >>> On Mon, Dec 2, 2019 at 12:54 PM Mikhail Petrov > >>> wrote: > Hi, Ivan. > > > Unfortunately, we came to no decision yet. As I mentioned above this > event is disabled by default and no node will receive this event without > an explicit subscription. In my case, that event is assumed to be used > on node locally to share joining node info between security and > discovery components. I have no idea how to solve this problem without > publishing a new event too. But I'm concerned about the acceptance of > that solution. Maybe it can be solved some other way? I will appreciate > any suggestion or review PR [1] with event implementation. > > > [1] https://github.com/apache/ignite/pull/7057 > > > Regards, > > Mikhail. > > On 02.12.2019 10:38, Ivan Pavlukhin wrote: > > Mikhail, Andrey, > > > > Have you come to a common decision here? As for me, it sounds useful > > to expose node join failure details somehow. The thing to decide on is > > a mechanism to expose it. Unfortunately, immediately have no idea > > better than using events. > > > >> What is purpose of the special cluster wide event? Node is not joined > >> to the topology. Why topology nodes should know something about this > >> node? > > Was this question answered? I suppose that at least coordinator will > > receive the event, will not it? > > > > чт, 28 нояб. 2019 г. в 10:10, Mikhail Petrov : > >> Hi, Ivan. > >> > >> I'm sorry that the discussion was moved in private channel. The problem > >> I'm trying to solve is described below in the thread. The security > >> plugin in my case stores and analyzesinfo about a node that failed to > >> join. > >> > >> > >> Regards, > >> > >> Mikhail. > >> > >> > >> > >> Forwarded Message > >> Subject:Re: Joining node validation failure event. > >> Date: Thu, 21 Nov 2019 21:43:33 +0300 > >> From: Mikhail Petrov > >> To: Andrey Gura > >> > >> > >> > >> Hi, Andrey. > >> > >> In my task security plugin running on the coordinator must locally > >> handle the situation when some node is trying to join the topology with > >> the invalid configuration. I can't handle the response on a node that > >> failed to connect because it's untrusted. > >> > >> Actually I'm only concerned about one validation -- when all Ignite > >> components validate new node. > >> > >> In my case plugin must be able to obtain general and security subject > >> information from joining TcpDiscoveryNode attributes. But I have no > >> idea > >> how to share information between the security and discovery components > >> without recording event and listening it locally. > >> > >> This event is assumed to be disable by default and in my case used only > >> on local node so it's not look like "cluster wide event". > >> > >> Also I propose to record this event in dedicated utilityPool so it > >> can't >
