Re: {DISCUSSION] Cluster read-only mode.
Sergey, > On inactive cluster caches aren't started. So you can't get data from cache. As a user I do not understand what does it mean for cache to be started. IgniteCache does not provide start/stop methods. I am trying to figure out why there should be 2 modes (not from an implementation point of view but from a user one). And my concerns are simple. Supporting 2 modes (and 4 combinations) can make more complex both an experience for users and a maintenance for developers. вт, 4 июн. 2019 г. в 16:30, Sergey Antonov : > > Maxim, thank you for reproducer. It looks like a bug. I will fix it! > > вт, 4 июн. 2019 г. в 15:58, Maxim Muzafarov : > > > >> We throw CacheException on each update to read-only cluster. User code > > must handle CacheException correctly .You could find test on it in > > ClusterReadOnlyModeTest#testDataStreamerReadOnly() > > > > In this test, DataStreamer starts when the cluster already changes its > > mode, but not before. Please, check my reproducer [1]. CacheException > > is not thrown. Am I missing something? > > > > [1] > > https://github.com/Mmuzaf/ignite/blob/readonly_streamer/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/ClusterReadOnlyModeTest.java#L72 > > > > On Tue, 4 Jun 2019 at 14:42, Sergey Antonov > > wrote: > > > > > > Hello, Maxim! > > > > > > >> Do we have an IEP for this feature? > > > No, we don't have. > > > > > > >> How we guarantee that all cache operations delivered (or not yet) to > > > backups are not rejected by applied read-only request? > > > I reused cluster activation mechanism. So enabling read-only mode > > generates > > > an exchange on cluster. Cluster exchange guarantees update consistency > > > between primary and backups. > > > > > > >> We should cancel the DataStreamer task or allow it to be finished. > > > We throw CacheException on each update to read-only cluster. User code > > must > > > handle CacheException correctly .You could find test on it in > > > ClusterReadOnlyModeTest#testDataStreamerReadOnly() > > > > > > пн, 3 июн. 2019 г. в 20:15, Maxim Muzafarov : > > > > > > > Sergey, > > > > > > > > Do we have an IEP for this feature? > > > > > > > > What should happen when on an active cluster with put operations if we > > > > receive a read-only state change request? How we guarantee that all > > > > cache operations delivered (or not yet) to backups are not rejected by > > > > applied read-only request? I haven't found such tests in your PR. > > > > > > > > I've downloaded your branch and run some tests locally. I've tried > > > > DataStreamer cache loads (allowOverwrite mode false) with a concurrent > > > > cluster change state request to read-only mode and I've got strange > > > > behaviour. My test scenario was: > > > > 1) Start DataStremer cache load; > > > > 2) change cluster to read-only state; > > > > 3) change state back to normal; > > > > > > > > When the state has been changed to `read-only` I've flooded with a lot > > > > of `Failed to perform cache operation (cluster is in read-only mode)` > > > > errors, but when I've reverted the state back the DataStreamer > > > > continue its load without any error. I think we should not allow such > > > > behaviour. We should cancel the DataStreamer task or allow it to be > > > > finished. > > > > > > > > On Fri, 31 May 2019 at 13:00, Sergey Antonov < > > antonovserge...@gmail.com> > > > > wrote: > > > > > > > > > > Hello, Zhenya, Maxim! > > > > > > > > > > Thank you for your replies! > > > > > > > > > > >> Should we also allow writes to the DistributedMetaStorage and if > > not > > > > why? > > > > > Yes. DistributedMetastorage available for updates with enabled > > read-only > > > > > mode. I added test about it to ClusterReadOnlyModeSelfTest > > > > > > > > > > >> What's the purpose for ignite-sys-cache updates still be > > available ? > > > > > ignite-sys-cache is using in the different subcomponents, for > > example, > > > > > security. > > > > > > > > > > чт, 30 мая 2019 г. в 20:30, Zhenya Stanilovsky > > > > : > > > > > > > > > > > hi, Sergey. > > > > > > What's the purpose for ignite-sys-cache updates still be available > > ? > > > > > > > > > > > > thanks ! > > > > > > > > > > > > > Hello Igniters! > > > > > > > > > > > > > > I'm working on cluster read-only mode [1] feature. In this mode > > > > cluster > > > > > > > will be available only for read operations, all data modification > > > > > > > operations in user caches will be rejected > > > > > > > with ClusterReadOnlyModeCheckedException. This feature could be > > > > helpfull > > > > > > > for maintenance works (control.sh idle_verify/validate_indexes). > > > > > > > > > > > > > > A few points about cluster read-only mode: > > > > > > > 1) Read-only mode could be enabled on active cluster only. > > > > > > > 2) Read-only mode doens't store on PDS (i.e. after cluster > > restart > > > > > > > enabled > > > > > > > read-only mode will be forgotten) > > > > > > > 3) Updates to ignite-sys-cache will b
Re: [DISCUSSION] Release Apache Ignite 2.7.5-rc4
Dmitriy, Are there any differences between 2.7.5-rc4 and a previous RC? вт, 4 июн. 2019 г. в 20:35, Dmitriy Pavlov : > > Dear Community, > > > > Please use this thread for all non-voting, discussion, questions related to > this release. > > > > Cast your vote here > https://lists.apache.org/thread.html/35cbc2d4c5b769155dc8aec15edd808a25c5cf48a5e12637528e931d@%3Cdev.ignite.apache.org%3E > > > Since NuGet is a post-release step, I will proceed with it shortly. > > > > Sincerely, > > Dmitriy Pavlov -- Best regards, Ivan Pavlukhin
Re: [TC] Examples suite issues
Nikolay, 1. Why does your PR show only single file changed? 2. Can you describe the problem once more, please? Is it about :ignite-examples module not being build in ~Build Apache Ignite~ ? > On 4 Jun 2019, at 20:34, Nikolay Izhikov wrote: > > Hello, Igniters. > > Root of the issue is in our maven profiles hierarchy. > > "Build Ignite" suite on TC activate 'all-scala' and examples module have only > 'scala' profile. > To reduce copy-pase I added 'scala' profile of examples module activation > based on system property 'scala-examples'. > > So, for now my PR [1] fixes both issues with examples module: > > 1. Unnecessary dependecies that leads to Spark examples crash. > 2. Spark examples not tested on TC. > > After merge of PR [1] we should add this property to "Build Ignite" suite. > > Is there any objections? > > [1] https://github.com/apache/ignite/pull/6587 > > [2] > https://ci.ignite.apache.org/viewLog.html?buildId=4041670&buildTypeId=IgniteTests24Java8_Examples&tab=testsInfo&branch_IgniteTests24Java8=pull%2F6587%2Fhead > > > В Вт, 04/06/2019 в 16:56 +0300, Nikolay Izhikov пишет: >> I moved jackson dependencies to maven-javadoc-plugin additional dependencies. >> Seems, it solves both issues: Data Frame examples and javadoc generation. >> >> But, Data Frame examples still doesn't shows in tc log. >> >> Will investigate it, shortly. >> >> https://ci.ignite.apache.org/viewLog.html?buildId=4039945&tab=queuedBuildOverviewTab >> >> https://ci.ignite.apache.org/viewLog.html?buildId=4039943&;;; >> >> В Вт, 04/06/2019 в 15:45 +0300, Павлухин Иван пишет: >>> Would be great if it is possible. >>> >>> вт, 4 июн. 2019 г. в 15:43, Nikolay Izhikov : I used to think it's forbidden to use direct version in pom.xml. We should keep all versions in parent pom. В Вт, 04/06/2019 в 15:36 +0300, Павлухин Иван пишет: > Nikolay, > > It was already observed that removal of jackson dependencies from > examples module would lead to a Javadoc build job failure (for > examples module). One way to fix it is changing jackson dependency > versions: > >com.fasterxml.jackson.core >jackson-core >2.6.7 > > > >com.fasterxml.jackson.core >jackson-databind >2.6.7 > > > >com.fasterxml.jackson.core >jackson-annotations >2.6.7 > > > вт, 4 июн. 2019 г. в 14:57, Nikolay Izhikov : >> >> Igniters >> >> I've found the root issue. >> >> jackson dependency was added in IGNITE-10803 and that's brokes Spark >> Examples. >> >> Alex, Yury, please, clarify, do we really need this dependencies? >> >> В Вт, 04/06/2019 в 11:42 +0300, Павлухин Иван пишет: >>> But -Pscala is already activated. But it seems that a build job in >>> question does not compile classes and retrieves a compiled package >>> from ~Build Apache Ignite~ job. Need to check if spark examples are >>> present. >>> >>> вт, 4 июн. 2019 г. в 09:52, Павлухин Иван : Nikolay, It looks like that profile -Pscala should be activated to run IgniteExamplesSparkSelfTestSuite. I was able to start that suite execution on my machine but it failed during execution. вт, 4 июн. 2019 г. в 03:08, Denis Magda : > > Hey Peter, Anton, > > Are any of you able to help Nickolay with [1]? > > - > Denis > > > On Mon, Jun 3, 2019 at 6:17 AM Nikolay Izhikov > wrote: > >> Hello, Igniters. >> >> We have several issues with Examples suite on TC >> >> 1. IgniteExamplesSparkSelfTestSuite seems configure, bu not runs on >> suite >> execution. [1] >> >> 2. Spark Data Frame examples doesn't work in master. Seems some >> classpath >> issues [2] >> >> I will resolve issue #2. >> Can anyone help me with #1? >> >> >> [1] >> https://ci.ignite.apache.org/viewLog.html?buildId=4027327&buildTypeId=IgniteTests24Java8_Examples&tab=buildParameters >> [2] https://issues.apache.org/jira/browse/IGNITE-11881 >> -- Best regards, Ivan Pavlukhin >>> >>> >>> > > > >>> >>> >>>
Re: [TC] Examples suite issues
> 1. Why does your PR show only single file changed? Because this change fix the issue described in the ticket [1] > 2. Can you describe the problem once more, please? Is it about :ignite-examples module not being build in ~Build Apache Ignite~ ? Spark examples activated only if `scala` profile enabled. There is no `scala` profile in "~Build Apache Ignite~", only `all-scala`. [1] https://issues.apache.org/jira/browse/IGNITE-11881 ср, 5 июн. 2019 г. в 10:59, Petr Ivanov : > Nikolay, > > > 1. Why does your PR show only single file changed? > 2. Can you describe the problem once more, please? Is it about > :ignite-examples module not being build in ~Build Apache Ignite~ ? > > > > > On 4 Jun 2019, at 20:34, Nikolay Izhikov wrote: > > > > Hello, Igniters. > > > > Root of the issue is in our maven profiles hierarchy. > > > > "Build Ignite" suite on TC activate 'all-scala' and examples module have > only 'scala' profile. > > To reduce copy-pase I added 'scala' profile of examples module > activation based on system property 'scala-examples'. > > > > So, for now my PR [1] fixes both issues with examples module: > > > > 1. Unnecessary dependecies that leads to Spark examples crash. > > 2. Spark examples not tested on TC. > > > > After merge of PR [1] we should add this property to "Build Ignite" > suite. > > > > Is there any objections? > > > > [1] https://github.com/apache/ignite/pull/6587 > > > > [2] > https://ci.ignite.apache.org/viewLog.html?buildId=4041670&buildTypeId=IgniteTests24Java8_Examples&tab=testsInfo&branch_IgniteTests24Java8=pull%2F6587%2Fhead > > > > > > В Вт, 04/06/2019 в 16:56 +0300, Nikolay Izhikov пишет: > >> I moved jackson dependencies to maven-javadoc-plugin additional > dependencies. > >> Seems, it solves both issues: Data Frame examples and javadoc > generation. > >> > >> But, Data Frame examples still doesn't shows in tc log. > >> > >> Will investigate it, shortly. > >> > >> > https://ci.ignite.apache.org/viewLog.html?buildId=4039945&tab=queuedBuildOverviewTab > >> > >> https://ci.ignite.apache.org/viewLog.html?buildId=4039943&;;; > >> > >> В Вт, 04/06/2019 в 15:45 +0300, Павлухин Иван пишет: > >>> Would be great if it is possible. > >>> > >>> вт, 4 июн. 2019 г. в 15:43, Nikolay Izhikov : > > I used to think it's forbidden to use direct version in pom.xml. > We should keep all versions in parent pom. > > > В Вт, 04/06/2019 в 15:36 +0300, Павлухин Иван пишет: > > Nikolay, > > > > It was already observed that removal of jackson dependencies from > > examples module would lead to a Javadoc build job failure (for > > examples module). One way to fix it is changing jackson dependency > > versions: > > > >com.fasterxml.jackson.core > >jackson-core > >2.6.7 > > > > > > > >com.fasterxml.jackson.core > >jackson-databind > >2.6.7 > > > > > > > >com.fasterxml.jackson.core > >jackson-annotations > >2.6.7 > > > > > > вт, 4 июн. 2019 г. в 14:57, Nikolay Izhikov : > >> > >> Igniters > >> > >> I've found the root issue. > >> > >> jackson dependency was added in IGNITE-10803 and that's brokes > Spark Examples. > >> > >> Alex, Yury, please, clarify, do we really need this dependencies? > >> > >> В Вт, 04/06/2019 в 11:42 +0300, Павлухин Иван пишет: > >>> But -Pscala is already activated. But it seems that a build job in > >>> question does not compile classes and retrieves a compiled package > >>> from ~Build Apache Ignite~ job. Need to check if spark examples are > >>> present. > >>> > >>> вт, 4 июн. 2019 г. в 09:52, Павлухин Иван : > > Nikolay, > > It looks like that profile -Pscala should be activated to run > IgniteExamplesSparkSelfTestSuite. I was able to start that suite > execution on my machine but it failed during execution. > > вт, 4 июн. 2019 г. в 03:08, Denis Magda : > > > > Hey Peter, Anton, > > > > Are any of you able to help Nickolay with [1]? > > > > - > > Denis > > > > > > On Mon, Jun 3, 2019 at 6:17 AM Nikolay Izhikov < > nizhi...@apache.org> wrote: > > > >> Hello, Igniters. > >> > >> We have several issues with Examples suite on TC > >> > >> 1. IgniteExamplesSparkSelfTestSuite seems configure, bu not > runs on suite > >> execution. [1] > >> > >> 2. Spark Data Frame examples doesn't work in master. Seems some > classpath > >> issues [2] > >> > >> I will resolve issue #2. > >> Can anyone help me with #1? > >> > >> > >> [1] > >> > https://ci.ignite.apache.org/viewLog.html?buildId=4027327&buildTypeId=IgniteTests24Java8_Examples&tab=buildParameters > >> [2] https:
Re: {DISCUSSION] Cluster read-only mode.
Ivan, >From a user's point of view: Inactive cluster - deny writes, deny reads. Read-only cluster - deny writes, allow reads. ср, 5 июн. 2019 г. в 10:44, Павлухин Иван : > Sergey, > > > On inactive cluster caches aren't started. So you can't get data from > cache. > > As a user I do not understand what does it mean for cache to be > started. IgniteCache does not provide start/stop methods. > > I am trying to figure out why there should be 2 modes (not from an > implementation point of view but from a user one). And my concerns are > simple. Supporting 2 modes (and 4 combinations) can make more complex > both an experience for users and a maintenance for developers. > > вт, 4 июн. 2019 г. в 16:30, Sergey Antonov : > > > > Maxim, thank you for reproducer. It looks like a bug. I will fix it! > > > > вт, 4 июн. 2019 г. в 15:58, Maxim Muzafarov : > > > > > >> We throw CacheException on each update to read-only cluster. User > code > > > must handle CacheException correctly .You could find test on it in > > > ClusterReadOnlyModeTest#testDataStreamerReadOnly() > > > > > > In this test, DataStreamer starts when the cluster already changes its > > > mode, but not before. Please, check my reproducer [1]. CacheException > > > is not thrown. Am I missing something? > > > > > > [1] > > > > https://github.com/Mmuzaf/ignite/blob/readonly_streamer/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/ClusterReadOnlyModeTest.java#L72 > > > > > > On Tue, 4 Jun 2019 at 14:42, Sergey Antonov > > > > wrote: > > > > > > > > Hello, Maxim! > > > > > > > > >> Do we have an IEP for this feature? > > > > No, we don't have. > > > > > > > > >> How we guarantee that all cache operations delivered (or not yet) > to > > > > backups are not rejected by applied read-only request? > > > > I reused cluster activation mechanism. So enabling read-only mode > > > generates > > > > an exchange on cluster. Cluster exchange guarantees update > consistency > > > > between primary and backups. > > > > > > > > >> We should cancel the DataStreamer task or allow it to be finished. > > > > We throw CacheException on each update to read-only cluster. User > code > > > must > > > > handle CacheException correctly .You could find test on it in > > > > ClusterReadOnlyModeTest#testDataStreamerReadOnly() > > > > > > > > пн, 3 июн. 2019 г. в 20:15, Maxim Muzafarov : > > > > > > > > > Sergey, > > > > > > > > > > Do we have an IEP for this feature? > > > > > > > > > > What should happen when on an active cluster with put operations > if we > > > > > receive a read-only state change request? How we guarantee that all > > > > > cache operations delivered (or not yet) to backups are not > rejected by > > > > > applied read-only request? I haven't found such tests in your PR. > > > > > > > > > > I've downloaded your branch and run some tests locally. I've tried > > > > > DataStreamer cache loads (allowOverwrite mode false) with a > concurrent > > > > > cluster change state request to read-only mode and I've got strange > > > > > behaviour. My test scenario was: > > > > > 1) Start DataStremer cache load; > > > > > 2) change cluster to read-only state; > > > > > 3) change state back to normal; > > > > > > > > > > When the state has been changed to `read-only` I've flooded with a > lot > > > > > of `Failed to perform cache operation (cluster is in read-only > mode)` > > > > > errors, but when I've reverted the state back the DataStreamer > > > > > continue its load without any error. I think we should not allow > such > > > > > behaviour. We should cancel the DataStreamer task or allow it to be > > > > > finished. > > > > > > > > > > On Fri, 31 May 2019 at 13:00, Sergey Antonov < > > > antonovserge...@gmail.com> > > > > > wrote: > > > > > > > > > > > > Hello, Zhenya, Maxim! > > > > > > > > > > > > Thank you for your replies! > > > > > > > > > > > > >> Should we also allow writes to the DistributedMetaStorage and > if > > > not > > > > > why? > > > > > > Yes. DistributedMetastorage available for updates with enabled > > > read-only > > > > > > mode. I added test about it to ClusterReadOnlyModeSelfTest > > > > > > > > > > > > >> What's the purpose for ignite-sys-cache updates still be > > > available ? > > > > > > ignite-sys-cache is using in the different subcomponents, for > > > example, > > > > > > security. > > > > > > > > > > > > чт, 30 мая 2019 г. в 20:30, Zhenya Stanilovsky > > > > > : > > > > > > > > > > > > > hi, Sergey. > > > > > > > What's the purpose for ignite-sys-cache updates still be > available > > > ? > > > > > > > > > > > > > > thanks ! > > > > > > > > > > > > > > > Hello Igniters! > > > > > > > > > > > > > > > > I'm working on cluster read-only mode [1] feature. In this > mode > > > > > cluster > > > > > > > > will be available only for read operations, all data > modification > > > > > > > > operations in user caches will be rejected > > > > > > > > with ClusterReadOnlyModeCheckedException. This feature could > be > > > > > helpf
Re: [TC] Examples suite issues
> On 5 Jun 2019, at 11:49, Nikolay Izhikov wrote: > >> 1. Why does your PR show only single file changed? > > Because this change fix the issue described in the ticket [1] So changes in intermediate commits were reverted in branch? > >> 2. Can you describe the problem once more, please? Is it about > :ignite-examples module not being build in ~Build Apache Ignite~ ? > > Spark examples activated only if `scala` profile enabled. > There is no `scala` profile in "~Build Apache Ignite~", only `all-scala`. Let's add 'scala' profile into build. > > > > [1] https://issues.apache.org/jira/browse/IGNITE-11881 > > ср, 5 июн. 2019 г. в 10:59, Petr Ivanov : > >> Nikolay, >> >> >> 1. Why does your PR show only single file changed? >> 2. Can you describe the problem once more, please? Is it about >> :ignite-examples module not being build in ~Build Apache Ignite~ ? >> >> >> >>> On 4 Jun 2019, at 20:34, Nikolay Izhikov wrote: >>> >>> Hello, Igniters. >>> >>> Root of the issue is in our maven profiles hierarchy. >>> >>> "Build Ignite" suite on TC activate 'all-scala' and examples module have >> only 'scala' profile. >>> To reduce copy-pase I added 'scala' profile of examples module >> activation based on system property 'scala-examples'. >>> >>> So, for now my PR [1] fixes both issues with examples module: >>> >>> 1. Unnecessary dependecies that leads to Spark examples crash. >>> 2. Spark examples not tested on TC. >>> >>> After merge of PR [1] we should add this property to "Build Ignite" >> suite. >>> >>> Is there any objections? >>> >>> [1] https://github.com/apache/ignite/pull/6587 >>> >>> [2] >> https://ci.ignite.apache.org/viewLog.html?buildId=4041670&buildTypeId=IgniteTests24Java8_Examples&tab=testsInfo&branch_IgniteTests24Java8=pull%2F6587%2Fhead >>> >>> >>> В Вт, 04/06/2019 в 16:56 +0300, Nikolay Izhikov пишет: I moved jackson dependencies to maven-javadoc-plugin additional >> dependencies. Seems, it solves both issues: Data Frame examples and javadoc >> generation. But, Data Frame examples still doesn't shows in tc log. Will investigate it, shortly. >> https://ci.ignite.apache.org/viewLog.html?buildId=4039945&tab=queuedBuildOverviewTab https://ci.ignite.apache.org/viewLog.html?buildId=4039943&;;; В Вт, 04/06/2019 в 15:45 +0300, Павлухин Иван пишет: > Would be great if it is possible. > > вт, 4 июн. 2019 г. в 15:43, Nikolay Izhikov : >> >> I used to think it's forbidden to use direct version in pom.xml. >> We should keep all versions in parent pom. >> >> >> В Вт, 04/06/2019 в 15:36 +0300, Павлухин Иван пишет: >>> Nikolay, >>> >>> It was already observed that removal of jackson dependencies from >>> examples module would lead to a Javadoc build job failure (for >>> examples module). One way to fix it is changing jackson dependency >>> versions: >>> >>> com.fasterxml.jackson.core >>> jackson-core >>> 2.6.7 >>> >>> >>> >>> com.fasterxml.jackson.core >>> jackson-databind >>> 2.6.7 >>> >>> >>> >>> com.fasterxml.jackson.core >>> jackson-annotations >>> 2.6.7 >>> >>> >>> вт, 4 июн. 2019 г. в 14:57, Nikolay Izhikov : Igniters I've found the root issue. jackson dependency was added in IGNITE-10803 and that's brokes >> Spark Examples. Alex, Yury, please, clarify, do we really need this dependencies? В Вт, 04/06/2019 в 11:42 +0300, Павлухин Иван пишет: > But -Pscala is already activated. But it seems that a build job in > question does not compile classes and retrieves a compiled package > from ~Build Apache Ignite~ job. Need to check if spark examples are > present. > > вт, 4 июн. 2019 г. в 09:52, Павлухин Иван : >> >> Nikolay, >> >> It looks like that profile -Pscala should be activated to run >> IgniteExamplesSparkSelfTestSuite. I was able to start that suite >> execution on my machine but it failed during execution. >> >> вт, 4 июн. 2019 г. в 03:08, Denis Magda : >>> >>> Hey Peter, Anton, >>> >>> Are any of you able to help Nickolay with [1]? >>> >>> - >>> Denis >>> >>> >>> On Mon, Jun 3, 2019 at 6:17 AM Nikolay Izhikov < >> nizhi...@apache.org> wrote: >>> Hello, Igniters. We have several issues with Examples suite on TC 1. IgniteExamplesSparkSelfTestSuite seems configure, bu not >> runs on suite execution. [1] 2. Spark Data Frame examples doesn't work in master. Seems some >> classpath issues [2] I will resolve
Re: {DISCUSSION] Cluster read-only mode.
Alex and all, > From a user's point of view: > Inactive cluster - deny writes, deny reads. > Read-only cluster - deny writes, allow reads. I would be great to explain what is the need to deny both reads and writes. What tasks are not possible to solve when cluster is in read-only state? ср, 5 июн. 2019 г. в 11:55, Alex Plehanov : > > Ivan, > > From a user's point of view: > Inactive cluster - deny writes, deny reads. > Read-only cluster - deny writes, allow reads. > > ср, 5 июн. 2019 г. в 10:44, Павлухин Иван : > > > Sergey, > > > > > On inactive cluster caches aren't started. So you can't get data from > > cache. > > > > As a user I do not understand what does it mean for cache to be > > started. IgniteCache does not provide start/stop methods. > > > > I am trying to figure out why there should be 2 modes (not from an > > implementation point of view but from a user one). And my concerns are > > simple. Supporting 2 modes (and 4 combinations) can make more complex > > both an experience for users and a maintenance for developers. > > > > вт, 4 июн. 2019 г. в 16:30, Sergey Antonov : > > > > > > Maxim, thank you for reproducer. It looks like a bug. I will fix it! > > > > > > вт, 4 июн. 2019 г. в 15:58, Maxim Muzafarov : > > > > > > > >> We throw CacheException on each update to read-only cluster. User > > code > > > > must handle CacheException correctly .You could find test on it in > > > > ClusterReadOnlyModeTest#testDataStreamerReadOnly() > > > > > > > > In this test, DataStreamer starts when the cluster already changes its > > > > mode, but not before. Please, check my reproducer [1]. CacheException > > > > is not thrown. Am I missing something? > > > > > > > > [1] > > > > > > https://github.com/Mmuzaf/ignite/blob/readonly_streamer/modules/core/src/test/java/org/apache/ignite/internal/processors/cache/ClusterReadOnlyModeTest.java#L72 > > > > > > > > On Tue, 4 Jun 2019 at 14:42, Sergey Antonov > > > > > > wrote: > > > > > > > > > > Hello, Maxim! > > > > > > > > > > >> Do we have an IEP for this feature? > > > > > No, we don't have. > > > > > > > > > > >> How we guarantee that all cache operations delivered (or not yet) > > to > > > > > backups are not rejected by applied read-only request? > > > > > I reused cluster activation mechanism. So enabling read-only mode > > > > generates > > > > > an exchange on cluster. Cluster exchange guarantees update > > consistency > > > > > between primary and backups. > > > > > > > > > > >> We should cancel the DataStreamer task or allow it to be finished. > > > > > We throw CacheException on each update to read-only cluster. User > > code > > > > must > > > > > handle CacheException correctly .You could find test on it in > > > > > ClusterReadOnlyModeTest#testDataStreamerReadOnly() > > > > > > > > > > пн, 3 июн. 2019 г. в 20:15, Maxim Muzafarov : > > > > > > > > > > > Sergey, > > > > > > > > > > > > Do we have an IEP for this feature? > > > > > > > > > > > > What should happen when on an active cluster with put operations > > if we > > > > > > receive a read-only state change request? How we guarantee that all > > > > > > cache operations delivered (or not yet) to backups are not > > rejected by > > > > > > applied read-only request? I haven't found such tests in your PR. > > > > > > > > > > > > I've downloaded your branch and run some tests locally. I've tried > > > > > > DataStreamer cache loads (allowOverwrite mode false) with a > > concurrent > > > > > > cluster change state request to read-only mode and I've got strange > > > > > > behaviour. My test scenario was: > > > > > > 1) Start DataStremer cache load; > > > > > > 2) change cluster to read-only state; > > > > > > 3) change state back to normal; > > > > > > > > > > > > When the state has been changed to `read-only` I've flooded with a > > lot > > > > > > of `Failed to perform cache operation (cluster is in read-only > > mode)` > > > > > > errors, but when I've reverted the state back the DataStreamer > > > > > > continue its load without any error. I think we should not allow > > such > > > > > > behaviour. We should cancel the DataStreamer task or allow it to be > > > > > > finished. > > > > > > > > > > > > On Fri, 31 May 2019 at 13:00, Sergey Antonov < > > > > antonovserge...@gmail.com> > > > > > > wrote: > > > > > > > > > > > > > > Hello, Zhenya, Maxim! > > > > > > > > > > > > > > Thank you for your replies! > > > > > > > > > > > > > > >> Should we also allow writes to the DistributedMetaStorage and > > if > > > > not > > > > > > why? > > > > > > > Yes. DistributedMetastorage available for updates with enabled > > > > read-only > > > > > > > mode. I added test about it to ClusterReadOnlyModeSelfTest > > > > > > > > > > > > > > >> What's the purpose for ignite-sys-cache updates still be > > > > available ? > > > > > > > ignite-sys-cache is using in the different subcomponents, for > > > > example, > > > > > > > security. > > > > > > > > > > > > > > чт, 30 мая 2019 г. в 20:30, Zhenya Stanilovsky > > > >
Re: [TC] Examples suite issues
Petr. Current PR solves the following issue: "Data Frame examples crashes in master". We need to solve one more issue: "Data Frame examples not tested on TC" - this require "scala" profile in "Build Ignite" suite. В Ср, 05/06/2019 в 12:05 +0300, Petr Ivanov пишет: > > On 5 Jun 2019, at 11:49, Nikolay Izhikov wrote: > > > > > 1. Why does your PR show only single file changed? > > > > Because this change fix the issue described in the ticket [1] > > So changes in intermediate commits were reverted in branch? > > > > > > > 2. Can you describe the problem once more, please? Is it about > > > > :ignite-examples module not being build in ~Build Apache Ignite~ ? > > > > Spark examples activated only if `scala` profile enabled. > > There is no `scala` profile in "~Build Apache Ignite~", only `all-scala`. > > Let's add 'scala' profile into build. > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-11881 > > > > ср, 5 июн. 2019 г. в 10:59, Petr Ivanov : > > > > > Nikolay, > > > > > > > > > 1. Why does your PR show only single file changed? > > > 2. Can you describe the problem once more, please? Is it about > > > :ignite-examples module not being build in ~Build Apache Ignite~ ? > > > > > > > > > > > > > On 4 Jun 2019, at 20:34, Nikolay Izhikov wrote: > > > > > > > > Hello, Igniters. > > > > > > > > Root of the issue is in our maven profiles hierarchy. > > > > > > > > "Build Ignite" suite on TC activate 'all-scala' and examples module have > > > > > > only 'scala' profile. > > > > To reduce copy-pase I added 'scala' profile of examples module > > > > > > activation based on system property 'scala-examples'. > > > > > > > > So, for now my PR [1] fixes both issues with examples module: > > > > > > > > 1. Unnecessary dependecies that leads to Spark examples crash. > > > > 2. Spark examples not tested on TC. > > > > > > > > After merge of PR [1] we should add this property to "Build Ignite" > > > > > > suite. > > > > > > > > Is there any objections? > > > > > > > > [1] https://github.com/apache/ignite/pull/6587 > > > > > > > > [2] > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4041670&buildTypeId=IgniteTests24Java8_Examples&tab=testsInfo&branch_IgniteTests24Java8=pull%2F6587%2Fhead > > > > > > > > > > > > В Вт, 04/06/2019 в 16:56 +0300, Nikolay Izhikov пишет: > > > > > I moved jackson dependencies to maven-javadoc-plugin additional > > > > > > dependencies. > > > > > Seems, it solves both issues: Data Frame examples and javadoc > > > > > > generation. > > > > > > > > > > But, Data Frame examples still doesn't shows in tc log. > > > > > > > > > > Will investigate it, shortly. > > > > > > > > > > > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4039945&tab=queuedBuildOverviewTab > > > > > > > > > > https://ci.ignite.apache.org/viewLog.html?buildId=4039943& > > > > > > > > > > В Вт, 04/06/2019 в 15:45 +0300, Павлухин Иван пишет: > > > > > > Would be great if it is possible. > > > > > > > > > > > > вт, 4 июн. 2019 г. в 15:43, Nikolay Izhikov : > > > > > > > > > > > > > > I used to think it's forbidden to use direct version in pom.xml. > > > > > > > We should keep all versions in parent pom. > > > > > > > > > > > > > > > > > > > > > В Вт, 04/06/2019 в 15:36 +0300, Павлухин Иван пишет: > > > > > > > > Nikolay, > > > > > > > > > > > > > > > > It was already observed that removal of jackson dependencies > > > > > > > > from > > > > > > > > examples module would lead to a Javadoc build job failure (for > > > > > > > > examples module). One way to fix it is changing jackson > > > > > > > > dependency > > > > > > > > versions: > > > > > > > > > > > > > > > > com.fasterxml.jackson.core > > > > > > > > jackson-core > > > > > > > > 2.6.7 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > com.fasterxml.jackson.core > > > > > > > > jackson-databind > > > > > > > > 2.6.7 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > com.fasterxml.jackson.core > > > > > > > > jackson-annotations > > > > > > > > 2.6.7 > > > > > > > > > > > > > > > > > > > > > > > > вт, 4 июн. 2019 г. в 14:57, Nikolay Izhikov > > > > > > > > : > > > > > > > > > > > > > > > > > > Igniters > > > > > > > > > > > > > > > > > > I've found the root issue. > > > > > > > > > > > > > > > > > > jackson dependency was added in IGNITE-10803 and that's brokes > > > > > > Spark Examples. > > > > > > > > > > > > > > > > > > Alex, Yury, please, clarify, do we really need this > > > > > > > > > dependencies? > > > > > > > > > > > > > > > > > > В Вт, 04/06/2019 в 11:42 +0300, Павлухин Иван пишет: > > > > > > > > > > But -Pscala is already activated. But it seems that a build > > > > > > > > > > job in > > > > > > > > > > question does not compile classes and retrieves a compiled > > > > > > > > > > package > > > > > > > > > > from ~Build Apache Ignite~ job. Need to check if spark > > > > > > > > > > examples are >
Re: Lightweight version of partitions map exchange
Maksim, I agree with you that we should implement current issue and do not allow lightweight PME if there are MOVING partitions in the cluster. But now I'm investigating issue about finalizing update counters cause it assumes that finalizing happens on exchange and all cache updates are completed. Here we can wrong process update counters gaps and break recently merged IGNITE-10078. And about phase 2, correct me if I misunderstood you. You suggest do not move primary partitions on rebalancing completing (do not change affinity assignment)? In this case, nodes recently join to cluster will not have primary partitions and won't get a load after rebalancing. чт, 30 мая 2019 г. в 19:55, Maxim Muzafarov : > > Igniters, > > > I've looked through Nikita's changes and I think for the current issue > [1] we should not allow the existence of MOVING partitions in the > cluster (it must be stable) to run the lightweight PME on BLT node > leave event occurred to achieve truly unlocked operations and here are > my thoughts why. > > In general, as Nikita mentioned above, the existence of MOVING > partitions in the cluster means that the rebalance procedure is > currently running. It owns cache partitions locally and sends in the > background (with additional timeout) the actual statuses of his local > partitions to the coordinator node. So, we will always have a lag > between local node partition states and all other cluster nodes > partitions states. This lag can be very huge since previous > #scheduleResendPartitions() is cancelled when a new cache group > rebalance finished. Without the fair partition states synchronization > (without full PME) and in case of local affinity recalculation on BLT > node leave event, other nodes will mark such partitions LOST in most > of the cases, which in fact are present in the cluster and saved on > some node under checkpoint. I see that it cannot be solved by saving > transition states of such partitions on each node. > > As for the case when the coordinator will calculate affinity and send > "full map" to other nodes, I think it is better here to focus on > designing a new lightweight PME when the rebalancing process finishes. > Сurrently full distributed PME will occur anyway by the coordinator by > sending CacheAffinityChaneMessage, but I think we can avoid it here, > since no new MOVING or OWNING node partition states are introduced and > all the previous mappings are still valid. We don't need a distributed > PME if we will leave partition primaries on those nodes where they > were, just set correct partition statuses via a light discovery > message. > > So, my plan here can be: > Phase 1. Lightweight PME on BLT node leave on a stable cluster (no > MOVING partitions); > Phase 2. Lightweight PME on BLT node finishes its rebalance procedure. > > Folks, Nikita, > WDYT? > > [1] https://issues.apache.org/jira/browse/IGNITE-9913 > > On Fri, 24 May 2019 at 13:31, Nikita Amelchev wrote: > > > > Hello, Igniters! > > > > I am working on the implementation of lightweight PME for the case of > > a BLT node leave. [1] > > > > There is a question: whether to allow lightweight PME if the cluster > > has MOVING partitions? > > > > The problems that may happen if allow: > > - Nodes can differently select the primary node from current OWNING > > backups. > > - One part of nodes can mark a partition as LOST and another one as OWNING. > > > > We can take states of the partitions from the node2part map. The root > > cause of those problems is that when rebalancing ends (get the last > > message), it updates partition state of the local node to OWNING (and > > schedules partitions resend). This may lead to different affinity > > re-calculations on nodes. > > > > I see two solutions: > > > > 1. Nodes will store “moving-owning” transition of partitions state > > until the rebalancing ends. Each node will locally recalculate the > > affinity on node left event. > > 2. The coordinator will calculate affinity and send "full map" to > > nodes. In this case, nodes still should wait for topology change event > > (to get correct topology in discovery). > > > > If disallow lightweight PME when the cluster has MOVING partitions - > > there are no problems and it works fine. > > > > Any thoughts? > > > > 1. https://issues.apache.org/jira/browse/IGNITE-9913 > > > > пт, 29 мар. 2019 г. в 15:00, Nikita Amelchev : > > > > > > Pavel, > > > I have provided MTCGA bot status in Jira issue comments. [1] > > > > > > Eduard, > > > Yes, for current implementation it will be distributed PME if > > > in-memory caches configured. > > > > > > 1. https://issues.apache.org/jira/browse/IGNITE-9913 > > > > > > пт, 29 мар. 2019 г. в 14:49, Eduard Shangareev > > > : > > > > > > > > Nikita, > > > > > > > > It sounds cool. But I didn't get about in-memory caches. The baseline is > > > > not used for their affinity calculation. > > > > So, this improvement would be switched off for them or completely (when > > > > such cache
Re: [TC] Examples suite issues
I've added 'scala' profile to ~Build Apache Ignite~. Can you check please that everything is OK now? > On 5 Jun 2019, at 12:16, Nikolay Izhikov wrote: > > Petr. > > Current PR solves the following issue: "Data Frame examples crashes in > master". > > We need to solve one more issue: > > "Data Frame examples not tested on TC" - this require "scala" profile in > "Build Ignite" suite. > > > В Ср, 05/06/2019 в 12:05 +0300, Petr Ivanov пишет: >>> On 5 Jun 2019, at 11:49, Nikolay Izhikov wrote: >>> 1. Why does your PR show only single file changed? >>> >>> Because this change fix the issue described in the ticket [1] >> >> So changes in intermediate commits were reverted in branch? >> >> >>> 2. Can you describe the problem once more, please? Is it about >>> >>> :ignite-examples module not being build in ~Build Apache Ignite~ ? >>> >>> Spark examples activated only if `scala` profile enabled. >>> There is no `scala` profile in "~Build Apache Ignite~", only `all-scala`. >> >> Let's add 'scala' profile into build. >> >> >>> >>> >>> >>> [1] https://issues.apache.org/jira/browse/IGNITE-11881 >>> >>> ср, 5 июн. 2019 г. в 10:59, Petr Ivanov : >>> Nikolay, 1. Why does your PR show only single file changed? 2. Can you describe the problem once more, please? Is it about :ignite-examples module not being build in ~Build Apache Ignite~ ? > On 4 Jun 2019, at 20:34, Nikolay Izhikov wrote: > > Hello, Igniters. > > Root of the issue is in our maven profiles hierarchy. > > "Build Ignite" suite on TC activate 'all-scala' and examples module have only 'scala' profile. > To reduce copy-pase I added 'scala' profile of examples module activation based on system property 'scala-examples'. > > So, for now my PR [1] fixes both issues with examples module: > > 1. Unnecessary dependecies that leads to Spark examples crash. > 2. Spark examples not tested on TC. > > After merge of PR [1] we should add this property to "Build Ignite" suite. > > Is there any objections? > > [1] https://github.com/apache/ignite/pull/6587 > > [2] https://ci.ignite.apache.org/viewLog.html?buildId=4041670&buildTypeId=IgniteTests24Java8_Examples&tab=testsInfo&branch_IgniteTests24Java8=pull%2F6587%2Fhead > > > В Вт, 04/06/2019 в 16:56 +0300, Nikolay Izhikov пишет: >> I moved jackson dependencies to maven-javadoc-plugin additional dependencies. >> Seems, it solves both issues: Data Frame examples and javadoc generation. >> >> But, Data Frame examples still doesn't shows in tc log. >> >> Will investigate it, shortly. >> >> https://ci.ignite.apache.org/viewLog.html?buildId=4039945&tab=queuedBuildOverviewTab >> >> https://ci.ignite.apache.org/viewLog.html?buildId=4039943& >> >> В Вт, 04/06/2019 в 15:45 +0300, Павлухин Иван пишет: >>> Would be great if it is possible. >>> >>> вт, 4 июн. 2019 г. в 15:43, Nikolay Izhikov : I used to think it's forbidden to use direct version in pom.xml. We should keep all versions in parent pom. В Вт, 04/06/2019 в 15:36 +0300, Павлухин Иван пишет: > Nikolay, > > It was already observed that removal of jackson dependencies from > examples module would lead to a Javadoc build job failure (for > examples module). One way to fix it is changing jackson dependency > versions: > > com.fasterxml.jackson.core > jackson-core > 2.6.7 > > > > com.fasterxml.jackson.core > jackson-databind > 2.6.7 > > > > com.fasterxml.jackson.core > jackson-annotations > 2.6.7 > > > вт, 4 июн. 2019 г. в 14:57, Nikolay Izhikov : >> >> Igniters >> >> I've found the root issue. >> >> jackson dependency was added in IGNITE-10803 and that's brokes Spark Examples. >> >> Alex, Yury, please, clarify, do we really need this dependencies? >> >> В Вт, 04/06/2019 в 11:42 +0300, Павлухин Иван пишет: >>> But -Pscala is already activated. But it seems that a build job in >>> question does not compile classes and retrieves a compiled package >>> from ~Build Apache Ignite~ job. Need to check if spark examples are >>> present. >>> >>> вт, 4 июн. 2019 г. в 09:52, Павлухин Иван : Nikolay, It looks like that profile -Pscala should be activated to run IgniteExamplesSparkSelfTestSuite. I was able to start that suite execution on my machine but it failed during e
[jira] [Created] (IGNITE-11895) .NET Resharper inspections got broken after update
Alexandr Shapkin created IGNITE-11895: - Summary: .NET Resharper inspections got broken after update Key: IGNITE-11895 URL: https://issues.apache.org/jira/browse/IGNITE-11895 Project: Ignite Issue Type: Improvement Components: platforms Affects Versions: 2.7 Reporter: Alexandr Shapkin Assignee: Alexandr Shapkin Resharper Inspection add new violation after migration from 2018.1.4 to version 2019.1 Lets suppress this new warning -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [TC] Examples suite issues
Folks, I've checked PR [1] and it seems to me it fixes the main issue with a spark examples. After adding `scala` profile the [Example] Suite in the master branch starts to fail - [2] and it works fine in the PR branch [3] (Javadocs also compiles). Not sure, but adding this profile also fixes the [Check Code Style] Suite too which is failing [4] in the master branch on compile `ignite-scalar` module. I will investigate it more shortly. [1] https://github.com/apache/ignite/pull/6587 [2] https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Examples&branch_IgniteTests24Java8=%3Cdefault%3E&tab=buildTypeStatusDiv [3] https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_Examples&branch_IgniteTests24Java8=pull%2F6587%2Fhead&tab=buildTypeStatusDiv [4] https://ci.ignite.apache.org/viewLog.html?buildId=4046113&buildTypeId=IgniteTests24Java8_CheckCodeStyle On Wed, 5 Jun 2019 at 12:35, Petr Ivanov wrote: > > I've added 'scala' profile to ~Build Apache Ignite~. > Can you check please that everything is OK now? > > > On 5 Jun 2019, at 12:16, Nikolay Izhikov wrote: > > > > Petr. > > > > Current PR solves the following issue: "Data Frame examples crashes in > > master". > > > > We need to solve one more issue: > > > > "Data Frame examples not tested on TC" - this require "scala" profile in > > "Build Ignite" suite. > > > > > > В Ср, 05/06/2019 в 12:05 +0300, Petr Ivanov пишет: > >>> On 5 Jun 2019, at 11:49, Nikolay Izhikov wrote: > >>> > 1. Why does your PR show only single file changed? > >>> > >>> Because this change fix the issue described in the ticket [1] > >> > >> So changes in intermediate commits were reverted in branch? > >> > >> > >>> > 2. Can you describe the problem once more, please? Is it about > >>> > >>> :ignite-examples module not being build in ~Build Apache Ignite~ ? > >>> > >>> Spark examples activated only if `scala` profile enabled. > >>> There is no `scala` profile in "~Build Apache Ignite~", only `all-scala`. > >> > >> Let's add 'scala' profile into build. > >> > >> > >>> > >>> > >>> > >>> [1] https://issues.apache.org/jira/browse/IGNITE-11881 > >>> > >>> ср, 5 июн. 2019 г. в 10:59, Petr Ivanov : > >>> > Nikolay, > > > 1. Why does your PR show only single file changed? > 2. Can you describe the problem once more, please? Is it about > :ignite-examples module not being build in ~Build Apache Ignite~ ? > > > > > On 4 Jun 2019, at 20:34, Nikolay Izhikov wrote: > > > > Hello, Igniters. > > > > Root of the issue is in our maven profiles hierarchy. > > > > "Build Ignite" suite on TC activate 'all-scala' and examples module have > > only 'scala' profile. > > To reduce copy-pase I added 'scala' profile of examples module > > activation based on system property 'scala-examples'. > > > > So, for now my PR [1] fixes both issues with examples module: > > > > 1. Unnecessary dependecies that leads to Spark examples crash. > > 2. Spark examples not tested on TC. > > > > After merge of PR [1] we should add this property to "Build Ignite" > > suite. > > > > Is there any objections? > > > > [1] https://github.com/apache/ignite/pull/6587 > > > > [2] > > https://ci.ignite.apache.org/viewLog.html?buildId=4041670&buildTypeId=IgniteTests24Java8_Examples&tab=testsInfo&branch_IgniteTests24Java8=pull%2F6587%2Fhead > > > > > > В Вт, 04/06/2019 в 16:56 +0300, Nikolay Izhikov пишет: > >> I moved jackson dependencies to maven-javadoc-plugin additional > > dependencies. > >> Seems, it solves both issues: Data Frame examples and javadoc > > generation. > >> > >> But, Data Frame examples still doesn't shows in tc log. > >> > >> Will investigate it, shortly. > >> > >> > > https://ci.ignite.apache.org/viewLog.html?buildId=4039945&tab=queuedBuildOverviewTab > >> > >> https://ci.ignite.apache.org/viewLog.html?buildId=4039943& > >> > >> В Вт, 04/06/2019 в 15:45 +0300, Павлухин Иван пишет: > >>> Would be great if it is possible. > >>> > >>> вт, 4 июн. 2019 г. в 15:43, Nikolay Izhikov : > > I used to think it's forbidden to use direct version in pom.xml. > We should keep all versions in parent pom. > > > В Вт, 04/06/2019 в 15:36 +0300, Павлухин Иван пишет: > > Nikolay, > > > > It was already observed that removal of jackson dependencies from > > examples module would lead to a Javadoc build job failure (for > > examples module). One way to fix it is changing jackson dependency > > versions: > > > > com.fasterxml.jackson.core > > jackson-core > > 2.6.7 > > > > > > > > com.fasterxml.jackson.core > > jackson-databind >
[jira] [Created] (IGNITE-11896) [TC Bot] Comment JIRA may not work for case aliased TC is used
Dmitriy Pavlov created IGNITE-11896: --- Summary: [TC Bot] Comment JIRA may not work for case aliased TC is used Key: IGNITE-11896 URL: https://issues.apache.org/jira/browse/IGNITE-11896 Project: Ignite Issue Type: Bug Reporter: Dmitriy Pavlov Assignee: Dmitriy Pavlov 'Comment JIRA' will not work for PR -less contribution in pr.html report -- This message was sent by Atlassian JIRA (v7.6.3#76005)
Re: [DISCUSSION] Release Apache Ignite 2.7.5-rc4
Hi Ivan, In code there are 2 differences: - TLSv1.2 is used as a default protocol for Communication in case SSL is used. - version changed for all pom files, so apache-ignite-src.zip now contains correct version instead of -SNAPSHOT. Sincerely, Dmitriy Pavlov ср, 5 июн. 2019 г. в 10:49, Павлухин Иван : > Dmitriy, > > Are there any differences between 2.7.5-rc4 and a previous RC? > > вт, 4 июн. 2019 г. в 20:35, Dmitriy Pavlov : > > > > Dear Community, > > > > > > > > Please use this thread for all non-voting, discussion, questions related > to > > this release. > > > > > > > > Cast your vote here > > > https://lists.apache.org/thread.html/35cbc2d4c5b769155dc8aec15edd808a25c5cf48a5e12637528e931d@%3Cdev.ignite.apache.org%3E > > > > > > Since NuGet is a post-release step, I will proceed with it shortly. > > > > > > > > Sincerely, > > > > Dmitriy Pavlov > > > > -- > Best regards, > Ivan Pavlukhin >
Re: [VOTE] Release Apache Ignite 2.7.5-rc4
+1
Built C++ and .Net successfully, started .Net <-> Java <-> C++ SSL Cluster
on Java 11 and Java 8 without tuning any flags.
--
Ilya Kasnacheev
ср, 5 июн. 2019 г. в 09:44, Nikolay Izhikov :
> +1 (binding)
>
> В Ср, 05/06/2019 в 01:11 +0100, Denis Magda пишет:
> > +1 (binding)
> >
> > Nice to have clusters nodes started with ignite.sh in my JVM 11
> environment
> > without any settings.
> >
> > -
> > Denis
> >
> >
> > On Tue, Jun 4, 2019 at 6:34 PM Dmitriy Pavlov
> wrote:
> >
> > > Dear Community,
> > >
> > >
> > >
> > > We have uploaded release candidate to
> > > https://dist.apache.org/repos/dist/dev/ignite/2.7.5-rc4/
> > >
> > >
> > >
> > > The following staging can be used for a dependent project for testing:
> > >
> > >
> https://repository.apache.org/content/repositories/orgapacheignite-1463/
> > >
> > >
> > >
> > > This is half-release before 2.8 containing Java 11 support and fixes
> for
> > > Native Persistence storage.
> > >
> > > Tag name is 2.7.5-rc4:
> > >
> > >
> > >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=tag;h=refs/tags/2.7.5-rc4
> > >
> > > 2.7.5 changes:
> > >
> > > * Added Java 11 support
> > >
> > > * Fixed infinite looping during SSL handshake, affecting Java
> 11/Windows
> > >
> > > * Fixed storage corruption case after incorrectly rotated page
> > >
> > > * Erroneous WAL record after incorrectly rotated page processed
> > > automatically
> > >
> > > * Addressed ignite.sh failure on Mac OS and Linux, affecting Java 11
> > >
> > > * Launch scripts and some Ignite initialization steps were fixed for
> Java
> > > 12
> > >
> > > * Fixed indexes corruption on node stop under load
> > >
> > > * Fixed case of node crash during node deactivation
> > >
> > > * Error message with advice about required JVM parameters printed when
> Java
> > > 9+ is used
> > >
> > > * Introduced SYSTEM_CRITICAL_OPERATION_TIMEOUT failure type
> > >
> > >
> > >
> > > Complete list of closed issues:
> > >
> > >
> > >
> https://issues.apache.org/jira/issues/?jql=(project%20%3D%20'Ignite'%20AND%20fixVersion%20in%20('2.7.5')%20AND%20status%20IN%20(Resolved%2C%20Closed))%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20
> > >
> > > DEVNOTES
> > >
> > >
> > >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=DEVNOTES.txt;h=307189059ae70834ead4c127cc18295ce4d0735a;hb=ignite-2.7.5
> > >
> > > RELEASE NOTES
> > >
> > >
> > >
> https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=RELEASE_NOTES.txt;h=0a7d2d395b3c9ca1a69ce3f8f1657940fa12f972;hb=ignite-2.7.5
> > >
> > > The vote is formal, see voting guidelines
> > > https://www.apache.org/foundation/voting.html
> > >
> > > +1 - to accept Apache Ignite 2.7.5-rc4
> > >
> > > 0 - don't care either way
> > >
> > > -1 - DO NOT accept Apache Ignite Ignite 2.7.5-rc4 (explain why)
> > >
> > >
> > >
> > > See notes on how to verify release here
> > >
> > > https://www.apache.org/info/verification.html
> > >
> > > and
> > >
> > >
> https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-ReleaseVerification
> > >
> > > This vote will be open for at least 72 hours till June 7, 18:00 UTC.
> > >
> > >
> https://www.timeanddate.com/countdown/to?year=2019&month=6&day=7&hour=18&min=0&sec=0&p0=utc-1
> > >
> > >
> > >
> > > Best Regards,
> > >
> > > Dmitriy Pavlov
> > >
>
Re: [ML] Deployment of user-defined preprocessors
Alexey,
This is a cool change, do you create a ticket for it?
If no I can create one.
Best regards,
Yuriy Babak
пт, 31 мая 2019 г. в 14:20, Алексей Платонов :
> Hi, Igniters!
> Currently we don't have an ability to deploy automatically user-defined
> preprocessors and vectorizers. Client's code should be deployed manually to
> Ignite server nodes.
>
> I have an idea how to fix it. If we pass user's classloader and one of
> user-defined classes from fit-level to
> ComputeUtils.affinityCallWithRetries() then we wiil be able to use
> GridPeerDeployAware interface to send informtation about this classloader
> to server nodes.
>
> To support this ability we can define interfaces like these:
>
> public interface DeployableObject {
> public List getDependencies();
> }
>
> and
>
> public interface DeployingContext {
> public Class userClass();
> public ClassLoader clientClassLoader();
> }
>
> DeployableObject will be mark for our ignite-ml final classes like trainers
> or concrete preprocessors and it can be able to return all dependencies
> that should be deployed to server nodes if it's needed. If these
> dependencies are DeployableObjects too then depenndencies will be unfolded
> recursively. Classes that isn't defined as DeployableObject will be
> recognized as user-defined (NOTE: all leaf classes in our hierarchy will be
> DeployableObject).
>
> This list of DeployableObjects will be user for define user class loader
> and one of these objects will be used for passing to GridPeerDeployAware.
>
> So, this logic allows to pass user-defined Preprocessors and Vectorizers to
> training algorithms and pipelines.
>
> What do you think?
>
> Sincerely
> Alexey Platonov
>
Re: [VOTE] Release Apache Ignite 2.7.5-rc4
+1 (binding)
On Wed, Jun 5, 2019 at 4:39 PM Ilya Kasnacheev
wrote:
>
> +1
>
> Built C++ and .Net successfully, started .Net <-> Java <-> C++ SSL Cluster
> on Java 11 and Java 8 without tuning any flags.
> --
> Ilya Kasnacheev
>
>
> ср, 5 июн. 2019 г. в 09:44, Nikolay Izhikov :
>
> > +1 (binding)
> >
> > В Ср, 05/06/2019 в 01:11 +0100, Denis Magda пишет:
> > > +1 (binding)
> > >
> > > Nice to have clusters nodes started with ignite.sh in my JVM 11
> > environment
> > > without any settings.
> > >
> > > -
> > > Denis
> > >
> > >
> > > On Tue, Jun 4, 2019 at 6:34 PM Dmitriy Pavlov
> > wrote:
> > >
> > > > Dear Community,
> > > >
> > > >
> > > >
> > > > We have uploaded release candidate to
> > > > https://dist.apache.org/repos/dist/dev/ignite/2.7.5-rc4/
> > > >
> > > >
> > > >
> > > > The following staging can be used for a dependent project for testing:
> > > >
> > > >
> > https://repository.apache.org/content/repositories/orgapacheignite-1463/
> > > >
> > > >
> > > >
> > > > This is half-release before 2.8 containing Java 11 support and fixes
> > for
> > > > Native Persistence storage.
> > > >
> > > > Tag name is 2.7.5-rc4:
> > > >
> > > >
> > > >
> > https://gitbox.apache.org/repos/asf?p=ignite.git;a=tag;h=refs/tags/2.7.5-rc4
> > > >
> > > > 2.7.5 changes:
> > > >
> > > > * Added Java 11 support
> > > >
> > > > * Fixed infinite looping during SSL handshake, affecting Java
> > 11/Windows
> > > >
> > > > * Fixed storage corruption case after incorrectly rotated page
> > > >
> > > > * Erroneous WAL record after incorrectly rotated page processed
> > > > automatically
> > > >
> > > > * Addressed ignite.sh failure on Mac OS and Linux, affecting Java 11
> > > >
> > > > * Launch scripts and some Ignite initialization steps were fixed for
> > Java
> > > > 12
> > > >
> > > > * Fixed indexes corruption on node stop under load
> > > >
> > > > * Fixed case of node crash during node deactivation
> > > >
> > > > * Error message with advice about required JVM parameters printed when
> > Java
> > > > 9+ is used
> > > >
> > > > * Introduced SYSTEM_CRITICAL_OPERATION_TIMEOUT failure type
> > > >
> > > >
> > > >
> > > > Complete list of closed issues:
> > > >
> > > >
> > > >
> > https://issues.apache.org/jira/issues/?jql=(project%20%3D%20'Ignite'%20AND%20fixVersion%20in%20('2.7.5')%20AND%20status%20IN%20(Resolved%2C%20Closed))%20ORDER%20BY%20priority%20%20%20%20%20%20%20%20%20%20%20%20%20
> > > >
> > > > DEVNOTES
> > > >
> > > >
> > > >
> > https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=DEVNOTES.txt;h=307189059ae70834ead4c127cc18295ce4d0735a;hb=ignite-2.7.5
> > > >
> > > > RELEASE NOTES
> > > >
> > > >
> > > >
> > https://gitbox.apache.org/repos/asf?p=ignite.git;a=blob;f=RELEASE_NOTES.txt;h=0a7d2d395b3c9ca1a69ce3f8f1657940fa12f972;hb=ignite-2.7.5
> > > >
> > > > The vote is formal, see voting guidelines
> > > > https://www.apache.org/foundation/voting.html
> > > >
> > > > +1 - to accept Apache Ignite 2.7.5-rc4
> > > >
> > > > 0 - don't care either way
> > > >
> > > > -1 - DO NOT accept Apache Ignite Ignite 2.7.5-rc4 (explain why)
> > > >
> > > >
> > > >
> > > > See notes on how to verify release here
> > > >
> > > > https://www.apache.org/info/verification.html
> > > >
> > > > and
> > > >
> > > >
> > https://cwiki.apache.org/confluence/display/IGNITE/Release+Process#ReleaseProcess-ReleaseVerification
> > > >
> > > > This vote will be open for at least 72 hours till June 7, 18:00 UTC.
> > > >
> > > >
> > https://www.timeanddate.com/countdown/to?year=2019&month=6&day=7&hour=18&min=0&sec=0&p0=utc-1
> > > >
> > > >
> > > >
> > > > Best Regards,
> > > >
> > > > Dmitriy Pavlov
> > > >
> >
Re: [DISCUSSION] Release Apache Ignite 2.7.5-rc4
Hi Igniters, Pavel T., I'm struggling to find clear steps for building/uploading NuGet packages. How can I double check if it was built or not? What should I do? In some internal notes, I've found that NuGet upload can't be undone, and in https://cwiki.apache.org/confluence/display/IGNITE/Release+Process it is a post-release step. Please advise, both here or in private. I will update the instructions accordingly. Sincerely Dmitriy Pavlov ср, 5 июн. 2019 г. в 15:27, Dmitriy Pavlov : > Hi Ivan, > > In code there are 2 differences: > - TLSv1.2 is used as a default protocol for Communication in case SSL is > used. > - version changed for all pom files, so apache-ignite-src.zip now contains > correct version instead of -SNAPSHOT. > > Sincerely, > Dmitriy Pavlov > > > ср, 5 июн. 2019 г. в 10:49, Павлухин Иван : > >> Dmitriy, >> >> Are there any differences between 2.7.5-rc4 and a previous RC? >> >> вт, 4 июн. 2019 г. в 20:35, Dmitriy Pavlov : >> > >> > Dear Community, >> > >> > >> > >> > Please use this thread for all non-voting, discussion, questions >> related to >> > this release. >> > >> > >> > >> > Cast your vote here >> > >> https://lists.apache.org/thread.html/35cbc2d4c5b769155dc8aec15edd808a25c5cf48a5e12637528e931d@%3Cdev.ignite.apache.org%3E >> > >> > >> > Since NuGet is a post-release step, I will proceed with it shortly. >> > >> > >> > >> > Sincerely, >> > >> > Dmitriy Pavlov >> >> >> >> -- >> Best regards, >> Ivan Pavlukhin >> >
Re: [DISCUSSION] Release Apache Ignite 2.7.5-rc4
I've found only this step https://ci.ignite.apache.org/viewType.html?buildTypeId=ApacheIgniteReleaseJava8_PrepareVote3BuildNuGetPackages&tab=buildTypeStatusDiv&branch_Releases_ApacheIgniteMain=__all_branches__ It requires some build counter. Peter I, could you please chime in? ср, 5 июн. 2019 г. в 18:02, Dmitriy Pavlov : > Hi Igniters, Pavel T., > > I'm struggling to find clear steps for building/uploading NuGet packages. > How can I double check if it was built or not? What should I do? > > In some internal notes, I've found that NuGet upload can't be undone, and > in https://cwiki.apache.org/confluence/display/IGNITE/Release+Process it > is a post-release step. > > Please advise, both here or in private. I will update the instructions > accordingly. > > Sincerely > Dmitriy Pavlov > > ср, 5 июн. 2019 г. в 15:27, Dmitriy Pavlov : > >> Hi Ivan, >> >> In code there are 2 differences: >> - TLSv1.2 is used as a default protocol for Communication in case SSL is >> used. >> - version changed for all pom files, so apache-ignite-src.zip now >> contains correct version instead of -SNAPSHOT. >> >> Sincerely, >> Dmitriy Pavlov >> >> >> ср, 5 июн. 2019 г. в 10:49, Павлухин Иван : >> >>> Dmitriy, >>> >>> Are there any differences between 2.7.5-rc4 and a previous RC? >>> >>> вт, 4 июн. 2019 г. в 20:35, Dmitriy Pavlov : >>> > >>> > Dear Community, >>> > >>> > >>> > >>> > Please use this thread for all non-voting, discussion, questions >>> related to >>> > this release. >>> > >>> > >>> > >>> > Cast your vote here >>> > >>> https://lists.apache.org/thread.html/35cbc2d4c5b769155dc8aec15edd808a25c5cf48a5e12637528e931d@%3Cdev.ignite.apache.org%3E >>> > >>> > >>> > Since NuGet is a post-release step, I will proceed with it shortly. >>> > >>> > >>> > >>> > Sincerely, >>> > >>> > Dmitriy Pavlov >>> >>> >>> >>> -- >>> Best regards, >>> Ivan Pavlukhin >>> >>
Re: [DISCUSSION] Release Apache Ignite 2.7.5-rc4
I managed to get some build using build number from 1. Release build step (for that case - 20) The result can be found here https://ci.ignite.apache.org/viewLog.html?buildId=4049646&buildTypeId=ApacheIgniteReleaseJava8_PrepareVote3BuildNuGetPackages&tab=artifacts#%2Fsrc Pavel T., Peter I., Please confirm that it is sufficient. I'll update the release manager notes with steps required to run this configuration. Sincerely Dmitriy Pavlov ср, 5 июн. 2019 г. в 18:07, Dmitriy Pavlov : > I've found only this step > > https://ci.ignite.apache.org/viewType.html?buildTypeId=ApacheIgniteReleaseJava8_PrepareVote3BuildNuGetPackages&tab=buildTypeStatusDiv&branch_Releases_ApacheIgniteMain=__all_branches__ > > > It requires some build counter. > > Peter I, could you please chime in? > > ср, 5 июн. 2019 г. в 18:02, Dmitriy Pavlov : > >> Hi Igniters, Pavel T., >> >> I'm struggling to find clear steps for building/uploading NuGet packages. >> How can I double check if it was built or not? What should I do? >> >> In some internal notes, I've found that NuGet upload can't be undone, and >> in https://cwiki.apache.org/confluence/display/IGNITE/Release+Process it >> is a post-release step. >> >> Please advise, both here or in private. I will update the instructions >> accordingly. >> >> Sincerely >> Dmitriy Pavlov >> >> ср, 5 июн. 2019 г. в 15:27, Dmitriy Pavlov : >> >>> Hi Ivan, >>> >>> In code there are 2 differences: >>> - TLSv1.2 is used as a default protocol for Communication in case SSL is >>> used. >>> - version changed for all pom files, so apache-ignite-src.zip now >>> contains correct version instead of -SNAPSHOT. >>> >>> Sincerely, >>> Dmitriy Pavlov >>> >>> >>> ср, 5 июн. 2019 г. в 10:49, Павлухин Иван : >>> Dmitriy, Are there any differences between 2.7.5-rc4 and a previous RC? вт, 4 июн. 2019 г. в 20:35, Dmitriy Pavlov : > > Dear Community, > > > > Please use this thread for all non-voting, discussion, questions related to > this release. > > > > Cast your vote here > https://lists.apache.org/thread.html/35cbc2d4c5b769155dc8aec15edd808a25c5cf48a5e12637528e931d@%3Cdev.ignite.apache.org%3E > > > Since NuGet is a post-release step, I will proceed with it shortly. > > > > Sincerely, > > Dmitriy Pavlov -- Best regards, Ivan Pavlukhin >>>
[jira] [Created] (IGNITE-11897) Use `bytearray` as a Python type for Ignite `ByteArray`
Dmitry Melnichuk created IGNITE-11897: - Summary: Use `bytearray` as a Python type for Ignite `ByteArray` Key: IGNITE-11897 URL: https://issues.apache.org/jira/browse/IGNITE-11897 Project: Ignite Issue Type: Improvement Components: thin client Reporter: Dmitry Melnichuk Assignee: Dmitry Melnichuk This optimization will allow client to read and write `ByteArray` values without iterating over single bytes, thereby improving performance. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[MTCGA]: new failures in builds [4047322] needs to be handled
Hi Igniters, I've detected some new issue on TeamCity to be handled. You are more than welcomed to help. If your changes can lead to this failure(s): We're grateful that you were a volunteer to make the contribution to this project, but things change and you may no longer be able to finalize your contribution. Could you respond to this email and indicate if you wish to continue and fix test failures or step down and some committer may revert you commit. *New Critical Failure in master Platform .NET https://ci.ignite.apache.org/viewType.html?buildTypeId=IgniteTests24Java8_PlatformNet&branch=%3Cdefault%3E&tab=buildTypeStatusDiv Changes may lead to failure were done by - dmitriy.govoruk...@gmail.com https://ci.ignite.apache.org/viewModification.html?modId=885984 - Here's a reminder of what contributors were agreed to do https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute - Should you have any questions please contact dev@ignite.apache.org Best Regards, Apache Ignite TeamCity Bot https://github.com/apache/ignite-teamcity-bot Notification generated at 08:23:20 06-06-2019
