Re: [ML] Tickets for beginners
Hi, Lev! I'll return with explanation on next week, maybe I need to adds some details to this ticket. P.S. About dataset - you could search among the datasets presented in MLSandbox class or could add your own (I'd prefer small datasets from http://archive.ics.uci.edu/ml/index.php) пт, 14 февр. 2020 г. в 10:02, Лев Киселев : > Hello everyone > I'd like to take following task: > > https://issues.apache.org/jira/browse/IGNITE-12384?jql=project%20%3D%20IGNITE%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20component%20%3D%20ML%20AND%20labels%20%3D%20newbie > > But I do not fully understand what exactly needs to be done. > As far as I understand encoders are needed to convert categorical variables > to numerical. So, to demonstrate how they works I need some dataset with > several categorical features. Should I search for it or generate by myself? > Then, do I need to solve some regression/classification problem in order > to demonstrate "power" of using categorical variables to fit prediction or > classification models? >
[ML] Tickets for beginners
Hello everyone I'd like to take following task: https://issues.apache.org/jira/browse/IGNITE-12384?jql=project%20%3D%20IGNITE%20AND%20status%20in%20(Open%2C%20Reopened)%20AND%20component%20%3D%20ML%20AND%20labels%20%3D%20newbie But I do not fully understand what exactly needs to be done. As far as I understand encoders are needed to convert categorical variables to numerical. So, to demonstrate how they works I need some dataset with several categorical features. Should I search for it or generate by myself? Then, do I need to solve some regression/classification problem in order to demonstrate "power" of using categorical variables to fit prediction or classification models?
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Maxim, There are some of the tasks that are being moved from a release to a release or exist for a while and might be skipped for 2.8 if nobody is willing to document them. A could of examples of such tickets: https://issues.apache.org/jira/browse/IGNITE-10331 https://issues.apache.org/jira/browse/IGNITE-7704 How about preparing a list of the tickets that represent either a new functionality added in 2.8 (new metrics, service grid improvements, etc.) or change in behavior (it might be the case we need to update the baseline topology or rebalancing pages)? Once we get the list, we'll find the names of contributors and they will be able to cooperate with Artem who agreed to assist with this effort. The docs are needed before we announce the release and start bragging about new capabilities. - Denis On Thu, Feb 13, 2020 at 8:46 AM Maxim Muzafarov wrote: > Denis, > > Actually, I've already filtered documentation issues previously and > left only major documentation tasks. Should I shrink the list more? > > On Thu, 13 Feb 2020 at 18:58, Denis Magda wrote: > > > > Maxim, > > > > Thanks for the list. How many of those tickets relate to new capabilities > > or changed behavior in 2.8? You can probably come up with such a > sub-list. > > This filter returns all the documentation tickets we have in JIRA, and, > > indeed, many of them can be pushed to further releases. > > > > - > > Denis > > > > > > On Thu, Feb 13, 2020 at 3:19 AM Maxim Muzafarov > wrote: > > > > > Denis, > > > > > > > > > We still need additional work over the whole documentation, not only > > > resolving comments for the new monitoring feature [2]. > > > Here is the full list of issues related to documentation - [1]. > > > > > > Examples need to be extended too. For instance, > > > - suspend/resume for pessimistic transactions > > > - default Ignite work dir location (changed in 2.7.6 right?) > > > - baseline auto-adjustment feature > > > > > > > > > [1] > > > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8#ApacheIgnite2.8-Unresolveddocumentationtasks > > > [2] https://issues.apache.org/jira/browse/IGNITE-12408 > > > > > > On Thu, 13 Feb 2020 at 00:48, Denis Magda wrote: > > > > > > > > Hi Maxim, > > > > > > > > Do you have any understanding in regards to documentation readiness? > I do > > > > remember Nikolay was creating a page for the new metrics framework > and > > > > Artem stepped in as a reviewer. But not sure if that supposedly the > > > largest > > > > item is completed and if the other pages need to be updated. > > > > > > > > - > > > > Denis > > > > > > > > > > > > On Tue, Feb 11, 2020 at 6:19 AM Maxim Muzafarov > > > wrote: > > > > > > > > > Igniters, > > > > > > > > > > > > > > > Current the 2.8 release status > > > > > > > > > > > > > > > 1. The PR with RELEASE_NOTES fully updated [1]. > > > > > > > > > > 2. Previously mentioned performance drop has not been confirmed. > Run > > > > > many times in different environments. All test results within the > > > > > margin of error. > > > > > In-memory, putAll, 4 nodes, 1 client > > > > > IgnitePutAllBenchmark: +1% > > > > > IgnitePutAllTxBenchmark: -6% > > > > > > > > > > 3. Waiting for the vote completion > > > > > (Allow or prohibit a joint use of @deprecated and > @IgniteExperimental) > > > > > > > > > > 4. Mark MVCC with IgniteExperimental [2]. > > > > > > > > > > 5. Wait for ML examples to be fixed [3]. > > > > > > > > > > [1] https://github.com/apache/ignite/pull/7367/files > > > > > [2] > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html > > > > > [3] https://issues.apache.org/jira/browse/IGNITE-12657 > > > > > > > > > > On Tue, 11 Feb 2020 at 15:08, Ivan Bessonov > > > > wrote: > > > > > > > > > > > > Hello Igniters, > > > > > > > > > > > > I'd like to add one more fix to the release: [1] > > > > > > It adds versioning to internal classes of distributed metastorage > > > > > component. > > > > > > Without this fix it would be much harder to update these classes > > > without > > > > > > breaking binary compatibility. > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12638 > > > > > > > > > > > > ср, 5 февр. 2020 г. в 22:33, Maxim Muzafarov >: > > > > > > > > > > > > > Ivan, > > > > > > > > > > > > > > > Should not we state in release notes what new experimental > API > > > was > > > > > added? > > > > > > > > > > > > > > I think we should. Will do. > > > > > > > Just not to miss anything that we should mark with > > > > > > > @IgniteExperimental: Consistency Check [1], Monitoring [2] > anything > > > > > > > else? > > > > > > > > > > > > > > > As Flink integration was moved to external repository how > Ignite > > > 2.8 > > > > > > > users will be able to use that integration? > > > > > > > > > > > > > > Since ignite-extension has a separate release cycle (right?), > it is > > > > > > > better to release ignite-extension rather than cherry-pick this > > >
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Denis, Actually, I've already filtered documentation issues previously and left only major documentation tasks. Should I shrink the list more? On Thu, 13 Feb 2020 at 18:58, Denis Magda wrote: > > Maxim, > > Thanks for the list. How many of those tickets relate to new capabilities > or changed behavior in 2.8? You can probably come up with such a sub-list. > This filter returns all the documentation tickets we have in JIRA, and, > indeed, many of them can be pushed to further releases. > > - > Denis > > > On Thu, Feb 13, 2020 at 3:19 AM Maxim Muzafarov wrote: > > > Denis, > > > > > > We still need additional work over the whole documentation, not only > > resolving comments for the new monitoring feature [2]. > > Here is the full list of issues related to documentation - [1]. > > > > Examples need to be extended too. For instance, > > - suspend/resume for pessimistic transactions > > - default Ignite work dir location (changed in 2.7.6 right?) > > - baseline auto-adjustment feature > > > > > > [1] > > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8#ApacheIgnite2.8-Unresolveddocumentationtasks > > [2] https://issues.apache.org/jira/browse/IGNITE-12408 > > > > On Thu, 13 Feb 2020 at 00:48, Denis Magda wrote: > > > > > > Hi Maxim, > > > > > > Do you have any understanding in regards to documentation readiness? I do > > > remember Nikolay was creating a page for the new metrics framework and > > > Artem stepped in as a reviewer. But not sure if that supposedly the > > largest > > > item is completed and if the other pages need to be updated. > > > > > > - > > > Denis > > > > > > > > > On Tue, Feb 11, 2020 at 6:19 AM Maxim Muzafarov > > wrote: > > > > > > > Igniters, > > > > > > > > > > > > Current the 2.8 release status > > > > > > > > > > > > 1. The PR with RELEASE_NOTES fully updated [1]. > > > > > > > > 2. Previously mentioned performance drop has not been confirmed. Run > > > > many times in different environments. All test results within the > > > > margin of error. > > > > In-memory, putAll, 4 nodes, 1 client > > > > IgnitePutAllBenchmark: +1% > > > > IgnitePutAllTxBenchmark: -6% > > > > > > > > 3. Waiting for the vote completion > > > > (Allow or prohibit a joint use of @deprecated and @IgniteExperimental) > > > > > > > > 4. Mark MVCC with IgniteExperimental [2]. > > > > > > > > 5. Wait for ML examples to be fixed [3]. > > > > > > > > [1] https://github.com/apache/ignite/pull/7367/files > > > > [2] > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html > > > > [3] https://issues.apache.org/jira/browse/IGNITE-12657 > > > > > > > > On Tue, 11 Feb 2020 at 15:08, Ivan Bessonov > > wrote: > > > > > > > > > > Hello Igniters, > > > > > > > > > > I'd like to add one more fix to the release: [1] > > > > > It adds versioning to internal classes of distributed metastorage > > > > component. > > > > > Without this fix it would be much harder to update these classes > > without > > > > > breaking binary compatibility. > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12638 > > > > > > > > > > ср, 5 февр. 2020 г. в 22:33, Maxim Muzafarov : > > > > > > > > > > > Ivan, > > > > > > > > > > > > > Should not we state in release notes what new experimental API > > was > > > > added? > > > > > > > > > > > > I think we should. Will do. > > > > > > Just not to miss anything that we should mark with > > > > > > @IgniteExperimental: Consistency Check [1], Monitoring [2] anything > > > > > > else? > > > > > > > > > > > > > As Flink integration was moved to external repository how Ignite > > 2.8 > > > > > > users will be able to use that integration? > > > > > > > > > > > > Since ignite-extension has a separate release cycle (right?), it is > > > > > > better to release ignite-extension rather than cherry-pick this > > change > > > > > > back to 2.8. I also think it is not a blocker for the release, but > > we > > > > > > should do our best make the first ignite-extension release as > > earlier > > > > > > as possible. > > > > > > > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-10663 > > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-11848 > > > > > > > > > > > > On Wed, 5 Feb 2020 at 22:07, Ivan Pavlukhin > > > > wrote: > > > > > > > > > > > > > > Maxim, > > > > > > > > > > > > > > A couple of questions: > > > > > > > 1. We added an annotation to designate experimental API. Should > > not > > > > we > > > > > > > state in release notes what new experimental API was added? > > Perhaps > > > > in > > > > > > > a separate block. > > > > > > > 2. As Flink integration was moved to external repository how > > Ignite > > > > > > > 2.8 users will be able to use that integration? > > > > > > > > > > > > > > ср, 5 февр. 2020 г. в 21:21, Maxim Muzafarov > >: > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > > > > I've prepared RELEASE_NOTES pull-request [1] to the
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Maxim, Thanks for the list. How many of those tickets relate to new capabilities or changed behavior in 2.8? You can probably come up with such a sub-list. This filter returns all the documentation tickets we have in JIRA, and, indeed, many of them can be pushed to further releases. - Denis On Thu, Feb 13, 2020 at 3:19 AM Maxim Muzafarov wrote: > Denis, > > > We still need additional work over the whole documentation, not only > resolving comments for the new monitoring feature [2]. > Here is the full list of issues related to documentation - [1]. > > Examples need to be extended too. For instance, > - suspend/resume for pessimistic transactions > - default Ignite work dir location (changed in 2.7.6 right?) > - baseline auto-adjustment feature > > > [1] > https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8#ApacheIgnite2.8-Unresolveddocumentationtasks > [2] https://issues.apache.org/jira/browse/IGNITE-12408 > > On Thu, 13 Feb 2020 at 00:48, Denis Magda wrote: > > > > Hi Maxim, > > > > Do you have any understanding in regards to documentation readiness? I do > > remember Nikolay was creating a page for the new metrics framework and > > Artem stepped in as a reviewer. But not sure if that supposedly the > largest > > item is completed and if the other pages need to be updated. > > > > - > > Denis > > > > > > On Tue, Feb 11, 2020 at 6:19 AM Maxim Muzafarov > wrote: > > > > > Igniters, > > > > > > > > > Current the 2.8 release status > > > > > > > > > 1. The PR with RELEASE_NOTES fully updated [1]. > > > > > > 2. Previously mentioned performance drop has not been confirmed. Run > > > many times in different environments. All test results within the > > > margin of error. > > > In-memory, putAll, 4 nodes, 1 client > > > IgnitePutAllBenchmark: +1% > > > IgnitePutAllTxBenchmark: -6% > > > > > > 3. Waiting for the vote completion > > > (Allow or prohibit a joint use of @deprecated and @IgniteExperimental) > > > > > > 4. Mark MVCC with IgniteExperimental [2]. > > > > > > 5. Wait for ML examples to be fixed [3]. > > > > > > [1] https://github.com/apache/ignite/pull/7367/files > > > [2] > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html > > > [3] https://issues.apache.org/jira/browse/IGNITE-12657 > > > > > > On Tue, 11 Feb 2020 at 15:08, Ivan Bessonov > wrote: > > > > > > > > Hello Igniters, > > > > > > > > I'd like to add one more fix to the release: [1] > > > > It adds versioning to internal classes of distributed metastorage > > > component. > > > > Without this fix it would be much harder to update these classes > without > > > > breaking binary compatibility. > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12638 > > > > > > > > ср, 5 февр. 2020 г. в 22:33, Maxim Muzafarov : > > > > > > > > > Ivan, > > > > > > > > > > > Should not we state in release notes what new experimental API > was > > > added? > > > > > > > > > > I think we should. Will do. > > > > > Just not to miss anything that we should mark with > > > > > @IgniteExperimental: Consistency Check [1], Monitoring [2] anything > > > > > else? > > > > > > > > > > > As Flink integration was moved to external repository how Ignite > 2.8 > > > > > users will be able to use that integration? > > > > > > > > > > Since ignite-extension has a separate release cycle (right?), it is > > > > > better to release ignite-extension rather than cherry-pick this > change > > > > > back to 2.8. I also think it is not a blocker for the release, but > we > > > > > should do our best make the first ignite-extension release as > earlier > > > > > as possible. > > > > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-10663 > > > > > [2] https://issues.apache.org/jira/browse/IGNITE-11848 > > > > > > > > > > On Wed, 5 Feb 2020 at 22:07, Ivan Pavlukhin > > > wrote: > > > > > > > > > > > > Maxim, > > > > > > > > > > > > A couple of questions: > > > > > > 1. We added an annotation to designate experimental API. Should > not > > > we > > > > > > state in release notes what new experimental API was added? > Perhaps > > > in > > > > > > a separate block. > > > > > > 2. As Flink integration was moved to external repository how > Ignite > > > > > > 2.8 users will be able to use that integration? > > > > > > > > > > > > ср, 5 февр. 2020 г. в 21:21, Maxim Muzafarov >: > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > I've prepared RELEASE_NOTES pull-request [1] to the 2.8 > release. > > > > > > > > > > > > > > Currently, IEP-35 monitoring issues are not included in this > PR. > > > Will > > > > > > > do it soon. > > > > > > > Please, take a look. > > > > > > > > > > > > > > > > > > > > > [1] https://github.com/apache/ignite/pull/7367/files > > > > > > > > > > > > > > On Mon, 3 Feb 2020 at 14:38, Maxim Muzafarov < > mmu...@apache.org> > > > > > wrote: > > > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > > > >
Re: Provide an access to RELEASE suites on TeamCity
Thank you, Petr. Sure, will not. On Thu, 13 Feb 2020 at 18:21, Petr Ivanov wrote: > > Added you to Ignite Release Managers group — you should now be able to run > Release build. > > Please, do not modify anything there without dev-list. > > > > On 13 Feb 2020, at 17:19, Maxim Muzafarov wrote: > > > > Igniters, > > > > > > Can anyone provide access to release suites on TeamCity? > > > > [1] Main Release Build (have no access) > > [2] Apache Ignite Nightly Release (read-only access, can't run it, > > can't configure) > > > > login: mmuzaf > > > > [1] > > https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild > > [2] > > https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_NightlyRelease_RunApacheIgniteNightlyRelease_Releases_ApacheIgniteNightly=ignite-2.8=buildTypeStatusDiv >
Re: Provide an access to RELEASE suites on TeamCity
Added you to Ignite Release Managers group — you should now be able to run Release build. Please, do not modify anything there without dev-list. > On 13 Feb 2020, at 17:19, Maxim Muzafarov wrote: > > Igniters, > > > Can anyone provide access to release suites on TeamCity? > > [1] Main Release Build (have no access) > [2] Apache Ignite Nightly Release (read-only access, can't run it, > can't configure) > > login: mmuzaf > > [1] > https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild > [2] > https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_NightlyRelease_RunApacheIgniteNightlyRelease_Releases_ApacheIgniteNightly=ignite-2.8=buildTypeStatusDiv
Re: [DISCUSS] Relevance of pessimistic tx log properties in TransactionConfiguration
I think yes. чт, 13 февр. 2020 г. в 16:52, Ivan Pavlukhin : > Hi, > > It seems I found some more unused configuration properties: > * TransactionConfiguration.PessimisticTxLogSize > * TransactionConfiguration.PessimisticTxLogLinger > > Can we deprecate/remove them? > > Best regards, > Ivan Pavlukhin > -- Best regards, Alexei Scherbakov
[jira] [Created] (IGNITE-12679) Simplify javadocs in SQL classes
Ivan Pavlukhin created IGNITE-12679: --- Summary: Simplify javadocs in SQL classes Key: IGNITE-12679 URL: https://issues.apache.org/jira/browse/IGNITE-12679 Project: Ignite Issue Type: Bug Components: sql Reporter: Ivan Pavlukhin Fix For: 2.9 Update javadocs according to comments in PR https://github.com/apache/ignite/pull/7412 -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] Relevance of CacheConfiguration.DefaultLockTimeout
Ivan, Nothing from my side as well - looks like the property is not used.
[jira] [Created] (IGNITE-12678) GridLongList fails on add when created with explicit zero size
Konstantin Orlov created IGNITE-12678: - Summary: GridLongList fails on add when created with explicit zero size Key: IGNITE-12678 URL: https://issues.apache.org/jira/browse/IGNITE-12678 Project: Ignite Issue Type: Bug Components: data structures Reporter: Konstantin Orlov -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: ML examples EvaluatorExample and MultipleMetricsExample looks the same
Should I wait any ML-related issues in examples? I could wait a day or two if you are going to continue your testing deals with ML and examples to push changes together. Are you running your examples on the release staging as a ignite dependency, isn't it? чт, 13 февр. 2020 г. в 14:37, Stepan Pilschikov : > filled ticket - https://issues.apache.org/jira/browse/IGNITE-12673 > > чт, 13 февр. 2020 г. в 13:46, Alexey Zinoviev : > >> I suppose one ticket is enough, collect where all your nites and thoughts >> >> чт, 13 февр. 2020 г., 12:12 Stepan Pilschikov > >: >> >>> And one (three) more thing >>> >>> In TutorialStepByStepExample we running 17 examples >>> First 12 logging is pretty good and looks like "Tutorial step N: name" >>> -> model -> accuracy -> "Tutorial step N: completed" >>> But then starting with 13 this pattern is kind of broke, step start and >>> step completion is missing >>> >>> Also want to admin that >>> Step_8_CV_with_Param_Grid_and_metrics_and_pipeline is haven't step >>> completion log >>> And complete log for Step_9_Scaling_With_Stacking looks like 'Tutorial >>> step 5 (scaling) example completed' >>> >>> i don't know, should i made some ticket? One ticket for each problem or >>> one for all or you can fix it in one commit? because fixes quite small >>> >>> ср, 12 февр. 2020 г. в 16:31, Alexey Zinoviev : >>> Yes, you are right. Missed resources, I'll remove them tomorrow ср, 12 февр. 2020 г. в 16:23, Stepan Pilschikov < pilshchikov@gmail.com>: > With mnist_tf_model in same directory > All others looks right > > ср, 12 февр. 2020 г. в 16:21, Stepan Pilschikov < > pilshchikov@gmail.com>: > >> Hmm, interesting, got it >> Also found >> apache-ignite-2.8.0-bin/examples/src/main/resources/models/mleap >> I think its also should be removed then >> >> ср, 12 февр. 2020 г. в 16:11, Alexey Zinoviev > >: >> >>> Please, have a look to the next ticket >>> https://issues.apache.org/jira/browse/IGNITE-12659 >>> Also tensorflow packages are blockers for IGFS deprecation and >>> removal. >>> >>> ср, 12 февр. 2020 г. в 15:58, Alexey Zinoviev < >>> zaleslaw@gmail.com>: >>> They are not lost, they are removed from release branch due to a lot of bugs, cves and so on, don't worry about that. ср, 12 февр. 2020 г., 14:31 Stepan Pilschikov < pilshchikov@gmail.com>: > Also lost ignite-ml-tensorflow-model-parser and ignite-tensorflow > in libs/optional > > > ср, 12 февр. 2020 г. в 14:25, Stepan Pilschikov < > pilshchikov@gmail.com>: > >> But now have new problems >> I build ignite-2.8 build on TC >> https://ci.ignite.apache.org/buildConfiguration/Releases_ApacheIgniteMain_ReleaseBuild/5043672 >> Previous >> https://ci.ignite.apache.org/buildConfiguration/Releases_ApacheIgniteMain_ReleaseBuild/4988324 >> >> New build much lighter >> Previous: 5.7 Gb >> Current: 4.3 Gb >> And its looks like something missing, on a first look i notice >> that /libs/optional/ignite-ml-mleap-model-parser lost completely >> >> >> ср, 12 февр. 2020 г. в 14:18, Stepan Pilschikov < >> pilshchikov@gmail.com>: >> >>> Thanks for fix >>> Duplicated examples deleted and tutorial works on real (1+ node) >>> cluster now >>> I think ticket can be closed, have no power here >>> >>> вт, 11 февр. 2020 г. в 20:44, Alexey Zinoviev < >>> zaleslaw@gmail.com>: >>> Sorry, for that, but I it touches ML-related stuff only and doesn't influence on any another module. I merged them to master later (currently, I have a 4 tickets in a queue). Will keep in mind for the next fixes вт, 11 февр. 2020 г. в 18:58, Maxim Muzafarov < mmu...@apache.org>: > Alexey, > > Is the approach when we cherry-pick fixes to the 2.8 only > after all > fixes have verified in the master branch better? I just want > to be > sure the release branch to be stable enough. > > On Tue, 11 Feb 2020 at 18:36, Alexey Zinoviev < > zaleslaw@gmail.com> wrote: > > > > Fixed both in release branch 2.8. Please verify and let me > know > > > > вт, 11 февр. 2020 г. в 14:58, Stepan Pilschikov < > pilshchikov@gmail.com>: > > > > > And one more > > > https://issues.apache.org/jira/browse/IGNITE-12658 - > > > TutorialStepByStepExample example failed if cluster with > more then 1 node
Provide an access to RELEASE suites on TeamCity
Igniters, Can anyone provide access to release suites on TeamCity? [1] Main Release Build (have no access) [2] Apache Ignite Nightly Release (read-only access, can't run it, can't configure) login: mmuzaf [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_ApacheIgniteMain_ReleaseBuild [2] https://ci.ignite.apache.org/viewType.html?buildTypeId=Releases_NightlyRelease_RunApacheIgniteNightlyRelease_Releases_ApacheIgniteNightly=ignite-2.8=buildTypeStatusDiv
[jira] [Created] (IGNITE-12677) Extend test coverage for GridLongList serialization
Konstantin Orlov created IGNITE-12677: - Summary: Extend test coverage for GridLongList serialization Key: IGNITE-12677 URL: https://issues.apache.org/jira/browse/IGNITE-12677 Project: Ignite Issue Type: Task Components: binary Reporter: Konstantin Orlov Assignee: Konstantin Orlov -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12676) .NET: Add partition-based AffinityCall and AffinityRun overloads
Pavel Tupitsyn created IGNITE-12676: --- Summary: .NET: Add partition-based AffinityCall and AffinityRun overloads Key: IGNITE-12676 URL: https://issues.apache.org/jira/browse/IGNITE-12676 Project: Ignite Issue Type: Improvement Components: platforms Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Add partition-based AffinityCall and AffinityRun overloads to ICompute. See corresponding methods in Java (IgniteCompute). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[DISCUSS] Relevance of pessimistic tx log properties in TransactionConfiguration
Hi, It seems I found some more unused configuration properties: * TransactionConfiguration.PessimisticTxLogSize * TransactionConfiguration.PessimisticTxLogLinger Can we deprecate/remove them? Best regards, Ivan Pavlukhin
Re: [DISCUSS] Relevance of CacheConfiguration.DefaultLockTimeout
Anybody? Best regards, Ivan Pavlukhin вт, 28 янв. 2020 г. в 18:29, Ivan Pavlukhin : > > Hi, > > I failed to find where CacheConfiguration.DefaultLockTimeout property > is used. Does anyone know how is it used? If it is not used I suppose > we can mark this property deprecated. > > Please share your thoughts! > > -- > Best regards, > Ivan Pavlukhin
Re: [DISCUSS] Public API deprecation rules
I've created a wiki page based on the vote results [1]. Feel free to update the page if it seems incomplete. [1] https://cwiki.apache.org/confluence/display/IGNITE/Development+Guidelines чт, 6 февр. 2020 г. в 20:17, Alexey Goncharuk : > Ivan, thanks, I agree, added this clause: > > The vote we are going to have is reduced to two choices so far: > * Never deprecate the old APIs unless the new APIs are stable and > released without @IgniteExperimental. The old APIs javadoc may be updated > with a reference to new APIs to encourage users to evaluate new APIs. The > deprecation and new API release may happen simultaneously if the new API is > not marked with @IgniteExperimental or the annotation is removed in the > same release. > * Allow to deprecate the old APIs even when new APIs are marked with > @IgniteExperimental to explicitly notify users that old APIs will be > removed in the next major release AND new APIs are available > Neither of cases prohibits deprecation of an API without a replacement if > community decided so. > > Are we good to go? >
[jira] [Created] (IGNITE-12675) Binary types registered in two stages when entry processor is used
Konstantin Orlov created IGNITE-12675: - Summary: Binary types registered in two stages when entry processor is used Key: IGNITE-12675 URL: https://issues.apache.org/jira/browse/IGNITE-12675 Project: Ignite Issue Type: Bug Components: binary Reporter: Konstantin Orlov When you try to add new value with entry processor, binary type registration proceeded in two stages: - class registration - binary metadata registration Perhaps it could be achieved by only one stage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12674) Extend test coverage for binary types registration
Konstantin Orlov created IGNITE-12674: - Summary: Extend test coverage for binary types registration Key: IGNITE-12674 URL: https://issues.apache.org/jira/browse/IGNITE-12674 Project: Ignite Issue Type: Task Components: binary Reporter: Konstantin Orlov Assignee: Konstantin Orlov -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSSION] Deprecation of obsolete rebalancing functionality
But in combination with BLT it will work as intended - no rebalancing under the cover. чт, 13 февр. 2020 г. в 14:39, Alexei Scherbakov < alexey.scherbak...@gmail.com>: > Of course, stable topology will be just a hint. > > Any node can leave at any moment. > > чт, 13 февр. 2020 г. в 14:35, Alexei Scherbakov < > alexey.scherbak...@gmail.com>: > >> 1. Yes >> >> 2. This is right but doesn't sound like a bug. The rebalancing will be >> finished before releasing syncFut and partitions will contain all necessary >> data (but are still in moving state). >> >> 3. No, local node doesn't wait the rebalancing on all grid nodes. >> >> Actually, I think SYNC mode should be dropped as well. Instead we must >> provide the convenient public API to wait for "stable" topology. >> >> >> чт, 13 февр. 2020 г. в 14:09, Maxim Muzafarov : >> >>> Pavel, >>> >>> It's still a big question regarding SYNC rebalance mode. Here is my >>> thoughts. >>> >>> 1. Yes, we must rebalance such caches prior to ASYNC one (if the >>> rebalanceOrder configuration will be removed). >>> >>> 2. When persistence is enabled and when WAL is disabled (on the first >>> rebalance start), I think we should finish syncFuture only on >>> checkpoint like we are enabling the WAL state for cache group and >>> simultaneously owning all MOVING partitions. But currently, I've seen >>> that syncFuture finishes when there are no remaining partitions left >>> [1]. >>> Is it correct? Seems like a bug. >>> >>> 3. In my understanding, a new local node can start only when ALL SYNC >>> cache groups have been fully rebalanced on ALL nodes, right? But how >>> about late affinity assignment here? It seems that SYNC caches will be >>> rebalanced locally on the node, the node will start, but other nodes >>> still think this node is not operational (late affinity assignment not >>> occurred yet). >>> >>> >>> [1] >>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionDemander.java#L1561 >>> >>> On Thu, 13 Feb 2020 at 12:57, Pavel Pereslegin wrote: >>> > >>> > > +1 to deprecate rebalanceOrder and remove related functionality, >>> > Meant to "rework related functionality" not "remove". >>> > >>> > чт, 13 февр. 2020 г. в 12:47, Pavel Pereslegin : >>> > > >>> > > Hello, >>> > > >>> > > +1 to deprecate rebalanceOrder and remove related functionality, >>> > > should we create a separate ticket for this? >>> > > >>> > > Btw, as I understand, SYNC mode is only useful for in-memory caches, >>> > > because when persistence is enabled (and WAL is disabled during >>> > > rebalancing), even "ignite-sys-cache" owns partitions only after all >>> > > cache groups are rebalanced. Thus, even utility cache is still >>> > > inoperable after node startup when persistence is enabled. Do we >>> > > really need to wait for SYNC caches when a node starts with enabled >>> > > persistence or should we enabled WAL for SYNC-caches? >>> > > >>> > > чт, 13 февр. 2020 г. в 11:13, Ivan Rakov : >>> > > > >>> > > > Hello, >>> > > > >>> > > > +1 from me for rebalance delay deprecation. >>> > > > I can imagine only one actual case for this option: prevent >>> excessive load >>> > > > on the cluster in case of temporary short-term topology changes >>> (e.g. node >>> > > > is stopped for a while and then returned back). >>> > > > Now it's handled by baseline auto adjustment in a much more >>> correct way: >>> > > > partitions are not reassigned within a maintenance interval >>> (unlike with >>> > > > the rebalance delay). >>> > > > I also don't think that ability to configure rebalance delay per >>> cache is >>> > > > crucial. >>> > > > >>> > > > > rebalanceOrder is also useless, agreed. >>> > > > +1 >>> > > > Except for one case: we may want to rebalance caches with >>> > > > CacheRebalanceMode.SYNC first. But anyway, this behavior doesn't >>> require a >>> > > > separate property to be enabled. >>> > > > >>> > > > On Wed, Feb 12, 2020 at 4:54 PM Alexei Scherbakov < >>> > > > alexey.scherbak...@gmail.com> wrote: >>> > > > >>> > > > > Maxim, >>> > > > > >>> > > > > rebalanceDelay was introduced before the BLT appear in the >>> product to solve >>> > > > > scenarios which are now solved by BLT. >>> > > > > >>> > > > > It's pointless for me having it in the product since BLT was >>> introduced. >>> > > > > >>> > > > > I do not think delaying rebalancing per cache group has any >>> meaning. I >>> > > > > cannot image any reason for it. >>> > > > > >>> > > > > rebalanceOrder is also useless, agreed. >>> > > > > >>> > > > > >>> > > > > >>> > > > > >>> > > > > ср, 12 февр. 2020 г. в 16:19, Maxim Muzafarov >> >: >>> > > > > >>> > > > > > Alexey, >>> > > > > > >>> > > > > > Why do you think delaying of historical rebalance (on BLT node >>> join) >>> > > > > > for particular cache groups is not the real world use case? >>> Probably >>> > > > > > the same topic may be started on user-list to collect more use >>> cases
Re: [DISCUSSION] Deprecation of obsolete rebalancing functionality
Of course, stable topology will be just a hint. Any node can leave at any moment. чт, 13 февр. 2020 г. в 14:35, Alexei Scherbakov < alexey.scherbak...@gmail.com>: > 1. Yes > > 2. This is right but doesn't sound like a bug. The rebalancing will be > finished before releasing syncFut and partitions will contain all necessary > data (but are still in moving state). > > 3. No, local node doesn't wait the rebalancing on all grid nodes. > > Actually, I think SYNC mode should be dropped as well. Instead we must > provide the convenient public API to wait for "stable" topology. > > > чт, 13 февр. 2020 г. в 14:09, Maxim Muzafarov : > >> Pavel, >> >> It's still a big question regarding SYNC rebalance mode. Here is my >> thoughts. >> >> 1. Yes, we must rebalance such caches prior to ASYNC one (if the >> rebalanceOrder configuration will be removed). >> >> 2. When persistence is enabled and when WAL is disabled (on the first >> rebalance start), I think we should finish syncFuture only on >> checkpoint like we are enabling the WAL state for cache group and >> simultaneously owning all MOVING partitions. But currently, I've seen >> that syncFuture finishes when there are no remaining partitions left >> [1]. >> Is it correct? Seems like a bug. >> >> 3. In my understanding, a new local node can start only when ALL SYNC >> cache groups have been fully rebalanced on ALL nodes, right? But how >> about late affinity assignment here? It seems that SYNC caches will be >> rebalanced locally on the node, the node will start, but other nodes >> still think this node is not operational (late affinity assignment not >> occurred yet). >> >> >> [1] >> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionDemander.java#L1561 >> >> On Thu, 13 Feb 2020 at 12:57, Pavel Pereslegin wrote: >> > >> > > +1 to deprecate rebalanceOrder and remove related functionality, >> > Meant to "rework related functionality" not "remove". >> > >> > чт, 13 февр. 2020 г. в 12:47, Pavel Pereslegin : >> > > >> > > Hello, >> > > >> > > +1 to deprecate rebalanceOrder and remove related functionality, >> > > should we create a separate ticket for this? >> > > >> > > Btw, as I understand, SYNC mode is only useful for in-memory caches, >> > > because when persistence is enabled (and WAL is disabled during >> > > rebalancing), even "ignite-sys-cache" owns partitions only after all >> > > cache groups are rebalanced. Thus, even utility cache is still >> > > inoperable after node startup when persistence is enabled. Do we >> > > really need to wait for SYNC caches when a node starts with enabled >> > > persistence or should we enabled WAL for SYNC-caches? >> > > >> > > чт, 13 февр. 2020 г. в 11:13, Ivan Rakov : >> > > > >> > > > Hello, >> > > > >> > > > +1 from me for rebalance delay deprecation. >> > > > I can imagine only one actual case for this option: prevent >> excessive load >> > > > on the cluster in case of temporary short-term topology changes >> (e.g. node >> > > > is stopped for a while and then returned back). >> > > > Now it's handled by baseline auto adjustment in a much more correct >> way: >> > > > partitions are not reassigned within a maintenance interval (unlike >> with >> > > > the rebalance delay). >> > > > I also don't think that ability to configure rebalance delay per >> cache is >> > > > crucial. >> > > > >> > > > > rebalanceOrder is also useless, agreed. >> > > > +1 >> > > > Except for one case: we may want to rebalance caches with >> > > > CacheRebalanceMode.SYNC first. But anyway, this behavior doesn't >> require a >> > > > separate property to be enabled. >> > > > >> > > > On Wed, Feb 12, 2020 at 4:54 PM Alexei Scherbakov < >> > > > alexey.scherbak...@gmail.com> wrote: >> > > > >> > > > > Maxim, >> > > > > >> > > > > rebalanceDelay was introduced before the BLT appear in the >> product to solve >> > > > > scenarios which are now solved by BLT. >> > > > > >> > > > > It's pointless for me having it in the product since BLT was >> introduced. >> > > > > >> > > > > I do not think delaying rebalancing per cache group has any >> meaning. I >> > > > > cannot image any reason for it. >> > > > > >> > > > > rebalanceOrder is also useless, agreed. >> > > > > >> > > > > >> > > > > >> > > > > >> > > > > ср, 12 февр. 2020 г. в 16:19, Maxim Muzafarov > >: >> > > > > >> > > > > > Alexey, >> > > > > > >> > > > > > Why do you think delaying of historical rebalance (on BLT node >> join) >> > > > > > for particular cache groups is not the real world use case? >> Probably >> > > > > > the same topic may be started on user-list to collect more use >> cases >> > > > > > from real users. >> > > > > > >> > > > > > In general, I support reducing the number of available rebalance >> > > > > > configuration parameters, but we should do it really carefully. >> > > > > > I can also propose - rebalanceOrder param for removing. >> > > > > > >> > > > > > On Wed,
Re: ML examples EvaluatorExample and MultipleMetricsExample looks the same
filled ticket - https://issues.apache.org/jira/browse/IGNITE-12673 чт, 13 февр. 2020 г. в 13:46, Alexey Zinoviev : > I suppose one ticket is enough, collect where all your nites and thoughts > > чт, 13 февр. 2020 г., 12:12 Stepan Pilschikov : > >> And one (three) more thing >> >> In TutorialStepByStepExample we running 17 examples >> First 12 logging is pretty good and looks like "Tutorial step N: name" -> >> model -> accuracy -> "Tutorial step N: completed" >> But then starting with 13 this pattern is kind of broke, step start and >> step completion is missing >> >> Also want to admin that >> Step_8_CV_with_Param_Grid_and_metrics_and_pipeline is haven't step >> completion log >> And complete log for Step_9_Scaling_With_Stacking looks like 'Tutorial >> step 5 (scaling) example completed' >> >> i don't know, should i made some ticket? One ticket for each problem or >> one for all or you can fix it in one commit? because fixes quite small >> >> ср, 12 февр. 2020 г. в 16:31, Alexey Zinoviev : >> >>> Yes, you are right. Missed resources, I'll remove them tomorrow >>> >>> ср, 12 февр. 2020 г. в 16:23, Stepan Pilschikov < >>> pilshchikov@gmail.com>: >>> With mnist_tf_model in same directory All others looks right ср, 12 февр. 2020 г. в 16:21, Stepan Pilschikov < pilshchikov@gmail.com>: > Hmm, interesting, got it > Also found > apache-ignite-2.8.0-bin/examples/src/main/resources/models/mleap > I think its also should be removed then > > ср, 12 февр. 2020 г. в 16:11, Alexey Zinoviev >: > >> Please, have a look to the next ticket >> https://issues.apache.org/jira/browse/IGNITE-12659 >> Also tensorflow packages are blockers for IGFS deprecation and >> removal. >> >> ср, 12 февр. 2020 г. в 15:58, Alexey Zinoviev > >: >> >>> They are not lost, they are removed from release branch due to a lot >>> of bugs, cves and so on, don't worry about that. >>> >>> ср, 12 февр. 2020 г., 14:31 Stepan Pilschikov < >>> pilshchikov@gmail.com>: >>> Also lost ignite-ml-tensorflow-model-parser and ignite-tensorflow in libs/optional ср, 12 февр. 2020 г. в 14:25, Stepan Pilschikov < pilshchikov@gmail.com>: > But now have new problems > I build ignite-2.8 build on TC > https://ci.ignite.apache.org/buildConfiguration/Releases_ApacheIgniteMain_ReleaseBuild/5043672 > Previous > https://ci.ignite.apache.org/buildConfiguration/Releases_ApacheIgniteMain_ReleaseBuild/4988324 > > New build much lighter > Previous: 5.7 Gb > Current: 4.3 Gb > And its looks like something missing, on a first look i notice > that /libs/optional/ignite-ml-mleap-model-parser lost completely > > > ср, 12 февр. 2020 г. в 14:18, Stepan Pilschikov < > pilshchikov@gmail.com>: > >> Thanks for fix >> Duplicated examples deleted and tutorial works on real (1+ node) >> cluster now >> I think ticket can be closed, have no power here >> >> вт, 11 февр. 2020 г. в 20:44, Alexey Zinoviev < >> zaleslaw@gmail.com>: >> >>> Sorry, for that, but I it touches ML-related stuff only and >>> doesn't influence on any another module. >>> I merged them to master later (currently, I have a 4 tickets in >>> a queue). >>> >>> Will keep in mind for the next fixes >>> >>> вт, 11 февр. 2020 г. в 18:58, Maxim Muzafarov >> >: >>> Alexey, Is the approach when we cherry-pick fixes to the 2.8 only after all fixes have verified in the master branch better? I just want to be sure the release branch to be stable enough. On Tue, 11 Feb 2020 at 18:36, Alexey Zinoviev < zaleslaw@gmail.com> wrote: > > Fixed both in release branch 2.8. Please verify and let me know > > вт, 11 февр. 2020 г. в 14:58, Stepan Pilschikov < pilshchikov@gmail.com>: > > > And one more > > https://issues.apache.org/jira/browse/IGNITE-12658 - > > TutorialStepByStepExample example failed if cluster with more then 1 node > > > > вт, 11 февр. 2020 г. в 11:52, Alexey Zinoviev < zaleslaw@gmail.com>: > > > >> Great, thank you so much, will fix next week > >> > >> вт, 11 февр. 2020 г., 11:39 Stepan Pilschikov < pilshchikov@gmail.com > >> >: > >> > >>> Hi, Alexey, could you please looking on one small accident happen in ML > >>>
[jira] [Created] (IGNITE-12673) ML examples logging
Stepan Pilschikov created IGNITE-12673: -- Summary: ML examples logging Key: IGNITE-12673 URL: https://issues.apache.org/jira/browse/IGNITE-12673 Project: Ignite Issue Type: Bug Components: examples, ml Affects Versions: 2.8 Reporter: Stepan Pilschikov Compile of several minor fixes for ML examples: 1. In TutorialStepByStepExample we running 17 examples First 12 logging is pretty good and looks like "Tutorial step N: name" -> model -> accuracy -> "Tutorial step N: completed" But then starting with 13 this pattern is kind of broke, step start and step completion is missing 2. Step_8_CV_with_Param_Grid_and_metrics_and_pipeline is haven't step completion log 3. Complete log for Step_9_Scaling_With_Stacking looks like 'Tutorial step 5 (scaling) example completed' -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSSION] Deprecation of obsolete rebalancing functionality
1. Yes 2. This is right but doesn't sound like a bug. The rebalancing will be finished before releasing syncFut and partitions will contain all necessary data (but are still in moving state). 3. No, local node doesn't wait the rebalancing on all grid nodes. Actually, I think SYNC mode should be dropped as well. Instead we must provide the convenient public API to wait for "stable" topology. чт, 13 февр. 2020 г. в 14:09, Maxim Muzafarov : > Pavel, > > It's still a big question regarding SYNC rebalance mode. Here is my > thoughts. > > 1. Yes, we must rebalance such caches prior to ASYNC one (if the > rebalanceOrder configuration will be removed). > > 2. When persistence is enabled and when WAL is disabled (on the first > rebalance start), I think we should finish syncFuture only on > checkpoint like we are enabling the WAL state for cache group and > simultaneously owning all MOVING partitions. But currently, I've seen > that syncFuture finishes when there are no remaining partitions left > [1]. > Is it correct? Seems like a bug. > > 3. In my understanding, a new local node can start only when ALL SYNC > cache groups have been fully rebalanced on ALL nodes, right? But how > about late affinity assignment here? It seems that SYNC caches will be > rebalanced locally on the node, the node will start, but other nodes > still think this node is not operational (late affinity assignment not > occurred yet). > > > [1] > https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionDemander.java#L1561 > > On Thu, 13 Feb 2020 at 12:57, Pavel Pereslegin wrote: > > > > > +1 to deprecate rebalanceOrder and remove related functionality, > > Meant to "rework related functionality" not "remove". > > > > чт, 13 февр. 2020 г. в 12:47, Pavel Pereslegin : > > > > > > Hello, > > > > > > +1 to deprecate rebalanceOrder and remove related functionality, > > > should we create a separate ticket for this? > > > > > > Btw, as I understand, SYNC mode is only useful for in-memory caches, > > > because when persistence is enabled (and WAL is disabled during > > > rebalancing), even "ignite-sys-cache" owns partitions only after all > > > cache groups are rebalanced. Thus, even utility cache is still > > > inoperable after node startup when persistence is enabled. Do we > > > really need to wait for SYNC caches when a node starts with enabled > > > persistence or should we enabled WAL for SYNC-caches? > > > > > > чт, 13 февр. 2020 г. в 11:13, Ivan Rakov : > > > > > > > > Hello, > > > > > > > > +1 from me for rebalance delay deprecation. > > > > I can imagine only one actual case for this option: prevent > excessive load > > > > on the cluster in case of temporary short-term topology changes > (e.g. node > > > > is stopped for a while and then returned back). > > > > Now it's handled by baseline auto adjustment in a much more correct > way: > > > > partitions are not reassigned within a maintenance interval (unlike > with > > > > the rebalance delay). > > > > I also don't think that ability to configure rebalance delay per > cache is > > > > crucial. > > > > > > > > > rebalanceOrder is also useless, agreed. > > > > +1 > > > > Except for one case: we may want to rebalance caches with > > > > CacheRebalanceMode.SYNC first. But anyway, this behavior doesn't > require a > > > > separate property to be enabled. > > > > > > > > On Wed, Feb 12, 2020 at 4:54 PM Alexei Scherbakov < > > > > alexey.scherbak...@gmail.com> wrote: > > > > > > > > > Maxim, > > > > > > > > > > rebalanceDelay was introduced before the BLT appear in the product > to solve > > > > > scenarios which are now solved by BLT. > > > > > > > > > > It's pointless for me having it in the product since BLT was > introduced. > > > > > > > > > > I do not think delaying rebalancing per cache group has any > meaning. I > > > > > cannot image any reason for it. > > > > > > > > > > rebalanceOrder is also useless, agreed. > > > > > > > > > > > > > > > > > > > > > > > > > ср, 12 февр. 2020 г. в 16:19, Maxim Muzafarov : > > > > > > > > > > > Alexey, > > > > > > > > > > > > Why do you think delaying of historical rebalance (on BLT node > join) > > > > > > for particular cache groups is not the real world use case? > Probably > > > > > > the same topic may be started on user-list to collect more use > cases > > > > > > from real users. > > > > > > > > > > > > In general, I support reducing the number of available rebalance > > > > > > configuration parameters, but we should do it really carefully. > > > > > > I can also propose - rebalanceOrder param for removing. > > > > > > > > > > > > On Wed, 12 Feb 2020 at 15:50, Alexei Scherbakov > > > > > > wrote: > > > > > > > > > > > > > > Maxim, > > > > > > > > > > > > > > In general rebalanceDelay is used to delay/disable rebalance > then > > > > > > topology > > > > > > > is changed. > > > > > > > Right now we have BLT to avoid unnecesary
[jira] [Created] (IGNITE-12672) Query annotations are not working if statement keywords are in lower case
Evgeniy Rudenko created IGNITE-12672: Summary: Query annotations are not working if statement keywords are in lower case Key: IGNITE-12672 URL: https://issues.apache.org/jira/browse/IGNITE-12672 Project: Ignite Issue Type: Bug Components: sql Reporter: Evgeniy Rudenko Assignee: Evgeniy Rudenko Fix For: 2.9 SQL queries defined using org.apache.ignite.springdata.repository.config.Query annotations are not working if key words (such as UPDATE or DELETE) are written using lower case. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: Data vanished from cluster after INACTIVE/ACTIVE switch
Hello, everyone. I’d like to propose to make behavior of cluster deactivation the same independently on usage type: in code or manually with control.sh or JMX. Let’s put it in one place like IgniteCluster#state(ClusterState newState, boolean force) and require forced call for in-memory data. While I was doing the ticket I realized that the suggested previously solution cannot be complete. To prevent data loss via JMX I would need to stop executing IgniteMXBean#active(false). But this causes stopping of Ignite#active(false) too. The problem relies in single implementation for both interfaces in IgniteKernal#active(boolean). It has to act differently depending on where it is called from: JMX/CLI or code. Any thoughts? чт, 30 янв. 2020 г. в 13:08, Alexey Goncharuk : > I agree on CLI and JMX because those interfaces can be used by a system > administrator and can be invoked by mistake. > > As for the Java API, personally, I find it strange to add 'force' or > 'confirm' flags to it because it is very unlikely that such an invocation > is done by mistake. Such mistakes are caught during the testing phase and > developers will end up hard-coding 'true' as a flag anyways. >
Re: Apache Ignite 2.8 RELEASE [Time, Scope, Manager]
Denis, We still need additional work over the whole documentation, not only resolving comments for the new monitoring feature [2]. Here is the full list of issues related to documentation - [1]. Examples need to be extended too. For instance, - suspend/resume for pessimistic transactions - default Ignite work dir location (changed in 2.7.6 right?) - baseline auto-adjustment feature [1] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.8#ApacheIgnite2.8-Unresolveddocumentationtasks [2] https://issues.apache.org/jira/browse/IGNITE-12408 On Thu, 13 Feb 2020 at 00:48, Denis Magda wrote: > > Hi Maxim, > > Do you have any understanding in regards to documentation readiness? I do > remember Nikolay was creating a page for the new metrics framework and > Artem stepped in as a reviewer. But not sure if that supposedly the largest > item is completed and if the other pages need to be updated. > > - > Denis > > > On Tue, Feb 11, 2020 at 6:19 AM Maxim Muzafarov wrote: > > > Igniters, > > > > > > Current the 2.8 release status > > > > > > 1. The PR with RELEASE_NOTES fully updated [1]. > > > > 2. Previously mentioned performance drop has not been confirmed. Run > > many times in different environments. All test results within the > > margin of error. > > In-memory, putAll, 4 nodes, 1 client > > IgnitePutAllBenchmark: +1% > > IgnitePutAllTxBenchmark: -6% > > > > 3. Waiting for the vote completion > > (Allow or prohibit a joint use of @deprecated and @IgniteExperimental) > > > > 4. Mark MVCC with IgniteExperimental [2]. > > > > 5. Wait for ML examples to be fixed [3]. > > > > [1] https://github.com/apache/ignite/pull/7367/files > > [2] > > http://apache-ignite-developers.2346864.n4.nabble.com/Mark-MVCC-with-IgniteExperimental-td45669.html > > [3] https://issues.apache.org/jira/browse/IGNITE-12657 > > > > On Tue, 11 Feb 2020 at 15:08, Ivan Bessonov wrote: > > > > > > Hello Igniters, > > > > > > I'd like to add one more fix to the release: [1] > > > It adds versioning to internal classes of distributed metastorage > > component. > > > Without this fix it would be much harder to update these classes without > > > breaking binary compatibility. > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-12638 > > > > > > ср, 5 февр. 2020 г. в 22:33, Maxim Muzafarov : > > > > > > > Ivan, > > > > > > > > > Should not we state in release notes what new experimental API was > > added? > > > > > > > > I think we should. Will do. > > > > Just not to miss anything that we should mark with > > > > @IgniteExperimental: Consistency Check [1], Monitoring [2] anything > > > > else? > > > > > > > > > As Flink integration was moved to external repository how Ignite 2.8 > > > > users will be able to use that integration? > > > > > > > > Since ignite-extension has a separate release cycle (right?), it is > > > > better to release ignite-extension rather than cherry-pick this change > > > > back to 2.8. I also think it is not a blocker for the release, but we > > > > should do our best make the first ignite-extension release as earlier > > > > as possible. > > > > > > > > > > > > [1] https://issues.apache.org/jira/browse/IGNITE-10663 > > > > [2] https://issues.apache.org/jira/browse/IGNITE-11848 > > > > > > > > On Wed, 5 Feb 2020 at 22:07, Ivan Pavlukhin > > wrote: > > > > > > > > > > Maxim, > > > > > > > > > > A couple of questions: > > > > > 1. We added an annotation to designate experimental API. Should not > > we > > > > > state in release notes what new experimental API was added? Perhaps > > in > > > > > a separate block. > > > > > 2. As Flink integration was moved to external repository how Ignite > > > > > 2.8 users will be able to use that integration? > > > > > > > > > > ср, 5 февр. 2020 г. в 21:21, Maxim Muzafarov : > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > I've prepared RELEASE_NOTES pull-request [1] to the 2.8 release. > > > > > > > > > > > > Currently, IEP-35 monitoring issues are not included in this PR. > > Will > > > > > > do it soon. > > > > > > Please, take a look. > > > > > > > > > > > > > > > > > > [1] https://github.com/apache/ignite/pull/7367/files > > > > > > > > > > > > On Mon, 3 Feb 2020 at 14:38, Maxim Muzafarov > > > > wrote: > > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > > > > > > > > Let me share the current status of the release. > > > > > > > > > > > > > > 1. > > > > > > > Waiting for the issues [1] [2] (discussed previously this > > thread) to > > > > > > > be tested by TC.Bot and merged to the 2.8 release branch. > > > > > > > > > > > > > > 2. > > > > > > > Only 2 release BLOCKER issues left. I'm planning to move these > > issues > > > > > > > to 2.8.1 release. > > > > > > > The issue [4] (Error during purges by expiration: Unknown page > > type) > > > > > > > will be covered by [1] [2]. > > > > > > > The issue [3] (Apache Ignite Cluster(Amazon S3 Based Discovery) > > Nodes > > > > > > > getting down) probably require additional
Re: [DISCUSSION] Deprecation of obsolete rebalancing functionality
Pavel, It's still a big question regarding SYNC rebalance mode. Here is my thoughts. 1. Yes, we must rebalance such caches prior to ASYNC one (if the rebalanceOrder configuration will be removed). 2. When persistence is enabled and when WAL is disabled (on the first rebalance start), I think we should finish syncFuture only on checkpoint like we are enabling the WAL state for cache group and simultaneously owning all MOVING partitions. But currently, I've seen that syncFuture finishes when there are no remaining partitions left [1]. Is it correct? Seems like a bug. 3. In my understanding, a new local node can start only when ALL SYNC cache groups have been fully rebalanced on ALL nodes, right? But how about late affinity assignment here? It seems that SYNC caches will be rebalanced locally on the node, the node will start, but other nodes still think this node is not operational (late affinity assignment not occurred yet). [1] https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/preloader/GridDhtPartitionDemander.java#L1561 On Thu, 13 Feb 2020 at 12:57, Pavel Pereslegin wrote: > > > +1 to deprecate rebalanceOrder and remove related functionality, > Meant to "rework related functionality" not "remove". > > чт, 13 февр. 2020 г. в 12:47, Pavel Pereslegin : > > > > Hello, > > > > +1 to deprecate rebalanceOrder and remove related functionality, > > should we create a separate ticket for this? > > > > Btw, as I understand, SYNC mode is only useful for in-memory caches, > > because when persistence is enabled (and WAL is disabled during > > rebalancing), even "ignite-sys-cache" owns partitions only after all > > cache groups are rebalanced. Thus, even utility cache is still > > inoperable after node startup when persistence is enabled. Do we > > really need to wait for SYNC caches when a node starts with enabled > > persistence or should we enabled WAL for SYNC-caches? > > > > чт, 13 февр. 2020 г. в 11:13, Ivan Rakov : > > > > > > Hello, > > > > > > +1 from me for rebalance delay deprecation. > > > I can imagine only one actual case for this option: prevent excessive load > > > on the cluster in case of temporary short-term topology changes (e.g. node > > > is stopped for a while and then returned back). > > > Now it's handled by baseline auto adjustment in a much more correct way: > > > partitions are not reassigned within a maintenance interval (unlike with > > > the rebalance delay). > > > I also don't think that ability to configure rebalance delay per cache is > > > crucial. > > > > > > > rebalanceOrder is also useless, agreed. > > > +1 > > > Except for one case: we may want to rebalance caches with > > > CacheRebalanceMode.SYNC first. But anyway, this behavior doesn't require a > > > separate property to be enabled. > > > > > > On Wed, Feb 12, 2020 at 4:54 PM Alexei Scherbakov < > > > alexey.scherbak...@gmail.com> wrote: > > > > > > > Maxim, > > > > > > > > rebalanceDelay was introduced before the BLT appear in the product to > > > > solve > > > > scenarios which are now solved by BLT. > > > > > > > > It's pointless for me having it in the product since BLT was introduced. > > > > > > > > I do not think delaying rebalancing per cache group has any meaning. I > > > > cannot image any reason for it. > > > > > > > > rebalanceOrder is also useless, agreed. > > > > > > > > > > > > > > > > > > > > ср, 12 февр. 2020 г. в 16:19, Maxim Muzafarov : > > > > > > > > > Alexey, > > > > > > > > > > Why do you think delaying of historical rebalance (on BLT node join) > > > > > for particular cache groups is not the real world use case? Probably > > > > > the same topic may be started on user-list to collect more use cases > > > > > from real users. > > > > > > > > > > In general, I support reducing the number of available rebalance > > > > > configuration parameters, but we should do it really carefully. > > > > > I can also propose - rebalanceOrder param for removing. > > > > > > > > > > On Wed, 12 Feb 2020 at 15:50, Alexei Scherbakov > > > > > wrote: > > > > > > > > > > > > Maxim, > > > > > > > > > > > > In general rebalanceDelay is used to delay/disable rebalance then > > > > > topology > > > > > > is changed. > > > > > > Right now we have BLT to avoid unnecesary rebalancing when topology > > > > > > is > > > > > > changed. > > > > > > If a node left from cluster topology no rebalancing happens until > > > > > > the > > > > > node > > > > > > explicitly removed from baseline topology. > > > > > > > > > > > > I would like to know real world scenarios which can not be covered > > > > > > by > > > > BLT > > > > > > configuration. > > > > > > > > > > > > > > > > > > > > > > > > ср, 12 февр. 2020 г. в 15:16, Maxim Muzafarov : > > > > > > > > > > > > > Alexey, > > > > > > > > > > > > > > > All scenarios where rebalanceDelay has meaning are handled by > > > > > baseline > > > > > > > topology now. > > > > > > > > > > > > > > Can you,
Re: ML examples EvaluatorExample and MultipleMetricsExample looks the same
I suppose one ticket is enough, collect where all your nites and thoughts чт, 13 февр. 2020 г., 12:12 Stepan Pilschikov : > And one (three) more thing > > In TutorialStepByStepExample we running 17 examples > First 12 logging is pretty good and looks like "Tutorial step N: name" -> > model -> accuracy -> "Tutorial step N: completed" > But then starting with 13 this pattern is kind of broke, step start and > step completion is missing > > Also want to admin that Step_8_CV_with_Param_Grid_and_metrics_and_pipeline > is haven't step completion log > And complete log for Step_9_Scaling_With_Stacking looks like 'Tutorial > step 5 (scaling) example completed' > > i don't know, should i made some ticket? One ticket for each problem or > one for all or you can fix it in one commit? because fixes quite small > > ср, 12 февр. 2020 г. в 16:31, Alexey Zinoviev : > >> Yes, you are right. Missed resources, I'll remove them tomorrow >> >> ср, 12 февр. 2020 г. в 16:23, Stepan Pilschikov < >> pilshchikov@gmail.com>: >> >>> With mnist_tf_model in same directory >>> All others looks right >>> >>> ср, 12 февр. 2020 г. в 16:21, Stepan Pilschikov < >>> pilshchikov@gmail.com>: >>> Hmm, interesting, got it Also found apache-ignite-2.8.0-bin/examples/src/main/resources/models/mleap I think its also should be removed then ср, 12 февр. 2020 г. в 16:11, Alexey Zinoviev : > Please, have a look to the next ticket > https://issues.apache.org/jira/browse/IGNITE-12659 > Also tensorflow packages are blockers for IGFS deprecation and removal. > > ср, 12 февр. 2020 г. в 15:58, Alexey Zinoviev >: > >> They are not lost, they are removed from release branch due to a lot >> of bugs, cves and so on, don't worry about that. >> >> ср, 12 февр. 2020 г., 14:31 Stepan Pilschikov < >> pilshchikov@gmail.com>: >> >>> Also lost ignite-ml-tensorflow-model-parser and ignite-tensorflow in >>> libs/optional >>> >>> >>> ср, 12 февр. 2020 г. в 14:25, Stepan Pilschikov < >>> pilshchikov@gmail.com>: >>> But now have new problems I build ignite-2.8 build on TC https://ci.ignite.apache.org/buildConfiguration/Releases_ApacheIgniteMain_ReleaseBuild/5043672 Previous https://ci.ignite.apache.org/buildConfiguration/Releases_ApacheIgniteMain_ReleaseBuild/4988324 New build much lighter Previous: 5.7 Gb Current: 4.3 Gb And its looks like something missing, on a first look i notice that /libs/optional/ignite-ml-mleap-model-parser lost completely ср, 12 февр. 2020 г. в 14:18, Stepan Pilschikov < pilshchikov@gmail.com>: > Thanks for fix > Duplicated examples deleted and tutorial works on real (1+ node) > cluster now > I think ticket can be closed, have no power here > > вт, 11 февр. 2020 г. в 20:44, Alexey Zinoviev < > zaleslaw@gmail.com>: > >> Sorry, for that, but I it touches ML-related stuff only and >> doesn't influence on any another module. >> I merged them to master later (currently, I have a 4 tickets in a >> queue). >> >> Will keep in mind for the next fixes >> >> вт, 11 февр. 2020 г. в 18:58, Maxim Muzafarov > >: >> >>> Alexey, >>> >>> Is the approach when we cherry-pick fixes to the 2.8 only after >>> all >>> fixes have verified in the master branch better? I just want to >>> be >>> sure the release branch to be stable enough. >>> >>> On Tue, 11 Feb 2020 at 18:36, Alexey Zinoviev < >>> zaleslaw@gmail.com> wrote: >>> > >>> > Fixed both in release branch 2.8. Please verify and let me know >>> > >>> > вт, 11 февр. 2020 г. в 14:58, Stepan Pilschikov < >>> pilshchikov@gmail.com>: >>> > >>> > > And one more >>> > > https://issues.apache.org/jira/browse/IGNITE-12658 - >>> > > TutorialStepByStepExample example failed if cluster with >>> more then 1 node >>> > > >>> > > вт, 11 февр. 2020 г. в 11:52, Alexey Zinoviev < >>> zaleslaw@gmail.com>: >>> > > >>> > >> Great, thank you so much, will fix next week >>> > >> >>> > >> вт, 11 февр. 2020 г., 11:39 Stepan Pilschikov < >>> pilshchikov@gmail.com >>> > >> >: >>> > >> >>> > >>> Hi, Alexey, could you please looking on one small accident >>> happen in ML >>> > >>> examples: >>> > >>> https://issues.apache.org/jira/browse/IGNITE-12657 - 2 >>> examples exactly >>> > >>> the same. >>> > >>> >>> > >>> Also want to say that all previous examples have one >>> common approach for >>> >
Re: [DISCUSSION] Deprecation of obsolete rebalancing functionality
> +1 to deprecate rebalanceOrder and remove related functionality, Meant to "rework related functionality" not "remove". чт, 13 февр. 2020 г. в 12:47, Pavel Pereslegin : > > Hello, > > +1 to deprecate rebalanceOrder and remove related functionality, > should we create a separate ticket for this? > > Btw, as I understand, SYNC mode is only useful for in-memory caches, > because when persistence is enabled (and WAL is disabled during > rebalancing), even "ignite-sys-cache" owns partitions only after all > cache groups are rebalanced. Thus, even utility cache is still > inoperable after node startup when persistence is enabled. Do we > really need to wait for SYNC caches when a node starts with enabled > persistence or should we enabled WAL for SYNC-caches? > > чт, 13 февр. 2020 г. в 11:13, Ivan Rakov : > > > > Hello, > > > > +1 from me for rebalance delay deprecation. > > I can imagine only one actual case for this option: prevent excessive load > > on the cluster in case of temporary short-term topology changes (e.g. node > > is stopped for a while and then returned back). > > Now it's handled by baseline auto adjustment in a much more correct way: > > partitions are not reassigned within a maintenance interval (unlike with > > the rebalance delay). > > I also don't think that ability to configure rebalance delay per cache is > > crucial. > > > > > rebalanceOrder is also useless, agreed. > > +1 > > Except for one case: we may want to rebalance caches with > > CacheRebalanceMode.SYNC first. But anyway, this behavior doesn't require a > > separate property to be enabled. > > > > On Wed, Feb 12, 2020 at 4:54 PM Alexei Scherbakov < > > alexey.scherbak...@gmail.com> wrote: > > > > > Maxim, > > > > > > rebalanceDelay was introduced before the BLT appear in the product to > > > solve > > > scenarios which are now solved by BLT. > > > > > > It's pointless for me having it in the product since BLT was introduced. > > > > > > I do not think delaying rebalancing per cache group has any meaning. I > > > cannot image any reason for it. > > > > > > rebalanceOrder is also useless, agreed. > > > > > > > > > > > > > > > ср, 12 февр. 2020 г. в 16:19, Maxim Muzafarov : > > > > > > > Alexey, > > > > > > > > Why do you think delaying of historical rebalance (on BLT node join) > > > > for particular cache groups is not the real world use case? Probably > > > > the same topic may be started on user-list to collect more use cases > > > > from real users. > > > > > > > > In general, I support reducing the number of available rebalance > > > > configuration parameters, but we should do it really carefully. > > > > I can also propose - rebalanceOrder param for removing. > > > > > > > > On Wed, 12 Feb 2020 at 15:50, Alexei Scherbakov > > > > wrote: > > > > > > > > > > Maxim, > > > > > > > > > > In general rebalanceDelay is used to delay/disable rebalance then > > > > topology > > > > > is changed. > > > > > Right now we have BLT to avoid unnecesary rebalancing when topology is > > > > > changed. > > > > > If a node left from cluster topology no rebalancing happens until the > > > > node > > > > > explicitly removed from baseline topology. > > > > > > > > > > I would like to know real world scenarios which can not be covered by > > > BLT > > > > > configuration. > > > > > > > > > > > > > > > > > > > > ср, 12 февр. 2020 г. в 15:16, Maxim Muzafarov : > > > > > > > > > > > Alexey, > > > > > > > > > > > > > All scenarios where rebalanceDelay has meaning are handled by > > > > baseline > > > > > > topology now. > > > > > > > > > > > > Can you, please, provide more details here e.g. the whole list of > > > > > > scenarios where rebalanceDelay is used and how these handled by > > > > > > baseline topology? > > > > > > > > > > > > Actually, I doubt that it covers exactly all the cases due to > > > > > > rebalanceDelay is a "per cache group property" rather than > > > > > > "baseline" > > > > > > is meaningful for the whole topology. > > > > > > > > > > > > On Wed, 12 Feb 2020 at 12:58, Alexei Scherbakov > > > > > > wrote: > > > > > > > > > > > > > > I've meant baseline topology. > > > > > > > > > > > > > > ср, 12 февр. 2020 г. в 12:41, Alexei Scherbakov < > > > > > > > alexey.scherbak...@gmail.com>: > > > > > > > > > > > > > > > > > > > > > > > V.Pyatkov > > > > > > > > > > > > > > > > Doesn't rebalance topology solves it ? > > > > > > > > > > > > > > > > ср, 12 февр. 2020 г. в 12:31, V.Pyatkov : > > > > > > > > > > > > > > > >> Hi, > > > > > > > >> > > > > > > > >> I am sure we can to reduce this ability, but do not completely. > > > > > > > >> We can use rebalance delay for disable it until manually > > > > triggered. > > > > > > > >> > > > > > > > >> CacheConfiguration#setRebalanceDelay(-1) > > > > > > > >> > > > > > > > >> It may helpful for cluster where can not allow performance drop > > > > from > > > > > > > >> rebalance at any time. > > > > > > > >> > > > > > > > >> > > > > > > > >> > > > > > > > >> -- > > > > > > > >> Sent
Re: [DISCUSSION] Deprecation of obsolete rebalancing functionality
Hello, +1 to deprecate rebalanceOrder and remove related functionality, should we create a separate ticket for this? Btw, as I understand, SYNC mode is only useful for in-memory caches, because when persistence is enabled (and WAL is disabled during rebalancing), even "ignite-sys-cache" owns partitions only after all cache groups are rebalanced. Thus, even utility cache is still inoperable after node startup when persistence is enabled. Do we really need to wait for SYNC caches when a node starts with enabled persistence or should we enabled WAL for SYNC-caches? чт, 13 февр. 2020 г. в 11:13, Ivan Rakov : > > Hello, > > +1 from me for rebalance delay deprecation. > I can imagine only one actual case for this option: prevent excessive load > on the cluster in case of temporary short-term topology changes (e.g. node > is stopped for a while and then returned back). > Now it's handled by baseline auto adjustment in a much more correct way: > partitions are not reassigned within a maintenance interval (unlike with > the rebalance delay). > I also don't think that ability to configure rebalance delay per cache is > crucial. > > > rebalanceOrder is also useless, agreed. > +1 > Except for one case: we may want to rebalance caches with > CacheRebalanceMode.SYNC first. But anyway, this behavior doesn't require a > separate property to be enabled. > > On Wed, Feb 12, 2020 at 4:54 PM Alexei Scherbakov < > alexey.scherbak...@gmail.com> wrote: > > > Maxim, > > > > rebalanceDelay was introduced before the BLT appear in the product to solve > > scenarios which are now solved by BLT. > > > > It's pointless for me having it in the product since BLT was introduced. > > > > I do not think delaying rebalancing per cache group has any meaning. I > > cannot image any reason for it. > > > > rebalanceOrder is also useless, agreed. > > > > > > > > > > ср, 12 февр. 2020 г. в 16:19, Maxim Muzafarov : > > > > > Alexey, > > > > > > Why do you think delaying of historical rebalance (on BLT node join) > > > for particular cache groups is not the real world use case? Probably > > > the same topic may be started on user-list to collect more use cases > > > from real users. > > > > > > In general, I support reducing the number of available rebalance > > > configuration parameters, but we should do it really carefully. > > > I can also propose - rebalanceOrder param for removing. > > > > > > On Wed, 12 Feb 2020 at 15:50, Alexei Scherbakov > > > wrote: > > > > > > > > Maxim, > > > > > > > > In general rebalanceDelay is used to delay/disable rebalance then > > > topology > > > > is changed. > > > > Right now we have BLT to avoid unnecesary rebalancing when topology is > > > > changed. > > > > If a node left from cluster topology no rebalancing happens until the > > > node > > > > explicitly removed from baseline topology. > > > > > > > > I would like to know real world scenarios which can not be covered by > > BLT > > > > configuration. > > > > > > > > > > > > > > > > ср, 12 февр. 2020 г. в 15:16, Maxim Muzafarov : > > > > > > > > > Alexey, > > > > > > > > > > > All scenarios where rebalanceDelay has meaning are handled by > > > baseline > > > > > topology now. > > > > > > > > > > Can you, please, provide more details here e.g. the whole list of > > > > > scenarios where rebalanceDelay is used and how these handled by > > > > > baseline topology? > > > > > > > > > > Actually, I doubt that it covers exactly all the cases due to > > > > > rebalanceDelay is a "per cache group property" rather than "baseline" > > > > > is meaningful for the whole topology. > > > > > > > > > > On Wed, 12 Feb 2020 at 12:58, Alexei Scherbakov > > > > > wrote: > > > > > > > > > > > > I've meant baseline topology. > > > > > > > > > > > > ср, 12 февр. 2020 г. в 12:41, Alexei Scherbakov < > > > > > > alexey.scherbak...@gmail.com>: > > > > > > > > > > > > > > > > > > > > V.Pyatkov > > > > > > > > > > > > > > Doesn't rebalance topology solves it ? > > > > > > > > > > > > > > ср, 12 февр. 2020 г. в 12:31, V.Pyatkov : > > > > > > > > > > > > > >> Hi, > > > > > > >> > > > > > > >> I am sure we can to reduce this ability, but do not completely. > > > > > > >> We can use rebalance delay for disable it until manually > > > triggered. > > > > > > >> > > > > > > >> CacheConfiguration#setRebalanceDelay(-1) > > > > > > >> > > > > > > >> It may helpful for cluster where can not allow performance drop > > > from > > > > > > >> rebalance at any time. > > > > > > >> > > > > > > >> > > > > > > >> > > > > > > >> -- > > > > > > >> Sent from: > > http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > > >> > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > Best regards, > > > > > > > Alexei Scherbakov > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Best regards, > > > > > > Alexei Scherbakov > > > > > > > > > > > > > > > > > -- > > > > > > > > Best regards, > > > > Alexei Scherbakov > > > > > > > > > -- > > > > Best regards,
Re: ML examples EvaluatorExample and MultipleMetricsExample looks the same
And one (three) more thing In TutorialStepByStepExample we running 17 examples First 12 logging is pretty good and looks like "Tutorial step N: name" -> model -> accuracy -> "Tutorial step N: completed" But then starting with 13 this pattern is kind of broke, step start and step completion is missing Also want to admin that Step_8_CV_with_Param_Grid_and_metrics_and_pipeline is haven't step completion log And complete log for Step_9_Scaling_With_Stacking looks like 'Tutorial step 5 (scaling) example completed' i don't know, should i made some ticket? One ticket for each problem or one for all or you can fix it in one commit? because fixes quite small ср, 12 февр. 2020 г. в 16:31, Alexey Zinoviev : > Yes, you are right. Missed resources, I'll remove them tomorrow > > ср, 12 февр. 2020 г. в 16:23, Stepan Pilschikov >: > >> With mnist_tf_model in same directory >> All others looks right >> >> ср, 12 февр. 2020 г. в 16:21, Stepan Pilschikov < >> pilshchikov@gmail.com>: >> >>> Hmm, interesting, got it >>> Also found >>> apache-ignite-2.8.0-bin/examples/src/main/resources/models/mleap >>> I think its also should be removed then >>> >>> ср, 12 февр. 2020 г. в 16:11, Alexey Zinoviev : >>> Please, have a look to the next ticket https://issues.apache.org/jira/browse/IGNITE-12659 Also tensorflow packages are blockers for IGFS deprecation and removal. ср, 12 февр. 2020 г. в 15:58, Alexey Zinoviev : > They are not lost, they are removed from release branch due to a lot > of bugs, cves and so on, don't worry about that. > > ср, 12 февр. 2020 г., 14:31 Stepan Pilschikov < > pilshchikov@gmail.com>: > >> Also lost ignite-ml-tensorflow-model-parser and ignite-tensorflow in >> libs/optional >> >> >> ср, 12 февр. 2020 г. в 14:25, Stepan Pilschikov < >> pilshchikov@gmail.com>: >> >>> But now have new problems >>> I build ignite-2.8 build on TC >>> https://ci.ignite.apache.org/buildConfiguration/Releases_ApacheIgniteMain_ReleaseBuild/5043672 >>> Previous >>> https://ci.ignite.apache.org/buildConfiguration/Releases_ApacheIgniteMain_ReleaseBuild/4988324 >>> >>> New build much lighter >>> Previous: 5.7 Gb >>> Current: 4.3 Gb >>> And its looks like something missing, on a first look i notice that >>> /libs/optional/ignite-ml-mleap-model-parser lost completely >>> >>> >>> ср, 12 февр. 2020 г. в 14:18, Stepan Pilschikov < >>> pilshchikov@gmail.com>: >>> Thanks for fix Duplicated examples deleted and tutorial works on real (1+ node) cluster now I think ticket can be closed, have no power here вт, 11 февр. 2020 г. в 20:44, Alexey Zinoviev < zaleslaw@gmail.com>: > Sorry, for that, but I it touches ML-related stuff only and > doesn't influence on any another module. > I merged them to master later (currently, I have a 4 tickets in a > queue). > > Will keep in mind for the next fixes > > вт, 11 февр. 2020 г. в 18:58, Maxim Muzafarov : > >> Alexey, >> >> Is the approach when we cherry-pick fixes to the 2.8 only after >> all >> fixes have verified in the master branch better? I just want to be >> sure the release branch to be stable enough. >> >> On Tue, 11 Feb 2020 at 18:36, Alexey Zinoviev < >> zaleslaw@gmail.com> wrote: >> > >> > Fixed both in release branch 2.8. Please verify and let me know >> > >> > вт, 11 февр. 2020 г. в 14:58, Stepan Pilschikov < >> pilshchikov@gmail.com>: >> > >> > > And one more >> > > https://issues.apache.org/jira/browse/IGNITE-12658 - >> > > TutorialStepByStepExample example failed if cluster with more >> then 1 node >> > > >> > > вт, 11 февр. 2020 г. в 11:52, Alexey Zinoviev < >> zaleslaw@gmail.com>: >> > > >> > >> Great, thank you so much, will fix next week >> > >> >> > >> вт, 11 февр. 2020 г., 11:39 Stepan Pilschikov < >> pilshchikov@gmail.com >> > >> >: >> > >> >> > >>> Hi, Alexey, could you please looking on one small accident >> happen in ML >> > >>> examples: >> > >>> https://issues.apache.org/jira/browse/IGNITE-12657 - 2 >> examples exactly >> > >>> the same. >> > >>> >> > >>> Also want to say that all previous examples have one common >> approach for >> > >>> meaningful output, like ">>> Something important". In this >> two i seeing >> > >>> different one, could you please also pay attention to it. >> > >>> >> > >>> Best regards, >> > >>> Stepan >> > >>> >> > >> >> >
[RESULT][VOTE] Allow or prohibit a joint use of @deprecated and @IgniteExperimental
Dear Ignite community, The vote is now closed. The votes spread the following way: [+1 Allow]: 5 votes: Vyacheslav Daradur Tkalenko Kirill Nikolay Izhikov Ilya Kasnacheev Maxim Muzafarov [-1 Prohibit]: 23 votes: Sergey Antonov Ivan Pavlukhin Philipp Masharov Alexey Zinoviev Yuri Gerzhedovich Mekhanikov Denis Andrey Mashenkov Roman Kondakov Ivan Rakov Anton Kalashnikov Zhenya Stanilovsky Pavel Tupitsyn Dmitriy Govorukhin Vyacheslav Koptilin Alexei Scherbakov Konstantin Orlov Alexey Kuznetsov Anton Vinogradov Denis Magda Alex Plehanov Andrey Gura Sergey Chugunov Alexey Goncharuk Vote thread: http://apache-ignite-developers.2346864.n4.nabble.com/VOTE-Allow-or-prohibit-a-joint-use-of-deprecated-and-IgniteExperimental-td45749.html The Ignite community ruled in favor of prohibiting the simultaneous use of @deprecated and @IgniteExperimental. I will update the wiki shortly.
Re: [DISCUSSION] Deprecation of obsolete rebalancing functionality
Hello, +1 from me for rebalance delay deprecation. I can imagine only one actual case for this option: prevent excessive load on the cluster in case of temporary short-term topology changes (e.g. node is stopped for a while and then returned back). Now it's handled by baseline auto adjustment in a much more correct way: partitions are not reassigned within a maintenance interval (unlike with the rebalance delay). I also don't think that ability to configure rebalance delay per cache is crucial. > rebalanceOrder is also useless, agreed. +1 Except for one case: we may want to rebalance caches with CacheRebalanceMode.SYNC first. But anyway, this behavior doesn't require a separate property to be enabled. On Wed, Feb 12, 2020 at 4:54 PM Alexei Scherbakov < alexey.scherbak...@gmail.com> wrote: > Maxim, > > rebalanceDelay was introduced before the BLT appear in the product to solve > scenarios which are now solved by BLT. > > It's pointless for me having it in the product since BLT was introduced. > > I do not think delaying rebalancing per cache group has any meaning. I > cannot image any reason for it. > > rebalanceOrder is also useless, agreed. > > > > > ср, 12 февр. 2020 г. в 16:19, Maxim Muzafarov : > > > Alexey, > > > > Why do you think delaying of historical rebalance (on BLT node join) > > for particular cache groups is not the real world use case? Probably > > the same topic may be started on user-list to collect more use cases > > from real users. > > > > In general, I support reducing the number of available rebalance > > configuration parameters, but we should do it really carefully. > > I can also propose - rebalanceOrder param for removing. > > > > On Wed, 12 Feb 2020 at 15:50, Alexei Scherbakov > > wrote: > > > > > > Maxim, > > > > > > In general rebalanceDelay is used to delay/disable rebalance then > > topology > > > is changed. > > > Right now we have BLT to avoid unnecesary rebalancing when topology is > > > changed. > > > If a node left from cluster topology no rebalancing happens until the > > node > > > explicitly removed from baseline topology. > > > > > > I would like to know real world scenarios which can not be covered by > BLT > > > configuration. > > > > > > > > > > > > ср, 12 февр. 2020 г. в 15:16, Maxim Muzafarov : > > > > > > > Alexey, > > > > > > > > > All scenarios where rebalanceDelay has meaning are handled by > > baseline > > > > topology now. > > > > > > > > Can you, please, provide more details here e.g. the whole list of > > > > scenarios where rebalanceDelay is used and how these handled by > > > > baseline topology? > > > > > > > > Actually, I doubt that it covers exactly all the cases due to > > > > rebalanceDelay is a "per cache group property" rather than "baseline" > > > > is meaningful for the whole topology. > > > > > > > > On Wed, 12 Feb 2020 at 12:58, Alexei Scherbakov > > > > wrote: > > > > > > > > > > I've meant baseline topology. > > > > > > > > > > ср, 12 февр. 2020 г. в 12:41, Alexei Scherbakov < > > > > > alexey.scherbak...@gmail.com>: > > > > > > > > > > > > > > > > > V.Pyatkov > > > > > > > > > > > > Doesn't rebalance topology solves it ? > > > > > > > > > > > > ср, 12 февр. 2020 г. в 12:31, V.Pyatkov : > > > > > > > > > > > >> Hi, > > > > > >> > > > > > >> I am sure we can to reduce this ability, but do not completely. > > > > > >> We can use rebalance delay for disable it until manually > > triggered. > > > > > >> > > > > > >> CacheConfiguration#setRebalanceDelay(-1) > > > > > >> > > > > > >> It may helpful for cluster where can not allow performance drop > > from > > > > > >> rebalance at any time. > > > > > >> > > > > > >> > > > > > >> > > > > > >> -- > > > > > >> Sent from: > http://apache-ignite-developers.2346864.n4.nabble.com/ > > > > > >> > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > Best regards, > > > > > > Alexei Scherbakov > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Best regards, > > > > > Alexei Scherbakov > > > > > > > > > > > > > -- > > > > > > Best regards, > > > Alexei Scherbakov > > > > > -- > > Best regards, > Alexei Scherbakov >
