Re: Recap of Ignite ML presentation at ApacheCon

2019-09-18 Thread Denis Magda
Alexey, thanks, that's the source code I named as Payments Fraud Detection: https://github.com/dmagda/ignite-ml-examples/blob/master/logistic-regression-example/src/main/java/org/apache/ignite/example/ClientNode.java - Denis On Wed, Sep 18, 2019 at 2:27 AM Alexey Zinoviev wrote: > Cool

Re: [SparkDataFrame] Query Optimization. Prototype

2019-09-18 Thread liyuj
Hi Nikolay, Because in the discussion below, there is a list describing that ROWNUM is not applicable to Ignite, and I want to know why. 在 2019/9/18 下午11:14, Nikolay Izhikov 写道: Hello, liyuj. Please, clarify. Do you want to contribute this to Ignite? What explanation do you expect? В Ср,

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-18 Thread Denis Magda
Igniters, 2.8 release cycle can last for many months due to a number of issues and regressions we need to address. It can easily slip to the next year, especially considering the holiday season. An alternate solution can be a series of Ignite 2.7.x releases that are becoming more hardened with

Re: TDE Master key rotation (Phase-2)

2019-09-18 Thread Nikita Amelchev
Nikolay, thanks for participating. I have supplemented the design and clarify these moments. [1] [1] https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=95652381 ср, 18 сент. 2019 г. в 16:48, Nikolay Izhikov : > > Hello, Nikita. > > Thanks for starting this discussion. > > 1. We

[jira] [Created] (IGNITE-12198) GridDiscoveryManager uses hardcoded failure handler

2019-09-18 Thread Rohit Joshi (Jira)
Rohit Joshi created IGNITE-12198: Summary: GridDiscoveryManager uses hardcoded failure handler Key: IGNITE-12198 URL: https://issues.apache.org/jira/browse/IGNITE-12198 Project: Ignite Issue

Re: How to free up space on disc after removing entries from IgniteCache with enabled PDS?

2019-09-18 Thread Alexey Goncharuk
Denis, It's not fundamental, but quite complex. In postgres, for example, this is not maintained automatically and store compaction is performed using the full vacuum command, which acquires exclusive table lock, so no concurrent activities on the table are possible. The solution which Anton

[jira] [Created] (IGNITE-12197) Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl

2019-09-18 Thread Andrey Gura (Jira)
Andrey Gura created IGNITE-12197: Summary: Incorrect way for getting value of persistent enabled in CacheGroupMetricsImpl Key: IGNITE-12197 URL: https://issues.apache.org/jira/browse/IGNITE-12197

[DISCUSSION] The rebalancing process metrics update

2019-09-18 Thread Maxim Muzafarov
Igniters, Currently, from my perspective, the Apache Ignite has a very raw rebalance process metrics. Moreover, the most interesting metrics are related to the Cache, not a CacheGroup and require enabling cache statistics which can affect node performance. Some of the metrics are not working as

[jira] [Created] (IGNITE-12196) [Phase-4] Deprecate old rebalancing cache metrics

2019-09-18 Thread Maxim Muzafarov (Jira)
Maxim Muzafarov created IGNITE-12196: Summary: [Phase-4] Deprecate old rebalancing cache metrics Key: IGNITE-12196 URL: https://issues.apache.org/jira/browse/IGNITE-12196 Project: Ignite

Re: Apache Ignite 2.7.6 (Time, Scope, and Release manager)

2019-09-18 Thread Alexey Goncharuk
Igniters, I am following the release process procedure [1] and currently on docs update step. I can login to the admin panel, but I can only see the main Ignite docs project. Who can add me to the remaining projects (C#/.NET, C++, SQL, Integrations*, Ignite for Spark, Tools) or perhaps clone the

[jira] [Created] (IGNITE-12195) [Phase-3] Rebalance HitRate metrics

2019-09-18 Thread Maxim Muzafarov (Jira)
Maxim Muzafarov created IGNITE-12195: Summary: [Phase-3] Rebalance HitRate metrics Key: IGNITE-12195 URL: https://issues.apache.org/jira/browse/IGNITE-12195 Project: Ignite Issue Type:

[jira] [Created] (IGNITE-12194) [Phase-2] Calculate expected rebalancing cache group keys

2019-09-18 Thread Maxim Muzafarov (Jira)
Maxim Muzafarov created IGNITE-12194: Summary: [Phase-2] Calculate expected rebalancing cache group keys Key: IGNITE-12194 URL: https://issues.apache.org/jira/browse/IGNITE-12194 Project: Ignite

[jira] [Created] (IGNITE-12193) [Phase-1] Rebalancing cache group keys metric counters

2019-09-18 Thread Maxim Muzafarov (Jira)
Maxim Muzafarov created IGNITE-12193: Summary: [Phase-1] Rebalancing cache group keys metric counters Key: IGNITE-12193 URL: https://issues.apache.org/jira/browse/IGNITE-12193 Project: Ignite

Re: [SparkDataFrame] Query Optimization. Prototype

2019-09-18 Thread Nikolay Izhikov
Hello, liyuj. Please, clarify. Do you want to contribute this to Ignite? What explanation do you expect? В Ср, 18/09/2019 в 22:46 +0800, liyuj пишет: > Hi, > > Can anyone explain how difficult it is to implement ROWNUM? > > This is a very common requirement. > > 在 2018/1/23 下午4:05, Serge

Re: [SparkDataFrame] Query Optimization. Prototype

2019-09-18 Thread liyuj
Hi, Can anyone explain how difficult it is to implement ROWNUM? This is a very common requirement. 在 2018/1/23 下午4:05, Serge Puchnin 写道: yes, the Cust function is supporting both Ignite and H2. I've updated the documentation for next system functions: CASEWHEN Function, CAST, CONVERT, TABLE

Re: TDE Master key rotation (Phase-2)

2019-09-18 Thread Nikolay Izhikov
Hello, Nikita. Thanks for starting this discussion. 1. We should add prerequisites for "master key rotation process" in design. Seems, it should be, "New master key available to EncryptionSPI for each server node". 2. Please, use code formatting in wiki. It's make reading easier. 3. Please,

[jira] [Created] (IGNITE-12192) visorcmd hangs if you ^D to quit

2019-09-18 Thread Stephen Darlington (Jira)
Stephen Darlington created IGNITE-12192: --- Summary: visorcmd hangs if you ^D to quit Key: IGNITE-12192 URL: https://issues.apache.org/jira/browse/IGNITE-12192 Project: Ignite Issue

Re: [IEP-35] Monitoring & Profiling. Phase 2

2019-09-18 Thread Alex Plehanov
One more point to discuss: Wouldn't it be better to have enabled system views by default? To enable views admin must restart the node, sometimes it's an issue. Views cost almost nothing in terms of performance until they are explicitly requested, so is their a reason to disable views by default?

TDE Master key rotation (Phase-2)

2019-09-18 Thread Nikita Amelchev
Hi, Igniters. I'm going to implement the ability to rotate the master encryption key (TDE Phase 2). [1] Master key rotation required in case of it compromising or at the end of crypto period(key validity period). I prepared the design. [2] In briefly, master keys will be identified by String

[jira] [Created] (IGNITE-12191) Add support of system view and metric to Visor

2019-09-18 Thread Nikolay Izhikov (Jira)
Nikolay Izhikov created IGNITE-12191: Summary: Add support of system view and metric to Visor Key: IGNITE-12191 URL: https://issues.apache.org/jira/browse/IGNITE-12191 Project: Ignite

[jira] [Created] (IGNITE-12190) Throw an exception for enabled text queries on persistent caches

2019-09-18 Thread Yuriy Shuliha (Jira)
Yuriy Shuliha created IGNITE-12190: --- Summary: Throw an exception for enabled text queries on persistent caches Key: IGNITE-12190 URL: https://issues.apache.org/jira/browse/IGNITE-12190 Project:

[jira] [Created] (IGNITE-12189) Implement correct limit for TextQuery

2019-09-18 Thread Yuriy Shuliha (Jira)
Yuriy Shuliha created IGNITE-12189: --- Summary: Implement correct limit for TextQuery Key: IGNITE-12189 URL: https://issues.apache.org/jira/browse/IGNITE-12189 Project: Ignite Issue Type:

[jira] [Created] (IGNITE-12188) Metric CacheGroupMetrics.IndexBuildCountPartitionsLeft doesn't work correctly if there is more then one cache in cache group

2019-09-18 Thread Aleksey Plekhanov (Jira)
Aleksey Plekhanov created IGNITE-12188: -- Summary: Metric CacheGroupMetrics.IndexBuildCountPartitionsLeft doesn't work correctly if there is more then one cache in cache group Key: IGNITE-12188 URL:

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-18 Thread Nikolay Izhikov
Start of December for 2.8 sounds good. В Ср, 18/09/2019 в 13:38 +0300, Alexey Zinoviev пишет: > Sounds good for me, I will be ready to release ML module till 1 December > (end of September is not appropriate for ML module due to known issues and > bugs) > If the whole Ignite could be released to

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-18 Thread Alexey Zinoviev
Sounds good for me, I will be ready to release ML module till 1 December (end of September is not appropriate for ML module due to known issues and bugs) If the whole Ignite could be released to this date I will be happy in another case I suggest the minor release by the formula mentioned above.

Re: Non-blocking PME Phase One (Node fail)

2019-09-18 Thread Anton Vinogradov
Alexey, >> Can you describe the complete protocol changes first The current goal is to find a better way. I had at least 5 scenarios discarded because of finding corner cases (Thanks to Alexey Scherbakov, Aleksei Plekhanov and Nikita Amelchev). That's why I explained what we able to improve and

[jira] [Created] (IGNITE-12187) [IEP-35] Move GridMetricManager and dependent classes to org.apache.ignite.internal.managers

2019-09-18 Thread Nikolay Izhikov (Jira)
Nikolay Izhikov created IGNITE-12187: Summary: [IEP-35] Move GridMetricManager and dependent classes to org.apache.ignite.internal.managers Key: IGNITE-12187 URL:

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-18 Thread Maxim Muzafarov
Folks, Agree, we should start a discussion about releasing Apache Ignite 2.8 version right after 2.7.6 will be announced. Am I missing something? On Wed, 18 Sep 2019 at 12:33, Nikolay Izhikov wrote: > > > of 2.7 release > > It's 2.8, of course. Sorry, for typo. > > > В Ср, 18/09/2019 в 12:36

[jira] [Created] (IGNITE-12186) TDE - Phase-2. Master key rotation.

2019-09-18 Thread Amelchev Nikita (Jira)
Amelchev Nikita created IGNITE-12186: Summary: TDE - Phase-2. Master key rotation. Key: IGNITE-12186 URL: https://issues.apache.org/jira/browse/IGNITE-12186 Project: Ignite Issue Type:

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-18 Thread Nikolay Izhikov
> of 2.7 release It's 2.8, of course. Sorry, for typo. В Ср, 18/09/2019 в 12:36 +0300, Nikolay Izhikov пишет: > Hello, Alexey > > > we have no chance to > > release the whole master branch > > Why do you think so? > From my point of view, we should start discussion of 2.7 release, shortly(end

Re: Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-18 Thread Nikolay Izhikov
Hello, Alexey > we have no chance to > release the whole master branch Why do you think so? From my point of view, we should start discussion of 2.7 release, shortly(end of September of similar) В Ср, 18/09/2019 в 12:04 +0300, Alexey Zinoviev пишет: > Dear Igniters, for the whole year I'm

Re: ML stable and performance

2019-09-18 Thread Alexey Zinoviev
I've started the discussion here http://apache-ignite-developers.2346864.n4.nabble.com/Proposed-Release-2-7-7-2-7-6-ML-extra-fixed-bugs-td43537.html пт, 13 сент. 2019 г. в 22:12, Alexey Zinoviev : > The reason was that the last year there is no significant releases of > Ignite between 2.7 and

Re: Recap of Ignite ML presentation at ApacheCon

2019-09-18 Thread Alexey Zinoviev
Cool presentation, thank you for evangelism for ML module Could you share the link on the code from demo: Payments Fraud Detection? вт, 17 сент. 2019 г. в 02:25, Denis Magda : > Hey Ignite dev and user communities, > > Those of you who are passionate about ML and Ignite ML in particular, we >

Proposed Release: 2.7.7 = 2.7.6 + ML + extra fixed bugs

2019-09-18 Thread Alexey Zinoviev
Dear Igniters, for the whole year I'm waiting for the next release of Apache Ignite with number 2.8 or 3.0. As I understand we have no chance to release the whole master branch, but we have cool changes in ML component that were not released since 2.7 (Nomember 2018). I'm ready to become a

[jira] [Created] (IGNITE-12185) [IEP-35] Add metrics - how many indexes are left to rebuild

2019-09-18 Thread Aleksandr Brazhnikov (Jira)
Aleksandr Brazhnikov created IGNITE-12185: - Summary: [IEP-35] Add metrics - how many indexes are left to rebuild Key: IGNITE-12185 URL: https://issues.apache.org/jira/browse/IGNITE-12185

[jira] [Created] (IGNITE-12184) [IEP-35] Add metrics - status rebild indexes

2019-09-18 Thread Aleksandr Brazhnikov (Jira)
Aleksandr Brazhnikov created IGNITE-12184: - Summary: [IEP-35] Add metrics - status rebild indexes Key: IGNITE-12184 URL: https://issues.apache.org/jira/browse/IGNITE-12184 Project: Ignite

[jira] [Created] (IGNITE-12183) [IEP-35] Add metrics rebalancing

2019-09-18 Thread Aleksandr Brazhnikov (Jira)
Aleksandr Brazhnikov created IGNITE-12183: - Summary: [IEP-35] Add metrics rebalancing Key: IGNITE-12183 URL: https://issues.apache.org/jira/browse/IGNITE-12183 Project: Ignite Issue

Re: Non-blocking PME Phase One (Node fail)

2019-09-18 Thread Alexey Goncharuk
Anton, I looked through the presentation and the ticket but did not find any new protocol description that you are going to implement. Can you describe the complete protocol changes first (to save time for both you and during the later review)? Some questions that I have in mind: * It looks

[jira] [Created] (IGNITE-12182) ExecutorService is inconsistent with Compute and Services, runs on clients

2019-09-18 Thread Stephen Darlington (Jira)
Stephen Darlington created IGNITE-12182: --- Summary: ExecutorService is inconsistent with Compute and Services, runs on clients Key: IGNITE-12182 URL: https://issues.apache.org/jira/browse/IGNITE-12182

Non-blocking PME Phase One (Node fail)

2019-09-18 Thread Anton Vinogradov
Igniters, Recently we had a discussion devoted to the non-blocking PME. We agreed that the most important case is a blocking on node failure and it can be splitted to: 1) Affected partition’s operations latency will be increased by node failure detection duration. So, some operations may be

Re: Text queries/indexes (GridLuceneIndex, @QueryTextFiled)

2019-09-18 Thread Alexei Scherbakov
Denis, I like the idea of throwing an exception for enabled text queries on persistent caches. Also I'm fine with proposed limit for unsorted searches. Yury, please proceed with ticket creation. вт, 17 сент. 2019 г., 22:06 Denis Magda : > Igniters, > > I see nothing wrong with Yury's proposal