[jira] [Commented] (IGNITE-11993) Print warning if awaiting next wal segment it too long

2019-07-17 Thread Ignite TC Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887696#comment-16887696
 ] 

Ignite TC Bot commented on IGNITE-11993:


{panel:title=--> Run :: All: Possible 
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Platform C++ (Linux Clang){color} [[tests 506 JVM CRASH , Exit 
Code , Failure on metric 
|https://ci.ignite.apache.org/viewLog.html?buildId=4345318]]

{color:#d04437}Client Nodes{color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=4345278]]
* IgniteClientReconnectTestSuite: 
IgniteClientReconnectContinuousProcessorTest.testMessageListenerReconnectAndStopFromServer

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4345386&buildTypeId=IgniteTests24Java8_RunAll]

> Print warning if awaiting next wal segment it too long
> --
>
> Key: IGNITE-11993
> URL: https://issues.apache.org/jira/browse/IGNITE-11993
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> We must print warn to log, if awaiting next WAL segment more then defined 
> threshold.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-10761) GridCacheProcessor should add info about cache in excecption message, if applicable.

2019-07-17 Thread Ignite TC Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887693#comment-16887693
 ] 

Ignite TC Bot commented on IGNITE-10761:


{panel:title=--> Run :: All: Possible 
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}PDS (Indexing){color} [[tests 
3|https://ci.ignite.apache.org/viewLog.html?buildId=4343593]]
* IgnitePdsWithIndexingCoreTestSuite: 
IgniteCacheGroupsWithRestartsTest.testCleaningGarbageAfterCacheDestroyedAndNodeStop_ControlConsoleUtil

{color:#d04437}Platform .NET{color} [[tests 1 JVM CRASH , Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=4343599]]

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4343628&buildTypeId=IgniteTests24Java8_RunAll]

> GridCacheProcessor should add info about cache in excecption message, if 
> applicable. 
> -
>
> Key: IGNITE-10761
> URL: https://issues.apache.org/jira/browse/IGNITE-10761
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Sergey Antonov
>Assignee: Kirill Tkalenko
>Priority: Major
> Fix For: 2.8
>
>
> We should add info about problem cache in exception message. 



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11989) Preload predicate not used in GridCachePreloader

2019-07-17 Thread Anton Vinogradov (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887686#comment-16887686
 ] 

Anton Vinogradov commented on IGNITE-11989:
---

[~Mmuzaf]
LGTM, +1 to devlist, 
but ... in case no tests failed, seems this is obsolete useless feature :) 

> Preload predicate not used in GridCachePreloader
> 
>
> Key: IGNITE-11989
> URL: https://issues.apache.org/jira/browse/IGNITE-11989
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Minor
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {code:title=GridCachePreloader.java}
> /**
>  * @return Preload predicate. If not {@code null}, will evaluate each 
> preloaded entry during
>  *  send and receive, and if predicate evaluates to {@code false}, 
> entry will be skipped.
>  */
> public IgnitePredicate preloadPredicate();
> {code}
> This is internal cache preload predicate, which is not used and not tested 
> for entry preloading. Can be removed to keep code simple.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11981) [IEP-35] Create MU shortcut instead of static import of MetricUtils

2019-07-17 Thread Anton Vinogradov (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887662#comment-16887662
 ] 

Anton Vinogradov commented on IGNITE-11981:
---

[~NIzhikov] 
My proposal was not about MetricUtils -> MU, 
but about replacing static method import with explicit MetricUtils.method() 
call.

> [IEP-35] Create MU shortcut instead of static import of MetricUtils
> ---
>
> Key: IGNITE-11981
> URL: https://issues.apache.org/jira/browse/IGNITE-11981
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>
> More Ignite-way of coding is the usage of short cut classes.
> We should use MU instead of static import of {{MetricUtils}}.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11988) control.sh validate_indexes SQL Index issue add information about group and cache id

2019-07-17 Thread Ignite TC Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887660#comment-16887660
 ] 

Ignite TC Bot commented on IGNITE-11988:


{panel:title=--> Run :: All: Possible 
Blockers|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Cache 6{color} [[tests 0 TIMEOUT , Exit Code 
|https://ci.ignite.apache.org/viewLog.html?buildId=4343930]]

{color:#d04437}ZooKeeper (Discovery) 2{color} [[tests 
8|https://ci.ignite.apache.org/viewLog.html?buildId=4343900]]
* ZookeeperDiscoverySpiTestSuite2: internal.IgniteClientReconnectCacheTest

{color:#d04437}Platform C++ (Win x64 / Release){color} [[tests 1 
BuildFailureOnMessage 
|https://ci.ignite.apache.org/viewLog.html?buildId=4343902]]

{color:#d04437}MVCC Cache 4{color} [[tests 
1|https://ci.ignite.apache.org/viewLog.html?buildId=4343964]]
* IgniteCacheMvccTestSuite4: 
IgniteCacheConfigurationTemplateTest.testCreateFromTemplate

{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4343976&buildTypeId=IgniteTests24Java8_RunAll]

> control.sh validate_indexes SQL Index issue add information about group and 
> cache id
> 
>
> Key: IGNITE-11988
> URL: https://issues.apache.org/jira/browse/IGNITE-11988
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> At the moment we have the following output in case of SQL index problems:
> {noformat}
> SQL Index 
> [cache=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
>  idx=_key_PK] ValidateIndexesPartitionResult 
> [consistentId=10.116.241.93:47500, sqlIdxName=_key_PK]
>  IndexValidationIssue [key=678073218895971307, 
> cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
>  idxName=_key_PK], class org.apache.ignite.IgniteCheckedException: Key is 
> present in SQL index, but can't be found in CacheDataTree.
>  IndexValidationIssue [key=2495557143516676100, 
> cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
>  idxName=_key_PK], class org.apache.ignite.IgniteCheckedException: Key is 
> present in SQL index, but can't be found in CacheDataTree.
>  IndexValidationIssue [key=null, 
> cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
>  idxName=_key_PK], class java.lang.AssertionError: itemId=9, directCnt=9, 
> indirectCnt=0, page=000133230112 [3883, 3669, 3456, 3242, 3029, 2815, 
> 2602, 2386, 1747][][free=2101]
>  IndexValidationIssue [key=2760988046554825752, 
> cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
>  idxName=_key_PK], class org.apache.ignite.IgniteCheckedException: Key is 
> present in SQL index, but can't be found in CacheDataTree.
> {noformat}
> We print info about cache name only. 
> Now shoud add group and cache id.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-11995) control.sh if experimental command disabled - don't show help for experemental commands

2019-07-17 Thread Kirill Tkalenko (JIRA)
Kirill Tkalenko created IGNITE-11995:


 Summary: control.sh if experimental command disabled - don't show 
help for experemental commands
 Key: IGNITE-11995
 URL: https://issues.apache.org/jira/browse/IGNITE-11995
 Project: Ignite
  Issue Type: Improvement
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko
 Fix For: 2.8


If experimental command disabled:
 * don't show WALCommand help
 * if user ask for help for particular command - print out warning about 
experimental commands instead of ignoring user request



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Comment Edited] (IGNITE-11989) Preload predicate not used in GridCachePreloader

2019-07-17 Thread Pavel Pereslegin (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887275#comment-16887275
 ] 

Pavel Pereslegin edited comment on IGNITE-11989 at 7/17/19 5:54 PM:


[~Mmuzaf]
LGTM, 
But, I cannot find in {{git}} history the reasons why this feature was added 
and how it was originally used, so may be it worth to clarify this on the dev 
list.


was (Author: xtern):
[~Mmuzaf]
LGTM, 
I cannot find in {{git}} history the reasons why this feature was added and how 
it was originally used, so may be it worth to clarify this on the dev list.

> Preload predicate not used in GridCachePreloader
> 
>
> Key: IGNITE-11989
> URL: https://issues.apache.org/jira/browse/IGNITE-11989
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Minor
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {code:title=GridCachePreloader.java}
> /**
>  * @return Preload predicate. If not {@code null}, will evaluate each 
> preloaded entry during
>  *  send and receive, and if predicate evaluates to {@code false}, 
> entry will be skipped.
>  */
> public IgnitePredicate preloadPredicate();
> {code}
> This is internal cache preload predicate, which is not used and not tested 
> for entry preloading. Can be removed to keep code simple.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Comment Edited] (IGNITE-11989) Preload predicate not used in GridCachePreloader

2019-07-17 Thread Pavel Pereslegin (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887275#comment-16887275
 ] 

Pavel Pereslegin edited comment on IGNITE-11989 at 7/17/19 5:53 PM:


[~Mmuzaf]
LGTM, 
I cannot find in {{git}} history the reasons why this feature was added and how 
it was originally used, so may be it worth to clarify this on the dev list.


was (Author: xtern):
[~Mmuzaf]
LGTM, 
I cannot find in {{git}} history the reasons why this feature was added and how 
it was originally used.

> Preload predicate not used in GridCachePreloader
> 
>
> Key: IGNITE-11989
> URL: https://issues.apache.org/jira/browse/IGNITE-11989
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Minor
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {code:title=GridCachePreloader.java}
> /**
>  * @return Preload predicate. If not {@code null}, will evaluate each 
> preloaded entry during
>  *  send and receive, and if predicate evaluates to {@code false}, 
> entry will be skipped.
>  */
> public IgnitePredicate preloadPredicate();
> {code}
> This is internal cache preload predicate, which is not used and not tested 
> for entry preloading. Can be removed to keep code simple.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11989) Preload predicate not used in GridCachePreloader

2019-07-17 Thread Pavel Pereslegin (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887275#comment-16887275
 ] 

Pavel Pereslegin commented on IGNITE-11989:
---

[~Mmuzaf]
LGTM, 
I cannot find in {{git}} history the reasons why this feature was added and how 
it was originally used.

> Preload predicate not used in GridCachePreloader
> 
>
> Key: IGNITE-11989
> URL: https://issues.apache.org/jira/browse/IGNITE-11989
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Minor
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {code:title=GridCachePreloader.java}
> /**
>  * @return Preload predicate. If not {@code null}, will evaluate each 
> preloaded entry during
>  *  send and receive, and if predicate evaluates to {@code false}, 
> entry will be skipped.
>  */
> public IgnitePredicate preloadPredicate();
> {code}
> This is internal cache preload predicate, which is not used and not tested 
> for entry preloading. Can be removed to keep code simple.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11989) Preload predicate not used in GridCachePreloader

2019-07-17 Thread Maxim Muzafarov (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887244#comment-16887244
 ] 

Maxim Muzafarov commented on IGNITE-11989:
--

[~avinogradov], [~xtern]

Folks, can you review my changes?

> Preload predicate not used in GridCachePreloader
> 
>
> Key: IGNITE-11989
> URL: https://issues.apache.org/jira/browse/IGNITE-11989
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Minor
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {code:title=GridCachePreloader.java}
> /**
>  * @return Preload predicate. If not {@code null}, will evaluate each 
> preloaded entry during
>  *  send and receive, and if predicate evaluates to {@code false}, 
> entry will be skipped.
>  */
> public IgnitePredicate preloadPredicate();
> {code}
> This is internal cache preload predicate, which is not used and not tested 
> for entry preloading. Can be removed to keep code simple.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11989) Preload predicate not used in GridCachePreloader

2019-07-17 Thread Ignite TC Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887241#comment-16887241
 ] 

Ignite TC Bot commented on IGNITE-11989:


{panel:title=--> Run :: All: No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4343860&buildTypeId=IgniteTests24Java8_RunAll]

> Preload predicate not used in GridCachePreloader
> 
>
> Key: IGNITE-11989
> URL: https://issues.apache.org/jira/browse/IGNITE-11989
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Minor
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> {code:title=GridCachePreloader.java}
> /**
>  * @return Preload predicate. If not {@code null}, will evaluate each 
> preloaded entry during
>  *  send and receive, and if predicate evaluates to {@code false}, 
> entry will be skipped.
>  */
> public IgnitePredicate preloadPredicate();
> {code}
> This is internal cache preload predicate, which is not used and not tested 
> for entry preloading. Can be removed to keep code simple.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-11994) [TC Bot] Prepare new view to select base branch and other build parameters

2019-07-17 Thread Dmitriy Pavlov (JIRA)
Dmitriy Pavlov created IGNITE-11994:
---

 Summary: [TC Bot] Prepare new view to select base branch and other 
build parameters
 Key: IGNITE-11994
 URL: https://issues.apache.org/jira/browse/IGNITE-11994
 Project: Ignite
  Issue Type: Improvement
Reporter: Dmitriy Pavlov
Assignee: Dmitriy Pavlov


New view required for reports view and for VISAs creation for non-standard 
master branches,

additional features may be contributed to this new view later



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11992) Improvements for new security approach

2019-07-17 Thread Stepachev Maksim (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16887111#comment-16887111
 ] 

Stepachev Maksim commented on IGNITE-11992:
---

PR: https://github.com/apache/ignite/pull/6702/

> Improvements for new security approach
> --
>
> Key: IGNITE-11992
> URL: https://issues.apache.org/jira/browse/IGNITE-11992
> Project: Ignite
>  Issue Type: Improvement
>  Components: security
>Affects Versions: 2.8
>Reporter: Stepachev Maksim
>Assignee: Stepachev Maksim
>Priority: Major
> Fix For: 2.8
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> 1. ZookeaperDiscoveryImpl doesn't implement security into itself.
>  As a result: Caused by: class org.apache.ignite.spi.IgniteSpiException: 
> Security context isn't certain.
> 2. The visor tasks lost permission. 
>  The method VisorQueryUtils#scheduleQueryStart makes a new thread and loses 
> context.
> 3. The GridRestProcessor does tasks outside "withContext" section. As result 
> context loses.
> 4. The GridRestProcessor isn't client, we can't read security subject from 
> node attribute. 
>  We should transmit secCtx for fake nodes and secSubjId for real.
> 5. NoOpIgniteSecurityProcessor should include a disabled processor and 
> validate it too if it is not null. It is important for a client node. 
> For example:
> Into IgniteKernal#securityProcessor method createComponent return a 
> GridSecurityProcessor. For server nodes are enabled, but for clients aren't. 
> The clients aren't able to pass validation for this reason. 
> 6. ATTR_SECURITY_SUBJECT was removed. It broke compatibility.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-11993) Print warning if awaiting next wal segment it too long

2019-07-17 Thread Kirill Tkalenko (JIRA)
Kirill Tkalenko created IGNITE-11993:


 Summary: Print warning if awaiting next wal segment it too long
 Key: IGNITE-11993
 URL: https://issues.apache.org/jira/browse/IGNITE-11993
 Project: Ignite
  Issue Type: Improvement
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko
 Fix For: 2.8


We must print warn to log, if awaiting next WAL segment more then defined 
threshold.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-11992) Improvements for new security approach

2019-07-17 Thread Stepachev Maksim (JIRA)
Stepachev Maksim created IGNITE-11992:
-

 Summary: Improvements for new security approach
 Key: IGNITE-11992
 URL: https://issues.apache.org/jira/browse/IGNITE-11992
 Project: Ignite
  Issue Type: Improvement
  Components: security
Affects Versions: 2.8
Reporter: Stepachev Maksim
Assignee: Stepachev Maksim
 Fix For: 2.8


1. ZookeaperDiscoveryImpl doesn't implement security into itself.
 As a result: Caused by: class org.apache.ignite.spi.IgniteSpiException: 
Security context isn't certain.
2. The visor tasks lost permission. 
 The method VisorQueryUtils#scheduleQueryStart makes a new thread and loses 
context.
3. The GridRestProcessor does tasks outside "withContext" section. As result 
context loses.
4. The GridRestProcessor isn't client, we can't read security subject from node 
attribute. 
 We should transmit secCtx for fake nodes and secSubjId for real.
5. NoOpIgniteSecurityProcessor should include a disabled processor and validate 
it too if it is not null. It is important for a client node. 
For example:
Into IgniteKernal#securityProcessor method createComponent return a 
GridSecurityProcessor. For server nodes are enabled, but for clients aren't. 
The clients aren't able to pass validation for this reason. 
6. ATTR_SECURITY_SUBJECT was removed. It broke compatibility.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11979) Add ability to set default parallelizm of rebuild indexes in configuration

2019-07-17 Thread Denis Chudov (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886993#comment-16886993
 ] 

Denis Chudov commented on IGNITE-11979:
---

[~jooger], could you also review my changes, please?

> Add ability to set default parallelizm of rebuild indexes in configuration
> --
>
> Key: IGNITE-11979
> URL: https://issues.apache.org/jira/browse/IGNITE-11979
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Denis Chudov
>Assignee: Denis Chudov
>Priority: Major
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> We can't change SchemaIndexCacheVisitorImpl#DFLT_PARALLELISM at the moment:
> {code:java}
> /** Default degree of parallelism. */
> private static final int DFLT_PARALLELISM = Math.min(4, Math.max(1, 
> Runtime.getRuntime().availableProcessors() / 4));
> {code}
> On huge servers with a lot of cores (such as 56) we will rebuild indexes in 4 
> threads. I think we should have ability to set DFLT_PARALLELISM in Ignite 
> configuration.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-11991) [TC Bot]: Rerun possible blockers sets incorrect "SCALE FACTOR" build parameter

2019-07-17 Thread Pavel Kuznetsov (JIRA)
Pavel Kuznetsov created IGNITE-11991:


 Summary: [TC Bot]: Rerun possible blockers sets incorrect "SCALE 
FACTOR" build parameter
 Key: IGNITE-11991
 URL: https://issues.apache.org/jira/browse/IGNITE-11991
 Project: Ignite
  Issue Type: Improvement
Reporter: Pavel Kuznetsov


1) Issue "Trigger build" during PR check from the Bot UI. Scheduled build has 
parameter SCALE FACTOR=0.1 which is ok, because we want to get results sooner.
2) Issue "Rerun possible blockers" and see that SF for the "rerun" TC builds is 
1.0 (default).

Expected that SF of the rerun builds should be the same as in the initial 
RunAll build.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11986) Failed to deserialize object with given class loader: sun.misc.Launcher$AppClassLoader

2019-07-17 Thread JIRA


[ 
https://issues.apache.org/jira/browse/IGNITE-11986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886926#comment-16886926
 ] 

Jean-Denis Giguère commented on IGNITE-11986:
-

I can make it working by removing
$HADOOP_HOME/hadoop-3.1.2/share/hadoop/yarn/lib/geronimo-jcache_1.0_spec-1.0-alpha-1.jar

I'm not sure removing it is a valid use case for many. What could be an better 
way to manage this?

I update the repo with the sample code to provide a docker-compose environment 
to make it easier to reproduce. 



>  Failed to deserialize object with given class loader: 
> sun.misc.Launcher$AppClassLoader
> ---
>
> Key: IGNITE-11986
> URL: https://issues.apache.org/jira/browse/IGNITE-11986
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: mas
> Environment: Ignite master: commit 
> {{1a2c35caf805769ca4e3f169d7a5c72c31147e41}}
> spark 2.4.3
> hadoop 3.1.2
> OpenJDK 8
> scala 2.11.12
>  
>Reporter: Jean-Denis Giguère
>Priority: Major
> Attachments: server-not-ok.log, spark.log
>
>
> h1. Current situation
> Trying to create connect to a remote ignite cluster from {{spark-submit}}, I 
> get the error message given in the error log attached.
> See code snippet here : 
> https://github.com/jdenisgiguere/ignite_failed_unmarshal_discovery_data
> h2. Expected situation
> We shall be able to connect to a remote Ignite even when we are using Hadoop 
> 3.1.x. 
> h3. Steps to reproduce
> See: [https://github.com/jdenisgiguere/ignite_failed_unmarshal_discovery_data]



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-11990) Optimize heap usage for TcpDiscoveryNodeAddedMessage stored in pending messages in ServerImpl

2019-07-17 Thread Vladimir Malinovskiy (JIRA)
Vladimir Malinovskiy created IGNITE-11990:
-

 Summary: Optimize heap usage for TcpDiscoveryNodeAddedMessage 
stored in pending messages in ServerImpl
 Key: IGNITE-11990
 URL: https://issues.apache.org/jira/browse/IGNITE-11990
 Project: Ignite
  Issue Type: Improvement
Reporter: Vladimir Malinovskiy
Assignee: Vladimir Malinovskiy


We are storing pending discovery messages in deserialized form. Pending message 
could be heavy, for example TcpDiscoveryNodeAddedMessage. I think we should 
store only information requeired for resending messages across ring. In case of 
TcpDiscoveryNodeAddedMessage we couldn't store unmarhalled data in 
DiscoveryDataPacket



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-10761) GridCacheProcessor should add info about cache in excecption message, if applicable.

2019-07-17 Thread Kirill Tkalenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kirill Tkalenko updated IGNITE-10761:
-
Ignite Flags:   (was: Docs Required)

> GridCacheProcessor should add info about cache in excecption message, if 
> applicable. 
> -
>
> Key: IGNITE-10761
> URL: https://issues.apache.org/jira/browse/IGNITE-10761
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Sergey Antonov
>Assignee: Kirill Tkalenko
>Priority: Major
> Fix For: 2.8
>
>
> We should add info about problem cache in exception message. 



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-11988) control.sh validate_indexes SQL Index issue add information about group and cache id

2019-07-17 Thread Kirill Tkalenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kirill Tkalenko updated IGNITE-11988:
-
Description: 
At the moment we have the following output in case of SQL index problems:

{noformat}
SQL Index 
[cache=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
 idx=_key_PK] ValidateIndexesPartitionResult [consistentId=10.116.241.93:47500, 
sqlIdxName=_key_PK]
 IndexValidationIssue [key=678073218895971307, 
cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
 idxName=_key_PK], class org.apache.ignite.IgniteCheckedException: Key is 
present in SQL index, but can't be found in CacheDataTree.
 IndexValidationIssue [key=2495557143516676100, 
cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
 idxName=_key_PK], class org.apache.ignite.IgniteCheckedException: Key is 
present in SQL index, but can't be found in CacheDataTree.
 IndexValidationIssue [key=null, 
cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
 idxName=_key_PK], class java.lang.AssertionError: itemId=9, directCnt=9, 
indirectCnt=0, page=000133230112 [3883, 3669, 3456, 3242, 3029, 2815, 2602, 
2386, 1747][][free=2101]
 IndexValidationIssue [key=2760988046554825752, 
cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
 idxName=_key_PK], class org.apache.ignite.IgniteCheckedException: Key is 
present in SQL index, but can't be found in CacheDataTree.
{noformat}

We print info about cache name only. 
Now shoud add group and cache id.

> control.sh validate_indexes SQL Index issue add information about group and 
> cache id
> 
>
> Key: IGNITE-11988
> URL: https://issues.apache.org/jira/browse/IGNITE-11988
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
> Fix For: 2.8
>
>
> At the moment we have the following output in case of SQL index problems:
> {noformat}
> SQL Index 
> [cache=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
>  idx=_key_PK] ValidateIndexesPartitionResult 
> [consistentId=10.116.241.93:47500, sqlIdxName=_key_PK]
>  IndexValidationIssue [key=678073218895971307, 
> cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
>  idxName=_key_PK], class org.apache.ignite.IgniteCheckedException: Key is 
> present in SQL index, but can't be found in CacheDataTree.
>  IndexValidationIssue [key=2495557143516676100, 
> cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
>  idxName=_key_PK], class org.apache.ignite.IgniteCheckedException: Key is 
> present in SQL index, but can't be found in CacheDataTree.
>  IndexValidationIssue [key=null, 
> cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
>  idxName=_key_PK], class java.lang.AssertionError: itemId=9, directCnt=9, 
> indirectCnt=0, page=000133230112 [3883, 3669, 3456, 3242, 3029, 2815, 
> 2602, 2386, 1747][][free=2101]
>  IndexValidationIssue [key=2760988046554825752, 
> cacheName=com.sbt.processing.replication.dpl.data.ReplicationApplyStateV1Entity_DPL_union-module,
>  idxName=_key_PK], class org.apache.ignite.IgniteCheckedException: Key is 
> present in SQL index, but can't be found in CacheDataTree.
> {noformat}
> We print info about cache name only. 
> Now shoud add group and cache id.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-11989) Preload predicate not used in GridCachePreloader

2019-07-17 Thread Maxim Muzafarov (JIRA)
Maxim Muzafarov created IGNITE-11989:


 Summary: Preload predicate not used in GridCachePreloader
 Key: IGNITE-11989
 URL: https://issues.apache.org/jira/browse/IGNITE-11989
 Project: Ignite
  Issue Type: Improvement
Reporter: Maxim Muzafarov
Assignee: Maxim Muzafarov
 Fix For: 2.8


{code:title=GridCachePreloader.java}
/**
 * @return Preload predicate. If not {@code null}, will evaluate each 
preloaded entry during
 *  send and receive, and if predicate evaluates to {@code false}, 
entry will be skipped.
 */
public IgnitePredicate preloadPredicate();
{code}

This is internal cache preload predicate, which is not used and not tested for 
entry preloading. Can be removed to keep code simple.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-11988) control.sh validate_indexes SQL Index issue add information about group and cache id

2019-07-17 Thread Kirill Tkalenko (JIRA)
Kirill Tkalenko created IGNITE-11988:


 Summary: control.sh validate_indexes SQL Index issue add 
information about group and cache id
 Key: IGNITE-11988
 URL: https://issues.apache.org/jira/browse/IGNITE-11988
 Project: Ignite
  Issue Type: Improvement
Reporter: Kirill Tkalenko
Assignee: Kirill Tkalenko
 Fix For: 2.8






--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Assigned] (IGNITE-10761) GridCacheProcessor should add info about cache in excecption message, if applicable.

2019-07-17 Thread Kirill Tkalenko (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-10761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kirill Tkalenko reassigned IGNITE-10761:


Assignee: Kirill Tkalenko

> GridCacheProcessor should add info about cache in excecption message, if 
> applicable. 
> -
>
> Key: IGNITE-10761
> URL: https://issues.apache.org/jira/browse/IGNITE-10761
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Sergey Antonov
>Assignee: Kirill Tkalenko
>Priority: Major
> Fix For: 2.8
>
>
> We should add info about problem cache in exception message. 



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Resolved] (IGNITE-11953) BTree corruption caused by byte array values

2019-07-17 Thread Dmitriy Govorukhin (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dmitriy Govorukhin resolved IGNITE-11953.
-
Resolution: Fixed

Fixed in IGNITE-11982

> BTree corruption caused by byte array values
> 
>
> Key: IGNITE-11953
> URL: https://issues.apache.org/jira/browse/IGNITE-11953
> Project: Ignite
>  Issue Type: Bug
>Reporter: Dmitriy Govorukhin
>Assignee: Dmitriy Govorukhin
>Priority: Major
> Fix For: 2.8
>
>
> In some cases for caches with cache group, we can get BTree corruption 
> exception.
> {code}
> 09:53:58,890][SEVERE][sys-stripe-10-#11][] Critical system error detected. 
> Will be handled accordingly to configured handler [hnd=CustomFailureHandler 
> [ignoreCriticalErrors=false, disabled=false][StopNodeOrHaltFailureHandler 
> [tryStop=false, timeout=0]], failureCtx=FailureContext [type=CRITICAL_ERROR, 
> err=class o.a.i.i.transactions.IgniteTxHeuristicCheckedException: Committing 
> a transaction has produced runtime exception]]class 
> org.apache.ignite.internal.transactions.IgniteTxHeuristicCheckedException: 
> Committing a transaction has produced runtime exception
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter.heuristicException(IgniteTxAdapter.java:800)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:922)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:799)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.localFinish(GridDhtTxLocal.java:608)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.finishTx(GridDhtTxLocal.java:478)
>   at 
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.commitDhtLocalAsync(GridDhtTxLocal.java:535)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finishDhtLocal(IgniteTxHandler.java:1055)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.finish(IgniteTxHandler.java:931)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxFinishRequest(IgniteTxHandler.java:887)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$200(IgniteTxHandler.java:117)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:209)
>   at 
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$3.apply(IgniteTxHandler.java:207)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1129)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:594)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:393)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:319)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:109)
>   at 
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:308)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1568)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1196)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
>   at 
> org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1092)
>   at 
> org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:504)
>   at 
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
>   at java.lang.Thread.run(Thread.java:748)
> Caused by: class 
> org.apache.ignite.internal.processors.cache.persistence.tree.CorruptedTreeException:
>  Runtime failure on search row: SearchRow [key=KeyCacheObjectImpl [part=427, 
> val=Grkg1DUF3yQE6tC9Se50mi5w.T, hasValBytes=true], hash=1872857770, 
> cacheId=-420893003]
>   at 
> org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.invoke(BPlusTree.java:1811)
>   at 
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke0(IgniteCacheOffheapManagerImpl.java:1620)
>   at 
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManage

[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-07-17 Thread Nikolay Izhikov (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886780#comment-16886780
 ] 

Nikolay Izhikov commented on IGNITE-11927:
--

[~agura]

Configuration code removed from this PR.
Please, review.

> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Comment Edited] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-07-17 Thread Nikolay Izhikov (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886762#comment-16886762
 ] 

Nikolay Izhikov edited comment on IGNITE-11927 at 7/17/19 8:28 AM:
---

Hello, Andrey.

> 2. We enable or disable metrics for one source of metrics. So we can just 
> remove MetricRegistry from GridMetricManager when metrics disabled.
 > 3. There is no need for disabled flag in MetricRegistry. As we discused 
 > early, when metric disabled they don't consume any memory. MetricsRegistry 
 > should be collected by GC. After enabling it can be created and registered 
 > into GridMetricManager again.

I don't understand your proposal.

Metric instances are stored as a class field in the places where they updated.
You can take a {{GridJobProcessor}} or {{CacheMetricImpl}} as examples.

How and why we should clear these variables on disabling?
Can you provide simple pseudo code for disable\enable processing.

> 4. Please, separate buckets configuration and metrics enabling/disabling into 
> two different tickets.

OK

> 5. Please, create review in Upsource.

https://reviews.ignite.apache.org/ignite/review/IGNT-CR-1061

> 6. Please, run TC and take a look to results before code review.

https://ci.ignite.apache.org/viewQueued.html?itemId=4343158


was (Author: nizhikov):
Hello, Andrey.

> 2. We enable or disable metrics for one source of metrics. So we can just 
> remove MetricRegistry from GridMetricManager when metrics disabled.
 > 3. There is no need for disabled flag in MetricRegistry. As we discused 
 > early, when metric disabled they don't consume any memory. MetricsRegistry 
 > should be collected by GC. After enabling it can be created and registered 
 > into GridMetricManager again.

I don't understand your proposal.

Metric instances are stored as a class field in the places where they updated.
You can take a {{GridJobProcessor}} or {{CacheMetricImpl}} as examples.

How and why we should clear these variables on disabling?
Can you provide simple pseudo code for disable\enable processing.

> 4. Please, separate buckets configuration and metrics enabling/disabling into 
> two different tickets.

OK

> 5. Please, create review in Upsource.

https://reviews.ignite.apache.org/ignite/review/IGNT-CR-1061

> 6. Please, run TC and take a look to results before code review.

In progress: https://ci.ignite.apache.org/viewLog.html?buildId=4343042&;

> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Created] (IGNITE-11987) [IEP-35] Add ability to configure metrics

2019-07-17 Thread Nikolay Izhikov (JIRA)
Nikolay Izhikov created IGNITE-11987:


 Summary: [IEP-35] Add ability to configure metrics
 Key: IGNITE-11987
 URL: https://issues.apache.org/jira/browse/IGNITE-11987
 Project: Ignite
  Issue Type: Improvement
Reporter: Nikolay Izhikov
Assignee: Nikolay Izhikov


Ignite should be able to:

* Enable or disable an arbitrary subset of the metrics. User should be able to 
do it in runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-11987) [IEP-35] Add ability to configure metrics

2019-07-17 Thread Nikolay Izhikov (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikolay Izhikov updated IGNITE-11987:
-
Description: 
Ignite should be able to:

* Configure Histogram metrics
* Configure HitRate metrics.

We should provide 2 ways to configure metric:

1. -Configuration file.- Discussed on dev-list. Agreed to go with the simplest 
solution - JMX method.
2. JMX method.

  was:
Ignite should be able to:

* Enable or disable an arbitrary subset of the metrics. User should be able to 
do it in runtime.


> [IEP-35] Add ability to configure metrics
> -
>
> Key: IGNITE-11987
> URL: https://issues.apache.org/jira/browse/IGNITE-11987
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>
> Ignite should be able to:
> * Configure Histogram metrics
> * Configure HitRate metrics.
> We should provide 2 ways to configure metric:
> 1. -Configuration file.- Discussed on dev-list. Agreed to go with the 
> simplest solution - JMX method.
> 2. JMX method.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-07-17 Thread Nikolay Izhikov (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikolay Izhikov updated IGNITE-11927:
-
Description: 
Ignite should be able to:

* Enable or disable an arbitrary subset of the metrics. User should be able to 
do it in runtime.

  was:
Ignite should be able to:

* Enable or disable an arbitrary subset of the metrics. User should be able to 
do it in runtime.
* Configure Histogram metrics
* Configure HitRate metrics.

We should provide 2 ways to configure metric:

1. -Configuration file.- Discussed on dev-list. Agreed to go with the simplest 
solution - JMX method.
2. JMX method.


> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Updated] (IGNITE-11927) [IEP-35] Add ability to enable\disable subset of metrics

2019-07-17 Thread Nikolay Izhikov (JIRA)


 [ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nikolay Izhikov updated IGNITE-11927:
-
Summary: [IEP-35] Add ability to enable\disable subset of metrics  (was: 
[IEP-35] Add ability to configure subset of metrics)

> [IEP-35] Add ability to enable\disable subset of metrics
> 
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.
> * Configure Histogram metrics
> * Configure HitRate metrics.
> We should provide 2 ways to configure metric:
> 1. -Configuration file.- Discussed on dev-list. Agreed to go with the 
> simplest solution - JMX method.
> 2. JMX method.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11927) [IEP-35] Add ability to configure subset of metrics

2019-07-17 Thread Nikolay Izhikov (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886762#comment-16886762
 ] 

Nikolay Izhikov commented on IGNITE-11927:
--

Hello, Andrey.

> 2. We enable or disable metrics for one source of metrics. So we can just 
> remove MetricRegistry from GridMetricManager when metrics disabled.
 > 3. There is no need for disabled flag in MetricRegistry. As we discused 
 > early, when metric disabled they don't consume any memory. MetricsRegistry 
 > should be collected by GC. After enabling it can be created and registered 
 > into GridMetricManager again.

I don't understand your proposal.

Metric instances are stored as a class field in the places where they updated.
You can take a {{GridJobProcessor}} or {{CacheMetricImpl}} as examples.

How and why we should clear these variables on disabling?
Can you provide simple pseudo code for disable\enable processing.

> 4. Please, separate buckets configuration and metrics enabling/disabling into 
> two different tickets.

OK

> 5. Please, create review in Upsource.

https://reviews.ignite.apache.org/ignite/review/IGNT-CR-1061

> 6. Please, run TC and take a look to results before code review.

In progress: https://ci.ignite.apache.org/viewLog.html?buildId=4343042&;

> [IEP-35] Add ability to configure subset of metrics
> ---
>
> Key: IGNITE-11927
> URL: https://issues.apache.org/jira/browse/IGNITE-11927
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Nikolay Izhikov
>Priority: Major
>  Labels: IEP-35
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Ignite should be able to:
> * Enable or disable an arbitrary subset of the metrics. User should be able 
> to do it in runtime.
> * Configure Histogram metrics
> * Configure HitRate metrics.
> We should provide 2 ways to configure metric:
> 1. -Configuration file.- Discussed on dev-list. Agreed to go with the 
> simplest solution - JMX method.
> 2. JMX method.



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)


[jira] [Commented] (IGNITE-11966) Using AdaptiveLoadBalancingSpi without IgniteConfiguration.setIncludeEventTypes(EventType.EVT_TASK_FINISHED, EventType.EVT_TASK_FAILED) leads to memory leak

2019-07-17 Thread Ignite TC Bot (JIRA)


[ 
https://issues.apache.org/jira/browse/IGNITE-11966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16886751#comment-16886751
 ] 

Ignite TC Bot commented on IGNITE-11966:


{panel:title=--> Run :: All: No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
[TeamCity *--> Run :: All* 
Results|https://ci.ignite.apache.org/viewLog.html?buildId=4337709&buildTypeId=IgniteTests24Java8_RunAll]

> Using AdaptiveLoadBalancingSpi without 
> IgniteConfiguration.setIncludeEventTypes(EventType.EVT_TASK_FINISHED,  
> EventType.EVT_TASK_FAILED) leads to memory leak
> -
>
> Key: IGNITE-11966
> URL: https://issues.apache.org/jira/browse/IGNITE-11966
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.7.5
>Reporter: Andrey Aleksandrov
>Assignee: Ivan Bessonov
>Priority: Major
> Fix For: 2.8
>
> Attachments: Top_Consumers.zip
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> You can use in debug mode any of tests related to AdaptiveLoadBalancingSpi 
> like GridAdaptiveLoadBalancingSpiSelfTest.
> In case if you don't subscribe on EventType.EVT_TASK_FINISHED,  
> EventType.EVT_TASK_FAILED then next code will be never called:
> case EVT_TASK_FINISHED:
> case EVT_TASK_FAILED: {
>  TaskEvent taskEvt = (TaskEvent)evt;
>  taskTops.remove(taskEvt.taskSessionId()); //this
> As a result, you will see the growth of memory usage.
> [^Top_Consumers.zip]
>  if (log.isDebugEnabled())
>  log.debug("Removed task topology from topology cache for session: " +
>  taskEvt.taskSessionId());
>  break;
> }



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)