Re: Should we enable write throttling by default?

[Dmitry Pavlov]

Yes, it is correct.

But still, right after implementation I was for enabling this mode by
detault.
- It is not simple to explain need to enable this feature to each user.
- Moreover write freezing after peak load may be considered by some Ignite
users as bug. Not all of them will write to user list.

чт, 16 нояб. 2017 г. в 10:47, Vladimir Ozerov :

> AFAIK throttling may decrease peak throughput.
>
> Alex, Ivan,
> Is it correct?
>
> On Thu, Nov 16, 2017 at 1:12 AM, Dmitriy Setrakyan 
> wrote:
>
> > Thanks Denis, looks very nice!
> >
> > Igniters, let's stop asking users to enable flags in order for the system
> > to behave properly. All such flags should be enabled by default.
> >
> > D.
> >
> > On Wed, Nov 15, 2017 at 11:26 AM, Denis Magda  wrote:
> >
> > > Igniters,
> > >
> > > I finished documenting pages write throttling and checkpointing buffers
> > > adjustment sections under the performance notes:
> > > https://apacheignite.readme.io/docs/durable-memory-tuning#
> > > section-pages-writes-throttling
> > > https://apacheignite.readme.io/docs/durable-memory-tuning#
> > > section-checkpointing-buffer-size
> > >
> > > Is there any reason why the write throttling shouldn’t be enabled by
> > > default? I couldn’t come with any negative effect.
> > >
> > > —
> > > Denis
> > >
> > >
> >
>

Re: Should we enable write throttling by default?

[Vladimir Ozerov]

AFAIK throttling may decrease peak throughput.

Alex, Ivan,
Is it correct?

On Thu, Nov 16, 2017 at 1:12 AM, Dmitriy Setrakyan 
wrote:

> Thanks Denis, looks very nice!
>
> Igniters, let's stop asking users to enable flags in order for the system
> to behave properly. All such flags should be enabled by default.
>
> D.
>
> On Wed, Nov 15, 2017 at 11:26 AM, Denis Magda  wrote:
>
> > Igniters,
> >
> > I finished documenting pages write throttling and checkpointing buffers
> > adjustment sections under the performance notes:
> > https://apacheignite.readme.io/docs/durable-memory-tuning#
> > section-pages-writes-throttling
> > https://apacheignite.readme.io/docs/durable-memory-tuning#
> > section-checkpointing-buffer-size
> >
> > Is there any reason why the write throttling shouldn’t be enabled by
> > default? I couldn’t come with any negative effect.
> >
> > —
> > Denis
> >
> >
>

[jira] [Created] (IGNITE-6927) Web Console: Create build with maven for Web Console direct-install.

[Andrey Novikov (JIRA)]

Andrey Novikov created IGNITE-6927:
--

 Summary: Web Console: Create build with maven for Web Console 
direct-install.
 Key: IGNITE-6927
 URL: https://issues.apache.org/jira/browse/IGNITE-6927
 Project: Ignite
  Issue Type: Sub-task
  Security Level: Public (Viewable by anyone)
  Components: wizards
Affects Versions: 2.0
Reporter: Andrey Novikov
Priority: Minor
 Fix For: 2.4


* Add build to maven for Web Console direct-install.
* Add section with instructions into DEVNOTES.txt



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6926) Web console: SimpleWorkerPool select wrong worker in special case

[Alexey Kuznetsov (JIRA)]

Alexey Kuznetsov created IGNITE-6926:


 Summary: Web console: SimpleWorkerPool select wrong worker in 
special case
 Key: IGNITE-6926
 URL: https://issues.apache.org/jira/browse/IGNITE-6926
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: wizards
Affects Versions: 2.3
Reporter: Alexey Kuznetsov
Assignee: Alexey Kuznetsov
 Fix For: 2.4


In case when several tasks submitted at once to newly created pool it has a bug 
of selecting next available worked.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Ignite Enhancement Proposal #6 (Metrics)

[Denis Magda]

Ok, closed the previously existed as a duplicate and add another usability 
issue to fix in the scope of this IEP:
https://issues.apache.org/jira/browse/IGNITE-6925 


—
Denis

> On Nov 15, 2017, at 9:21 AM, Anton Vinogradov  
> wrote:
> 
> Denis,
> 
> It looks like [1] is a duplicate of [2] and [3]
> 
> [1] https://issues.apache.org/jira/browse/IGNITE-5796
> [2] https://issues.apache.org/jira/browse/IGNITE-6903
> [3] https://issues.apache.org/jira/browse/IGNITE-6902
> 
> Please close issue as duplicate in case that's true.
> 
> On Wed, Nov 15, 2017 at 11:18 AM, Vladimir Ozerov 
> wrote:
> 
>> Dima,
>> 
>> This appears to be a INFRA's bug. I filed a ticket [1].
>> 
>> [1] https://issues.apache.org/jira/browse/INFRA-15487
>> 
>> On Tue, Nov 14, 2017 at 9:25 PM, Dmitriy Setrakyan 
>> wrote:
>> 
>>> On Tue, Nov 14, 2017 at 10:12 AM, Anton Vinogradov <
>>> avinogra...@gridgain.com
 wrote:
>>> 
 Dmitriy,
 
 It looks like a confluence bug.
 Please login and push refresh button at issues list.
 
>>> 
>>> Works now. It is unfortunate that it does not work for the community
>>> members who do not login. Perhaps we can provide a link to the Jira
>> filter,
>>> so folks could click on it and view the tickets.
>>> 
>> 

[jira] [Created] (IGNITE-6925) Simplify cache metrics activation

[Denis Magda (JIRA)]

Denis Magda created IGNITE-6925:
---

 Summary: Simplify cache metrics activation
 Key: IGNITE-6925
 URL: https://issues.apache.org/jira/browse/IGNITE-6925
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
Reporter: Denis Magda


The user needs to do 3 things to enabled cache metrics:
- set {{statisticsEnabled}} to {{true}}.
- set not a dummy {{EventsStorageSpi}}
- list metrics of the interest.

This process has to be reduced to 2 steps or, preferably, to 1.

More details are here: 
http://apache-ignite-developers.2346864.n4.nabble.com/Annoying-extra-steps-for-enabling-metrics-td21865.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: request for contributors permissions

[Denis Magda]

Hello Denis,

Seems that someone has already given your the permission and forgot to reply 
here.

—
Denis

> On Nov 15, 2017, at 4:08 AM, Denis Loginov  wrote:
> 
> Hello!
> 
> I want to resolve bugs using my account: da.loginov
> Please, provide me with necessary permissions.
> My GutHub account: Denis-Log
> 
> Thanks in advance.
> 
> 
> Best regards,
> Denis

Re: Request for Contributers permission

[Denis Magda]

Hello Turik,

Done!

Some extra info that might be useful for you.

Get familiar with Ignite development process described here:
https://cwiki.apache.org/confluence/display/IGNITE/Development+Process

Instructions on how to contribute can be found here:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute

Project setup in Intellij IDEA:
https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup

—
Denis

> On Nov 14, 2017, at 6:14 PM, techbysample  wrote:
> 
> 
> Ignite Community,
> 
> Would you please provide me with contributor permissions? 
> 
> JIRA account: netmille
> 
> Best,
> Turik
> 
> 
> 
> 
> --
> Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/

Re: Request for contributors permissions

[Denis Magda]

Hi Michael,

Ready!

Done!

Some extra info that might be useful for you.

Get familiar with Ignite development process described here:
https://cwiki.apache.org/confluence/display/IGNITE/Development+Process

Instructions on how to contribute can be found here:
https://cwiki.apache.org/confluence/display/IGNITE/How+to+Contribute

Project setup in Intellij IDEA:
https://cwiki.apache.org/confluence/display/IGNITE/Project+Setup 


—
Denis

> On Nov 15, 2017, at 3:08 AM, Michael Zhuravlev  wrote:
> 
> Hello!
> I want to resolve bugs. Please, grant me permission for assign tasks in
> jira. My jira account MikeZhur.
> 
> Regards,
> Mike Zhuravlev

Re: Why isn't Apache Ignite on stackshare.io

[Denis Magda]

Interesting source. Will look into it in a couple of days.

Thanks for sharing.

—
Denis

> On Nov 15, 2017, at 8:23 AM, Dmitriy Setrakyan  wrote:
> 
> Hi Jason, great point!
> 
> Denis, what do you think? Here is a link to how you can add Ignite to the
> Stackshare:
> https://stackshare.io/stacks
> 
> Jason, perhaps you can be our first customer testimonial there :)
> 
> D.
> 
> 
> On Wed, Nov 15, 2017 at 1:50 AM, Jason Man, CLSA  wrote:
> 
>> Just curious, why isn't Apache Ignite on Stackshare.io?  Stackshare.io is
>> really picking up in momentum as a developer community for sharing what
>> stacks are used and where.  Could be useful in promoting usage in the
>> open-source community.
>> 
>> I can see that Hazelcast is:
>> https://stackshare.io/hazelcast
>> 
>> Regards,
>> Jason
>> 
>> The content of this communication is intended for the recipient and is
>> subject to CLSA Legal and Regulatory Notices.
>> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
>> you upon request.
>> CLSA is ISO14001 certified and committed to reducing environmental impact.
>> 

[GitHub] ignite pull request #3040: ignite-6924: Fixed missed CacheStoreSessionListen...

[sk0x50]

GitHub user sk0x50 opened a pull request:

https://github.com/apache/ignite/pull/3040

ignite-6924: Fixed missed CacheStoreSessionListener#onSessionStart() call

CacheStoreSessionListener#onSessionStart() is not called in case of 
'WriteBehind' mode is enabled and 'writeCache' size exceeds critical size.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/sk0x50/ignite ignite-6924

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3040.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3040


commit 6fd1695ee3a228915230d6478e1a8db98ac9b974
Author: Slava Koptilin 
Date:   2017-11-15T22:20:14Z

ignite-6924: Fixed missed CacheStoreSessionListener#onSessionStart() call




---

Re: Should we enable write throttling by default?

[Dmitriy Setrakyan]

Thanks Denis, looks very nice!

Igniters, let's stop asking users to enable flags in order for the system
to behave properly. All such flags should be enabled by default.

D.

On Wed, Nov 15, 2017 at 11:26 AM, Denis Magda  wrote:

> Igniters,
>
> I finished documenting pages write throttling and checkpointing buffers
> adjustment sections under the performance notes:
> https://apacheignite.readme.io/docs/durable-memory-tuning#
> section-pages-writes-throttling
> https://apacheignite.readme.io/docs/durable-memory-tuning#
> section-checkpointing-buffer-size
>
> Is there any reason why the write throttling shouldn’t be enabled by
> default? I couldn’t come with any negative effect.
>
> —
> Denis
>
>

Should we enable write throttling by default?

[Denis Magda]

Igniters,

I finished documenting pages write throttling and checkpointing buffers 
adjustment sections under the performance notes:
https://apacheignite.readme.io/docs/durable-memory-tuning#section-pages-writes-throttling
https://apacheignite.readme.io/docs/durable-memory-tuning#section-checkpointing-buffer-size

Is there any reason why the write throttling shouldn’t be enabled by default? I 
couldn’t come with any negative effect.

—
Denis

[GitHub] ignite pull request #3000: ignite-6767: reset of the topVer on loading of th...

[sk0x50]

Github user sk0x50 closed the pull request at:

https://github.com/apache/ignite/pull/3000


---

[jira] [Created] (IGNITE-6924) CacheStoreSessionListener#onSessionStart() is not called in case of 'WriteBehind' mode is enabled and 'writeCache' size exceeds critical size.

[Vyacheslav Koptilin (JIRA)]

Vyacheslav Koptilin created IGNITE-6924:
---

 Summary: CacheStoreSessionListener#onSessionStart() is not called 
in case of 'WriteBehind' mode is enabled and 'writeCache' size exceeds critical 
size.
 Key: IGNITE-6924
 URL: https://issues.apache.org/jira/browse/IGNITE-6924
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: cache
Affects Versions: 2.4
Reporter: Vyacheslav Koptilin
Assignee: Vyacheslav Koptilin






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Ignite Enhancement Proposal #6 (Metrics)

[Anton Vinogradov]

Denis,

It looks like [1] is a duplicate of [2] and [3]

[1] https://issues.apache.org/jira/browse/IGNITE-5796
[2] https://issues.apache.org/jira/browse/IGNITE-6903
[3] https://issues.apache.org/jira/browse/IGNITE-6902

Please close issue as duplicate in case that's true.

On Wed, Nov 15, 2017 at 11:18 AM, Vladimir Ozerov 
wrote:

> Dima,
>
> This appears to be a INFRA's bug. I filed a ticket [1].
>
> [1] https://issues.apache.org/jira/browse/INFRA-15487
>
> On Tue, Nov 14, 2017 at 9:25 PM, Dmitriy Setrakyan 
> wrote:
>
> > On Tue, Nov 14, 2017 at 10:12 AM, Anton Vinogradov <
> > avinogra...@gridgain.com
> > > wrote:
> >
> > > Dmitriy,
> > >
> > > It looks like a confluence bug.
> > > Please login and push refresh button at issues list.
> > >
> >
> > Works now. It is unfortunate that it does not work for the community
> > members who do not login. Perhaps we can provide a link to the Jira
> filter,
> > so folks could click on it and view the tickets.
> >
>

Re: collocated compute and off-heap cache

[Dmitriy Setrakyan]

On Wed, Nov 15, 2017 at 8:38 AM, Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:

> Dmitriy,
>
> There will be no performance overhead for reads, but there will be a
> significant performance overhead for writes because each update must be
> changed in offheap, and since the offheap will be very small, there will be
> a lot of pages reads and spills.
>

Sounds very strange. Why do we need to read a page if we don't need to read
the entry? Moreover, we do not even need to write into that page, as far as
I know. Are you sure there is no room for optimization here?

Re: collocated compute and off-heap cache

[Alexey Goncharuk]

Dmitriy,

There will be no performance overhead for reads, but there will be a
significant performance overhead for writes because each update must be
changed in offheap, and since the offheap will be very small, there will be
a lot of pages reads and spills.

2017-11-15 6:53 GMT+03:00 Dmitriy Setrakyan :

> Igniters,
>
> I am noticing that some users struggle with performance when using
> collocated compute and off-heap cache. The main reason is that collocated
> computations access data locally, directly on the server, and with off-heap
> cache all binary objects need to be deserialized for every access.
>
> A much more efficient approach is to enable on-heap cache, so the data is
> deserialized once and then stored in the on-heap cache. However, the
> disadvantage of this approach is that the data foot print in memory doubles
> because the data is now stores in both, on-heap and off-heap caches.
>
> What if we suggested the following configuration for the collocated compute
> users:
>
>- enable the *on-heap cache* and make it large enough to fit all the
> data
>- enable *Ignite native persistence* in BACKGROUND mode, so there is no
>performance overhead for persisting data
>- make *off-heap cache* very small compared to the on-heap cache.
>
> This way, the memory will be consumed by the on-heap cache mostly, the data
> will be cached in deserialized form, and there should be no performance
> degradation.
>
> Will this approach work?
>
> D.
>

Re: Why isn't Apache Ignite on stackshare.io

[Dmitriy Setrakyan]

Hi Jason, great point!

Denis, what do you think? Here is a link to how you can add Ignite to the
Stackshare:
https://stackshare.io/stacks

Jason, perhaps you can be our first customer testimonial there :)

D.


On Wed, Nov 15, 2017 at 1:50 AM, Jason Man, CLSA  wrote:

> Just curious, why isn't Apache Ignite on Stackshare.io?  Stackshare.io is
> really picking up in momentum as a developer community for sharing what
> stacks are used and where.  Could be useful in promoting usage in the
> open-source community.
>
> I can see that Hazelcast is:
> https://stackshare.io/hazelcast
>
> Regards,
> Jason
>
> The content of this communication is intended for the recipient and is
> subject to CLSA Legal and Regulatory Notices.
> These can be viewed at https://www.clsa.com/disclaimer.html or sent to
> you upon request.
> CLSA is ISO14001 certified and committed to reducing environmental impact.
>

request for contributors permissions

[Denis Loginov]

Hello!

I want to resolve bugs using my account: da.loginov
Please, provide me with necessary permissions.
My GutHub account: Denis-Log

Thanks in advance.


Best regards,
Denis

[jira] [Created] (IGNITE-6923) Cache metrics are updated in timeout-worker potentially delaying critical code execution due to current implementation issues.

[Alexei Scherbakov (JIRA)]

Alexei Scherbakov created IGNITE-6923:
-

 Summary: Cache metrics are updated in timeout-worker potentially 
delaying critical code execution due to current implementation issues.
 Key: IGNITE-6923
 URL: https://issues.apache.org/jira/browse/IGNITE-6923
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
Affects Versions: 2.3
Reporter: Alexei Scherbakov
 Fix For: 2.4


Some metrics are using full cache iteration for calculation.

See stack trace for example.

{noformat}
"grid-timeout-worker-#39%DPL_GRID%DplGridNodeName%" #152 prio=5 os_prio=0 
tid=0x7f1009a03000 nid=0x5caa runnable [0x7f0f059d9000] 
   java.lang.Thread.State: RUNNABLE 
at java.util.HashMap.containsKey(HashMap.java:595) 
at java.util.HashSet.contains(HashSet.java:203) 
at 
java.util.Collections$UnmodifiableCollection.contains(Collections.java:1032) 
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$3.apply(IgniteCacheOffheapManagerImpl.java:339)
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$3.apply(IgniteCacheOffheapManagerImpl.java:337)
at 
org.apache.ignite.internal.util.lang.gridfunc.TransformFilteringIterator.hasNext:@TransformFilteringIterator.java:90)
at 
org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45)
 
at 
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.cacheEntriesCount(IgniteCacheOffheapManagerImpl.java:293)
at 
org.apache.ignite.internal.processors.cache.CacheMetricsImpl.getOffHeapPrimaryEntriesCount(CacheMetricsImpl.java:240)
at 
org.apache.ignite.internal.processors.cache.CacheMetricsSnapshot.(CacheMetricsSnapshot.java:271)
 
at 
org.apache.ignite.internal.processors.cache.GridCacheAdapter.localMetrics(GridCacheAdapter.java:3217)
 
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$7.cacheMetrics(GridDiscoveryManager.java:1151)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$7.nonHeapMemoryUsed(GridDiscoveryManager.java:1121)
at 
org.apache.ignite.internal.managers.discovery.GridDiscoveryManager$7.metrics(GridDiscoveryManager.java:1087)
 
at 
org.apache.ignite.spi.discovery.tcp.internal.TcpDiscoveryNode.metrics(TcpDiscoveryNode.java:269)
 
at 
org.apache.ignite.internal.IgniteKernal$3.run(IgniteKernal.java:1175) 
at 
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$CancelableTask.onTimeout(GridTimeoutProcessor.java:256)
- locked <0x7f115f5bf890> (a 
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$CancelableTask)
at 
org.apache.ignite.internal.processors.timeout.GridTimeoutProcessor$TimeoutWorker.body(GridTimeoutProcessor.java:158)
at 
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110) 
at java.lang.Thread.run(Thread.java:748)
{noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6922) Class can not undeploy from grid in some specific cases

[Vladislav Pyatkov (JIRA)]

Vladislav Pyatkov created IGNITE-6922:
-

 Summary: Class can not undeploy from grid in some specific cases
 Key: IGNITE-6922
 URL: https://issues.apache.org/jira/browse/IGNITE-6922
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
Reporter: Vladislav Pyatkov






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #3039: IGNITE-6911 .NET: Optionally disable Java console...

[ptupitsyn]

GitHub user ptupitsyn opened a pull request:

https://github.com/apache/ignite/pull/3039

IGNITE-6911 .NET: Optionally disable Java console redirect



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6911

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3039.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3039


commit 9f84958da67459d1bc3faeb784ceb065397565dd
Author: Pavel Tupitsyn 
Date:   2017-11-15T13:35:33Z

IGNITE-6911 .NET: Optionally disable Java console redirect




---

[GitHub] ignite pull request #3038: Ignite gg 13060

[dkarachentsev]

GitHub user dkarachentsev opened a pull request:

https://github.com/apache/ignite/pull/3038

Ignite gg 13060



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-gg-13060

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3038.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3038


commit 5ac9afc719138e37a7d97d9d9db05243eee9a942
Author: Evgenii Zhuravlev 
Date:   2017-06-22T09:36:14Z

IGNITE-5399 add test to testsuite

commit a935d40a80e2f928a84a145aba540a45b156687f
Author: Evgenii Zhuravlev 
Date:   2017-06-22T12:10:32Z

GG-12256 Minor fixes

commit 7e2468770a4eb47a4f61204d8c2000b6ab67c967
Author: nikolay_tikhonov 
Date:   2017-06-22T13:13:01Z

IGNITE-GG-12197 Fixed "Ignore events for discarded update in CLOCK mode".

Signed-off-by: nikolay_tikhonov 

commit 5858efd406bb54352de14a0a7e7f21c2ac7bf899
Author: sboikov 
Date:   2016-12-16T16:23:29Z

IGNITE-2412 - Do not acquire asyncSemaphore for synchronous operations 
(cherry-picked from master)

(cherry picked from commit 82b4073)

commit 113a1380da34ea804d68757d39926da97dee09b6
Author: Alexey Goncharuk 
Date:   2017-06-13T05:20:22Z

GG-12355: Backported IO latency test.

commit 540ca449f1bd2386d3ba0722afb21dd3a504d044
Author: Alexey Goncharuk 
Date:   2017-06-13T17:55:38Z

GG-12355: Added discovery ring latency test + made it available from MBean 
(cherry-picked from master).

commit 0fc6271d8e39125bf5ee341e50a2665a04fc8b1e
Author: Andrey V. Mashenkov 
Date:   2017-06-21T10:42:12Z

GG-12350: GridDhtAtomicSingleUpdateRequest misses topologyVersion() method 
override.

commit f8224d13cf9a6432ba65e0016370ba51bbb544e9
Author: Alexey Goncharuk 
Date:   2017-06-15T19:49:45Z

GG-12299: Make sure concurrent type registrations do not trigger multiple 
cache updates.

commit 4ffc3acfa1bc43bea8c79bfd1864787c15cfc4de
Author: Alexey Goncharuk 
Date:   2017-06-20T04:59:09Z

IGNITE-5528 - IS_EVICT_DISABLED flag is not cleared on cache store 
exception.

commit 8cd9e829380f4c91cc9bb126169863286d1cb323
Author: Andrey V. Mashenkov 
Date:   2017-06-21T12:40:14Z

GG-12353: Added local binary context flag.

Backport of IGNITE-5223 with fixes.

commit 9036ad239d68eff663bc73a81baab2826b054d9a
Author: Andrey V. Mashenkov 
Date:   2017-06-21T15:25:31Z

Added MBean for system cache executors.

commit ed34a5dc681ea8f284f4d25c5575ad46569cc600
Author: Andrey V. Mashenkov 
Date:   2017-06-21T15:33:55Z

Partial fix of IGNITE-5562.

commit d427021f329292fb69d348ba949ad1f8f1e9089e
Author: Andrey V. Mashenkov 
Date:   2017-06-21T16:30:27Z

IGNITE-5552: ServiceProcessor recalculates all service assignments even if 
there is a pending topology change.

commit f1b9cdc0716a1b23f54d68ce0fe19eb85107567d
Author: Alexey Goncharuk 
Date:   2017-06-14T18:37:54Z

GG-12354: Partial fix of IGNITE-5473: Introduce troubleshooting logger.

commit beb2409cfe2045789443d47de735d879961d371e
Author: Andrey V. Mashenkov 
Date:   2017-06-23T09:26:06Z

GG-12352: Forcible node drop makes cluster instable in some cases.
Disable forcible node drop by default.

commit 802f18fc250cbae8959192c78bb28dc525ed3cf7
Author: AMRepo 
Date:   2017-06-22T21:24:57Z

Fix compilation

commit 39d2dec85a3c571dfdb1cd6189b53ae2413a5d22
Author: Andrey V. Mashenkov 
Date:   2017-06-23T10:41:30Z

Merge branch 'ignite-1.7.12-b2' into ignite-1.8.8

# Conflicts:
#   modules/core/src/main/java/org/apache/ignite/internal/GridTopic.java
#   
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoManager.java
#   
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/GridIoMessageFactory.java
#   
modules/core/src/main/java/org/apache/ignite/internal/managers/communication/IgniteIoTestMessage.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/GridCacheAdapter.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/cache/distributed/dht/atomic/GridDhtAtomicCache.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/service/GridServiceProcessor.java
#   
modules/core/src/main/java/org/apache/ignite/internal/processors/task/GridTaskThreadContextKey.java
#   
modules/core/src/main/java/org/apache/ignite/spi/communication/tcp/TcpCommunicationSpi.java
#   
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheTestSuite5.java

commit 96445156143b46b664f785b353463dd435ac342d
Author: Andrey V. Mashenkov 
Date:   2017-06-23T10:42:44Z

Merge remote-tracking branch 'origin/ignite-1.8.8' into ignite-1.8.8

# Conflicts:
#   
modules/core/src/test/java/org/apache/ignite/testsuites/IgniteCacheTestSuite5.java

comm

[GitHub] ignite pull request #3031: IGNITE-6896 .NET: support Multidimensional Arrays...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3031


---

Re: Add emergency node closing handler to public Ignite API

[Anton Vinogradov]

Vova,

I'll refactor IEP-7 [1], most likely merge it with IEP-5 [2], and let you
know that overall design ready and clear :)

[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-7%3A+Ignite+internal+problems+detection#IEP-7:Igniteinternalproblemsdetection-SystemThreadRegestry
.
[2]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-5+Cluster+reaction+if+node+detects+an+extraordinary+situations

On Wed, Nov 15, 2017 at 3:21 PM, Vladimir Ozerov 
wrote:

> It would be nice to see the whole design first before going into low-level
> details. Without it we are jumping from topic to topic. Were the list
> events and reaction to these events discussed previously? At this point it
> is not clear why nodes should be forcefully stopped without any
> alternative.
>
> For example, consider the following cases:
> 1) Exchange thread died. This is critical situation. But as a part of
> analysis administrator might want to dump threads before killing the node.
> He can do that programmatically, which is difficult and require knowledge
> of Java, or can do that through management utilities, such as jstack or
> VisualVM. What is more user friendly?
> 2) We start a service with multiple data regions. One data region is
> configured incorrectly, what causes IOOME on multiple nodes. Why do you
> think that the whole cluster (or many nodes) should be restarted? This is
> potential data loss in all caches (not only in affected) and interruption
> of service. Instead, administrator might decide to gradually reconfigure
> and restart nodes one by one, instead of killing them all immediately.
>
> This is why we need the design first.
>
> On Wed, Nov 15, 2017 at 2:39 PM, Anton Vinogradov <
> avinogra...@gridgain.com>
> wrote:
>
> > According to [1]
> >
> > Reasons are:
> > - IgniteOutOfMemoryException
> > - Persistence errors
> > - ExchangeWorker exits with error
> >
> > [1]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > 7%3A+Ignite+internal+problems+detection
> >
> > On Wed, Nov 15, 2017 at 2:24 PM, Vladimir Ozerov 
> > wrote:
> >
> > > I am not quite I understand how tasks are split. How can we discuss
> > > graceful shutdown without discussing the reasons of this shutdown? What
> > > leads to it?
> > >
> > > On Wed, Nov 15, 2017 at 2:10 PM, Anton Vinogradov <
> > > avinogra...@gridgain.com>
> > > wrote:
> > >
> > > > Vova,
> > > >
> > > > Currently we have a lot IEPs to improve grid monitoring and behavior.
> > > >
> > > > Let's split tasks to:
> > > >
> > > > 1) Graceful shutdown.
> > > > In this case we'd like to provide user ability to do something,
> > > > LifecycleBean is what we looking for, thanks for tips!
> > > > But, we have to keep shutdown reason somewhere.
> > > > In case you know where it already kept , please let us know.
> > > >
> > > > 2) OOM or any other reason cause node crash.
> > > > In this case some watchdog (like [1] or [2]) should monitor node
> alive
> > > >
> > > > 3) GC and deadlock(java and tx) issues
> > > > Should be monitored by special thread [3] or published by metrics [4]
> > > >
> > > > 4) Throughput, latency and space issues
> > > > Special metrics should be developed according to [5]
> > > >
> > > > Andrey asking about case #1 (graceful shutdown), lets discuss only
> this
> > > > case.
> > > >
> > > > [1] https://issues.apache.org/jira/browse/IGNITE-6587
> > > > [2] https://wrapper.tanukisoftware.com/doc/english/download.jsp
> > > > [3] https://issues.apache.org/jira/browse/IGNITE-6171
> > > > [4]
> > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > > > 7%3A+Ignite+internal+problems+detection
> > > > [5]
> > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > > > 6%3A+Metrics+improvements
> > > >
> > > >
> > > > On Wed, Nov 15, 2017 at 1:34 PM, Vladimir Ozerov <
> voze...@gridgain.com
> > >
> > > > wrote:
> > > >
> > > > > AFAIK the idea was not only to shutdown the node, but also to give
> > user
> > > > > (e.g. administrator) ability to observe the problem from the
> outside,
> > > > e.g.
> > > > > through JMX. E.g. if we detect Java-level deadlock, it doesn't mean
> > > that
> > > > > the only possible solution is node shutdown. In addition it could
> be
> > > > no-op,
> > > > > e.g. to give user chance to collect additional system info, or
> simply
> > > > > because this particular deadlock is resolvable (e.g.
> > > > > Lock.lockInterruptibly()). So as we need to expose health info
> > through
> > > > JMX
> > > > > anyway, we could also give user programmatic access to it as well.
> > > > > Alternatively, we can expose this info through JMX only and ask
> user
> > to
> > > > get
> > > > > instance of that bean manually.
> > > > >
> > > > > On Wed, Nov 15, 2017 at 1:19 PM, Anton Vinogradov <
> > > > > avinogra...@gridgain.com>
> > > > > wrote:
> > > > >
> > > > > > Vova,
> > > > > >
> > > > > > Could you point to metric you're talking about?
> > > > > >
> > > > > > On Wed, Nov 15, 2017 at 1:06 PM, Andrey Kuznetsov <
> > stku...@gmail.c

[GitHub] ignite pull request #3037: [IGNITE-6702] Get rid of values deserialization i...

[gg-shq]

GitHub user gg-shq opened a pull request:

https://github.com/apache/ignite/pull/3037

[IGNITE-6702] Get rid of values deserialization in H2TreeIndex.getRowCount

Committing preliminary version to run tests on TC

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6702

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3037.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3037


commit a7541202bb835c6e0dff004c06ea222384a53549
Author: gg-shq 
Date:   2017-11-15T12:20:04Z

IGNITE-6702: Change H2TreeIndex.getRowCount() to use BPlusTree.size() 
method instead of BPlusTree.find() + iteration

commit c31145be5ee11a7bd3cb72f960e03e9212a964d8
Author: gg-shq 
Date:   2017-11-15T12:20:30Z

IGNITE-6702: Change H2TreeIndex.getRowCount() to use BPlusTree.size() 
method instead of BPlusTree.find() + iteration




---

Re: Add emergency node closing handler to public Ignite API

[Vladimir Ozerov]

It would be nice to see the whole design first before going into low-level
details. Without it we are jumping from topic to topic. Were the list
events and reaction to these events discussed previously? At this point it
is not clear why nodes should be forcefully stopped without any
alternative.

For example, consider the following cases:
1) Exchange thread died. This is critical situation. But as a part of
analysis administrator might want to dump threads before killing the node.
He can do that programmatically, which is difficult and require knowledge
of Java, or can do that through management utilities, such as jstack or
VisualVM. What is more user friendly?
2) We start a service with multiple data regions. One data region is
configured incorrectly, what causes IOOME on multiple nodes. Why do you
think that the whole cluster (or many nodes) should be restarted? This is
potential data loss in all caches (not only in affected) and interruption
of service. Instead, administrator might decide to gradually reconfigure
and restart nodes one by one, instead of killing them all immediately.

This is why we need the design first.

On Wed, Nov 15, 2017 at 2:39 PM, Anton Vinogradov 
wrote:

> According to [1]
>
> Reasons are:
> - IgniteOutOfMemoryException
> - Persistence errors
> - ExchangeWorker exits with error
>
> [1]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 7%3A+Ignite+internal+problems+detection
>
> On Wed, Nov 15, 2017 at 2:24 PM, Vladimir Ozerov 
> wrote:
>
> > I am not quite I understand how tasks are split. How can we discuss
> > graceful shutdown without discussing the reasons of this shutdown? What
> > leads to it?
> >
> > On Wed, Nov 15, 2017 at 2:10 PM, Anton Vinogradov <
> > avinogra...@gridgain.com>
> > wrote:
> >
> > > Vova,
> > >
> > > Currently we have a lot IEPs to improve grid monitoring and behavior.
> > >
> > > Let's split tasks to:
> > >
> > > 1) Graceful shutdown.
> > > In this case we'd like to provide user ability to do something,
> > > LifecycleBean is what we looking for, thanks for tips!
> > > But, we have to keep shutdown reason somewhere.
> > > In case you know where it already kept , please let us know.
> > >
> > > 2) OOM or any other reason cause node crash.
> > > In this case some watchdog (like [1] or [2]) should monitor node alive
> > >
> > > 3) GC and deadlock(java and tx) issues
> > > Should be monitored by special thread [3] or published by metrics [4]
> > >
> > > 4) Throughput, latency and space issues
> > > Special metrics should be developed according to [5]
> > >
> > > Andrey asking about case #1 (graceful shutdown), lets discuss only this
> > > case.
> > >
> > > [1] https://issues.apache.org/jira/browse/IGNITE-6587
> > > [2] https://wrapper.tanukisoftware.com/doc/english/download.jsp
> > > [3] https://issues.apache.org/jira/browse/IGNITE-6171
> > > [4]
> > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > > 7%3A+Ignite+internal+problems+detection
> > > [5]
> > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > > 6%3A+Metrics+improvements
> > >
> > >
> > > On Wed, Nov 15, 2017 at 1:34 PM, Vladimir Ozerov  >
> > > wrote:
> > >
> > > > AFAIK the idea was not only to shutdown the node, but also to give
> user
> > > > (e.g. administrator) ability to observe the problem from the outside,
> > > e.g.
> > > > through JMX. E.g. if we detect Java-level deadlock, it doesn't mean
> > that
> > > > the only possible solution is node shutdown. In addition it could be
> > > no-op,
> > > > e.g. to give user chance to collect additional system info, or simply
> > > > because this particular deadlock is resolvable (e.g.
> > > > Lock.lockInterruptibly()). So as we need to expose health info
> through
> > > JMX
> > > > anyway, we could also give user programmatic access to it as well.
> > > > Alternatively, we can expose this info through JMX only and ask user
> to
> > > get
> > > > instance of that bean manually.
> > > >
> > > > On Wed, Nov 15, 2017 at 1:19 PM, Anton Vinogradov <
> > > > avinogra...@gridgain.com>
> > > > wrote:
> > > >
> > > > > Vova,
> > > > >
> > > > > Could you point to metric you're talking about?
> > > > >
> > > > > On Wed, Nov 15, 2017 at 1:06 PM, Andrey Kuznetsov <
> stku...@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Vladimir,
> > > > > >
> > > > > > Could you please refine, what are local metrics? Should I extend
> > > Ignite
> > > > > > interface by adding something similar to dataRegionMetrics() or
> > there
> > > > is
> > > > > > some universal mechanism to handle metrics?
> > > > > >
> > > > > > 2017-11-15 8:30 GMT+03:00 Vladimir Ozerov  >:
> > > > > > >
> > > > > > > This information should be available through local metrics, so
> > that
> > > > it
> > > > > is
> > > > > > > accessible from Ignite instance.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Losing data during restarting cluster with persistence enabled

[Andrey Mashenkov]

Hi Vyacheslav,

Key to partition mapping shouldn't depends on topology, and shouldn't
changed unstable topology.
Looks like you've missed smth.

Would you please share configuration?
Does all nodes share same RockDB database or each node has its own copy?



On Wed, Nov 15, 2017 at 12:22 AM, Vyacheslav Daradur 
wrote:

> Hi, Igniters!
>
> I’m using partitioned Ignite cache with RocksDB as 3rd party persistence
> store.
> I've got an issue: if cache rebalancing is switched on, then it’s
> possible to lose some data.
>
> Basic scenario:
> 1) Start Ignite cluster and fill a cache with RocksDB persistence;
> 2) Stop all nodes
> 3) Start Ignite cluster and validate data
>
> This works fine while rebalancing is switched off.
>
> If rebalancing switched on: when I call Ignition#stopAll, some nodes
> go down sequentially and while one node having gone down another start
> rebalancing. When nodes started affinity function works with a full
> set of nodes and may define a wrong partition for a key because the
> previous state was changed at rebalancing.
>
> Maybe I'm doing something wrong. How can I avoid rebalancing while
> stopping all nodes in the cluster?
>
> Could you give me any advice, please?
>
> --
> Best Regards, Vyacheslav D.
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Add emergency node closing handler to public Ignite API

[Anton Vinogradov]

According to [1]

Reasons are:
- IgniteOutOfMemoryException
- Persistence errors
- ExchangeWorker exits with error

[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-7%3A+Ignite+internal+problems+detection

On Wed, Nov 15, 2017 at 2:24 PM, Vladimir Ozerov 
wrote:

> I am not quite I understand how tasks are split. How can we discuss
> graceful shutdown without discussing the reasons of this shutdown? What
> leads to it?
>
> On Wed, Nov 15, 2017 at 2:10 PM, Anton Vinogradov <
> avinogra...@gridgain.com>
> wrote:
>
> > Vova,
> >
> > Currently we have a lot IEPs to improve grid monitoring and behavior.
> >
> > Let's split tasks to:
> >
> > 1) Graceful shutdown.
> > In this case we'd like to provide user ability to do something,
> > LifecycleBean is what we looking for, thanks for tips!
> > But, we have to keep shutdown reason somewhere.
> > In case you know where it already kept , please let us know.
> >
> > 2) OOM or any other reason cause node crash.
> > In this case some watchdog (like [1] or [2]) should monitor node alive
> >
> > 3) GC and deadlock(java and tx) issues
> > Should be monitored by special thread [3] or published by metrics [4]
> >
> > 4) Throughput, latency and space issues
> > Special metrics should be developed according to [5]
> >
> > Andrey asking about case #1 (graceful shutdown), lets discuss only this
> > case.
> >
> > [1] https://issues.apache.org/jira/browse/IGNITE-6587
> > [2] https://wrapper.tanukisoftware.com/doc/english/download.jsp
> > [3] https://issues.apache.org/jira/browse/IGNITE-6171
> > [4]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > 7%3A+Ignite+internal+problems+detection
> > [5]
> > https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> > 6%3A+Metrics+improvements
> >
> >
> > On Wed, Nov 15, 2017 at 1:34 PM, Vladimir Ozerov 
> > wrote:
> >
> > > AFAIK the idea was not only to shutdown the node, but also to give user
> > > (e.g. administrator) ability to observe the problem from the outside,
> > e.g.
> > > through JMX. E.g. if we detect Java-level deadlock, it doesn't mean
> that
> > > the only possible solution is node shutdown. In addition it could be
> > no-op,
> > > e.g. to give user chance to collect additional system info, or simply
> > > because this particular deadlock is resolvable (e.g.
> > > Lock.lockInterruptibly()). So as we need to expose health info through
> > JMX
> > > anyway, we could also give user programmatic access to it as well.
> > > Alternatively, we can expose this info through JMX only and ask user to
> > get
> > > instance of that bean manually.
> > >
> > > On Wed, Nov 15, 2017 at 1:19 PM, Anton Vinogradov <
> > > avinogra...@gridgain.com>
> > > wrote:
> > >
> > > > Vova,
> > > >
> > > > Could you point to metric you're talking about?
> > > >
> > > > On Wed, Nov 15, 2017 at 1:06 PM, Andrey Kuznetsov  >
> > > > wrote:
> > > >
> > > > > Vladimir,
> > > > >
> > > > > Could you please refine, what are local metrics? Should I extend
> > Ignite
> > > > > interface by adding something similar to dataRegionMetrics() or
> there
> > > is
> > > > > some universal mechanism to handle metrics?
> > > > >
> > > > > 2017-11-15 8:30 GMT+03:00 Vladimir Ozerov :
> > > > > >
> > > > > > This information should be available through local metrics, so
> that
> > > it
> > > > is
> > > > > > accessible from Ignite instance.
> > > > > >
> > > > >
> > > >
> > >
> >
>

Re: Add emergency node closing handler to public Ignite API

[Vladimir Ozerov]

I am not quite I understand how tasks are split. How can we discuss
graceful shutdown without discussing the reasons of this shutdown? What
leads to it?

On Wed, Nov 15, 2017 at 2:10 PM, Anton Vinogradov 
wrote:

> Vova,
>
> Currently we have a lot IEPs to improve grid monitoring and behavior.
>
> Let's split tasks to:
>
> 1) Graceful shutdown.
> In this case we'd like to provide user ability to do something,
> LifecycleBean is what we looking for, thanks for tips!
> But, we have to keep shutdown reason somewhere.
> In case you know where it already kept , please let us know.
>
> 2) OOM or any other reason cause node crash.
> In this case some watchdog (like [1] or [2]) should monitor node alive
>
> 3) GC and deadlock(java and tx) issues
> Should be monitored by special thread [3] or published by metrics [4]
>
> 4) Throughput, latency and space issues
> Special metrics should be developed according to [5]
>
> Andrey asking about case #1 (graceful shutdown), lets discuss only this
> case.
>
> [1] https://issues.apache.org/jira/browse/IGNITE-6587
> [2] https://wrapper.tanukisoftware.com/doc/english/download.jsp
> [3] https://issues.apache.org/jira/browse/IGNITE-6171
> [4]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 7%3A+Ignite+internal+problems+detection
> [5]
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-
> 6%3A+Metrics+improvements
>
>
> On Wed, Nov 15, 2017 at 1:34 PM, Vladimir Ozerov 
> wrote:
>
> > AFAIK the idea was not only to shutdown the node, but also to give user
> > (e.g. administrator) ability to observe the problem from the outside,
> e.g.
> > through JMX. E.g. if we detect Java-level deadlock, it doesn't mean that
> > the only possible solution is node shutdown. In addition it could be
> no-op,
> > e.g. to give user chance to collect additional system info, or simply
> > because this particular deadlock is resolvable (e.g.
> > Lock.lockInterruptibly()). So as we need to expose health info through
> JMX
> > anyway, we could also give user programmatic access to it as well.
> > Alternatively, we can expose this info through JMX only and ask user to
> get
> > instance of that bean manually.
> >
> > On Wed, Nov 15, 2017 at 1:19 PM, Anton Vinogradov <
> > avinogra...@gridgain.com>
> > wrote:
> >
> > > Vova,
> > >
> > > Could you point to metric you're talking about?
> > >
> > > On Wed, Nov 15, 2017 at 1:06 PM, Andrey Kuznetsov 
> > > wrote:
> > >
> > > > Vladimir,
> > > >
> > > > Could you please refine, what are local metrics? Should I extend
> Ignite
> > > > interface by adding something similar to dataRegionMetrics() or there
> > is
> > > > some universal mechanism to handle metrics?
> > > >
> > > > 2017-11-15 8:30 GMT+03:00 Vladimir Ozerov :
> > > > >
> > > > > This information should be available through local metrics, so that
> > it
> > > is
> > > > > accessible from Ignite instance.
> > > > >
> > > >
> > >
> >
>

Re: Add emergency node closing handler to public Ignite API

[Anton Vinogradov]

Vova,

Currently we have a lot IEPs to improve grid monitoring and behavior.

Let's split tasks to:

1) Graceful shutdown.
In this case we'd like to provide user ability to do something,
LifecycleBean is what we looking for, thanks for tips!
But, we have to keep shutdown reason somewhere.
In case you know where it already kept , please let us know.

2) OOM or any other reason cause node crash.
In this case some watchdog (like [1] or [2]) should monitor node alive

3) GC and deadlock(java and tx) issues
Should be monitored by special thread [3] or published by metrics [4]

4) Throughput, latency and space issues
Special metrics should be developed according to [5]

Andrey asking about case #1 (graceful shutdown), lets discuss only this
case.

[1] https://issues.apache.org/jira/browse/IGNITE-6587
[2] https://wrapper.tanukisoftware.com/doc/english/download.jsp
[3] https://issues.apache.org/jira/browse/IGNITE-6171
[4]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-7%3A+Ignite+internal+problems+detection
[5]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-6%3A+Metrics+improvements


On Wed, Nov 15, 2017 at 1:34 PM, Vladimir Ozerov 
wrote:

> AFAIK the idea was not only to shutdown the node, but also to give user
> (e.g. administrator) ability to observe the problem from the outside, e.g.
> through JMX. E.g. if we detect Java-level deadlock, it doesn't mean that
> the only possible solution is node shutdown. In addition it could be no-op,
> e.g. to give user chance to collect additional system info, or simply
> because this particular deadlock is resolvable (e.g.
> Lock.lockInterruptibly()). So as we need to expose health info through JMX
> anyway, we could also give user programmatic access to it as well.
> Alternatively, we can expose this info through JMX only and ask user to get
> instance of that bean manually.
>
> On Wed, Nov 15, 2017 at 1:19 PM, Anton Vinogradov <
> avinogra...@gridgain.com>
> wrote:
>
> > Vova,
> >
> > Could you point to metric you're talking about?
> >
> > On Wed, Nov 15, 2017 at 1:06 PM, Andrey Kuznetsov 
> > wrote:
> >
> > > Vladimir,
> > >
> > > Could you please refine, what are local metrics? Should I extend Ignite
> > > interface by adding something similar to dataRegionMetrics() or there
> is
> > > some universal mechanism to handle metrics?
> > >
> > > 2017-11-15 8:30 GMT+03:00 Vladimir Ozerov :
> > > >
> > > > This information should be available through local metrics, so that
> it
> > is
> > > > accessible from Ignite instance.
> > > >
> > >
> >
>

Request for contributors permissions

[Michael Zhuravlev]

Hello!
I want to resolve bugs. Please, grant me permission for assign tasks in
jira. My jira account MikeZhur.

Regards,
Mike Zhuravlev

[GitHub] ignite pull request #3015: IGNITE-6836: Implemented query timeout.

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/ignite/pull/3015


---

[jira] [Created] (IGNITE-6921) Optimise tx/queries tracking when mvcc is enabled and local caches are used

[Igor Seliverstov (JIRA)]

Igor Seliverstov created IGNITE-6921:


 Summary: Optimise tx/queries tracking when mvcc is enabled and 
local caches are used
 Key: IGNITE-6921
 URL: https://issues.apache.org/jira/browse/IGNITE-6921
 Project: Ignite
  Issue Type: Improvement
  Security Level: Public (Viewable by anyone)
Reporter: Igor Seliverstov


Seems we don't need to request mvcc version and track tx/queries on mvcc 
coordinator when only local caches are used.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Add emergency node closing handler to public Ignite API

[Vladimir Ozerov]

AFAIK the idea was not only to shutdown the node, but also to give user
(e.g. administrator) ability to observe the problem from the outside, e.g.
through JMX. E.g. if we detect Java-level deadlock, it doesn't mean that
the only possible solution is node shutdown. In addition it could be no-op,
e.g. to give user chance to collect additional system info, or simply
because this particular deadlock is resolvable (e.g.
Lock.lockInterruptibly()). So as we need to expose health info through JMX
anyway, we could also give user programmatic access to it as well.
Alternatively, we can expose this info through JMX only and ask user to get
instance of that bean manually.

On Wed, Nov 15, 2017 at 1:19 PM, Anton Vinogradov 
wrote:

> Vova,
>
> Could you point to metric you're talking about?
>
> On Wed, Nov 15, 2017 at 1:06 PM, Andrey Kuznetsov 
> wrote:
>
> > Vladimir,
> >
> > Could you please refine, what are local metrics? Should I extend Ignite
> > interface by adding something similar to dataRegionMetrics() or there is
> > some universal mechanism to handle metrics?
> >
> > 2017-11-15 8:30 GMT+03:00 Vladimir Ozerov :
> > >
> > > This information should be available through local metrics, so that it
> is
> > > accessible from Ignite instance.
> > >
> >
>

[jira] [Created] (IGNITE-6920) Web console: Prepare Web Console package with simple deploy.

[Andrey Novikov (JIRA)]

Andrey Novikov created IGNITE-6920:
--

 Summary: Web console: Prepare Web Console package with simple 
deploy.
 Key: IGNITE-6920
 URL: https://issues.apache.org/jira/browse/IGNITE-6920
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: wizards
Affects Versions: 2.0
Reporter: Andrey Novikov
Priority: Minor


* Package Web Console backend into an executable that can be run even on 
devices without Node.js installed.
* Let Web Console backend will be used to serve static files (compiled Web 
Console frontend)
* Let Web Console backend download and run as child process mongoDB. if mongoDB 
is not installed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #2745: Ignite 1.8.12

[sk0x50]

Github user sk0x50 closed the pull request at:

https://github.com/apache/ignite/pull/2745


---

Re: Add emergency node closing handler to public Ignite API

[Anton Vinogradov]

Vova,

Could you point to metric you're talking about?

On Wed, Nov 15, 2017 at 1:06 PM, Andrey Kuznetsov  wrote:

> Vladimir,
>
> Could you please refine, what are local metrics? Should I extend Ignite
> interface by adding something similar to dataRegionMetrics() or there is
> some universal mechanism to handle metrics?
>
> 2017-11-15 8:30 GMT+03:00 Vladimir Ozerov :
> >
> > This information should be available through local metrics, so that it is
> > accessible from Ignite instance.
> >
>

Re: Add emergency node closing handler to public Ignite API

[Andrey Kuznetsov]

Vladimir,

Could you please refine, what are local metrics? Should I extend Ignite
interface by adding something similar to dataRegionMetrics() or there is
some universal mechanism to handle metrics?

2017-11-15 8:30 GMT+03:00 Vladimir Ozerov :
>
> This information should be available through local metrics, so that it is
> accessible from Ignite instance.
>

[jira] [Created] (IGNITE-6919) Web console: incorrect character in tab name under IE11

[Pavel Konstantinov (JIRA)]

Pavel Konstantinov created IGNITE-6919:
--

 Summary: Web console: incorrect character in tab name under IE11
 Key: IGNITE-6919
 URL: https://issues.apache.org/jira/browse/IGNITE-6919
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
Reporter: Pavel Konstantinov
Priority: Trivial






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Why isn't Apache Ignite on stackshare.io

[Jason Man, CLSA]

Just curious, why isn't Apache Ignite on Stackshare.io?  Stackshare.io is 
really picking up in momentum as a developer community for sharing what stacks 
are used and where.  Could be useful in promoting usage in the open-source 
community.

I can see that Hazelcast is:
https://stackshare.io/hazelcast

Regards,
Jason

The content of this communication is intended for the recipient and is subject 
to CLSA Legal and Regulatory Notices.
These can be viewed at https://www.clsa.com/disclaimer.html or sent to you upon 
request.
CLSA is ISO14001 certified and committed to reducing environmental impact.

[jira] [Created] (IGNITE-6918) SQL: support window functions

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-6918:
---

 Summary: SQL: support window functions
 Key: IGNITE-6918
 URL: https://issues.apache.org/jira/browse/IGNITE-6918
 Project: Ignite
  Issue Type: New Feature
  Security Level: Public (Viewable by anyone)
  Components: sql
Reporter: Vladimir Ozerov


E.g. {{PARTITION ... OVER}}. Design is to be defined. See [1] as a starting 
point.

[1] https://en.wikipedia.org/wiki/SQL_window_function



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (IGNITE-6917) SQL: implement COP command for efficient data loading

[Vladimir Ozerov (JIRA)]

Vladimir Ozerov created IGNITE-6917:
---

 Summary: SQL: implement COP command for efficient data loading
 Key: IGNITE-6917
 URL: https://issues.apache.org/jira/browse/IGNITE-6917
 Project: Ignite
  Issue Type: Task
  Security Level: Public (Viewable by anyone)
  Components: sql
Reporter: Vladimir Ozerov
 Fix For: 2.4


Inspired by Postgres [1]

Common use case - bulk data load through JDBC/ODBC interface. Currently it is 
only possible to execute single commands one by one. We already can batch them 
to improve performance, but there is still big room for improvement.

We should think of a completely new command - {{COPY}}. It will accept a file 
(or input stream in general case) on the client side, then transfer data to the 
cluster, and then execute update inside the cluster, e.g. through streamer. 

First of all we need to create quick and dirty prototype to assess potential 
performance improvement. It speedup is confirmed, we should build base 
implementation which will accept only files. But at the same time we should 
understand how it will evolve in future: multiple file formats (probably 
including Hadoop formarts, e.g. Parquet), escape characters, input streams, 
etc..

[1] https://www.postgresql.org/docs/9.6/static/sql-copy.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[GitHub] ignite pull request #3036: IGNITE-6916: node joining with enabled pds and em...

[zstan]

GitHub user zstan opened a pull request:

https://github.com/apache/ignite/pull/3036

IGNITE-6916: node joining with enabled pds and empty disc space cause…

…s exchange to hang

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/gridgain/apache-ignite ignite-6916

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/ignite/pull/3036.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3036


commit 56d27cb452d64c493dfb00d72578fd49f321795c
Author: Evgeny Stanilovskiy 
Date:   2017-11-15T08:51:25Z

IGNITE-6916: node joining with enabled pds and empty disc space causes 
exchange to hang




---

[jira] [Created] (IGNITE-6916) A node joining with enabled persistence and empty disc space causes exchange to hang.

[Stanilovsky Evgeny (JIRA)]

Stanilovsky Evgeny created IGNITE-6916:
--

 Summary: A node joining with enabled persistence and empty disc 
space causes exchange to hang.
 Key: IGNITE-6916
 URL: https://issues.apache.org/jira/browse/IGNITE-6916
 Project: Ignite
  Issue Type: Bug
  Security Level: Public (Viewable by anyone)
  Components: persistence
Affects Versions: 2.4
Reporter: Stanilovsky Evgeny
Assignee: Stanilovsky Evgeny
 Fix For: 2.4


If no more free disk space occurs, while node joining cluster, it will be 
hanging.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: Ignite Enhancement Proposal #6 (Metrics)

[Vladimir Ozerov]

Dima,

This appears to be a INFRA's bug. I filed a ticket [1].

[1] https://issues.apache.org/jira/browse/INFRA-15487

On Tue, Nov 14, 2017 at 9:25 PM, Dmitriy Setrakyan 
wrote:

> On Tue, Nov 14, 2017 at 10:12 AM, Anton Vinogradov <
> avinogra...@gridgain.com
> > wrote:
>
> > Dmitriy,
> >
> > It looks like a confluence bug.
> > Please login and push refresh button at issues list.
> >
>
> Works now. It is unfortunate that it does not work for the community
> members who do not login. Perhaps we can provide a link to the Jira filter,
> so folks could click on it and view the tickets.
>