Re: Ignite 2.7 Errors

[Andrey Kuznetsov]

Philip, if you can bear with so huge JVM pauses, then there is no use to
investigate stacktraces anymore. Just increase systemWorkerBlockedTimeout
parameter of IgniteConfiguration appropriately, as described in
https://apacheignite.readme.io/docs/critical-failures-handling#section-critical-workers-health-check
and Ignite 2.7 won't report these failures.

чт, 21 мар. 2019 г. в 20:42, Philip Wu :

> hello, Andrey -
>
> for your 2nd question, in Ignite 2.5, we have 15 mins + JVM paused as well,
> but no IgniteException, was working fine.
>
> 2019-03-15 19:08:46,088 WARNING [ (jvm-pause-detector-worker)] Possible too
> long JVM pause: 1001113 milliseconds.
>
> 2019-03-15 19:08:46,280 INFO  [IgniteKernal%XXXGrid
> (grid-timeout-worker-#71%XXXGrid%)]
> Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> ^-- Node [id=1dc0de55, name=EnfusionGrid, uptime=01:49:39.992]
> ^-- H/N/C [hosts=1, nodes=1, CPUs=32]
> ^-- CPU [cur=100%, avg=39.77%, GC=1042.83%]
> ^-- PageMemory [pages=2300496]
> ^-- Heap [used=295123MB, free=14.48%, comm=345088MB]
> ^-- Non heap [used=442MB, free=-1%, comm=463MB]
> ^-- Outbound messages queue [size=0]
> ^-- Public thread pool [active=0, idle=0, qSize=0]
> ^-- System thread pool [active=0, idle=1, qSize=0]
> 2019-03-15 19:08:46,280 INFO  [IgniteKernal%XXXGrid
> (grid-timeout-worker-#71%XXXGrid%)] FreeList [name=XXXGrid, buckets=256,
> dataPages=1, reusePages=0]
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>


-- 
Best regards,
  Andrey Kuznetsov.

infinite localhost to localhost connections

[Andrey Lobachev]

Hello!

We have Apache ignite cluster v2.7 with Zookeeper discovery. After some
time in the log on each node appeared these messages:

[2019-03-21T22:48:35,536][INFO
][grid-nio-worker-tcp-comm-1-#25][TcpCommunicationSpi] Accepted incoming
communication connection [locAddr=/127.0.0.1:47100, rmtAddr=/127.0.0.1:57186
]
 [2019-03-21T22:48:59,119][INFO
][grid-nio-worker-tcp-comm-2-#26][TcpCommunicationSpi] Accepted incoming
communication connection [locAddr=/127.0.0.1:47100, rmtAddr=/127.0.0.1:57188
]
 [2019-03-21T22:49:46,173][INFO
][grid-nio-worker-tcp-comm-3-#27][TcpCommunicationSpi] Accepted incoming
communication connection [locAddr=/127.0.0.1:47100, rmtAddr=/127.0.0.1:57194
]
 [2019-03-21T22:50:01,658][INFO
][grid-nio-worker-tcp-comm-0-#24][TcpCommunicationSpi] Accepted incoming
communication connection [locAddr=/127.0.0.1:47100, rmtAddr=/127.0.0.1:57200
]

And then:

[2019-03-21T22:50:02,660][ERROR][grid-nio-worker-tcp-comm-0-#24][TcpCommunicationSpi]
Failed to process selector key [ses=GridSelectorNioSessionImpl
[worker=DirectNioClientWorker [super=AbstractNioClientWorker [
 idx=0, bytesRcvd=5508546, bytesSent=1747567, bytesRcvd0=0, bytesSent0=18,
select=true, super=GridWorker [name=grid-nio-worker-tcp-comm-0,
igniteInstanceName=null, finished=false, heartbeatTs=1553197801651, hashC
 ode=1278667839, interrupted=false,
runner=grid-nio-worker-tcp-comm-0-#24]]],
writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecover
 y=null, outRecovery=null, super=GridNioSessionImpl [locAddr=/
127.0.0.1:47100, rmtAddr=/127.0.0.1:57200, createTime=1553197801651,
closeTime=0, bytesSent=18, bytesRcvd=0, bytesSent0=18, bytesRcvd0=0,
sndSchedTime
 =1553197801651, lastSndTime=1553197801651, lastRcvTime=1553197801651,
readsPaused=false, filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=o.a.i.i.util.nio.GridDirectParser@64f568c9, directMode=true],
 GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]
 java.io.IOException: Connection reset by peer
 at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
~[?:1.8.0_191]
 at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
~[?:1.8.0_191]
 at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
~[?:1.8.0_191]
 at sun.nio.ch.IOUtil.read(IOUtil.java:192) ~[?:1.8.0_191]
 at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:380)
~[?:1.8.0_191]
 at
org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1275)
~[ignite-core-2.7.0.jar:2.7.0]
 at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeysOptimized(GridNioServer.java:2389)
[ignite-core-2.7.0.jar:2.7.0]
 at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2156)
[ignite-core-2.7.0.jar:2.7.0]
 at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1797)
[ignite-core-2.7.0.jar:2.7.0]
 at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
[ignite-core-2.7.0.jar:2.7.0]
 at java.lang.Thread.run(Thread.java:748) [?:1.8.0_191]
 [2019-03-21T22:50:02,661][WARN
][grid-nio-worker-tcp-comm-0-#24][TcpCommunicationSpi] Closing NIO session
because of unhandled exception [cls=class
o.a.i.i.util.nio.GridNioException, msg=Connection reset by  peer]

These message persists after restart and even in the case of one node
running in separate zkRootPath. There are some environments with same
configuration which dont have similar behavior. What could it be?

Re: Ignite 2.7 Errors

[Philip Wu]

hello, Andrey - 

for your 2nd question, in Ignite 2.5, we have 15 mins + JVM paused as well,
but no IgniteException, was working fine.

2019-03-15 19:08:46,088 WARNING [ (jvm-pause-detector-worker)] Possible too
long JVM pause: 1001113 milliseconds.

2019-03-15 19:08:46,280 INFO  [IgniteKernal%XXXGrid
(grid-timeout-worker-#71%XXXGrid%)]
Metrics for local node (to disable set 'metricsLogFrequency' to 0)
^-- Node [id=1dc0de55, name=EnfusionGrid, uptime=01:49:39.992]
^-- H/N/C [hosts=1, nodes=1, CPUs=32]
^-- CPU [cur=100%, avg=39.77%, GC=1042.83%]
^-- PageMemory [pages=2300496]
^-- Heap [used=295123MB, free=14.48%, comm=345088MB]
^-- Non heap [used=442MB, free=-1%, comm=463MB]
^-- Outbound messages queue [size=0]
^-- Public thread pool [active=0, idle=0, qSize=0]
^-- System thread pool [active=0, idle=1, qSize=0]
2019-03-15 19:08:46,280 INFO  [IgniteKernal%XXXGrid
(grid-timeout-worker-#71%XXXGrid%)] FreeList [name=XXXGrid, buckets=256,
dataPages=1, reusePages=0]





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.7 Errors

[Philip Wu]

Hello, Andrey - 

for your 1st question, I do have a straceTrace in other
*FailureProcessor*.log

Thread [name="tcp-disco-msg-worker-#2%XXXGrid%", id=445, state=RUNNABLE,
blockCnt=0, waitCnt=320611]
at sun.management.ThreadImpl.dumpThreads0(Native Method)
at sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:454)
at o.a.i.i.util.IgniteUtils.dumpThreads(IgniteUtils.java:1364)
at
o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:128)
- locked o.a.i.i.processors.failure.FailureProcessor@6d7b017b
at
o.a.i.i.processors.failure.FailureProcessor.process(FailureProcessor.java:104)
at
o.a.i.i.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1829)
at
o.a.i.i.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826)
at o.a.i.i.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233)
at o.a.i.i.util.worker.GridWorker.onIdle(GridWorker.java:297)
at
o.a.i.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663)
at
o.a.i.spi.discovery.tcp.ServerImpl$RingMessageWorker$$Lambda$211/210453.run(Unknown
Source)
at
o.a.i.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181)
at
o.a.i.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
at o.a.i.i.util.worker.GridWorker.run(GridWorker.java:120)
at
o.a.i.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
at o.a.i.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Can't add new key/value pair to existing cache via sql command

[kcheng.mvp]

or can I log a feature ticket?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Can't add new key/value pair to existing cache via sql command

[kcheng.mvp]

is there any timeline to release such limit? I have use ignite in production
in my sub-system. it works so far so good, the only one exception is that
every time I add new table I have to define a new cache. I think it's ok
when the system is small. In fact I would like to use ignite in my another
big system. it's obviously that new tables will be added with business
involved. This limit is my only consideration to take it into my new system.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.7 Errors

[Andrey Kuznetsov]

Sorry, my mistake. I meant the last message you provided, but it doesn't
contain stacktrace, only brief thread information.
Anyway, 15 minutes long JVM pauses are suspicious. Do you have the same
message on 2.5 or 2.6?

Best regards,
Andrey Kuznetsov.

чт, 21 марта 2019, 19:49 Philip Wu p...@enfusionsystems.com:

> before that , there was:
>
> 2019-03-20 22:28:45,028 WARNING [G (tcp-disco-msg-worker-#2%XXXGrid%)]
> Thread [name="grid-nio-worker-tcp-comm-1-#73%XXXGrid%", id=415,
> state=RUNNABLE, blockCnt=0, waitCnt=0]
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite 2.7 Errors

[Philip Wu]

before that , there was:

2019-03-20 22:28:45,028 WARNING [G (tcp-disco-msg-worker-#2%XXXGrid%)]
Thread [name="grid-nio-worker-tcp-comm-1-#73%XXXGrid%", id=415,
state=RUNNABLE, blockCnt=0, waitCnt=0]




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.7 Errors

[Philip Wu]

Hello, Andrey - actually this is the sequence of events in time order:

2019-03-20 22:28:44,999 WARNING [IgniteKernal%XXXGrid
(jvm-pause-detector-worker)] Possible too long JVM pause: 928937
milliseconds

2019-03-20 22:28:45,014 SEVERE [G (tcp-disco-msg-worker-#2%XXXGrid%)]
Blocked system-critical thread has been detected. This can lead to
cluster-wide undefined behaviour [threadName=grid-nio-worker-tcp-comm-1,
blockedFor=928s]

019-03-20 22:28:45,021 WARN  [FailoverTransport (ActiveMQ Transport:
Transport ) failed , attempting to automatically reconnect:
java.io.EOFException

2019-03-20 22:28:45,021 ERROR
[ActiveMQDelegate=>MasterServiceGlobalConnection (ActiveMQ Transport: 
transport Interrupted
2019-03-20 22:28:45,023 WARN  [FailoverTransport (ActiveMQ Transport: 
Transport () failed , attempting to automatically reconnect:
java.io.EOFException

2019-03-20 22:28:45,028 WARNING [G (tcp-disco-msg-worker-#2%EnfusionGrid%)]
Thread [name="grid-nio-worker-tcp-comm-1-#73%EnfusionGrid%", id=415,
state=RUNNABLE, blockCnt=0, waitCnt=0]

2019-03-20 22:28:45,028 WARN  [FailoverTransport (ActiveMQ Transport: )]
Transport  failed , attempting to automatically reconnect:
java.io.EOFException
2019-03-20 22:28:45,029 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] Exception in thread "ActiveMQ InactivityMonitor
WriteCheckTimer" java.lang.NullPointerException
2019-03-20 22:28:45,029 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] at
org.apache.activemq.transport.AbstractInactivityMonitor.writeCheck(AbstractInactivityMonitor.java:219)
2019-03-20 22:28:45,029 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] at
org.apache.activemq.transport.AbstractInactivityMonitor$3.run(AbstractInactivityMonitor.java:153)
2019-03-20 22:28:45,029 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] at
org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
2019-03-20 22:28:45,030 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] at java.util.TimerThread.mainLoop(Timer.java:555)
2019-03-20 22:28:45,030 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] at java.util.TimerThread.run(Timer.java:505)



2019-03-20 22:28:45,044 ERROR
[ActiveMQDelegate=>MasterServiceGlobalConnection (ActiveMQ Transport: ]
transport Interrupted

2019-03-20 22:28:45,044 SEVERE [ (tcp-disco-msg-worker-#2%XXXGrid%)]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext
[type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker
[name=grid-nio-worker-tcp-comm-1, igniteInstanceName=XXXGrid,
finished=false, heartbeatTs=1553137996031]]]: class
org.apache.ignite.IgniteException: GridWorker
[name=grid-nio-worker-tcp-comm-1, igniteInstanceName=XXXGrid,
finished=false, heartbeatTs=1553137996031]
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826)
at
org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233)
at
org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
at
org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)



2019-03-20 22:28:45,052 WARNING [FailureProcessor
(tcp-disco-msg-worker-#2%XXXGrid%)] No deadlocked threads detected.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite 2.7 Errors

[Philip Wu]

Hello, Andrey -

I see this:

2019-03-20 22:28:45,052 WARNING [FailureProcessor
(tcp-disco-msg-worker-#2%XXXGrid%)] No deadlocked threads detected.

is that what you mean?

--- 

Also, prior to crash, I see this:
not sure if it is related.

2019-03-20 22:28:45,029 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] Exception in thread "ActiveMQ InactivityMonitor
WriteCheckTimer" java.lang.NullPointerException
2019-03-20 22:28:45,029 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] at
org.apache.activemq.transport.AbstractInactivityMonitor.writeCheck(AbstractInactivityMonitor.java:219)
2019-03-20 22:28:45,029 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] at
org.apache.activemq.transport.AbstractInactivityMonitor$3.run(AbstractInactivityMonitor.java:153)
2019-03-20 22:28:45,029 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] at
org.apache.activemq.thread.SchedulerTimerTask.run(SchedulerTimerTask.java:33)
2019-03-20 22:28:45,030 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] at java.util.TimerThread.mainLoop(Timer.java:555)
2019-03-20 22:28:45,030 ERROR [stderr (ActiveMQ InactivityMonitor
WriteCheckTimer)] at java.util.TimerThread.run(Timer.java:505)





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Caches have distinct sets of data nodes

[Ilya Kasnacheev]

Hello!

Please try and share with us. You can also experiment with node filters for
non-persistent caches.

We actually have plans to enable baseline for non-persistent caches but
we're not there yet.

Regards,
-- 
Ilya Kasnacheev


чт, 21 мар. 2019 г. в 19:08, Tâm Nguyễn Mạnh :

> Hi,
>
> My data is affinity collocated. The issue is my 2nd node is not in a
> baseline yet. Persistent cache had not been re-balanced yet but non
> persistent cache had been done.
> Distributed join is not a good choice for us because of performance. I'm
> thinking about those solutions :
>
> 1. Don't sent request out of baseline node
> 2. Don't automatically re-balanced none persistent caches when cluster is
> persistent enabled.
> 3. Add node to baseline right after it join cluster.
> Is it possible to do this ?
>
> On Thu, Mar 21, 2019 at 6:15 PM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> Ignite assumes that your data is affinity collocated. While rebalancing
>> is in process, obviously this is not the case.
>>
>> I think you can have this problem go away by setting
>> distributedJoins=true. Have you tried that? Note that there is a
>> potentially large performance impact.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> чт, 21 мар. 2019 г. в 14:08, Tâm Nguyễn Mạnh > >:
>>
>>> Hi Igniters,
>>>
>>> Today i got this exception during testing cluster scaling.
>>> I have 2 caches X and Y in cluster, X is persistent and Y is none
>>> persistent.
>>>
>>> 1. I start 1st node A and activate cluster and populate data for X and Y
>>> 2. I start 2nd node B but not yet update topology. Few seconds later Y
>>> had been auto rebalanced.
>>>
>>> In A we have both X and Y data but in B we only have Y. Exception will
>>> be thrown when I try to join X and Y in sql query: Caches have distinct
>>> sets of data nodes [cache1=X, cache2=Y]
>>> AB
>>> == ==
>>> XY
>>> Y
>>> .
>>> I think it because of none persistent data is automaticly rebalanced
>>> whenver a node is join into cluster (event not in baseline yet) but not
>>> persistent data. Persistent data is only rebalanced whenever baseline is
>>> changed. Is it correct? And how do i prevent this exception during scaling ?
>>>
>>> --
>>> Thanks & Best Regards
>>>
>>> Tam, Nguyen Manh
>>>
>>>
>
> --
> Thanks & Best Regards
>
> Tam, Nguyen Manh
>
>

Re: Caches have distinct sets of data nodes

[Tâm Nguyễn Mạnh]

Hi,

My data is affinity collocated. The issue is my 2nd node is not in a
baseline yet. Persistent cache had not been re-balanced yet but non
persistent cache had been done.
Distributed join is not a good choice for us because of performance. I'm
thinking about those solutions :

1. Don't sent request out of baseline node
2. Don't automatically re-balanced none persistent caches when cluster is
persistent enabled.
3. Add node to baseline right after it join cluster.
Is it possible to do this ?

On Thu, Mar 21, 2019 at 6:15 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> Ignite assumes that your data is affinity collocated. While rebalancing is
> in process, obviously this is not the case.
>
> I think you can have this problem go away by setting
> distributedJoins=true. Have you tried that? Note that there is a
> potentially large performance impact.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 21 мар. 2019 г. в 14:08, Tâm Nguyễn Mạnh :
>
>> Hi Igniters,
>>
>> Today i got this exception during testing cluster scaling.
>> I have 2 caches X and Y in cluster, X is persistent and Y is none
>> persistent.
>>
>> 1. I start 1st node A and activate cluster and populate data for X and Y
>> 2. I start 2nd node B but not yet update topology. Few seconds later Y
>> had been auto rebalanced.
>>
>> In A we have both X and Y data but in B we only have Y. Exception will be
>> thrown when I try to join X and Y in sql query: Caches have distinct sets
>> of data nodes [cache1=X, cache2=Y]
>> AB
>> == ==
>> XY
>> Y
>> .
>> I think it because of none persistent data is automaticly rebalanced
>> whenver a node is join into cluster (event not in baseline yet) but not
>> persistent data. Persistent data is only rebalanced whenever baseline is
>> changed. Is it correct? And how do i prevent this exception during scaling ?
>>
>> --
>> Thanks & Best Regards
>>
>> Tam, Nguyen Manh
>>
>>

-- 
Thanks & Best Regards

Tam, Nguyen Manh

Re: Ignite 2.7 Errors

[Andrey Kuznetsov]

Hi, Philip!

There should be a stacktrace of the blocked worker itself in the log, with
warn level, before the message you cite, but after "Blocked system-critical
thread has been detected." Could you please share that trace? It can help
to understand failure cause.

Best regards,
Andrey Kuznetsov.

чт, 21 марта 2019, 18:34 Ilya Kasnacheev ilya.kasnach...@gmail.com:

> Hello!
>
> With NoOp handler this should be a purely cosmetic message.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> чт, 21 мар. 2019 г. в 18:15, Philip Wu :
>
>> Thanks, llya!
>>
>> Actually it happened in PROD system again last night ... even with
>> NoOpFailureHandler.
>>
>> I am rolling back to Ignite 2.5 or 2.6 for now. Thanks!
>>
>> 2019-03-20 22:28:45,044 SEVERE [ (tcp-disco-msg-worker-#2%XXXGrid%)]
>> Critical system error detected. Will be handled accordingly to configured
>> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
>> [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext
>> [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker
>> [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=XXXGrid,
>> finished=false, heartbeatTs=1553137996031]]]: class
>> org.apache.ignite.IgniteException: GridWorker
>> [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=XXXGrid,
>> finished=false, heartbeatTs=1553137996031]
>> at
>>
>> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831)
>> at
>>
>> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826)
>> at
>>
>> org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233)
>> at
>>
>> org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297)
>> at
>>
>> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663)
>> at
>>
>> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181)
>> at
>>
>> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
>> at
>> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
>> at
>>
>> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
>> at
>> org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>>
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>

Re: Ignite 2.7 Errors

[Ilya Kasnacheev]

Hello!

With NoOp handler this should be a purely cosmetic message.

Regards,
-- 
Ilya Kasnacheev


чт, 21 мар. 2019 г. в 18:15, Philip Wu :

> Thanks, llya!
>
> Actually it happened in PROD system again last night ... even with
> NoOpFailureHandler.
>
> I am rolling back to Ignite 2.5 or 2.6 for now. Thanks!
>
> 2019-03-20 22:28:45,044 SEVERE [ (tcp-disco-msg-worker-#2%XXXGrid%)]
> Critical system error detected. Will be handled accordingly to configured
> handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
> [ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext
> [type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker
> [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=XXXGrid,
> finished=false, heartbeatTs=1553137996031]]]: class
> org.apache.ignite.IgniteException: GridWorker
> [name=grid-nio-worker-tcp-comm-1, igniteInstanceName=XXXGrid,
> finished=false, heartbeatTs=1553137996031]
> at
>
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831)
> at
>
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826)
> at
>
> org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233)
> at
>
> org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297)
> at
>
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663)
> at
>
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181)
> at
>
> org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
> at
>
> org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
> at
> org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite 2.7 Errors

[Philip Wu]

Thanks, llya!

Actually it happened in PROD system again last night ... even with
NoOpFailureHandler.

I am rolling back to Ignite 2.5 or 2.6 for now. Thanks!

2019-03-20 22:28:45,044 SEVERE [ (tcp-disco-msg-worker-#2%XXXGrid%)]
Critical system error detected. Will be handled accordingly to configured
handler [hnd=NoOpFailureHandler [super=AbstractFailureHandler
[ignoredFailureTypes=[SYSTEM_WORKER_BLOCKED]]], failureCtx=FailureContext
[type=SYSTEM_WORKER_BLOCKED, err=class o.a.i.IgniteException: GridWorker
[name=grid-nio-worker-tcp-comm-1, igniteInstanceName=XXXGrid,
finished=false, heartbeatTs=1553137996031]]]: class
org.apache.ignite.IgniteException: GridWorker
[name=grid-nio-worker-tcp-comm-1, igniteInstanceName=XXXGrid,
finished=false, heartbeatTs=1553137996031]
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1831)
at
org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance$2.apply(IgnitionEx.java:1826)
at
org.apache.ignite.internal.worker.WorkersRegistry.onIdle(WorkersRegistry.java:233)
at
org.apache.ignite.internal.util.worker.GridWorker.onIdle(GridWorker.java:297)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.lambda$new$0(ServerImpl.java:2663)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorker.body(ServerImpl.java:7181)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$RingMessageWorker.body(ServerImpl.java:2700)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$MessageWorkerThread.body(ServerImpl.java:7119)
at
org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Changing of eviction policies for a cache at runtime

[Ilya Kasnacheev]

Hello!

Nope! Maybe in 3.0 or even later.

You can't change cache at runtime, however you can do
cache.withExpiryPolicy() when using that one.

Regards,
-- 
Ilya Kasnacheev


чт, 21 мар. 2019 г. в 16:21, Venkata Bhagavatula :

> Hi,
>
> Is it possible to change the eviction policies of a cache at runtime in
> Ignite?
> I came across wikipage :
> https://cwiki.apache.org/confluence/display/IGNITE/Allow+Configuration+Settings+Change+At+Runtime
>
>
> Is this implemented? or any plans for this?
>
> Thanks n Regards,
> Chalapathi.
>

Changing of eviction policies for a cache at runtime

[Venkata Bhagavatula]

Hi,

Is it possible to change the eviction policies of a cache at runtime in
Ignite?
I came across wikipage :
https://cwiki.apache.org/confluence/display/IGNITE/Allow+Configuration+Settings+Change+At+Runtime


Is this implemented? or any plans for this?

Thanks n Regards,
Chalapathi.

Re: Blog post "Introduction to the Apache(R) Ignite™ community structure"

[Dmitriy Pavlov]

Sorry for the late reply. Contributors are not ranked so weight is not
measured. The community always prefer to build consensus. If consensus is
not reached (this happens time-to-time), then feature or change should not
appear in the product.

By the way, Apache PMCs may veto changes with a technical justification
explaining why change is bad.

A final word in project development has PMC (it is a committee consisting
of all its members). Practically Apache Ignite Community prefers wider
consensus - from all contributors.

In case something is going wrong, there is our last resort:
https://www.apache.org/board/escalation  But I don't remember any
escalations.

Priority of implementing features is solved in another way: if someone
wants to implement something and the community does not reject it, so why
not? There is no single priority for all contributors.

ср, 17 окт. 2018 г. в 12:09, zaleslaw :

> [For discussion] It's interesting to know more about possible conflicts:
> when
> a few persons are involved in contribution as a commiters, for example, and
> they have different opinions about next steps in roadmap implementation or
> about certain feature. How to measure correctly their weights? One of them
> doing small bug fixes, another does large features. They are not ranked,
> except amount of commits or code lines in github. Who can say last word?
>
> Imagine, only they are both understand something in this feature and its
> priority. Votes couldn't help, right? How they should solve their conflict?
> Does you have any cases in your experience?
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Licencing cost

[Dmitriy Pavlov]

Hi,

Apache Ignite is always free for users without dependence on cores count or
size of the cluster. Apache Ignite it is developed and supported by Apache
Ignite Community, including users providing patches, enthusiasts and other
individuals.

Sincerely,
Dmitriy Pavlov

чт, 21 мар. 2019 г., 14:02 Stephen Darlington <
stephen.darling...@gridgain.com>:

> You should direct this question to GridGain rather than the Ignite open
> source community.
>
> Regards,
> Stephen
>
> > On 20 Mar 2019, at 16:11, austin solomon 
> wrote:
> >
> > Hi,
> >
> > Does the gridgain licensing cost vary depending on the number of physical
> > cores of each node?
> >
> > Can anyone tell me.
> >
> > Thanks,
> > Austin
> >
> >
> >
> > --
> > Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>
>
>

Re: Access a cache loaded by DataStreamer with SQL

[Ilya Kasnacheev]

Hello!

I think you need to have different types for keys too.

Regards,
-- 
Ilya Kasnacheev


ср, 20 мар. 2019 г. в 21:00, Mike Needham :

> I guess I am not understanding how to build this for multiple tables in a
> cache that can be loaded using the datastreamer and are queryable from
> DBeaver or tableau.
>
> I changed the code to be
>
>  IgniteCache testCache =
> ignite.getOrCreateCache(new CacheConfiguration<>("MAIN")
> .setIndexedTypes(Long.class, Employee.class)
> .setIndexedTypes(Long.class, Department.class)
> );
> Employee e = new Employee(1, "Test", 123.34f, 3);
> try (IgniteDataStreamer ds =
> ignite.dataStreamer("MAIN")) {
> ds.addData(1l, e);
> }
> Department d = new Department(1, "Main", 12, 3);
> try (IgniteDataStreamer ds2 =
> ignite.dataStreamer("MAIN")) {
> ds2.addData(1l, d);
> }
>
>
>
> which created the two in the cache, but does not load anything to the
> department table.
>
> On Wed, Mar 20, 2019 at 10:01 AM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello!
>>
>> I don't understand what you are doing here. Why do you have two employee
>> tables here? What is desired table structure?
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> ср, 20 мар. 2019 г. в 16:44, Mike Needham :
>>
>>> I have that part, what I dont understand is how I can create multiple
>>> "Tables" within a Cache(Schema)?  I have the following code that is using a
>>> simple Employee Class.
>>>
>>> IgniteCache testCache =
>>> ignite.getOrCreateCache(new CacheConfiguration<>("MAIN")
>>> .setIndexedTypes(Long.class, Employee.class)
>>> .setQueryEntities(Collections.singleton(
>>> new QueryEntity(Integer.class,
>>> String.class).setTableName("EMPLOYEE";
>>> Employee e = new Employee(1, "Test", 123.34f, 3);
>>> try (IgniteDataStreamer ds =
>>> ignite.dataStreamer("MAIN")) {
>>> ds.addData(1l, e);
>>> }
>>>
>>> how would one go about adding a second "Table" to the MAIN cache so that
>>> it is queryable from DBeaver or other tools?
>>>
>>>
>>> On Wed, Mar 20, 2019 at 4:29 AM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 Please take a look at
 https://apacheignite.readme.io/docs/cache-queries#section-query-configuration-by-annotations

 Regards,
 --
 Ilya Kasnacheev


 вт, 19 мар. 2019 г. в 20:25, Mike Needham :

> Do you have an example of how that could be done.  I am struggling to
> figure out how to set this up.
>
> On Mon, Mar 18, 2019 at 2:00 AM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello!
>>
>> The best approach is to use .setIndexedTypes() instead of
>> setQueryEntities(), and annotate complex types in question with
>> @QuerySqlField.
>> This way you can then pour those types into cache and it will work
>> transparently.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 15 мар. 2019 г. в 18:28, Mike Needham :
>>
>>> Perfect, now the next question is how would you do this for a more
>>> complex object/table?  Either one defined in a separate object or via 
>>> SQL
>>> DDL?
>>>
>>> On Fri, Mar 15, 2019 at 9:05 AM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 You will have to specify schema name (or cache name?) in ALLCAPS
 when creating cache.

 Regards,
 --
 Ilya Kasnacheev


 пт, 15 мар. 2019 г. в 16:45, Mike Needham :

> I see.  did not have the "person" for the schema.  Is there a way
> to not have the quotes around that?
>
> On Fri, Mar 15, 2019 at 7:59 AM ilya.kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello!
>>
>> Definitely works for me in DBeaver with this exact code:
>>
>> <
>> http://apache-ignite-users.70518.x6.nabble.com/file/t1312/dbeaver-tables.png>
>>
>>
>> Some of DBeaver's introspection does not work but statements are
>> solid.
>>
>> Regards,
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>
> --
> *Some days it just not worth chewing through the restraints*
>

>>>
>>> --
>>> *Some days it just not worth chewing through the restraints*
>>>
>>
>
> --
> *Some days it just not worth chewing through the restraints*
>

>>>
>>> --
>>> *Some days it just not worth chewing through the restraints*
>>>
>>
>
> --
> *Some days it just not worth chewing through the 

Re: Caches have distinct sets of data nodes

[Ilya Kasnacheev]

Hello!

Ignite assumes that your data is affinity collocated. While rebalancing is
in process, obviously this is not the case.

I think you can have this problem go away by setting distributedJoins=true.
Have you tried that? Note that there is a potentially large performance
impact.

Regards,
-- 
Ilya Kasnacheev


чт, 21 мар. 2019 г. в 14:08, Tâm Nguyễn Mạnh :

> Hi Igniters,
>
> Today i got this exception during testing cluster scaling.
> I have 2 caches X and Y in cluster, X is persistent and Y is none
> persistent.
>
> 1. I start 1st node A and activate cluster and populate data for X and Y
> 2. I start 2nd node B but not yet update topology. Few seconds later Y had
> been auto rebalanced.
>
> In A we have both X and Y data but in B we only have Y. Exception will be
> thrown when I try to join X and Y in sql query: Caches have distinct sets
> of data nodes [cache1=X, cache2=Y]
> AB
> == ==
> XY
> Y
> .
> I think it because of none persistent data is automaticly rebalanced
> whenver a node is join into cluster (event not in baseline yet) but not
> persistent data. Persistent data is only rebalanced whenever baseline is
> changed. Is it correct? And how do i prevent this exception during scaling ?
>
> --
> Thanks & Best Regards
>
> Tam, Nguyen Manh
>
>

Caches have distinct sets of data nodes

[Tâm Nguyễn Mạnh]

Hi Igniters,

Today i got this exception during testing cluster scaling.
I have 2 caches X and Y in cluster, X is persistent and Y is none
persistent.

1. I start 1st node A and activate cluster and populate data for X and Y
2. I start 2nd node B but not yet update topology. Few seconds later Y had
been auto rebalanced.

In A we have both X and Y data but in B we only have Y. Exception will be
thrown when I try to join X and Y in sql query: Caches have distinct sets
of data nodes [cache1=X, cache2=Y]
AB
== ==
XY
Y
.
I think it because of none persistent data is automaticly rebalanced
whenver a node is join into cluster (event not in baseline yet) but not
persistent data. Persistent data is only rebalanced whenever baseline is
changed. Is it correct? And how do i prevent this exception during scaling ?

-- 
Thanks & Best Regards

Tam, Nguyen Manh

Re: Licencing cost

[Stephen Darlington]

You should direct this question to GridGain rather than the Ignite open source 
community.

Regards,
Stephen

> On 20 Mar 2019, at 16:11, austin solomon  wrote:
> 
> Hi,
> 
> Does the gridgain licensing cost vary depending on the number of physical
> cores of each node?
> 
> Can anyone tell me.
> 
> Thanks,
> Austin
> 
> 
> 
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Tomcat version for Web session clustering in Ignite 2.7

[Ilya Kasnacheev]

Hello!

Why don't you try, share results with us?

Regards,
-- 
Ilya Kasnacheev


чт, 21 мар. 2019 г. в 13:08, Rout, Biswajeet :

> Hi,
>
> Can I know the tomcat version it supports for Web Session Clustering in
> Ignite 2.7.
> I see in the document it has mentioned that, it has tested in tomcat 6 and
> 7. Does it supports tomcat 8?
>
> [image: image.png]
>
> --
>
> 
>
> Biswajeet
> Rout
> DTIX , DELPHI
>
> O +18136176308 | M 9703349977
> biswajeet.r...@verizon.com
>

Tomcat version for Web session clustering in Ignite 2.7

[Rout, Biswajeet]

Hi,

Can I know the tomcat version it supports for Web Session Clustering in
Ignite 2.7.
I see in the document it has mentioned that, it has tested in tomcat 6 and
7. Does it supports tomcat 8?

[image: image.png]

-- 



Biswajeet
Rout
DTIX , DELPHI

O +18136176308 | M 9703349977
biswajeet.r...@verizon.com

Re: Ignite 2.7 Errors

[Ilya Kasnacheev]

Hello!

We are tweaking this mechanism so maybe you will wish to reenable it in 2.8.

Regards,
-- 
Ilya Kasnacheev


ср, 20 мар. 2019 г. в 18:30, Philip Wu :

> Thank you, IIya!
>
> We ended up using
>
> cfg.setFailureHandler(new NoOpFailureHandler());
>
> it silenced the errors and no more stack dumps, etc. and it seems to work
> like in 2.5 and 2.6, with no other changes.
>
> I am still curious if in the future I can take that line out if 2.7 is more
> stable or 2.8.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: How to disable replication over multicast for development nodes?

[ibelyakov]

Yes, you're right. 

If you wish to use specific port just add it to the value:
127.0.0.1:47500

or ports range, in case you're going to use multiple nodes in the cluster:
127.0.0.1:47500..47509

Regards,
Igor



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Support User defined aggregate function ?

[Tâm Nguyễn Mạnh]

 Hi Igniters,

Can we support user defined aggreate function ?

-- 
Thanks & Best Regards

Tam, Nguyen Manh

Re: Support User defined aggregate function ?

[Tâm Nguyễn Mạnh]

Hi,

Sorry for that
I will move this to u...@ignite.apache.or. 

Thank you

On Wed, Mar 20, 2019 at 8:45 PM Ilya Kasnacheev 
wrote:

> Hello!
>
> As far as I know there is no such support. It is recommended to use
> job/task API instead:
> https://apacheignite.readme.io/docs/compute-tasks
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> ср, 20 мар. 2019 г. в 09:24, Tâm Nguyễn Mạnh :
>
>> Hi Igniters,
>>
>> Can we support user defined aggreate function ?
>>
>> --
>> Thanks & Best Regards
>>
>> Tam, Nguyen Manh
>>
>

-- 
Thanks & Best Regards

Tam, Nguyen Manh