Re: Ignite communicating with non ignite servers

[Ilya Kasnacheev]

Hello!

I don't think it should cause heap dumps. Here you are showing just a
warning. This warning may be ignored.

It's outside of scope of Apache Ignite to disable something else to try
connecting to it. If you have invasive security port scanning, you will
expect to see warnings/errors in the logs of any network application.

Regards,
-- 
Ilya Kasnacheev


вт, 22 сент. 2020 г. в 16:26, ignite_user2016 :

> We have SSL enabled on all servers but some how it s trying to attempt
> connection on SSL causing heap dumps. Is there a way to disable to external
> server try connecting to ignite ?
>
> 2020-09-10 22:52:47,029 WARN [grid-nio-worker-tcp-comm-3-#27%NAME_GRID%]
> org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi Client
> disconnected abruptly due to network connection loss or because the
> connection was left open on application shutdown. [cls=class
> o.a.i.i.util.nio.GridNioException, msg=Failed to decode SSL data:
> GridSelectorNioSessionImpl [worker=DirectNioClientWorker
> [super=AbstractNioClientWorker [idx=3, bytesRcvd=13315002728, bytesSent=0,
> bytesRcvd0=18, bytesSent0=0, select=true, super=GridWorker
> [name=grid-nio-worker-tcp-comm-3, igniteInstanceName=WebGrid,
> finished=false, heartbeatTs=1599796365124, hashCode=1230825885,
> interrupted=false, runner=grid-nio-worker-tcp-comm-3-#27%WebGrid%]]],
> writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
> readBuf=java.nio.DirectByteBuffer[pos=18 lim=18 cap=32768],
> inRecovery=null,
> outRecovery=null, closeSocket=true,
>
> outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1
> ,
> super=GridNioSessionImpl [locAddr=/*IG_SERVER1*:47101, rmtAddr=/*SEC_SCAN*
> SERVER:52082, createTime=1599796365124, closeTime=0, bytesSent=0,
> bytesRcvd=18, bytesSent0=0, bytesRcvd0=18, sndSchedTime=1599796365124,
> lastSndTime=1599796365124, lastRcvTime=1599796367026, readsPaused=false,
> filterChain=FilterChain[filters=[GridNioCodecFilter
> [parser=o.a.i.i.util.nio.GridDirectParser@20ca1d6a, directMode=true],
> GridConnectionBytesVerifyFilter, SSL filter], accepted=true,
> markedForClose=false]]]
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: [External]Re: how to achieve timeout for get/put/remove operations

[Ilya Kasnacheev]

Hello!

Futures will accept a timeout value in their get() method and will only
wait for that timeout, you do not have to count milliseconds by hand.

Regards,
-- 
Ilya Kasnacheev


вт, 22 сент. 2020 г. в 15:31, Kamlesh Joshi :

> Thanks ilya. Does that mean, we have to specifically maintain a time
> counter and act accordingly on it?
>
>
>
> I was hoping, if there is any API where we could just pass timeout value
> as well, as a parameter and Ignite would handle internally.
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
> *From:* Ilya Kasnacheev 
> *Sent:* 22 September 2020 17:10
> *To:* user@ignite.apache.org
> *Subject:* [External]Re: how to achieve timeout for get/put/remove
> operations
>
>
>
> The e-mail below is from an external source. Please do not open
> attachments or click links from an unknown or suspicious origin.
>
> Hello!
>
>
>
> You can use e.g. putAsync() and then wait on the future for the specified
> time only. If it did not complete, take your actions.
>
>
>
> Regards,
>
> --
>
> Ilya Kasnacheev
>
>
>
>
>
> вт, 22 сент. 2020 г. в 14:27, Kamlesh Joshi :
>
> Hi Igniters,
>
>
>
> How to achieve/get timeout error for particular operation
> (get/put/remove), if it crosses certain timeout value then I can take
> certain action based on that. Is there any Ignite API readily available for
> this?
>
>
>
> Any pointers would be helpful !
>
>
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>

Re: Ignite communicating with non ignite servers

[ignite_user2016]

We have SSL enabled on all servers but some how it s trying to attempt
connection on SSL causing heap dumps. Is there a way to disable to external
server try connecting to ignite ? 

2020-09-10 22:52:47,029 WARN [grid-nio-worker-tcp-comm-3-#27%NAME_GRID%]
org.apache.ignite.spi.communication.tcp.TcpCommunicationSpi Client
disconnected abruptly due to network connection loss or because the
connection was left open on application shutdown. [cls=class
o.a.i.i.util.nio.GridNioException, msg=Failed to decode SSL data:
GridSelectorNioSessionImpl [worker=DirectNioClientWorker
[super=AbstractNioClientWorker [idx=3, bytesRcvd=13315002728, bytesSent=0,
bytesRcvd0=18, bytesSent0=0, select=true, super=GridWorker
[name=grid-nio-worker-tcp-comm-3, igniteInstanceName=WebGrid,
finished=false, heartbeatTs=1599796365124, hashCode=1230825885,
interrupted=false, runner=grid-nio-worker-tcp-comm-3-#27%WebGrid%]]],
writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
readBuf=java.nio.DirectByteBuffer[pos=18 lim=18 cap=32768], inRecovery=null,
outRecovery=null, closeSocket=true,
outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1,
super=GridNioSessionImpl [locAddr=/*IG_SERVER1*:47101, rmtAddr=/*SEC_SCAN*
SERVER:52082, createTime=1599796365124, closeTime=0, bytesSent=0,
bytesRcvd=18, bytesSent0=0, bytesRcvd0=18, sndSchedTime=1599796365124,
lastSndTime=1599796365124, lastRcvTime=1599796367026, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=o.a.i.i.util.nio.GridDirectParser@20ca1d6a, directMode=true],
GridConnectionBytesVerifyFilter, SSL filter], accepted=true,
markedForClose=false]]]



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: [External]Re: how to achieve timeout for get/put/remove operations

[Kamlesh Joshi]

Hi Stephen,

Earlier we faced issues, where cluster got completely frozen and due to some 
issues (may be one node failed) it took lot of time to respond (all the read 
requests were piled up). So, we are trying to build a solution around this 
problem.

To conclude this, we might have to maintain our own timeouts and use getAsync 
API that is the only way we can achieve this (as we are not using any 
transactions).

Thanks and Regards,
Kamlesh Joshi

From: Stephen Darlington 
Sent: 22 September 2020 18:11
To: user@ignite.apache.org
Subject: Re: [External]Re: how to achieve timeout for get/put/remove operations


The e-mail below is from an external source. Please do not open attachments or 
click links from an unknown or suspicious origin.


Are you actually experiencing this problem? Or is this theoretical?

As Ilya says, there’s no built-in timeouts for basic operations like get/put 
but then, if they take very long Something Bad likely has happened.

There are timeouts available for operations that can take a long time. For 
example, methods in Transaction and IgniteLock both have timeouts.

Regards,
Stephen


On 22 Sep 2020, at 13:30, Kamlesh Joshi 
mailto:kamlesh.jo...@ril.com>> wrote:

Thanks ilya. Does that mean, we have to specifically maintain a time counter 
and act accordingly on it?

I was hoping, if there is any API where we could just pass timeout value as 
well, as a parameter and Ignite would handle internally.

Thanks and Regards,
Kamlesh Joshi

From: Ilya Kasnacheev 
mailto:ilya.kasnach...@gmail.com>>
Sent: 22 September 2020 17:10
To: user@ignite.apache.org
Subject: [External]Re: how to achieve timeout for get/put/remove operations

The e-mail below is from an external source. Please do not open attachments or 
click links from an unknown or suspicious origin.
Hello!

You can use e.g. putAsync() and then wait on the future for the specified time 
only. If it did not complete, take your actions.

Regards,
--
Ilya Kasnacheev


вт, 22 сент. 2020 г. в 14:27, Kamlesh Joshi 
mailto:kamlesh.jo...@ril.com>>:
Hi Igniters,

How to achieve/get timeout error for particular operation (get/put/remove), if 
it crosses certain timeout value then I can take certain action based on that. 
Is there any Ignite API readily available for this?

Any pointers would be helpful !


Thanks and Regards,
Kamlesh Joshi

"Confidentiality Warning: This message and any attachments are intended only 
for the use of the intended recipient(s). 
are confidential and may be privileged. If you are not the intended recipient. 
you are hereby notified that any 
review. re-transmission. conversion to hard copy. copying. circulation or other 
use of this message and any attachments is 
strictly prohibited. If you are not the intended recipient. please notify the 
sender immediately by return email. 
and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure 
no viruses are present in this email. 
The company cannot accept responsibility for any loss or damage arising from 
the use of this email or attachment."

Re: [External]Re: how to achieve timeout for get/put/remove operations

[Stephen Darlington]

Are you actually experiencing this problem? Or is this theoretical? 

As Ilya says, there’s no built-in timeouts for basic operations like get/put 
but then, if they take very long Something Bad likely has happened.

There are timeouts available for operations that can take a long time. For 
example, methods in Transaction and IgniteLock both have timeouts.

Regards,
Stephen

> On 22 Sep 2020, at 13:30, Kamlesh Joshi  wrote:
> 
> Thanks ilya. Does that mean, we have to specifically maintain a time counter 
> and act accordingly on it?
>  
> I was hoping, if there is any API where we could just pass timeout value as 
> well, as a parameter and Ignite would handle internally.
>  
> Thanks and Regards,
> Kamlesh Joshi
>  
> From: Ilya Kasnacheev  > 
> Sent: 22 September 2020 17:10
> To: user@ignite.apache.org 
> Subject: [External]Re: how to achieve timeout for get/put/remove operations
>  
> The e-mail below is from an external source. Please do not open attachments 
> or click links from an unknown or suspicious origin.
> 
> Hello!
>  
> You can use e.g. putAsync() and then wait on the future for the specified 
> time only. If it did not complete, take your actions.
>  
> Regards,
> -- 
> Ilya Kasnacheev
>  
>  
> вт, 22 сент. 2020 г. в 14:27, Kamlesh Joshi  >:
> Hi Igniters,
>  
> How to achieve/get timeout error for particular operation (get/put/remove), 
> if it crosses certain timeout value then I can take certain action based on 
> that. Is there any Ignite API readily available for this?
>  
> Any pointers would be helpful !
>  
>  
> Thanks and Regards,
> Kamlesh Joshi
>  

RE: [External]Re: how to achieve timeout for get/put/remove operations

[Kamlesh Joshi]

Thanks ilya. Does that mean, we have to specifically maintain a time counter 
and act accordingly on it?

I was hoping, if there is any API where we could just pass timeout value as 
well, as a parameter and Ignite would handle internally.

Thanks and Regards,
Kamlesh Joshi

From: Ilya Kasnacheev 
Sent: 22 September 2020 17:10
To: user@ignite.apache.org
Subject: [External]Re: how to achieve timeout for get/put/remove operations


The e-mail below is from an external source. Please do not open attachments or 
click links from an unknown or suspicious origin.
Hello!

You can use e.g. putAsync() and then wait on the future for the specified time 
only. If it did not complete, take your actions.

Regards,
--
Ilya Kasnacheev


вт, 22 сент. 2020 г. в 14:27, Kamlesh Joshi 
mailto:kamlesh.jo...@ril.com>>:
Hi Igniters,

How to achieve/get timeout error for particular operation (get/put/remove), if 
it crosses certain timeout value then I can take certain action based on that. 
Is there any Ignite API readily available for this?

Any pointers would be helpful !


Thanks and Regards,
Kamlesh Joshi


"Confidentiality Warning: This message and any attachments are intended only 
for the use of the intended recipient(s), are confidential and may be 
privileged. If you are not the intended recipient, you are hereby notified that 
any review, re-transmission, conversion to hard copy, copying, circulation or 
other use of this message and any attachments is strictly prohibited. If you 
are not the intended recipient, please notify the sender immediately by return 
email and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure 
no viruses are present in this email. The company cannot accept responsibility 
for any loss or damage arising from the use of this email or attachment."
"Confidentiality Warning: This message and any attachments are intended only 
for the use of the intended recipient(s). 
are confidential and may be privileged. If you are not the intended recipient. 
you are hereby notified that any 
review. re-transmission. conversion to hard copy. copying. circulation or other 
use of this message and any attachments is 
strictly prohibited. If you are not the intended recipient. please notify the 
sender immediately by return email. 
and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure 
no viruses are present in this email. 
The company cannot accept responsibility for any loss or damage arising from 
the use of this email or attachment."

Re: how to achieve timeout for get/put/remove operations

[Ilya Kasnacheev]

Hello!

You can use e.g. putAsync() and then wait on the future for the specified
time only. If it did not complete, take your actions.

Regards,
-- 
Ilya Kasnacheev


вт, 22 сент. 2020 г. в 14:27, Kamlesh Joshi :

> Hi Igniters,
>
>
>
> How to achieve/get timeout error for particular operation
> (get/put/remove), if it crosses certain timeout value then I can take
> certain action based on that. Is there any Ignite API readily available for
> this?
>
>
>
> Any pointers would be helpful !
>
>
>
>
>
> *Thanks and Regards,*
>
> *Kamlesh Joshi*
>
>
>
>
> "*Confidentiality Warning*: This message and any attachments are intended
> only for the use of the intended recipient(s), are confidential and may be
> privileged. If you are not the intended recipient, you are hereby notified
> that any review, re-transmission, conversion to hard copy, copying,
> circulation or other use of this message and any attachments is strictly
> prohibited. If you are not the intended recipient, please notify the sender
> immediately by return email and delete this message and any attachments
> from your system.
>
> *Virus Warning:* Although the company has taken reasonable precautions to
> ensure no viruses are present in this email. The company cannot accept
> responsibility for any loss or damage arising from the use of this email or
> attachment."
>

how to achieve timeout for get/put/remove operations

[Kamlesh Joshi]

Hi Igniters,

How to achieve/get timeout error for particular operation (get/put/remove), if 
it crosses certain timeout value then I can take certain action based on that. 
Is there any Ignite API readily available for this?

Any pointers would be helpful !


Thanks and Regards,
Kamlesh Joshi

"Confidentiality Warning: This message and any attachments are intended only 
for the use of the intended recipient(s). 
are confidential and may be privileged. If you are not the intended recipient. 
you are hereby notified that any 
review. re-transmission. conversion to hard copy. copying. circulation or other 
use of this message and any attachments is 
strictly prohibited. If you are not the intended recipient. please notify the 
sender immediately by return email. 
and delete this message and any attachments from your system.

Virus Warning: Although the company has taken reasonable precautions to ensure 
no viruses are present in this email. 
The company cannot accept responsibility for any loss or damage arising from 
the use of this email or attachment."

Re: Ignite communicating with non ignite servers

[Ilya Kasnacheev]

Hello!

Well, you could enable SSL on all ports, in this case you can block off the
security scanner.

Regards,
-- 
Ilya Kasnacheev


пн, 21 сент. 2020 г. в 19:03, ignite_user2016 :

> Recently, we migrated ignite to JDK11, all works well except when we run
> our
> security scan, ignite node tries to connect on that servers and result in
> out of memory and heap dump errors.
>
> Is it possible where we can stop that scan server connecting to ignite ?
>
> Any configuration ?
>
> help is much appreciated.
>
> And I have observed that ignite visor is also broken where it cant give us
> the states for nodes, memory and CPU.
>
> Thanks..
> Rishi
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite Thin Client - IDLE TIMEOUT

[Pavel Tupitsyn]

Hi,

You can set the idle timeout on the server side with
ClientConnectorConfiguration.setIdleTimeout

On Tue, Sep 22, 2020 at 8:16 AM Mahesh Renduchintala <
mahesh.renduchint...@aline-consulting.com> wrote:

> Hi,
>
> is there a way one can set IdleTImeOut for a thin client?
>
> I did not see such configuration in ClientConfiguration.
>
> regards
> Mahesh
>
>
>
>

Re: Apache ignite statefulsets pods abruptly restarts

[Stephen Darlington]

The on-heap cache is in addition to the off-heap cache.

8Gb (off-heap)
+ 8Gb (on-heap cache)
+ ~4Gb (heap space for running Ignite)
= ~ 20Gb

Way more than the 11Gb you have allocated.

Three steps:
Turn off the on-heap cache
Decrease the data region to maybe 6Gb (in Ignite configuration)
Allocate around 4Gb to your heap (by setting JVM_OPTS)

> On 22 Sep 2020, at 06:10, Sanjaya Kumar Sahoo  wrote:
> 
> Hi, 
> 
> Thanks for your reply, we have set on heap as below in ignite configuration 
> xml,  
> https://apacheignite.readme.io/docs/memory-configuration#section-on-heap-caching
>  
> 
> 
> Basically we have set as true in  xml configuration as   name="onheapCacheEnabled" value="true">.
> 
> The idea to move on heap with persistence enable is to maximize the latency 
> of query use case
> 
> Am I doing anything wrong here ?
> 
> 
> Thanks,
> Sanjaya 
> 
> On Mon, Sep 21, 2020 at 10:11 PM Evgenii Zhuravlev  > wrote:
> There is no such thing as "on heap cache only.". It's possible to enable an 
> additional cache level in heap, but it still will be storing all data in the 
> off heap. So, right now you need at least 10.25+8gb+ Checkpoint buffer size 
> for your Ignite node.
> 
> Evgenii
> 
> пн, 21 сент. 2020 г. в 09:29, Sanjaya  >:
> Hi All, 
> 
> In out production environment, ignite v2.8.1 is install as a kubernetes
> stateful sets pods inside Azure Kubernetes cluster. There are 2 pods
> running.
> 
> Ignite is persistence enabled, with on heap cache only.
> 
> The pod is running with below guaranteed resources
> Memory : 11 GB
> CPU: 3 core
> 
> Ignite is given heap as : 10.25 GB  
> The total data region size is as : 8GB
> 
> 
> We are getting below error when 2 caches joins each other without any
> indexing, one of PODS jvm simply restarts, we are not sure whats going on.
> The usecase is that ignite cache grid hold all master data and gets loads
> from postgres, and plannned to being called from 30+ differen pods for same
> kind of queries.
> 
> We are completely stuck in this usecase, and thinking if ignite is right for
> this usecase. 
> 
> 
> The stack trace as is below
> =
>   AND (A__Z0.ASSET_UID = B__Z1.ASSET_UID
> ORDER BY 9, 1]
> [09:43:10,370][WARNING][jvm-pause-detector-worker][IgniteKernal] Possible
> too long JVM pause: 872 milliseconds.
> [09:43:10,630][WARNING][client-connector-#52][IgniteH2Indexing] Long running
> query is finished [time=4316ms, type=MAP, distributedJoin=false,
> enforceJoinOrder=true, lazy=false, schema=CRTX, node=TcpDiscoveryNode
> [id=4093191a-f958-4b4b-bf55-ae774d450fa2,
> consistentId=4ed84cd6-d24c-4b2e-b61b-e747b0a6e6ba, addrs=ArrayList
> [10.188.0.108, 127.0.0.1], sockAddrs=HashSet
> [ignite-0.ignite.ignite.svc.cluster.local/10.188.0.108:47500 <>,
> /127.0.0.1:47500 <>], discPort=47500, order=2, intOrder=2,
> lastExchangeTime=1600681390383, loc=true, ver=2.8.1#20200521-sha1:86422096,
> isClient=false], reqId=145, segment=0, sql='SELECT
> A__Z0.ASSET_UID __C0_0,
> A__Z0.ATTRIBUTE_CODE __C0_1,
> B__Z1.TYPE __C0_2,
> A__Z0.NUMVALUE __C0_3,
> A__Z0.UNIT_SYMBOL __C0_4,
> A__Z0.ALNVALUE __C0_5,
> A__Z0.CHANGEDATE __C0_6,
> B__Z1.CHANGEDATE __C0_7,
> A__Z0.ORG_ID __C0_8
> FROM CRTX.ASSET B__Z1 
>  INNER JOIN CRTX.ASSETSPEC A__Z0 
>  ON TRUE
> WHERE (B__Z1.LOCATION_UID = 'R02ERUS010843') AND ((A__Z0.ORG_ID = ?4) AND
> (((A__Z0.CHANGEDATE > ?2) OR (B__Z1.CHANGEDATE > ?3)) AND ((B__Z1.TYPE = ?1)
> AND (A__Z0.ASSET_UID = B__Z1.ASSET_UID
> ORDER BY 9, 1', plan=SELECT
> A__Z0.ASSET_UID AS __C0_0,
> A__Z0.ATTRIBUTE_CODE AS __C0_1,
> B__Z1.TYPE AS __C0_2,
> A__Z0.NUMVALUE AS __C0_3,
> A__Z0.UNIT_SYMBOL AS __C0_4,
> A__Z0.ALNVALUE AS __C0_5,
> A__Z0.CHANGEDATE AS __C0_6,
> B__Z1.CHANGEDATE AS __C0_7,
> A__Z0.ORG_ID AS __C0_8
> FROM CRTX.ASSET B__Z1
> /* CRTX.ASSET.__SCAN_ */
> /* WHERE (B__Z1.LOCATION_UID = 'R02ERUS010843')
> AND (B__Z1.TYPE = ?1)
> */
> /* scanCount: 377126 */
> INNER JOIN CRTX.ASSETSPEC A__Z0
> /* CRTX."_key_PK": ASSET_UID = B__Z1.ASSET_UID */
> ON 1=1
> WHERE (B__Z1.LOCATION_UID = 'R02ERUS010843')
> AND ((A__Z0.ORG_ID = ?4)
> AND (((A__Z0.CHANGEDATE > ?2)
> OR (B__Z1.CHANGEDATE > ?3))
> AND ((B__Z1.TYPE = ?1)
> AND (A__Z0.ASSET_UID = B__Z1.ASSET_UID
> ORDER BY 9, 1]
> /opt/ignite/apache-ignite/bin/ignite.sh: line 207:74 Killed   
>   
> "$JAVA" ${JVM_OPTS} ${QUIET} "${RESTART_SUCCESS_OPT}" ${JMX_MON:-}
> -DIGNITE_HOME="${IGNITE_HOME}" -DIGNITE_PROG_NAME="$0" ${JVM_XOPTS} -cp
> "${CP}" ${MAIN_CLASS} "${CONFIG}"
> 
> 
> 
> 
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/ <>