Re: Ignite Client Node OOM Issue
Hi, *1) What load profile do you have? * Ans: We have 2 clusters, each having 2 nodes, one cluster is having approx 15 GB data (Replication) & second cluster is having approx 5 GB data (Partitioned) with eviction policy. *2) Do you use SQL queries?* Ans: Yes, We are using. *3) Is it possible to share your client node configuration?* Ans: Yes, I have attached below. Thanks, On Fri, 6 Nov 2020 at 18:50, Vladimir Pligin wrote: > What load profile do you have? Do you use SQL queries? Is it possible to > share your client node configuration? > > > > -- > Sent from: http://apache-ignite-users.70518.x6.nabble.com/ > http://www.springframework.org/schema/beans; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; xmlns:util="http://www.springframework.org/schema/util; xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util.xsd;> 10.201.30.52:58500
Ignite Client Node Stopped : OOM Error
Hi, We are using Apache Ignite 2.7.0 binary and servers are using Linux OS & app servers are using Windows OS.We are using Apache Ignite .Net APIs. Recently we have noticed that our application is not stopped with an OOM error. App server has 32 GB RAM & We are specifying JVM Heap = 8 GB I am sharing a client log and could you please suggest why the client node is not responding & what is the possible cause and how to resolve this issue? Thanks & Regards, logs.rar Description: Binary data
Ignite Client Node OOM Issue
HI, We are using Apache Ignite 2.7.0 binary and servers are using Linux OS & app servers are using Windows OS.We are using Apache Ignite .Net APIs. Recently we have noticed that our application (Client Node) is stopping with OOM error. App server has 32 GB RAM & we are specifying JVM Heap = 8 GB. App server utilization at the OOM error was 80% and we have still 20% memory left, still we are facing this issue. 1) How to check memory usage for the client node? 2) Is there any way by looking at logs we can understand, OOM might occur? 3) Could you please suggest, how to overcome this issue? I am sharing an application crash log. Thanks & Regards, OOM-Error.rar Description: Binary data
Re: Ignite Client Node Stopped With Out Of Memory
Hi Paval, As we are not setting explicitly QueryBase.Pagesize for SqlQuery and SqlFieldQuery so default will be used as 1024. We have not found so far any example by looking the same we can try to explicitly set the Query base.Pagesize. Can we have any reference by checking that we can try to set the lower value and can able to replicate the scenario? Thanks, On Wed, 28 Oct, 2020, 7:53 pm Ravi Makwana, wrote: > > > > > On Wed, 28 Oct, 2020, 5:55 pm Pavel Tupitsyn, > wrote: > >> I found a bug in Ignite [1] which probably causes the issue on your side. >> >> Looks like you are running a query (is it a ScanQuery or SqlQuery?) and >> the size of one results page exceeds 2GB. >> Please try using a smaller value for *QueryBase.PageSize*. >> >> If you use the default value of 1024, your cache entries are very large. >> What does the data look like? >> >> [1] https://issues.apache.org/jira/browse/IGNITE-13635 >> >> On Wed, Oct 28, 2020 at 1:43 PM Ravi Makwana >> wrote: >> >>> Hi, >>> >>> Our service is running with 64 bit and we have verified the same in our >>> app server too. >>> >>> Any finding from the logs? >>> >>> Is there any way to replicate it? >>> >>> Thanks, >>> >>> >>> On Wed, 28 Oct 2020 at 15:47, Pavel Tupitsyn >>> wrote: >>> >>>> Looks like the app is running in 32 bit mode, which can't use more than >>>> 2GB of memory. >>>> JVM and memory regions pre-allocate all of it, leaving nothing for .NET >>>> to use. >>>> >>>> Please check the `Platform` column in the Task Manager - does it say >>>> `32 bit`? >>>> If yes, then try disabling `Prefer 32 bit` in the project properties. >>>> >>>> On Wed, Oct 28, 2020 at 1:06 PM Ravi Makwana >>>> wrote: >>>> >>>>> Hi Pavel, >>>>> >>>>> Thanks for the information. >>>>> >>>>> We have noticed our service is taking max 2GB memory which is noticed >>>>> in task manager and stopping app pool at same time server memory >>>>> utilization is 80% so we have still memory spare 20%. >>>>> >>>>> Still we are not able to correlate the issue, I would like to share >>>>> some more logs. Could you suggest what is happening at the moment? >>>>> >>>>> Thanks, >>>>> >>>>>
Re: Ignite Client Node Stopped With Out Of Memory
On Wed, 28 Oct, 2020, 5:55 pm Pavel Tupitsyn, wrote: > I found a bug in Ignite [1] which probably causes the issue on your side. > > Looks like you are running a query (is it a ScanQuery or SqlQuery?) and > the size of one results page exceeds 2GB. > Please try using a smaller value for *QueryBase.PageSize*. > > If you use the default value of 1024, your cache entries are very large. > What does the data look like? > > [1] https://issues.apache.org/jira/browse/IGNITE-13635 > > On Wed, Oct 28, 2020 at 1:43 PM Ravi Makwana > wrote: > >> Hi, >> >> Our service is running with 64 bit and we have verified the same in our >> app server too. >> >> Any finding from the logs? >> >> Is there any way to replicate it? >> >> Thanks, >> >> >> On Wed, 28 Oct 2020 at 15:47, Pavel Tupitsyn >> wrote: >> >>> Looks like the app is running in 32 bit mode, which can't use more than >>> 2GB of memory. >>> JVM and memory regions pre-allocate all of it, leaving nothing for .NET >>> to use. >>> >>> Please check the `Platform` column in the Task Manager - does it say `32 >>> bit`? >>> If yes, then try disabling `Prefer 32 bit` in the project properties. >>> >>> On Wed, Oct 28, 2020 at 1:06 PM Ravi Makwana >>> wrote: >>> >>>> Hi Pavel, >>>> >>>> Thanks for the information. >>>> >>>> We have noticed our service is taking max 2GB memory which is noticed >>>> in task manager and stopping app pool at same time server memory >>>> utilization is 80% so we have still memory spare 20%. >>>> >>>> Still we are not able to correlate the issue, I would like to share >>>> some more logs. Could you suggest what is happening at the moment? >>>> >>>> Thanks, >>>> >>>>
Re: Ignite Client Node Stopped With Out Of Memory
Hi, Our service is running with 64 bit and we have verified the same in our app server too. Any finding from the logs? Is there any way to replicate it? Thanks, On Wed, 28 Oct 2020 at 15:47, Pavel Tupitsyn wrote: > Looks like the app is running in 32 bit mode, which can't use more than > 2GB of memory. > JVM and memory regions pre-allocate all of it, leaving nothing for .NET to > use. > > Please check the `Platform` column in the Task Manager - does it say `32 > bit`? > If yes, then try disabling `Prefer 32 bit` in the project properties. > > On Wed, Oct 28, 2020 at 1:06 PM Ravi Makwana > wrote: > >> Hi Pavel, >> >> Thanks for the information. >> >> We have noticed our service is taking max 2GB memory which is noticed in >> task manager and stopping app pool at same time server memory utilization >> is 80% so we have still memory spare 20%. >> >> Still we are not able to correlate the issue, I would like to share some >> more logs. Could you suggest what is happening at the moment? >> >> Thanks, >> >>
Ignite Client Node Stopped With Out Of Memory
Hi, We are using Apache Ignite 2.7.0 binary and servers are using Linux OS & app servers are using Windows OS.We are using Apache Ignite .Net APIs. Recently we have noticed that our application is stopping due to a client node throwing Out Of Memory error which we have seen in ignite client node log file. App server has 32 GB RAM & We are specifying JVM Heap = 8 GB & Data Region Min = 1 GB & Max = 10 GB. *Exception:* class org.apache.ignite.IgniteException: Platform error:System.OutOfMemoryException: Insufficient memory to continue the execution of the program. at System.Runtime.InteropServices.Marshal.ReAllocHGlobal(IntPtr pv, IntPtr cb) at Apache.Ignite.Core.Impl.Memory.PlatformMemoryUtils.ReallocatePooled(Int64 memPtr, Int32 cap) at Apache.Ignite.Core.Impl.Unmanaged.UnmanagedCallbacks.MemoryReallocate(Int64 memPtr, Int64 cap, Int64 unused, Void* arg) at Apache.Ignite.Core.Impl.Unmanaged.UnmanagedCallbacks.InLongLongLongObjectOutLong(Int32 type, Int64 val1, Int64 val2, Int64 val3, IntPtr arg) at org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.loggerLog(PlatformProcessorImpl.java:404) at org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:460) at org.apache.ignite.internal.processors.platform.PlatformProcessorImpl.processInStreamOutLong(PlatformProcessorImpl.java:512) at org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67) at org.apache.ignite.internal.processors.platform.callback.PlatformCallbackUtils.inLongLongLongObjectOutLong(Native Method) at org.apache.ignite.internal.processors.platform.callback.PlatformCallbackGateway.memoryReallocate(PlatformCallbackGateway.java:840) at org.apache.ignite.internal.processors.platform.memory.PlatformExternalMemory.reallocate(PlatformExternalMemory.java:48) at org.apache.ignite.internal.processors.platform.memory.PlatformOutputStreamImpl.ensureCapacity(PlatformOutputStreamImpl.java:305) at org.apache.ignite.internal.processors.platform.memory.PlatformOutputStreamImpl.unsafeEnsure(PlatformOutputStreamImpl.java:222) at org.apache.ignite.internal.binary.BinaryWriterExImpl.doWriteBinaryObject(BinaryWriterExImpl.java:962) at org.apache.ignite.internal.binary.BinaryClassDescriptor.write(BinaryClassDescriptor.java:744) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal0(BinaryWriterExImpl.java:223) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:164) at org.apache.ignite.internal.binary.BinaryWriterExImpl.marshal(BinaryWriterExImpl.java:151) at org.apache.ignite.internal.binary.BinaryWriterExImpl.writeObjectDetached(BinaryWriterExImpl.java:1506) at org.apache.ignite.internal.processors.platform.cache.query.PlatformQueryCursor.write(PlatformQueryCursor.java:43) at org.apache.ignite.internal.processors.platform.cache.query.PlatformQueryCursor.write(PlatformQueryCursor.java:28) at org.apache.ignite.internal.processors.platform.cache.query.PlatformAbstractQueryCursor.processOutStream(PlatformAbstractQueryCursor.java:85) at org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.outStream(PlatformTargetProxyImpl.java:93) Can please suggest what is the root cause for the above issue? Thanks & Regards,
Re: Ignite SqlFieldQuery Unicode Characters Support
Hi, We have started our ignite nodes (server & client) with mentioned jvm parameter and it works fine. Thanks for the quick response. Regards, Ravi Makwana On Mon, 8 Jun 2020 at 18:49, Ilya Kasnacheev wrote: > Hello! > > I have met an error like this one, and my recommendation is that on all > nodes, client and server, you should set file.encoding to the same value > (UTF-8 usually, however CJK multi-byte encodings should also work). > > Make sure all of your JVMs are launched with -Dfile.encoding=UTF-8. > ignite.sh should do that already. Make sure you are using the same on > Windows and Linux. > > Regards, > -- > Ilya Kasnacheev > > > пт, 5 июн. 2020 г. в 14:31, Ravi Makwana : > >> Hi, >> >> We are using Apache ignite version 2.7 and we have one node cluster with >> 150 caches out of 150 We have 1 cache which contains Unicode characters >> like Chinese language. >> >> Server node is running on Linux OS & Client node is running on Windows OS. >> >> Client side we are using sqlquery & sqlfieldquery to fetch and put & >> data-streamer to insert data using .net ignite apis. >> >> In ignite cache,we have AddressFull property which contains Unicode >> characters like Chinese language so while inserting we have inserted >> specified data into AddressFull property which is mentioned below. >> >> *"北京市朝阳区安贞西里四区甲1号, 北京, 中国, 100011"* >> >> While fetching data from ignite cache we have observed that we are not >> getting inserted data properly. >> >> We have performed below mentioned sqlquery using SqlFieldQuery. >> >> SELECT HotelCode,AddressFull FROM HotelOtherLanguageMasterModel WHERE >> HotelCode = '12345' >> >> We got below mentioned result data for AddressFull property. >> >> *"北京市æœ�阳区安贞西里四区甲1å�·, 北京, ä¸å›½, 100011"* >> >> Based on this test we have couple of questions on the same. >> >> 1) Is SqlFieldQuery .net api supporting Unicode characters? >> 2) If Yes, How do we overcome this issue? >> >> Regards, >> Ravi Makwana >> >
Ignite SqlFieldQuery Unicode Characters Support
Hi, We are using Apache ignite version 2.7 and we have one node cluster with 150 caches out of 150 We have 1 cache which contains Unicode characters like Chinese language. Server node is running on Linux OS & Client node is running on Windows OS. Client side we are using sqlquery & sqlfieldquery to fetch and put & data-streamer to insert data using .net ignite apis. In ignite cache,we have AddressFull property which contains Unicode characters like Chinese language so while inserting we have inserted specified data into AddressFull property which is mentioned below. *"北京市朝阳区安贞西里四区甲1号, 北京, 中国, 100011"* While fetching data from ignite cache we have observed that we are not getting inserted data properly. We have performed below mentioned sqlquery using SqlFieldQuery. SELECT HotelCode,AddressFull FROM HotelOtherLanguageMasterModel WHERE HotelCode = '12345' We got below mentioned result data for AddressFull property. *"北京市æœ�阳区安贞西里四区甲1å�·, 北京, ä¸å›½, 100011"* Based on this test we have couple of questions on the same. 1) Is SqlFieldQuery .net api supporting Unicode characters? 2) If Yes, How do we overcome this issue? Regards, Ravi Makwana
