Have you configured near cache on the client? Do you buffer data somewere ? Share the details of the loading process.
2016-05-24 16:07 GMT+03:00 Tomek W <[email protected]>: > Thanks for you answer. > > > I didn't measure it. Simply, I was getting GC overflow error, so I > succesively increased it. > How to measure it ? Why client requires so much memory ? > > 2016-05-24 14:28 GMT+02:00 Alexei Scherbakov <[email protected] > >: > >> Just use this: >> >> -server -Xms10G -Xmx10G -XX:+UseParNewGC -XX:+UseConcMarkSweepGC >> -XX:+UseTLAB -XX:NewSize=128m -XX:MaxNewSize=128m >> -XX:MaxTenuringThreshold=0 -XX:SurvivorRatio=1024 >> -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=60 >> -XX:+DisableExplicitGC >> >> How do you measure client memory usage ? >> >> 2016-05-24 15:04 GMT+03:00 Tomek W <[email protected]>: >> >>> Sorry, >>> I made a mistake - I wanted to say that I am going to use ON_HEAP. >>> Can you suggest my more details about tuning ? >>> I have client (which run hot loading data from postgresql) and server >>> node (keep cache - data). >>> Now client also requires ~4GB data. Why ? After all, it doesn't keep >>> data, only run hot loading. >>> >>> >>> >>> >>> >>> 2016-05-24 13:44 GMT+02:00 Alexei Scherbakov < >>> [email protected]>: >>> >>>> Try to start with some larger number, if default value is too low for >>>> you. >>>> On example, set it to 50000, and see if the performance is OK. >>>> If not, increase to 100000 etc. >>>> I can't help you further without knowing your data access patterns. >>>> >>>> BTW, for 10G heap it is probably better to use ONHEAP_TIERED, as Val >>>> suggested. >>>> Don't forget to tune GC as described here: >>>> >>>> >>>> https://apacheignite.readme.io/docs/jvm-and-system-tuning#jvm-tuning-for-clusters-with-on_heap-caches >>>> >>>> >>>> 2016-05-23 22:05 GMT+03:00 Tomek W <[email protected]>: >>>> >>>>> Ok, I am going to use OFF_HEAP. >>>>> >>>>> >>>>> On each node I am going to use about 10 GB. (My ram is 16GB). >>>>> Can you help me adjust configuration for this aim ? >>>>> It is very important for me. >>>>> Aim: Extremely fast sql quries. >>>>> >>>>> >>>>> 2016-05-23 18:13 GMT+02:00 Alexei Scherbakov < >>>>> [email protected]>: >>>>> >>>>>> Hi, >>>>>> >>>>>> Generally speaking, settings setSqlOnheapRowCacheSize to larger >>>>>> value increases >>>>>> SQL performance in OFFHEAP_TIERED mode, but also means more job for >>>>>> GC, >>>>>> so it should be used with care. >>>>>> >>>>>> The value should be set to the size of your application's >>>>>> working(frequently accessed) data set. >>>>>> >>>>>> 2016-05-23 13:07 GMT+03:00 vkulichenko <[email protected] >>>>>> >: >>>>>> >>>>>>> Are you using offheap? What is your data size? >>>>>>> >>>>>>> Generally, I would recommend to use on-heap with SQL queries if this >>>>>>> possible (unless you have a very big data sets and want to avoid >>>>>>> having >>>>>>> large heap sizes). If you still have to use offheap, you can try >>>>>>> playing >>>>>>> with this parameter and see what performance you get with different >>>>>>> values. >>>>>>> The optimal value depends on a particular application. >>>>>>> >>>>>>> -Val >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> View this message in context: >>>>>>> http://apache-ignite-users.70518.x6.nabble.com/off-heap-indexes-setSqlOnheapRowCacheSize-how-does-it-improve-efficiency-tp5070p5092.html >>>>>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com. >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Best regards, >>>>>> Alexei Scherbakov >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> >>>> Best regards, >>>> Alexei Scherbakov >>>> >>> >>> >> >> >> -- >> >> Best regards, >> Alexei Scherbakov >> > > -- Best regards, Alexei Scherbakov
