Re: Memory Usage of a connection
On Fri, Aug 31, 2012 at 11:27 AM, Peter Schuller peter.schul...@infidyne.com wrote: Could these 500 connections/second cause (on average) 2600Mb memory usage per 2 second ~ 1300Mb/second. or For 1 connection around 2-3Mb. In terms of garbage generated it's much less about number of connections as it is about what you're doing with them. Are you for example requesting large amounts of data? Large or many columns (or both), etc. Essentially all working data that your request touches is allocated on the heap and contributes to allocation rate and ParNew frequency. write requests are simple counter increments and in memtables existing in memory. There is negligible read traffic (100/200 reads/second). Also, increasing write traffic si the one that increases gc frequency while keeping read traffic constant. So the gc should be independent of reads. -- / Peter Schuller (@scode, http://worldmodscode.wordpress.com)
Re: Memory Usage of a connection
PS: everything above is in bytes, not bits. On Fri, Aug 31, 2012 at 11:03 AM, rohit bhatia rohit2...@gmail.com wrote: I was wondering how much would be the memory usage of an established connection in cassandra's heap space. We are noticing extremely frequent young generation garbage collections (3.2gb young generation, ParNew gc every 2 seconds) at a traffic of 20,000qps for 8 nodes. We do connection pooling but with 1 connection for 6 requests with phpcassa. So, essentially every node has on an average 500 connections created/destroyed every second. Could these 500 connections/second cause (on average) 2600Mb memory usage per 2 second ~ 1300Mb/second. or For 1 connection around 2-3Mb. Is this value expected? (our write requests are simple counter increments and cannot take up 500KB per request as calculation suggests, rather should take up only a few hundred bytes). Thanks Rohit
Re: Memory Usage of a connection
Could these 500 connections/second cause (on average) 2600Mb memory usage per 2 second ~ 1300Mb/second. or For 1 connection around 2-3Mb. In terms of garbage generated it's much less about number of connections as it is about what you're doing with them. Are you for example requesting large amounts of data? Large or many columns (or both), etc. Essentially all working data that your request touches is allocated on the heap and contributes to allocation rate and ParNew frequency. -- / Peter Schuller (@scode, http://worldmodscode.wordpress.com)
