What constitutes an "extreme write" ?
On 2013-05-03 15:45:33
+0000, Edward Capriolo said:
If your writes are so extreme that metables are flushing all the time,
the best you can do is turn off all caches, do bloom filters off heap,
and then instruct cassandra to use large portions of the heap as
memtables.
On Fri, May 3, 2013 at 11:40 AM, Bryan Talbot <btal...@aeriagames.com> wrote:
It's true that a 16GB heap is generally not a good idea; however, it's
not clear from the data provided what problem you're trying to solve.
What is it that you don't like about the default settings?
-Bryan
On Fri, May 3, 2013 at 4:27 AM, Oleg Dulin <oleg.du...@gmail.com> wrote:
Here is my question. It can't possibly be a good set up to use 16gig
heap space, but this is the best I can do. Setting it to default never
worked well for me, setting it to 8g doesn't work well either. It can't
keep up with flushing memtables. It is possibly that someone at some
point may have broken something in the config files. If I were to look
for hints there, what should I look at ?
Look at my gc log from Cassandra:
Starts off like this:
2013-04-29T08:53:44.548-0400: 5.386: [GC 1677824K->11345K(16567552K),
0.0509880 secs]
2 2013-04-29T08:53:47.701-0400: 8.539: [GC
1689169K->42027K(16567552K), 0.1269180 secs]
3 2013-04-29T08:54:05.361-0400: 26.199: [GC
1719851K->231763K(16567552K), 0.1436070 secs]
4 2013-04-29T08:55:44.797-0400: 125.635: [GC
1909587K->1480096K(16567552K), 1.2626270 secs]
5 2013-04-29T08:58:44.367-0400: 305.205: [GC
3157920K->2358588K(16567552K), 1.1198150 secs]
6 2013-04-29T09:01:12.167-0400: 453.005: [GC
4036412K->3634298K(16567552K), 1.0098650 secs]
7 2013-04-29T09:03:35.204-0400: 596.042: [GC
5312122K->4339703K(16567552K), 0.4597180 secs]
8 2013-04-29T09:04:51.562-0400: 672.400: [GC
6017527K->4956381K(16567552K), 0.5361800 secs]
9 2013-04-29T09:04:59.205-0400: 680.043: [GC
6634205K->5131825K(16567552K), 0.1741690 secs]
10 2013-04-29T09:05:06.638-0400: 687.476: [GC
6809649K->5027933K(16567552K), 0.0607470 secs]
11 2013-04-29T09:05:13.908-0400: 694.747: [GC
6705757K->5012439K(16567552K), 0.0624410 secs]
12 2013-04-29T09:05:20.909-0400: 701.747: [GC
6690263K->5039538K(16567552K), 0.0618750 secs]
13 2013-04-29T09:06:35.914-0400: 776.752: [GC
6717362K->5819204K(16567552K), 0.5738550 secs]
14 2013-04-29T09:08:05.589-0400: 866.428: [GC
7497028K->6678597K(16567552K), 0.6781900 secs]
15 2013-04-29T09:08:12.458-0400: 873.296: [GC
8356421K->6865736K(16567552K), 0.1423040 secs]
16 2013-04-29T09:08:18.690-0400: 879.529: [GC
8543560K->6742902K(16567552K), 0.0516470 secs]
17 2013-04-29T09:08:24.914-0400: 885.752: [GC
8420726K->6725877K(16567552K), 0.0517290 secs]
18 2013-04-29T09:08:31.008-0400: 891.846: [GC
8403701K->6741781K(16567552K), 0.0532540 secs]
19 2013-04-29T09:08:37.201-0400: 898.039: [GC
8419605K->6759614K(16567552K), 0.0563290 secs]
20 2013-04-29T09:08:43.493-0400: 904.331: [GC
8437438K->6772147K(16567552K), 0.0569580 secs]
21 2013-04-29T09:08:49.757-0400: 910.595: [GC
8449971K->6776883K(16567552K), 0.0558070 secs]
22 2013-04-29T09:08:55.973-0400: 916.812: [GC
8454707K->6789404K(16567552K), 0.0577230 secs]
……
look what it is today:
41536 2013-05-03T07:17:13.519-0400: 339814.357: [GC
9178946K->9176740K(16567552K), 0.0265830 secs]
41537 2013-05-03T07:17:19.556-0400: 339820.394: [GC
10854564K->9178449K(16567552K), 0.0253180 secs]
41538 2013-05-03T07:17:24.390-0400: 339825.228: [GC
10856273K->9179073K(16567552K), 0.0266450 secs]
41539 2013-05-03T07:17:30.729-0400: 339831.567: [GC
10856897K->9178629K(16567552K), 0.0261150 secs]
41540 2013-05-03T07:17:35.584-0400: 339836.422: [GC
10856453K->9178586K(16567552K), 0.0250870 secs]
41541 2013-05-03T07:17:38.514-0400: 339839.352: [GC
10856410K->9179314K(16567552K), 0.0258120 secs]
41542 2013-05-03T07:17:43.200-0400: 339844.038: [GC
10857138K->9180160K(16567552K), 0.0250150 secs]
41543 2013-05-03T07:17:46.566-0400: 339847.404: [GC
10857984K->9179071K(16567552K), 0.0264420 secs]
41544 2013-05-03T07:17:52.913-0400: 339853.751: [GC
10856895K->9179870K(16567552K), 0.0262430 secs]
41545 2013-05-03T07:17:58.303-0400: 339859.141: [GC
10857694K->9179209K(16567552K), 0.0255130 secs]
41546 2013-05-03T07:18:03.427-0400: 339864.265: [GC
10857033K->9178316K(16567552K), 0.0263140 secs]
41547 2013-05-03T07:18:11.657-0400: 339872.495: [GC
10856140K->9178351K(16567552K), 0.0265340 secs]
41548 2013-05-03T07:18:17.429-0400: 339878.267: [GC
10856175K->9179067K(16567552K), 0.0254820 secs]
41549 2013-05-03T07:18:21.251-0400: 339882.089: [GC
10856891K->9179680K(16567552K), 0.0264210 secs]
41550 2013-05-03T07:18:25.062-0400: 339885.900: [GC
10857504K->9178985K(16567552K), 0.0267200 secs]
--
Regards,
Oleg Dulin
NYC Java Big Data Engineer
http://www.olegdulin.com/
--
Regards,
Oleg Dulin
NYC Java Big Data Engineer
http://www.olegdulin.com/
