Still seems MEM. However it's hard to convince that constantly writing(even great amount of data) needs so much MEM(16GB). The process is quite simple,
input_data -> memtable -> flush to disk right? What does cassandra need so much MEM for? Thanks! ?? 2010-06-02???? 16:24 +0800??lwl?????? > No. > But I did some capacity tests about another distributed system. > Your former test cost too much MEM, it was the bottleneck. > caches and JVM cost MEM, so I suggested to decrease them. > > > What is the bottleneck of your current test now? > > > ?? 2010??6??2?? ????4:13??Shuai Yuan <yuansh...@supertool.net.cn>?? > ???? > Hi, > > I tried, > > 1-consistency level ZERO > > 2-JVM heap 4GB > > 3-normal Memtable cache > > and now I have about 30% improvment. > > However I want to know if you have also done w/r benchmark and > what's > the result? > > ?? 2010-06-02???? 11:35 +0800??lwl?????? > > > and, why did you set "JVM has 8G heap"? > > 8g, seems too big. > > > > ?? 2010??6??2?? ????11:20??lwl <lwl.ro...@gmail.com>?????? > > 3.32 concurrent read & 128 write in > storage-conf.xml, other > > cache > > enlarged as well. > > -------------------- > > > > > > maybe you can try to decrease the size of caches. > > > > ?? 2010??6??2?? ????11:14??Shuai Yuan > > <yuansh...@supertool.net.cn>?????? > > > > > > ?? 2010-06-02???? 10:37 +0800??lwl?????? > > > is all the 4 servers' MEM almost 100%? > > > > > > Yes > > > > > > > ?? 2010??6??2?? ????10:12??Shuai Yuan > > <yuansh...@supertool.net.cn>?? > > > ???? > > > Thanks lwl. > > > > > > Then is there anyway of tuning > this, faster > > flush to disk or > > > else? > > > > > > Cheers, > > > > > > Kevin > > > > > > ?? 2010-06-02???? 09:57 +0800??lwl > ?????? > > > > > > > MEM: almost 100% (16GB) > > > > ----------------- > > > > maybe this is the bottleneck. > > > > writing concerns Memtable and > SSTable in > > memory. > > > > > > > > ?? 2010??6??2?? ????9:48??Shuai > Yuan > > > <yuansh...@supertool.net.cn>?? > > > > ???? > > > > ?? 2010-06-01???? 15:00 > -0500?? > > Jonathan Shook?????? > > > > > Also, what are you > meaning > > specifically by 'slow'? > > > Which > > > > measurements > > > > > are you looking at. > What are > > your baseline > > > constraints for > > > > your test > > > > > system? > > > > > > > > > > > > > Actually, the problem is > the > > utilizaton of > > > resources(for a > > > > single > > > > machine): > > > > CPU: 700% / 1600% (16 > cores) > > > > MEM: almost 100% (16GB) > > > > Swap: almost 0% > > > > Disk IO(write): > 20~30MB / 200MB > > (7.2k raid5, > > > benchmarked > > > > previously) > > > > NET: up to 100Mbps / > 950Mbps > > (1Gbps, tuned and > > > benchmarked > > > > previously) > > > > > > > > So the speed of > generating load, > > about 15M/s as > > > reported > > > > before seems > > > > quite slow to me. I > assume the > > system should get at > > > least > > > > about 50MB/s > > > > of Disk IO speed. > > > > > > > > MEM? I don't think it > plays a > > major role in this > > > writing game. > > > > What's > > > > the bottleneck of the > system? > > > > > > > > P.S > > > > about Consistency Level, > I've > > tried ONE/DCQUORUM and > > > found ONE > > > > is about > > > > 10-15% faster. However > that's > > neither a promising > > > result. > > > > > > > > Thanks! > > > > > > > > Kevin > > > > > > > > > > > > > > 2010/6/1 ?????? > > <shiyingjie1...@gmail.com>: > > > > > > Hi, It would be > better if we > > know which > > > Consistency Level > > > > did you choose, > > > > > > and what is the > schema of test > > data? > > > > > > > > > > > > ?? 2010??6??1?? ???? > 4:48?? > > Shuai Yuan > > > > > <yuansh...@supertool.net.cn>?????? > > > > > >> > > > > > >> Hi all, > > > > > >> > > > > > >> I'm testing writing > speed of > > cassandra with 4 > > > servers. > > > > I'm confused by > > > > > >> the behavior of > cassandra. > > > > > >> > > > > > >> ---env--- > > > > > >> load-data app > written in c++, > > using > > > libcassandra (w/ > > > > modified batch > > > > > >> insert) > > > > > >> 20 writing threads > in 2 > > processes running on 2 > > > servers > > > > > >> > > > > > >> ---optimization--- > > > > > >> 1.turn log level to > INFO > > > > > >> 2.JVM has 8G heap > > > > > >> 3.32 concurrent > read & 128 > > write in > > > storage-conf.xml, > > > > other cache > > > > > >> enlarged as well. > > > > > >> > > > > > >> ---result--- > > > > > >> 1-monitoring by > > `date;nodetool -h host ring` > > > > > >> I add all load > together and > > measure the writing > > > speed by > > > > > >> (load_difference / > > time_difference), and I get > > > about > > > > 15MB/s for the > > > > > >> whole cluster. > > > > > >> > > > > > >> 2-monitoring by > `iostat -m > > 10` > > > > > >> I can watch the > disk_io from > > the system level > > > and have > > > > about 10MB/s - > > > > > >> 65MB/s for a single > machine. > > Very big variance > > > over time. > > > > > >> > > > > > >> 3-monitoring by > `iptraf -g` > > > > > >> In this way I watch > the > > communication between > > > servers and > > > > get about > > > > > >> 10MB/s for a single > machine. > > > > > >> > > > > > >> ---opinion--- > > > > > >> So, have you > checked the > > writing speed of > > > cassandra? I > > > > feel it's quite > > > > > >> slow currently. > > > > > >> > > > > > >> Could anyone > confirm this is > > the normal writing > > > speed of > > > > cassandra, or > > > > > >> please provide > someway of > > improving it? > > > > > >> -- > > > > > >> Kevin Yuan > > > > > >> www.yuan-shuai.info > > > > > >> > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Kevin Yuan > > > > www.yuan-shuai.info > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Shuai Yuan ???? > > > Supertool Corp. ?????????????????? > ???????? > > > > > > www.yuan-shuai.info > > > > > > > > > > > > > > > -- > > > > Shuai Yuan ???? > > Supertool Corp. ?????????????????????????? > > www.yuan-shuai.info > > > > > > > > > > > > > > > > > -- > > Shuai Yuan ???? > Supertool Corp. ?????????????????????????? > www.yuan-shuai.info > > > > > -- Kevin Yuan www.yuan-shuai.info
