I'm afraid I can't use iostat on a VPS. Shows me a "Cannot find disk data" warning and dies. Googled it a bit and found a bunch of "iostat doesn't work from inside containers" and "it would be a security issue if you could have that kkind of access to physical drivers from inside a virtual machine".
It's a shame dedicated servers are so overpriced and aren't really an option. The cheapest ones cost at least 25x (yes, I'm not exaggerating) the VPS I rent. Brazil's network is a piece of shit. Still, gotta get the best I can from these VPSes. If I could solidly link the stuttering to i/o, I can argue with the host and see if they can place me with calmer neighbours. Would a SSD, still using a virtual machine, help me somehow? _pilger On 9 April 2014 23:23, Yun Huang Yong <[email protected]> wrote: > Try watching: > > $ iostat -x 1 > > You want to pay attention to the r_await/w_await which are the average > read and write times respectively, in milliseconds. > > If any of those numbers goes above 15 you will get the stuttering you > describe. > > Basically it indicates that the underlying disk(s) can not serve your > read/write requests within 15 milliseconds. As mentioned previously > 1000ms/66 ticket = time budget of 15ms per tick. > > If that is the case talk to the host about possibly migrating to another > physical host, or change providers. There's nothing you can do inside the > VM. > > With VPS hosting there are a lot of anecdotes about various providers. At > the end of the day your *consistent* performance level depends on what your > provider does in terms of locking down resource usage between VMs, how many > VMs they try to cram onto each box, and what your neighbours are doing. > > In an ideal world providers would provide specifications down to things > like disk latency, IOPS, CPU scheduling etc but in reality most customers > won't understand it anyway, or prefer to pay a low price for "burst" > ability even if that means massive oversubscription > > Unfortunately that means VPS is basically a crapshoot, especially within > the budget market. You try VPSes until you land with a provider & host that > works for your budget. On the bright side at least with the server > registration now you can migrate between VPS providers without losing all > of your players :] > > > On 10/04/2014 3:48 AM, pilger wrote: > >> Hey guys, >> >> Switched to Ubuntu 13 x86 and things got a little better but the hiccups >> are still there. >> >> I've noticed the "wa" value of top varies a bit during the hiccups. Here's >> what I have observed: >> >> [image: Inline images 1] >> >> Could that be related to the problem? I've googled it briefly and found it >> has something to do with disk i/o. Is that correct!? The problem is caused >> by slow disk!? >> >> Thanks in advance! >> >> >> _pilger >> >> >> On 8 April 2014 11:14, pilger <[email protected]> wrote: >> >> The VPS is running OpenVZ virtualization, so I guess it should be fine. >>> >>> I also have a friend on the same hosting company with the same plan and >>> Debian 7 x64 (which was the SO at the time I first posted about the >>> problem) that works way better than my VPS. Not sure why. >>> >>> >>> _pilger >>> >>> >>> On 8 April 2014 11:06, pilger <[email protected]> wrote: >>> >>> Weasels, whew. Long text. But it was a good share of experience there. >>>> My >>>> VPS host offers Ubuntu 13 as the latest SO there, so I switched to it. >>>> On >>>> preliminary tests it did well, let's see how it goes when the server >>>> gets >>>> full and we reach peak time of day were the lousy neighbours are all >>>> screaming around. >>>> >>>> I've had previous experiences with Linux but for other purposes and, >>>> since I do some programming around, the learning curve isn't too steep >>>> for >>>> me. Networking is still a mystery to me, though. So I went here for your >>>> help. >>>> >>>> Many thanks for all the info! >>>> >>>> Thanks for all the input, Yun. Seems interesting but a rather long read, >>>> so I'll take my time to digest the whole information. I'll also have to >>>> produce another server to host the monitor, which I don't have available >>>> right now, so I guess collectd will be postponed for a while, despite >>>> of it >>>> being a good method to see what the hell is going on. >>>> >>>> >>>> >>>> >>>> _pilger >>>> >>>> >>>> On 7 April 2014 21:55, Yun Huang Yong <[email protected]> wrote: >>>> >>>> collectd is fairly light which is why it is popular as a collection >>>>> agent. >>>>> >>>>> I'm going to assume you are comfortable installing packages and >>>>> configuring software. I don't have the time to write copy pasta >>>>> instructions. I *strongly* recommend you read all of this & the links >>>>> before you begin, and make sure you understand what is required of you. >>>>> >>>>> The key components are: >>>>> - https://collectd.org/ >>>>> - http://graphite.wikidot.com/ >>>>> >>>>> You need to get collectd running on each of your TF2 hosts. Basically >>>>> apt-get but see note below regarding collectd versions. >>>>> >>>>> You'll then want to setup Graphite on another machine. You *could* run >>>>> it on your TF2 host but Carbon can get I/O hungry (it is tunable) and >>>>> that >>>>> will create more problems for you so I strongly recommend running >>>>> Graphite >>>>> on another machine. >>>>> >>>>> Also, having Graphite on another machine (with the collectd collector, >>>>> below) makes it easy for you to have multiple TF2 hosts, or migrate TF2 >>>>> hosts. >>>>> >>>>> In my setup I have my Graphite host running on an Ubuntu VM at home >>>>> with >>>>> 6 external servers reporting to it. >>>>> >>>>> Here's a picture of the overall setup: >>>>> https://dl.dropboxusercontent.com/u/8110989/2014/collectd-graphite.png >>>>> >>>>> Back to setup... with collectd running on your TF2 host, and Graphite >>>>> on >>>>> another host, how do you connect them? >>>>> >>>>> https://collectd.org/wiki/index.php/Networking_introduction >>>>> >>>>> Your Graphite host *also* needs to run collectd in order to act as a >>>>> collection server for your TF2 host's collectd to send it data. Edit >>>>> config >>>>> on both TF2 host & Graphite host -- both sides need to run the collectd >>>>> network plugin, Graphite host as server, TF2 host as client. >>>>> >>>>> Your Graphite host's collectd also needs to run the write_graphite >>>>> plugin to write the network collected data to Graphite. >>>>> >>>>> https://collectd.org/wiki/index.php/Plugin:Write_Graphite >>>>> >>>>> <Plugin write_graphite> >>>>> <Carbon> >>>>> Host "localhost" >>>>> Port "2003" >>>>> EscapeCharacter "_" >>>>> </Carbon> >>>>> </Plugin> >>>>> >>>>> Note: if you Google collectd + graphite you may be confused by many >>>>> blog >>>>> posts refer to custom written plugins which were necessary before >>>>> collectd >>>>> had its write_graphite plugin. >>>>> >>>>> Note 2: since you're on Debian note that the write_graphite plugin was >>>>> added with collectd 5.1. You may need to get it from backports or >>>>> something. >>>>> >>>>> For Graphite... >>>>> >>>>> This is a reasonable overview but may be out of date: >>>>> http://graphite.wikidot.com/installation >>>>> >>>>> Read ^ as an overview but maybe follow the current instructions here: >>>>> https://graphite.readthedocs.org/en/latest/install.html >>>>> >>>>> You need to pay attention to the storage-schemas.conf but you can more >>>>> or less ignore other instructions about feeding data into Graphite. >>>>> With >>>>> the collectd write_graphite plugin your data will automagically be fed >>>>> from >>>>> collectd -> localhost:2003 which is Carbon (Graphite's collector). >>>>> >>>>> Good luck :] >>>>> >>>>> PS: I am happy to answer specific questions about the collectd/graphite >>>>> setup but if you ask general sysadmin stuff I probably won't respond. >>>>> >>>>> >>>>> On 8/04/2014 12:50 AM, pilger wrote: >>>>> >>>>> I've noticed the yellow bars mainly on the Mem field. Don't know if >>>>>> that >>>>>> might be related. Could it? >>>>>> >>>>>> About collectd, it seems very nice and a lot easier to visualize but >>>>>> you >>>>>> talked greek to me up there. Would you point me to some tutorial or >>>>>> show >>>>>> me some ropes on how to get it running so I can find the bottlenecks? >>>>>> Does it use a lot of resource!? >>>>>> >>>>>> >>>>>> _pilger >>>>>> >>>>>> >>>>>> On 7 April 2014 11:35, Yun Huang Yong <[email protected] >>>>>> <mailto:[email protected]>> wrote: >>>>>> >>>>>> Your concern about noisy VPS neighbours will show up as CPU >>>>>> steal - >>>>>> htop shows this as yellow bars by default. >>>>>> >>>>>> Disk latency could also be an issue. >>>>>> >>>>>> 66 tick means each tick has a time budget of around 15ms >>>>>> (1000/66). >>>>>> If disk latency exceeds 15ms you will get stuttering - I had this >>>>>> happen on servers in the past. >>>>>> >>>>>> e.g. >>>>>> https://dl.dropboxusercontent.__com/u/8110989/2013/np1-disk- >>>>>> __latency.png >>>>>> >>>>>> <https://dl.dropboxusercontent.com/u/ >>>>>> 8110989/2013/np1-disk-latency. >>>>>> png> >>>>>> >>>>>> Stuttery server leading up to 08/03 (US style month/day, August >>>>>> last >>>>>> year). Host migrated my server to another less loaded machine, >>>>>> great >>>>>> for a few weeks then as that machine also became more heavily >>>>>> utilised (by other customers) it started to stutter again. >>>>>> >>>>>> FWIW I use collectd to gather these metrics on each host, feeding >>>>>> into a single collectd collector which then uses collectd's >>>>>> write_graphite plugin to write all the data into graphite for >>>>>> storage & graphing. collectd's default 10s polling is great for >>>>>> picking up transient issues, and graphite_web makes the >>>>>> visualisation easy. >>>>>> >>>>>> >>>>>> On 7/04/2014 10:26 PM, pilger wrote: >>>>>> >>>>>> Hey guys, thanks for the replies. >>>>>> >>>>>> * The RAM seems all right when I look at it with htop; >>>>>> * We tried CentOS but the network was behaving poorly >>>>>> with it >>>>>> so we >>>>>> >>>>>> switched to Debian x64 and it became a lot better; >>>>>> * net_splitpacket_maxrate was set to 50000 while the rates >>>>>> were from >>>>>> >>>>>> 30000 to 60000. I've now set the splitpacket to 100000 >>>>>> and >>>>>> the rates >>>>>> to 50000 to 100000 as you guys suggested. Gotta wait a >>>>>> bit >>>>>> for the >>>>>> server to get full so I can check if it worked; >>>>>> >>>>>> Wouldn't the htop or any other monitoring tool show something >>>>>> wrong even >>>>>> it being a VPS!? >>>>>> >>>>>> But, anyway, as I mentioned before, the problem occurs with >>>>>> the >>>>>> server >>>>>> practically empty. So I don't think it is related to CPU >>>>>> being >>>>>> overloaded... could I be wrong on this? Could my VPS >>>>>> neighbours >>>>>> be >>>>>> leeching on my CPU even it being supposedly reserved to my >>>>>> service? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> >>>>>> >>>>>> _pilger >>>>>> >>>>>> >>>>>> On 7 April 2014 02:10, John <lists.valve@nuclearfallout.__ >>>>>> net >>>>>> <mailto:[email protected]> >>>>>> <mailto:lists.valve@__nuclearfallout.net >>>>>> >>>>>> <mailto:[email protected]>>> wrote: >>>>>> >>>>>> Its not the RAM. Its packet loss from server side - >>>>>> you >>>>>> won't >>>>>> see it on net graph as its only client side. >>>>>> >>>>>> >>>>>> Packet loss should show in net_graph output either way. >>>>>> But, to be >>>>>> safe, certainly run MTR tests. >>>>>> >>>>>> >>>>>> I've had this happen to me lots of times. Been >>>>>> running >>>>>> servers >>>>>> since the 1.5 days. Ditch your host and also ditch >>>>>> Debian BS. >>>>>> >>>>>> >>>>>> Recent versions of Debian work well for game servers, so >>>>>> ditching it >>>>>> would not be necessary. >>>>>> >>>>>> You should confer with your host on the status of your >>>>>> hardware and >>>>>> whether a performance limitation is involved, such as >>>>>> I/O >>>>>> delays. >>>>>> You should also double-check server-side rates, >>>>>> including >>>>>> by making >>>>>> sure that net_splitpacket_maxrate is set sufficiently >>>>>> high >>>>>> (such as >>>>>> 100000). These symptoms seem along the lines of what I >>>>>> would expect >>>>>> from net_splitpacket_maxrate being low. >>>>>> >>>>>> >>>>>> Ask ant corporation or enterprise, all use CentOS. >>>>>> >>>>>> >>>>>> CentOS is marketed to enterprise and works well for such >>>>>> applications because of its older, stable, well-tested >>>>>> software >>>>>> packages and extended RHEL support for those older >>>>>> packages. For >>>>>> game servers, it is not ideal, since those older >>>>>> packages >>>>>> often lack >>>>>> useful features and performance tweaks. Debian is >>>>>> usually a >>>>>> better >>>>>> choice for game servers. >>>>>> >>>>>> >>>>>> If you're interested in hosting DDoS protected >>>>>> servers, >>>>>> email me >>>>>> - I can help you. >>>>>> >>>>>> >>>>>> Be very careful with hosts that claim to offer DDoS >>>>>> protection. >>>>>> There is an extremely limited number who do it right, >>>>>> and a >>>>>> very >>>>>> large number who do not. >>>>>> >>>>>> -John >>>>>> >>>>>> >>>>>> ___________________________________________________ >>>>>> >>>>>> >>>>>> To unsubscribe, edit your list preferences, or view the >>>>>> list >>>>>> archives, please visit: >>>>>> https://list.valvesoftware.____com/cgi-bin/mailman/listinfo/ >>>>>> ____hlds >>>>>> >>>>>> <https://list.valvesoftware.__com/cgi-bin/mailman/listinfo/_ >>>>>> _hlds >>>>>> <https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _________________________________________________ >>>>>> To unsubscribe, edit your list preferences, or view the list >>>>>> archives, please visit: >>>>>> https://list.valvesoftware.__com/cgi-bin/mailman/listinfo/_ >>>>>> _hlds >>>>>> <https://list.valvesoftware.com/cgi-bin/mailman/listinfo/ >>>>>> hlds> >>>>>> >>>>>> >>>>>> >>>>>> _________________________________________________ >>>>>> To unsubscribe, edit your list preferences, or view the list >>>>>> archives, please visit: >>>>>> https://list.valvesoftware.__com/cgi-bin/mailman/listinfo/__hlds >>>>>> <https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> To unsubscribe, edit your list preferences, or view the list archives, >>>>>> please visit: >>>>>> https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds >>>>>> >>>>>> >>>>>> >>>>> _______________________________________________ >>>>> To unsubscribe, edit your list preferences, or view the list archives, >>>>> please visit: >>>>> https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds >>>>> >>>>> >>>> >>>> >>> _______________________________________________ >> To unsubscribe, edit your list preferences, or view the list archives, >> please visit: >> https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux >> >> > > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list archives, > please visit: > https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux > _______________________________________________ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux
