So I've got two demos here. The first one, go to about tick 19000 and wait to around the point where Fred 2.0 joins. The server drops to 5 frames. I removed Sourcemod and recorded on another server. I wasn't able to get those huge drops but there are big var spikes ~10 and it *seems* to happen as clients join.
http://dropcanvas.com/8itl3/1 http://dropcanvas.com/8itl3/2 I can't think of what else it could be. We haven't changed anything about the setup from before Steampipe to after. On Mon, May 6, 2013 at 1:00 PM, Bjorn Wielens <uniac...@yahoo.ca> wrote: > > > Thanks for investigating, Fletcher. > > I can tell you that in our case, we have an 8 core system with no CPU > affinity assigned (with 12 TF2 servers running comfortably with CPU to > spare, srcds generally consumes 30% but can get mid 40s when full of > players), Watching top, the load is fairly evenly distributed across the > individual cores, and monitoring resource usage showed only one process > spiking in usage at a time. > > > For what it's worth, I haven't had the issue recur after disabling replay > on the servers in question. For me, that would mean the next step is to > re-enable it and see if the problem returns - I'll post back here when I > get those results (though it's entirely possible that the additional replay > overhead is the metaphorical straw on the camel's back) and any additional > information I uncover. > > > > > > ________________________________ > From: Fletcher Dunn <fletch...@valvesoftware.com> > To: Half-Life dedicated Linux server mailing list < > hlds_linux@list.valvesoftware.com> > Cc: "Half-Life dedicated Win32 server mailing list ( > h...@list.valvesoftware.com)" <h...@list.valvesoftware.com> > Sent: Monday, May 6, 2013 1:41:18 PM > Subject: Re: [hlds_linux] Huge sv drops and var spikes in net_graph 4. > > > I was able to get a trace from a Windows server operator that was > experiencing a similar problem. The problem in that case was that it was > spending all of its time compressing snapshots. The problem was > exacerbated by some process affinity settings which were counterproductive > and putting all the threads on the same core, while other cores were idle. > > Tracking the overall CPU usage can be quite deceiving when investigating > "lag". Lower CPU usage is not necessarily better! For example, in this > case if the process affinity were not set and more cores were allowed to be > used, then the CPU usage on this 4-core system would have gone above 25% > for a shorter burst, which would have been preferable to it being pegged at > 25% (a single core saturated for a longer period of time). > > I think it is a reasonable guess that his problem is similar to the one > you guys are seeing. (Of course they could be totally unrelated, so take > all this with a grain of salt.) But assuming that they are similar > problems, there are two things to investigate. First, did something change > that is doing something subtly different with process affinity that it > wasn't doing before, or is the process affinity thing totally a config > issue with one server? (Or if anybody else is tweaking processor > affinities, do you fully understand the implications of it?) When you are > looking at the CPU usage, is the process pegging a single CPU, or is it > pegging at some round fraction like 1/4 or 1/8? The second thing to > investigate is, what is happening at the high level that is causing the > snapshot data to be so big? Do network activity spikes correlates with the > CPU spikes? If that is the case, then we can use -netspike or other tools > that you guys are > probably more familiar with than I am to track down what the entity data > is that is causing all the busywork. > > We have very powerful performance analysis tools we can use on Windows to > track these problems down. I only wish more windows server operators users > would get setup with them! Please help us help you! It's not that > difficult to get set up. Email me directly and I can get you a CD key. > https://support.steampowered.com/kb_article.php?ref=3059-RTXB-2672 > > -----Original Message----- > From: hlds_linux-boun...@list.valvesoftware.com [mailto: > hlds_linux-boun...@list.valvesoftware.com] On Behalf Of Essay Tew Phaun > Sent: Saturday, May 04, 2013 11:54 AM > To: Half-Life dedicated Linux server mailing list > Subject: Re: [hlds_linux] Huge sv drops and var spikes in net_graph 4. > > Hmm. We didn't have this problem before Steampipe though. We just use the > STV with a couple of slots open on it so that an admin can connect and > watch for reported players. > > > On Sat, May 4, 2013 at 2:44 PM, ics <i...@ics-base.net> wrote: > > > STV is known to cause issues and hogs a lot of memory. That's why > > rarely no one uses it on public servers. > > > > -ics > > > > Essay Tew Phaun kirjoitti: > > > > Interesting. We're not hosting a replay, but we do have STV. > >> > >> > >> On Sat, May 4, 2013 at 12:06 PM, Bjorn Wielens <uniac...@yahoo.ca> > wrote: > >> > >> Sounds like the same problem I'm having (see last few posts in 'CPU > >>> Increase' thread). > >>> > >>> > >>> > >>> > >>> ______________________________**__ > >>> From: Michael Johansen <michs...@live.no> > >>> To: Half-Life dedicated Linux server mailing list < > >>> hlds_linux@list.valvesoftware.**com > >>> <hlds_linux@list.valvesoftware.com>> > >>> Sent: Saturday, May 4, 2013 12:43:59 PM > >>> Subject: Re: [hlds_linux] Huge sv drops and var spikes in net_graph 4. > >>> > >>> > >>> We've noticed this as well, no complaints though as the players does > >>> not seem to notice it very much. > >>> > >>> Date: Sat, 4 May 2013 11:40:54 -0400 > >>>> From: sc2p...@gmail.com > >>>> To: > >>>> hlds_linux@list.valvesoftware.**com<hlds_linux@list.valvesoftware.c > >>>> om> > >>>> Subject: [hlds_linux] Huge sv drops and var spikes in net_graph 4. > >>>> > >>>> We never had this issue prior to Steampipe, but we're dealing with > >>>> these very random and rare spikes of lag where the server framerate > >>>> will > >>>> > >>> suddenly > >>> > >>>> drop from 66 to below 20 and var will spike to 20+. We're hosting 6 > >>>> 24 > >>>> > >>> slot > >>> > >>>> servers on the following: > >>>> > >>>> E3-1230v2 > >>>> 16GB > >>>> CentOS 6.4 64bit > >>>> > >>>> I was able to capture a screenshot of one of the drops but they're > >>>> very hard to catch since they appear very very briefly. > >>>> > >>>> http://i.imgur.com/wxp8P3N.png > >>>> > >>>> server.cfg > >>>> > >>>> sv_minrate 35000 > >>>> sv_maxrate 200000 > >>>> sv_maxupdaterate 67 > >>>> sv_minupdaterate 66 > >>>> sv_mincmdrate 66 > >>>> sv_maxcmdrate 67 > >>>> sv_client_cmdrate_difference 100 > >>>> > >>>> > >>>> Thanks > >>>> ______________________________**_________________ > >>>> To unsubscribe, edit your list preferences, or view the list > >>>> archives, > >>>> > >>> please visit: > >>> > >>>> https://list.valvesoftware.**com/cgi-bin/mailman/listinfo/**hlds_li > >>>> nux<https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_li > >>>> nux> > >>>> > >>> ______________________________**_________________ > >>> To unsubscribe, edit your list preferences, or view the list > >>> archives, please visit: > >>> https://list.valvesoftware.**com/cgi-bin/mailman/listinfo/**hlds_lin > >>> ux<https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linu > >>> x> ______________________________**_________________ > >>> To unsubscribe, edit your list preferences, or view the list > >>> archives, please visit: > >>> https://list.valvesoftware.**com/cgi-bin/mailman/listinfo/**hlds_lin > >>> ux<https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linu > >>> x> > >>> > >>> ______________________________**_________________ > >> To unsubscribe, edit your list preferences, or view the list > >> archives, please visit: > >> https://list.valvesoftware.**com/cgi-bin/mailman/listinfo/**hlds_linu > >> x<https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux> > >> > > > > > > ______________________________**_________________ > > To unsubscribe, edit your list preferences, or view the list archives, > > please visit: > > https://list.valvesoftware.**com/cgi-bin/mailman/listinfo/**hlds_linux > > <https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux> > > > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list archives, > please visit: > https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux > > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list archives, > please visit: > https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux > _______________________________________________ > To unsubscribe, edit your list preferences, or view the list archives, > please visit: > https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux > _______________________________________________ To unsubscribe, edit your list preferences, or view the list archives, please visit: https://list.valvesoftware.com/cgi-bin/mailman/listinfo/hlds_linux
