Of Obtaining Memory and Memory Leaks. (If you speak C or C++ in your
sleep, you don't need to read this. If you feel like replying to say
"TLDR", just jump to the last paragraph.)
With programming languages like C and C++, there are two ways of
obtaining memory: static and dynamic allocation.
Static allocation happens when a variable is declared. This memory is
already pre-reserved in the executable (in the section named .data or
_DATA or something like that) so anytime you do this, you make the .exe
or binary grow.
void func()
{
int d;
}
This snippet reserves enough bytes to hold a variable of the size "int",
which, in most cases, is 32 bits.
On the other hand, dynamic memory is created by explicitly asking for
it. This happens in C via the function "malloc" and in C++ via the
keyword "new". (You can use malloc() in C++ too, but people will
probably make fun of you.)
void func()
{
int *d, *e;
d = malloc(sizeof(int));
e = new int;
}
This snippet statically allocates 32 or 64 (depending on if your
operating system is 32- or 64-bit) bits each for the pointers "d" and
"e". When the program runs, the two lines ask for additional 32 bits of
memory each -- this is the dynamic allocation.
Since most of C++ code is embedded in classes which are instantiated,
nearly all of memory in a C++ program is dynamically allocated.
Now, consider that whenever our func() is called, we obtain 2x32=64 bits
of memory. If we were to call func() 134,217,728 times, we'd end up
taking up a gig of memory, and maybe even run out of it -- and the
operating system starts swapping, as ics said. The way out is to tell
the operating system to free the dynamically allocated memory we don't
need anymore. (We can do this since no other piece of this stupid
example program can or will access "d" or "e" once the function ends.)
void func()
{
int *d, *e;
d = malloc(sizeof(int));
e = new int;
// do stuff
free(d);
delete e;
}
This leads to us returning our 64 bits for the kernel to use for
something different. Sure, we still have the other 64 or 128 bits of
memory to store the pointers, but if you end up working with large
structures or classes, a pointer is really tiny. (malloc()'d memory is
free()'d, "new"d memory is "delete"d. Mixing them up might make the
program crash eventually... and people will laugh at you even more.)
Now, the main reason for memory leaks is that somebody malloc()'d or
new'd a piece of memory and forgot to free() or delete it. The problem
in large programs is that when memory is reserved, it might be e.g.
passed as a return value and therefore cannot be free()'d right at the
end of the function. It is then up to the calling function or some sort
of cleanup function to get rid of this memory again when the time is
right. (Freeing the memory and accessing it later will cause problems,
so you really need to find the right timepoint.) Should somebody forget
to do this, the program basically wastes the memory since the pointer
(the address to the data) is lost but the data wasn't marked for
removal, so the kernel can't just give it away again. The technical term
for such a situation is "memory leak".
So it's really not about what operating system you are using, it is
about a programmer asking for memory and forgetting to give it back.
Such a massive codebase as the Source engine and the game code is a
burden for any programmer, and getting rid of those leaks is often not
that easy since you need to know the code very well to be able to say,
"at this point, nobody needs this data anymore". However, three
megabytes per map already is quite a lot.
I hope this shone some light on the issue... and that it wasn't too
complicated to understand. This, after all, isn't [hlcoders].
~~ Ondra
On 08.08.07 16:29 Uhr, ics wrote:
Its definitely not the fault of linux or the way it is built. Usually
when process doesnt need memory or it is shut down, it frees the memory
it was using and that can be given by the system for other processes but
in srcds case, it tends to keep the memory and not freeing it, untill
its shut down. I'm sure srcds process doesnt need all that memory itself
that it is eating - it just couldnt free it. Yes of course the memory
usage is different in linux and windows, but in linux, the system tends
to reserve the whole memory for itself and then distributes it for the
processes that need it. I think you are referring to it.
However, if 1 process uses all the memory, at some point, there is no
free memory for the system to share. Then machine starts swapping and
thats bad. For example, while i watch mem usage on my desktop with
'top', it says that 910632k of the mem is used and 125092k is free.
Freshly booted 2 hours ago. This is of course the reserved memory by the
system, not the real thing. Real values for free memory seen with 'free'
are 259568 in use and 776156 free. Servers work same way. Eventually the
memory runs out. Not a problem if you reboot your gameserver daily but
still the leak has been annoying.
-ics
Svensk Ljud & Ljus Produktion kirjoitti:
I dont think its all a bug, its the way how linux is built.
As long as we have more memory in the box then its using at start and
full,
then its just no problems how its moving the free memory around !
Its only a problem for thouse who is used to look at how windows is
using
the memory, its not the same way !
Peter
----- Original Message -----
From: "ics" <[EMAIL PROTECTED]>
To: <hlds_linux@list.valvesoftware.com>
Sent: Wednesday, August 08, 2007 3:11 PM
Subject: Re: [hlds_linux] Source Engine/Dedicated Server Beta
forget zblock, it is not the main reason. Every CSS server has that
memory leak, vanilla or plugin enhanced. Leave your server on for a week
and you will see that its eating 4-6 times the memory that it ate at
when you first time started it. This leak has been in there for years,
good thing that its finally fixed (if it really is).
-ics
Thomas h kirjoitti:
I only suspected zblock to be "the bad guy", because i didnt notice
any of
the non-zblock running server to reach a level higher than 250-300mb
of ram.
But that could just as well be that they rarely change map on those
servers,
making in consistent with the memleak bug in the src engine.
Regards
Thomas Hjorth
http://DSRack.dk
[ Picked text/plain from multipart/alternative ]
Thomas Hjorth, that's actually the first time I ever heard anyone say
that
zBlock
were to have a memory leak, and seeing as ow zBlock barely actuallyt
uses
any
memory at all I can't imagine it leaking near a GB in only a few
days. I'll
check it
out though.
On 8/8/07, Thomas h <[EMAIL PROTECTED]> wrote:
>
> The memleak is there, though I though it was with zblock, since only
those
> servers used way to much memory. I had 1 server using 1.1GB of
memory
> within
> a few days. Restarting the server cleans up the mess, ofc.
>
> Regards
> Thomas Hjorth
> http://DSRack.dk
>
> >[ Picked text/plain from multipart/alternative ]
> >- Fixed a server memory leak (2-3 megs per map change)
> >
> >Holy mother of God. I am speechless if this is true. =o
> >
> >-tsuehpsyde
> >SourceKills.com
> >
> >On 8/7/07, Guy Watkins <[EMAIL PROTECTED]> wrote:
> > >
> > > What was the smoke grenade exploit? Was it the one that allowed
> someone
> > > to
> > > jump really high?
> > >
> > > Thanks,
> > > Guy
> > >
> > > } -----Original Message-----
> > > } From: [EMAIL PROTECTED]
[mailto:hlds_linux-
> > > } [EMAIL PROTECTED] On Behalf Of Jason Ruymen
> > > } Sent: Tuesday, August 07, 2007 9:47 PM
> > > } To: hlds_linux@list.valvesoftware.com;
[EMAIL PROTECTED]
> > > } Subject: [hlds_linux] Source Engine/Dedicated Server Beta
> > > }
> > > } A new Beta is available for the Source Engine and Source
Dedicated
> > > } Server. To participate, run hldsupdatetool with the command
"-beta
> > > } community". The specific changes include:
> > > }
> > > } - Fixed hl2mp bug where if you +used a weapon it would not
respawn
> > > } anymore
> > > } - Fixed hl2mp spectator being able to lock the player he's
viewing
> by
> > > } typing "jointeam 0"
> > > } - Fixed a CS:S smoke grenade exploit
> > > } - Added a way for server plugins to ask players if they'd
like to
> > > } connect to a different server
> > > } - Made materials handle sv_pure correctly when they've
included
> other
> > > } materials
> > > } - Fixed Windows dedicated server bug slowing down Steam
updates
> > > } - Fixed a server memory leak (2-3 megs per map change)
> > > } - Fixed IVEngineServer::FadeClientVolume
> > > } - Fixed certain models showing up all white when sv_pure
flushed
> them
> > > } - Made the server print a client's steam ID and show sv_pure
> warnings
> >if
> > > } sv_pure_kick_clients is 0
> > > } - Fixed a bug preventing sv_pure CRCs from verifying
correctly on
> > > } certain custom maps
> > > }
> > > } Jason
> > > }
> > > } _______________________________________________
> > > } To unsubscribe, edit your list preferences, or view the list
> archives,
> > > } please visit:
> > > } http://list.valvesoftware.com/mailman/listinfo/hlds_linux
> > >
> > >
> > > _______________________________________________
> > > To unsubscribe, edit your list preferences, or view the list
archives,
> > > please visit:
> > > http://list.valvesoftware.com/mailman/listinfo/hlds_linux
> > >
> >--
> >
> >_______________________________________________
> >To unsubscribe, edit your list preferences, or view the list
archives,
> >please visit:
> >http://list.valvesoftware.com/mailman/listinfo/hlds_linux
>
> _________________________________________________________________
> Vælg selv hvordan du vil kommunikere - skrift, tale, video eller
billeder
> med MSN Messenger: http://messenger.msn.dk
>
>
> _______________________________________________
> To unsubscribe, edit your list preferences, or view the list
archives,
> please visit:
> http://list.valvesoftware.com/mailman/listinfo/hlds_linux
>
--
___________________________
Wim 'TheUnknownFactor' Barelds
[EMAIL PROTECTED]
--
_______________________________________________
To unsubscribe, edit your list preferences, or view the list archives, please
visit:
http://list.valvesoftware.com/mailman/listinfo/hlds_linux
