Re: Lessons learned from string-fix and consequences on StringNg design
Kinkie wrote: On Fri, Feb 13, 2009 at 11:36 AM, Henrik Nordstrom wrote: tor 2009-02-12 klockan 17:19 +0100 skrev Kinkie: It would IMHO make sense to: 1. introduce StringNg::size_type, which should be a _signed_ 32-bit integer 2. introduce a static const StringNg::npos = -1 to be used in place of std::string::npos Or use size_t in the API while using a smaler type in the internal representation. size_t is only making a mess for printf style operations, and it's relatively easy to deal with (and have to be dealt with at a number of other locations..) Also doable. Currently there is no overflow-checks. Should they be added or should we rely in the callers? Anyone in favour of having some debugs() when a Buffer surpasses some predefined dimension? Definately. With a 32-bit signed size it _should_ be large enough for most uses. A debugs when it goes close to or over the signed rollover would be critical IMO. Maybe more, but debugs & assert for now. Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE13 Current Beta Squid 3.1.0.5
Re: Squid 3.1.0.5 uses memory without bounds
On 02/13/2009 01:40 PM, Steinar H. Gunderson wrote: > On Fri, Feb 13, 2009 at 11:55:28AM +0100, Steinar H. Gunderson wrote: > >> Here's a new run. I only let it run for a few minutes, though, so the numbers >> are pretty small (and their relative sizes may not be representable for a >> long-running Squid): >> > > ...and here's a pprof graph: > > http://home.samfundet.no/~sesse/pprof7409.0.ps > > It shows delayRead() as the culprit like the last one, but there's more > details since it doesn't inline everything. > Sorry, I do not know much about pprof. The graph shows that 200MB out of 500MB was allocated to close callbacks for deferred reads (the current suspect), right? Is there an indication that those objects were actually leaked? How long did this test run, and does the 200MB/test_time memory consumption speed correlate with what you have observed in longer tests? Finally, the other 300MB are due to pooled allocations. Those are tracked and reported by cache manager. Do you see any abnormal memory stats when you query your longer-running Squids? Thank you, Alex.
Re: votes..
fre 2009-02-13 klockan 10:25 -0700 skrev Alex Rousskov: > Your recollection is correct. I think it is even documented on the > Squid3 submission instructions wiki page. Indeed it is. http://wiki.squid-cache.org/MergeProcedure#head-3c69e15b207172e0eeb128fef209554218c60eda A little more complicated than I remember giving core and non-core different weights in their votes (core members apparently 50% heavier than the rest..). > A new form of governance! Or is it what elves use? :-) Not new. The elves community are very old you know. In fact the model are used quite a lot, just not explicitly stated so often.. Regards Henrik
Re: Squid 3.1.0.5 uses memory without bounds
On Fri, Feb 13, 2009 at 11:55:28AM +0100, Steinar H. Gunderson wrote: > Here's a new run. I only let it run for a few minutes, though, so the numbers > are pretty small (and their relative sizes may not be representable for a > long-running Squid): ...and here's a pprof graph: http://home.samfundet.no/~sesse/pprof7409.0.ps It shows delayRead() as the culprit like the last one, but there's more details since it doesn't inline everything. /* Steinar */ -- Homepage: http://www.sesse.net/
Re: [MERGE] libsquid-compat shuffling pt 1
On 02/06/2009 08:02 PM, Amos Jeffries wrote: > Alex Rousskov wrote: >> s/libsquid-compat/compat/g >> >> I think we should be including headers as compat/foo/... and not just >> foo/... -I directives need to be adjusted for that to happen. This >> will help with future moves of the compat directory and with avoiding >> file name clashes. > We discussed using "compat/" before. Yes. Did we reach an agreement that squid_compat is required? > There are about 1150 possible reasons to add "squid-" to the directory > name: > http://www.google.co.nz/search?hl=en&q=%2Fcompat%2Fcompat.h Or about 1150 reasons why it is not necessary (since most of those projects probably work fine and I am sure many of them do not prefix directories with a project name). Header clashes are most likely the result of wrong compiler options. Think about it: Millions of projects are using non-unique header names (even without any directories) and build fine. The preprocessor is designed to handle conflicts like this if you give it the right options. The problem is real only when you start installing headers. Squid does not. > This is like an OS API layer used by Squid. Should not be mistaken for > part of the squid codebase itself. I think this code is a part of our codebase, but let's postpone that part of the debate. Thank you, Alex. P.S. I commented on the latest submission before I saw this message; sorry.
Re: Lessons learned from string-fix and consequences on StringNg design
On 02/13/2009 10:31 AM, Kinkie wrote: >> As for overflow checks, I would throw in grow() or equivalent if the >> size gets too big. Adding debugging when the size exceeds some large >> threshold is probably not needed because we can grep for that without an >> explicit debug statement. Or did you want to add a level-1 warning to >> the admin? >> > The latter. Say that a StringNg can be up to 2Gb big. > I'd rather level-1 warn at .5Gb and throw at 2Gb than just throw at 1 > (but then things will have gone bad quite long before that). If you want to warn, then warn and throw at the same time -- there is little an admin can do after seeing a "we think Squid strings are getting too large but we are not going to do anything at this point; please check back later" warning. Not a big deal either way though. Thanks, Alex.
Re: [MERGE] SourceLayout: squid compat library
Alex Rousskov has voted tweak. Status is now: Conditionally approved Comment: s/squid-compat/compat/g Why do you insist on the "squid-" prefix in that directory name? I still think we should always be including headers as compat/foo/... and not just foo/... I believe you have fixed a few places, but some remain (possibly in the compat directory only?). Please check. BTW, is there new code here or just renaming/moving stuff around? In other words, do we need to review the code? For details, see: http://bundlebuggy.aaronbentley.com/project/squid/request/%3C498FEFF9.1030209%40treenet.co.nz%3E Project: Squid
Re: Lessons learned from string-fix and consequences on StringNg design
On Fri, Feb 13, 2009 at 5:51 PM, Alex Rousskov wrote: > On 02/12/2009 09:19 AM, Kinkie wrote: >> Hi all, >>the work on string-fix is nearing completion, and it yielded some >> interesting insight on an issue also affecting the StringNg design, >> I'd like to have some input on: storage size and API >> >> StringNg uses currently size_t as its storage size, which is probably >> a bit impractical. A on 64bit platforms, 2^64 bytes-long single-chunk >> string doesn't really make sense. However, it is necessary for one >> specific bit of API, which is the useage of std::string::npos as a >> return value for "not found" in the various find() functions. >> >> It would IMHO make sense to: >> 1. introduce StringNg::size_type, which should be a _signed_ 32-bit integer >> 2. introduce a static const StringNg::npos = -1 to be used in place of >> std::string::npos >> >> 2Gb-long Strings should suffice for a while, and the String design is >> probably not very sane for Strings whose max is not known in advance >> and are bigger than a few Mb anyways. >> >> What do you guys think? > StringNg::size_type should be added. I believe that was one of the > review items. It is. I had forgotten. > StringNg::npos should be added. Sorry if I missed that during the review > (IIRC, I thought you already had it). It's not, will add. > Whether StringNg::size_type is std::string::size_type, size_t, > std::size_type, or something else is not very important for now, IMO. If > you feel like saving a few bytes on a 64-bit platform, feel free to do > that. Naturally, the more non-standard your StringNg::size_type is, the > more conversions (explicit or implicit) we would have to deal with. If properly parametrized, changing that is relatively easy (done that twice with OldString). psize() and friend macros help a lot there. > As for overflow checks, I would throw in grow() or equivalent if the > size gets too big. Adding debugging when the size exceeds some large > threshold is probably not needed because we can grep for that without an > explicit debug statement. Or did you want to add a level-1 warning to > the admin? The latter. Say that a StringNg can be up to 2Gb big. I'd rather level-1 warn at .5Gb and throw at 2Gb than just throw at 1 (but then things will have gone bad quite long before that). -- /kinkie
Re: votes..
On 02/11/2009 03:17 PM, Henrik Nordstrom wrote: > tor 2009-02-12 klockan 08:25 +1100 skrev Robert Collins: > >> On Wed, 2009-02-11 at 23:20 +0200, Tsantilas Christos wrote: >> >>> May I ask, about the votes, which is the procedure? Who is giving access >>> to bundlebuggy? Or it is something I do not need it? >>> >> I can add users, but I don't consider myself the decider - rather the >> community here is. >> > iirc the defined voting procedure is that all registered developers have > vote rights (that's everyone subscribed on squid-dev I suppose, within > reason), and the core team have veto rights. > Your recollection is correct. I think it is even documented on the Squid3 submission instructions wiki page. In practice, I am guessing Robert cannot or does not want to add all squid-dev subscribers automatically so one needs to ask him to be added to the BB voting registry. Robert, is there a way to auto-add all squid-dev? Also, BB cannot distinguish core votes from others, right? > it's supposed to be an enlightened community, not a democracy or > monarchy. A new form of governance! Or is it what elves use? :-) Cheers, Alex.
Re: Lessons learned from string-fix and consequences on StringNg design
On 02/12/2009 09:19 AM, Kinkie wrote: > Hi all, >the work on string-fix is nearing completion, and it yielded some > interesting insight on an issue also affecting the StringNg design, > I'd like to have some input on: storage size and API > > StringNg uses currently size_t as its storage size, which is probably > a bit impractical. A on 64bit platforms, 2^64 bytes-long single-chunk > string doesn't really make sense. However, it is necessary for one > specific bit of API, which is the useage of std::string::npos as a > return value for "not found" in the various find() functions. > > It would IMHO make sense to: > 1. introduce StringNg::size_type, which should be a _signed_ 32-bit integer > 2. introduce a static const StringNg::npos = -1 to be used in place of > std::string::npos > > 2Gb-long Strings should suffice for a while, and the String design is > probably not very sane for Strings whose max is not known in advance > and are bigger than a few Mb anyways. > > What do you guys think? StringNg::size_type should be added. I believe that was one of the review items. StringNg::npos should be added. Sorry if I missed that during the review (IIRC, I thought you already had it). Whether StringNg::size_type is std::string::size_type, size_t, std::size_type, or something else is not very important for now, IMO. If you feel like saving a few bytes on a 64-bit platform, feel free to do that. Naturally, the more non-standard your StringNg::size_type is, the more conversions (explicit or implicit) we would have to deal with. As for overflow checks, I would throw in grow() or equivalent if the size gets too big. Adding debugging when the size exceeds some large threshold is probably not needed because we can grep for that without an explicit debug statement. Or did you want to add a level-1 warning to the admin? HTH, Alex.
Squid Developer? (fwd)
Duane: I'm a second year student at the Harvard Business School. Prior to coming back to school for my MBA I was the co-founder of something called Project Honey Pot (www.projecthoneypot.org). Project Honey Pot tracks malicious behavior online. We work with law enforcement agencies worldwide and have been instrumental in several of the high profile cases shutting down spam gangs and other online criminals. Generally, Project Honey Pot has operated as a public service project with contributions from members in more than 120 countries around the world. Talking with some of the entrepreneurial faculty around HBS I'm being coaxed into believing that there's a potentially interesting and disruptive business opportunity that could be spun off of the Project's data in order to protect websites. They're very interested in making the introductions to get this idea funded. Imagine something like a CDN for any website with the benefits of advanced, threat-based firewall++ protection. I've poked around with my technical team from Project Honey Pot as well as some CS friends I have over at MIT. Squid has repeatedly come up as a potential base platform onto which we could build the layers of the service. I was wondering if in the process of managing the Squid development these days you'd run across any talented, eager programmers who might be searching for a startup opportunity that has significant backing, a large market opportunity, and where they could play a meaningful role in growing a new company. I'm agnostic as to geographic locations so long as the person is talented and self-motivated. If any names spring to mind, I hope you won't hesitate shooting them my way. Best wishes, Matthew Prince. Email: matthew -at- mba2009 -dot- hbs -dot- edu
Re: Memory leak in ICAPXaction
On 02/11/2009 06:06 PM, Amos Jeffries wrote: >> Amos Jeffries wrote: >> >>> This is also popping out, and very confusing since ICAPXaction global is >>> only created once right? >>> >> it is the "new PconnPool" line. It is create only once. It says its >> reachable because the time of shutdown the icapPconnPool points on it. >> > ... which is preventing memPools deallocating any of it... > >> Should not be a problem... >> > Aye, the code _looks right_ but there is something fishy going on. > Unless I'm fully mistaken on what a block is, a single call to xmalloc > should not be generating: "155,949 blocks" in case #12, or again a single > call to xcalloc generating "3,122 blocks" for case #2. > I am guessing that the blocks reported are not the blocks allocated by that single malloc call to create icapPconnPool. They are the blocks allocated elsewhere (e.g., ICAP connection objects, callbacks, buffers, etc.) and tied to the pool. Valgrind did not say "leaked". It said "reachable". Reachable via icapPconnPool global. BTW, if we want to make sure everything is deallocated at shutdown, we can add glue do that. I am not sure this should be a top-priority item though. >>> ==23075== 16,245,636 bytes in 155,949 blocks are still reachable in loss >>> record 29 of 30 >>> ==23075==at 0x4C2260E: malloc (vg_replace_malloc.c:207) >>> ==23075==by 0x5C4F07: xmalloc (util.c:506) >>> ==23075==by 0x5A591D: _GLOBAL__I_ICAPXaction.cc (SquidNew.h:48) >>> ==23075==by 0x5C9155: (within /usr/sbin/squid3) >>> ==23075==by 0x47E9AA: (within /usr/sbin/squid3) >>> ==23075== >>> ==23075== >>> ==23075== 44,001,178 bytes in 3,122 blocks are still reachable in loss >>> record 30 of 30 >>> >> Are the 44Mbytes normal for a PconnPool object? >> > I hope not. It's meant to be a simple hash of currently active persistent > connections. > It is a collection of idle persistent connections with all associated data. Depending on the load and the time of the snapshot, 44MB can be normal. There could be thousands of idle connections, especially if Squid just stopped processing transactions. HTH, Alex.
Re: Squid 3.1.0.5 uses memory without bounds
On 02/13/2009 03:55 AM, Steinar H. Gunderson wrote: > On Fri, Feb 13, 2009 at 09:01:35PM +1300, Amos Jeffries wrote: > >> A few things are still a little hidden due to the compiler optimizations >> and valgrind behavior, would it be possible that you rebuild squid with >> --disable-optimizations and also make sure that --enable-valgrind is >> used. Then generate a new valgrind trace from the rebuild Squid? >> > > Here's a new run. I only let it run for a few minutes, though, so the numbers > are pretty small (and their relative sizes may not be representable for a > lon Hi Steinar, Was --enable-valgrind used last time you reconfigured and rebuilt Squid from scratch? Are you on 32- or 64-bit platform? Would you be able to run Squid with full debugging enabled? It may take a few iterations if we need to add debugging statements, but we should be able to see the leak based on debugging messages alone. If leaks occur all the time, you would not have to run Squid for a long time. Developers, If valgrind is not lying, we are leaking close handlers for deferred reads and possibly other close handlers. I do not see any bugs in the related code, but that does not mean there aren't any. The cbdata escape into non-C++ world and CbDataList-related manipulations in DeferredReadManager::delayRead would be my primary suspects even though I do not see anything seriously broken in that code. If, somehow, the callback data gets invalidated without CbDataList knowledge, we may leak the associated object (CbDataList::element). I do not see how that can happen though. Thank you, Alex. > ==26729== Syscall param write(buf) points to uninitialised byte(s) > ==26729==at 0x4E33720: __write_nocancel (in /lib/libpthread-2.7.so) > ==26729==by 0x50F7BC: default_write_method(int, char const*, int) > (fd.cc:167) > ==26729==by 0x5CB5F2: UFSCleanLog::write(StoreEntry const&) > (store_dir_ufs.cc:941) > ==26729==by 0x593778: storeDirWriteCleanLogs (store_dir.cc:475) > ==26729==by 0x5595DD: SquidShutdown() (main.cc:1686) > ==26729==by 0x55B259: SquidMain(int, char**) (main.cc:1352) > ==26729==by 0x55B3D8: SquidMainSafe(int, char**) (main.cc:1110) > ==26729==by 0x55B4D1: main (main.cc:1103) > ==26729== Address 0xcd71201 is 1 bytes inside a block of size 16,384 > alloc'd > ==26729==at 0x4C206F4: calloc (vg_replace_malloc.c:397) > ==26729==by 0x619727: xcalloc (util.c:688) > ==26729==by 0x5CC26C: UFSSwapDir::writeCleanStart() > (store_dir_ufs.cc:883) > ==26729==by 0x593591: storeDirWriteCleanLogs (store_dir.cc:445) > ==26729==by 0x5595DD: SquidShutdown() (main.cc:1686) > ==26729==by 0x55B259: SquidMain(int, char**) (main.cc:1352) > ==26729==by 0x55B3D8: SquidMainSafe(int, char**) (main.cc:1110) > ==26729==by 0x55B4D1: main (main.cc:1103) > ==26729== > ==26729== Syscall param write(buf) points to uninitialised byte(s) > ==26729==at 0x4E33720: __write_nocancel (in /lib/libpthread-2.7.so) > ==26729==by 0x50F7BC: default_write_method(int, char const*, int) > (fd.cc:167) > ==26729==by 0x5CB88B: UFSSwapDir::writeCleanDone() > (store_dir_ufs.cc:971) > ==26729==by 0x593892: storeDirWriteCleanLogs (store_dir.cc:487) > ==26729==by 0x5595DD: SquidShutdown() (main.cc:1686) > ==26729==by 0x55B259: SquidMain(int, char**) (main.cc:1352) > ==26729==by 0x55B3D8: SquidMainSafe(int, char**) (main.cc:1110) > ==26729==by 0x55B4D1: main (main.cc:1103) > ==26729== Address 0xcd71201 is 1 bytes inside a block of size 16,384 > alloc'd > ==26729==at 0x4C206F4: calloc (vg_replace_malloc.c:397) > ==26729==by 0x619727: xcalloc (util.c:688) > ==26729==by 0x5CC26C: UFSSwapDir::writeCleanStart() > (store_dir_ufs.cc:883) > ==26729==by 0x593591: storeDirWriteCleanLogs (store_dir.cc:445) > ==26729==by 0x5595DD: SquidShutdown() (main.cc:1686) > ==26729==by 0x55B259: SquidMain(int, char**) (main.cc:1352) > ==26729==by 0x55B3D8: SquidMainSafe(int, char**) (main.cc:1110) > ==26729==by 0x55B4D1: main (main.cc:1103) > ==26729== > ==26729== Invalid read of size 1 > ==26729==at 0x5B0F6B: isOpen(int) (comm.cc:284) > ==26729==by 0x5B1D0A: comm_remove_close_handler(int, > RefCount&) (comm.cc:1704) > ==26729==by 0x5B1E15: > DeferredReadManager::popHead(CbDataListContainer&) > (comm.cc:2521) > ==26729==by 0x5B1EE0: DeferredReadManager::flushReads() (comm.cc:2557) > ==26729==by 0x5B1F97: DeferredReadManager::~DeferredReadManager() > (comm.cc:2479) > ==26729==by 0x55FCD8: MemObject::~MemObject() (MemObject.cc:129) > ==26729==by 0x58900F: StoreEntry::destroyMemObject() (store.cc:376) > ==26729==by 0x589765: destroyStoreEntry(void*) (store.cc:389) > ==26729==by 0x612AE6: hashFreeItems (hash.c:308) > ==26729==by 0x591E53: StoreHashIndex::~StoreHashIndex() > (store_dir.cc:718) > ==26729==b
Re: Lessons learned from string-fix and consequences on StringNg design
On Fri, Feb 13, 2009 at 11:36 AM, Henrik Nordstrom wrote: > tor 2009-02-12 klockan 17:19 +0100 skrev Kinkie: >> It would IMHO make sense to: >> 1. introduce StringNg::size_type, which should be a _signed_ 32-bit integer >> 2. introduce a static const StringNg::npos = -1 to be used in place of >> std::string::npos > > Or use size_t in the API while using a smaler type in the internal > representation. > > size_t is only making a mess for printf style operations, and it's > relatively easy to deal with (and have to be dealt with at a number of > other locations..) Also doable. Currently there is no overflow-checks. Should they be added or should we rely in the callers? Anyone in favour of having some debugs() when a Buffer surpasses some predefined dimension? -- /kinkie
Re: Squid 3.1.0.5 uses memory without bounds
On Fri, Feb 13, 2009 at 09:01:35PM +1300, Amos Jeffries wrote: > A few things are still a little hidden due to the compiler optimizations > and valgrind behavior, would it be possible that you rebuild squid with > --disable-optimizations and also make sure that --enable-valgrind is > used. Then generate a new valgrind trace from the rebuild Squid? Here's a new run. I only let it run for a few minutes, though, so the numbers are pretty small (and their relative sizes may not be representable for a long-running Squid): ==26729== Syscall param write(buf) points to uninitialised byte(s) ==26729==at 0x4E33720: __write_nocancel (in /lib/libpthread-2.7.so) ==26729==by 0x50F7BC: default_write_method(int, char const*, int) (fd.cc:167) ==26729==by 0x5CB5F2: UFSCleanLog::write(StoreEntry const&) (store_dir_ufs.cc:941) ==26729==by 0x593778: storeDirWriteCleanLogs (store_dir.cc:475) ==26729==by 0x5595DD: SquidShutdown() (main.cc:1686) ==26729==by 0x55B259: SquidMain(int, char**) (main.cc:1352) ==26729==by 0x55B3D8: SquidMainSafe(int, char**) (main.cc:1110) ==26729==by 0x55B4D1: main (main.cc:1103) ==26729== Address 0xcd71201 is 1 bytes inside a block of size 16,384 alloc'd ==26729==at 0x4C206F4: calloc (vg_replace_malloc.c:397) ==26729==by 0x619727: xcalloc (util.c:688) ==26729==by 0x5CC26C: UFSSwapDir::writeCleanStart() (store_dir_ufs.cc:883) ==26729==by 0x593591: storeDirWriteCleanLogs (store_dir.cc:445) ==26729==by 0x5595DD: SquidShutdown() (main.cc:1686) ==26729==by 0x55B259: SquidMain(int, char**) (main.cc:1352) ==26729==by 0x55B3D8: SquidMainSafe(int, char**) (main.cc:1110) ==26729==by 0x55B4D1: main (main.cc:1103) ==26729== ==26729== Syscall param write(buf) points to uninitialised byte(s) ==26729==at 0x4E33720: __write_nocancel (in /lib/libpthread-2.7.so) ==26729==by 0x50F7BC: default_write_method(int, char const*, int) (fd.cc:167) ==26729==by 0x5CB88B: UFSSwapDir::writeCleanDone() (store_dir_ufs.cc:971) ==26729==by 0x593892: storeDirWriteCleanLogs (store_dir.cc:487) ==26729==by 0x5595DD: SquidShutdown() (main.cc:1686) ==26729==by 0x55B259: SquidMain(int, char**) (main.cc:1352) ==26729==by 0x55B3D8: SquidMainSafe(int, char**) (main.cc:1110) ==26729==by 0x55B4D1: main (main.cc:1103) ==26729== Address 0xcd71201 is 1 bytes inside a block of size 16,384 alloc'd ==26729==at 0x4C206F4: calloc (vg_replace_malloc.c:397) ==26729==by 0x619727: xcalloc (util.c:688) ==26729==by 0x5CC26C: UFSSwapDir::writeCleanStart() (store_dir_ufs.cc:883) ==26729==by 0x593591: storeDirWriteCleanLogs (store_dir.cc:445) ==26729==by 0x5595DD: SquidShutdown() (main.cc:1686) ==26729==by 0x55B259: SquidMain(int, char**) (main.cc:1352) ==26729==by 0x55B3D8: SquidMainSafe(int, char**) (main.cc:1110) ==26729==by 0x55B4D1: main (main.cc:1103) ==26729== ==26729== Invalid read of size 1 ==26729==at 0x5B0F6B: isOpen(int) (comm.cc:284) ==26729==by 0x5B1D0A: comm_remove_close_handler(int, RefCount&) (comm.cc:1704) ==26729==by 0x5B1E15: DeferredReadManager::popHead(CbDataListContainer&) (comm.cc:2521) ==26729==by 0x5B1EE0: DeferredReadManager::flushReads() (comm.cc:2557) ==26729==by 0x5B1F97: DeferredReadManager::~DeferredReadManager() (comm.cc:2479) ==26729==by 0x55FCD8: MemObject::~MemObject() (MemObject.cc:129) ==26729==by 0x58900F: StoreEntry::destroyMemObject() (store.cc:376) ==26729==by 0x589765: destroyStoreEntry(void*) (store.cc:389) ==26729==by 0x612AE6: hashFreeItems (hash.c:308) ==26729==by 0x591E53: StoreHashIndex::~StoreHashIndex() (store_dir.cc:718) ==26729==by 0x593F0A: RefCount::dereference(StoreHashIndex const*) (RefCount.h:100) ==26729==by 0x593F27: RefCount::~RefCount() (RefCount.h:52) ==26729== Address 0x70d8 is not stack'd, malloc'd or (recently) free'd ==26729== ==26729== Process terminating with default action of signal 11 (SIGSEGV) ==26729== Access not within mapped region at address 0x70D8 ==26729==at 0x5B0F6B: isOpen(int) (comm.cc:284) ==26729==by 0x5B1D0A: comm_remove_close_handler(int, RefCount&) (comm.cc:1704) ==26729==by 0x5B1E15: DeferredReadManager::popHead(CbDataListContainer&) (comm.cc:2521) ==26729==by 0x5B1EE0: DeferredReadManager::flushReads() (comm.cc:2557) ==26729==by 0x5B1F97: DeferredReadManager::~DeferredReadManager() (comm.cc:2479) ==26729==by 0x55FCD8: MemObject::~MemObject() (MemObject.cc:129) ==26729==by 0x58900F: StoreEntry::destroyMemObject() (store.cc:376) ==26729==by 0x589765: destroyStoreEntry(void*) (store.cc:389) ==26729==by 0x612AE6: hashFreeItems (hash.c:308) ==26729==by 0x591E53: StoreHashIndex::~StoreHashIndex() (store_dir.cc:718) ==26729==by 0x593F0A: RefCount::dereference(StoreHashIndex const*) (RefCount.h:100) ==26729==b
Re: Lessons learned from string-fix and consequences on StringNg design
tor 2009-02-12 klockan 17:19 +0100 skrev Kinkie: > It would IMHO make sense to: > 1. introduce StringNg::size_type, which should be a _signed_ 32-bit integer > 2. introduce a static const StringNg::npos = -1 to be used in place of > std::string::npos Or use size_t in the API while using a smaler type in the internal representation. size_t is only making a mess for printf style operations, and it's relatively easy to deal with (and have to be dealt with at a number of other locations..) Regards Henrik
Re: Squid 3.1.0.5 uses memory without bounds
Steinar H. Gunderson wrote: On Sun, Feb 08, 2009 at 12:38:42AM +0100, Steinar H. Gunderson wrote: I'm using Squid 3.1.0.5 on my amd64 system (Debian unstable), and I'm having problems that it seems to use RAM without bounds (about 1GB/week) even though my cache_mem is only 128MB. I ran it through valgrind, and it showed a few leaks and lots of memory that was still reachable. (I started Squid using -NYC, and exited it with Ctrl-C.) I hope someone can find and fix these issues; restarting Squid every week is not a very good option :-) Hi Steinar, Alex has been looking into this one as well now. I've cc'd so you can send him a trace as well if you continue to have squid-dev issues. A few things are still a little hidden due to the compiler optimizations and valgrind behavior, would it be possible that you rebuild squid with --disable-optimizations and also make sure that --enable-valgrind is used. Then generate a new valgrind trace from the rebuild Squid? Thanks. Amos -- Please be using Current Stable Squid 2.7.STABLE6 or 3.0.STABLE13 Current Beta Squid 3.1.0.5
