On Tue, 20 Jul 2010 14:46:48 -0500 Dave Goodell <[email protected]> wrote:
> On Jul 20, 2010, at 2:07 PM CDT, Török Edwin wrote: > > > On Tue, 20 Jul 2010 13:34:12 -0500 > > Dave Goodell <[email protected]> wrote: > > > >> On Jul 20, 2010, at 10:32 AM CDT, Török Edwin wrote: > >> > >>> While it is possible to teach valgrind about ClamAV's pools (see > >>> attached mpool.patch for example), I think valgrind will be more > >>> effective if I just disable ClamAV's pools (and let it use > >>> malloc). > >> > >> This sounds like a practical solution to me. There is a > >> RUNNING_ON_VALGRIND client request so that you can do this at > >> runtime instead of having to reconfigure/recompile. > > > > I could use that and some env var maybe to switch allocators when > > debugging. It is easier than rebuilding/switching build tree. > > I don't want to do that always because: > > - valgrind would not be testing the same code that is used in > > releases > > - sometimes bugs depend on memory layout/alignment, and they only > > occur with mempool, not with libc allocator. It might be a bug in > > the pool allocator itself. > > It's hard to use valgrind to debug your memory allocator if you are > doing anything nontrivial, mainly because you have to tell Valgrind > what's going on and it will blindly trust you. Presumably you added > this allocator because you really know what you are doing and you > really need the custom allocator for performance. Yeah, lower memory fragmentation, and reduced memory usage. The good part is that I can free the entire pool, and start again with a new pool to avoid some of the fragmentation. > The tradeoff here > is that you have to be very careful in the allocator code itself. > The best you can hope to get from Valgrind in that case is that in > order to use the client requests you have to think carefully about > the code and might catch bugs that way... > > [...] > > My problem with NOACCESS is that I don't know if it is NOACCESS > > because that memory zone wasn't mapped/allocated, or because it is > > the bookkeeping info. With readonly at least I'd know the address > > was valid at some point, for some purpose. But you are right, not > > ideal either, it might be some totally unrelated memory region that > > happens to be readonly. > [...] > >>> 3. If I get passed an invalid pointer (one that the pool allocator > >>> didn't allocate for example), then if I mark my allocation > >>> information block as UNDEFINED I actually mark random memory and > >>> valgrind can't help me > >> > >> Can't you tell if the given pointer is valid based on the address > >> value? > >> Don't you track the valid address ranges anyway in order to > >> do cleanup at the end? > > > > I could check that it is within the address ranges allocated by the > > pool. I can't tell if it is the start of an allocation, or some > > value in the middle, or before an allocation. > > > > I might be able to detect that the bookkeeping information is > > corrupt in some cases, but not always. > > > > Hmm, if I set the bookeeping info (and padding) as NOACCESS, then I > > should be able to check that the memory range where I think the > > bookkeeping information is, is marked NOACCESS, and that the pointer > > itself is DEF/UNDEF. > > Is there a way to do that quitely (without spamming valgrind's log)? > > VALGRIND_CHECK_MEM_IS_ADDRESSABLE docs say it spams the logs, > > maybe I can use VALGRIND_GET_VBITS? Will have to do some > > experiments. > > I don't think you should try to overload the meaning of valgrind's > Valid and Addressable bits in order to keep track of other > information about memory in your program. That approach will likely > just lead to false positives and won't catch many likely errors in > client code. Instead just use the RUNNING_ON_VALGRIND test to check > whether you should do some extra allocation tracking of your own. It > will probably be much simpler in the end. Just stuff all the valid > pointers into a hash table or other map data structure on allocation, > check for them when resizing or deallocating, and delete them on > deallocation. > > > Not strictly related to this discussion, but could I mark all > > anonymous mmaps (not used by the pool related) with > > MALLOCLIKE_BLOCK, and munmaps with FREELIKE_BLOCK to detect > > anon-map leaks? I think valgrind doesn't track mmap leaks on its > > own. > > Yes, but I think you'll have to do one or the other. IIRC valgrind > doesn't like it when you try to associate a piece of memory with two > different memory pools. > > >> Otherwise, how about "magic number" in the allocation header that > >> you can assert() is correct after you mark it DEFINED? Do you > >> actually need the code to keep operating after the client code > >> passes the allocator an invalid pointer? > > > > I have a #define for that already :) > > But there were bugs that didn't show up when I define it (because it > > increases size of bookkeeping info, which changes all the > > paddings/alignments). > > I actually hide the bookkeeping info in the padding sometimes, in > > best case scenario the overhead is only 2 bytes. > > Bugs in the allocator or bugs in the client code? > > The separate tracking approach mentioned above might make more sense > instead if your application is really that sensitive to alignment > issues. I think they were bugs while adding new features to the allocator. I sure missed valgrind there. But there was an application bug due to alignment too (fixed now), the allocator wasn't returning properly aligned memory. Now of course valgrind wouldn't help me debug alignment issues, I've just thought of this as an example of bug that might depend on memory layout. Best regards, --Edwin ------------------------------------------------------------------------------ This SF.net email is sponsored by Sprint What will you do first with EVO, the first 4G phone? Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first _______________________________________________ Valgrind-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/valgrind-users
