Thanks Tom, that helps a ton. Please copy-paste that into the docs. :)
-jonathan On Jun 15, 2012, at 10:43 AM, Tom Ward <[email protected]> wrote: > Hi guys, > > Some good questions there, hopefully the responses below will help understand > what goes on: > > What happens to an STLAllocator vector<> when you've resized it past Nuke's > max usage amount…? Does it simply allocate more memory outside the max_usage > range, start to use virtual memory if that exceed the available RAM, crash, > or what? > > When you call any of the allocate() functions (either directly or indirectly > through the STL Allocator), the memory system tracks how much memory you've > asked for and, if it reaches the user defined max usage, will attempt to free > memory by calling any registered free callbacks. Once it either gets below > the max usage, or runs out of free callbacks to call, it then allocates the > requested memory regardless and returns the pointer to the user. > > It will always try and allocate the memory, and only return a bad_alloc > exception if the system fails to allocate any memory, the same as if you > called new explicitly. > > To add to those questions, if I'm allocating something large within my node > what are the chances nuke will remove it from memory to avoid overrunning > the max? If it's removed from memory how do I know it's been removed and > re-populate it? > > The memory system will never delete any memory explicitly, and only requests > for memory to be freed through user-registered free callbacks. Obviously if > you're allocating large buffers (for instance if you need to store an entire > image buffer across engine calls) then you should register a free callback so > that Nuke can attempt to adhere to the max usage set in preferences. > > 2) Instead of simply using STLAllocator for these large vectors should I be > wrapping a cache class around these vectors that uses the Memory manager more > specifically? Is the SLAllocator intended for relatively small allocations? > > In the current version of Nuke the Memory system effectively calls malloc > under the hood, guaranteeing 16-byte alignment. For the next version of Nuke, > we've added custom allocators that are better suited for things like lots of > small allocations (e.g. in the 3D system) which can be used by plug-in > developers. This not only saves memory and limits the number of system > allocations, but also helps prevent fragmentation. > > For the next major release of Nuke there's a some more documentation on how > all these things work, as well some minor improvements to the memory > information callbacks. > > Hope that all helps > > Tom > > On Fri, Jun 15, 2012 at 10:16 AM, Colin Doncaster <[email protected]> > wrote: > To add to those questions, if I'm allocating something large within my node > what are the chances nuke will remove it from memory to avoid overrunning the > max? If it's removed from memory how do I know it's been removed and > re-populate it? > > There doesn't seem to be a clear way of locking the allocated memory during > the execution of my node. > > Thanks > > On 2012-06-15, at 1:00 PM, Jonathan Egstad wrote: > >> Hi Tom, >> >> Unfortunately it kinda doesn't for me - I started to use the STLAllocator >> class for some large vectors but experienced crashes in some of my code and >> backed out. It was unclear to me at the time what was at fault and could >> easily have been my mistake, but I realized I really didn't understand what >> the STLAllocator was going to do when it ran past Nuke's max limit, so I >> went back to the default allocator. >> >> I didn't find any explanation of the behavior of the STLAllocator itself - >> there are no comments on the class at all. And although there's some >> explanation to be inferred from the Memory class docs it isn't clear what >> happens when memory limits are exceeded - only a reference to >> 'reduce_current_usage()' which doesn't appear to relate to the STLAllocator >> class much. >> >> So two questions: >> 1) What happens to an STLAllocator vector<> when you've resized it past >> Nuke's max usage amount…? Does it simply allocate more memory outside the >> max_usage range, start to use virtual memory if that exceed the available >> RAM, crash, or what? >> >> 2) Instead of simply using STLAllocator for these large vectors should I be >> wrapping a cache class around these vectors that uses the Memory manager >> more specifically? Is the SLAllocator intended for relatively small >> allocations? >> >> Thanks, >> >> -jonathan >> >> On Jun 14, 2012, at 9:15 PM, Tom Ward wrote: >> >>> Hi Colin, >>> >>> no, Nuke doesn't overload new/delete so any STL containers you use won't >>> have their memory tracked by Nuke. There is a header in DDImage called >>> STLAllocator.h that has an STL allocator that calls into DD::Image::Memory, >>> as well as a couple of typedef's for vector and deque. >>> >>> Hope that helps... >>> >>> On Thu, Jun 14, 2012 at 6:53 AM, Colin Doncaster >>> <[email protected]> wrote: >>> Hi there - >>> >>> I'm looking at DD::Image::Memory and was wondering how standard template >>> containers ( vector, map, etc. ) fit within this context? I assume >>> new/delete is overloaded and the templates still happily report their usage >>> or do we need to explicitly call allocate? >>> >>> This is what we're assuming, but is that correct? >>> >>> Thanks_______________________________________________ >>> Nuke-dev mailing list >>> [email protected], http://forums.thefoundry.co.uk/ >>> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-dev >>> >>> >>> >>> -- >>> Tom Ward, Software Engineer >>> The Foundry, 6th Floor, The Communications Building, >>> 48 Leicester Square, London, UK, WC2H 7LT >>> Tel: +44 (0)20 7434 0449 Web: www.thefoundry.co.uk >>> >>> The Foundry Visionmongers Ltd. >>> Registered in England and Wales No: 4642027 >>> >>> _______________________________________________ >>> Nuke-dev mailing list >>> [email protected], http://forums.thefoundry.co.uk/ >>> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-dev >> >> _______________________________________________ >> Nuke-dev mailing list >> [email protected], http://forums.thefoundry.co.uk/ >> http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-dev > > > _______________________________________________ > Nuke-dev mailing list > [email protected], http://forums.thefoundry.co.uk/ > http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-dev > > > > > -- > Tom Ward, Software Engineer > The Foundry, 6th Floor, The Communications Building, > 48 Leicester Square, London, UK, WC2H 7LT > Tel: +44 (0)20 7434 0449 Web: www.thefoundry.co.uk > > The Foundry Visionmongers Ltd. > Registered in England and Wales No: 4642027 > > _______________________________________________ > Nuke-dev mailing list > [email protected], http://forums.thefoundry.co.uk/ > http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-dev
_______________________________________________ Nuke-dev mailing list [email protected], http://forums.thefoundry.co.uk/ http://support.thefoundry.co.uk/cgi-bin/mailman/listinfo/nuke-dev
