Re: [boost] Re: Re: Re: what happened to allocators in boost?
--- Peter Dimov [EMAIL PROTECTED] wrote: E. Gladyshev wrote: --- Peter Dimov [EMAIL PROTECTED] wrote: unless there are very solid reasons for the allocator parameter. ;-) I agree, but the problme is that I don't know whether I have a solid reason or not. I have to ship my class to someone else. She might have a solid reason, I don't know. If I supply an allocator parameter, then I am covered and I can sleep well. :) Well... if you ask me, it is better to not offer the functionality until someone explicitly asks for it and gives a reasonable scenario as an example. Features are easy to add, hard to remove, and there is always the possibility that you pre-included the wrong feature. Peter, using your example I put together an allocator class that can be used in shared_ptr and STL at the same time. It is not too pretty but it works. If we could only customize the counter allocations, that would be it. namespace boostx { template typename A class allocator : public A { public: typedef A::value_type type; allocator() {} type* create() { type* p = allocate(1, 0); try { p = new(p) type; } catch(...) { deallocate( p, 1 ); throw; } return p; } void operator()( type* p ) { try { p-~T(); } catch(...) { deallocate(p, 1); throw; } deallocate(p, 1); } }; }; template class T, class A = boostx::allocator std::allocatorT struct X { boost::shared_ptrT _data; std::vectorT,A _list; X( const A al = A() ) : _list(al) { A tmp(al); _data = boost::shared_ptrT( tmp.create(), tmp ); } }; OR if you don't mind dropping the 'const' from the constructor template typename T, class A = boostx::allocator std::allocatorT struct X { boost::shared_ptrT _data; std::vectorT,A _list; X( A al = A() ) : _list(al), _data( al.create(), al ) { } }; Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
--- E. Gladyshev [EMAIL PROTECTED] wrote: --- Peter Dimov [EMAIL PROTECTED] wrote: E. Gladyshev wrote: --- Peter Dimov [EMAIL PROTECTED] wrote: unless there are very solid reasons for the allocator parameter. ;-) I agree, but the problme is that I don't know whether I have a solid reason or not. I have to ship my class to someone else. She might have a solid reason, I don't know. If I supply an allocator parameter, then I am covered and I can sleep well. :) Well... if you ask me, it is better to not offer the functionality until someone explicitly asks for it and gives a reasonable scenario as an example. Features are easy to add, hard to remove, and there is always the possibility that you pre-included the wrong feature. Peter, using your example I put together an allocator class that can be used in shared_ptr and STL at the same time. It is not too pretty but it works. If we could only customize the counter allocations, that would be it. namespace boostx { template typename A class allocator : public A { public: typedef A::value_type type; allocator() {} type* create() { type* p = allocate(1, 0); try { p = new(p) type; } catch(...) { deallocate( p, 1 ); throw; } return p; } void operator()( type* p ) { try { p-~T(); } catch(...) { deallocate(p, 1); throw; } deallocate(p, 1); } }; }; template class T, class A = boostx::allocator std::allocatorT struct X { boost::shared_ptrT _data; std::vectorT,A _list; X( const A al = A() ) : _list(al) { A tmp(al); _data = boost::shared_ptrT( tmp.create(), tmp ); } }; OR if you don't mind dropping the 'const' from the constructor template typename T, class A = boostx::allocator std::allocatorT struct X { boost::shared_ptrT _data; std::vectorT,A _list; X( A al = A() ) : _list(al), _data( al.create(), al ) { } }; Eugene I forgot to mention, if the 'const' is dropped from the constructor, 'A' must be stateless. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
--- E. Gladyshev [EMAIL PROTECTED] wrote: --- Peter Dimov [EMAIL PROTECTED] wrote: E. Gladyshev wrote: --- Peter Dimov [EMAIL PROTECTED] wrote: unless there are very solid reasons for the allocator parameter. ;-) I agree, but the problme is that I don't know whether I have a solid reason or not. I have to ship my class to someone else. She might have a solid reason, I don't know. If I supply an allocator parameter, then I am covered and I can sleep well. :) Well... if you ask me, it is better to not offer the functionality until someone explicitly asks for it and gives a reasonable scenario as an example. Features are easy to add, hard to remove, and there is always the possibility that you pre-included the wrong feature. Peter, using your example I put together an allocator class that can be used in shared_ptr and STL at the same time. It is not too pretty but it works. If we could only customize the counter allocations, that would be it. namespace boostx { template typename A class allocator : public A { public: typedef A::value_type type; allocator() {} type* create() { type* p = allocate(1, 0); try { p = new(p) type; } catch(...) { deallocate( p, 1 ); throw; } return p; } void operator()( type* p ) { try { p-~T(); } catch(...) { deallocate(p, 1); throw; } deallocate(p, 1); } }; }; template class T, class A = boostx::allocator std::allocatorT struct X { boost::shared_ptrT _data; std::vectorT,A _list; X( const A al = A() ) : _list(al) { A tmp(al); _data = boost::shared_ptrT( tmp.create(), tmp ); } }; OR if you don't mind dropping the 'const' from the constructor template typename T, class A = boostx::allocator std::allocatorT struct X { boost::shared_ptrT _data; std::vectorT,A _list; X( A al = A() ) : _list(al), _data( al.create(), al ) { } }; Sorry, disregard my prev e-mail. I should have said that 'A' must always be stateless. But then, what is the point passing the A thing around... I guess my solution is a limited one. It is hard to get around it w/o a built-in allocator support. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
E. Gladyshev wrote: I guess my question is that, is boost redefining the memory management concepts that have been established by STL? Yes and no. The STL uses allocators for containers. Most non-containers do not have an allocator parameter, although many standard classes do allocate memory. ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
Andreas Huber wrote: I know, I just mentioned this because Peter seems to suggest that the need for memory management customization is rare if the compiler folks did their job right. But it is. Compiler folks not doing their job is the primary motivation for custom memory management. The C++ community has been trained to take for granted that it's perfectly acceptable for the system allocator to not work well. It does not _have_ to be this way. If you can write a better allocator, so can they. Of course things aren't that simple since the general allocator can not afford the luxury to only handle one particular application's memory use pattern. On the other hand, per-project allocators are usually written under pressure, not tested well, have bugs, so the playing field is fairly even. :-) Many project specific allocators I have seen can't even match dlmalloc, let alone a native commercial replacement. However, for a certain class of systems one almost never can avoid customization, no matter how ingenious the platform is. This may well be true, but I'm not convinced that platform-specific customizations (an implementation detail) absolutely need to be supported by a portable mechanism affecting the interface. ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
--- Peter Dimov [EMAIL PROTECTED] wrote: E. Gladyshev wrote: I guess my question is that, is boost redefining the memory management concepts that have been established by STL? Yes and no. The STL uses allocators for containers. Most non-containers do not have an allocator parameter, although many standard classes do allocate memory. I agree. It means the STL is not consistent either. So what are the boost recomendations for using STL containers and boost in the same class? I am using STL and trying to use boost in my daily development. What can I do to implement consistent classes in terms of memory management. For example, if I need to implement a class A that has a container and pointer to a data type. I have two options: template typename T struct A { shared_ptrT _data; std::listT _list; }; template typename T, template A = std::allocatorT struct A { shared_ptrT _data; std::listT, A _list; A : _data( new T() ) {...} }; The problem with the first defenition is that I am limiting the built-in STL functionality in terms of memory management. The problem with the second definition is that if I expose the allocator template parameter, the user of my class will expect that all memory allocations of type T are going to be using her allocator but boost::shared_ptr doesn't support it. Could you please advise me? Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
--- Gregory Colvin [EMAIL PROTECTED] wrote: For shared_ptr the count is allocated by the following line in the shared_count constructor: new sp_counted_base_implP, D(p, d); So it might be possible to make the allocation customizable by specializing sp_counted_base_impl. I think it would be great. However there is another problem. You have to new your object. shared_ptrMyClass s( new MyClass ); You cannot use allocator like you would expect: shared_ptr MyClass, std::allocatorMyClass s; Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
E. Gladyshev wrote: --- Gregory Colvin [EMAIL PROTECTED] wrote: For shared_ptr the count is allocated by the following line in the shared_count constructor: new sp_counted_base_implP, D(p, d); So it might be possible to make the allocation customizable by specializing sp_counted_base_impl. I think it would be great. However there is another problem. You have to new your object. shared_ptrMyClass s( new MyClass ); You can allocate your object however you want: shared_ptrMyClass s(MyClass::allocate(), MyClass::deallocate); and of course there is also the textbook way of defining a suitable MyClass::operator new. You cannot use allocator like you would expect: shared_ptr MyClass, std::allocatorMyClass s; Leaving aside the issue of whether specializing std::allocatorMyClass is a good idea: shared_ptr MyClass, MyAllocator s; note that you now wouldn't be able to pass s to a function that expects shared_ptrMyClass since the type is not the same. ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
On Tuesday 26 August 2003 01:23 pm, Peter Dimov wrote: Leaving aside the issue of whether specializing std::allocatorMyClass is a good idea: shared_ptr MyClass, MyAllocator s; note that you now wouldn't be able to pass s to a function that expects shared_ptrMyClass since the type is not the same. There's no reason shared_ptr couldn't support a constructor that takes an allocator as an argument. Then sp_counted_base_impl would also take an Allocator parameter (heh, the allocator it was in fact allocated with) and would know how to deallocate itself via a (copy of) this allocator. We'd get allocator support without putting the allocator into the type of shared_ptr (just like we have deleter support). Doug ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
On Tuesday, Aug 26, 2003, at 10:41 America/Denver, E. Gladyshev wrote: --- Gregory Colvin [EMAIL PROTECTED] wrote: For shared_ptr the count is allocated by the following line in the shared_count constructor: new sp_counted_base_implP, D(p, d); So it might be possible to make the allocation customizable by specializing sp_counted_base_impl. I think it would be great. Then I suggest giving it a try. However there is another problem. You have to new your object. shared_ptrMyClass s( new MyClass ); You cannot use allocator like you would expect: shared_ptr MyClass, std::allocatorMyClass s; Right. The shared_ptr design carefully avoids any extra class template parameters, in favor of parameterizing the constructor. My idea, if it works, would be to specialize sp_counted_base_impl on the pointer type P, e.g. MyClass*, so you just write shared_ptrMyClass s( new MyClass ); or on the deleter type D, so you would write, e.g. shared_ptrMyClass s( new MyClass, MyDeleterMyAllocatorMyClass ); The idea is that your sp_counted_base_impl specialization provides a custom operator new and operator delete that uses your allocator. Alternatively, we could consider adding shared_ptr constructor that takes an allocator argument. ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
E. Gladyshev wrote: I am using STL and trying to use boost in my daily development. What can I do to implement consistent classes in terms of memory management. For example, if I need to implement a class A that has a container and pointer to a data type. I have two options: template typename T struct A { shared_ptrT _data; std::listT _list; }; template typename T, template A = std::allocatorT struct A { shared_ptrT _data; std::listT, A _list; A : _data( new T() ) {...} }; The problem with the first defenition is that I am limiting the built-in STL functionality in terms of memory management. The problem with the second definition is that if I expose the allocator template parameter, the user of my class will expect that all memory allocations of type T are going to be using her allocator but boost::shared_ptr doesn't support it. It does... to an extent. You can't customize the count allocations, but they aren't T allocations. For the T allocation you'll need something along the lines of: templateclass A struct X { typedef typename A::value_type T; static T * create(A a) { T * p = a.allocate(1); try { new(p) T; } catch(...) { a.deallocate(p, 1); throw; } return p; } static void destroy(A a, T * p) { p-~T(); a.deallocate(p, 1); } }; The above doesn't respect A::pointer etc. Now you can _almost_ do the straightforward thing: template class T, class A = std::allocatorT struct Y { shared_ptrT _data; std::listT, A _list; Y(): _data( XA::create(), XA::destroy ) {...} }; except that XA::create and ::destroy take a reference to an allocator. This should remind you that you forgot the allocator argument in the constructor: Y(A const a = A()): ... Now _list will make itself a copy of that 'a'. This immediately hints at a potential problem: template class T, class A = std::allocatorT struct Z { std::listT, A _list; std::listT, A _list2; }; where _list and _list2 will make their own copies and can't be made to share a single allocator instance. But let's ignore that (presumably the author of a stateful A will use a drumroll shared_ptr underneath so that copies use the same heap) and get back to our list+shared_ptr. Now who should own the allocator that is used to construct or later destroy *_data? Should we use _list.get_allocator() for that? We can't since a copy of Y will copy _list but share _data, and the original may get destroyed along with the allocator. So we'll need to do something along the lines of: Y(A const a = A()): _list(a), _data() { A a2(a); // original is const T * p = XA::create(a2); // evaluation order _data.reset( p, bind(XA::destroy, a2, _1) ); } Not very pretty, but that's what you need to pay for being std::allocator-based; its interface is tailored for containers. For such a case I'd definitely consider going back to template typename T struct A { shared_ptrT _data; std::listT _list; }; unless there are very solid reasons for the allocator parameter. ;-) Or you can write your own alloc_ptrT, A, of course, if you want to go the allocator route; it is quite a different beast compared to shared_ptr as it doesn't support polymorphism, for example, so it deserves its own name. But your code still looks somewhat artificial to me; why would you want to deep copy _list but shallow copy _data? It doesn't make much sense. ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
Andreas Huber wrote: Peter Dimov wrote: The usual approach is to borrow an entry from the 2*N heap and split it in two when the N heap is empty. [snip] Which brings back non-determinism, as the 2*N heap could be full also and thus be borrowing from 4*N already. You can of course guarantee an upper limit for an allocation, as some heap must have a slot available. Actually it's possible that all free memory already went to the N/2 heap but this case is (even more) unsolvable with predetermined heap sizes, too. :-) However, for some systems this upper limit is already too slow, so someone inevitably has to configure heap sizes *before* startup. I agree, but I still don't see why a quality system allocator on such a system should not give you the ability to do so. ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
Douglas Gregor wrote: There's no reason shared_ptr couldn't support a constructor that takes an allocator as an argument. Minimal but complete. You can already achieve this functionality with the current interface. ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
--- Peter Dimov [EMAIL PROTECTED] wrote: [...] templateclass A struct X { typedef typename A::value_type T; static T * create(A a) { T * p = a.allocate(1); try { new(p) T; } catch(...) { a.deallocate(p, 1); throw; } return p; } static void destroy(A a, T * p) { p-~T(); a.deallocate(p, 1); } }; The above doesn't respect A::pointer etc. Now you can _almost_ do the straightforward thing: template class T, class A = std::allocatorT struct Y { shared_ptrT _data; std::listT, A _list; Y(): _data( XA::create(), XA::destroy ) {...} }; I would change the definition of create/destroy in X: static T * create(A a = A() ); static void destroy(T * p, A a= A()); Anyway it seems like a good solution, but it won't work pretty for the reasons that you described later in your post. shared_ptr should keep its own copy of A just like std::list does. Then it'll be consistent with what people are used to in STL. except that XA::create and ::destroy take a reference to an allocator. This should remind you that you forgot the allocator argument in the constructor: Good point, I just ommitted it for simplicity sake. Y(A const a = A()): ... Now _list will make itself a copy of that 'a'. This immediately hints at a potential problem: template class T, class A = std::allocatorT struct Z { std::listT, A _list; std::listT, A _list2; }; where _list and _list2 will make their own copies and can't be made to share a single allocator instance. But let's ignore that (presumably the author of a stateful A will use a drumroll shared_ptr underneath so that copies use the same heap) and get back to our list+shared_ptr. Yes, I am aware of this problem, shared_ptr or a static heap member will work. From my experience, it is exactly the case. Typically you don't want to keep per-instance data in A. Now who should own the allocator that is used to construct or later destroy *_data? Should we use _list.get_allocator() for that? We can't since a copy of Y will copy _list but share _data, and the original may get destroyed along with the allocator. So we'll need to do something along the lines of: Y(A const a = A()): _list(a), _data() { A a2(a); // original is const T * p = XA::create(a2); // evaluation order _data.reset( p, bind(XA::destroy, a2, _1) ); } Not very pretty, but that's what you need to pay for being std::allocator-based; its interface is tailored for containers. For such a Exactly, like I said before, shared_prt should keep its own copy of A. case I'd definitely consider going back to template typename T struct A { shared_ptrT _data; std::listT _list; }; unless there are very solid reasons for the allocator parameter. ;-) I agree, but the problme is that I don't know whether I have a solid reason or not. I have to ship my class to someone else. She might have a solid reason, I don't know. If I supply an allocator parameter, then I am covered and I can sleep well. :) Or you can write your own alloc_ptrT, A, of course, if you want to go the allocator route; it is quite a different beast compared to shared_ptr as it doesn't support polymorphism, for example, so it deserves its own name. Perhaps, alloc_ptr should be included in boost. It'll give us a standard way to integrate boost and STL. But your code still looks somewhat artificial to me; why would you want to deep copy _list but shallow copy _data? It doesn't make much sense. It is just an example. I agree, it does look artificial, but you can imaging something along the lines: template typename T, template typename A { shared_ptr T, AT _data; shared_ptr std::list T,AT , Astd::listT,AT _list; }; It uses template template parameters and both _data and _list are shared. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
E. Gladyshev wrote: --- Peter Dimov [EMAIL PROTECTED] wrote: unless there are very solid reasons for the allocator parameter. ;-) I agree, but the problme is that I don't know whether I have a solid reason or not. I have to ship my class to someone else. She might have a solid reason, I don't know. If I supply an allocator parameter, then I am covered and I can sleep well. :) Well... if you ask me, it is better to not offer the functionality until someone explicitly asks for it and gives a reasonable scenario as an example. Features are easy to add, hard to remove, and there is always the possibility that you pre-included the wrong feature. ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
--- Peter Dimov [EMAIL PROTECTED] wrote: E. Gladyshev wrote: --- Peter Dimov [EMAIL PROTECTED] wrote: unless there are very solid reasons for the allocator parameter. ;-) I agree, but the problme is that I don't know whether I have a solid reason or not. I have to ship my class to someone else. She might have a solid reason, I don't know. If I supply an allocator parameter, then I am covered and I can sleep well. :) Well... if you ask me, it is better to not offer the functionality until someone explicitly asks for it and gives a reasonable scenario as an example. Features are easy to add, hard to remove, and there is always the possibility that you pre-included the wrong feature. Ok, I'll let my users know who is to blame. :) I agree, if I am to use boost, providing an allocator parameter doesn't make much sense. shared_ptr news the ref counter and in general, boost doesn't allow a full memory management customization anyway. I agree that the STL allocators may not be the answer but then, what is. Counting on the standard allocator is certainly not an answer either. I am afraid that some of the boost libraries (as they are now) won't work for a big group of real time and embedded developers, where customization is the key. Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
Re: [boost] Re: Re: Re: what happened to allocators in boost?
--- Andreas Huber [EMAIL PROTECTED] wrote: [...] BTW: Having separate heaps is one of the reasons why I could not use boost::shared_ptr. I ended up writing my own. :( Yeah, I ended up using intrusive_ptr. Not always an option either ... In fact it is not clear how to marry STL and boost::shared_ptr. I love boost::shared_ptr but because of the lack of the allocator concept, STL and boost just don't live together (including other boost libraries). I guess my question is that, is boost redefining the memory management concepts that have been established by STL? I am a bit suprised by small number of comments from boost developers on this issue. IMO, if boost is to become a C++ standard, we'd have to resolve this conflict. In the mean-time, what are the boost recomendations on how to use STL and boost together? I am really confused, am I missing something here? Eugene __ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com ___ Unsubscribe other changes: http://lists.boost.org/mailman/listinfo.cgi/boost