Re: How do you deal with scoped allocations?
On 09.12.2013 10:13, John Colvin wrote: On Sunday, 8 December 2013 at 19:00:29 UTC, Namespace wrote: Just using new and GC.free would be fine, but there is really no need to burden the GC with this at all; it's a textbook case for C's malloc/free. The GC relies also on malloc / free. Yes, it does. However, obviously it does a lot more than just call malloc/free. If you statically know the lifetime of a chunk of memory, then there really is no point invoking all that extra heavyweight code* when you can just DIY at no extra cost.** *and adding to the collection burden (if triggered while the memory is in scope) **However, take care with unique pointers from inside the allocated block to resources outside the block, as the GC might not be able to find them later (see P.S.). I'm not sold on preferring C's malloc/free instead of the GC. - The operations GC.malloc and GC.free are very similar to the operations of the C functions (if no collection is triggered). The current implementation uses OS calls to get more memory from the system, only few extra data is allocated via malloc. GC.malloc/free might be missing a few optimizations (like lock-free allocations for small memory chunks), but they were a lot faster than what dmc's C runtime used to be - before the latter was switched to simply call the OS heap functions very recently. - if the GC.malloced memory does not contain pointers (e.g. allocated with new int[N]), it isn't scanned by the GC. Determining whether a pointer actually refers to GC managed memory or not might be faster for unmanaged memory, but this depends on the memory addresses. - if the GC.malloced memory might contain pointers to GC managed memory (like object references), you need to add and remove the memory range so that the GC scans the memory, but especially removing the range gets pretty expensive if a lot of ranges are added. P.S. does anyone know how the GC interacts with core.stdc.free? I.e. if you free a pointer, but don't null the pointer, presumably the GC will still scan the memory despite it being freed. Isn't this undefined behaviour? Like thedeemon said, a pointer not pointing to GC managed memory is ignored, so this should do no harm.
Re: How do you deal with scoped allocations?
08.12.2013 2:32, Namespace пишет: Since my last thread doesn't get much attention I like to ask here: How did you deal with temporary memory? The algorithm is always this: 1. Use function stack frame for small allocations. 2. Use thread local stack allocator if temporary allocations corresponds LIFO principle and use thread local heap otherwise. As for usability it must be a single function call and D type system have to rest. And what do you use? As I answered in previous thread: http://forum.dlang.org/thread/nxrxojbzbrfwv...@forum.dlang.org#post-l77th1:24icc:241:40digitalmars.com -- Денис В. Шеломовский Denis V. Shelomovskij
Re: How do you deal with scoped allocations?
On Sunday, 8 December 2013 at 19:00:29 UTC, Namespace wrote: Just using new and GC.free would be fine, but there is really no need to burden the GC with this at all; it's a textbook case for C's malloc/free. The GC relies also on malloc / free. Yes, it does. However, obviously it does a lot more than just call malloc/free. If you statically know the lifetime of a chunk of memory, then there really is no point invoking all that extra heavyweight code* when you can just DIY at no extra cost.** *and adding to the collection burden (if triggered while the memory is in scope) **However, take care with unique pointers from inside the allocated block to resources outside the block, as the GC might not be able to find them later (see P.S.). P.S. does anyone know how the GC interacts with core.stdc.free? I.e. if you free a pointer, but don't null the pointer, presumably the GC will still scan the memory despite it being freed. Isn't this undefined behaviour?
Re: How do you deal with scoped allocations?
On Monday, 9 December 2013 at 09:13:20 UTC, John Colvin wrote: P.S. does anyone know how the GC interacts with core.stdc.free? I.e. if you free a pointer, but don't null the pointer, presumably the GC will still scan the memory despite it being freed. Isn't this undefined behaviour? From what I understand, if you malloc a buffer it lies outside GC's managed heap, so a pointer to this place will not be followed during GC scan. Unless you explicitly told it to, by calling GC.addRange().
Re: How do you deal with scoped allocations?
On Sunday, 8 December 2013 at 18:55:26 UTC, Namespace wrote: On Sunday, 8 December 2013 at 18:33:39 UTC, Joseph Rushton Wakeling wrote: On 08/12/13 19:18, thedeemon wrote: I just use scope(exit) delete buf; Deprecated or at least disapproved of, no? Sadly yes. Unless I'm mistaken, the only thing that was deprecated was scoped as an attribute to variables. The scope *statement* is till valid, and one of D's more powerful arguments in simple and exception safe programming.
Re: How do you deal with scoped allocations?
On Monday, 9 December 2013 at 12:29:44 UTC, monarch_dodra wrote: On Sunday, 8 December 2013 at 18:55:26 UTC, Namespace wrote: On Sunday, 8 December 2013 at 18:33:39 UTC, Joseph Rushton Wakeling wrote: On 08/12/13 19:18, thedeemon wrote: I just use scope(exit) delete buf; Deprecated or at least disapproved of, no? Sadly yes. Unless I'm mistaken, the only thing that was deprecated was scoped as an attribute to variables. The scope *statement* is till valid, and one of D's more powerful arguments in simple and exception safe programming. 'delete' was meant. ;)
Re: How do you deal with scoped allocations?
On Monday, 9 December 2013 at 12:32:36 UTC, Namespace wrote: On Monday, 9 December 2013 at 12:29:44 UTC, monarch_dodra wrote: On Sunday, 8 December 2013 at 18:55:26 UTC, Namespace wrote: On Sunday, 8 December 2013 at 18:33:39 UTC, Joseph Rushton Wakeling wrote: On 08/12/13 19:18, thedeemon wrote: I just use scope(exit) delete buf; Deprecated or at least disapproved of, no? Sadly yes. Unless I'm mistaken, the only thing that was deprecated was scoped as an attribute to variables. The scope *statement* is till valid, and one of D's more powerful arguments in simple and exception safe programming. 'delete' was meant. ;) *slaps face*
Re: How do you deal with scoped allocations?
08-Dec-2013 02:32, Namespace пишет:
Since my last thread doesn't get much attention I like to ask here: How
did you deal with temporary memory? Let's assume that the size is only
known at runtime.
I have this situation e.g. in Dgame in the capture method: I get the
pixel data from my Window with glReadPixel but it is reversed. So I have
to reverse it again, but I still need at least temporary memory for one
pixel-line which stores the currently swapped pixels. So how would you
solve such a situation?
Since D doesn't offer VLA's and alloca is broken (besides the ugly syntax),
I use a scoped wrapper (since scope doesn't do the job):
struct scoped(A : T[], T) {
T[] arr;
alias arr this;
this(T[] arr) {
this.arr = arr;
writefln(Get %d %s's (ptr = %x), arr.length, T.stringof,
arr.ptr);
}
~this() {
GC.free(this.arr.ptr);
this.arr = null;
GC.minimize();
This is slow. Just use malloc free, why touch GC at all?
}
}
void main() {
// need temp memory
scoped!(int[]) arr = new int[n];
}
And what do you use?
--
Dmitry Olshansky
Re: How do you deal with scoped allocations?
On Saturday, 7 December 2013 at 22:32:59 UTC, Namespace wrote:
Since my last thread doesn't get much attention I like to ask
here: How did you deal with temporary memory? Let's assume that
the size is only known at runtime.
I have this situation e.g. in Dgame in the capture method: I
get the pixel data from my Window with glReadPixel but it is
reversed. So I have to reverse it again, but I still need at
least temporary memory for one pixel-line which stores the
currently swapped pixels. So how would you solve such a
situation?
Since D doesn't offer VLA's and alloca is broken (besides the
ugly syntax),
I use a scoped wrapper (since scope doesn't do the job):
struct scoped(A : T[], T) {
T[] arr;
alias arr this;
this(T[] arr) {
this.arr = arr;
writefln(Get %d %s's (ptr = %x), arr.length, T.stringof,
arr.ptr);
}
~this() {
GC.free(this.arr.ptr);
this.arr = null;
GC.minimize();
}
}
void main() {
// need temp memory
scoped!(int[]) arr = new int[n];
}
And what do you use?
From my probably somewhat incomplete understanding of such things:
This is not a good use-case for the gc. Use the c heap or just
let the gc do its job normally.
Re: How do you deal with scoped allocations?
On Sunday, 8 December 2013 at 00:17:48 UTC, Adam D. Ruppe wrote:
On Sunday, 8 December 2013 at 00:16:46 UTC, Adam D. Ruppe wrote:
the same general pattern there of static array up to a certain
size.
static array being T[max_size], not static T[max_size]
just so it uses the stack for most things.
So rather something like this:
import std.stdio;
struct Helper(T, uint StackSize = 128) {
T[StackSize] buffer = void;
T[] allocate(size_t n) {
if (n = StackSize)
return buffer[0 .. n];
return new T[n];
}
}
void main() {
auto h = Helper!int();
int[] arr = h.allocate(512);
}
Yes? I don't like it, because it isn't a one liner. Isn't it
possible in one line?
A personal dream of me would be: scope int[] arr = new int[n]; or
the implementation of DIP46. :)
Re: How do you deal with scoped allocations?
On Sunday, 8 December 2013 at 09:14:44 UTC, Dmitry Olshansky
wrote:
08-Dec-2013 02:32, Namespace пишет:
Since my last thread doesn't get much attention I like to ask
here: How
did you deal with temporary memory? Let's assume that the size
is only
known at runtime.
I have this situation e.g. in Dgame in the capture method: I
get the
pixel data from my Window with glReadPixel but it is reversed.
So I have
to reverse it again, but I still need at least temporary
memory for one
pixel-line which stores the currently swapped pixels. So how
would you
solve such a situation?
Since D doesn't offer VLA's and alloca is broken (besides the
ugly syntax),
I use a scoped wrapper (since scope doesn't do the job):
struct scoped(A : T[], T) {
T[] arr;
alias arr this;
this(T[] arr) {
this.arr = arr;
writefln(Get %d %s's (ptr = %x), arr.length,
T.stringof,
arr.ptr);
}
~this() {
GC.free(this.arr.ptr);
this.arr = null;
GC.minimize();
This is slow. Just use malloc free, why touch GC at all?
}
}
void main() {
// need temp memory
scoped!(int[]) arr = new int[n];
}
And what do you use?
Because it's more D'ish. That is what D offers.
Why is it slower than malloc + free?
Re: How do you deal with scoped allocations?
On Sunday, 8 December 2013 at 09:25:41 UTC, Namespace wrote:
On Sunday, 8 December 2013 at 09:14:44 UTC, Dmitry Olshansky
wrote:
08-Dec-2013 02:32, Namespace пишет:
Since my last thread doesn't get much attention I like to ask
here: How
did you deal with temporary memory? Let's assume that the
size is only
known at runtime.
I have this situation e.g. in Dgame in the capture method: I
get the
pixel data from my Window with glReadPixel but it is
reversed. So I have
to reverse it again, but I still need at least temporary
memory for one
pixel-line which stores the currently swapped pixels. So how
would you
solve such a situation?
Since D doesn't offer VLA's and alloca is broken (besides the
ugly syntax),
I use a scoped wrapper (since scope doesn't do the job):
struct scoped(A : T[], T) {
T[] arr;
alias arr this;
this(T[] arr) {
this.arr = arr;
writefln(Get %d %s's (ptr = %x), arr.length,
T.stringof,
arr.ptr);
}
~this() {
GC.free(this.arr.ptr);
this.arr = null;
GC.minimize();
This is slow. Just use malloc free, why touch GC at all?
}
}
void main() {
// need temp memory
scoped!(int[]) arr = new int[n];
}
And what do you use?
Because it's more D'ish. That is what D offers.
Why is it slower than malloc + free?
Well, for a start you're calling GC.minimize every time you leave
the scope, which is an expensive call.
Just using new and GC.free would be fine, but there is really no
need to burden the GC with this at all; it's a textbook case for
C's malloc/free.
Re: How do you deal with scoped allocations?
08-Dec-2013 13:25, Namespace пишет: On Sunday, 8 December 2013 at 09:14:44 UTC, Dmitry Olshansky wrote: 08-Dec-2013 02:32, Namespace пишет: [snip] And what do you use? Because it's more D'ish. There is no such objective quality as being D'ish. GC.malloc/GC.free are no better then malloc/free in code style, readability or observable effect. That is what D offers. D offers you C run-time as well. In fact it builds on top of it. See also (hopefully) soon to be formally reviewed std.allocator. Why is it slower than malloc + free? 1. Because GC.minimize does a hell of a work to optimize the whole GC heap. For starters it does collection so as to get rid of floating garbage (AFAIK). 2. You have a trivial use case of deterministic allocation - you know precisely when to deallocate a block. In this setting putting allocated block into tracing GC heap is a waste of time should a collection happen. -- Dmitry Olshansky
Re: How do you deal with scoped allocations?
On 12/8/13 4:29 AM, Dmitry Olshansky wrote: 08-Dec-2013 13:25, Namespace пишет: On Sunday, 8 December 2013 at 09:14:44 UTC, Dmitry Olshansky wrote: 08-Dec-2013 02:32, Namespace пишет: [snip] And what do you use? Because it's more D'ish. There is no such objective quality as being D'ish. GC.malloc/GC.free are no better then malloc/free in code style, readability or observable effect. If you put (arguably by mistake) a scoped member inside a class, the malloc'd memory will leak but the GC-allocated memory will be ultimately collected. Andrei
Re: How do you deal with scoped allocations?
On Saturday, 7 December 2013 at 22:32:59 UTC, Namespace wrote: Since my last thread doesn't get much attention I like to ask here: How did you deal with temporary memory? I just use scope(exit) delete buf; Your solution seems to do a GC every time some buffer goes out of scope, which is slooow.
Re: How do you deal with scoped allocations?
On 08/12/13 19:18, thedeemon wrote: I just use scope(exit) delete buf; Deprecated or at least disapproved of, no?
Re: How do you deal with scoped allocations?
On Sunday, 8 December 2013 at 18:33:39 UTC, Joseph Rushton Wakeling wrote: On 08/12/13 19:18, thedeemon wrote: I just use scope(exit) delete buf; Deprecated or at least disapproved of, no? Sadly yes.
Re: How do you deal with scoped allocations?
I just use scope(exit) delete buf; Your solution seems to do a GC every time some buffer goes out of scope, which is slooow. I don't need scoped allocations that often. Otherwise I wouldn't use GC.minimize ;) or wouldn't use the GC at all. John Colvin: Well, for a start you're calling GC.minimize every time you leave the scope, which is an expensive call. That was my own decision and is of course not required, but I like to clean up the wasted space. [1] And as I said, I don't use scope allocations that often. Just using new and GC.free would be fine, but there is really no need to burden the GC with this at all; it's a textbook case for C's malloc/free. The GC relies also on malloc / free. Dmitry Olshansky: There is no such objective quality as being D'ish. GC.malloc/GC.free are no better then malloc/free in code style, readability or observable effect. But I don't use GC.malloc / GC.free. I use new which is IMO a lot nicer and more clean. Of course I now that new is only D syntax for malloc but it is still more clean. D offers you C run-time as well. In fact it builds on top of it. See also (hopefully) soon to be formally reviewed std.allocator. It is still C even if D offers the access to the C standard lib. 1. Because GC.minimize does a hell of a work to optimize the whole GC heap. For starters it does collection so as to get rid of floating garbage (AFAIK). See [1]
How do you deal with scoped allocations?
Since my last thread doesn't get much attention I like to ask
here: How did you deal with temporary memory? Let's assume that
the size is only known at runtime.
I have this situation e.g. in Dgame in the capture method: I get
the pixel data from my Window with glReadPixel but it is
reversed. So I have to reverse it again, but I still need at
least temporary memory for one pixel-line which stores the
currently swapped pixels. So how would you solve such a situation?
Since D doesn't offer VLA's and alloca is broken (besides the
ugly syntax),
I use a scoped wrapper (since scope doesn't do the job):
struct scoped(A : T[], T) {
T[] arr;
alias arr this;
this(T[] arr) {
this.arr = arr;
writefln(Get %d %s's (ptr = %x), arr.length, T.stringof,
arr.ptr);
}
~this() {
GC.free(this.arr.ptr);
this.arr = null;
GC.minimize();
}
}
void main() {
// need temp memory
scoped!(int[]) arr = new int[n];
}
And what do you use?
Re: How do you deal with scoped allocations?
On Saturday, 7 December 2013 at 22:32:59 UTC, Namespace wrote: Since D doesn't offer VLA's and alloca is broken (besides the ugly syntax), I use a scoped wrapper (since scope doesn't do the job): I like to use a helper template that defines a static array with a typical size. Then, if the runtime requirement is less than that, it can simply slice into the static array, and if not, then do the alloc/free pair.
Re: How do you deal with scoped allocations?
On Saturday, 7 December 2013 at 23:21:05 UTC, Adam D. Ruppe wrote:
On Saturday, 7 December 2013 at 22:32:59 UTC, Namespace wrote:
Since D doesn't offer VLA's and alloca is broken (besides the
ugly syntax),
I use a scoped wrapper (since scope doesn't do the job):
I like to use a helper template that defines a static array
with a typical size. Then, if the runtime requirement is less
than that, it can simply slice into the static array, and if
not, then do the alloc/free pair.
You mean something like that?
struct Helper(T, uint StackSize = 128) {
static T[StackSize] buffer = void;
static T[] opCall(size_t n) {
if (n = StackSize)
return buffer[0 .. n];
return new T[n];
}
}
void main() {
int[] arr = Helper!int(512);
}
Re: How do you deal with scoped allocations?
On Saturday, 7 December 2013 at 23:26:05 UTC, Namespace wrote: You mean something like that? Yes.
Re: How do you deal with scoped allocations?
On Saturday, 7 December 2013 at 23:26:05 UTC, Namespace wrote:
struct Helper(T, uint StackSize = 128) {
static T[StackSize] buffer = void;
Using a static variable here means that there is only one such
buffer per thread (and e.g. it wouldn't be possible to use the
helper twice in the same call stack) - is this really what you
want?
David
Re: How do you deal with scoped allocations?
On Sunday, 8 December 2013 at 00:05:42 UTC, David Nadlinger wrote:
On Saturday, 7 December 2013 at 23:26:05 UTC, Namespace wrote:
struct Helper(T, uint StackSize = 128) {
static T[StackSize] buffer = void;
Using a static variable here means that there is only one such
buffer per thread (and e.g. it wouldn't be possible to use the
helper twice in the same call stack) - is this really what you
want?
David
That is not what I want or what I use. But it seems that Adam D.
Ruppe use such a thing. What would you use?
Re: How do you deal with scoped allocations?
On Sunday, 8 December 2013 at 00:11:18 UTC, Namespace wrote: But it seems that Adam D. Ruppe use such a thing. What would you use? I didn't pay close enough attention; I don't use static, just the same general pattern there of static array up to a certain size.
Re: How do you deal with scoped allocations?
On Sunday, 8 December 2013 at 00:16:46 UTC, Adam D. Ruppe wrote: the same general pattern there of static array up to a certain size. static array being T[max_size], not static T[max_size] just so it uses the stack for most things.
