Re: static void arrays under garbage control?
On 2015-02-26 10:07, Steven Schveighoffer wrote:
Static data I believe is always scanned conservatively because no
type information is stored for it ever, even on allocation (i.e.
program startup).
ouh, the confusion goes on... are you saying that
{
// will be all scanned by GC for
// potential pointers into GC managed memory:
void[16] buffer0 = void;
ubyte[16] buffer1;
uint[4] buffer3;
// will not be scanned by GC for pointers:
void[] buffer4 = cast(void[])(new ubyte[16]);
uint[] buffer5 = cast(uint[])(new ubyte[16]);
}
Re: static void arrays under garbage control?
On 2/25/15 8:15 PM, captaindet wrote: if i understand correctly, static arrays are exempt from GC scanning for memory pointers http://dlang.org/garbage.html : Pointers in D can be broadly divided into two categories: Those that point to garbage collected memory, and those that do not. Examples of the latter are pointers created by calls to C's malloc(), pointers received from C library routines, pointers to static data. but there is also a warning for void arrays http://dlang.org/arrays.html : The garbage collector will scan void[] arrays for pointers, since such an array may have been implicitly converted from an array of pointers or an array of elements that contain pointers. does this warning only apply to dynamic void[] arrays but not to static void[CTconstant] arrays? (because sometimes the docs also mean static arrays even if just type[] is written.) thanks! ps: this is for 32bit apps Somewhat missing in this disscusion is: the GC does not know what an array's type currently is, it only knows what it was when it was allocated. So for instance: ubyte[] arr = cast(ubyte[])(new void[100]); // scanned for pointers void[] arr = new ubyte[100]; // not scanned for pointers. In fact, the GC has no idea of type at all. It just knows about memory blocks, and whether those blocks are flagged as having pointers or not having pointers. The call to new is what tells the GC hm.. this is a type that may contain pointers, set that flag! Static data I believe is always scanned conservatively because no type information is stored for it ever, even on allocation (i.e. program startup). -Steve
Re: static void arrays under garbage control?
On 2/26/15 11:57 AM, captaindet wrote:
On 2015-02-26 10:07, Steven Schveighoffer wrote:
Static data I believe is always scanned conservatively because no
type information is stored for it ever, even on allocation (i.e.
program startup).
ouh, the confusion goes on... are you saying that
{
// will be all scanned by GC for
// potential pointers into GC managed memory:
void[16] buffer0 = void;
ubyte[16] buffer1;
uint[4] buffer3;
Yes, all 3 are stack based. This is not static data, which would be like
uint[4] at module level. Those are also treated as scannable, as the
entire stack of every thread is scanned.
// will not be scanned by GC for pointers:
void[] buffer4 = cast(void[])(new ubyte[16]);
uint[] buffer5 = cast(uint[])(new ubyte[16]);
Correct, since they are allocated as ubyte[]. uint[] also would not be
scanned.
-Steve
Re: static void arrays under garbage control?
On 02/26/2015 10:07 AM, Steven Schveighoffer wrote: // will not be scanned by GC for pointers: void[] buffer4 = cast(void[])(new ubyte[16]); uint[] buffer5 = cast(uint[])(new ubyte[16]); Correct, since they are allocated as ubyte[]. uint[] also would not be scanned. I've been under the impression that current GC being imprecise, even an unfortunate bit-pattern in an int would keep otherwise dead objects alive. Is that still true for a single int? And is 'new int[N]' better in this regard because it never keeps objects alive? Ali
Re: static void arrays under garbage control?
On 2015-02-26 12:07, Steven Schveighoffer wrote:
On 2/26/15 11:57 AM, captaindet wrote:
On 2015-02-26 10:07, Steven Schveighoffer wrote:
Static data I believe is always scanned conservatively because no
type information is stored for it ever, even on allocation (i.e.
program startup).
ouh, the confusion goes on... are you saying that
{
// will be all scanned by GC for
// potential pointers into GC managed memory:
void[16] buffer0 = void;
ubyte[16] buffer1;
uint[4] buffer3;
Yes, all 3 are stack based. This is not static data, which would be like
uint[4] at module level. Those are also treated as scannable, as the entire
stack of every thread is scanned.
// will not be scanned by GC for pointers:
void[] buffer4 = cast(void[])(new ubyte[16]);
uint[] buffer5 = cast(uint[])(new ubyte[16]);
Correct, since they are allocated as ubyte[]. uint[] also would not be scanned.
-Steve
got it, finally! thanks to everyone for their help.
/det
Re: static void arrays under garbage control?
On 2/26/15 2:28 PM, Ali Çehreli wrote: On 02/26/2015 10:07 AM, Steven Schveighoffer wrote: // will not be scanned by GC for pointers: void[] buffer4 = cast(void[])(new ubyte[16]); uint[] buffer5 = cast(uint[])(new ubyte[16]); Correct, since they are allocated as ubyte[]. uint[] also would not be scanned. I've been under the impression that current GC being imprecise, even an unfortunate bit-pattern in an int would keep otherwise dead objects alive. Is that still true for a single int? It depends on where the int is, not whether it's an int or not. An int on the stack will be scanned as if it were a pointer, because the GC does not know the type for those bits. An int on the heap will be scanned if it is in a block marked as having pointers. And is 'new int[N]' better in this regard because it never keeps objects alive? Correct, at the point you allocate new int[N], the type system is aware that the block will NOT contain pointers, so the GC is informed of this. Now, if you said new int[][N], the block contains pointers, as each element has a pointer and length. But the lengths will ALSO be scanned even though they aren't pointers. This is a heuristic for the scanner. It's very blunt, but it's better than completely imprecise :) A truly precise scanner would be informed not only of the block bit patterns (which bits are alive, which are pointers, etc), but also of the stack frame and static data bit patterns. Such a scanner would be better at avoiding leaked memory, but possibly perform worse, since it may spend extra time parsing type data when the entire block is all pointers, etc. It also requires more metadata, and can affect cache coherency. I'll put forth right now, I am NOT the person to ask about this in any detail :) -Steve
Re: static void arrays under garbage control?
On Wed, Feb 25, 2015 at 08:20:37PM -0600, captaindet via Digitalmars-d-learn
wrote:
[...]
struct Stuff2Do{
static ubyte[1024*1024] buffer4speed = void; // even if untyped at this
point
// more
}
[...]
Tangential note: be careful with putting a large static array inside a
struct. Structs are passed by value, which means that if you didn't
allocate that struct on the heap and you pass it around to various
functions, you will overflow your stack very quickly -- every function
call that passes the struct will consume 1MB of stack space. Many OSes
do not allocate that much space for the runtime stack.
T
--
Real Programmers use cat a.out.
Re: static void arrays under garbage control?
On Wed, 25 Feb 2015 20:20:37 -0600, captaindet wrote:
On 2015-02-25 19:24, Adam D. Ruppe wrote:
does this warning only apply to dynamic void[] arrays but not to
static void[CTconstant] arrays?
Both of those will be scanned for pointers.
thanks, adam,
so i should always use
struct Stuff2Do{
static ubyte[1024*1024] buffer4speed = void; // even if untyped at
this point
// more
}
to avoid that the GC is scanning my buffer for pointers?
compiler is clever enough to not mark `ubyte` arrays as GC ranges.
signature.asc
Description: PGP signature
Re: static void arrays under garbage control?
On 2015-02-25 20:45, H. S. Teoh via Digitalmars-d-learn wrote:
On Wed, Feb 25, 2015 at 08:20:37PM -0600, captaindet via Digitalmars-d-learn
wrote:
[...]
struct Stuff2Do{
static ubyte[1024*1024] buffer4speed = void; // even if untyped at this
point
// more
}
[...]
Tangential note: be careful with putting a large static array inside a
struct. Structs are passed by value, which means that if you didn't
allocate that struct on the heap and you pass it around to various
functions, you will overflow your stack very quickly -- every function
call that passes the struct will consume 1MB of stack space. Many OSes
do not allocate that much space for the runtime stack.
T
this is why i tried to get away with a 'static' static array, which only exists once for all Stuff2Do structs. but as it seems, i'll rather need on the order of 512MB buffer, so i will probably go with c.stdlib.malloc
/det
Re: static void arrays under garbage control?
On 2015-02-25 19:24, Adam D. Ruppe wrote:
does this warning only apply to dynamic void[] arrays but not to static
void[CTconstant] arrays?
Both of those will be scanned for pointers.
thanks, adam,
so i should always use
struct Stuff2Do{
static ubyte[1024*1024] buffer4speed = void; // even if untyped at this
point
// more
}
to avoid that the GC is scanning my buffer for pointers?
Re: static void arrays under garbage control?
You don't have to do all that, if it is typed ubyte[] instead of void[], it shouldn't be scanned at all anyway.
static void arrays under garbage control?
if i understand correctly, static arrays are exempt from GC scanning for memory pointers http://dlang.org/garbage.html : Pointers in D can be broadly divided into two categories: Those that point to garbage collected memory, and those that do not. Examples of the latter are pointers created by calls to C's malloc(), pointers received from C library routines, pointers to static data. but there is also a warning for void arrays http://dlang.org/arrays.html : The garbage collector will scan void[] arrays for pointers, since such an array may have been implicitly converted from an array of pointers or an array of elements that contain pointers. does this warning only apply to dynamic void[] arrays but not to static void[CTconstant] arrays? (because sometimes the docs also mean static arrays even if just type[] is written.) thanks! ps: this is for 32bit apps
Re: static void arrays under garbage control?
On Thursday, 26 February 2015 at 01:15:37 UTC, captaindet wrote: pointers to static data. Keep in mind that this is pointers TO static data - that is, if the object itself is static, it is not to be collected. static int[5] foo = [1,2,3,4,5]; // foo will never be collected, even if there are no pointers to it But, the static array itself will be scanned: static Object[2] objects; The objects stored in there won't be garbage collected because this array is still scanned. So does this warning only apply to dynamic void[] arrays but not to static void[CTconstant] arrays? Both of those will be scanned for pointers.
