Re: D2 postgresql interface - Phobos2?
Hi Piotr, How cool. Very glad you're going native D. I've used Npgsql a lot and also the more standard data.sqlclient interface from c# so I'm happy you're modeling after that API. In your general API, will you support the more advanced features like creating functions, refcursors, preplanning queries, etc? Also, your base db object looks very usefull. Do you have any sense when you would have code ready for testing purposes (don't take this as pressure, just curious)? Or for others to review? Maybe people like Mandeep or myself could help on the coding front? As I guess a DB interface will be used a lot. Andrei (or Phobos team?), would you consider a DB interface in Phobos or are you already planning something yourselves for Phobos or feel this does not belong in Phobos and should stay outside? Many thanks, fil
Printing wide characters in console
I think I found a solution for it. This code changes encoding of string to
ansi which is default string encoding in windows console.
import std.c.windows.windows;
string ansi(string str)
{
wchar[] src = toUTF16(str).dup;
char[] dest;
dest.length = str.length;
WideCharToMultiByte(0, 0, src.ptr, -1, dest.ptr, dest.length, null,
null);
return cast(string)dest;
}
It's inconvinient to call this function every time printing string, I think
simillar code should be in the standard library.
Threads, shread and TLS
So, I thought I sort of understood shared and now I think I don't.
If I have a class:
class foo {
int x;
static int y;
shared static int z;
}
So x is one instance per class and is thread-local?
y is one instance per thread?
z is one instance per application, i.e., global?
If that's true (and I realize it might not be), and I want to initialize these
variables in constructors, how does that work?
I think
class foo {
...(as before)
this() { x = 2; } // ok
static this() { y = 3; } // is this called once per thread?
shared static this() { z = 3;} // also, okay, called before main
}
but I don't understand what happens with threads and the static this
constructor. How/when are the thread-local copies constructed? How do you
initialize/construct the thread-local static data?
Thanks!
Adam
Re: D2 postgresql interface - Phobos2?
How cool. Very glad you're going native D. I've used Npgsql a lot and also the more standard data.sqlclient interface from c# so I'm happy you're modeling after that API. In your general API, will you support the more advanced features like creating functions, refcursors, preplanning queries, etc? I plan to support most of postgres features. Preplanning or preparing queries is already done using prepare() method of PGCommand just like in Npgsql. Cursors also should be available to users. In case of functions I assume native D functions linked to postgres. These must be compiled to shared library and loaded within server but AFAIK shared library support is not complete in D2 (maybe I'm misinformed?). Also there will be support for other advanced features like asynchronous notifications (using NOTIFY channel, payload; syntax). Also, your base db object looks very usefull. Do you have any sense when you would have code ready for testing purposes (don't take this as pressure, just curious)? Or for others to review? Maybe people like Mandeep or myself could help on the coding front? As I guess a DB interface will be used a lot. Non query prepared statements are already working. For general API, I need to finish query result handling and binary formatting of compound types and arrays. Then I will be working on ORM API. I will post source code when result handling is done :) regards, Piotr
Re: How does dsource create the multiple projects?
On Thu, 06 Jan 2011 06:25:16 -0500, Sclytrack sclytr...@fake.com wrote: How does dsource handle the multiple projects? Is that each projects its own trac environment. Does it use sqlite? I know not a D related question. Each project is its own trac environment. What DB is used, I'm not sure. -Steve
Re: Threads, shread and TLS
On Fri, 07 Jan 2011 09:24:18 -0500, Adam Conner-Sax
adam_conner_...@yahoo.com wrote:
So, I thought I sort of understood shared and now I think I don't.
If I have a class:
class foo {
int x;
static int y;
shared static int z;
}
So x is one instance per class and is thread-local?
y is one instance per thread?
z is one instance per application, i.e., global?
Yes to all
If that's true (and I realize it might not be), and I want to initialize
these
variables in constructors, how does that work?
I think
class foo {
...(as before)
this() { x = 2; } // ok
static this() { y = 3; } // is this called once per thread?
shared static this() { z = 3;} // also, okay, called before main
}
but I don't understand what happens with threads and the static this
constructor. How/when are the thread-local copies constructed? How do
you
initialize/construct the thread-local static data?
static this() is run upon thread creation (and once at the beginning of
the program for the main thread), and static ~this() is run when a thread
is destroyed.
All static this() and shared static this() functions are assembled by the
runtime into an array of constructors, built in the correct order based on
import dependencies. So each time a thread starts, it simply goes through
an array and calls each function.
-Steve
Variants for interfaces
Hi,
Is it possible to point a variable to an interface. The below code does not
compile.
module testD;
import std.stream;
import std.stdio;
import std.variant;
interface A {
void func1();
}
class AC: A {
void func1() {
writeln(func1);
}
}
int main() {
A a = new AC();
a.func1();
Variant b = Variant(a);
return 0;
}
It says that opEquals is not defined for the interface.
Thanks Regards
Mandeep
Re: Threads, shread and TLS
== Quote from Adam Conner-Sax (adam_conner_...@yahoo.com)'s article
So, I thought I sort of understood shared and now I think I don't.
If I have a class:
class foo {
int x;
static int y;
shared static int z;
}
So x is one instance per class and is thread-local?
y is one instance per thread?
z is one instance per application, i.e., global?
If that's true (and I realize it might not be), and I want to initialize these
variables in constructors, how does that work?
I think
class foo {
...(as before)
this() { x = 2; } // ok
static this() { y = 3; } // is this called once per thread?
shared static this() { z = 3;} // also, okay, called before main
}
but I don't understand what happens with threads and the static this
constructor. How/when are the thread-local copies constructed? How do you
initialize/construct the thread-local static data?
Thanks!
Adam
Nevermind. Answered myself with the following:
import core.thread;
import std.c.stdio;
class foo {
int a;
shared int b;
static int x;
shared static int y;
shared static int[] arr1;
shared static int[] arr2;
this() { a = 1; b=10; }
static this() { x=100; arr1 ~= x; }
shared static this() { y=1000; arr2 ~= y; }
static void A() { x++; y++; }
void B() { a++; b++; }
void report() {
printf(a=%i; b=%i; x=%i; y=%i; arr1.length=%i;
arr2.length=%i\n,a,b,x,y,arr1.length, arr2.length);
}
}
void main()
{
auto f = new foo();
void call_foo_functions() { f.A(); f.B(); f.report(); }
auto tg = new ThreadGroup();
foreach (k; 0..3) {
auto t = new Thread(call_foo_functions);
tg.add(t);
t.start();
}
tg.joinAll();
printf(back in main: );
f.report();
}
which output:
a=2; b=11; x=101; y=1001; arr1.length=2; arr2.length=1
a=3; b=12; x=101; y=1002; arr1.length=2; arr2.length=1
a=4; b=13; x=101; y=1003; arr1.length=3; arr2.length=1
back in main: a=4; b=13; x=100; y=1003; arr1.length=3; arr2.length=1
which all makes sense (to me) except a. Why is a acting global? Is it since it
isn't static so it belongs to the class and there is only one copy of the class?
Then what makes a and b different?
Also, though maybe it's obvious to everybody else, I think the docs should
explain
someplace that static this() gets called once *per thread*. Maybe I just
missed it.
Adam
begin 644 TLS_Test.d
M:6UP;W)T(-OF4N=AR96%D.PII;7!OG0@W1D+F,NW1D:6\[@IC;%S
MR!F;V\@PH@B`@:6YT($[B`@VAAF5D(EN=!B.PH*(!S=%T:6,@
M:6YT('@[B`@VAAF5D('-T871I8R!I;G0@3L*(!S:%R960@W1A=EC
M(EN=%M=(%RC$[B`@VAAF5D('-T871I8R!I;G1;72!AG(R.PH*(!T
M:ES*D@R!A(#T@,3...@8ctq,#...@?0h@('-T871I8R!T:ES*D@R!X/3$P
M,#...@87)R,2!^/2!X.R!]B`@VAAF5D('-T871I8R!T:ES*D@R!Y/3$P
M,#`[(%RC(@?CT@3L@('T*B`@W1A=EC('9O:6...@02@I('L@LK.R!Y
M*RL[('T*(!V;VED($(H*2![($k...@8blk.r!]b...@=f]i9!R97!OG0H
M*2![B`@(!PFEN=8H(F$])6D[((])6D[('@])6D[('D])6D[(%RC$N
M;5N9W1H/25I.R!AG(R+FQE;F=T:#TE:5QN(BQA+(LQY+%RC$N;5N
M9W1H+!AG(R+FQE;F=T:D[b...@?0i]@H*G9O:60@;6%I;b...@i(`I[@H@
M(%U=\...@9b`](YE=R!F;V\H*3L*(!V;VED(-A;Q?9F]O7V9U;F-T:6]N
Mr...@i('l...@9by!*D[(8...@i.r!f+g)E]R=@I.R!](`H@(%U=\...@=@
M/2!N97@5AR96%D1W)O=7`H*3L*(!F;W)E86-H(AK.R`P+BXS*2![B`@
M(!A=71O('0@/2!N97@5AR96%D*9C86QL7V9O;U]F=6YC=EO;G,I.PH@
M(`...@=N861D*'0I.PH@(`...@=YS=%R=@I.PH@('T*(!T9RYJ;VEN06QL
M*D[b...@+r\@83TQ/PH@(\O((],3,_b...@+r\@#TQ,#`_b...@+r\@3TQ
M,#`SB`@')I;G1F*)B86-K(EN(UA:6XZ((I.PH@(8NF5P;W)T*D[
-B!](`H@(`@@H*@``
`
end
Re: Threads, shread and TLS
On Fri, 07 Jan 2011 09:55:33 -0500, Adam Conner-Sax
adam_conner_...@yahoo.com wrote:
== Quote from Adam Conner-Sax (adam_conner_...@yahoo.com)'s article
So, I thought I sort of understood shared and now I think I don't.
If I have a class:
class foo {
int x;
static int y;
shared static int z;
}
So x is one instance per class and is thread-local?
y is one instance per thread?
z is one instance per application, i.e., global?
If that's true (and I realize it might not be), and I want to
initialize these
variables in constructors, how does that work?
I think
class foo {
...(as before)
this() { x = 2; } // ok
static this() { y = 3; } // is this called once per thread?
shared static this() { z = 3;} // also, okay, called before main
}
but I don't understand what happens with threads and the static this
constructor. How/when are the thread-local copies constructed? How do
you
initialize/construct the thread-local static data?
Thanks!
Adam
Nevermind. Answered myself with the following:
import core.thread;
import std.c.stdio;
class foo {
int a;
shared int b;
static int x;
shared static int y;
shared static int[] arr1;
shared static int[] arr2;
this() { a = 1; b=10; }
static this() { x=100; arr1 ~= x; }
shared static this() { y=1000; arr2 ~= y; }
static void A() { x++; y++; }
void B() { a++; b++; }
void report() {
printf(a=%i; b=%i; x=%i; y=%i; arr1.length=%i;
arr2.length=%i\n,a,b,x,y,arr1.length, arr2.length);
}
}
void main()
{
auto f = new foo();
void call_foo_functions() { f.A(); f.B(); f.report(); }
auto tg = new ThreadGroup();
foreach (k; 0..3) {
auto t = new Thread(call_foo_functions);
tg.add(t);
t.start();
}
tg.joinAll();
printf(back in main: );
f.report();
}
which output:
a=2; b=11; x=101; y=1001; arr1.length=2; arr2.length=1
a=3; b=12; x=101; y=1002; arr1.length=2; arr2.length=1
a=4; b=13; x=101; y=1003; arr1.length=3; arr2.length=1
back in main: a=4; b=13; x=100; y=1003; arr1.length=3; arr2.length=1
which all makes sense (to me) except a. Why is a acting global? Is it
since it
isn't static so it belongs to the class and there is only one copy of
the class?
Then what makes a and b different?
OK, so here is what is happening :)
call_foo_functions is a delegate, which means it has a frame pointer to
the main function. So all three threads' f is the *same* f (the one
defined in main()).
I would suggest that you move call_foo_functions outside main, and
instantiate an additional f *inside* the function. This would be more
correct.
Incidentally, it appears that this allows untagged sharing (i.e. sharing
data that isn't tagged with shared) I wonder if this issue has been
reported before? Sean?
Also, though maybe it's obvious to everybody else, I think the docs
should explain
someplace that static this() gets called once *per thread*. Maybe I
just missed it.
Yes, the documentation is out of date. Could you file a bugzilla report
on this?
-Steve
Re: Threads, shread and TLS
Thanks!
It's not really about correctness as much as trying to understand how these
different storage classes work. I understand that there is only one foo object.
I wanted to see which parts are thread-local and which are shared and how the
constructors work.
I'm working (way over my head) on a more complex threading issue and I realized
that I didn't quite understand the storage classes and constructors. Now I get
it
a bit more.
By untagged sharing do you mean the a variable which is shared in the sense
that all threads see the same copy but does not have storage class shared?
Yes,
that confuses me too. Should it be an error?
I have never dealt with bugzilla but I will try to figure out how to do what you
ask :)
Thanks again.
Adam
== Quote from Steven Schveighoffer (schvei...@yahoo.com)'s article
On Fri, 07 Jan 2011 09:55:33 -0500, Adam Conner-Sax
adam_conner_...@yahoo.com wrote:
== Quote from Adam Conner-Sax (adam_conner_...@yahoo.com)'s article
So, I thought I sort of understood shared and now I think I don't.
If I have a class:
class foo {
int x;
static int y;
shared static int z;
}
So x is one instance per class and is thread-local?
y is one instance per thread?
z is one instance per application, i.e., global?
If that's true (and I realize it might not be), and I want to
initialize these
variables in constructors, how does that work?
I think
class foo {
...(as before)
this() { x = 2; } // ok
static this() { y = 3; } // is this called once per thread?
shared static this() { z = 3;} // also, okay, called before main
}
but I don't understand what happens with threads and the static this
constructor. How/when are the thread-local copies constructed? How do
you
initialize/construct the thread-local static data?
Thanks!
Adam
Nevermind. Answered myself with the following:
import core.thread;
import std.c.stdio;
class foo {
int a;
shared int b;
static int x;
shared static int y;
shared static int[] arr1;
shared static int[] arr2;
this() { a = 1; b=10; }
static this() { x=100; arr1 ~= x; }
shared static this() { y=1000; arr2 ~= y; }
static void A() { x++; y++; }
void B() { a++; b++; }
void report() {
printf(a=%i; b=%i; x=%i; y=%i; arr1.length=%i;
arr2.length=%i\n,a,b,x,y,arr1.length, arr2.length);
}
}
void main()
{
auto f = new foo();
void call_foo_functions() { f.A(); f.B(); f.report(); }
auto tg = new ThreadGroup();
foreach (k; 0..3) {
auto t = new Thread(call_foo_functions);
tg.add(t);
t.start();
}
tg.joinAll();
printf(back in main: );
f.report();
}
which output:
a=2; b=11; x=101; y=1001; arr1.length=2; arr2.length=1
a=3; b=12; x=101; y=1002; arr1.length=2; arr2.length=1
a=4; b=13; x=101; y=1003; arr1.length=3; arr2.length=1
back in main: a=4; b=13; x=100; y=1003; arr1.length=3; arr2.length=1
which all makes sense (to me) except a. Why is a acting global? Is it
since it
isn't static so it belongs to the class and there is only one copy of
the class?
Then what makes a and b different?
OK, so here is what is happening :)
call_foo_functions is a delegate, which means it has a frame pointer to
the main function. So all three threads' f is the *same* f (the one
defined in main()).
I would suggest that you move call_foo_functions outside main, and
instantiate an additional f *inside* the function. This would be more
correct.
Incidentally, it appears that this allows untagged sharing (i.e. sharing
data that isn't tagged with shared) I wonder if this issue has been
reported before? Sean?
Also, though maybe it's obvious to everybody else, I think the docs
should explain
someplace that static this() gets called once *per thread*. Maybe I
just missed it.
Yes, the documentation is out of date. Could you file a bugzilla report
on this?
-Steve
Re: D Language info collected and combined into one CHM file.
david wang Wrote: Sorry, surrely that I've posted the attachment (CHM file), but I can not see it. Does anyone can kindly help me to point out that how to transfer chm file to this BBS? waiting for kindly reply. David. http://docs.google.com or maybe http://prowiki.org/wiki4d/wiki.cgi?LanguageSpecification Personally I don't like attachments to these forms, and maybe others are like me. Also the website documentation is distributed with the dmd zip file for offline viewing.
Re: Variants for interfaces
Mandeep Singh Brar Wrote:
Hi,
Is it possible to point a variable to an interface. The below code does not
compile.
module testD;
import std.stream;
import std.stdio;
import std.variant;
interface A {
void func1();
}
class AC: A {
void func1() {
writeln(func1);
}
}
int main() {
A a = new AC();
a.func1();
Variant b = Variant(a);
return 0;
}
It says that opEquals is not defined for the interface.
Thanks Regards
Mandeep
I suggest submitting it as a bug:
http://d.puremagic.com/issues/
Re: D Language info collected and combined into one CHM file.
Vladimir already made an automatic script that does this, see: http://thecybershadow.net/d/docs/
Re: How the GC distinguishes code from data
It assumes everything on the stack is pointers, at the moment, I believe Uh-oh... not the answer I wanted to hear, but I was half-expecting this. So doesn't that mean that, at the moment, D will leak memory? If it's not on the garbage collected heap, it won't scan it unless you tell it to. But what if it's a void[] on a non-GC heap? Doesn't the language say that needs to be scanned too?
Re: Threads, shread and TLS
On Fri, 07 Jan 2011 11:30:17 -0500, Adam Conner-Sax adam_conner_...@yahoo.com wrote: Thanks! It's not really about correctness as much as trying to understand how these different storage classes work. I understand that there is only one foo object. I wanted to see which parts are thread-local and which are shared and how the constructors work. A class instance can be shared or unshared. Either way, changing the data on the same instance updates the same instance, there is not a copy of the whole world in each thread, just a copy of the thread local storage block. So you are sort of conflating 'per instance' with 'per thread.' int x -- per instance (shared or not) static int x -- per thread (in the TLS block) shared static int x -- per process, not in any instance. I'm working (way over my head) on a more complex threading issue and I realized that I didn't quite understand the storage classes and constructors. Now I get it a bit more. By untagged sharing do you mean the a variable which is shared in the sense that all threads see the same copy but does not have storage class shared? Yes, that confuses me too. Should it be an error? Yes, if you have a piece of data that shared and not marked with __gshared or shared, then we have a problem. The problem is that a lot of code assumes that situation cannot happen without casts (you can always cast and override the type system), so you can make assumptions based on that. For example, a lot of C++/java code is written *just in case* an object is shared. With D, the hope is that you are *guaranteed* that it is shared or not, so you can optimize your code that way (i.e. use a lock or not). As long as the possibility exists that code not marked as shared can be easily shared without a cast, we cannot make those assumptions. I think it should be filed as a bug, but I'm not sure if someone's already reported it. I have never dealt with bugzilla but I will try to figure out how to do what you ask :) Go to http://d.puremagic.com/issues/enter_bug.cgi You will have to create a user before filing a bug, but bugzilla is relatively straightforward to use. -Steve
Re: How the GC distinguishes code from data
Kinda sorta. I haven't had any problems from that. If you allocate very large blocks in the garbage collector you may face trouble :-) Haha okay, thanks. :) (This makes me shiver quite a bit...) You have to add it to the garbage collector's list of roots But if I need to do that, then what would be the difference between void[] and ubyte[]?
Re: How the GC distinguishes code from data
%u wfunct...@hotmail.com wrote: You have to add it to the garbage collector's list of roots But if I need to do that, then what would be the difference between void[] and ubyte[]? None what so ever. If you want to mark some memory with special bits, use setattr in core.memory. -- Simen
auto declarations
auto a = 1, b = null; int a = 1, *b = null; The first is accepted by dmd, and it should result in typeof(a) == int and typeof(b) == void*. It is somewhat contradictory to the error message resulting from the second: multiple declarations must have the same type, not int and int* I am skeptical of dmd's permitting the first. Does anyone else see any utility in it?
Re: auto declarations
Ellery Newcomer wrote: auto a = 1, b = null; int a = 1, *b = null; The first is accepted by dmd, and it should result in typeof(a) == int and typeof(b) == void*. It is somewhat contradictory to the error message resulting from the second: multiple declarations must have the same type, not int and int* I am skeptical of dmd's permitting the first. Does anyone else see any utility in it? Personally, I like it. In second line you specify int type, and list of *int* variables. In first line you specify list of initialized variables which types should be inferred automatically. I see no reason why first line should not be permitted. If one would need several variables of one type, why he wouldn't specify exact type instead of using 'auto'?
Re: How the GC distinguishes code from data
On Fri, 07 Jan 2011 16:39:20 -0500, %u wfunct...@hotmail.com wrote:
None what so ever.
Huh.. then what about what is said in this link?
http://d.puremagic.com/issues/show_bug.cgi?id=5326#c1
I was told that void[] could contain references, but that ubyte[] would
not, and
that the GC would need to scan the former but not the latter. Is that
wrong?
First, you should understand that the GC does not know what data is in a
memory block. It has no idea that the block is a void[] or a ubyte[] or a
class instance or whatever it is. All it knows is that it's data. What
makes it scan a block is a bit set on the block indicating that it
contains pointers. This bit is set by the higher-level runtime routines
(like the ones that create an array) which use the TypeInfo to determine
whether to set the NO_SCAN bit or not.
Second, memory that is not part of D's allocation is *not* scanned or
marked, no matter where it is. Essentially the mark routine goes like
this (pseudocode):
foreach(root; roots)
if(root.hasPointers) // notice this has nothing to do with type
foreach(pointer; root)
if(pointer.pointsAt.GCHeapBlock)
pointer.heapBlock.mark = true;
while(changesWereMade)
foreach(heapBlock; heap)
if(heapBlock.hasPointers)
foreach(pointer; heapBlock)
if(pointer.pointsAt.GCHeapBlock)
{
pointer.heapBlock.mark = true;
changesWereMade = true;
}
// free memory
foreach(heapBlock; heap)
if(!heapBlock.mark)
free(heapBlock)
So essentially, you can see if you allocated memory for example with
malloc, and you didn't add it as a root, it's neither scanned nor marked.
It does not participate whatsoever with the collection cycle, no matter
what the type of the data is.
Now, you should also realize that just because an array is a void[]
doesn't necessarily make it marked as containing pointers. It is quite
possible to implicitly cast a ubyte[] to a void[], and this does not
change the NO_SCAN bit in the memory block. Data *allocated* as a void[]
(which I highly recommend *not* doing) will be conservatively marked as
containing pointers. This is probably where you get the notion that
void[] contains pointers.
-Steve
Re: auto declarations
On Friday, January 07, 2011 13:32:42 Ellery Newcomer wrote: auto a = 1, b = null; int a = 1, *b = null; The first is accepted by dmd, and it should result in typeof(a) == int and typeof(b) == void*. It is somewhat contradictory to the error message resulting from the second: multiple declarations must have the same type, not int and int* I am skeptical of dmd's permitting the first. Does anyone else see any utility in it? The second should definitely _not_ be allowed. * definitely goes with the type in D (as it should have in C), not the variable. So, the *b = null makes no sense. However, I'm vere suprised that the first one succeeds. I think that it should be reported as a bug. All variables declared on the same line are supposed to have the same type. - Jonathan M Davis
Re: How the GC distinguishes code from data
First, you should understand that the GC does not know what data is in a memory block. That is exactly why I was wondering how it figures things out. :) Data *allocated* as a void[] (which I highly recommend *not* doing) will be conservatively marked as containing pointers. Ah, all right, that clears things up! Thank you!!
Re: D Language info collected and combined into one CHM file.
Great! Thank you. Please view the download link: https://docs.google.com/leaf?id=0B38se3xzJrbuMjNlMTQ5MzUtYTM1NC00M2UyLWJjODctMDZlM2Y2ZWQwYjlmhl=enauthkey=CPvh1ZoG (if you can not see the page correctly shown, just refresh the page) this D_Language.chm was produced yesterday and collected the latest docs from Digitalmars.com. Maybe this document is useful for someone. Best regards. David.
std.container.Array/RefCounted(T) leaking memory?
Hi,
This code seems to leak memory, as the memory isn't reclaimed:
//Test memory here: low
{
auto b = Array!(bool)();
b.length = 1024 * 1024 * 128 * 8;
//Test memory here: high
}
//Test memory here: high
Am I missing something about how Array(T) (and RefCounted) works, or is this
really a bug?
Thank you!
