Re: scope attribute vs scope keyword vs scope storage class
On Wednesday, 5 February 2014 at 15:43:45 UTC, Dicebot wrote: For delegates it actually has a meaning. No deprecation. It's the same meaning as for scope classes though. And the same unsafety.
Re: std.parallelism: How to wait all tasks finished?
On Wednesday, 5 February 2014 at 15:38:14 UTC, Cooler wrote:
On Tuesday, 4 February 2014 at 03:26:04 UTC, Dan Killebrew
wrote:
It seems to me that worker threads will continue as long as
the queue isn't empty. So if a task adds another task to the
pool, some worker will process the newly enqueued task.
No. After taskPool.finish() no way to add new tasks to the
queue. taskPool.put will not add new tasks.
Then perhaps you need to create a new TaskPool (and make sure
that workers add their tasks to the correct task pool), so
that you can wait on the first task pool, then wait on the
second task pool, etc.
auto phase1 = new TaskPool();
//make sure all new tasks are added to phase1
phase1.finish(true);
auto phase2 = new TaskPool();
//make sure all new tasks are added to phase2
phase2.finish(true);
Will not help. I don't know beforehand what tasks will be
created. procData is recursive and it decides create new task or
not.
Something like this? (not tested...)
shared bool more = true;
...
...
...
void procData(){
if(...)
{
taskPool.put(task(procData));
more = true;
}
}
while(true)
{
taskPool.finish(true);
if (!more) break;
else more = false;
}
Re: std.parallelism: How to wait all tasks finished?
On Thursday, 6 February 2014 at 14:52:36 UTC, Cooler wrote:
On Thursday, 6 February 2014 at 14:42:57 UTC, Cooler wrote:
On Thursday, 6 February 2014 at 11:30:17 UTC, Andrea Fontana
wrote:
On Wednesday, 5 February 2014 at 15:38:14 UTC, Cooler wrote:
On Tuesday, 4 February 2014 at 03:26:04 UTC, Dan Killebrew
wrote:
It seems to me that worker threads will continue as long
as the queue isn't empty. So if a task adds another task
to the pool, some worker will process the newly enqueued
task.
No. After taskPool.finish() no way to add new tasks to the
queue. taskPool.put will not add new tasks.
Then perhaps you need to create a new TaskPool (and make
sure that workers add their tasks to the correct task
pool), so that you can wait on the first task pool, then
wait on the second task pool, etc.
auto phase1 = new TaskPool();
//make sure all new tasks are added to phase1
phase1.finish(true);
auto phase2 = new TaskPool();
//make sure all new tasks are added to phase2
phase2.finish(true);
Will not help. I don't know beforehand what tasks will be
created. procData is recursive and it decides create new
task or
not.
Something like this? (not tested...)
shared bool more = true;
...
...
...
void procData(){
if(...)
{
taskPool.put(task(procData));
more = true;
}
}
while(true)
{
taskPool.finish(true);
if (!more) break;
else more = false;
}
It is closer, but after taskPool.finish() all tries to
taskPool.put() will be rejected. Let's me clear example.
import std.stdio, std.parallelism, core.thread;
shared int i;
void procData(){
synchronized ++i;
if(i = 100)
return;
foreach(i; 0 .. 100)
taskPool.put(task(procData)); // New tasks will be
rejected after
// taskPool.finish()
}
void main(){
taskPool.put(task(procData));
Thread.sleep(1.msecs); // The final output of i depends on
duration here
taskPool.finish(true);
writefln(i = %s, i);
}
In the example above the total number of tasks executed
depends on sleep duration.
Forgot to say - I know how to solve the topic problem. My
question is What is the BEST way?.
One of my idea - may be introduce new function, named for
example
wait, that will block until there are working tasks?
What about sync ++taskCount when you put() something and
--taskCount when task is done? And on main while(i 0)
Thread.yield(); ?
Re: std.parallelism: How to wait all tasks finished?
On Thursday, 6 February 2014 at 16:07:51 UTC, Andrea Fontana
wrote:
On Thursday, 6 February 2014 at 14:52:36 UTC, Cooler wrote:
On Thursday, 6 February 2014 at 14:42:57 UTC, Cooler wrote:
On Thursday, 6 February 2014 at 11:30:17 UTC, Andrea Fontana
wrote:
On Wednesday, 5 February 2014 at 15:38:14 UTC, Cooler wrote:
On Tuesday, 4 February 2014 at 03:26:04 UTC, Dan Killebrew
wrote:
It seems to me that worker threads will continue as long
as the queue isn't empty. So if a task adds another task
to the pool, some worker will process the newly enqueued
task.
No. After taskPool.finish() no way to add new tasks to
the queue. taskPool.put will not add new tasks.
Then perhaps you need to create a new TaskPool (and make
sure that workers add their tasks to the correct task
pool), so that you can wait on the first task pool, then
wait on the second task pool, etc.
auto phase1 = new TaskPool();
//make sure all new tasks are added to phase1
phase1.finish(true);
auto phase2 = new TaskPool();
//make sure all new tasks are added to phase2
phase2.finish(true);
Will not help. I don't know beforehand what tasks will be
created. procData is recursive and it decides create new
task or
not.
Something like this? (not tested...)
shared bool more = true;
...
...
...
void procData(){
if(...)
{
taskPool.put(task(procData));
more = true;
}
}
while(true)
{
taskPool.finish(true);
if (!more) break;
else more = false;
}
It is closer, but after taskPool.finish() all tries to
taskPool.put() will be rejected. Let's me clear example.
import std.stdio, std.parallelism, core.thread;
shared int i;
void procData(){
synchronized ++i;
if(i = 100)
return;
foreach(i; 0 .. 100)
taskPool.put(task(procData)); // New tasks will be
rejected after
// taskPool.finish()
}
void main(){
taskPool.put(task(procData));
Thread.sleep(1.msecs); // The final output of i depends on
duration here
taskPool.finish(true);
writefln(i = %s, i);
}
In the example above the total number of tasks executed
depends on sleep duration.
Forgot to say - I know how to solve the topic problem. My
question is What is the BEST way?.
One of my idea - may be introduce new function, named for
example
wait, that will block until there are working tasks?
What about sync ++taskCount when you put() something and
--taskCount when task is done? And on main while(i 0)
Thread.yield(); ?
Something like this:
import std.stdio, std.parallelism, core.thread;
import std.random;
shared size_t taskCount;
shared size_t i;
void procData()
in { synchronized ++i; }
out { synchronized --taskCount; }
body
{
if (i 100)
return;
foreach(i; 0 .. 100)
{
taskPool.put(task(procData));
synchronized ++taskCount;
}
}
void main(){
taskCount = 2;
taskPool.put(task(procData));
taskPool.put(task(procData));
while(taskCount 0)
Thread.yield();
}
Re: std.parallelism: How to wait all tasks finished?
On Mon, 2014-02-03 at 00:00 +, Cooler wrote: I have several tasks. Each task may or may not create another task. What is the best way to wait until all tasks finished? What you are describing here is a classic fork/join architecture. The tasks are structured as a tree with synchronization handled by the sub-nodes. As far as I am aware std.parallelism focuses on data parallelism which is a scatter/gather (aka map/reduce) model of just a single layer. All the code fragments in the thread have, I believe, been predicated on working with a thread pool as an explicit global entity. I think the problems have stemmed from taking this viewpoint. I would suggest following the way the Java fork/join framework (based on Doug Lea's original) works. There is an underlying global thread pool, but the user code uses the fork/join abstraction layer in order to create the tree of synchronization dependencies. In this case instead of working with tasks directly there needs to be a type whose job it is to be a non-leaf node in the tree that handles synchronization whilst nonetheless creating tasks and submitting them to the pool. This is clearly something that could turn into an addition to std.parallelism or be std.forkjoin. Sorry I have no actual code to offer, but the overall design of what is needed is well understood, at least in the Java context. C++ has a long way to go to catch up, as does D. The other thing that then sits on this is lazy stream parallelism, which is what Java 8 is adding to the mix. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: What is the best way to iterate over string chars?
On Thursday, 6 February 2014 at 18:50:28 UTC, Cooler wrote:
Example3:
foreach(i, c; s){
// do something with c and i
}
this is the best. You can also specific c to be char or wchar or
dchar specifically if you want it do do some UTF decoding for
you. (char is the default - note that this will send you each
byte of a multi-byte sequence. dchar on the other hand will
automatically combine those)
Re: scope attribute vs scope keyword vs scope storage class
On Wednesday, 5 February 2014 at 15:43:45 UTC, Dicebot wrote: D documentation has rather incosistent naming for attribute groups. - scope classes are deprecated, but usage of scope as storage class is still legal (it is expected to be a no-op for now) Couldn't scope allocating a class on the stack just be considered an optimization that can be applied if the scope storage class become fully implemented? - scope storage class for function parameters is also accepted as no-op. For delegates it actually has a meaning. No deprecation. No-op just because it hasn't been implemented yet, right?
Re: Is it possible to convert a delegate into a function?
On Thursday, 6 February 2014 at 21:26:06 UTC, Gary Willoughby
wrote:
I know you can convert functions into delegates using:
http://dlang.org/phobos/std_functional.html#.toDelegate but can
you do this the other way around?
I have a method that needs a function pointer but it would be
nice to use a converted delegate pointer instead.
auto toFunction(DG)(DG dg)
{
assert(!dg.ptr);
return dg.funcptr;
}
void foo(int function() input)
{
import std.stdio;
writeln(input());
}
void main()
{
int boo()
{
return 42;
}
// foo(boo); // fails
foo(toFunction(boo)); // prints 42
}
Re: scope attribute vs scope keyword vs scope storage class
On Thursday, 6 February 2014 at 19:01:52 UTC, Brad Anderson wrote: On Wednesday, 5 February 2014 at 15:43:45 UTC, Dicebot wrote: D documentation has rather incosistent naming for attribute groups. - scope classes are deprecated, but usage of scope as storage class is still legal (it is expected to be a no-op for now) Couldn't scope allocating a class on the stack just be considered an optimization that can be applied if the scope storage class become fully implemented? I think so. Scope classes were unsafe because of leaking references but if `scope` is actually implemented to assure safety it becomes perfectly valid thing to do. - scope storage class for function parameters is also accepted as no-op. For delegates it actually has a meaning. No deprecation. No-op just because it hasn't been implemented yet, right? Yes. And there is no clear definition either.
Re: Is it possible to convert a delegate into a function?
On Thursday, 6 February 2014 at 21:26:06 UTC, Gary Willoughby wrote: I know you can convert functions into delegates using: http://dlang.org/phobos/std_functional.html#.toDelegate but can you do this the other way around? I have a method that needs a function pointer but it would be nice to use a converted delegate pointer instead. A function is the address of a block of code in memory. A delegate also has the address to a block of code (a function) but it also has the pointer to an object that is to be passed as the first parameter to the function. It's pretty easy to drop a parameter, so writing a wrapper method that passes all the other parameters on to the target function is pretty easy. But unless the function that you're calling can accept a reference to the object that the delegate references and use that and the function pointer to reconstruct the delegate, you're definitely out of luck. Whether there is a means to manually construct a delegate or not, I'm not sure.
traits getOverload of a template method
How do i get aliases to overloads of a template method like
Class A
{
int a(T)(T tq,T tw);
int a(T)(T tq);
}
__traits(getOverloads, A, a(int))doesnt work
Re: Is it possible to convert a delegate into a function?
On Thursday, 6 February 2014 at 22:15:00 UTC, Dicebot wrote:
assert(!dg.ptr);
Note that Dicebot's version works only because he chose a
delegate that points to something that doesn't need an external
state. The below method, Foo.dump(), would fail the assert
because it requires a reference to an instance of Foo in order to
grab the state of a.value or b.value.
It does look like one is allowed to write into the delegate .ptr
and .funcptr variables, so reconstructing a delegate from parts
is possible. (I'm surprised that the definition of funcptr is
void function() instead of void function(Foo)?).
import std.stdio;
class Foo {
private:
int value;
public:
this(int v) {
value = v;
}
void dump() {
writeln(value);
}
}
void doDump(void* obj, void function() func) {
void delegate() dg;
dg.ptr = obj;
dg.funcptr = func;
dg();
}
void main() {
Foo a = new Foo(42);
Foo b = new Foo(255);
auto dgA = a.dump;
auto dgB = b.dump;
writefln(%x %x, dgA.funcptr, dgB.funcptr); // Same
void function() func = dgA.funcptr;
doDump(dgA.ptr, func);
doDump(dgB.ptr, func);
}
Re: Templates: generic return null;
Am Mon, 03 Feb 2014 10:25:17 +
schrieb Chris wend...@tcd.ie:
MyStruct(T) {
T[T] attributes;
//
public auto getAttribute(T attr) {
if (!(attr in attributes)) {
return null; // Doesn't work for numbers!
}
return attributes[attr];
}
}
void main() {
auto myStr = MyStruct!int(0); // Error
}
MyStruct(T) {
T[T] attributes;
//
public auto getAttribute(T attr) {
return attr in attributes;
}
}
There you go.
--
Marco
Re: Python calling D
On Tuesday, 4 February 2014 at 16:17:30 UTC, Russel Winder wrote: On Tue, 2014-02-04 at 12:45 +, Artem Tarasov wrote: But it does lead to a working system :-) Any particular reason you aren't using CeleriD to build this shared lib? CeleriD uses some hooks to call rt_init when the library loads. https://bitbucket.org/ariovistus/pyd/src/92b9962b429ed33afa7048cf1923fd76d0fe8977/infrastructure/d/?at=default see so_ctor.c and python_so_linux_boilerplate.d Also, note however you do it, if you have multiple shared libs, don't call initialize/term more than once. segfaults happen.
Re: 3d vector struct
Am Mon, 03 Feb 2014 22:01:14 + schrieb Stanislav Blinov stanislav.bli...@gmail.com: Return-by-value being optimized as a move might be one more reason why you would like to use slices instead of variables to store coordinates (since that would mean just moving a pointer and a size_t), but that might have to wait until custom allocators finally arrive. 3 doubles is only one machine word more than an array slice and there are no indirections, allocations and length attribute to deal with (which is always 3 here). -- Marco
Re: Performant method for reading huge text files
Am Tue, 04 Feb 2014 00:04:22 + schrieb Rene Zwanenburg renezwanenb...@gmail.com: On Monday, 3 February 2014 at 23:50:54 UTC, bearophile wrote: Rene Zwanenburg: The problem is speed. I'm using LockingTextReader in std.stdio, but it't not nearly fast enough. On my system it only reads about 3 MB/s with one core spending all it's time in IO calls. Are you reading the text by lines? In Bugzilla there is a byLineFast: https://d.puremagic.com/issues/show_bug.cgi?id=11810 Bye, bearophile Nope, I'm feeding it to csvReader which uses an input range of characters. Come to think of it.. Well this is embarassing, I've been sloppy with my profiling :). It appears the time is actually spent converting strings to doubles, done by csvReader to read a row into my Record struct. No way to speed that up I suppose. Still I find it surprising that parsing doubles is so slow. Parsing textual representations of numbers is slow. The other way around is faster. You have to check all kinds of stuff, like preceding +/-, starts with a dot, are all characters '0' to '9', is there an exponent? Is it NaN or nan? Floating point math is slow, but when you store the intermediate results while parsing inside an integer, you may run out of digits if the number string is long. On the other hand repeated floating point math will introduce some error as you append digits. Here is the ~400 lines version in Phobos: https://github.com/D-Programming-Language/phobos/blob/master/std/conv.d#L2250 -- Marco
App release build crashes
I was attempting to to build my app COMPO using a Makefile this morning, and at first this worked to some extent, but after some fiddling with compiler flags to make a release version that then had problems, I reverted to my original makefile as of yesterday. That builds and links the app, but when I run it I get Fatal Error while loading '/usr/lib/i386-linux-gnu/libphobos2.so.0.64': The module 'std.regex' is already defined in './compo'. My Makefile has: LFLAGS = -L-L/usr/lib/i386-linux-gnu -L-L/usr/local/lib -L-L/home/steve/COMPO -L-lusps4cb -L-lgtkd-2 -L-lrsvg -L-lphobos2 -L-ldl -L-lpthread -L-lm -L-lrt compo: $(OBJFILES) gcc -o compo $(OBJFILES) $(LFLAGS) If I use static phobos2 -L-l:libphobos2.a, it runs OK. Building via CodeBlocks using phobos2 shared library works OK judging by the smaller executable size. What is wrong with my linker flags?
