Re: A hash table implementation benchmark
On 10/01/2014 10:00 PM, thedeemon wrote: Here's another benchmark: D AAs vs. Vibe.d's open addressing hashes vs. Robin Hood hashing: http://www.infognition.com/blog/2014/on_robin_hood_hashing.html What a coincidence. :) Your blog article was written just two weeks ago. Ali
cgi.d - fastcgi - LightTPD is not cooperative
This is a cross-post from the issue i started on github (i figure i'll probably get a quicker response here) FastCGI dosen't cooperate with LightTPD on windows. as soon as lighttpd starts up and initializes fastcgi unknown listenType (0) is printed out and the server shuts down I've isolated that this is caulsed by the reciveloop on line 2675
Obedient threads
What is the best way to kill a thread when it pleases the owner
(main thread)? The background is playing audio files. The
playback happens in a separate thread so the owner can keep on
listening to events triggered by the user (like stop, pause). I
have to queue audio files, wait until one has finished, then play
the next etc. Communicating between threads does not work with
this technique:
Owner:
// create thread ...
...
auto isFinished = receiveOnly!bool();
Thread:
// play file ...
if (finished) {
ownerTid.send(true);
}
Of course, now Owner waits until it receives the message and is
deaf to any user input until Thread has finished, thus there is
no way to interrupt the playback. Is there a simple and elegant D
way to solve this? Slots are thread local so the observer doesn't
know what's going on in another thread, does it?
Re: Obedient threads
Just use non-blocking receives in main thread's event loop. When you get a message from child thread that it's finished playing and you decide you don't need that thread anymore, send a message to child you're dismissed. The child should also have some loop to check for incoming messages non-blockingly. Upon receiving such message it should exit.
Re: Obedient threads
On Thursday, 2 October 2014 at 10:33:02 UTC, thedeemon wrote: Just use non-blocking receives in main thread's event loop. When you get a message from child thread that it's finished playing and you decide you don't need that thread anymore, send a message to child you're dismissed. The child should also have some loop to check for incoming messages non-blockingly. Upon receiving such message it should exit. Thanks. I was thinking of something like that, only I haven't found a way to set up non-blocking receives. What am I missing. I'm sure it's something trivial.
Re: Obedient threads
On Thu, 02 Oct 2014 11:36:06 + Chris via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: Thanks. I was thinking of something like that, only I haven't found a way to set up non-blocking receives. What am I missing. I'm sure it's something trivial. you can use receiveTimeout! to check if there is some message available. signature.asc Description: PGP signature
Re: Find Semantically Correct Word Splits in UTF-8 Strings
On Wednesday, 1 October 2014 at 21:34:40 UTC, Nordlöw wrote: On Wednesday, 1 October 2014 at 17:09:57 UTC, monarch_dodra wrote: Does that even work? takeExactly would pop up to N *codepoints*, whereas your string only has N *codeunits*. Your're right again :) If forgot that takeExactly auto-decodes. Technically, it only pops. It's front/popFront that auto-decode.
Re: cgi.d - fastcgi - LightTPD is not cooperative
Try running the program yourself with a port argument yourprogram.exe --port 3000 for example, then have lighttpd configured to connect to that port for the application. That's what I had to do for nginx on Windows, lighttpd might be the same thing.
Re: Obedient threads
On Thursday, 2 October 2014 at 13:05:00 UTC, ketmar via Digitalmars-d-learn wrote: On Thu, 02 Oct 2014 11:36:06 + Chris via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: you can use receiveTimeout! to check if there is some message available. That won't do. It blocks the main thread too (for the duration of timeout), and it might abandon the thread too early. If you do it like in Ali's example[1], the main thread is blocked in the sense that it does not listen to input. [1] http://ddili.org/ders/d.en/concurrency.html
Re: cgi.d - fastcgi - LightTPD is not cooperative
On Thursday, 2 October 2014 at 13:25:14 UTC, Adam D. Ruppe wrote:
Try running the program yourself with a port argument
yourprogram.exe --port 3000
for example, then have lighttpd configured to connect to that
port for the application. That's what I had to do for nginx on
Windows, lighttpd might be the same thing.
Ok That Worked, but the GenericMain never runs
module main;
//import std.stdio;
import arsd.cgi;
void Req(Cgi c)
{
new File(C:/c/a.text).write(hello);
c.write(This is a test);
c.close();
}
mixin GenericMain!(Req);
when running in the browser the page just continues to load and
the lighttpd console shows no output from it and it dosent write
anything in the file.
Re: Obedient threads
On Thu, 02 Oct 2014 13:49:33 + Chris via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: That won't do. It blocks the main thread too (for the duration of timeout) don't do hour-long timeouts. 1ms timeout is actually poll-and-receive. can't see any problems with it. signature.asc Description: PGP signature
Re: cgi.d - fastcgi - LightTPD is not cooperative
On Thursday, 2 October 2014 at 13:53:48 UTC, Sycam_Inc wrote: when running in the browser the page just continues to load and the lighttpd console shows no output from it and it dosent write anything in the file. What url did you use in the browser and what's your lighttpd config look like? The program might be correct and the config is just off.
Re: cgi.d - fastcgi - LightTPD is not cooperative
On Thursday, 2 October 2014 at 14:17:42 UTC, Adam D. Ruppe wrote: On Thursday, 2 October 2014 at 13:53:48 UTC, Sycam_Inc wrote: when running in the browser the page just continues to load and the lighttpd console shows no output from it and it dosent write anything in the file. What url did you use in the browser and what's your lighttpd config look like? The program might be correct and the config is just off. Folders C:\c - server folder C:\c\cgi-bin - cgi folder C:\c\cgi-bin\test.exe - cgi file example url: loclahost\cgi-bin\test.exe Relevent Config: server.document-root = C:\c fastcgi.debug = 1 fastcgi.server = ( /cgi-bin/ = ( localhost = ( max-procs = 1, bin-path = server.document-root + \cgi-bin\Test.exe --port 9000 , host = 127.0.0.1, port = 9000 ) ) ) p.s. lighttpd exits on the event of a fast-cgi process not starting
How do I check if a function got CTFE?
I'd like to check if a function got CTFE, ie, the compiler was able to replace my foo(s); by the computed value at compile-time. I'm trying to convert the binary executable to assembly by using objconv tool but I'm finding it very diffucult to find anything in there, since some converters I've used which does ELF to ASM keep the function name, e.g, foo() function is a foo label somewhere in the file but this convert doesn't and use some numbers instead of. I don't know if it's related how is the windows object file format designed.
Re: How do I check if a function got CTFE?
On Thursday, 2 October 2014 at 17:56:29 UTC, AsmMan wrote: I'd like to check if a function got CTFE, ie, the compiler was able to replace my foo(s); by the computed value at compile-time. You have to explicitly force ctfe with context, it is never done automatically, and if it fails, the build will fail and you get a compile time error. So if you write enum f = foo(); or static f = foo(); or similar initializations and the build succeeds, you know it got ctfe'd. Otherwise, it wasn't.
Re: Obedient threads
On 10/02/2014 06:49 AM, Chris wrote:
On Thursday, 2 October 2014 at 13:05:00 UTC, ketmar via
Digitalmars-d-learn wrote:
On Thu, 02 Oct 2014 11:36:06 +
Chris via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote:
you can use receiveTimeout! to check if there is some message available.
That won't do. It blocks the main thread too (for the duration of
timeout), and it might abandon the thread too early. If you do it like
in Ali's example[1], the main thread is blocked in the sense that it
does not listen to input.
[1] http://ddili.org/ders/d.en/concurrency.html
To add to what ketmar said, even 0.msecs works.
In such a case moving the other tasks out of the main thread may be a
better option. main can safely block on the message queue while another
thread interacts with the outside world. It would be a cleaner event
loop that way.
I've just improved that example to interact with the user. The worker
produces a random message periodically. The user can ask for the most
recent message by entering yes. (I made it so that reading the message
also clears it.)
import std.stdio;
import std.concurrency;
import core.thread;
import std.string;
import std.random;
import std.array;
void workerFunc(size_t count, Duration duration)
{
writefln(There will be %s messages every %s., count, duration);
foreach (i; 0 .. count) {
Thread.sleep(duration);
ownerTid.send(format(message %s: %s, i, uniform(0, 100)));
}
writeln(workerFunc exiting);
}
struct ResultPlease
{}
void interactor()
{
bool done = false;
while (!done) {
write(Would you like to see the result? );
string response = readln.chomp;
if (response == yes) {
ownerTid.send(ResultPlease(), thisTid);
const result = receiveOnly!string();
if (result.empty) {
writeln(Sorry, no result yet.);
} else {
writefln(`The result is %s`, result);
}
} else {
writeln(Ok, no more interaction. Bye.);
done = true;
}
}
}
void main()
{
spawnLinked(workerFunc, 4, 5.seconds);
spawnLinked(interactor);
string result;
Tid[] completed;
while (completed.length 2) {
receive(
(string message) {
result = message;
},
(ResultPlease request, Tid requestor) {
requestor.send(result);
result = ;
},
(LinkTerminated e) {
completed ~= e.tid;
}
);
}
}
Ali
Re: How do I check if a function got CTFE?
On Thursday, 2 October 2014 at 17:56:29 UTC, AsmMan wrote: I'd like to check if a function got CTFE, ie, the compiler was able to replace my foo(s); by the computed value at compile-time. I'm trying to convert the binary executable to assembly by using objconv tool but I'm finding it very diffucult to find anything in there, since some converters I've used which does ELF to ASM keep the function name, e.g, foo() function is a foo label somewhere in the file but this convert doesn't and use some numbers instead of. I don't know if it's related how is the windows object file format designed. Sorry, can't help with you objconv, COFF, ELF, etc. A little something about terminology though: CTFE in the narrower sense refers to those occurrences of compile time evaluation that are specified to happen. Static initializers are CTFE-ed, for example. CTFE is not an optimization that a compiler may or may not do. The term for the optimization would be constant folding, I think. Your question, as I understand it, is about constant folding, not about CTFE.
Re: How do I check if a function got CTFE?
On Thursday, 2 October 2014 at 18:02:30 UTC, Adam D. Ruppe wrote: On Thursday, 2 October 2014 at 17:56:29 UTC, AsmMan wrote: I'd like to check if a function got CTFE, ie, the compiler was able to replace my foo(s); by the computed value at compile-time. You have to explicitly force ctfe with context, it is never done automatically, and if it fails, the build will fail and you get a compile time error. That's the point. I thought the compiler did it by checking things like constant arguments + function purity or so. This was exactly my issue. Thanks!
Re: How do I check if a function got CTFE?
On Thursday, 2 October 2014 at 18:17:12 UTC, anonymous wrote: On Thursday, 2 October 2014 at 17:56:29 UTC, AsmMan wrote: I'd like to check if a function got CTFE, ie, the compiler was able to replace my foo(s); by the computed value at compile-time. I'm trying to convert the binary executable to assembly by using objconv tool but I'm finding it very diffucult to find anything in there, since some converters I've used which does ELF to ASM keep the function name, e.g, foo() function is a foo label somewhere in the file but this convert doesn't and use some numbers I was thiking the dmd compiler did CTFE without someone ask for this, in the way as I've mentioned, checking for constant arguments + function's purity and if all this is true, it did the CTFE rather than generate code to compute it at run-time. In the case of it did happen, I just wanted to know. It was my misunderstsooding how it does works in dmd.
Re: How do I check if a function got CTFE?
On Thursday, 2 October 2014 at 18:42:56 UTC, AsmMan wrote: I was thiking the dmd compiler did CTFE without someone ask for this, in the way as I've mentioned, checking for constant arguments + function's purity and if all this is true, it did the CTFE rather than generate code to compute it at run-time. In the case of it did happen, I just wanted to know. It was my misunderstsooding how it does works in dmd. Yeah, that would be constant folding. The compiler is free to do that. It's just not called CTFE then.
Re: How do I check if a function got CTFE?
On Thu, Oct 02, 2014 at 07:04:56PM +, anonymous via Digitalmars-d-learn wrote: On Thursday, 2 October 2014 at 18:42:56 UTC, AsmMan wrote: I was thiking the dmd compiler did CTFE without someone ask for this, in the way as I've mentioned, checking for constant arguments + function's purity and if all this is true, it did the CTFE rather than generate code to compute it at run-time. In the case of it did happen, I just wanted to know. It was my misunderstsooding how it does works in dmd. Yeah, that would be constant folding. The compiler is free to do that. It's just not called CTFE then. CTFE grew out of constant-folding in dmd. In the dmd code, it is essentially still just a constant-folder, albeit a superpowered one. :-P T -- It is of the new things that men tire --- of fashions and proposals and improvements and change. It is the old things that startle and intoxicate. It is the old things that are young. -- G.K. Chesterton
Re: How do I check if a function got CTFE?
On Thursday, 2 October 2014 at 18:42:56 UTC, AsmMan wrote: I was thiking the dmd compiler did CTFE without someone ask for this, in the way as I've mentioned, checking for constant arguments + function's purity and if all this is true, it did the CTFE rather than generate code to compute it at run-time. In the case of it did happen, I just wanted to know. It was my misunderstsooding how it does works in dmd. A convenient way to force ctfe is eval: http://dlang.org/function.html (search for eval!) Though you'd change const for enum. Unfortunately, it's not in Phobos, but it should be! https://issues.dlang.org/show_bug.cgi?id=11811
Re: How do I check if a function got CTFE?
On Thursday, 2 October 2014 at 17:56:29 UTC, AsmMan wrote: I'd like to check if a function got CTFE, ie, the compiler was able to replace my foo(s); by the computed value at compile-time. I'm trying to convert the binary executable to assembly by using objconv tool but I'm finding it very diffucult to find anything in there, since some converters I've used which does ELF to ASM keep the function name, e.g, foo() function is a foo label somewhere in the file but this convert doesn't and use some numbers instead of. I don't know if it's related how is the windows object file format designed. You could use __ctfe http://forum.dlang.org/thread/yzioyjhiqedktswkw...@forum.dlang.org
Hunting down rogue memory allocations?
Say i have created a program written in D, what tools are available for me to track memory allocations? If you write a program and its performance is slow because you suspect too many allocations are taking place in unrecognised areas, what tools or techniques do you use to find where they are and eliminate them?
Re: Hunting down rogue memory allocations?
On Thursday, 2 October 2014 at 20:16:56 UTC, Gary Willoughby wrote: Say i have created a program written in D, what tools are available for me to track memory allocations? If you write a program and its performance is slow because you suspect too many allocations are taking place in unrecognised areas, what tools or techniques do you use to find where they are and eliminate them? If *time* spent by allocations is a problem, profile with `perf top` (assuming you have Linux): Look for 'gc', 'malloc', 'calloc', etc. (Plain perf record will also work, but not be as quick/interactive. CodeXL works too.) See https://perf.wiki.kernel.org/index.php/Tutorial - you probably need some specific arguments to get caller info, didn't use it for a while so I don't remember. If e.g. the GC is an issue, it should be immediately evident with some GC function taking e.g. over 5% of time. Usually the actual overhead will be much higher but divided into multiple smaller functions that will take little time individually. Drill down into one of these and look at its callers. Eliminate the most common source of calls, look again, repeat. Usually removing a source of alloc calls will result in better speedup than the profiler suggests. If *space* is a problem, Valgrind doesn't work with most D programs for some reason (probably D's fault), so, good luck with that. But eliminating the biggest time wasters usually helps space as well (or rather, it makes it more controllable as you know better where you allocate memory).
Re: Obedient threads
On Thursday, 2 October 2014 at 18:08:40 UTC, Ali Çehreli wrote:
On 10/02/2014 06:49 AM, Chris wrote:
On Thursday, 2 October 2014 at 13:05:00 UTC, ketmar via
Digitalmars-d-learn wrote:
On Thu, 02 Oct 2014 11:36:06 +
Chris via Digitalmars-d-learn
digitalmars-d-learn@puremagic.com wrote:
you can use receiveTimeout! to check if there is some message
available.
That won't do. It blocks the main thread too (for the duration
of
timeout), and it might abandon the thread too early. If you do
it like
in Ali's example[1], the main thread is blocked in the sense
that it
does not listen to input.
[1] http://ddili.org/ders/d.en/concurrency.html
To add to what ketmar said, even 0.msecs works.
In such a case moving the other tasks out of the main thread
may be a better option. main can safely block on the message
queue while another thread interacts with the outside world. It
would be a cleaner event loop that way.
I've just improved that example to interact with the user. The
worker produces a random message periodically. The user can ask
for the most recent message by entering yes. (I made it so
that reading the message also clears it.)
import std.stdio;
import std.concurrency;
import core.thread;
import std.string;
import std.random;
import std.array;
void workerFunc(size_t count, Duration duration)
{
writefln(There will be %s messages every %s., count,
duration);
foreach (i; 0 .. count) {
Thread.sleep(duration);
ownerTid.send(format(message %s: %s, i, uniform(0,
100)));
}
writeln(workerFunc exiting);
}
struct ResultPlease
{}
void interactor()
{
bool done = false;
while (!done) {
write(Would you like to see the result? );
string response = readln.chomp;
if (response == yes) {
ownerTid.send(ResultPlease(), thisTid);
const result = receiveOnly!string();
if (result.empty) {
writeln(Sorry, no result yet.);
} else {
writefln(`The result is %s`, result);
}
} else {
writeln(Ok, no more interaction. Bye.);
done = true;
}
}
}
void main()
{
spawnLinked(workerFunc, 4, 5.seconds);
spawnLinked(interactor);
string result;
Tid[] completed;
while (completed.length 2) {
receive(
(string message) {
result = message;
},
(ResultPlease request, Tid requestor) {
requestor.send(result);
result = ;
},
(LinkTerminated e) {
completed ~= e.tid;
}
);
}
}
Ali
Thanks Ali, you're a legend! I was actually thinking of creating
a separate thread as an event listener instead of using main.
I'll try that now. Somehow the 1.msecs solution doesn't seem
clean enough.
Re: Hunting down rogue memory allocations?
On Thursday, 2 October 2014 at 20:31:29 UTC, Kiith-Sa wrote: On Thursday, 2 October 2014 at 20:16:56 UTC, Gary Willoughby wrote: Say i have created a program written in D, what tools are available for me to track memory allocations? If you write a program and its performance is slow because you suspect too many allocations are taking place in unrecognised areas, what tools or techniques do you use to find where they are and eliminate them? If *time* spent by allocations is a problem, profile with `perf top` (assuming you have Linux): Look for 'gc', 'malloc', 'calloc', etc. (Plain perf record will also work, but not be as quick/interactive. CodeXL works too.) See https://perf.wiki.kernel.org/index.php/Tutorial - you probably need some specific arguments to get caller info, didn't use it for a while so I don't remember. If e.g. the GC is an issue, it should be immediately evident with some GC function taking e.g. over 5% of time. Usually the actual overhead will be much higher but divided into multiple smaller functions that will take little time individually. Drill down into one of these and look at its callers. Eliminate the most common source of calls, look again, repeat. Usually removing a source of alloc calls will result in better speedup than the profiler suggests. If *space* is a problem, Valgrind doesn't work with most D programs for some reason (probably D's fault), so, good luck with that. But eliminating the biggest time wasters usually helps space as well (or rather, it makes it more controllable as you know better where you allocate memory). Great thanks, I'll look into those.
Re: Obedient threads
On Thu, 02 Oct 2014 20:42:49 + Chris via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: I'll try that now. Somehow the 1.msecs solution doesn't seem clean enough. it seems that you want thread messaging to be integrated in your event loop. sorry, there is no easy way to do it now. maybe if libasync will become std.async we will have such feature in phobos out-of-the-box. signature.asc Description: PGP signature
curl and proxy
auto http = HTTP(dlang.org);
http.onReceive = (ubyte[] data)
{
writeln(cast(string) (data));
return data.length;
};
http.proxy = 192.168.111.111;
http.proxyPort = 1788;
WHAT HERE ?
http.perform();
//
how to make Сurl authorize on a proxy.
I specify proxyUser and proxyPassword?
