Re: Threading challenge: calculate fib(45) while spinning
On Friday, 15 October 2021 at 03:35:44 UTC, jfondren wrote:
The book, "The Go Programming Language" has this simple
goroutine example:
[...]
Here is a similar implementation using the concurrency library:
```d
import concurrency;
import concurrency.stream;
import concurrency.sender : justFrom;
import concurrency.operations : via, race;
import concurrency.thread : ThreadSender;
import core.time : msecs;
import std.stdio : writef, writefln, stdout;
import core.thread : Thread;
void main() @safe {
enum chars = `-\|/`;
auto spinner = infiniteStream(0)
.scan((int acc, int _) => acc + 1, 0)
.collect((int i) shared @trusted {
writef("\r%c", chars[i % chars.length]);
stdout.flush();
Thread.sleep(100.msecs);
})
.via(ThreadSender());
enum n = 45;
auto work = justFrom(() => fib(n));
auto result = race(spinner, work).syncWait.value;
writefln("\rFibonacci(%d) = %d", n, result.get);
}
int fib(int x) pure @safe @nogc {
if (x < 2)
return x;
return fib(x - 1) + fib(x - 2);
}
```
Go has language support so it is a bit unfair to compare it.
But this code will properly handle errors (in case `writef` or
`flush` throws), and as well as having an explicit
synchronization point so that the final `writeln` is always after
the spinner is done.
Re: Threading challenge: calculate fib(45) while spinning
On 10/15/21 10:01 AM, Ali Çehreli wrote: > writefln!"\rFibonacci(%d) = %d"(n, fibN); That '\r' bothered me because the actual work needs to know what the spinner is doing to clear its remaining character. I would expect the original go code had the same problem. -Steve
Re: Threading challenge: calculate fib(45) while spinning
On 10/14/21 8:54 PM, Ali Çehreli wrote:
>writefln!"\rFibonacci(%d) = %d"(n, fibN);
That '\r' bothered me because the actual work needs to know what the
spinner is doing to clear its remaining character.
>receiveTimeout(delay,
> (OwnerTerminated msg) {
And there is a race condition because the spinner can print an extra
character by the time it receives the OwnerTerminated message. (You can
observe this by adding e.g. Thread.sleep(300.msecs) after the
"\rFibonnacci..." line above.)
So, I improved it by removing both of those concerns as well as adding
the following:
- An optional message when spinning (it can be further improved because
there is an extra space character if the message is empty)
- A withSpinner() function to work with any delegate
The only requirement is that the delegate should not output to stdout if
we want a clean output.
import std.stdio : stdout, writef, writefln;
import std.concurrency : receiveTimeout, send, spawn;
import std.traits : ReturnType;
import core.thread : Duration, msecs, Thread;
import std.range : cycle, repeat, walkLength;
import std.format : format;
void main() {
enum n = 45;
int fibN; // Left mutable not to complicate the example
withSpinner({
fibN = fib(n); // slow
},
format!"Calculating fib(%s)"(n));
writefln!"Fibonacci(%d) = %d"(n, fibN);
}
// The delegate 'dg' should not output to stdout.
void withSpinner(Dg)(Dg dg,
string message = null,
Duration delay = 100.msecs) {
shared(bool) spinnerDone = false;
auto s = spawn(&spinner, message, delay, &spinnerDone);
// Do work while spinning
dg();
// Tell the spinner to stop (the value does not matter)
s.send(0x0FF);
// Busy(ish) wait until the spinner is done
while (!spinnerDone) {
Thread.yield();
}
}
void spinner(string message,
Duration delay,
shared(bool) * done) {
foreach (c; `-\|/`.cycle) {
if (receiveTimeout(delay, (int _) {})) {
// Stop request received
// Clear the spinning message
writef("\r%s \r", " ".repeat(walkLength(message)));
// Tell the user we are done
*done = true;
return;
}
writef!"\r%s %c"(message, c);
stdout.flush();
}
}
auto fib(int x) {
if (x < 2) {
return x;
}
return fib(x-1) + fib(x-2);
}
Ali
Re: Threading challenge: calculate fib(45) while spinning
On 10/14/21 11:35 PM, jfondren wrote:
The book, "The Go Programming Language" has this simple goroutine example:
```go
func main() {
go spinner(100 * time.Millisecond)
const n = 45
fibN := fib(n) // slow
fmt.Printf("\rFibonacci(%d) = %d\n", n, fibN)
}
func spinner(delay time.Duration) {
for {
for _, r := range `-\|/` {
fmt.Printf("\r%c", r)
time.Sleep(delay)
}
}
}
func fib(x int) int {
if x < 2 {
return x
}
return fib(x-1) + fib(x-2)
}
```
Attempt #1, with std.concurrency:
```d
import std.concurrency : spawn;
import core.thread : Thread;
import std.stdio : writefln, writef, stdout;
import std.datetime : msecs, Duration;
void main() @safe {
(() @trusted { spawn(&spinner, 100.msecs); })();
const n = 45;
const fibN = fib(n); // slow
writefln!"\rFibonacci(%d) = %d"(n, fibN);
}
void spinner(Duration delay) @safe {
(() @trusted { Thread.getThis.isDaemon(true); })();
while (true) {
foreach (char c; `-\|/`) {
writef!"\r%c"(c);
(() @trusted { stdout.flush; })();
(() @trusted { Thread.sleep(delay); })();
}
}
}
int fib(int x) pure @safe @nogc {
if (x < 2)
return x;
return fib(x - 1) + fib(x - 2);
}
```
This version has two problems:
1. a race condition with `isDaemon`: if `main()` ends before
`isDaemon(true)` is called, then the program never ends because the
kill-non-daemon-threads module destructor is called while the new thread
isn't a daemon thread.
You can also just spawn a thread directly with `Thread`, which I believe
allows you to set the daemon-ness from `main`.
2. it crashes about 10% of the time on exit (in dmd, gdc, and ldc).
valgrind on a gdc build complains about "Conditional jump or move
depends on uninitialised value(s)" early on.
The crash is likely because you are using D i/o utilities, and the
runtime is shut down. Technically it shouldn't cause a problem, but
possibly there are things that are needed deep inside `writef`.
If you switch to `printf`, it will probably work.
-Steve
Re: Threading challenge: calculate fib(45) while spinning
On Friday, 15 October 2021 at 03:54:17 UTC, Ali Çehreli wrote:
On 10/14/21 8:35 PM, jfondren wrote:
[...]
Here is one that uses receiveTimeout and OwnerTerminated:
import std.stdio;
import std.concurrency;
import core.thread;
void main() {
spawnLinked(&spinner, 100.msecs);
enum n = 45;
const fibN = fib(n); // slow
writefln!"\rFibonacci(%d) = %d"(n, fibN);
}
void spinner(const(Duration) delay) {
for (;;) {
foreach (r; `-\|/`) {
writef!"\r%c"(r);
stdout.flush();
bool done;
receiveTimeout(delay,
(OwnerTerminated msg) {
done = true;
});
if (done) {
return;
}
}
}
}
auto fib(int x) {
if (x < 2) {
return x;
}
return fib(x-1) + fib(x-2);
}
Ali
This is a "similar" approach to what Erlang does. I have always
liked it ☀️
Re: Threading challenge: calculate fib(45) while spinning
On 10/14/21 9:17 PM, jfondren wrote:
On Friday, 15 October 2021 at 03:54:17 UTC, Ali Çehreli wrote:
On 10/14/21 8:35 PM, jfondren wrote:
The book, "The Go Programming Language" has this simple goroutine
example:
Here is one that uses receiveTimeout and OwnerTerminated:
Very nice, replacing Thread.sleep with receiveTimeout and getting
graceful interruption for free. This also doesn't crash.
Cool. :)
Actually, it can be shorter by checking the return value of receiveTimeout:
if (receiveTimeout(delay, (OwnerTerminated msg) {})) {
return;
}
I didn't use this method earlier because I was afraid an unexpected
message might make receiveTimeout return 'true'. But I've tested just
now: Only the expected OwnerTerminated makes it return 'true'.
Ali
Re: Threading challenge: calculate fib(45) while spinning
On Friday, 15 October 2021 at 03:54:17 UTC, Ali Çehreli wrote: On 10/14/21 8:35 PM, jfondren wrote: The book, "The Go Programming Language" has this simple goroutine example: Here is one that uses receiveTimeout and OwnerTerminated: Very nice, replacing Thread.sleep with receiveTimeout and getting graceful interruption for free. This also doesn't crash.
Re: Threading challenge: calculate fib(45) while spinning
On 10/14/21 8:35 PM, jfondren wrote:
The book, "The Go Programming Language" has this simple goroutine example:
Here is one that uses receiveTimeout and OwnerTerminated:
import std.stdio;
import std.concurrency;
import core.thread;
void main() {
spawnLinked(&spinner, 100.msecs);
enum n = 45;
const fibN = fib(n); // slow
writefln!"\rFibonacci(%d) = %d"(n, fibN);
}
void spinner(const(Duration) delay) {
for (;;) {
foreach (r; `-\|/`) {
writef!"\r%c"(r);
stdout.flush();
bool done;
receiveTimeout(delay,
(OwnerTerminated msg) {
done = true;
});
if (done) {
return;
}
}
}
}
auto fib(int x) {
if (x < 2) {
return x;
}
return fib(x-1) + fib(x-2);
}
Ali
Re: Threading to prevent GUI Freeze
On 01/04/2016 06:31 AM, TheDGuy wrote:
> I tried it with "std.concurrency" like this:
>
> bool drawCallback(Scoped!Context cr, Widget widget){
> writeln("init");
> spawn(&render, cr, widget);
The first parameter to render() is Scoped!Context but render() takes a
Context:
> void render(Context cr, Widget widget){
Unless there is implicit conversion from Scoped!Context to Context, it
won't work. Perhaps you need to call an accessor like the following?
spawn(&render, cr.some_accessor(), widget);
Ali
Re: Threading to prevent GUI Freeze
On Monday, 4 January 2016 at 18:04:34 UTC, TheDGuy wrote: On Monday, 4 January 2016 at 17:33:28 UTC, Gerald wrote: On Monday, 4 January 2016 at 16:13:50 UTC, TheDGuy wrote: [...] Yes, you need it. The extern (C) function is what GDK invokes on idle. In any GUI application there is a lot of idle time waiting for events, what the addThreadIdle allows you to do is take advantage of this and tell GTK that whenever it's sitting around doing nothing, give this function a call. [...] Okay, thanks alot for your help. I think i will need some time to understand this but one last question: Do the errors come from the fact at i didn't use those GTK thread mechanisms or that my function is not "spawnable"? "std.concurrency.spawn(F, T...)(F fn, T args) if(isSpawnable!(F,T))" "Error: template std.concurrency.spawn cannot deduce function from argument types!()(void delegate(Context cr, Widget widget), Scoped Widget), candidates are:" Keep in mind I know nothing about Cairo and I don't have time to try your code, but what happens if you remove the Scoped template from the Context parameter? Also, have you checked if Cairo is thread-safe the way you are using it in the spawned function? I'm not sure if Cairo has the same restrictions that GTK widgets do.
Re: Threading to prevent GUI Freeze
On Monday, 4 January 2016 at 17:33:28 UTC, Gerald wrote: On Monday, 4 January 2016 at 16:13:50 UTC, TheDGuy wrote: [...] Yes, you need it. The extern (C) function is what GDK invokes on idle. In any GUI application there is a lot of idle time waiting for events, what the addThreadIdle allows you to do is take advantage of this and tell GTK that whenever it's sitting around doing nothing, give this function a call. [...] Okay, thanks alot for your help. I think i will need some time to understand this but one last question: Do the errors come from the fact at i didn't use those GTK thread mechanisms or that my function is not "spawnable"? "std.concurrency.spawn(F, T...)(F fn, T args) if(isSpawnable!(F,T))" "Error: template std.concurrency.spawn cannot deduce function from argument types!()(void delegate(Context cr, Widget widget), Scoped Widget), candidates are:"
Re: Threading to prevent GUI Freeze
On Monday, 4 January 2016 at 16:13:50 UTC, TheDGuy wrote: Thanks for your example code. Do i need those extern (C) function? Yes, you need it. The extern (C) function is what GDK invokes on idle. In any GUI application there is a lot of idle time waiting for events, what the addThreadIdle allows you to do is take advantage of this and tell GTK that whenever it's sitting around doing nothing, give this function a call. The idea is that you spawn a thread that does your work and when GTK invokes your thread idle callback where you can check the status of the thread and update the UI accordingly. For example, let's say you need to render a highly complicated graph that takes a few minutes to complete. You could spawn a thread that renders it into an off-line buffer of some sort and once completed sends a message to the GTK main thread using std.concurrency. At some point GTK calls your thread idle callback and you simply invoke the std.concurrency.receive and see a message saying the graph has been rendered is available, and if so, copy it into the appropriate GTK widget. The important thing to understand is that the thread idle callback happens in the GTK main thread so it is completely safe to update GTK widgets from here. The other thing to understand is that whatever you work you do in the callback must be short, if you don't return in a reasonable amount of time you are blocking the main GTK thread. As a result it really should only be used as a mechanism to track work progress in whatever threads you have spawned. The GTK thread idle callback works beautifully with D's std.concurrency send and receive mechanism. Note the code I pointed you to in that D github Issue abstracts the extern (C) function from you and allows you to use normal D delegates as callbacks. The issue was created to get this incorporated into GtkD as I agree the framework should abstract this. Why is it not possible to write the value to the TreeView in D? I don't understand what you mean as of course it's possible to update value's in a TreeView. Do you mean why am I updating it from the callback (i.e. the C function)? The code here is an artificial example where it is simply updating the treeview with an iterating number generated in a separate thread. The results being posted to the TreeView could just as easily be a resultset from a long running database query, a complicated mathematical expression, etc. Hopefully the previous explanation helps you understand what the callback is doing. You can see a more real world example of GtkD multi-threading in this application I wrote called Visual Grep (https://github.com/gnunn1/vgrep), it puts a GTK GUI around grep with all searches running in background threads and updating Gtk TreeView as results come in. It also uses the delegate code linked in that issue which originally came from an application called grestful. Note Visual Grep was my first ever D/GtkD program so there is some ugh code in there, but hopefully it can act as an additional source of info for you.
Re: Threading to prevent GUI Freeze
I wrote a demo for GtkD showing how multi-threading and D work together, it's in the demos/gtkD/DemoMultithread folder of GtkD, hopefully it will be helpful. However this example it is based on using the GTk threadIdle callback which is generally preferred over the locking methods you show above, obviously though your use case may vary but keep in mind the locking methods have been deprecated, see the GTK 3 reference manual here: https://developer.gnome.org/gdk3/stable/gdk3-Threads.html You also see this GtkD issue here for https://github.com/gtkd-developers/GtkD/issues/137 for some code on how to use Delgates with gdk_threads_add_idle (i.e. GtkD gdk.Threads.threadsAddIdle). Thanks for your example code. Do i need those extern (C) function? Why is it not possible to write the value to the TreeView in D?
Re: Threading to prevent GUI Freeze
On Monday, 4 January 2016 at 15:28:56 UTC, TheDGuy wrote: On Monday, 4 January 2016 at 15:07:12 UTC, Luis wrote: On Monday, 4 January 2016 at 14:31:04 UTC, TheDGuy wrote: [...] Before doing anything with threads and GTK, you should read this : http://blogs.operationaldynamics.com/andrew/software/gnome-desktop/gtk-thread-awareness Okay, so i have to do it like this on every function i use from GTKD? threadsInit(); threadsEnter(); GtkAllocation size; widget.getAllocation(size); threadsLeave(); I wrote a demo for GtkD showing how multi-threading and D work together, it's in the demos/gtkD/DemoMultithread folder of GtkD, hopefully it will be helpful. However this example it is based on using the GTk threadIdle callback which is generally preferred over the locking methods you show above, obviously though your use case may vary but keep in mind the locking methods have been deprecated, see the GTK 3 reference manual here: https://developer.gnome.org/gdk3/stable/gdk3-Threads.html You also see this GtkD issue here for https://github.com/gtkd-developers/GtkD/issues/137 for some code on how to use Delgates with gdk_threads_add_idle (i.e. GtkD gdk.Threads.threadsAddIdle).
Re: Threading to prevent GUI Freeze
On Monday, 4 January 2016 at 15:07:12 UTC, Luis wrote: On Monday, 4 January 2016 at 14:31:04 UTC, TheDGuy wrote: [...] Before doing anything with threads and GTK, you should read this : http://blogs.operationaldynamics.com/andrew/software/gnome-desktop/gtk-thread-awareness Okay, so i have to do it like this on every function i use from GTKD? threadsInit(); threadsEnter(); GtkAllocation size; widget.getAllocation(size); threadsLeave();
Re: Threading to prevent GUI Freeze
On Monday, 4 January 2016 at 14:31:04 UTC, TheDGuy wrote:
Hello,
i use GTKD to draw some stuff on a DrawingArea. Because it
needs some time to calculate i want to outsource those
calculation so that the GUI doesn't freeze.
I tried it with "std.concurrency" like this:
bool drawCallback(Scoped!Context cr, Widget widget){
writeln("init");
spawn(&render, cr, widget);
return true;
}
void render(Context cr, Widget widget){
Renderer renderer = new Renderer(new Vector3D(0,0,0), cr,
widget);
int i = 0;
while(i < 4){
renderer.renderOneStep();
i++;
}
renderer.DisplayResult();
}
But i get:
"std.concurrency.spawn(F, T...)(F fn, T args)
if(isSpawnable!(F,T))"
"Error: template std.concurrency.spawn cannot deduce function
from argument types!()(void delegate(Context cr, Widget
widget), Scoped Widget), candidates are:"
Before doing anything with threads and GTK, you should read this
:
http://blogs.operationaldynamics.com/andrew/software/gnome-desktop/gtk-thread-awareness
Re: Threading Questions
On Friday, 9 October 2015 at 04:04:42 UTC, bitwise wrote: Ah, I see. I thought you meant illegal meant it won't compile. Wouldn't it be more correct to say that it's undefined behaviour? I's probably not as undefined as in C case, i.e. it doesn't break safety guarantees, only the application's high-level business logic gets confused by data races.
Re: Threading Questions
On Thursday, 8 October 2015 at 20:42:46 UTC, Kagamin wrote: On Thursday, 8 October 2015 at 13:44:46 UTC, bitwise wrote: That still doesn't explain what you mean about it being illegal in other languages or why you brought up C# in the first place. Illegal means the resulting program behaves incorrectly, potentially leading to silent failures and data corruption. C# is a language that allows such bugs, and D disallows them - treats such code as invalid and rejects. Ah, I see. I thought you meant illegal meant it won't compile. Wouldn't it be more correct to say that it's undefined behaviour? Bit
Re: Threading Questions
On Thursday, 8 October 2015 at 13:44:46 UTC, bitwise wrote: That still doesn't explain what you mean about it being illegal in other languages or why you brought up C# in the first place. Illegal means the resulting program behaves incorrectly, potentially leading to silent failures and data corruption. C# is a language that allows such bugs, and D disallows them - treats such code as invalid and rejects.
Re: Threading Questions
On Thursday, 8 October 2015 at 10:11:38 UTC, Kagamin wrote:
On Thursday, 8 October 2015 at 02:31:24 UTC, bitwise wrote:
If you have System.Collections.Generic.List(T) static class
member, there is nothing wrong with using it from multiple
threads like this:
The equivalent of your D example would be
class Foo {
static List numbers = new List();
void bar() {
new Thread(()=>{
numbers.Add(1);
}).Start();
}
}
That still doesn't explain what you mean about it being illegal
in other languages or why you brought up C# in the first place.
Bit
Re: Threading Questions
On Thursday, 8 October 2015 at 02:31:24 UTC, bitwise wrote:
If you have System.Collections.Generic.List(T) static class
member, there is nothing wrong with using it from multiple
threads like this:
The equivalent of your D example would be
class Foo {
static List numbers = new List();
void bar() {
new Thread(()=>{
numbers.Add(1);
}).Start();
}
}
Re: Threading Questions
On Wednesday, 7 October 2015 at 09:09:36 UTC, Kagamin wrote:
On Sunday, 4 October 2015 at 04:24:55 UTC, bitwise wrote:
I use C#(garbage collected) for making apps/games, and while,
_in_theory_, the GC is supposed to protect you from leaks,
memory is not the only thing that can leak. Threads need to be
stopped, graphics resources need to be released, etc.
XNA doesn't manage graphics resources?
On Monday, 5 October 2015 at 17:40:24 UTC, bitwise wrote:
I'm not sure what's going to be done with shared, but I do
think it's annoying that you can't do this:
shared Array!int numbers;
someThread... {
numbers.clear(); // 'clear' is not shared
}
So this means that on top of the already ridiculous number of
attributes D has, now you have to mark everything as shared
too =/
That's illegal in other languages too except that they allow
you to do it. If you want concurrent collections, you must code
them separately:
https://msdn.microsoft.com/en-us/library/system.collections.concurrent%28v=vs.110%29.aspx
I'm not sure what you mean by illegal. AFAIK 'shared' is unique
to D. As far as simply locking and then accessing a global
variable(class static member) in C#, there is no problem doing
that from multiple threads.
If you have System.Collections.Generic.List(T) static class
member, there is nothing wrong with using it from multiple
threads like this:
class Foo {
static List numbers = new List();
void bar() {
new Thread(()=>{
lock(numbers) {
numbers.Add(1);
}).Start();
}
}
Bit
Re: Threading Questions
On Sunday, 4 October 2015 at 04:24:55 UTC, bitwise wrote:
I use C#(garbage collected) for making apps/games, and while,
_in_theory_, the GC is supposed to protect you from leaks,
memory is not the only thing that can leak. Threads need to be
stopped, graphics resources need to be released, etc.
XNA doesn't manage graphics resources?
On Monday, 5 October 2015 at 17:40:24 UTC, bitwise wrote:
I'm not sure what's going to be done with shared, but I do
think it's annoying that you can't do this:
shared Array!int numbers;
someThread... {
numbers.clear(); // 'clear' is not shared
}
So this means that on top of the already ridiculous number of
attributes D has, now you have to mark everything as shared too
=/
That's illegal in other languages too except that they allow you
to do it. If you want concurrent collections, you must code them
separately:
https://msdn.microsoft.com/en-us/library/system.collections.concurrent%28v=vs.110%29.aspx
Re: Threading Questions
On Monday, 5 October 2015 at 20:18:18 UTC, Laeeth Isharc wrote: On Monday, 5 October 2015 at 17:40:24 UTC, bitwise wrote: You may be right. I wrote a simple download manager in D using message passing. It was a little awkward at first, but in general, the spawn/send/receive API seems very intuitive. It feels awkward because the data you're working with is out of reach, but I guess it's safer that way. Any possibility of a blog post on your experience of doing so ? ;) [I should start writing some directly, but for time being, until I have my blog up and running again, I write from time to time on Quora]. A few minutes of writing now and then can have a remarkably big impact as well as clarifying your own thoughts, and the time invested is amply repaid, even viewed from a narrowly self-interested perspective. Unfortunately, my time is limited right now. I do have another project, which I've decided will either be finished or discarded by the dawn of 2016. So in the near future, I should have more time for other things. I had same experience with learning message passing. Feels like learning to eat with chopsticks in the beginning, but soon enough it feels much more civilised when it's the right tool for the job. I like the way my Worker class works because when I don't need the thread anymore, I can simply discard the object that represents the thread. As long as the Worker object is higher up on the stack than anything it's working on, all is well, and the concept of spawn/join is not visible while programming. This works out ok, because while the jobs I'm doing are slow enough to make a UI thread lag, they aren't long-running enough to where waiting for the Worker's thread to join in the destructor becomes a problem. There may be a small lag as the Worker's destructor waits for the last job to finish and the thread to join, but it's only happens once in the lifetime of the worker, so it's not a big deal. If care is not taken, the above could be subject to these problems: 1) shared memory corruption 2) worker accessing dead memory if it's placed on the stack below what it's working on 3) queueing a long running task could freeze the program on ~Worker() If you're moving or copying data into a thread, then returning the result(which can be ignored) I think most of the above can be solved. It's still a bit foreign to me though, and C++ has no such construct yet afaik. I read a bit about std::future and so on, but I'm not sure if they're standard yet. The biggest blocker though, is that the project I'm using that Worker class in is a Unity3D plugin. They only very recently updated their iOS libs to allow libc++ > 98 Bit
Re: Threading Questions
On 10/5/15 1:40 PM, bitwise wrote: On Monday, 5 October 2015 at 00:23:21 UTC, Jonathan M Davis wrote: On Sunday, October 04, 2015 14:42:48 bitwise via Digitalmars-d-learn wrote: Since D is moving towards a phobos with no GC, what will happen to things that are classes like Condition and Mutex? Phobos and druntime will always use the GC for some things, and some things just plain need classes. Rather, we're trying to make it so that Phobos does not use the GC when it doesn't need to use the GC as well reduce how much the GC is required for stuff like string processing where lazy ranges can be used instead in many cases. I was under the impression that the idea was to _completely_ eliminate the GC. It says in Andre's 2015H1 vision statement: "We aim to make the standard library usable in its entirety without a garbage collector." No, the plan is to allow the user to choose how he wants to allocate. Many pieces of phobos make the assumption that the GC is fair game. I think the plan is to make those pieces instead allocate on the stack and provide a mechanism to move that allocation to the GC (Walter's Dconf talk was about this), or accept an allocator to use as a mechanism for allocating memory. -Steve
Re: Threading Questions
On Monday, 5 October 2015 at 17:40:24 UTC, bitwise wrote: You may be right. I wrote a simple download manager in D using message passing. It was a little awkward at first, but in general, the spawn/send/receive API seems very intuitive. It feels awkward because the data you're working with is out of reach, but I guess it's safer that way. Any possibility of a blog post on your experience of doing so ? ;) [I should start writing some directly, but for time being, until I have my blog up and running again, I write from time to time on Quora]. A few minutes of writing now and then can have a remarkably big impact as well as clarifying your own thoughts, and the time invested is amply repaid, even viewed from a narrowly self-interested perspective. I had same experience with learning message passing. Feels like learning to eat with chopsticks in the beginning, but soon enough it feels much more civilised when it's the right tool for the job.
Re: Threading Questions
On Monday, 5 October 2015 at 00:23:21 UTC, Jonathan M Davis wrote:
On Sunday, October 04, 2015 14:42:48 bitwise via
Digitalmars-d-learn wrote:
Since D is moving towards a phobos with no GC, what will
happen to things that are classes like Condition and Mutex?
Phobos and druntime will always use the GC for some things, and
some things just plain need classes. Rather, we're trying to
make it so that Phobos does not use the GC when it doesn't need
to use the GC as well reduce how much the GC is required for
stuff like string processing where lazy ranges can be used
instead in many cases.
I was under the impression that the idea was to _completely_
eliminate the GC. It says in Andre's 2015H1 vision statement:
"We aim to make the standard library usable in its entirety
without a garbage collector."
I understand the allocation/freeing of memory is expensive, but I
thought the actual sweep of the GC was a problem too, and that
disabling the GC to avoid the sweep was the plan for some people.
I don't know how long D's GC takes to sweep, but even a 5ms pause
would be unacceptable for a performance intensive game.
I guess if you use @nogc properly though, you could still safely
turn off the GC, right?
As for Condition and Mutex specifically, I don't know whey they
were ever classes except perhaps to take advantage of the
monitor in Object. Maybe they'll get changed to structs, maybe
they won't, but most D code is thread-local, and most of the
code that isn't is going to use message passing, which means
that explicit mutexes and conditions are unnecessary. So, most
code won't be impacted regardless of what we do with Condition
and Mutex.
You may be right. I wrote a simple download manager in D using
message passing. It was a little awkward at first, but in
general, the spawn/send/receive API seems very intuitive. It
feels awkward because the data you're working with is out of
reach, but I guess it's safer that way.
Regardless, I doubt that anything will be done with Condition
or Mutex until shared is revisted, which is supposed to happen
sometime soon but hasn't happened yet. What happens with shared
could completely change how Condition and Mutex are handled
(e.g. they don't support shared directly even though they
should probably have most of their members marked with shared,
because Sean Kelly didn't want to be doing anything with shared
that he'd have to change later).
- Jonathan M Davis
I'm not sure what's going to be done with shared, but I do think
it's annoying that you can't do this:
shared Array!int numbers;
someThread... {
numbers.clear(); // 'clear' is not shared
}
So this means that on top of the already ridiculous number of
attributes D has, now you have to mark everything as shared too =/
Bit
Re: Threading Questions
On Sunday, October 04, 2015 14:42:48 bitwise via Digitalmars-d-learn wrote: > Since D is moving towards a phobos with no GC, what will happen > to things that are classes like Condition and Mutex? Phobos and druntime will always use the GC for some things, and some things just plain need classes. Rather, we're trying to make it so that Phobos does not use the GC when it doesn't need to use the GC as well reduce how much the GC is required for stuff like string processing where lazy ranges can be used instead in many cases. As for Condition and Mutex specifically, I don't know whey they were ever classes except perhaps to take advantage of the monitor in Object. Maybe they'll get changed to structs, maybe they won't, but most D code is thread-local, and most of the code that isn't is going to use message passing, which means that explicit mutexes and conditions are unnecessary. So, most code won't be impacted regardless of what we do with Condition and Mutex. Regardless, I doubt that anything will be done with Condition or Mutex until shared is revisted, which is supposed to happen sometime soon but hasn't happened yet. What happens with shared could completely change how Condition and Mutex are handled (e.g. they don't support shared directly even though they should probably have most of their members marked with shared, because Sean Kelly didn't want to be doing anything with shared that he'd have to change later). - Jonathan M Davis
Re: Threading Questions
On Wednesday, 30 September 2015 at 10:32:01 UTC, Jonathan M Davis wrote: On Tuesday, September 29, 2015 22:38:42 Johannes Pfau via Digitalmars-d-learn wrote: [...] What I took from the answers to that SO question was that in general, it really doesn't matter whether a condition variable has spurious wakeups. You're going to have to check that the associated bool is true when you wake up anyway. Maybe without spurious wakeups, it wouldn't be required if only one thread was waiting for the signal, but you'd almost certainly still need an associated bool in case it becomes true prior to waiting. In addition, if you want to avoid locking up your program, it's ferquently the case that you want a timed wait so that you can check whether the program is trying to exit (or at least that the thread in question is being terminated), and you'd need a separate bool in that case as well so that you can check whether the condition has actually been signaled. So, ultimately, while spurious wakeups do seem wrong from a correctness perspective, when you look at what a condition variable needs to do, it usually doesn't matter that spurious wakeups exist, and a correctly used condition variable will just handle spurious wakeups as a side effect of how it's used. - Jonathan M Davis Yea, I guess you're right. The class in the example I posted was a crude reproduction of something I'm using right now in another project: http://codepad.org/M4fVyiXf I don't think it would make a difference whether it woke up randomly or not. I've been using this code regularly with no problems. Bit
Re: Threading Questions
On Tuesday, 29 September 2015 at 23:20:31 UTC, Steven
Schveighoffer wrote:
yeah, that could probably be done. One thing to note is that
these classes are from ages ago (probably close to 10 years).
New API suggestions may be allowed.
-Steve
I'm still thinking about my last rant, here... So by new API, do
you mean just adding a couple of new functions, or rewriting a
new Condition class(as is the plan for streams)?
Since D is moving towards a phobos with no GC, what will happen
to things that are classes like Condition and Mutex?
If DIP74 were implemented, Condition and Mutex could be made ref
counted, but DIP74 seems like something that will be very
complicated, and may not happen for a long time.
So the only other alternative is to make it a struct, but for a
Mutex, that would prevent you from doing this:
Mutex m = new Mutex();
synchronized(m) { }
I also don't mind the way that the current streams are made up of
a class hierarchy.
Although inheritance is overused sometimes, I don't think it's
bad. But, if I'm correct about the current trend in D, it seems
any new stream stuff will end up getting flattened into some
template/struct solution.
Any comments on this?
Thanks,
Bit
Re: Threading Questions
On Tuesday, 29 September 2015 at 19:10:58 UTC, Steven Schveighoffer wrote: An object that implements the Monitor interface may not actually be a mutex. For example, a pthread_cond_t requires a pthread_mutex_t to operate properly. Right! I feel like I should have caught the fact that ConditionVariable still has to use pthread_cond_t under the hood, and adopts all of it's behaviour and requirements as a result. 4. Technically, you shouldn't access member variables that are GC allocated from a dtor. I know it's a struct, but structs can be GC allocated as well. Right forgot about that. GC's are really beginning to get on my nerves.. IMO, RAII for GC is a horrible tradeoff. I'm still not sure I would like Rust, but their memory model is making it a very enticing proposition. I'm almost at the point where I just don't care how much convenience, or familiarity D can offer in other areas.. Its starting to seem like none of it is worth it with a GC-based memory model standing in the way. Maybe this is an exageration...D has a lot of great features..but it's the net benefit that will ultimately determine whether or not people use D. I use C#(garbage collected) for making apps/games, and while, _in_theory_, the GC is supposed to protect you from leaks, memory is not the only thing that can leak. Threads need to be stopped, graphics resources need to be released, etc.. So when I can't rely on RAII to free these things, I need to free them explicitly, which basically puts me right back where I started. Anyways, I realize this will probably be buried 3 pages deep in D-Learn by Monday, but at least I feel better :) Bit
Re: Threading Questions
On Friday, 25 September 2015 at 15:19:27 UTC, bitwise wrote: I know that all global variables are TLS unless explicitly marked as 'shared', but someone once told me something about 'shared' affecting member variables in that accessing them from a separate thread would return T.init instead of the actual value... or something like that. This seems to be wrong(thankfully). T.init is returned for TLS variable when accessed from a thread for which it wasn't initialized.
Re: Threading Questions
On Tuesday, September 29, 2015 22:38:42 Johannes Pfau via Digitalmars-d-learn
wrote:
> Am Tue, 29 Sep 2015 15:10:58 -0400
> schrieb Steven Schveighoffer :
>
> >
> > > 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just
> > > pass an "Object"?
> >
> > An object that implements the Monitor interface may not actually be a
> > mutex. For example, a pthread_cond_t requires a pthread_mutex_t to
> > operate properly. If you passed it anything that can act like a lock,
> > it won't work. So the Condition needs to know that it has an actual
> > Mutex, not just any lock-like object.
> >
> > I think I advocated in the past to Sean that Condition should provide
> > a default ctor that just constructs a mutex, but it doesn't look like
> > that was done.
> >
>
> But you'll need access to the Mutex in user code as well. And often you
> use multiple Conditions with one Mutex so a Condition doesn't really
> own the Mutex.
>
> > >
> > > 4) Will D's Condition ever experience spurious wakeups?
> >
> > What do you mean by "spurious"? If you notify a condition, anything
> > that is waiting on it can be woken up. Since the condition itself is
> > user defined, there is no way for the actual Condition to verify you
> > will only be woken up when it is satisfied.
> >
> > In terms of whether a condition could be woken when notify *isn't*
> > called, I suppose it's possible (perhaps interrupted by a signal?).
> > But I don't know why it would matter -- per above you should already
> > be checking the condition while within the lock.
>
> Spurious wakeup is a common term when talking about posix conditions
> and it does indeed mean a wait() call can return without ever calling
> notify():
> https://en.wikipedia.org/wiki/Spurious_wakeup
> http://stackoverflow.com/questions/8594591/why-does-pthread-cond-wait-have-spurious-wakeups
>
> And yes, this does happen for core.sync.condition as well. As a result
> you'll always have to check in a loop:
>
> synchronized(mutex)
> {
> while(some_flag_or_expression)
> {
> cond.wait();
> }
> }
>
> -
> synchronized(mutex)
> {
> some_flag_or_expression = true;
> cond.notify();
> }
What I took from the answers to that SO question was that in general, it
really doesn't matter whether a condition variable has spurious wakeups.
You're going to have to check that the associated bool is true when you wake
up anyway. Maybe without spurious wakeups, it wouldn't be required if only
one thread was waiting for the signal, but you'd almost certainly still need
an associated bool in case it becomes true prior to waiting. In addition, if
you want to avoid locking up your program, it's ferquently the case that you
want a timed wait so that you can check whether the program is trying to
exit (or at least that the thread in question is being terminated), and
you'd need a separate bool in that case as well so that you can check
whether the condition has actually been signaled. So, ultimately, while
spurious wakeups do seem wrong from a correctness perspective, when you look
at what a condition variable needs to do, it usually doesn't matter that
spurious wakeups exist, and a correctly used condition variable will just
handle spurious wakeups as a side effect of how it's used.
- Jonathan M Davis
Re: Threading Questions
On 9/29/15 4:38 PM, Johannes Pfau wrote: Am Tue, 29 Sep 2015 15:10:58 -0400 schrieb Steven Schveighoffer : 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just pass an "Object"? An object that implements the Monitor interface may not actually be a mutex. For example, a pthread_cond_t requires a pthread_mutex_t to operate properly. If you passed it anything that can act like a lock, it won't work. So the Condition needs to know that it has an actual Mutex, not just any lock-like object. I think I advocated in the past to Sean that Condition should provide a default ctor that just constructs a mutex, but it doesn't look like that was done. But you'll need access to the Mutex in user code as well. synchronized(condition.mutex) And often you use multiple Conditions with one Mutex so a Condition doesn't really own the Mutex. It's just a different option. Often times, you have a condition variable, and a mutex variable. It's not super-important, you can always do: new Condition(new Mutex); 4) Will D's Condition ever experience spurious wakeups? What do you mean by "spurious"? If you notify a condition, anything that is waiting on it can be woken up. Since the condition itself is user defined, there is no way for the actual Condition to verify you will only be woken up when it is satisfied. In terms of whether a condition could be woken when notify *isn't* called, I suppose it's possible (perhaps interrupted by a signal?). But I don't know why it would matter -- per above you should already be checking the condition while within the lock. Spurious wakeup is a common term when talking about posix conditions and it does indeed mean a wait() call can return without ever calling notify(): https://en.wikipedia.org/wiki/Spurious_wakeup http://stackoverflow.com/questions/8594591/why-does-pthread-cond-wait-have-spurious-wakeups OK thanks. 5) Why doesn't D's Condition.wait take a predicate? I assume this is because the answer to (4) is no. The actual "condition" that you are waiting on is up to you to check/define. He probably means that you could pass an expression to wait and wait would do the looping / check internally. That's probably a nicer API but not implemented. yeah, that could probably be done. One thing to note is that these classes are from ages ago (probably close to 10 years). New API suggestions may be allowed. I just wanted to stress that there isn't some sort of built-in condition predicate (like a boolean). -Steve
Re: Threading Questions
Am Tue, 29 Sep 2015 15:10:58 -0400
schrieb Steven Schveighoffer :
>
> > 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just
> > pass an "Object"?
>
> An object that implements the Monitor interface may not actually be a
> mutex. For example, a pthread_cond_t requires a pthread_mutex_t to
> operate properly. If you passed it anything that can act like a lock,
> it won't work. So the Condition needs to know that it has an actual
> Mutex, not just any lock-like object.
>
> I think I advocated in the past to Sean that Condition should provide
> a default ctor that just constructs a mutex, but it doesn't look like
> that was done.
>
But you'll need access to the Mutex in user code as well. And often you
use multiple Conditions with one Mutex so a Condition doesn't really
own the Mutex.
> >
> > 4) Will D's Condition ever experience spurious wakeups?
>
> What do you mean by "spurious"? If you notify a condition, anything
> that is waiting on it can be woken up. Since the condition itself is
> user defined, there is no way for the actual Condition to verify you
> will only be woken up when it is satisfied.
>
> In terms of whether a condition could be woken when notify *isn't*
> called, I suppose it's possible (perhaps interrupted by a signal?).
> But I don't know why it would matter -- per above you should already
> be checking the condition while within the lock.
Spurious wakeup is a common term when talking about posix conditions
and it does indeed mean a wait() call can return without ever calling
notify():
https://en.wikipedia.org/wiki/Spurious_wakeup
http://stackoverflow.com/questions/8594591/why-does-pthread-cond-wait-have-spurious-wakeups
And yes, this does happen for core.sync.condition as well. As a result
you'll always have to check in a loop:
synchronized(mutex)
{
while(some_flag_or_expression)
{
cond.wait();
}
}
-
synchronized(mutex)
{
some_flag_or_expression = true;
cond.notify();
}
>
> I think there are cases with multiple threads where you can
> potentially wake up the thread waiting on a condition AFTER the
> condition was already reset by another.
>
> > 5) Why doesn't D's Condition.wait take a predicate? I assume this is
> > because the answer to (4) is no.
>
> The actual "condition" that you are waiting on is up to you to
> check/define.
>
He probably means that you could pass an expression to wait and wait
would do the looping / check internally. That's probably a nicer API
but not implemented.
> > 6) Does 'shared' actually have any effect on non-global variables
> > beside the syntactic regulations?
>
> I believe shared doesn't alter code generation at all. It only
> prevents certain things and affects the type.
>
It shouldn't. I think in GDC it does generate different code, but that's
an implementation detail that needs to be fixed.
Re: Threading Questions
On 9/25/15 11:19 AM, bitwise wrote:
Hey, I've got a few questions if anybody's got a minute.
I'm trying to wrap my head around the threading situation in D. So far,
things seem to be working as expected, but I want to verify my solutions.
1) Are the following two snippets exactly equivalent(not just in
observable behaviour)?
a)
Mutex mut;
mut.lock();
scope(exit) mut.unlock();
b)
Mutex mut;
synchronized(mut) { }
Will 'synchronized' call 'lock' on the Mutex, or do something
else(possibly related to the interface Object.Monitor)?
Yes. A mutex object has it's internal lock as its monitor.
2) Phobos has 'Condition' which takes a Mutex in the constructor. The
documentation doesn't exactly specify this, but should I assume it works
the same as std::condition_variable in C++?
I am not sure about std::condition_variable. core.sync.condition works
like a standard condition
(https://en.wikipedia.org/wiki/Monitor_%28synchronization%29)
For example, is this correct?
Mutex mut;
Condition cond = new Condition(mut);
// mut must be locked before calling Condition.wait
synchronized(mut) // depends on answer to (1)
{
// wait() unlocks the mutex and enters wait state
// wait() must re-acquire the mutex before returning when cond is
signalled
cond.wait();
}
Yes, I believe it is.
3) Why do I have to pass a "Mutex" to "Condition"? Why can't I just pass
an "Object"?
An object that implements the Monitor interface may not actually be a
mutex. For example, a pthread_cond_t requires a pthread_mutex_t to
operate properly. If you passed it anything that can act like a lock, it
won't work. So the Condition needs to know that it has an actual Mutex,
not just any lock-like object.
I think I advocated in the past to Sean that Condition should provide a
default ctor that just constructs a mutex, but it doesn't look like that
was done.
4) Will D's Condition ever experience spurious wakeups?
What do you mean by "spurious"? If you notify a condition, anything that
is waiting on it can be woken up. Since the condition itself is user
defined, there is no way for the actual Condition to verify you will
only be woken up when it is satisfied.
In terms of whether a condition could be woken when notify *isn't*
called, I suppose it's possible (perhaps interrupted by a signal?). But
I don't know why it would matter -- per above you should already be
checking the condition while within the lock.
I think there are cases with multiple threads where you can potentially
wake up the thread waiting on a condition AFTER the condition was
already reset by another.
5) Why doesn't D's Condition.wait take a predicate? I assume this is
because the answer to (4) is no.
The actual "condition" that you are waiting on is up to you to check/define.
6) Does 'shared' actually have any effect on non-global variables beside
the syntactic regulations?
I believe shared doesn't alter code generation at all. It only prevents
certain things and affects the type.
I know that all global variables are TLS unless explicitly marked as
'shared', but someone once told me something about 'shared' affecting
member variables in that accessing them from a separate thread would
return T.init instead of the actual value... or something like that.
This seems to be wrong(thankfully).
No, this isn't true.
For example, I have created this simple Worker class which seems to work
fine without a 'shared' keyword in sight(thankfully). I'm wondering
though, if there would be any unexpected consequences of doing things
this way.
http://dpaste.com/2ZG2QZV
Some errors:
1. When calling notifyAll, you should ALWAYS have the mutex locked.
2. Since the mutex is protecting _run, it should only be
checked/modified with the lock held.
3. After you have been woken up, you should check that the condition is
satisfied.
4. Technically, you shouldn't access member variables that are GC
allocated from a dtor. I know it's a struct, but structs can be GC
allocated as well.
I would replace your if(tasks.empty) with while(tasks.empty && _run) to
fix issue 3.
-Steve
Re: Threading Questions
On Tue, 2015-09-29 at 03:05 +, bitwise via Digitalmars-d-learn wrote: > On Monday, 28 September 2015 at 11:47:38 UTC, Russel Winder wrote: > > I hadn't answered as I do not have answers to the questions you > > ask. My reason: people should not be doing their codes using > > these low-level shared memory techniques. Data parallel things > > should be using the std.parallelism module. Dataflow-style > > things should be using spawn and channels – akin to the way you > > do things in Go. > > > > So to give you an answer I would go back a stage, forget > > threads, mutexes, synchronized, etc. and ask what do you want > > you workers to do? If they are to do something and return a > > result then spawn and channel is exactly the right abstraction > > to use. Think "farmer–worker", the farmer spawns the workers > > and then collects their results. No shared memory anywyere – at > > least not mutable. > > https://www.youtube.com/watch?v=S7pGs7JU7eM > > Bit What's the tl;dr as text, I very, very rarely watch videos. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: Threading Questions
On Monday, 28 September 2015 at 11:47:38 UTC, Russel Winder wrote: I hadn't answered as I do not have answers to the questions you ask. My reason: people should not be doing their codes using these low-level shared memory techniques. Data parallel things should be using the std.parallelism module. Dataflow-style things should be using spawn and channels – akin to the way you do things in Go. So to give you an answer I would go back a stage, forget threads, mutexes, synchronized, etc. and ask what do you want you workers to do? If they are to do something and return a result then spawn and channel is exactly the right abstraction to use. Think "farmer–worker", the farmer spawns the workers and then collects their results. No shared memory anywyere – at least not mutable. https://www.youtube.com/watch?v=S7pGs7JU7eM Bit
Re: Threading Questions
I hadn't answered as I do not have answers to the questions you ask. My
reason: people should not be doing their codes using these low-level
shared memory techniques. Data parallel things should be using the
std.parallelism module. Dataflow-style things should be using spawn and
channels – akin to the way you do things in Go.
So to give you an answer I would go back a stage, forget threads,
mutexes, synchronized, etc. and ask what do you want you workers to do?
If they are to do something and return a result then spawn and channel
is exactly the right abstraction to use. Think "farmer–worker", the
farmer spawns the workers and then collects their results. No shared
memory anywyere – at least not mutable.
On Fri, 2015-09-25 at 15:19 +, bitwise via Digitalmars-d-learn
wrote:
> Hey, I've got a few questions if anybody's got a minute.
>
> I'm trying to wrap my head around the threading situation in D.
> So far, things seem to be working as expected, but I want to
> verify my solutions.
>
> 1) Are the following two snippets exactly equivalent(not just in
> observable behaviour)?
> a)
>
> Mutex mut;
> mut.lock();
> scope(exit) mut.unlock();
>
> b)
> Mutex mut;
> synchronized(mut) { }
>
> Will 'synchronized' call 'lock' on the Mutex, or do something
> else(possibly related to the interface Object.Monitor)?
>
> 2) Phobos has 'Condition' which takes a Mutex in the constructor.
> The documentation doesn't exactly specify this, but should I
> assume it works the same as std::condition_variable in C++?
>
> For example, is this correct?
>
> Mutex mut;
> Condition cond = new Condition(mut);
>
> // mut must be locked before calling Condition.wait
> synchronized(mut) // depends on answer to (1)
> {
> // wait() unlocks the mutex and enters wait state
> // wait() must re-acquire the mutex before returning when
> cond is signalled
> cond.wait();
> }
>
> 3) Why do I have to pass a "Mutex" to "Condition"? Why can't I
> just pass an "Object"?
>
> 4) Will D's Condition ever experience spurious wakeups?
>
> 5) Why doesn't D's Condition.wait take a predicate? I assume this
> is because the answer to (4) is no.
>
> 6) Does 'shared' actually have any effect on non-global variables
> beside the syntactic regulations?
>
> I know that all global variables are TLS unless explicitly marked
> as 'shared', but someone once told me something about 'shared'
> affecting member variables in that accessing them from a separate
> thread would return T.init instead of the actual value... or
> something like that. This seems to be wrong(thankfully).
>
> For example, I have created this simple Worker class which seems
> to work fine without a 'shared' keyword in sight(thankfully). I'm
> wondering though, if there would be any unexpected consequences
> of doing things this way.
>
> http://dpaste.com/2ZG2QZV
>
>
>
>
> Thanks!
> Bit
--
Russel.
=
Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net
41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk
London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
signature.asc
Description: This is a digitally signed message part
Re: Threading Questions
Sorry I don't know the answers but these questions are
interesting so BUMP ;)
On Friday, 25 September 2015 at 15:19:27 UTC, bitwise wrote:
1) Are the following two snippets exactly equivalent(not just
in observable behaviour)?
a)
Mutex mut;
mut.lock();
scope(exit) mut.unlock();
b)
Mutex mut;
synchronized(mut) { }
Will 'synchronized' call 'lock' on the Mutex, or do something
else(possibly related to the interface Object.Monitor)?
Don't know.
Is this Object monitor a mutex or something else?
6) Does 'shared' actually have any effect on non-global
variables beside the syntactic regulations?
Don't think so.
Re: Threading Questions
Pretty please? :)
Re: threading issues with D -> C -> Python
On 12/07/2014 03:12 PM, Michael wrote: now to figure out how to use them in the general case. This is great.. Thank you. I'm looking forward to being able to try the finished result. My build servers are broken at the moment, but I think I have this fixed, on linux at least.
Re: threading issues with D -> C -> Python
On Monday, 8 December 2014 at 01:17:16 UTC, Ellery Newcomer wrote: On 12/07/2014 03:12 PM, Michael wrote: On Saturday, 6 December 2014 at 00:40:49 UTC, Ellery Newcomer wrote: On 12/04/2014 10:55 PM, Ellery Newcomer wrote: I guess tomorrow I can try messing around with thread_attachThis, as the fullcollect happening in #2 might be screwing with python data. But you aren't really passing anything from python to d or vice versa, so I'm not sure why the gc would need to know about the python threads. by gum, thread_attachThis and thread_detachThis fix the issue! now to figure out how to use them in the general case. This is great.. Thank you. I'm looking forward to being able to try the finished result. It would be great if there were some convenient hook in python to stick these calls. Near as I can tell, there isn't. That leaves you with either explicitly calling attach and detach with an exposed api, or pyd obsessively checking whether the current thread is registered. Actually, I suppose with a thread local flag the latter wouldn't be too bad. Mind if I incorporate your example into pyd's test suite? Not at all. Go for it.
Re: threading issues with D -> C -> Python
On 12/07/2014 03:12 PM, Michael wrote: On Saturday, 6 December 2014 at 00:40:49 UTC, Ellery Newcomer wrote: On 12/04/2014 10:55 PM, Ellery Newcomer wrote: I guess tomorrow I can try messing around with thread_attachThis, as the fullcollect happening in #2 might be screwing with python data. But you aren't really passing anything from python to d or vice versa, so I'm not sure why the gc would need to know about the python threads. by gum, thread_attachThis and thread_detachThis fix the issue! now to figure out how to use them in the general case. This is great.. Thank you. I'm looking forward to being able to try the finished result. It would be great if there were some convenient hook in python to stick these calls. Near as I can tell, there isn't. That leaves you with either explicitly calling attach and detach with an exposed api, or pyd obsessively checking whether the current thread is registered. Actually, I suppose with a thread local flag the latter wouldn't be too bad. Mind if I incorporate your example into pyd's test suite?
Re: threading issues with D -> C -> Python
On Saturday, 6 December 2014 at 00:40:49 UTC, Ellery Newcomer wrote: On 12/04/2014 10:55 PM, Ellery Newcomer wrote: I guess tomorrow I can try messing around with thread_attachThis, as the fullcollect happening in #2 might be screwing with python data. But you aren't really passing anything from python to d or vice versa, so I'm not sure why the gc would need to know about the python threads. by gum, thread_attachThis and thread_detachThis fix the issue! now to figure out how to use them in the general case. This is great.. Thank you. I'm looking forward to being able to try the finished result.
Re: threading issues with D -> C -> Python
On 12/04/2014 10:55 PM, Ellery Newcomer wrote: I guess tomorrow I can try messing around with thread_attachThis, as the fullcollect happening in #2 might be screwing with python data. But you aren't really passing anything from python to d or vice versa, so I'm not sure why the gc would need to know about the python threads. by gum, thread_attachThis and thread_detachThis fix the issue! now to figure out how to use them in the general case.
Re: threading issues with D -> C -> Python
On 12/04/2014 02:11 PM, Michael wrote: On Thursday, 4 December 2014 at 03:22:05 UTC, Ellery Newcomer wrote: dustmite? Not sure what went wrong with dustmite, but every time I tried it it just started deleting all the files in the directory and setup.py would give errors. I manually deleted a reasonable chunk of the code and I'm left with these files which still seem to cause segfaults: Main code: http://pastebin.com/zqgNTk9w PyD definitions: http://pastebin.com/6mRH3KZZ setup.py: http://pastebin.com/i9Ph78UC test code that causes segfaults: http://pastebin.com/1ukzShVh Cheers, Michael. hmm.. looks like here it originates in python when it tries to acquire the GIL. specifically, pthread_cond_timedwait is segfaulting. in your code, execution inside a python thread makes it to receiveTimeout in get_image. it made it past receiveTimeout in acquire. then I guess there is a context switch. the main python thread throws an exception, but a number of things trigger the segfault. I think it's just the interpreter loop calling RestoreThread. backtrace looks like #0 pthread_cond_timedwait@@GLIBC_2.3.2 () at ../nptl/sysdeps/unix/sysv/linux/x86_64/pthread_cond_timedwait.S:238 #1 0x003799d07bb3 in PyCOND_TIMEDWAIT (cond=0x379a063220 , mut=0x379a0631e0 , us=5000) at /usr/src/debug/Python-3.3.2/Python/condvar.h:103 #2 take_gil (tstate=tstate@entry=0x604410) at /usr/src/debug/Python-3.3.2/Python/ceval_gil.h:224 #3 0x003799d081fb in PyEval_RestoreThread (tstate=tstate@entry=0x604410) ... It looks like this is the python main thread. I see two other threads. (i took out one of your python spawns) #2 looks to be your listener thread. std.concurrency.send seems to have gotten it into a gc_malloc, but it looks like it's just waiting. it's currently in sem_wait. this one would have been spawned in D code by #3 #3 is your other python thread. it is also in pthread_cond_timedwait. by its stack trace, receiveTimeout is just waiting. I guess tomorrow I can try messing around with thread_attachThis, as the fullcollect happening in #2 might be screwing with python data. But you aren't really passing anything from python to d or vice versa, so I'm not sure why the gc would need to know about the python threads. not exactly my area of expertise, this.
Re: threading issues with D -> C -> Python
On Thu, Dec 04, 2014 at 10:11:53PM +, Michael via Digitalmars-d-learn wrote: > On Thursday, 4 December 2014 at 03:22:05 UTC, Ellery Newcomer wrote: > > > >dustmite? > > Not sure what went wrong with dustmite, but every time I tried it it > just started deleting all the files in the directory and setup.py > would give errors. [...] Nothing is wrong with dustmite; one of its standard techniques to reduce code is to delete source files and see if the problem still happens (i.e., it's independent of that file). What you need to do is to craft a test script such that only the specific error you're looking for will return a success status, and everything else won't. Then when dustmite deletes an essential file, it will know that that was a wrong step and backtrack. T -- Food and laptops don't mix.
Re: threading issues with D -> C -> Python
On Thursday, 4 December 2014 at 03:22:05 UTC, Ellery Newcomer wrote: dustmite? Not sure what went wrong with dustmite, but every time I tried it it just started deleting all the files in the directory and setup.py would give errors. I manually deleted a reasonable chunk of the code and I'm left with these files which still seem to cause segfaults: Main code: http://pastebin.com/zqgNTk9w PyD definitions: http://pastebin.com/6mRH3KZZ setup.py: http://pastebin.com/i9Ph78UC test code that causes segfaults: http://pastebin.com/1ukzShVh Cheers, Michael.
Re: threading issues with D -> C -> Python
On 12/03/2014 06:56 PM, Michael wrote: On Thursday, 4 December 2014 at 02:31:51 UTC, Ellery Newcomer wrote: okay. that's not too surprising. If you can get me a minimal example, I'd be happy to have a look since pyd should probably support this case. Cool. Unfortunately most of the times I've attempted to reduce this down it always seems to work, but I think that's because I often did the example code in D. I'll play around with it and try to send you an example. Cheers, Michael. dustmite?
Re: threading issues with D -> C -> Python
On Thursday, 4 December 2014 at 02:31:51 UTC, Ellery Newcomer wrote: okay. that's not too surprising. If you can get me a minimal example, I'd be happy to have a look since pyd should probably support this case. Cool. Unfortunately most of the times I've attempted to reduce this down it always seems to work, but I think that's because I often did the example code in D. I'll play around with it and try to send you an example. Cheers, Michael.
Re: threading issues with D -> C -> Python
On 12/03/2014 04:43 PM, Michael wrote: On Wednesday, 3 December 2014 at 21:35:48 UTC, ketmar via Digitalmars-d-learn wrote: ah, dsource strikes back! that vile site keep biting us again and again. let's hope that new admins will kill it for good. Yeah. I've got the new PyD and it compiles and does everything I want much nicer, but it appears to have the exact same problems. When calling a python thread to my code, it can cause segfaults and hanging issues. Cheers, Michael. okay. that's not too surprising. If you can get me a minimal example, I'd be happy to have a look since pyd should probably support this case.
Re: threading issues with D -> C -> Python
On Wednesday, 3 December 2014 at 21:35:48 UTC, ketmar via Digitalmars-d-learn wrote: ah, dsource strikes back! that vile site keep biting us again and again. let's hope that new admins will kill it for good. Yeah. I've got the new PyD and it compiles and does everything I want much nicer, but it appears to have the exact same problems. When calling a python thread to my code, it can cause segfaults and hanging issues. Cheers, Michael.
Re: threading issues with D -> C -> Python
On Wed, 03 Dec 2014 20:41:46 + Michael via Digitalmars-d-learn wrote: > On Wednesday, 3 December 2014 at 06:11:56 UTC, Ellery Newcomer > wrote: > > are you looking at this pyd: > > https://bitbucket.org/ariovistus/pyd > > I'm looking at this one, which is what came up when googling > "python to D" > http://pyd.dsource.org/ ah, dsource strikes back! that vile site keep biting us again and again. let's hope that new admins will kill it for good. signature.asc Description: PGP signature
Re: threading issues with D -> C -> Python
On Wednesday, 3 December 2014 at 06:30:07 UTC, Russel Winder via Digitalmars-d-learn wrote: As far as I can tell PyD is still active, but in a non-funded FOSS way, i.e. work happens as and when volunteers put time and effort in. I haven't tried PyD recently but it worked fine last time I did. If can set out what you tried and what didn't work, maybe there is a PyD solution, or a fix to PyD to give a solution? Yeah apparently I might have used the wrong PyD. This might end up being all I need to do to fix my problem. D's "big problem" is shared objects/dynamic link libraries. Without them you cannot interwork with Python at all. I have tried experiments on Linux creating shared libraries from D code with C linkage entry points to create classic Python extensions, and it appears to work fine. Except for having to start up the D heap and thread management, should they be needed. But that is what PyD is there for. If I took my experiments any further I would end up recreating PyD or something like it. This is what I'm doing. I'm using the rt_init() function to setup the heap/thread management. Am I missing anything else here? It seems the issue is definitely more complicated than that. I'll also point out that: http://dlang.org/interfaceToC had no information on calling rt_init first, although it seems like it should. It sounds like you are in a similar situation except that you appear to have an extra layer of C code. I am not sure a layer of C is needed between Python and D, it would be good to know more about why you seem to need it. Well I wanted to compile D code and directly call it in Python, and since PyD didn't work for me, I instead tried the python-> C interface I already knew (https://docs.python.org/2/c-api/index.html) and worked my way through a C->D interface, which I understood to be relatively simple. My guess would be not properly initializing the D infrastructure from the incoming Python thread. I would suggest that you want to avoid threads crossing the boundaries and just pass data via a shared channel. Unix pipes seem to work well in this context since they provide a language independent data channel. Yeah I'm leaning in that direction myself, although I might try the other PyD library first. I wanted to be able to use the D message-passing libraries to do the thread-safety stuff at first, because it was much easier than the alternative, but I'm not sure that's true anymore.
Re: threading issues with D -> C -> Python
On Wednesday, 3 December 2014 at 06:11:56 UTC, Ellery Newcomer wrote: are you looking at this pyd: https://bitbucket.org/ariovistus/pyd I'm looking at this one, which is what came up when googling "python to D" http://pyd.dsource.org/
Re: threading issues with D -> C -> Python
On Wed, 2014-12-03 at 01:07 +, Michael via Digitalmars-d-learn wrote: > Hi. I'm new here and this is my first post. I'm not sure this is > the right subforum for it, but wasn't sure where else to put it > either. > > I've written a library to talk to some external hardware using a > socket. It uses the std.concurrency threads to send messages > between the main D-object for the hardware and the D-object for > the sockets. I then wanted to be able to call these functions > from Python. PyD appeared to be out of date, so I've been using a As far as I can tell PyD is still active, but in a non-funded FOSS way, i.e. work happens as and when volunteers put time and effort in. I haven't tried PyD recently but it worked fine last time I did. If can set out what you tried and what didn't work, maybe there is a PyD solution, or a fix to PyD to give a solution? > D -> C interface, and a C -> Python interface. The python code > will often run from different python threads, so I then added yet > another message-passing layer between the D->C interface and the > D->hardware interface. D's "big problem" is shared objects/dynamic link libraries. Without them you cannot interwork with Python at all. I have tried experiments on Linux creating shared libraries from D code with C linkage entry points to create classic Python extensions, and it appears to work fine. Except for having to start up the D heap and thread management, should they be needed. But that is what PyD is there for. If I took my experiments any further I would end up recreating PyD or something like it. It sounds like you are in a similar situation except that you appear to have an extra layer of C code. I am not sure a layer of C is needed between Python and D, it would be good to know more about why you seem to need it. Your use case is interesting as I have more or less given up on using D in a Python context. CPython naturally requires C linkage shared objects for non-Python code so C, C++ or D would be fine except that everyone does it in C or C++, very few people in the arena have even heard of D. PyPy brings it's own issues with non-Python code but the now have a C capability. With Cython, Pythran, and more recently Numba there is increasing less and less need for any user written non-Python code. Hardware control would be done in C and PyPy now has a C linkage mechanism. Of course for Python networking there is Twisted, Asyncio and Tornado so no Python folk are using non-Python code for handling networking. > My problem is that this code routinely causes segmentation > faults. I've spent a long time going through trying to figure out > exactly what the causes are. I think there have been some related > to D-exceptions not being handled gracefully by the C/Python > code. Some more by stdout writing from multiple threads (which > surprised me). > > I'm fairly sure I have tackled both of these issues, but it still > seems like Python threads and D threads don't mix well. When > running the same functions from D, I am able to get no errors, > but when run from Python/C it causes segfaults reliably. Without seeing your code, it is difficult to say what may or may not be the problem, but I would guess it is about D infrastructure start up. I recollect being able to reliably get segfaults this way. Are you using ctypes or CFFI on the Python side? > Sorry for the large exposition. I am currently at the point of > suspecting bugs in Phobos, but I am unskilled enough to tell for > sure, and would appreciate any help. > > The latest core dump gives a backtrace of almost entirely phobos > commands: > > #0 0x7fe789ad3b97 in gc.gc.Gcx.fullcollect() () from > /lib/libphobos2.so.0.66 > #1 0x7fe789ad3294 in gc.gc.Gcx.bigAlloc() () from > /lib/libphobos2.so.0.66 > #2 0x7fe789ad0df1 in gc.gc.GC.mallocNoSync() () from > /lib/libphobos2.so.0.66 > #3 0x7fe789ad0c15 in gc.gc.GC.malloc() () from > /lib/libphobos2.so.0.66 > #4 0x7fe789ad6470 in gc_malloc () from > /lib/libphobos2.so.0.66 > #5 0x7fe789ae6d36 in _d_newitemT () from > /lib/libphobos2.so.0.66 > #6 0x7fe789e57112 in > std.array.__T8AppenderTAaZ.Appender.__T3putTAxaZ.put() () from > /usr/lib/libv5camera.so > #7 0x7fe789e570b5 in > std.array.__T8AppenderTAaZ.Appender.__T3putTAxaZ.put() () from > /usr/lib/libv5camera.so > #8 0x7fe789e562dc in > std.array.__T8AppenderTAaZ.Appender.__T3putTAaZ.put() () from > /usr/lib/libv5camera.so > #9 0x7fe789e561ea in > std.array.__T8AppenderTAaZ.Appender.__T3putTxwZ.put() () from > /usr/lib/libv5camera.so > #10 0x7fe789e5617d in > std.format.__T10formatCharTS3std5array16__T8AppenderTAaZ8AppenderZ.formatChar() > > () from /usr/lib/libv5camera.so > #11 0x7fe789e56132 in > std.format.__T10formatCharTS3std5array16__T8AppenderTAaZ8AppenderZ.formatChar() > > () from /usr/lib/libv5camera.so > #12 0x7fe789e61f09 in > std.concurrency.MessageBox.__T3getTS4core4time8D
Re: threading issues with D -> C -> Python
On 12/02/2014 05:07 PM, Michael wrote: Hi. I'm new here and this is my first post. I'm not sure this is the right subforum for it, but wasn't sure where else to put it either. I've written a library to talk to some external hardware using a socket. It uses the std.concurrency threads to send messages between the main D-object for the hardware and the D-object for the sockets. I then wanted to be able to call these functions from Python. PyD appeared to be out of date, so I've been using a D -> C interface, and a C -> Python interface. The python code will often run from different python threads, so I then added yet another message-passing layer between the D->C interface and the D->hardware interface. are you looking at this pyd: https://bitbucket.org/ariovistus/pyd
Re: threading issues with D -> C -> Python
On Wed, 03 Dec 2014 01:07:42 + Michael via Digitalmars-d-learn wrote: btw, Adam Ruppe's "D Cookbook" has a chapter which describes how to call D library from C code. don't remember if it describes threading, though. signature.asc Description: PGP signature
Re: threading issues with D -> C -> Python
On Wed, 03 Dec 2014 01:07:42 + Michael via Digitalmars-d-learn wrote: all in all, you'd better not mixing D code with "alien" mulththreaded code and not using .a/.so libraries written in D in another language until you are familiar with D runtime and GC. those mixes are very fragile. signature.asc Description: PGP signature
Re: threading issues with D -> C -> Python
On Wed, 03 Dec 2014 02:52:27 + Michael via Digitalmars-d-learn wrote: by "using from C code" i mean that your main program is not written in D, it has no D `main()` and so on. i.e. you wrote, for example, some .a library in D and now you want to use that library in C code. signature.asc Description: PGP signature
Re: threading issues with D -> C -> Python
On Wed, 03 Dec 2014 02:52:27 + Michael via Digitalmars-d-learn wrote: > Okay. Well I am already not passing any D-allocated data. I'm > specifically creating variables/arrays on the C-stack, and then > passing the pointer of that to D and overwriting the data of the > C-stack pointer for any return values. I was worried about that > specific problem and I thought this would be a solution. I am > then able to tell python to use the C-stack variable without > having to worry about D trying to run any garbage collection on > it. if D code has any pointer to that data stored anywhere, GC will walk it and hit another thread's stack. and now it doesn't know where it is, and it can't pause that uknown thread so it will not mutate the area GC is scanning now. this *may* work, but it will segfault sooner or later. > Going the other way, I probably am passing some python strings > etc.. into D, but I would assume they are valid for the lifetime > of the function call, and that D would have no reason to try and > perform any garbage collection on them. D has conservative GC, so it will try to walk with the unknown data just in case that data contains some pointers. and GC can hit "false positives" there (something that *looks* like a pointer to some area) and other things. so to make the long story short: you should either register and deregister *all* your threads in GC (for D threads it's automatic process; for other threads you must do it manually), or don't use GC at all. besides, if you are using your D library from C code, you must call `rt_init()` once before calling any D code. this function will initialize D runtime. and you have to call `rt_term()` before exiting your program to deinitialize D runtime. signature.asc Description: PGP signature
Re: threading issues with D -> C -> Python
On Wednesday, 3 December 2014 at 02:41:11 UTC, ketmar via Digitalmars-d-learn wrote: On Wed, 03 Dec 2014 02:21:45 + Michael via Digitalmars-d-learn wrote: Thanks for this. Its definitely a step in the right direction. Would you mind explaining a bit more about the problem here, if you can? I don't fully understand why the garbage collector needs to know about the threads, and if so for how long does it need to know? If I put in "thread_attachThis();scope(exit)thread_detachThis();" it doesn't appear to fix my problems, so I'm definitely curious as to what is going on under the hood. you have to call `thread_attachThis();` in "alien" thread, not in D thread. i.e. if you created thread from python code, you have to call `thread_attachThis();` in that python thread (i don't know how you'll do that, but you must ;-). and you must call `thread_detachThis();` from the same python thread before exiting from it. garbage collector must know about all running threads so it can scan their stacks, variables and so on. as there is no portable way to set application-wide hooks on thread creation and termination, you must inform GC about that events manually. the other thing you can do is to not use any D allocated data in "alien" threads. i.e. don't pass anything that was allocated by D code to python thread and vice versa. if you want to pass some data to "alien" thread, `malloc()` the necessary space, copy data to it and pass malloc'ed pointer. don't forget to free that data in "alien" thread. but i think that this is not what you really want, as it means alot of allocations and copying, and complicates the whole thing alot. "alien" is the thread that was created outside of D code. Okay. Well I am already not passing any D-allocated data. I'm specifically creating variables/arrays on the C-stack, and then passing the pointer of that to D and overwriting the data of the C-stack pointer for any return values. I was worried about that specific problem and I thought this would be a solution. I am then able to tell python to use the C-stack variable without having to worry about D trying to run any garbage collection on it. Going the other way, I probably am passing some python strings etc.. into D, but I would assume they are valid for the lifetime of the function call, and that D would have no reason to try and perform any garbage collection on them.
Re: threading issues with D -> C -> Python
On Wed, 03 Dec 2014 02:21:45 + Michael via Digitalmars-d-learn wrote: > Thanks for this. Its definitely a step in the right direction. > Would you mind explaining a bit more about the problem here, if > you can? I don't fully understand why the garbage collector needs > to know about the threads, and if so for how long does it need to > know? If I put in > "thread_attachThis();scope(exit)thread_detachThis();" it doesn't > appear to fix my problems, so I'm definitely curious as to what > is going on under the hood. you have to call `thread_attachThis();` in "alien" thread, not in D thread. i.e. if you created thread from python code, you have to call `thread_attachThis();` in that python thread (i don't know how you'll do that, but you must ;-). and you must call `thread_detachThis();` from the same python thread before exiting from it. garbage collector must know about all running threads so it can scan their stacks, variables and so on. as there is no portable way to set application-wide hooks on thread creation and termination, you must inform GC about that events manually. the other thing you can do is to not use any D allocated data in "alien" threads. i.e. don't pass anything that was allocated by D code to python thread and vice versa. if you want to pass some data to "alien" thread, `malloc()` the necessary space, copy data to it and pass malloc'ed pointer. don't forget to free that data in "alien" thread. but i think that this is not what you really want, as it means alot of allocations and copying, and complicates the whole thing alot. "alien" is the thread that was created outside of D code. signature.asc Description: PGP signature
Re: threading issues with D -> C -> Python
Thanks for this. Its definitely a step in the right direction. Would you mind explaining a bit more about the problem here, if you can? I don't fully understand why the garbage collector needs to know about the threads, and if so for how long does it need to know? If I put in "thread_attachThis();scope(exit)thread_detachThis();" it doesn't appear to fix my problems, so I'm definitely curious as to what is going on under the hood. Cheers, Michael. On Wednesday, 3 December 2014 at 01:17:43 UTC, ketmar via Digitalmars-d-learn wrote: On Wed, 03 Dec 2014 01:07:42 + Michael via Digitalmars-d-learn wrote: I'm fairly sure I have tackled both of these issues, but it still seems like Python threads and D threads don't mix well. When running the same functions from D, I am able to get no errors, but when run from Python/C it causes segfaults reliably. you are right, D threads and other language/library threads aren't mix well. at least you have to use `thread_attachThis()` and `thread_detachThis()` from core.threads module to make sure that GC is aware of "alien" threads. and i assume that calling this functions from python will not be very easy. but it's better to not mix 'em at all if it is possible.
Re: threading issues with D -> C -> Python
On Wed, 03 Dec 2014 01:07:42 + Michael via Digitalmars-d-learn wrote: > I'm fairly sure I have tackled both of these issues, but it still > seems like Python threads and D threads don't mix well. When > running the same functions from D, I am able to get no errors, > but when run from Python/C it causes segfaults reliably. you are right, D threads and other language/library threads aren't mix well. at least you have to use `thread_attachThis()` and `thread_detachThis()` from core.threads module to make sure that GC is aware of "alien" threads. and i assume that calling this functions from python will not be very easy. but it's better to not mix 'em at all if it is possible. signature.asc Description: PGP signature
Re: Threading methodology
Am Sat, 07 Dec 2013 17:53:06 +0100 schrieb "Frustrated" : > I have to process n arrays in some partial order. Instead of all > working only on the n arrays and reusing them, [...] Wait, what partial order and how is it relevant? Who is "all" in "all working"? Why "only" the n arrays, I thought those are all? > if I "duplicate" > them(effectively write once read many) does that make things > simpler and allow threading to be used more efficiently? > > Basically, instead of having to worry about threads waiting on > other threads when they are writing the data I can just have each > thread write to it's own duplicate array(although hopefully won't > have to duplicate the buffer as that should not be necessary?). > This should then make it much easier to deal because there is > essentially no write contention. > > If I'm not mistaken this is basically the idea of TLS? (read many > write once time idea so that one doesn't have to lock memory > which stalls other threads in some cases and/or uses expensive > locks?) The idea of TLS in D as I understand it, is that your global variables are not shared between threads by default as is the case in C. So as you point out you need not lock them since each thread has its own copy. > The case I gave above is a little more simplified than the actual > issue but the idea is simply that by writing to a threads own > buffer reduces inter-threading issues and increases performance. > One issue I have is how to combine the data back. > > In fact, the real case is basically n arrays. Each array is > processed to create m arrays, which are processed to create l > arrays and so on. Not all processing of each array takes the same > time and I would like to multithread the overall processing by > allowing threads to pick which (sub)arrays need processing and > process them, but in their own space, and at the end send them > back to the main thread. (But one thread may need access to > another's buffer but only for reading so it shouldn't be a > problem) > > This uses a lot more memory but if there are N threads and n > arrays and N > n, no thread will be wasted. (in my case I would > have n*m*l which will almost always be larger than N so threads > will always be doing something except towards the end) > > Another way to see the problem is to think of a digraph where > each node processes all connected to it. All independent nodes > must be processed first which then works itself up to the most > dependent nodes. Every processed node uses a new buffer to write > to instead of writing to the same buffer which locks that buffer > and all other threads that may be trying to read from it(which is > the key in my mind that this method is better except it requires > more memory). > > I will of course mark nodes as being processed or not so threads > don't work on unprocessed buffers and this allows dependent nodes > to start up once all there dependencies become processed. > > Does any of this make sense? It is a little clearer now, but I didn't fully understand it yet. The sub-arrays need to be processed before the parent arrays can be completed? When does a thread need access to another thread's buffer? What if you have only 1 CPU core and there is only one thread and one buffer? Does your algorithm break? :p -- Marco
Re: Threading / Concurrency Problem
On Wednesday, 23 October 2013 at 12:41:58 UTC, Bauss wrote: Thanks. It worked! :) Connections is in thread local storage by default. If you want it to be global across threads, you have to mark it shared or __gshared.
Re: Threading / Concurrency Problem
On Wednesday, 23 October 2013 at 12:20:32 UTC, Meta wrote: On Wednesday, 23 October 2013 at 11:51:35 UTC, Bauss wrote: Working on a 2nd simple webserver in D. However I am facing a problem with some threading. I want the receiving of connections to happen in a 2nd thread, but after adding the connections to the collection it seems like the receive thread never loops through them, as if no connections were ever added. However I have checked and it is added. The thread for receiving is also working as I have checked if code outside of the foreach loop works and it does, so it's probably a problem with the collection. This makes me assume it's a concurrency issue and I'm not sure how to fix it. I'm using an associative array for the connections. Class[int] is the format I'm using. You can view my code here: If anyone knows the solution to this, it would be appreciated. I can't seem to figure it out. Code: http://pastebin.com/A61NPWUq Possibly try marking Connections as shared. I can't compile your example because it's missing import statements. Thanks. It worked! :)
Re: Threading / Concurrency Problem
On Wednesday, 23 October 2013 at 11:51:35 UTC, Bauss wrote: Working on a 2nd simple webserver in D. However I am facing a problem with some threading. I want the receiving of connections to happen in a 2nd thread, but after adding the connections to the collection it seems like the receive thread never loops through them, as if no connections were ever added. However I have checked and it is added. The thread for receiving is also working as I have checked if code outside of the foreach loop works and it does, so it's probably a problem with the collection. This makes me assume it's a concurrency issue and I'm not sure how to fix it. I'm using an associative array for the connections. Class[int] is the format I'm using. You can view my code here: If anyone knows the solution to this, it would be appreciated. I can't seem to figure it out. Code: http://pastebin.com/A61NPWUq Possibly try marking Connections as shared. I can't compile your example because it's missing import statements.
Re: Threading Question
On Oct 30, 2012, at 11:06 AM, Alex Rønne Petersen wrote: > On 30-10-2012 19:04, Sean Kelly wrote: >> >> >> The semaphore implementation for OSX is not signal-safe. Originally, >> druntime used the signal approach on OSX and had deadlock issues in the >> signal handler (OSX semaphores wrap a global mutex to obtain something from >> a shared pool). Also, semaphores on OSX are just ridiculously slow. The >> current method for suspending and scanning threads on OSX is much faster. I >> wish Posix in general had the same feature. Using signals is really a hack. >> >> It may be worth dropping use of SIGUSR1/2 in favor of the realtime signals >> as well, since SIGUSR1/2 are in pretty common use. >> > > Real time signals as in? SIGRTMIN - SIGRTMAX. Linux supports them, but I'm not sure which other Posix OSes.
Re: Threading Question
On 30-10-2012 19:04, Sean Kelly wrote: On Oct 25, 2012, at 11:18 PM, Jacob Carlborg wrote: On 2012-10-26 01:18, Sean Kelly wrote: On Oct 25, 2012, at 4:12 PM, Alex Rønne Petersen wrote: What's used on OS X? I forget... The method used is similar to how GC works on Windows--there's a kernel call that can be used to explicitly suspend a thread. I can't remember the function name offhand though. The Posix functions weren't working properly? The semaphore implementation for OSX is not signal-safe. Originally, druntime used the signal approach on OSX and had deadlock issues in the signal handler (OSX semaphores wrap a global mutex to obtain something from a shared pool). Also, semaphores on OSX are just ridiculously slow. The current method for suspending and scanning threads on OSX is much faster. I wish Posix in general had the same feature. Using signals is really a hack. It may be worth dropping use of SIGUSR1/2 in favor of the realtime signals as well, since SIGUSR1/2 are in pretty common use. Real time signals as in? -- Alex Rønne Petersen a...@lycus.org http://lycus.org
Re: Threading Question
On Oct 25, 2012, at 11:18 PM, Jacob Carlborg wrote: > On 2012-10-26 01:18, Sean Kelly wrote: >> On Oct 25, 2012, at 4:12 PM, Alex Rønne Petersen wrote: >>> >>> What's used on OS X? I forget... >> >> The method used is similar to how GC works on Windows--there's a kernel call >> that can be used to explicitly suspend a thread. I can't remember the >> function name offhand though. > > The Posix functions weren't working properly? The semaphore implementation for OSX is not signal-safe. Originally, druntime used the signal approach on OSX and had deadlock issues in the signal handler (OSX semaphores wrap a global mutex to obtain something from a shared pool). Also, semaphores on OSX are just ridiculously slow. The current method for suspending and scanning threads on OSX is much faster. I wish Posix in general had the same feature. Using signals is really a hack. It may be worth dropping use of SIGUSR1/2 in favor of the realtime signals as well, since SIGUSR1/2 are in pretty common use.
Re: Threading Question
On 2012-10-26 01:18, Sean Kelly wrote: On Oct 25, 2012, at 4:12 PM, Alex Rønne Petersen wrote: What's used on OS X? I forget... The method used is similar to how GC works on Windows--there's a kernel call that can be used to explicitly suspend a thread. I can't remember the function name offhand though. The Posix functions weren't working properly? -- /Jacob Carlborg
Re: Threading Question
Sean Kelly wrote: [snip] In D, the main thread will join all non-daemon threads before calling static dtors or performing any other cleanup. Daemon threads will continue to run until the process exits. So daemon threads are basically just C-style kernel threads. [snip] The GC in D uses SIGUSR1 and SIGUSR2 to coordinate collections on Posix OSes (except for OSX). You're free to use any other signal just as you would in C. Thanks for clarification Sean! Peter
Re: Threading Question
On Oct 25, 2012, at 4:12 PM, Alex Rønne Petersen wrote: > > What's used on OS X? I forget... The method used is similar to how GC works on Windows--there's a kernel call that can be used to explicitly suspend a thread. I can't remember the function name offhand though.
Re: Threading Question
On 26-10-2012 01:11, Sean Kelly wrote: On Oct 23, 2012, at 11:25 AM, Peter Sommerfeld wrote: Hi! I'm new to D an not a native speaker so I may misunderstand the following sentence of the thread documentation. "final @property void isDaemon(bool val); Sets the daemon status for this thread. While the runtime will wait for all normal threads to complete before tearing down the process, daemon threads are effectively ignored and thus will not prevent the process from terminating. In effect, daemon threads will be terminated automatically by the OS when the process exits." That sounds to me as if the daemon will be finished when the main-thread has finished. But in my understanding daemons will survive the termination of the main-thread and be killed by a signal (KILL, SIGTERM etc) or finish itself. I think that is the case here too. Is that true ? In D, the main thread will join all non-daemon threads before calling static dtors or performing any other cleanup. Daemon threads will continue to run until the process exits. So daemon threads are basically just C-style kernel threads. Another question: I cannot find a reference how D deals with OS SIGNALS, especially about differences between the platform (*nix; Windows, Mac). Can you explain or point me to the documentation? The GC in D uses SIGUSR1 and SIGUSR2 to coordinate collections on Posix OSes (except for OSX). You're free to use any other signal just as you would in C. What's used on OS X? I forget... -- Alex Rønne Petersen a...@lycus.org http://lycus.org
Re: Threading Question
On Oct 23, 2012, at 11:25 AM, Peter Sommerfeld wrote: > Hi! > > I'm new to D an not a native speaker so I may misunderstand > the following sentence of the thread documentation. > > "final @property void isDaemon(bool val); > > Sets the daemon status for this thread. While the runtime > will wait for all normal threads to complete before tearing > down the process, daemon threads are effectively ignored and > thus will not prevent the process from terminating. In effect, > daemon threads will be terminated automatically by the OS when > the process exits." > > That sounds to me as if the daemon will be finished when the > main-thread has finished. But in my understanding daemons will > survive the termination of the main-thread and be killed by > a signal (KILL, SIGTERM etc) or finish itself. > > I think that is the case here too. Is that true ? In D, the main thread will join all non-daemon threads before calling static dtors or performing any other cleanup. Daemon threads will continue to run until the process exits. So daemon threads are basically just C-style kernel threads. > Another question: I cannot find a reference how D deals with > OS SIGNALS, especially about differences between the platform > (*nix; Windows, Mac). Can you explain or point me to the > documentation? The GC in D uses SIGUSR1 and SIGUSR2 to coordinate collections on Posix OSes (except for OSX). You're free to use any other signal just as you would in C.
Re: Threading errors.
So what's the word on this? Will we have simple indexing of tuples via T[] or do we still need T.field[] and T._1 ? The situations is the same in 2.048 as it was in 2.047. There are more examples of tuples being used via T[] in some Phobos code examples (which don't compile and I've reported that). Otherwise if it stays the way it is I'll send Andrei a message to add the corrections to the errata. Philippe Sigaud Wrote: > On Tue, Jul 27, 2010 at 19:03, Sean Kelly wrote: > > > The next release, 2.048, should bring things in line with TDPL. I had > > meant to do this for 2.047, but was too busy with other work to finish in > > time. > > > > Do you mean, indexing tuples directly by a CT value as in msg[1]? Because > that was the only error here, I think. > > On Tue, Jul 27, 2010 at 19:03, Sean Kelly dir="ltr">< href="mailto:s...@invisibleduck.org";>s...@invisibleduck.org> > wrote: > The next release, 2.048, should bring things in line with TDPL. I had meant > to do this for 2.047, but was too busy with other work to finish in time. > Do you mean, indexing tuples directly by a CT > value as in msg[1]? Because that was the only error here, I > think. >
Re: Threading errors.
On Tue, Jul 27, 2010 at 19:03, Sean Kelly wrote: > The next release, 2.048, should bring things in line with TDPL. I had > meant to do this for 2.047, but was too busy with other work to finish in > time. > Do you mean, indexing tuples directly by a CT value as in msg[1]? Because that was the only error here, I think.
Re: Threading errors.
The next release, 2.048, should bring things in line with TDPL. I had meant to do this for 2.047, but was too busy with other work to finish in time.
Re: Threading errors.
== Quote from Rory Mcguire (rjmcgu...@gm_no_ail.com)'s article > Philippe Sigaud wrote: > >> > Also, in my case, the return; in writer must be commented out, or DMD > >> > complains it cannot be reached. > >> > > > > > >> > >> Interesting, I didn't have to comment out return; using dmd 2.047 on > >> linux > >> > > > > I think I have -w (warnings treated as errors?) always checked. That's my > > Code::Blocks default configuration for DMD. > > I was on Windows for this test. I never cheked if there was any difference > > between OSes for this :-) > :) thanks Philippe Actually, after making the changes that you suggest, Philippe my program works. Thanks. However, I did not have to comment out the return statement. But when I did compile using the -w flag, I do get the compiler error that you describe. Anyways, thanks for the coding suggestion.
Re: Threading errors.
Philippe Sigaud wrote: > On Tue, Jul 27, 2010 at 11:25, Rory Mcguire > wrote: > >> >> > Also, in my case, the return; in writer must be commented out, or DMD >> > complains it cannot be reached. >> > > >> >> Interesting, I didn't have to comment out return; using dmd 2.047 on >> linux >> > > I think I have -w (warnings treated as errors?) always checked. That's my > Code::Blocks default configuration for DMD. > I was on Windows for this test. I never cheked if there was any difference > between OSes for this :-) :) thanks Philippe
Re: Threading errors.
On Tue, Jul 27, 2010 at 11:25, Rory Mcguire wrote: > > > Also, in my case, the return; in writer must be commented out, or DMD > > complains it cannot be reached. > > > Interesting, I didn't have to comment out return; using dmd 2.047 on linux > I think I have -w (warnings treated as errors?) always checked. That's my Code::Blocks default configuration for DMD. I was on Windows for this test. I never cheked if there was any difference between OSes for this :-)
Re: Threading errors.
Philippe Sigaud wrote:
> On Mon, Jul 26, 2010 at 19:11, dcoder wrote:
>
>> == Quote from Rory Mcguire (rjmcgu...@gm_no_ail.com)'s article
>> > Dmitry Olshansky wrote:
>>
>
>
> std.typecons.Tuple fields cannot be indexed like arrays, Andrei made a
> mistake. To access field #n, use ._n or .field[n]. There is no difference
> between the two.
>
> void writer() {
> for( ;; ) {
>auto msg = receiveOnly!(Tid, int)(); // msg is a Tuple!(Tid, int),
>msg._0
> is a Tid, msg._1 is an int.
>writeln( "Secondary thread: ", msg._1);
>msg._0.send(thisTid);
> }
> }
>
> Also, in my case, the return; in writer must be commented out, or DMD
> complains it cannot be reached.
>
>
> Philippe
Interesting, I didn't have to comment out return; using dmd 2.047 on linux
Re: Threading errors.
On Mon, Jul 26, 2010 at 19:11, dcoder wrote:
> == Quote from Rory Mcguire (rjmcgu...@gm_no_ail.com)'s article
> > Dmitry Olshansky wrote:
>
std.typecons.Tuple fields cannot be indexed like arrays, Andrei made a
mistake. To access field #n, use ._n or .field[n]. There is no difference
between the two.
void writer() {
for( ;; ) {
auto msg = receiveOnly!(Tid, int)(); // msg is a Tuple!(Tid, int), msg._0
is a Tid, msg._1 is an int.
writeln( "Secondary thread: ", msg._1);
msg._0.send(thisTid);
}
}
Also, in my case, the return; in writer must be commented out, or DMD
complains it cannot be reached.
Philippe
Re: Threading errors.
== Quote from Rory Mcguire (rjmcgu...@gm_no_ail.com)'s article > Dmitry Olshansky wrote: > > On 26.07.2010 19:45, dcoder wrote: > >> $ dmd --help > >> Digital Mars D Compiler v2.042 > >> Copyright (c) 1999-2010 by Digital Mars written by Walter Bright > >> Documentation: http://www.digitalmars.com/d/2.0/index.html > >> > >> > >> > > I suggest updating compiler - it's should be 2.047 now. The outdated > > distribution could also feature some bugs in std.concurency. > Doesn't work for 2.047 either. > dthreadpass.d(19): Error: no [] operator overload for type Tuple!(Tid,int) > dthreadpass.d(20): Error: no [] operator overload for type Tuple!(Tid,int) Yep, I get that same error after upgrading my compiler. $ dmd --help Digital Mars D Compiler v2.047 Copyright (c) 1999-2010 by Digital Mars written by Walter Bright Documentation: http://www.digitalmars.com/d/2.0/index.html thanks.
Re: Threading errors.
Dmitry Olshansky wrote: > On 26.07.2010 19:45, dcoder wrote: >> Hello. >> >> > [snip] >> $ dmd --help >> Digital Mars D Compiler v2.042 >> Copyright (c) 1999-2010 by Digital Mars written by Walter Bright >> Documentation: http://www.digitalmars.com/d/2.0/index.html >> >> >> > I suggest updating compiler - it's should be 2.047 now. The outdated > distribution could also feature some bugs in std.concurency. >> So, what am I doing wrong here? >> >> thanks. >> >> > > Doesn't work for 2.047 either. dthreadpass.d(19): Error: no [] operator overload for type Tuple!(Tid,int) dthreadpass.d(20): Error: no [] operator overload for type Tuple!(Tid,int)
Re: Threading errors.
On 26.07.2010 19:45, dcoder wrote: Hello. [snip] $ dmd --help Digital Mars D Compiler v2.042 Copyright (c) 1999-2010 by Digital Mars written by Walter Bright Documentation: http://www.digitalmars.com/d/2.0/index.html I suggest updating compiler - it's should be 2.047 now. The outdated distribution could also feature some bugs in std.concurency. So, what am I doing wrong here? thanks. -- Dmitry Olshansky
Re: Threading
"Georg Wrede" wrote in message news:guc6ep$2im...@digitalmars.com... > Saaa wrote: > Steven Schveighoffer wrote: >>> i.e. I want thread 1 to initialize elements 0, 1, and 2, and thread 2 to >>> initialize elements 3, 4, and 5: >>> >>> thread1 = new ProcessingThread(arrayToInit[0..3]); >>> thread2 = new ProcessingThread(arrayToInit[3..6]); >>> thread1.start(); >>> thread2.start(); >>> >>> Should be completely thread safe. >> >> Could become more difficult if the distinction would be odd/even elements >> :P > > Why??? If your threads have no bugs (i.e. the "odd" thread doesn't read or > write the "even" elements, and vice versa), then this should be easy! > >> Or creatures on a grid who can only see the cell they stand on :D > > Then the only thing that has to be thread safe is the code that moves a > creature from cell to cell. And, of course, it has to guarantee that no > two creatures end up in the same cell. > I meant slicing would become more difficult.
Re: Threading
Saaa wrote: Steven Schveighoffer wrote: i.e. I want thread 1 to initialize elements 0, 1, and 2, and thread 2 to initialize elements 3, 4, and 5: thread1 = new ProcessingThread(arrayToInit[0..3]); thread2 = new ProcessingThread(arrayToInit[3..6]); thread1.start(); thread2.start(); Should be completely thread safe. Could become more difficult if the distinction would be odd/even elements :P Why??? If your threads have no bugs (i.e. the "odd" thread doesn't read or write the "even" elements, and vice versa), then this should be easy! Or creatures on a grid who can only see the cell they stand on :D Then the only thing that has to be thread safe is the code that moves a creature from cell to cell. And, of course, it has to guarantee that no two creatures end up in the same cell.
Re: Threading
>>> I probably should have phrased my question better with the array, what I was wondering is if it was safe for say two threads to right to the array at the same time as long as I'm sure they're not writing to the same index of the array? >>> >>> You can do anything you want without thread safety, but you run the risk >>> of deadlocks or corrupted memory. >> That is why the question was whether it was safe. > > If two threads are writing to two different sections of memory, yes it is > always safe :) I think that's one of the fundamental premises of threads > running anyways. If you couldn't do this, you couldn't have threads. I used to think of an array as one thing, thus making it unsafe to write to it from multiple threads at the same time :) I kind of thought he was asking along this conception. > >> >>> >>> The problem isn't writing to two different elements of an array, >> >>> the hard part is *ensuring* that you are writing to two different >>> elements >>> of an array. Multithreading code is tough to write correctly, you may >>> want to read a book on it. >> And sometimes it is extremely easy to ensure you are never writing to the >> same elements. > > If you are doing anything interesting with an array, this is not the > case. Might as well not pass the same array to both threads. > > Maybe the OP doesn't understand that you can slice up an array quite > easily. If you want to ensure two threads don't touch the same memory, > don't give both threads access to the same memory. That's the easiest way > to ensure thread safety. > > i.e. I want thread 1 to initialize elements 0, 1, and 2, and thread 2 to > initialize elements 3, 4, and 5: > > thread1 = new ProcessingThread(arrayToInit[0..3]); > thread2 = new ProcessingThread(arrayToInit[3..6]); > thread1.start(); > thread2.start(); > > Should be completely thread safe. > > -Steve Could become more difficult if the distinction would be odd/even elements :P Or creatures on a grid who can only see the cell they stand on :D But yes, slicing is neat!
Re: Threading
On Tue, 12 May 2009 10:12:48 -0400, Saaa wrote: I probably should have phrased my question better with the array, what I was wondering is if it was safe for say two threads to right to the array at the same time as long as I'm sure they're not writing to the same index of the array? You can do anything you want without thread safety, but you run the risk of deadlocks or corrupted memory. That is why the question was whether it was safe. If two threads are writing to two different sections of memory, yes it is always safe :) I think that's one of the fundamental premises of threads running anyways. If you couldn't do this, you couldn't have threads. The problem isn't writing to two different elements of an array, the hard part is *ensuring* that you are writing to two different elements of an array. Multithreading code is tough to write correctly, you may want to read a book on it. And sometimes it is extremely easy to ensure you are never writing to the same elements. If you are doing anything interesting with an array, this is not the case. Might as well not pass the same array to both threads. Maybe the OP doesn't understand that you can slice up an array quite easily. If you want to ensure two threads don't touch the same memory, don't give both threads access to the same memory. That's the easiest way to ensure thread safety. i.e. I want thread 1 to initialize elements 0, 1, and 2, and thread 2 to initialize elements 3, 4, and 5: thread1 = new ProcessingThread(arrayToInit[0..3]); thread2 = new ProcessingThread(arrayToInit[3..6]); thread1.start(); thread2.start(); Should be completely thread safe. -Steve
Re: Threading
> >> I probably should have phrased my question better with the array, what I >> was >> wondering is if it was safe for say two threads to right to the array at >> the same >> time as long as I'm sure they're not writing to the same index of the >> array? > > You can do anything you want without thread safety, but you run the risk > of deadlocks or corrupted memory. That is why the question was whether it was safe. > > The problem isn't writing to two different elements of an array, >the hard part is *ensuring* that you are writing to two different elements >of an array. Multithreading code is tough to write correctly, you may >want to read a book on it. And sometimes it is extremely easy to ensure you are never writing to the same elements.
Re: Threading
On Tue, 12 May 2009 05:11:49 -0400, Brok3n Halo
wrote:
I probably should have phrased my question better with the array, what I
was
wondering is if it was safe for say two threads to right to the array at
the same
time as long as I'm sure they're not writing to the same index of the
array?
You can do anything you want without thread safety, but you run the risk
of deadlocks or corrupted memory.
The problem isn't writing to two different elements of an array, the hard
part is *ensuring* that you are writing to two different elements of an
array. Multithreading code is tough to write correctly, you may want to
read a book on it.
D2 promises to be a lot better at helping you ensure this.
Also I'm still getting the "Error: Win32 Exception" with my test code
any idea
why? Attached is the latest version.
Just realized from reading your code, you are using D1 with Phobos, I have
no idea what bugs there are, or how to use threads there, so I can't
really help you.
I can tell you that your code ported to Tango runs without throwing an
exception, code below:
import tango.core.Thread;
import tango.io.Stdout;
int count = 0;
void main(){
Thread testthread;
void testfunc(){
while(count<1000){
++count;
}
}
testthread = new Thread(&testfunc);
testthread.start();
int lastCount = 0;
while(testthread.isRunning){
if(count != lastCount){
Stdout.formatln("{}", count);
lastCount = count;
}
}
}
output:
164
16789
23750
29998
36054
4472263
4482283
4488871
4495320
4501356
4507264
4513158
4518987
4524886
4530722
4536557
4542362
4548221
4554051
4559848
4565753
4571592
4577354
4583152
4588942
4594719
4600579
4606375
4612181
4617981
4623686
4629421
-Steve
Re: Threading
I have never used threads before, but it sounds safe to me as when the thread gets thrown of the core and later gets back nothing referred by the register has changed. Again, I am a newbie :) "Brok3n Halo" wrote in message news:gubegl$uu...@digitalmars.com... >I probably should have phrased my question better with the array, what I >was > wondering is if it was safe for say two threads to right to the array at > the same > time as long as I'm sure they're not writing to the same index of the > array? > > Also I'm still getting the "Error: Win32 Exception" with my test code any > idea > why? Attached is the latest version. >
Re: Threading
I probably should have phrased my question better with the array, what I was
wondering is if it was safe for say two threads to right to the array at the
same
time as long as I'm sure they're not writing to the same index of the array?
Also I'm still getting the "Error: Win32 Exception" with my test code any idea
why? Attached is the latest version.
begin 644 test.d
M;6]D=6QE('1EPT*"0EI9BAC;W5N="`A/2!L87-T0V]U;G0I>PT*"0D)=W)I=&5F;&XH
M(B5D(b...@8v]u;G0I.PT*"0D);&%S=$-O=6YT(#...@8v]u;G0[#0H)"7T-"@E]
%#0H-"GT`
`
end
Re: Threading
On Mon, 11 May 2009 13:46:39 -0400, Brok3n Halo wrote: ah-ha! I was just writing to let you know that's how my project started out and I went into this issue, when I decided to try it again and got an error message I know how to solve. Probably got it before too, but working off an all-nighter and my caffeine hasn't fully kicked in yet. Turns out it's expecting an int testfunc(), the reason it wasn't working before was I was trying to use a uint testfunc() Thanks for the heads up on the I/O issue with threading, the writefln wasn't producing any output once I got it going and thanks to that I know what the issue with that was right away, luckily my actual project doesn't require I/O on a thread. (I'm writing a multi-threaded raytracer for a class) Which brings me to my next 2 questions, are dynamic arrays thread safe in D? I imagine they should be as long as I don't have two threads writing to the same index at once and the array size doesn't change. They are no more thread safe than any other construct. There is no special care taken to ensure writes or reads are thread safe, it is up to you to ensure that. Also is there a thread safe linked list somewhere? As far as I know, there are no builtin linked lists in D's standard library. Tango, an alternate standard lib for D1, has linked lists, and dcollections, a collection package I wrote for both D1 phobos and Tango, has linked lists, neither of which has thread safety built in. -Steve
Re: Threading
ah-ha! I was just writing to let you know that's how my project started out and I went into this issue, when I decided to try it again and got an error message I know how to solve. Probably got it before too, but working off an all-nighter and my caffeine hasn't fully kicked in yet. Turns out it's expecting an int testfunc(), the reason it wasn't working before was I was trying to use a uint testfunc() Thanks for the heads up on the I/O issue with threading, the writefln wasn't producing any output once I got it going and thanks to that I know what the issue with that was right away, luckily my actual project doesn't require I/O on a thread. (I'm writing a multi-threaded raytracer for a class) Which brings me to my next 2 questions, are dynamic arrays thread safe in D? I imagine they should be as long as I don't have two threads writing to the same index at once and the array size doesn't change. Also is there a thread safe linked list somewhere?
Re: Threading
On Mon, 11 May 2009 11:45:34 -0400, Brok3n Halo wrote: Thanks for the quick reply but I had the & to begain with and I was getting an error. I'm not at my computer right now so unfortunetly I can't see what it was, but I think it was something along the lines of thread expecting a function but that's a delegate. I'll try and find the time to set up the project on my laptop, and see what the error was for sure in the next few hours. ah, different issue. Your thread function needs to return void. Thread should accept either a function pointer or a delegate. So it should look like: void testfunc() And kill the return 0. -Steve
