Re: [E-devel] Fwd: eo + eo base class + threads

[Jean-Philippe André]

On 19 April 2016 at 07:13, Cedric BAIL  wrote:

> On Fri, Apr 15, 2016 at 6:17 PM, Carsten Haitzler 
> wrote:
> > On Fri, 15 Apr 2016 13:40:41 -0700 Cedric BAIL 
> said:
> >> On Wed, Apr 13, 2016 at 4:37 PM, Carsten Haitzler  >
> >> wrote:
> >> > On Wed, 13 Apr 2016 11:16:58 -0700 Cedric BAIL 
> said:
> >> >> On Wed, Apr 13, 2016 at 3:47 AM, Carsten Haitzler <
> ras...@rasterman.com>
> >> >> wrote:
> >> >> > i'm wondering... shouldn't we make eo and eo base "threadsafe". ie
> a
> >> >> > basic eo object should have locks around things like:
> >> >>
> >> >> I am not to sure of the intended idea of making eo threadsafe for all
> >> >> of those function. If you are using the same object from two
> >> >> differents thread, it is pretty certain in my opinion that you will
> >> >> need to have lock to manage its use yourself or you will end up with
> >> >> crash. I don't think it is possible to build a threadsafe and safe to
> >> >> use API that won't require the user of that API to actually think
> >> >> about where to put lock. So if we advertise our API as threadsafe and
> >> >> the developer don't understand the limit of it (Like most Java
> >> >> developers for example), he will manage to have race condition and
> >> >> blame Eo for not being properly threadsafe. I believe it will result
> >> >> in a more complex use of eo, when can I rely on it being threadsafe,
> >> >> when can't I ? Which is why I don't think this is a good idea.
> >> >
> >> > you CANNOT build a threadsafe object ever if the base class isn't
> >> > threadsafe. i can NEVER do it. it's impossible to do.
> >>
> >> I do not believe it is possible to have a generic threadsafe object in
> >> C anyway. You need external synchronisation primitive or you will have
> >> race condition when passing parameter and returning value. Some
> >> primitive can be threadsafe, but the concept of threadsafety is
> >> unlikely to be broad.
> >
> > example:
> >
> > thread1:
> >
> > obj = eo_add(..);
> > eo_data_key_set(obj, "stuff", data);
> > send_msg(thread2, obj); // send msg to thread 2 to use obj
> > // some time later
> > for (;;) {
> >   eo_data_key_set(obj, "otherstuff", data2);
> >   sleep(1);
> >   datax = eo_data_key_get(obj, "otherstuff");
> >   eo_data_key_del(obj, "otherstuff");
> > }
> >
> > thread2:
> > wait_msg();
> > for (;;) {
> >   data3 = eo_data_get_get(obj, "stuff");
> >   process(data3);
> > }
>
> I guess, that sleep(1) is another synchronisation method in the
> example above (as is send_msg/wait_msg).
>
> > this above SHOULD work if obj was intended to be threadsafe. reality is
> we are
> > psetered day in and out to make efl threadsafe. all the time. reality is
> that
> > for gui studff this is nigh on impossible because of the model of
> keeping state
> > unrendered until going idle so changes need to be done within a lock of
> some
> > sort within the loop that owns the ui. most of efl is ui. but let's talk
> about
> > things like making the main loop an object so different threads can have
> > different loops. so you can do efl_loop_begin/end(mainloop); we have
> talked
> > about having inter-loop messaging similarly etc. this requires threadsafe
> > objects. if eo_base and below is not threadsafe then we are screwed in
> ever
> > being able to do this. you cannot have the above code which has no race
> > conditions at all because thread1 has guaranteed "stuff" is stored before
> > thread2 ever gets the message to use it and it doesnt ever mess with it
> again.
> > because keys are implemented using a list and a key lookup brings the
> key to
> > the front of the list (LRU) then 2 threads messing with the same list is
> going
> > to end up with a crash despite the programmer doing all the right things.
> > forcing them to create locks for every object separately to the object
> is just
> > poor poor poor design. that's another bit of data to remember, store,
> > delete/clean up when object is cleaned up and so on.
>
> As I have stated in my email, this is just making the internal kind of
> thread safe, but you are still opening huge synchronization issue that
> have to be addressed outside of the object. And this is exactly what
> your example show. So, this means that enabling internal thread safety
> per object, depend on that said object being useful in a thread or
> not. So why would we need that in the base object ?
>
> Remember we can always do in an inherited function :
>
> bob()
> {
>lock();
>super(bob());
>unlock();
>return ;
> }
>
> This will provide exactly the same kind of internal thread safety you
> are advocating without having to pay for its cost for object that
> don't need it.
>
> [snip]
>
> >> > i'm not saying to make all of efl threadsafe. but base class and core
> eo
> >> > stuff like the object table have to be to allow someone to build
> threadsafe
> >> > objects on top.
> >> >
> >> > we 

Re: [E-devel] Fwd: eo + eo base class + threads

[Cedric BAIL]

On Fri, Apr 15, 2016 at 6:17 PM, Carsten Haitzler  wrote:
> On Fri, 15 Apr 2016 13:40:41 -0700 Cedric BAIL  said:
>> On Wed, Apr 13, 2016 at 4:37 PM, Carsten Haitzler 
>> wrote:
>> > On Wed, 13 Apr 2016 11:16:58 -0700 Cedric BAIL  said:
>> >> On Wed, Apr 13, 2016 at 3:47 AM, Carsten Haitzler 
>> >> wrote:
>> >> > i'm wondering... shouldn't we make eo and eo base "threadsafe". ie a
>> >> > basic eo object should have locks around things like:
>> >>
>> >> I am not to sure of the intended idea of making eo threadsafe for all
>> >> of those function. If you are using the same object from two
>> >> differents thread, it is pretty certain in my opinion that you will
>> >> need to have lock to manage its use yourself or you will end up with
>> >> crash. I don't think it is possible to build a threadsafe and safe to
>> >> use API that won't require the user of that API to actually think
>> >> about where to put lock. So if we advertise our API as threadsafe and
>> >> the developer don't understand the limit of it (Like most Java
>> >> developers for example), he will manage to have race condition and
>> >> blame Eo for not being properly threadsafe. I believe it will result
>> >> in a more complex use of eo, when can I rely on it being threadsafe,
>> >> when can't I ? Which is why I don't think this is a good idea.
>> >
>> > you CANNOT build a threadsafe object ever if the base class isn't
>> > threadsafe. i can NEVER do it. it's impossible to do.
>>
>> I do not believe it is possible to have a generic threadsafe object in
>> C anyway. You need external synchronisation primitive or you will have
>> race condition when passing parameter and returning value. Some
>> primitive can be threadsafe, but the concept of threadsafety is
>> unlikely to be broad.
>
> example:
>
> thread1:
>
> obj = eo_add(..);
> eo_data_key_set(obj, "stuff", data);
> send_msg(thread2, obj); // send msg to thread 2 to use obj
> // some time later
> for (;;) {
>   eo_data_key_set(obj, "otherstuff", data2);
>   sleep(1);
>   datax = eo_data_key_get(obj, "otherstuff");
>   eo_data_key_del(obj, "otherstuff");
> }
>
> thread2:
> wait_msg();
> for (;;) {
>   data3 = eo_data_get_get(obj, "stuff");
>   process(data3);
> }

I guess, that sleep(1) is another synchronisation method in the
example above (as is send_msg/wait_msg).

> this above SHOULD work if obj was intended to be threadsafe. reality is we are
> psetered day in and out to make efl threadsafe. all the time. reality is that
> for gui studff this is nigh on impossible because of the model of keeping 
> state
> unrendered until going idle so changes need to be done within a lock of some
> sort within the loop that owns the ui. most of efl is ui. but let's talk about
> things like making the main loop an object so different threads can have
> different loops. so you can do efl_loop_begin/end(mainloop); we have talked
> about having inter-loop messaging similarly etc. this requires threadsafe
> objects. if eo_base and below is not threadsafe then we are screwed in ever
> being able to do this. you cannot have the above code which has no race
> conditions at all because thread1 has guaranteed "stuff" is stored before
> thread2 ever gets the message to use it and it doesnt ever mess with it again.
> because keys are implemented using a list and a key lookup brings the key to
> the front of the list (LRU) then 2 threads messing with the same list is going
> to end up with a crash despite the programmer doing all the right things.
> forcing them to create locks for every object separately to the object is just
> poor poor poor design. that's another bit of data to remember, store,
> delete/clean up when object is cleaned up and so on.

As I have stated in my email, this is just making the internal kind of
thread safe, but you are still opening huge synchronization issue that
have to be addressed outside of the object. And this is exactly what
your example show. So, this means that enabling internal thread safety
per object, depend on that said object being useful in a thread or
not. So why would we need that in the base object ?

Remember we can always do in an inherited function :

bob()
{
   lock();
   super(bob());
   unlock();
   return ;
}

This will provide exactly the same kind of internal thread safety you
are advocating without having to pay for its cost for object that
don't need it.

[snip]

>> > i'm not saying to make all of efl threadsafe. but base class and core eo
>> > stuff like the object table have to be to allow someone to build threadsafe
>> > objects on top.
>> >
>> > we already want to make the main loop "thread safe" in that one thread will
>> >
>> > mainloop.begin();
>> > or
>> > efl_loop_beging(mainloop);
>>
>> This call has no synchronisation issue as long as eo internal code to
>> resolve function is threadsafe, because it doesn't influence the
>> parameter, nor 

Re: [E-devel] Fwd: eo + eo base class + threads

[The Rasterman]

On Fri, 15 Apr 2016 13:40:41 -0700 Cedric BAIL  said:

> On Wed, Apr 13, 2016 at 4:37 PM, Carsten Haitzler 
> wrote:
> > On Wed, 13 Apr 2016 11:16:58 -0700 Cedric BAIL  said:
> >> On Wed, Apr 13, 2016 at 3:47 AM, Carsten Haitzler 
> >> wrote:
> >> > i'm wondering... shouldn't we make eo and eo base "threadsafe". ie a
> >> > basic eo object should have locks around things like:
> >>
> >> I am not to sure of the intended idea of making eo threadsafe for all
> >> of those function. If you are using the same object from two
> >> differents thread, it is pretty certain in my opinion that you will
> >> need to have lock to manage its use yourself or you will end up with
> >> crash. I don't think it is possible to build a threadsafe and safe to
> >> use API that won't require the user of that API to actually think
> >> about where to put lock. So if we advertise our API as threadsafe and
> >> the developer don't understand the limit of it (Like most Java
> >> developers for example), he will manage to have race condition and
> >> blame Eo for not being properly threadsafe. I believe it will result
> >> in a more complex use of eo, when can I rely on it being threadsafe,
> >> when can't I ? Which is why I don't think this is a good idea.
> >
> > you CANNOT build a threadsafe object ever if the base class isn't
> > threadsafe. i can NEVER do it. it's impossible to do.
> 
> I do not believe it is possible to have a generic threadsafe object in
> C anyway. You need external synchronisation primitive or you will have
> race condition when passing parameter and returning value. Some
> primitive can be threadsafe, but the concept of threadsafety is
> unlikely to be broad.

example:

thread1:

obj = eo_add(..);
eo_data_key_set(obj, "stuff", data);
send_msg(thread2, obj); // send msg to thread 2 to use obj
// some time later
for (;;) {
  eo_data_key_set(obj, "otherstuff", data2);
  sleep(1);
  datax = eo_data_key_get(obj, "otherstuff");
  eo_data_key_del(obj, "otherstuff");
}

thread2:
wait_msg();
for (;;) {
  data3 = eo_data_get_get(obj, "stuff");
  process(data3);
}

this above SHOULD work if obj was intended to be threadsafe. reality is we are
psetered day in and out to make efl threadsafe. all the time. reality is that
for gui studff this is nigh on impossible because of the model of keeping state
unrendered until going idle so changes need to be done within a lock of some
sort within the loop that owns the ui. most of efl is ui. but let's talk about
things like making the main loop an object so different threads can have
different loops. so you can do efl_loop_begin/end(mainloop); we have talked
about having inter-loop messaging similarly etc. this requires threadsafe
objects. if eo_base and below is not threadsafe then we are screwed in ever
being able to do this. you cannot have the above code which has no race
conditions at all because thread1 has guaranteed "stuff" is stored before
thread2 ever gets the message to use it and it doesnt ever mess with it again.
because keys are implemented using a list and a key lookup brings the key to
the front of the list (LRU) then 2 threads messing with the same list is going
to end up with a crash despite the programmer doing all the right things.
forcing them to create locks for every object separately to the object is just
poor poor poor design. that's another bit of data to remember, store,
delete/clean up when object is cleaned up and so on.

> > simple stuff like i am setting data keys in one thread and getting their
> > value in another...
> 
> Will always be a race condition if you don't have any synchronisation
> primitive between both side. The parameter of your function call and
> the returned value are always, by construction, outside of any
> potential lock, making it impossible to be threadsafe. We tried that
> with ecore and we all know it failed for that exact reason.

no you don't. see above.

> > i'm not saying to make all of efl threadsafe. but base class and core eo
> > stuff like the object table have to be to allow someone to build threadsafe
> > objects on top.
> >
> > we already want to make the main loop "thread safe" in that one thread will
> >
> > mainloop.begin();
> > or
> > efl_loop_beging(mainloop);
> 
> This call has no synchronisation issue as long as eo internal code to
> resolve function is threadsafe, because it doesn't influence the
> parameter, nor the return value and that's all we need and can do !
> Oh, and eo_ref/unref should already be threadsafe, but anyway in the
> example above, the mainloop should be refcounted when a thread get
> created and unref when it get destroyed. Otherwise the above code will
> always be race.

no - see above. if we dont lock simple things like key_data_set/get/del then
the object data structure internals get messed up. i'm not talking about the
input params or returns. i'm talking about the object internal state is