Re: [Factor-talk] Multithreading in Factor

[John Benediktsson]

>
>
> - is it possible to start the second Factor instance from the first
> one? I guess this is possible using run-detached, but maybe there is
> something Factor specific.
>

Phil Dawes spent some time working on a patchset to allow starting multiple
Factor VM's in their own threads.  You can see a test for it here:


https://github.com/slavapestov/factor/blob/master/extra/native-thread-test/native-thread-test.factor

I'm not sure if we merged all of his patches, but you can search his name
on the mailing list and there is a couple links on the issue I filed to
look into this:

https://github.com/slavapestov/factor/issues/1161

I imagine you want to build something a bit like the Python multiprocessing
module.  It shouldn't be that hard to do, but might require building a few
things. The server framework is pretty flexible and well tested (
factorcode.org, for example, runs on a Factor http server).  We have
serialization code, a few examples of RPC, but you might have to put it all
together for your needs.  Would make a great contribution!  Concurrency is
one of priorities after releasing a small 0.97.1.

Best,
John.



>
> Just to clarify, my end goal would be to be able to launch a few
> worker processes, say a fixed multiple of the number of available
> cores, and have a Factor frontend that delegates work to them. Servers
> are becoming more and more multicore, and it is hard to sell Factor
> without being able to make good use of them.
>
>
> 2014-11-21 0:05 GMT+01:00 Jon Harper :
> > Just to confirm that the following sequence of actions work (ie print
> > "hello" in the first factor instance)
> >
> > $ ./factor # first factor  process
> > USING: concurrency.distributed threads io.sockets io.servers ;
> > "127.0.0.1" 9000   start-server
> > FROM: concurrency.messaging => receive ;
> > : log-message ( -- ) receive . flush log-message ;
> > [ log-message ] "logger" spawn dup name>> register-remote-thread
> >
> >
> > $ ./factor # second factor process
> > USING: io.sockets concurrency.messaging concurrency.distributed ;
> > FROM: concurrency.messaging => send ;
> > "hello" "127.0.0.1" 9000  "logger"  send
> >
> > Jon
> >
> >
> > Jon
> >
> > On Thu, Nov 20, 2014 at 10:09 PM, Chris Double <
> chris.dou...@double.co.nz>
> > wrote:
> >>
> >> On Thu, Nov 20, 2014 at 8:24 AM, Andrea Ferretti
> >>  wrote:
> >> > I am trying to make this work, but I have issues with the line
> >> >
> >> > "myhost.com" 9001 start-server
> >>
> >> Looks like the API for starting servers/nodes has changed quite a bit.
> >> If you look at the following file you'll see examples of current
> >> usage:
> >>
> >> basis/concurrency/distributed/distributed-tests.factor
> >>
> >> --
> >> http://bluishcoder.co.nz
> >>
> >>
> >>
> --
> >> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> >> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> >> with Interactivity, Sharing, Native Excel Exports, App Integration &
> more
> >> Get technology previously reserved for billion-dollar corporations, FREE
> >>
> >>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> >> ___
> >> Factor-talk mailing list
> >> Factor-talk@lists.sourceforge.net
> >> https://lists.sourceforge.net/lists/listinfo/factor-talk
> >
> >
> >
> >
> --
> > Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> > from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> > with Interactivity, Sharing, Native Excel Exports, App Integration & more
> > Get technology previously reserved for billion-dollar corporations, FREE
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> > ___
> > Factor-talk mailing list
> > Factor-talk@lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/factor-talk
> >
>
>
> --
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk
>
--
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & 

Re: [Factor-talk] Multithreading in Factor

[Andrea Ferretti]

I think the missing point for me was . I knew how to
start a few server examples, but in general  requires
an encoding and a few parameters for configuration. What was not
obvious for me was how to create a server that would work with Factor
distributed message passing (for instance I had no idea what encoding
to choose because I don't know how serialization is performed).
 was what I needed.

I have a few more doubts:
- if I understand correctly, Factor has both an actor model and a CSP
implementation, the first one using named thread on processes - so
that you can send messages asynchronously provided you know the
address of the recipient - and the second one using channels, which
are a point of synchronization and decouple sender and receiver. Is
this correct?
- is it possible to start the second Factor instance from the first
one? I guess this is possible using run-detached, but maybe there is
something Factor specific.

Just to clarify, my end goal would be to be able to launch a few
worker processes, say a fixed multiple of the number of available
cores, and have a Factor frontend that delegates work to them. Servers
are becoming more and more multicore, and it is hard to sell Factor
without being able to make good use of them.


2014-11-21 0:05 GMT+01:00 Jon Harper :
> Just to confirm that the following sequence of actions work (ie print
> "hello" in the first factor instance)
>
> $ ./factor # first factor  process
> USING: concurrency.distributed threads io.sockets io.servers ;
> "127.0.0.1" 9000   start-server
> FROM: concurrency.messaging => receive ;
> : log-message ( -- ) receive . flush log-message ;
> [ log-message ] "logger" spawn dup name>> register-remote-thread
>
>
> $ ./factor # second factor process
> USING: io.sockets concurrency.messaging concurrency.distributed ;
> FROM: concurrency.messaging => send ;
> "hello" "127.0.0.1" 9000  "logger"  send
>
> Jon
>
>
> Jon
>
> On Thu, Nov 20, 2014 at 10:09 PM, Chris Double 
> wrote:
>>
>> On Thu, Nov 20, 2014 at 8:24 AM, Andrea Ferretti
>>  wrote:
>> > I am trying to make this work, but I have issues with the line
>> >
>> > "myhost.com" 9001 start-server
>>
>> Looks like the API for starting servers/nodes has changed quite a bit.
>> If you look at the following file you'll see examples of current
>> usage:
>>
>> basis/concurrency/distributed/distributed-tests.factor
>>
>> --
>> http://bluishcoder.co.nz
>>
>>
>> --
>> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
>> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
>> with Interactivity, Sharing, Native Excel Exports, App Integration & more
>> Get technology previously reserved for billion-dollar corporations, FREE
>>
>> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
>> ___
>> Factor-talk mailing list
>> Factor-talk@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/factor-talk
>
>
>
> --
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk
>

--
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Multithreading in Factor

[Jon Harper]

Just to confirm that the following sequence of actions work (ie print
"hello" in the first factor instance)

$ ./factor # first factor  process
USING: concurrency.distributed threads io.sockets io.servers ;
"127.0.0.1" 9000   start-server
FROM: concurrency.messaging => receive ;
: log-message ( -- ) receive . flush log-message ;
[ log-message ] "logger" spawn dup name>> register-remote-thread


$ ./factor # second factor process
USING: io.sockets concurrency.messaging concurrency.distributed ;
FROM: concurrency.messaging => send ;
"hello" "127.0.0.1" 9000  "logger"  send

Jon


Jon

On Thu, Nov 20, 2014 at 10:09 PM, Chris Double 
wrote:

> On Thu, Nov 20, 2014 at 8:24 AM, Andrea Ferretti
>  wrote:
> > I am trying to make this work, but I have issues with the line
> >
> > "myhost.com" 9001 start-server
>
> Looks like the API for starting servers/nodes has changed quite a bit.
> If you look at the following file you'll see examples of current
> usage:
>
> basis/concurrency/distributed/distributed-tests.factor
>
> --
> http://bluishcoder.co.nz
>
>
> --
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk
>
--
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Multithreading in Factor

[John D. Mitchell]

If you're talking about use in a single master process + multiple threads than 
check out the LMAX disruptor approach for inter-thread communication.
 
Cheers,
John

On Oct 18, 2014, at 08:30 , Andrea Ferretti  wrote:

> Thank you for you response. Regarding the possibility if delegating
> CPU intensive tasks to multiple tasks: what about communication? Is
> there an idiomatic way to make processes communicate without blocking?
> There is ZeroMQ, but as far as I know it is blocking. It would be nice
> to develop something like Erlang/Akka using a pool of processes and
> cooperative multithreading locally inside each process, but I am not
> sure what would be the best way to keep a channel among processes
> 
> 2014-10-15 14:40 GMT+02:00 Björn Lindqvist :
>> Hi Andrea,
>> 
>> I'm not an expert, so take what's written below with a grain of
>> salt. It mostly comes from what I've snapped up from varius places and
>> reading the mailing list archive (eg
>> http://search.gmane.org/?query=threading&group=comp.lang.factor.general&sort=relevance).
>> 
>> 2014-10-13 18:14 GMT+02:00 Andrea Ferretti :
>>> Hi, I have read in various places, including this mailing list, that
>>> Factor does not currently have support for (preemptive, kernel-level)
>>> multithreading, and that adding support for that would require a great
>>> deal of changes, since the underlying VM and many core words are not
>>> thread-safe.
>> 
>> It's true that Factor doesn't have preemptive, kernel-level threading
>> (I'll just write "threading" from now on when I refer to this
>> particular brand of threading) and that adding it would probably
>> require a lot of engineering and restructuring work. But if you dig
>> into the source, and read the previous discussions, it's clear that
>> adding threading was always the idea and the Factor VM has been
>> architected to make it simple to add in the future.
>> 
>> Whether it actually is simple or not, is a different matter. But it's
>> much different from, say, the CPython VM which is implemented in such
>> a way that it would be virtually impossible to add threading.
>> 
>> Most composite (non-primitive) words are thread-safe and the primitive
>> words are only "thread-unsafe" in that critical sections aren't
>> guarded by exclusion locks.
>> 
>>> Can anyone expand on this? Is there some place where people have
>>> collected some ideas about the issues that would arise and the areas
>>> that need work?
>> 
>> I don't think there is any particularly bloody issues. It's just a lot
>> of hard work.
>> 
>> For example interactions between threads and gc can be very tricky. If
>> two threads need to allocate memory, there needs to be some
>> synchronization so that they don't end up pointing to the same chunk
>> of memory. How do you make that both fast and safe?
>> 
>> What happens with a threads object references if another thread forces
>> a gc cycle? I guess all threads has to stop running during the gc so
>> that all object references can be updated. It's even more complicated
>> if one thread is in an ffi function which holds a pointer to a Factor
>> object.
>> 
>> What if another thread recompiles the same word a thread is running?
>> 
>>> If not, it would be nice to gather such information from people
>>> knowledgeable about the internals of factor, so that interested people
>>> could start make some contributions.
>>> 
>>> I don't know if personally I would be able to contribute, but I'd love
>>> to if I I found something I could handle
>> 
>> An alien wrapper for pthreads would be interesting. I've no idea if it
>> would kind of work or break Factor badly, non-the-less it would be
>> interesting to see.
>> 
>> While we're at the subject of threading.. It's a great feature but a
>> language VM can do without it and still be very useful. Python,
>> Erlang, Node.js and Ruby all proves that. If the goal of the
>> concurrency is IO throughput, then cooperative threads which Factor
>> already has works really well. For cpu intensive tasks you can often
>> start multiple processes instead of threading.
>> 
>> 
>> 
>> --
>> mvh/best regards Björn Lindqvist
>> 
>> --
>> Comprehensive Server Monitoring with Site24x7.
>> Monitor 10 servers for $9/Month.
>> Get alerted through email, SMS, voice calls or mobile push notifications.
>> Take corrective actions from your mobile device.
>> http://p.sf.net/sfu/Zoho
>> ___
>> Factor-talk mailing list
>> Factor-talk@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/factor-talk
> 
> --
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> 

Re: [Factor-talk] Multithreading in Factor

[John Benediktsson]

I wrote a few posts about servers, here is one simple echo server example:

http://re-factor.blogspot.com/2012/08/echo-server.html

Also, this is a "telnet" server implemented fairly simply:


https://github.com/mrjbq7/re-factor/blob/master/telnet-server/telnet-server.factor





On Thu, Nov 20, 2014 at 1:09 PM, Chris Double 
wrote:

> On Thu, Nov 20, 2014 at 8:24 AM, Andrea Ferretti
>  wrote:
> > I am trying to make this work, but I have issues with the line
> >
> > "myhost.com" 9001 start-server
>
> Looks like the API for starting servers/nodes has changed quite a bit.
> If you look at the following file you'll see examples of current
> usage:
>
> basis/concurrency/distributed/distributed-tests.factor
>
> --
> http://bluishcoder.co.nz
>
>
> --
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk
>
--
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Multithreading in Factor

[Chris Double]

On Thu, Nov 20, 2014 at 8:24 AM, Andrea Ferretti
 wrote:
> I am trying to make this work, but I have issues with the line
>
> "myhost.com" 9001 start-server

Looks like the API for starting servers/nodes has changed quite a bit.
If you look at the following file you'll see examples of current
usage:

basis/concurrency/distributed/distributed-tests.factor

-- 
http://bluishcoder.co.nz

--
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=157005751&iu=/4140/ostg.clktrk
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Multithreading in Factor

[Andrea Ferretti]

I am trying to make this work, but I have issues with the line

"myhost.com" 9001 start-server

in the documentation. start-server takes a threaded server as input,
and this just does not run as is. I could try with a threaded server,
but I am not sure how to configure it, and what encoding to use.

I am also slightly confused about:
1) whether to use remote channels or remote threads to implement messaging
2) whether I can spawn a second Factor instance from Factor itself or
if I should rather start it manually

Thank you
Andrea

2014-10-19 5:12 GMT+02:00, Chris Double :
> ZeroMQ has a non-blocking mode. You can get a file descriptor that can
> be passed to select/epoll/kqueue. Factor has its own channel and
> serialization system to. See:
>
> 
>
> On Sun, Oct 19, 2014 at 4:30 AM, Andrea Ferretti
>  wrote:
>> Thank you for you response. Regarding the possibility if delegating
>> CPU intensive tasks to multiple tasks: what about communication? Is
>> there an idiomatic way to make processes communicate without blocking?
>> There is ZeroMQ, but as far as I know it is blocking. It would be nice
>> to develop something like Erlang/Akka using a pool of processes and
>> cooperative multithreading locally inside each process, but I am not
>> sure what would be the best way to keep a channel among processes
>>
>> 2014-10-15 14:40 GMT+02:00 Björn Lindqvist :
>>> Hi Andrea,
>>>
>>> I'm not an expert, so take what's written below with a grain of
>>> salt. It mostly comes from what I've snapped up from varius places and
>>> reading the mailing list archive (eg
>>> http://search.gmane.org/?query=threading&group=comp.lang.factor.general&sort=relevance).
>>>
>>> 2014-10-13 18:14 GMT+02:00 Andrea Ferretti :
 Hi, I have read in various places, including this mailing list, that
 Factor does not currently have support for (preemptive, kernel-level)
 multithreading, and that adding support for that would require a great
 deal of changes, since the underlying VM and many core words are not
 thread-safe.
>>>
>>> It's true that Factor doesn't have preemptive, kernel-level threading
>>> (I'll just write "threading" from now on when I refer to this
>>> particular brand of threading) and that adding it would probably
>>> require a lot of engineering and restructuring work. But if you dig
>>> into the source, and read the previous discussions, it's clear that
>>> adding threading was always the idea and the Factor VM has been
>>> architected to make it simple to add in the future.
>>>
>>> Whether it actually is simple or not, is a different matter. But it's
>>> much different from, say, the CPython VM which is implemented in such
>>> a way that it would be virtually impossible to add threading.
>>>
>>> Most composite (non-primitive) words are thread-safe and the primitive
>>> words are only "thread-unsafe" in that critical sections aren't
>>> guarded by exclusion locks.
>>>
 Can anyone expand on this? Is there some place where people have
 collected some ideas about the issues that would arise and the areas
 that need work?
>>>
>>> I don't think there is any particularly bloody issues. It's just a lot
>>> of hard work.
>>>
>>> For example interactions between threads and gc can be very tricky. If
>>> two threads need to allocate memory, there needs to be some
>>> synchronization so that they don't end up pointing to the same chunk
>>> of memory. How do you make that both fast and safe?
>>>
>>> What happens with a threads object references if another thread forces
>>> a gc cycle? I guess all threads has to stop running during the gc so
>>> that all object references can be updated. It's even more complicated
>>> if one thread is in an ffi function which holds a pointer to a Factor
>>> object.
>>>
>>> What if another thread recompiles the same word a thread is running?
>>>
 If not, it would be nice to gather such information from people
 knowledgeable about the internals of factor, so that interested people
 could start make some contributions.

 I don't know if personally I would be able to contribute, but I'd love
 to if I I found something I could handle
>>>
>>> An alien wrapper for pthreads would be interesting. I've no idea if it
>>> would kind of work or break Factor badly, non-the-less it would be
>>> interesting to see.
>>>
>>> While we're at the subject of threading.. It's a great feature but a
>>> language VM can do without it and still be very useful. Python,
>>> Erlang, Node.js and Ruby all proves that. If the goal of the
>>> concurrency is IO throughput, then cooperative threads which Factor
>>> already has works really well. For cpu intensive tasks you can often
>>> start multiple processes instead of threading.
>>>
>>>
>>>
>>> --
>>> mvh/best regards Björn Lindqvist
>>>
>>> --
>>> Comprehensive Server Monitorin

Re: [Factor-talk] Multithreading in Factor

[Chris Double]

ZeroMQ has a non-blocking mode. You can get a file descriptor that can
be passed to select/epoll/kqueue. Factor has its own channel and
serialization system to. See:



On Sun, Oct 19, 2014 at 4:30 AM, Andrea Ferretti
 wrote:
> Thank you for you response. Regarding the possibility if delegating
> CPU intensive tasks to multiple tasks: what about communication? Is
> there an idiomatic way to make processes communicate without blocking?
> There is ZeroMQ, but as far as I know it is blocking. It would be nice
> to develop something like Erlang/Akka using a pool of processes and
> cooperative multithreading locally inside each process, but I am not
> sure what would be the best way to keep a channel among processes
>
> 2014-10-15 14:40 GMT+02:00 Björn Lindqvist :
>> Hi Andrea,
>>
>> I'm not an expert, so take what's written below with a grain of
>> salt. It mostly comes from what I've snapped up from varius places and
>> reading the mailing list archive (eg
>> http://search.gmane.org/?query=threading&group=comp.lang.factor.general&sort=relevance).
>>
>> 2014-10-13 18:14 GMT+02:00 Andrea Ferretti :
>>> Hi, I have read in various places, including this mailing list, that
>>> Factor does not currently have support for (preemptive, kernel-level)
>>> multithreading, and that adding support for that would require a great
>>> deal of changes, since the underlying VM and many core words are not
>>> thread-safe.
>>
>> It's true that Factor doesn't have preemptive, kernel-level threading
>> (I'll just write "threading" from now on when I refer to this
>> particular brand of threading) and that adding it would probably
>> require a lot of engineering and restructuring work. But if you dig
>> into the source, and read the previous discussions, it's clear that
>> adding threading was always the idea and the Factor VM has been
>> architected to make it simple to add in the future.
>>
>> Whether it actually is simple or not, is a different matter. But it's
>> much different from, say, the CPython VM which is implemented in such
>> a way that it would be virtually impossible to add threading.
>>
>> Most composite (non-primitive) words are thread-safe and the primitive
>> words are only "thread-unsafe" in that critical sections aren't
>> guarded by exclusion locks.
>>
>>> Can anyone expand on this? Is there some place where people have
>>> collected some ideas about the issues that would arise and the areas
>>> that need work?
>>
>> I don't think there is any particularly bloody issues. It's just a lot
>> of hard work.
>>
>> For example interactions between threads and gc can be very tricky. If
>> two threads need to allocate memory, there needs to be some
>> synchronization so that they don't end up pointing to the same chunk
>> of memory. How do you make that both fast and safe?
>>
>> What happens with a threads object references if another thread forces
>> a gc cycle? I guess all threads has to stop running during the gc so
>> that all object references can be updated. It's even more complicated
>> if one thread is in an ffi function which holds a pointer to a Factor
>> object.
>>
>> What if another thread recompiles the same word a thread is running?
>>
>>> If not, it would be nice to gather such information from people
>>> knowledgeable about the internals of factor, so that interested people
>>> could start make some contributions.
>>>
>>> I don't know if personally I would be able to contribute, but I'd love
>>> to if I I found something I could handle
>>
>> An alien wrapper for pthreads would be interesting. I've no idea if it
>> would kind of work or break Factor badly, non-the-less it would be
>> interesting to see.
>>
>> While we're at the subject of threading.. It's a great feature but a
>> language VM can do without it and still be very useful. Python,
>> Erlang, Node.js and Ruby all proves that. If the goal of the
>> concurrency is IO throughput, then cooperative threads which Factor
>> already has works really well. For cpu intensive tasks you can often
>> start multiple processes instead of threading.
>>
>>
>>
>> --
>> mvh/best regards Björn Lindqvist
>>
>> --
>> Comprehensive Server Monitoring with Site24x7.
>> Monitor 10 servers for $9/Month.
>> Get alerted through email, SMS, voice calls or mobile push notifications.
>> Take corrective actions from your mobile device.
>> http://p.sf.net/sfu/Zoho
>> ___
>> Factor-talk mailing list
>> Factor-talk@lists.sourceforge.net
>> https://lists.sourceforge.net/lists/listinfo/factor-talk
>
> --
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile devic

Re: [Factor-talk] Multithreading in Factor

[Andrea Ferretti]

Thank you for you response. Regarding the possibility if delegating
CPU intensive tasks to multiple tasks: what about communication? Is
there an idiomatic way to make processes communicate without blocking?
There is ZeroMQ, but as far as I know it is blocking. It would be nice
to develop something like Erlang/Akka using a pool of processes and
cooperative multithreading locally inside each process, but I am not
sure what would be the best way to keep a channel among processes

2014-10-15 14:40 GMT+02:00 Björn Lindqvist :
> Hi Andrea,
>
> I'm not an expert, so take what's written below with a grain of
> salt. It mostly comes from what I've snapped up from varius places and
> reading the mailing list archive (eg
> http://search.gmane.org/?query=threading&group=comp.lang.factor.general&sort=relevance).
>
> 2014-10-13 18:14 GMT+02:00 Andrea Ferretti :
>> Hi, I have read in various places, including this mailing list, that
>> Factor does not currently have support for (preemptive, kernel-level)
>> multithreading, and that adding support for that would require a great
>> deal of changes, since the underlying VM and many core words are not
>> thread-safe.
>
> It's true that Factor doesn't have preemptive, kernel-level threading
> (I'll just write "threading" from now on when I refer to this
> particular brand of threading) and that adding it would probably
> require a lot of engineering and restructuring work. But if you dig
> into the source, and read the previous discussions, it's clear that
> adding threading was always the idea and the Factor VM has been
> architected to make it simple to add in the future.
>
> Whether it actually is simple or not, is a different matter. But it's
> much different from, say, the CPython VM which is implemented in such
> a way that it would be virtually impossible to add threading.
>
> Most composite (non-primitive) words are thread-safe and the primitive
> words are only "thread-unsafe" in that critical sections aren't
> guarded by exclusion locks.
>
>> Can anyone expand on this? Is there some place where people have
>> collected some ideas about the issues that would arise and the areas
>> that need work?
>
> I don't think there is any particularly bloody issues. It's just a lot
> of hard work.
>
> For example interactions between threads and gc can be very tricky. If
> two threads need to allocate memory, there needs to be some
> synchronization so that they don't end up pointing to the same chunk
> of memory. How do you make that both fast and safe?
>
> What happens with a threads object references if another thread forces
> a gc cycle? I guess all threads has to stop running during the gc so
> that all object references can be updated. It's even more complicated
> if one thread is in an ffi function which holds a pointer to a Factor
> object.
>
> What if another thread recompiles the same word a thread is running?
>
>> If not, it would be nice to gather such information from people
>> knowledgeable about the internals of factor, so that interested people
>> could start make some contributions.
>>
>> I don't know if personally I would be able to contribute, but I'd love
>> to if I I found something I could handle
>
> An alien wrapper for pthreads would be interesting. I've no idea if it
> would kind of work or break Factor badly, non-the-less it would be
> interesting to see.
>
> While we're at the subject of threading.. It's a great feature but a
> language VM can do without it and still be very useful. Python,
> Erlang, Node.js and Ruby all proves that. If the goal of the
> concurrency is IO throughput, then cooperative threads which Factor
> already has works really well. For cpu intensive tasks you can often
> start multiple processes instead of threading.
>
>
>
> --
> mvh/best regards Björn Lindqvist
>
> --
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> ___
> Factor-talk mailing list
> Factor-talk@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/factor-talk

--
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

Re: [Factor-talk] Multithreading in Factor

[Björn Lindqvist]

Hi Andrea,

I'm not an expert, so take what's written below with a grain of
salt. It mostly comes from what I've snapped up from varius places and
reading the mailing list archive (eg
http://search.gmane.org/?query=threading&group=comp.lang.factor.general&sort=relevance).

2014-10-13 18:14 GMT+02:00 Andrea Ferretti :
> Hi, I have read in various places, including this mailing list, that
> Factor does not currently have support for (preemptive, kernel-level)
> multithreading, and that adding support for that would require a great
> deal of changes, since the underlying VM and many core words are not
> thread-safe.

It's true that Factor doesn't have preemptive, kernel-level threading
(I'll just write "threading" from now on when I refer to this
particular brand of threading) and that adding it would probably
require a lot of engineering and restructuring work. But if you dig
into the source, and read the previous discussions, it's clear that
adding threading was always the idea and the Factor VM has been
architected to make it simple to add in the future.

Whether it actually is simple or not, is a different matter. But it's
much different from, say, the CPython VM which is implemented in such
a way that it would be virtually impossible to add threading.

Most composite (non-primitive) words are thread-safe and the primitive
words are only "thread-unsafe" in that critical sections aren't
guarded by exclusion locks.

> Can anyone expand on this? Is there some place where people have
> collected some ideas about the issues that would arise and the areas
> that need work?

I don't think there is any particularly bloody issues. It's just a lot
of hard work.

For example interactions between threads and gc can be very tricky. If
two threads need to allocate memory, there needs to be some
synchronization so that they don't end up pointing to the same chunk
of memory. How do you make that both fast and safe?

What happens with a threads object references if another thread forces
a gc cycle? I guess all threads has to stop running during the gc so
that all object references can be updated. It's even more complicated
if one thread is in an ffi function which holds a pointer to a Factor
object.

What if another thread recompiles the same word a thread is running?

> If not, it would be nice to gather such information from people
> knowledgeable about the internals of factor, so that interested people
> could start make some contributions.
>
> I don't know if personally I would be able to contribute, but I'd love
> to if I I found something I could handle

An alien wrapper for pthreads would be interesting. I've no idea if it
would kind of work or break Factor badly, non-the-less it would be
interesting to see.

While we're at the subject of threading.. It's a great feature but a
language VM can do without it and still be very useful. Python,
Erlang, Node.js and Ruby all proves that. If the goal of the
concurrency is IO throughput, then cooperative threads which Factor
already has works really well. For cpu intensive tasks you can often
start multiple processes instead of threading.



--
mvh/best regards Björn Lindqvist

--
Comprehensive Server Monitoring with Site24x7.
Monitor 10 servers for $9/Month.
Get alerted through email, SMS, voice calls or mobile push notifications.
Take corrective actions from your mobile device.
http://p.sf.net/sfu/Zoho
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk

[Factor-talk] Multithreading in Factor

[Andrea Ferretti]

Hi, I have read in various places, including this mailing list, that
Factor does not currently have support for (preemptive, kernel-level)
multithreading, and that adding support for that would require a great
deal of changes, since the underlying VM and many core words are not
thread-safe.

Can anyone expand on this? Is there some place where people have
collected some ideas about the issues that would arise and the areas
that need work?

If not, it would be nice to gather such information from people
knowledgeable about the internals of factor, so that interested people
could start make some contributions.

I don't know if personally I would be able to contribute, but I'd love
to if I I found something I could handle

Andrea

--
Meet PCI DSS 3.0 Compliance Requirements with EventLog Analyzer
Achieve PCI DSS 3.0 Compliant Status with Out-of-the-box PCI DSS Reports
Are you Audit-Ready for PCI DSS 3.0 Compliance? Download White paper
Comply to PCI DSS 3.0 Requirement 10 and 11.5 with EventLog Analyzer
http://p.sf.net/sfu/Zoho
___
Factor-talk mailing list
Factor-talk@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/factor-talk