[Lift] Re: Concurrency control of database access in Lift/Comet
Thanks Kevin, I know, that this is more of a theoretical problem, but now that I have read so much about Actors and concurrent programming, I am actually curious about the underlying concurrency strategies taken by Scala. Infact I realize, that this is actually more a Scala question, than a Lift one. But it is interesting for lift too, as it it builds upon Scalas actor model. So as far as I understand you suggest, that I take the concurrency problem over to the database and leave it up to the database to handle concurrency like in the following flow schema: Many open Comet connections - one thread per one or more client socket connections - one database connection per thread - concurrency handling in DB This approach however means that my application will generate many open connections to the database and therefore much connection (IPC-) overhead. Also by doing so it doesn't actually use the advantages of Scalas concurrency model. My approach was instead something like this: Many open Comet connections - one thread per one or more client socket connections - one thread receiving messages from all other threads putting them in a message queue - one database connection - DB access Concurrency in such model is handled by the message system of Scala, which AFAIK is thread safe and doesn't use locks?! The advantage would be, that I just would need one database connection, which to me sounds more efficient. I think my question can be drilled down to the following. How efficient is Scala's concurrency model? Do Scala Actors use locks in the underpinning or are concurrency operations atomic? (I read here http://www.ibm.com/developerworks/java/library/j-scala04109.html that Scala uses locks under the hood too so after all I could finish up with the same problems like with shared memory and mutexes?) Best regards Gregor On 4 Okt., 21:06, Kevin Wright kev.lee.wri...@googlemail.com wrote: In my experience, the database engine itself does a pretty good job of managing concurrent connections like this out of the box, which is much of the reason why connection pooling is so effective. Of course, thinks can be a bit interesting on the database side if you want to get really obsessive about performance, with possibilities such as tinkering with page sizes, locking strategies, and such like, but it's extremely rare for this to be a bottleneck in an application. However... For the example you've given, I'd just run a dedicated actor to persist chat messages. There's no need to wait for a message to be persisted before displaying it to the user, so asynchronous messages can really help out here. On Sun, Oct 4, 2009 at 6:41 PM, donfranciscodequevedo donfranciscodequev...@gmail.com wrote: I have read that the Lift framework supports the CometActor model. As far as I understand this is achieved by creating many threads out of some thread pool, each of which handles one or more client socket connection to a client. My question is, what kind of approach Lift takes to handle access from such threads to a shared object, e.g. a database? Many thread based applications use locks to access shared data, which however won't scale well. I read that better models would be timestamp ordering or multiversion concurrency control like e.g. used by CouchDB. Perhaps this is also handled automatically by the database and I don't have to bother about it at all from my application? Is it save to use a database connection from different threads? A simple example that came to my mind would be a Comet chat application, where one wants to save the communication to a database. How would the concurrent write requests from two threads to the database be handled in such case? Best regards Gregor --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Lift group. To post to this group, send email to liftweb@googlegroups.com To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/liftweb?hl=en -~--~~~~--~~--~--~---
[Lift] Re: Concurrency control of database access in Lift/Comet
Its really more of a Java problem, or JDBC to be specific. The normal way to configure this would be to establish a pool of connections, when a thread, or actor, needs to interact with the database it takes a connection from the pool, uses it, then returns it to the pool. This is the same regardless of how you approach concurrency (actors/locks/dataflow/etc.). When every connection in the pool is in use, subsequent requests will block until one becomes available, thus limiting the number of simultaneous requests to the DB. Tuning is then handled by adjusting the pool size to a value that delivers greatest throughput from the database. Depending on your exact requirements, it's then possible to build a layer on to of this that can take advantage of asynchronous messages to actors On Mon, Oct 5, 2009 at 12:50 PM, donfranciscodequevedo donfranciscodequev...@gmail.com wrote: Thanks Kevin, I know, that this is more of a theoretical problem, but now that I have read so much about Actors and concurrent programming, I am actually curious about the underlying concurrency strategies taken by Scala. Infact I realize, that this is actually more a Scala question, than a Lift one. But it is interesting for lift too, as it it builds upon Scalas actor model. So as far as I understand you suggest, that I take the concurrency problem over to the database and leave it up to the database to handle concurrency like in the following flow schema: Many open Comet connections - one thread per one or more client socket connections - one database connection per thread - concurrency handling in DB This approach however means that my application will generate many open connections to the database and therefore much connection (IPC-) overhead. Also by doing so it doesn't actually use the advantages of Scalas concurrency model. My approach was instead something like this: Many open Comet connections - one thread per one or more client socket connections - one thread receiving messages from all other threads putting them in a message queue - one database connection - DB access Concurrency in such model is handled by the message system of Scala, which AFAIK is thread safe and doesn't use locks?! The advantage would be, that I just would need one database connection, which to me sounds more efficient. I think my question can be drilled down to the following. How efficient is Scala's concurrency model? Do Scala Actors use locks in the underpinning or are concurrency operations atomic? (I read here http://www.ibm.com/developerworks/java/library/j-scala04109.html that Scala uses locks under the hood too so after all I could finish up with the same problems like with shared memory and mutexes?) Best regards Gregor On 4 Okt., 21:06, Kevin Wright kev.lee.wri...@googlemail.com wrote: In my experience, the database engine itself does a pretty good job of managing concurrent connections like this out of the box, which is much of the reason why connection pooling is so effective. Of course, thinks can be a bit interesting on the database side if you want to get really obsessive about performance, with possibilities such as tinkering with page sizes, locking strategies, and such like, but it's extremely rare for this to be a bottleneck in an application. However... For the example you've given, I'd just run a dedicated actor to persist chat messages. There's no need to wait for a message to be persisted before displaying it to the user, so asynchronous messages can really help out here. On Sun, Oct 4, 2009 at 6:41 PM, donfranciscodequevedo donfranciscodequev...@gmail.com wrote: I have read that the Lift framework supports the CometActor model. As far as I understand this is achieved by creating many threads out of some thread pool, each of which handles one or more client socket connection to a client. My question is, what kind of approach Lift takes to handle access from such threads to a shared object, e.g. a database? Many thread based applications use locks to access shared data, which however won't scale well. I read that better models would be timestamp ordering or multiversion concurrency control like e.g. used by CouchDB. Perhaps this is also handled automatically by the database and I don't have to bother about it at all from my application? Is it save to use a database connection from different threads? A simple example that came to my mind would be a Comet chat application, where one wants to save the communication to a database. How would the concurrent write requests from two threads to the database be handled in such case? Best regards Gregor --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Lift group. To post to this group, send email to liftweb@googlegroups.com To unsubscribe from this group, send email to
[Lift] Re: Concurrency control of database access in Lift/Comet
Howdy, I think you misunderstand how Actors work. An Actor only consumes resources while it is processing an item in its mailbox. So if an Actor is hanging out, doing nothing, it will only consume memory (like any other object.) When the Actor receives a message, it allocates some additional resources to process the message. Those resources always include a thread. In Lift, those resources also include an S scope. When S (session state) comes into scope, a connection to the RDBMS may be allocated (depending on you application's transaction policy) so that all RDBMS interactions that take place during that message processing will be part of a single RDBMS transaction. There are no resources allocated while a browser is waiting for a Comet event (except an NIO connection and a few objects allocated in memory). It's only when something happens that will cause HTML to go to the browser that events get put into mailboxes and RDBMS connections get pulled out of a pool. I hope this helps clear up the questions you had. Thanks, David On Sun, Oct 4, 2009 at 10:41 AM, donfranciscodequevedo donfranciscodequev...@gmail.com wrote: I have read that the Lift framework supports the CometActor model. As far as I understand this is achieved by creating many threads out of some thread pool, each of which handles one or more client socket connection to a client. My question is, what kind of approach Lift takes to handle access from such threads to a shared object, e.g. a database? Many thread based applications use locks to access shared data, which however won't scale well. I read that better models would be timestamp ordering or multiversion concurrency control like e.g. used by CouchDB. Perhaps this is also handled automatically by the database and I don't have to bother about it at all from my application? Is it save to use a database connection from different threads? A simple example that came to my mind would be a Comet chat application, where one wants to save the communication to a database. How would the concurrent write requests from two threads to the database be handled in such case? Best regards Gregor -- Lift, the simply functional web framework http://liftweb.net Beginning Scala http://www.apress.com/book/view/1430219890 Follow me: http://twitter.com/dpp Surf the harmonics --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Lift group. To post to this group, send email to liftweb@googlegroups.com To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/liftweb?hl=en -~--~~~~--~~--~--~---
[Lift] Re: Concurrency control of database access in Lift/Comet
In my experience, the database engine itself does a pretty good job of managing concurrent connections like this out of the box, which is much of the reason why connection pooling is so effective. Of course, thinks can be a bit interesting on the database side if you want to get really obsessive about performance, with possibilities such as tinkering with page sizes, locking strategies, and such like, but it's extremely rare for this to be a bottleneck in an application. However... For the example you've given, I'd just run a dedicated actor to persist chat messages. There's no need to wait for a message to be persisted before displaying it to the user, so asynchronous messages can really help out here. On Sun, Oct 4, 2009 at 6:41 PM, donfranciscodequevedo donfranciscodequev...@gmail.com wrote: I have read that the Lift framework supports the CometActor model. As far as I understand this is achieved by creating many threads out of some thread pool, each of which handles one or more client socket connection to a client. My question is, what kind of approach Lift takes to handle access from such threads to a shared object, e.g. a database? Many thread based applications use locks to access shared data, which however won't scale well. I read that better models would be timestamp ordering or multiversion concurrency control like e.g. used by CouchDB. Perhaps this is also handled automatically by the database and I don't have to bother about it at all from my application? Is it save to use a database connection from different threads? A simple example that came to my mind would be a Comet chat application, where one wants to save the communication to a database. How would the concurrent write requests from two threads to the database be handled in such case? Best regards Gregor --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Lift group. To post to this group, send email to liftweb@googlegroups.com To unsubscribe from this group, send email to liftweb+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/liftweb?hl=en -~--~~~~--~~--~--~---
