entirely different machines, then youd have to partition out rows from the table yourself. you'd select some range of rows using LIMIT/ OFFSET on each machine to be processed. the exact count would depend on the total rows in the table and the total number of machines.
On Feb 25, 2009, at 8:50 AM, Daniel wrote: > > Hi, > > your suggestion sounds really reasonable but the point is that these > processes are running on different machines and I don't want to put a > lot of effort in synchronizing these threads. > > Isn't there any easy solution for having multiple processes working on > the same database table? > > Thanks. > > Cheers > > Am Dienstag, den 24.02.2009, 07:46 -0800 schrieb Michael Bayer: >> I would select out the total set of rows and then hand off groups of >> those rows, converted into serializable objects first, using the >> "imap" function of a multiprocessing.Pool object. It would be best >> if the rows are returned via a ResultProxy so that work can begin on >> results before all results are fetched. >> >> See >> http://docs.python.org/library/multiprocessing.html#module-multiprocessing.pool >> . Its critical that you call create_engine() within each >> subprocess if additional queries are to be issued in workers, and >> don't pass any other database resources like connections or cursors >> across. Pool allows initialization activities via the "initializer" >> argument. >> >> On Feb 24, 7:05 am, dagooglaa <danielso...@gmx.de> wrote: >>> Hi, >>> >>> I am using SQLAlchemy 0.4.8 with Python 2.4. >>> >>> I have a process which is selecting some rows, doing something with >>> the data of these rows and then writes a result into the database. >>> >>> How can I achieve a solution where multiple of these processes are >>> running without having them selecting the same rows and doing the >>> same >>> stuff? >>> >>> Normally I would suggest s.th. like Select for update or an trigger >>> which sets a timestamp after a select... >>> >>> Do you have any suggestions for having multiple processes working on >>> the same table but not on the same rows? >>> >>> Thanks >> > > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
