i'm answering the arhitectural part of your question. pylons is around MVC (model view controller) approach. u have squezeed model into the controller which isnt _bad as such but not appropriate for any complexier/larger modelling. controllers are about translating between model and view, input and output. so u need to separate the data-model off your controllers - they should only call methods of your model objects. And the model objects should care about sqlalchemy etc (well, given overall context, like sessions etc). i would further separate a bit the db-side from model, esp. as u have multi-db, caching etc things that model should not care about. your notion of data-manager is something about this last persistency layer. be ware, splitting a well-mixed-and-stirred (model+db+controller) into separates may be a tricky task here-there - u'll need to invent certain interfaces inbetween, starting with bridges/ adapters/ proxies/... and ending with completely new notions to allow for separate lifetimes/meanings. maybe u can get around by just separating the persistency, but that is postponing the real job for some other time - unless this is once-off thing, with no much future development.
ciao svilen www.svilendobrev.com On Monday 06 October 2008 04:48:07 Krishgy wrote: > Hi All, > > We are using sqlalchemy with Pylons project. > > Currently, pylons controllers are directly using the sqlalchemy for > handling the request. Due to our bad query handling approach, > contents and algorithm computation, our application works very > slow even for single user :-(. I can use the indexing, > lazy_loading to improve the performance. > > All our logic are put into the controllers, so our controller code > base increasing rapidly. > > I want to have a separate data manager to manage our web site data. > The datamanager shall be a python module which can use sqlalchemy, > memcached, and/or ( hadoop in later stage). > > Primary responsibilities of the data manager are > > 1. Manage the sqlalchemy to store and retrieve the data, DB > connection pooling > 2. Store the data into the memcached whenever appropriate. > 3. Use our custom twisted server to calculate the site statistics > such as content votes, user reputation, content ranking, > personalization 4. Use our custom twisted server to pre-build the > python object whenever there is a data change in the particular > model and save the python object to the Berkeley database. So we > will serve the data mostly from berkeley database which is ready to > be displayed. > > I will post my finding towards this approach here. > > My specific questions are, > > 1. The approach I have taken is good or bad? > 2. Is session = meta.Session() creates a heavy or light weight > object? 3. What is the best way to release the session object > returned by meta.Session() > 4. Can session object handle the connection polling in case our > master database died? Can it fallback to the slave database > automatically? 5. Do we really need a specific data manager? Can > you recommend pros and cons of having a specific data manager > module? > > I am sorry, my post is too big... Thank you so much for your help.. > > Thanks & Regards, > > Krish > > --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
