On Jun 1, 2007, at 10:27 PM, Mike Orr wrote:
> Multiple engines make my head spin. Why do you need that unless > you're connecting to two different databases in the same application? thats what it would be needed for, yes. > And even if you did, wouldn't you define a second top-level > session_context to bind it to, with its own different metadata and > tables, and never the twain shall mix? two separate sessions ? you might have a set of objects which are saved to more than one database, but are dealt with in one transaction. the SessionTransaction actually performs a poor-mans two-phase commit in this case, where it executes the flush() statements for each database before calling commit() on all of them. a ticket in trac exists which contains part of a patch that will wire in actual two-phase commit hooks for databases which support it directly (like postgres). while you might have the tables from the distinct databases associated with different MetaData objects, that doesnt change the fact that the session needs to know about the engines associated with each MetaData. but then again, the engine->MetaData binding idea is not really needed at all, except for the convenience of self- executing statement objects without needing to pass around an engine or a connection. > I don't want a registry of > engines or something in the 'g' object. What I want is a simple > out-of-the box configuration for simple sites, but more robust than > what Pylons currently has. Maybe we'll have to come up with separate > simple and advanced configurations if others need multiple engines and > whatnot. i think an object tailored to Pylons should be created here which encapsulates those concepts. > > I also don't like how pylons.database initializes a session_context at > import time rather than providing a create_session_context function, > so that's another thing to add to my patch. Otherwise if you can't > use pylons.database.create_engine() for some reason, you have to > duplicate a lot of code to recreate or bypass the default > session_context, and this includes writing stub functions because > SessionContext takes a session factory function with takes a > create_engine function, so there's no way to customize the > create_engine from the SessionContext constructor directly. yeah actually i think id just skip pylons.database altogether if i were writing a new app, and just make my own engine / session context. we're talking three lines of code here. > > Methinks SQLAlchemy is contributing to the problem with its long > hierarchy of engine -> metadata -> session -> session_context, but I > don't have the expertise to say what might be better. But certainly > it's annoying that: > > - SessionContext doesn't take both create_engine and make_session > arguments, or arguments to pass through to those, and build your ideal > engine -> session -> session_context hierarchy on the fly. Instead > you have to create a dummy make_session function just to tell it which > create_engine to use. This is part of why overriding Pylons' default > session_context requires reimplementing three whole functions. OK well i dont even like that SessionContext requires a session creation function at all, since for many cases its just the default create_session function. this whole deal with the session binding to the engine wasnt widely used before pylons. i would think the most obvious arguments wouldnt be create_engine, but just "engine", wouldnt it ? > > - This is a part of the previous, but SessionContext in the manual > says, "A common customization is a Session which needs to explicitly > bind to a particular Engine." Yes, so why doesn't SQLAlchemy provide > a way to handle this common case without the user having to define his > own make_session? Again, SessionContext -- or a > create_session_context function -- could do this for you if you pass > an 'engine' argument. oh ok, there we go. i dont have a problem with that. Dan (SessionContext's creator) hasnt been chiming in lately so i guess we can change his API a little bit :) Also the notion that SessionContext should be combined with Session is somethign to look at, although they are separate at the moment so that users can first get familiar with Session, and then when they realize they need to keep one of them around within threads they can then get their head around SessionContext. by combining it all together people would just be more overwhelmed and follow various flawed recipes more blindly than they do now :). > > - Metadata seems like an implementation detail. I have to define a > metadata just to pass it to my Table's, then I never use it again. its used for locating other tables when a ForeignKey is detected, as well as sorting them all for a create_all/drop_all. so a context of some kind is needed. > There is global_connect() which hides it, but its use seems > discouraged. Plus global_connect gets into all that DynamicMetaData > complication, such as whether it will autoconnect properly in the > other threads. Perhaps what I'm asking for is a global_bound_connect > or something? yes the whole global thing was something i never liked. I dont like application wide configurations that are bound at the module level. people are likely to get confused pretty quick when they start putting multiple applications, or sub-applications, within the same python interpreter and start getting name collisions and such. by exposing MetaData and Engine, the developer has to define the scope of these things by his or herself, and thats how it should be. I cringe when i see people using global_connect(). DynamicMetaData was then inspired exactly by this: http://www.sqlobject.org/class-sqlobject.dbconnection.ConnectionHub.html and it wasnt my idea at all, it was TG folks looking to adapt ConnectionHub to sqlalchemy. it originally began as a ProxyEngine, back when Tables were bound straight to the Engine. But these days, SA is a lot more high level than that, we have Session with its own engine, etc. so DMD is not so relevant now (unless youre doing lots of constructed SQL with thread-local engines). > I can see why the metadata can't be subsumed into the > engine because you may want to connect the same metadata back and > forth if you're copying data from one database to another. And I can > see why it can't be subsumed into create_session_context because you > need it before that to autoload the tables. So it's kind of the > centerpiece that holds those two together and also holds the tables. > So maybe making the metadata less obtrusive is insolvable. maybe a better interface would have been this: meta = MetaData() users = meta.create_table("users", Column("user_id")...) that would have circumvented the emotional impact of metadata being an argument rather than the factory. I was going for a particular look when i came up with Table/Column, i..e that it would look like regular SQL DDL. > > I don't know whether the answer is to collapse the hierarchy so it's > not function-calling-function-calling-function or what. Probably that > would be too disruptive to existing users and make SQLAlchemy less > flexible/useful. But I just wonder how much of this hierarchy is > really necessary/useful, and how much should just be subsumed in an > uber one-session-context-to-rule-them-all like pylons.database does. well Pylons has raised the bar a little by going for the session.bind_to=theengine idea, which was decidedly less common than the usual BoundMetaData idea. it is more flexible particularly for an application that shares the same set of Table objects among multiple databases, which is what Ben was very much going for, i.e. an application that supports multiple hosted accounts, each account having its own private database schema. I dont know if he remembers but he was IM'ing me about that all day long many months ago :). > > I'm only doing ctx.current.bind_to because Pylons makes that easy, and > because as a naive user I wouldn't know whether it's safe to stray > away from that recommendation. Users know not to share a session > between threads, that they should use session_context even though they > don't quite understand what it does, but they can get confused whether > an engine or metadata or anything else that's accessed through a > session_context is compatible with one declared or used at top level > in another thread. > > I would be in favor of a more straightforward approach in my model: > > engine = create_engine(...) > meta = BoundMetaData(engine) > ctx = create_session_context(engine, meta, session_options) > like I said we can make a nicer SessionContext constructor. but above, im not sure how much sense it makes to have BoundMetaData combined with the engine attached to the session directly. the most major issue im seeing in this area is that people are binding the engine twice like that and its for no reason, it creates confusion. if you have BoundMetaData, youre done with the engine. sessioncontext/session doesnt need it, it will use it from the tables directly. I think what we have to look at re: pylons is its insistence on even using the session(bind_to=engine) model in the first place. if it just used BoundMetaData, you can just create sessions freely without binding them to anything. the notion of "multple engines sharing the same Table objects" is just one particular execution model which i think is not at all the norm. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
