[sqlalchemy] Re: Slightly confusing error when session set up incorrectly
On Jun 29, 7:30 pm, Michael Bayer wrote: > On Jun 29, 2011, at 12:24 PM, Ben Sizer wrote: > >OK, this happens a lot with me, if anyone can please recommend what university >I should go to in order to learn to speak english correctlyHere's the >sentence: It's not incorrect, just ambiguous. I expect most people would have read it the way you intended it. But this wasn't my point and it's not directly relevant to my issue: whether it maintains an open connection for each engine or each request doesn't matter - what is important is that it won't make any attempt to connect at all until this point. It's arguably implied, but not explicitly stated. > > There is a similar implication > > in the 'What does the Session do?' paragraph also. > > Not seeing the "implication" in that one, please add some details to ticket > 2204 as to the verbiage you find confusing (log in as guest/guest). The part I meant is: "The Session begins in an essentially stateless form. Once queries are issued or other objects are persisted with it, it requests a connection resource from an Engine that is associated either with the Session itself or with the mapped Table objects being operated upon." This part isn't confusing, it's just that again it does not explicitly state that no connection happens at the start. Saying "When X occurs, Y also occurs" isn't logically the same as "Before X occurs, Y never occurs". So I've not added this to the ticket because I don't this anything about this needs to change. I have just been suggesting something extra to be added in the session config/creation docs: eg., "Note, this does not actually connect to the database yet. Only when your first query is issued will a physical connection be made." > >>> I expect I am not unusual > >>> in wanting an error in session configuration to fail as soon as > >>> possible, > > >> Nobody has ever raised this issue before to my recollection. > > > The problem with show-stopper bugs is that people often just give up > > and switch to something else rather than tell you about it. ;) > > "show stopper bug" is quite an exaggeration over what is essentially a small > documentation insufficiency. To you (and now to me), it's a small documentation insufficiency, because we know what the problem was. But to me 24 hours ago, this was a problem that stopped my app running entirely. > So there is a class of users, and a large one at that, that experience bugs > both large and small and don't immediately abandon the project. This class of > users continues to expand as the project has grown much stronger over the > course of many years, responding to user requests, competing products, etc. > There are of course users who abandon the project within 5 minutes, and I > would say if their level of tenacity is that low then they're probably better > off with whatever tool they end up using. Yes, of course. sqlalchemy is the best product of its kind, which is why I am here! I was just pointing out that sometimes it's the stuff that -isn't- reported that is actually really important for user retention. It just happens to be one of the things we tracked on the last product I worked on, and I got an eye-opening view into how most users stop using a product for reasons completely different to the ones people complain about. :) My level of tenacity is pretty low, I'll admit! But, I already fixed my bug, and with your help also understand the fix. I just came here to try and help others who might hit the same thing. If you don't think my suggestions help, that's fine. I do accept your point about not being able to adjust the docs to suit every individual user. > > [...] that means there would be 4 ways > > of approaching this, which seems a little much. > > I'm not sure what the "four ways" are here. As a user who has no real restriction on configuration, where my use case is simply, "connect to a database and get a session for it", it appears there are 4 different ways I can do that, and the docs tell me how to do all these but don't really explain why, meaning I was not sure whether I was using the wrong approach or a sub-optimal one. 1) import Session directly from sqlalchemy.orm.session and bind that to the engine with the keyword argument on creation. -- "Session is a regular Python class which can be directly instantiated." 2) call sessionmaker() to get a factory for unbound Session subclasses, and call .configure(bind=whatever) to bind it. -- "You can also associate a Engine with an existing sessionmaker() using the sessionmaker.configure() method" 3) call sessionmaker() to get a factory for unbound Session subclasses, and bind each to the engine with the keyword argument on creation. -- "you can also associate individual Session objects with an Engine on each invocation" 4) call sessionmaker(bind=some_engine) to get a factory for bound Sessions and create an already-bound instance. "However, to standardize how sessions are configured and
Re: [sqlalchemy] Re: Slightly confusing error when session set up incorrectly
On Jun 29, 2011, at 12:24 PM, Ben Sizer wrote: > >> the lazy initializing behavior of the Session is documented: >> >> http://www.sqlalchemy.org/docs/orm/session.html#unitofwork-transaction > > Not exactly. It says it maintains a connection for each SQL statement, OK, this happens a lot with me, if anyone can please recommend what university I should go to in order to learn to speak english correctlyHere's the sentence: As the Session receives requests to execute SQL statements using a particular Engine or Connection, it adds each individual Engine encountered to its transactional state and maintains an open connection for each one (note that a simple application normally has just one Engine). You're taking "each one" to mean, "the SQL statement". When I wrote it, "each one" means, "each Engine". Ticket http://www.sqlalchemy.org/trac/ticket/2204 has been added. > There is a similar implication > in the 'What does the Session do?' paragraph also. Not seeing the "implication" in that one, please add some details to ticket 2204 as to the verbiage you find confusing (log in as guest/guest). > >> But that doesn't have anything to do with the Engine, which is its own >> thing. As far as the Engine, the docs currently use the term "connect" in >> conjunction with create_engine() which is for simplicities' sake, but is >> technically inaccurate, perhaps come up with some term other than "connect", >> "configure a connection source" perhaps. We can add an explicit sentence >> to the top ofhttp://www.sqlalchemy.org/docs/core/connections.html. > >>> I expect I am not unusual >>> in wanting an error in session configuration to fail as soon as >>> possible, >> >> Nobody has ever raised this issue before to my recollection. > > The problem with show-stopper bugs is that people often just give up > and switch to something else rather than tell you about it. ;) "show stopper bug" is quite an exaggeration over what is essentially a small documentation insufficiency. There's more than 25000 messages on the Google Groups list as well as several thousand on previous mailing lists as well as the several thousand tickets in trac, virtually all of which originate from a user experiencing a bug either in SQLAlchemy, in its documentation, or within his or her own usage of the product. The vast majority of these issues have been resolved and those users went away happy. So there is a class of users, and a large one at that, that experience bugs both large and small and don't immediately abandon the project. This class of users continues to expand as the project has grown much stronger over the course of many years, responding to user requests, competing products, etc.There are of course users who abandon the project within 5 minutes, and I would say if their level of tenacity is that low then they're probably better off with whatever tool they end up using. SQLAlchemy is more about a big payoff for some upfront investment. > >> I suppose inefficiency is the only issue but its so unnecessarily >> inefficient, most people would consider it to be wrong behavior.A >> Session may be configured such that depending on what's requested of it, it >> can connect to any number of different engines - connecting to >> several/dozens/hundreds of engines unconditionally upon construction is not >> an option. > > Yeah, that's fine. I just think it would be good to have had something > about this documented right in at the top of the Session docs so that I think the specifics of how Session interacts with engines at a more detailed level is appropriate to be farther down the page, while language can be adjusted higher up as to not introduce misunderstandings. It's common that we get a user who had some kind of problem, that user got frustrated, then they request that very specific documentation for their specific issue be placed as some prominent paragraph somewhere.Unfortunately if we did that every time, the docs would be an enormous list of bulletpoints with no narrative at all. With user confusion, we always take into account the problem they had and that thinking goes into subsequent documentation revisions. >> >> They exist because the intended usage is that sessionmaker() be placed as a >> global variable in the module space of an application - it is then often the >> case that the actual Engine does not exist at this point, so cannot be >> passed to the sessionmaker(). A later configuration step can then call >> configure() to add new Engine configurations.This usage is driven by the >> needs of web frameworks and such. > > Ok, so: >Session = sessionmaker(bind=some_engine) is for when you know > which engine you need at import time; >Session.configure(bind=engine) is when you have decided on the > engine and want all subsequent sessions to use that engine, without > needing to keep a reference to that engi
[sqlalchemy] Re: Slightly confusing error when session set up incorrectly
On Jun 29, 6:46 am, Michael Bayer wrote: > On Jun 28, 2011, at 9:34 PM, Ben Sizer wrote: > > > What does the "None None" signify? Would it be possible to change this > > exception to be a bit more descriptive and a little less cryptic? eg. > > Include the URI that failed? > > That error is raised by SQLite and we just propagate it out. Ok, that's a bit annoying, but it does sound like something that is not your problem to solve. Maybe if I get time I'll contact the pysqlite maintainers as I think it would be useful to have a pseudo- statement in there at least. > > Additionally, I notice that sqlalchemy doesn't attempt to make an > > actual connection to the database until you perform the first query. > > That is true, all connections/transactions are lazy initializing. Sorry > this was a surprise, though I don't think this is much of an issue once you > get used to dealing with lazy initializing objects. > > The formal pattern at play with SQLAlchemy's connection pool, Engine and > Session behavior is called the Proxy Pattern, a decent description is at > http://sourcemaking.com/design_patterns/proxy. Yeah, that's all fine. I don't have a problem with this choice, just that it's not immediately obvious when following the docs. > the lazy initializing behavior of the Session is documented: > > http://www.sqlalchemy.org/docs/orm/session.html#unitofwork-transaction Not exactly. It says it maintains a connection for each SQL statement, which is not logically equivalent to saying there is no connection unless there has been an SQL statement. There is a similar implication in the 'What does the Session do?' paragraph also. > But that doesn't have anything to do with the Engine, which is its own thing. > As far as the Engine, the docs currently use the term "connect" in > conjunction with create_engine() which is for simplicities' sake, but is > technically inaccurate, perhaps come up with some term other than "connect", > "configure a connection source" perhaps. We can add an explicit sentence to > the top ofhttp://www.sqlalchemy.org/docs/core/connections.html. I must admit I don't fully understand engines, connections, sessions, etc. I will study the docs further! However when writing manual DB code I would usually have a step where I initialise the driver (which I had been thinking of as the create_engine stage), a step where I can issue statements (which I think of as the session stage), and a point temporally between those where I explicitly connect to the physical storage, which I had thought was automatically taking place when creating the engine but I now realise only happens on demand. It's not a problem as such, just a surprise. > > I expect I am not unusual > > in wanting an error in session configuration to fail as soon as > > possible, > > Nobody has ever raised this issue before to my recollection. The problem with show-stopper bugs is that people often just give up and switch to something else rather than tell you about it. ;) (Of course in this case, the bug is mine, but developers do like to blame their tools...) > I'm not sure lots of users are phased whether the stack trace starts at the > Session.configure() line or if it starts later as soon as their first unit > test tries to hit the database - they get the same error, see that the > connection URL is bad, and fix it. But this is exactly the problem: I had one unit test for setting up the database, and one for actually using the database. The first one passed - the second one failed. So I immediately think, "my usage of the database is wrong", not "I set up the database connection wrongly", because all the code for setting up the connection was covered the test that passed, too. That is what the unit tests were there for, after all - to separate out where a problem appears to be coming from, by splitting your code coverage across tests. > I suppose inefficiency is the only issue but its so unnecessarily > inefficient, most people would consider it to be wrong behavior. A Session > may be configured such that depending on what's requested of it, it can > connect to any number of different engines - connecting to > several/dozens/hundreds of engines unconditionally upon construction is not > an option. Yeah, that's fine. I just think it would be good to have had something about this documented right in at the top of the Session docs so that anyone starting out or writing unit tests for this sort of thing knows that they're not actually testing the db connection just because they made an engine and Session without error. Anywhere you pass in invalid data, you hope that the system catches that as soon as possible so that the problem's narrowed down, ideally to the exact statement that is incorrect. In this case, it's not efficient to do so, therefore it would be great for the docs to explicitly highlight that fact. > Feel free to suggest what verbiage you're looking for, it's not hitting me > strong
