On Jun 16, 2013, at 12:55 PM, Kevin S <kevinrst...@gmail.com> wrote:
> I can try to get another dbapi installed later this week and see if that > works. However, I had to jump through some hoops just to get pysybase working > in the first place, so I'm not terribly looking forward to trying to tackle > another one. > > I don't know much about how sessions are managed (I believe flask creates > scoped-session objects). Could it be something that is just not implemented > in the pysybase sqlalchemy dialect, but available in the dbapi? I'm not sure > exactly what to look for. not really. The DBAPI is a very simple API, it's pretty much mostly execute(), rollback(), and commit(). We have a test suite that runs against pysybase as well, it certainly has a lot of glitches, not the least of which is that pysybase last time I checked could not handle non-ASCII data in any way. If pysybase is halting the entire intepreter on a query, there's nothing about the DBAPI in the abstract which refers to that. It sounds like pysybase probably grabs the GIL on execute() while waiting for results, which would be pretty bad. Perhaps it has settings, either run time or compile time, which can modify its behavior in this regard. If it were me, I'd probably seek some way to not produce a web application directly against a Sybase database, as the severe lack of driver support will probably lead to many unsolvable scaling issues. I'd look to mirror the Sybase data in some other more modern system, either another RDBMS or a more cache-like system like Redis. > > On Saturday, June 15, 2013 3:33:36 PM UTC-4, Michael Bayer wrote: > > On Jun 14, 2013, at 3:18 PM, Kevin S <kevin...@gmail.com> wrote: > > > I am running into a problem while developing a flask application using > > flask-sqlalchemy. Now, I'm not even 100% sure my problem is sqlalchemy > > related, but I don't know how to debug this particular issue. > > > > To start, I have a sybase database that I want to see if I can build a > > report generating application for. The reports will all be custom SQL > > queries that are requested by our users, and they will be able to refresh > > throughout the day as they edit and clean up their data (we focus on a lot > > of data curation). We plan to do other things that merit the use of an ORM, > > and we have a lot of complex relationships. Anyway, that's why I'm first > > trying to get this to work in our flask + sqlalchemy stack. And it does > > work in fact. > > > > Now the problem is, my current application is not scalable, because any > > time I do a long query (say several seconds or more), flask will not accept > > any additional requests until that query finishes. (Note: I am running the > > application through cherrypy). I have tested various things to ensure that > > the application can handle multiple incoming requests. If I have it just > > loop through a big file, or even just sleep instead of doing a query, then > > I can bang away at it all I want from other browser windows, and it's fine. > > > > We also have a copy of our database that is in postgres (this is only for > > testing, and can't be a final solution, because it gets updated only once a > > week). So, I've found that if I hook the application up to the postgres > > version, I don't have this problem. I can initiate a long query in one > > browser tab, and any other page requests in subsequent windows come back > > fine. The problem is only when using Sybase. We have other applications > > that are not flask or sqlalchemy, and they don't seem to have this > > limitation. As far as I can tell, I've narrowed it down to as soon as it > > executes a query. The entire app will wait until that query finishes, not > > allowing any new connections. I have log statements in my request handlers, > > and even in my before_request method, and those will not print a thing > > until the moment that first query returns. > > > > Additional info: I am using Sybase 15 with the pysybase driver. > > I initiate the raw SQL queries like this: > > > > con = db.session.connection() > > results = con.execute(query) > > > > But I also see the same problem if I use object relationships via > > Object.query.all() or whatever. > > > > I don't expect anyone to specifically know about this sybase driver, but > > I'm wondering what more can I do to try to debug this? I'm mostly > > interested in figuring out where the limitation is coming from, i.e. is it > > the database, the driver, or the way I'm using the session. I can provide > > additional details if needed. > > well it's not a pooling issue because you don't have the issue with > Postgresql, so its a Sybase driver issue. you'd need to see if you can boil > down this same behavior to a single Python test script that uses the Sybase > DBAPI directly. > > Though that might only manage to prove its the Sybase DBAPI, and im not sure > how much those drivers are being supported. Have you tried a different > DBAPI ? > > > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to sqlalchemy+unsubscr...@googlegroups.com. > To post to this group, send email to sqlalchemy@googlegroups.com. > Visit this group at http://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/groups/opt_out. > > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at http://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/groups/opt_out.
