Re: [sqlalchemy] lockmode

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 12:35 AM, Michael Mileusnich wrote: It's just a basic select statement I do not see any other parms. SQL Server doesn't support FOR UPDATE unless in conjunction with DECLARE CURSOR which is not typical DBAPI usage so this feature is not supported by SQLAlchemy with MS-SQL

[sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Adrian Price-Whelan
Hi, I was just wondering at first whether there was a known issue with ARRAY types, but here is the query: spectra = session.query(Spectrum).join(SpectrumHeader).filter(SpectrumHeader.filename == spPlateFilename).all() It should return ~1000 objects equaling about 120MB of data. In Python, this

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 9:52 AM, Adrian Price-Whelan wrote: Hi, I was just wondering at first whether there was a known issue with ARRAY types, but here is the query: spectra = session.query(Spectrum).join(SpectrumHeader).filter(SpectrumHeader.filename == spPlateFilename).all() It should

Re: [sqlalchemy] problem with multi-table mapping - how to configure merge/cascade behaviour on mappers based on join/select?

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 10:24 AM, Harry Percival wrote: OK, so I will treat any classes mapped to a join of multiple tables as being a read-only API, and manually manage the write-API using relationship(). It doesn't look like I can define a relationship from the composite mapped class to

Re: [sqlalchemy] Graceful locking with sqlite

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 10:16 AM, Samuel wrote: Hi, I am trying to access (write to) an SQLite database from multiple threads (no ORM), resulting in the following error: OperationalError: (OperationalError) database is locked The engine is created using the default SingletonThreadPool. Is

[sqlalchemy] Re: how to run a stored procedure?

2010-07-19 Thread Lukasz Szybalski
On Jul 15, 3:35 pm, Lukasz Szybalski szybal...@gmail.com wrote: On Jul 15, 2:39 pm, David Gardner dgard...@creatureshop.com wrote: take a look at:http://www.sqlalchemy.org/docs/reference/sqlalchemy/expressions.html?... Now, Could you give me an example of it ? I understand how to run

[sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Adrian Price-Whelan
does it take a few seconds to fully fetch all the results and it only gets 1000 rows ? or is that just to get the initial result? I'm not sure what you mean by this - the query does return 1000 rows. also if any of the individual columns are very large BLOBs or perhaps very large PG

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 11:46 AM, Adrian Price-Whelan wrote: does it take a few seconds to fully fetch all the results and it only gets 1000 rows ? or is that just to get the initial result? I'm not sure what you mean by this - the query does return 1000 rows. if you run a query from a SQL

Re: [sqlalchemy] problem with multi-table mapping - how to configure merge/cascade behaviour on mappers based on join/select?

2010-07-19 Thread Harry Percival
Michael, thanks, as ever, for your help. So, I think I've managed to specify the relationships: j = join(movies_table,md_table).join(directors_table).join(genres_table) js = j.select(use_labels=True).alias('mdg') r0 = relationship(Movies,

Re: [sqlalchemy] problem with multi-table mapping - how to configure merge/cascade behaviour on mappers based on join/select?

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 12:04 PM, Harry Percival wrote: Michael, thanks, as ever, for your help. So, I think I've managed to specify the relationships: j = join(movies_table,md_table).join(directors_table).join(genres_table) js = j.select(use_labels=True).alias('mdg') r0 =

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread David Gardner
Try running that query directly against the database see how long that takes. Also try running explain on that query make sure it is using your indexes properly. Since you are only using a single filter make sure that the spectrum_header.filename has an index, and make sure your foreign key

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Adrian Price-Whelan
Here is some more detailed information trying the query multiple ways: Piping the command into psql and writing to a tmp file takes 12 seconds (tmp file is 241MB): time echo SELECT spectrum.pk AS spectrum_pk, spectrum.ra AS spectrum_ra, spectrum.dec AS spectrum_dec, spectrum.values AS

[sqlalchemy] Change __tablename__ at runtime?

2010-07-19 Thread Wade Leftwich
I've been using Sqlalchemy to help migrate a bunch of websites into and out of Drupal. Since a Drupal 'node' can involve fields from 10 or 12 tables, the Declarative approach has been a real timesaver. But now they're thrown me a serious curveball. It turns out that Drupal has a 'multisite' mode,

[sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Adrian Price-Whelan
Also, as a follow-up - inserting these ARRAYs into the database is very slow as well, slower than expected that is. Granted, it is looping over 1000 objects and inserting a few 4000 element arrays for each object, but doing one big SQL query takes considerably less time than session.add(object),

[sqlalchemy] Slides from the Advanced SQLAlchemy Customization tutorial at EuroPython

2010-07-19 Thread jason kirtland
Fellow Alchemers, I've posted the slides and code from the Advanced SQLAlchemy Customization tutorial I presented yesterday at EuroPython 2010 in Birmingham. Enjoy! http://discorporate.us/jek/talks/#d2010-07-18 Talk description: http://www.europython.eu/talks/talk_abstracts/#talk67 Cheers,

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 1:53 PM, Adrian Price-Whelan wrote: Here is some more detailed information trying the query multiple ways: Piping the command into psql and writing to a tmp file takes 12 seconds (tmp file is 241MB): time echo SELECT spectrum.pk AS spectrum_pk, spectrum.ra AS

Re: [sqlalchemy] Change __tablename__ at runtime?

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 1:56 PM, Wade Leftwich wrote: I've been using Sqlalchemy to help migrate a bunch of websites into and out of Drupal. Since a Drupal 'node' can involve fields from 10 or 12 tables, the Declarative approach has been a real timesaver. But now they're thrown me a serious

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Adrian Price-Whelan
First off, thanks for your quick replies! I will look into this, but I can tell you that the arrays are strictly numbers and the array columns are type numeric[] Thanks again, Adrian On Jul 19, 2010, at 3:47 PM, Michael Bayer wrote: On Jul 19, 2010, at 1:53 PM, Adrian Price-Whelan wrote:

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 3:55 PM, Adrian Price-Whelan wrote: First off, thanks for your quick replies! I will look into this, but I can tell you that the arrays are strictly numbers and the array columns are type numeric[] so its going to be doing that somewhat inefficient isinstance(list)

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread thatsanicehatyouhave
On Jul 19, 2010, at 4:08 PM, Michael Bayer wrote: so its going to be doing that somewhat inefficient isinstance(list) thing you see below, this appears to be how it handles arrays of arbitrary numbers of dimensions. This could be optimized if the ARRAY type accepted some clues as to how

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 4:38 PM, thatsanicehatyouh...@mac.com wrote: On Jul 19, 2010, at 4:08 PM, Michael Bayer wrote: so its going to be doing that somewhat inefficient isinstance(list) thing you see below, this appears to be how it handles arrays of arbitrary numbers of dimensions. This

[sqlalchemy] Re: how to run a stored procedure?

2010-07-19 Thread Lukasz Szybalski
) sqlalchemy.orm.exc.UnmappedClassError: Class ''20100719'' is not mapped Let me know, Thanks, Lucas -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread thatsanicehatyouhave
Hi Michael, Assuming I understood you correctly, I tried the code below. The result was the same (the query took 486 seconds). Since I autoload everything, I first adjust the column types to the class you defined. Did I misunderstand something? Thanks again for your help. Cheers, Demitri ---

Re: [sqlalchemy] Re: how to run a stored procedure?

2010-07-19 Thread Michael Bayer
=:end, params={'start':20100701,'end':20100719}) This gives the following error: session.execute(assp_ReportDailyTransactions @start_date=?, @end_date=?,20100701,20100719) Traceback (most recent call last): File stdin, line 1, in module File /home/lucas/tmp/sql2008/env/lib/python2.5/site

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 5:46 PM, thatsanicehatyouh...@mac.com wrote: Hi Michael, Assuming I understood you correctly, I tried the code below. The result was the same (the query took 486 seconds). Since I autoload everything, I first adjust the column types to the class you defined. Did I

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 5:46 PM, thatsanicehatyouh...@mac.com wrote: Hi Michael, Assuming I understood you correctly, I tried the code below. The result was the same (the query took 486 seconds). Since I autoload everything, I first adjust the column types to the class you defined. Did I

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Conor
On 07/19/2010 02:47 PM, Michael Bayer wrote: On Jul 19, 2010, at 1:53 PM, Adrian Price-Whelan wrote Here is some more detailed information trying the query multiple ways: Piping the command into psql and writing to a tmp file takes 12 seconds (tmp file is 241MB): time echo SELECT

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread thatsanicehatyouhave
Hi, Pasted below is a profile of the earlier code posted. I did update it with your new definition of ARRAY Michael, but that only shaved off 18 seconds (down to 468s total) when run without the profiler. The large number of __new__ calls roughly tracks with the number of numeric values

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Michael Bayer
On Jul 19, 2010, at 8:33 PM, thatsanicehatyouh...@mac.com wrote: Hi, Pasted below is a profile of the earlier code posted. I did update it with your new definition of ARRAY Michael, but that only shaved off 18 seconds (down to 468s total) when run without the profiler. The large

[sqlalchemy] Creating a custom Visitable

2010-07-19 Thread Mike Lewis
I'm interested in creating a visitable (more specifically, a SchemaItem) to support EXTENDS when creating tables. How would one go about this? Thanks, Mike -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to