[sqlalchemy] Unknown Issue Causing Extremely Slow Query

2010-07-16 Thread Adrian Price-Whelan
Hello --

I'm working with a database populated with data originally from a file 
structure of files that are ~150MB each. We are dealing with a lot of data that 
is being stored in the database using the 'ARRAY' datatype, specifically 
numeric[]. After loading some of the data into the database I tried performing 
a query to get back some data, and comparing it with code that reads directly 
from the file system - but the database query took ~50 times longer to 
complete. For instance, to retrieve 100 records that contain a few 4000 element 
arrays each using the code that reads the filesystem it took less than a 
second, but the query on the database took around 25 seconds to complete. Has 
anyone else had issues with array types slowing down queries or does this sound 
more like another issue?

Thanks!
Adrian

-- 
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en.



Re: [sqlalchemy] Unknown Issue Causing Extremely Slow Query

2010-07-16 Thread Michael Bayer
You absolutely need to turn in echoing and locate the specific SQL query which 
causes the issue.   Queries can take excessive time for a very wide variety of 
reasons.

On Jul 16, 2010, at 12:56 PM, Adrian Price-Whelan wrote:

 Hello --
 
 I'm working with a database populated with data originally from a file 
 structure of files that are ~150MB each. We are dealing with a lot of data 
 that is being stored in the database using the 'ARRAY' datatype, specifically 
 numeric[]. After loading some of the data into the database I tried 
 performing a query to get back some data, and comparing it with code that 
 reads directly from the file system - but the database query took ~50 times 
 longer to complete. For instance, to retrieve 100 records that contain a few 
 4000 element arrays each using the code that reads the filesystem it took 
 less than a second, but the query on the database took around 25 seconds to 
 complete. Has anyone else had issues with array types slowing down queries or 
 does this sound more like another issue?
 
 Thanks!
 Adrian
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 sqlalchemy group.
 To post to this group, send email to sqlalch...@googlegroups.com.
 To unsubscribe from this group, send email to 
 sqlalchemy+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/sqlalchemy?hl=en.
 

-- 
You received this message because you are subscribed to the Google Groups 
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to 
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/sqlalchemy?hl=en.