[sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Adrian Price-Whelan
and locate the specific SQL query which causes the issue.   Queries can take excessive time for a very wide variety of reasons. On Jul 16, 2010, at 12:56 PM, Adrian Price-Whelan wrote: Hello -- I'm working with a database populated with data originally from a file structure

[sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Adrian Price-Whelan
, 2010, at 9:52 AM, Adrian Price-Whelan wrote: Hi, I was just wondering at first whether there was a known issue with ARRAY types, but here is the query: spectra = session.query(Spectrum).join(SpectrumHeader).filter(SpectrumHeader.filename == spPlateFilename).all() It should return

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Adrian Price-Whelan
that the spectrum_header.filename has an index, and make sure your foreign key column spectrum.spectrum_header_pk is indexed as well. On 07/19/2010 08:46 AM, Adrian Price-Whelan wrote: does it take a few seconds to fully fetch all the results and it only gets 1000 rows ? or is that just to get

[sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Adrian Price-Whelan
), session.commit(). I thought that might be another clue, thanks! On Jul 19, 1:53 pm, Adrian Price-Whelan adrian@gmail.com wrote: Here is some more detailed information trying the query multiple ways: Piping the command into psql and writing to a tmp file takes 12 seconds (tmp file is 241MB

Re: [sqlalchemy] Re: Unknown Issue Causing Extremely Slow Query

2010-07-19 Thread Adrian Price-Whelan
First off, thanks for your quick replies! I will look into this, but I can tell you that the arrays are strictly numbers and the array columns are type numeric[] Thanks again, Adrian On Jul 19, 2010, at 3:47 PM, Michael Bayer wrote: On Jul 19, 2010, at 1:53 PM, Adrian Price-Whelan wrote

[sqlalchemy] Unknown Issue Causing Extremely Slow Query

2010-07-16 Thread Adrian Price-Whelan
Hello -- I'm working with a database populated with data originally from a file structure of files that are ~150MB each. We are dealing with a lot of data that is being stored in the database using the 'ARRAY' datatype, specifically numeric[]. After loading some of the data into the database I