RE: Slow MySQL queries with large data structures in memory

2011-12-22 Thread Patrick Oliver Glauner
Hi Benoit Since my notebook has limited RAM, I have used range(1000) and 'a' * 2000 instead of range(5000) and 'a' * 20. I could repoduce both scenarios on my virtual machine (Debian vagrant provided by Tibor) with a less significant time increase due to the lower

Re: Slow MySQL queries with large data structures in memory

2011-12-22 Thread Ferran Jorba
Hello Benoit, [...] In [4]: %time res = run_sql(SELECT id_bibrec FROM bibrec_bib03x LIMIT 100)CPU times: user 1.96 s, sys: 0.06 s, total: 2.02 s Wall time: 2.30 s Any idea about why we're seeing this and how we can fix it? It is quite a big problem for us as our citation

Re: Slow MySQL queries with large data structures in memory

2011-12-22 Thread Benoit Thiell
Hi Jorban. Thanks for taking the time to advise on this issue. The query I used is not the one that caused problem originally, I just chose it because it returns a large number of results. So I can't really change anything on this side. Also the query runs fine by itself, without dictionaries

Re: Slow MySQL queries with large data structures in memory

2011-12-22 Thread Benoit Thiell
Hi Roman. Thanks for your test. Here are the latest news: I tried with yet another MySQL bridge (https://launchpad.net/myconnpy) and got very similar results. This makes me think that it is not a problem that is specific to a Python MySQL bridge but maybe something lower level. So, on an advice

Re: Slow MySQL queries with large data structures in memory

2011-12-22 Thread Roman Chyla
Hi Benoit, My Python is behaving differently (see below). If I understand correctly, you are saying that a variable affects an operation of a function of a different scope - but I find it difficult to imagine. How is it possible? (other than consuming all RAM available and forcing Python to use