[sqlalchemy] Re: Slow running
News has a TEXT column [content] which is in average about 5000 in length. So ye, u found the problem :) And how to overcome (workaround) this in sqlalchemy? On 25 feb., 00:42, Michael Bayer mike...@zzzcomputing.com wrote: On Feb 24, 2010, at 6:15 PM, dusans wrote: Hi. I got this simple query thats running very slow: Im working on sqlalchemy 0.6 0.6beta1, sqlite3, python 2.5, windows xp 1. I've addet News.[title] to make it run faster, but its still very very slow == Code (10 seconds) == : Session.query(News.title, KernelResults).\ select_from(join(News, KernelResults, News.news_id == KernelResults.news_id_2)).\ filter(KernelResults.news_id_1 == c.news_id).\ filter(KernelResults.kernel_id == 0).\ order_by(desc(KernelResults.similarity))[: 8] == The SQL (50 miliseconds) ==: SELECT news.title AS news_title, kernel_results.id_row AS kernel_results_id_row, kernel_results.news_id_1 AS kernel_results_news_id_1, kernel_results.news_id_2 AS kernel_results_news_id_2, kernel_results.similarity AS kernel_results_similarity, kernel_results .kernel_id AS kernel_results_kernel_id FROM news JOIN kernel_results ON news.news_id = kernel_results.news_id_2 WHERE kernel_results.news_id_1 = 66 AND kernel_results.kernel_id = 0 ORDER BY kernel_results.similarity DESC LIMIT 8 OFFSET 0 Could someone please help me? im still fresh with sqlalchemy :) there's nothing apparent which would cause such slowness unless you have a high degree of network overhead when your SQLAlchemy application runs, or for example one or more of the columns returned represents an enormous text or binary field, perhaps. For 8 rows and a 50ms query the total time spent should be in the ballpark of well under 100 ms. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Slow running
I've profile it 1. Session.query: 2.8 seconds 2. Session.execute: 2.6 seconds 3. sqlite.execute: 2.6 seconds 4. If u run it directly via Sqlite Gui client (SQLite Expert) then i get: 50 ms Maybe its using some kind of cache. On 25 feb., 00:42, Michael Bayer mike...@zzzcomputing.com wrote: On Feb 24, 2010, at 6:15 PM, dusans wrote: Hi. I got this simple query thats running very slow: Im working on sqlalchemy 0.6 0.6beta1, sqlite3, python 2.5, windows xp 1. I've addet News.[title] to make it run faster, but its still very very slow == Code (10 seconds) == : Session.query(News.title, KernelResults).\ select_from(join(News, KernelResults, News.news_id == KernelResults.news_id_2)).\ filter(KernelResults.news_id_1 == c.news_id).\ filter(KernelResults.kernel_id == 0).\ order_by(desc(KernelResults.similarity))[: 8] == The SQL (50 miliseconds) ==: SELECT news.title AS news_title, kernel_results.id_row AS kernel_results_id_row, kernel_results.news_id_1 AS kernel_results_news_id_1, kernel_results.news_id_2 AS kernel_results_news_id_2, kernel_results.similarity AS kernel_results_similarity, kernel_results .kernel_id AS kernel_results_kernel_id FROM news JOIN kernel_results ON news.news_id = kernel_results.news_id_2 WHERE kernel_results.news_id_1 = 66 AND kernel_results.kernel_id = 0 ORDER BY kernel_results.similarity DESC LIMIT 8 OFFSET 0 Could someone please help me? im still fresh with sqlalchemy :) there's nothing apparent which would cause such slowness unless you have a high degree of network overhead when your SQLAlchemy application runs, or for example one or more of the columns returned represents an enormous text or binary field, perhaps. For 8 rows and a 50ms query the total time spent should be in the ballpark of well under 100 ms. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Constructors and Object Initialization...
Hello,
Guess I haven't understood all the magic behind object mapping...
I have a class like this :
class MyClass(object):
__init__(self, name):
self = session.query(MyClass).filter(MyClass.name==name).one()
print self.name # Outputs 'joe'
And I'm expecting that my instance of MyClass has its attributes filled
with the correct values, they are instead empty, while issuing for
example print self.name inside the __init__ method reports the expected
values :
obj = MyClass('joe')
obj.name
I've tried prepending __init__ with the @orm.reconstructor stuff without
success too.
Could someone enlighten me, and show me the way to handle this the right
way ?
Regards.
--
f3d
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Slow running
you can skip the loading of that column using deferred(), for example. still, 5000 * 8 is only 40K. should be well under 200 ms. On Feb 25, 2010, at 4:10 AM, dusans wrote: News has a TEXT column [content] which is in average about 5000 in length. So ye, u found the problem :) And how to overcome (workaround) this in sqlalchemy? On 25 feb., 00:42, Michael Bayer mike...@zzzcomputing.com wrote: On Feb 24, 2010, at 6:15 PM, dusans wrote: Hi. I got this simple query thats running very slow: Im working on sqlalchemy 0.6 0.6beta1, sqlite3, python 2.5, windows xp 1. I've addet News.[title] to make it run faster, but its still very very slow == Code (10 seconds) == : Session.query(News.title, KernelResults).\ select_from(join(News, KernelResults, News.news_id == KernelResults.news_id_2)).\ filter(KernelResults.news_id_1 == c.news_id).\ filter(KernelResults.kernel_id == 0).\ order_by(desc(KernelResults.similarity))[: 8] == The SQL (50 miliseconds) ==: SELECT news.title AS news_title, kernel_results.id_row AS kernel_results_id_row, kernel_results.news_id_1 AS kernel_results_news_id_1, kernel_results.news_id_2 AS kernel_results_news_id_2, kernel_results.similarity AS kernel_results_similarity, kernel_results .kernel_id AS kernel_results_kernel_id FROM news JOIN kernel_results ON news.news_id = kernel_results.news_id_2 WHERE kernel_results.news_id_1 = 66 AND kernel_results.kernel_id = 0 ORDER BY kernel_results.similarity DESC LIMIT 8 OFFSET 0 Could someone please help me? im still fresh with sqlalchemy :) there's nothing apparent which would cause such slowness unless you have a high degree of network overhead when your SQLAlchemy application runs, or for example one or more of the columns returned represents an enormous text or binary field, perhaps. For 8 rows and a 50ms query the total time spent should be in the ballpark of well under 100 ms. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Constructors and Object Initialization...
On Feb 24, 2010, at 7:59 PM, f3d wrote:
Hello,
Guess I haven't understood all the magic behind object mapping...
I have a class like this :
class MyClass(object):
__init__(self, name):
self = session.query(MyClass).filter(MyClass.name==name).one()
print self.name # Outputs 'joe'
And I'm expecting that my instance of MyClass has its attributes filled
with the correct values, they are instead empty, while issuing for
example print self.name inside the __init__ method reports the expected
values :
you can't change the value of self inside the __init__ method of an object.
That's just Python. You probably want to use __new__, as in
http://www.sqlalchemy.org/trac/wiki/UsageRecipes/UniqueObject .
obj = MyClass('joe')
obj.name
I've tried prepending __init__ with the @orm.reconstructor stuff without
success too.
Could someone enlighten me, and show me the way to handle this the right
way ?
Regards.
--
f3d
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] assert_unicode=True
Hello everyone, To refresh my familiarity with SQLAlchemy I wrote a sample app, and used: engine = create_engine(postgres://postgres:*...@localhost/ts, convert_unicode=True, assert_unicode=True) Then I passed a few strings in an object added via session.add() and one of those strings was not Unicode. SQLAlchemy displayed warning: C:\Python26\lib\site-packages\sqlalchemy-0.5.8-py2.6.egg\sqlalchemy\engine\default.py:242: SAWarning: Unicode type received non-unicode bind param value 'Jane Shmoe' param[key.encode(encoding)] = processors[key](compiled_params[key]) I was led to believe by doc on create_engine that this would raise an error: :param assert_unicode=False: When set to ``True`` alongside convert_unicode=``True``, asserts that incoming string bind parameters are instances of ``unicode``, otherwise raises an error. Only takes effect when ``convert_unicode==True``. I would need this to be error rather than warning just like docs say (well I'd rather have it crash then debug than write bad values into db) SQA is 0.5.8, win32 if that's of any importance. Regards, mk -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] assert_unicode=True
Le Thu, 25 Feb 2010 21:44:51 +0100, Marcin Krol mrk...@gmail.com a écrit : SAWarning: Unicode type received non-unicode bind param value 'Jane Shmoe' param[key.encode(encoding)] = processors[key](compiled_params[key]) [...] I would need this to be error rather than warning just like docs say (well I'd rather have it crash then debug than write bad values into db) To turn warnings into errors you just have to issue the right settings to the standard warnings module: http://docs.python.org/library/warnings.html Regards Antoine. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] assert_unicode=True
Marcin Krol wrote: Hello everyone, To refresh my familiarity with SQLAlchemy I wrote a sample app, and used: engine = create_engine(postgres://postgres:*...@localhost/ts, convert_unicode=True, assert_unicode=True) Then I passed a few strings in an object added via session.add() and one of those strings was not Unicode. SQLAlchemy displayed warning: C:\Python26\lib\site-packages\sqlalchemy-0.5.8-py2.6.egg\sqlalchemy\engine\default.py:242: SAWarning: Unicode type received non-unicode bind param value 'Jane Shmoe' param[key.encode(encoding)] = processors[key](compiled_params[key]) I was led to believe by doc on create_engine that this would raise an error: :param assert_unicode=False: When set to ``True`` alongside convert_unicode=``True``, asserts that incoming string bind parameters are instances of ``unicode``, otherwise raises an error. Only takes effect when ``convert_unicode==True``. I would need this to be error rather than warning just like docs say (well I'd rather have it crash then debug than write bad values into db) SQA is 0.5.8, win32 if that's of any importance. the assert_unicode flag is not ideal - a Unicode type sets it to warn which overrides the engine-level setting. The engine level setting is more for just all the String types to use. This qualifies as a bug but at the same time I think using the warnings filter is a better approach overall. Regards, mk -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Getting useful error messages
Ok...maybe I'm missing something, but why wasn't that raised in this case instead of the rather cryptic exception? On Feb 18, 4:59 pm, Michael Bayer mike...@zzzcomputing.com wrote: On Feb 18, 2010, at 1:16 PM, Hollister wrote: I ran into this also and it was caused by forgetting to map the class to the table. Make sure you have a line like: orm.mapper(TaskAction, taskaction_table) Mike: should there be a more specific error message for a missing mapping? there's a very specific error for that its sqlalchemy.orm.exc.UnmappedError. -aw On Feb 3, 11:04 am, Michael Bayer mike...@zzzcomputing.com wrote: King Simon-NFHD78 wrote: The line below the one you're complaining about is telling you what the column in question is: Invalid column expression 'class '__main__.TaskAction'' So somehow, you've passed your TaskAction class in a place where SA is expecting a column expression. I think we'd need to see the command that you actually typed in to work out what the problem is. Also, again, please upgrade to 0.5.8. Hundreds of bugs and sub-optimal behaviors have been fixed since 0.5.4p2 so you may get better results. Simon -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Getting useful error messages
Hollister wrote: Ok...maybe I'm missing something, but why wasn't that raised in this case instead of the rather cryptic exception? I'm looking back and I don't see that we've established specifically what this case is. Would need to see that in order to investigate. On Feb 18, 4:59 pm, Michael Bayer mike...@zzzcomputing.com wrote: On Feb 18, 2010, at 1:16 PM, Hollister wrote: I ran into this also and it was caused by forgetting to map the class to the table. Make sure you have a line like: orm.mapper(TaskAction, taskaction_table) Mike: should there be a more specific error message for a missing mapping? there's a very specific error for that its sqlalchemy.orm.exc.UnmappedError. -aw On Feb 3, 11:04 am, Michael Bayer mike...@zzzcomputing.com wrote: King Simon-NFHD78 wrote: The line below the one you're complaining about is telling you what the column in question is: Invalid column expression 'class '__main__.TaskAction'' So somehow, you've passed your TaskAction class in a place where SA is expecting a column expression. I think we'd need to see the command that you actually typed in to work out what the problem is. Also, again, please upgrade to 0.5.8. Hundreds of bugs and sub-optimal behaviors have been fixed since 0.5.4p2 so you may get better results. Simon -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] another problem with complex join
Manlio Perillo wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Michael Bayer ha scritto: On Feb 18, 2010, at 11:55 AM, Manlio Perillo wrote: Michael Bayer ha scritto: [...] so what I had in mind is that, if its given a join as the left side, it just does the natural thing, i.e. joins to the right. If the natural join isn't available, then it does its usual search through the whole thing. Here is another tentative patch. this is the idea and I've added change requests and a request for unit tests at #1714. Manlio -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAkuEBiAACgkQscQJ24LbaUTzDQCfSSJBXe9LzQvKFcDva3oqojxp WvQAn2k0Ykdn3hh1mmfFSg3gTOx1okPQ =TiLH -END PGP SIGNATURE- -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] sqlalchemy 6beta1 error
I could not get from sqlalchemy.databases.mysql import MSBinary working on my windows XP as well as my Ubuntu 32 and 64 bit machines. Are they dropped? Here are summary ActivePython 2.6.0.0 (ActiveState Software Inc.) based on Python 2.6 (r26:66714, Nov 11 2008, 10:21:19) [MSC v.1500 32 bit (Intel)] on win 32 Type help, copyright, credits or license for more information. import sqlalchemy sqlalchemy.__version__ '0.6beta1' from sqlalchemy.databases.mysql import MSBinary Traceback (most recent call last): File stdin, line 1, in module ImportError: No module named mysql -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
