[sqlalchemy] Re: Elixir performance
On Sep 5, 2007, at 3:52 PM, EricHolmberg wrote: > > Based upon these tests, I concluded that for rapid prototyping, > SqlAlchemy is a great fit for where I previously used hand-crafted SQL > queries along with MySQLdb. However, Elixir is still too slow for > most of my prototypes, so I'm sticking with straight SqlAlchemy. For > the speed-critical sections, C++ wins hands down and makes much better > usage of my finite hardware resources. Im assuming you're using SQLAlchemy Table/select/ResultProxy objects for the test here, whereas Elixir and TurboEntity make usage of the ORM package.SA 0.4 has made some big speed improvements in the ORM (even post-0.4beta5) so I'd be curious to see how it stacks up in that test (it will still be pretty slow compared to SQL expressions). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Elixir performance
Hi, >data. I did some benchmarks a while back to see how everything >stacked up as I was wondering if I was doing everything the hard way >(in C++) instead of using SqlAlchemy, etc. TurboEntity is the same as > > Great work Eric. I am quite surprised at the results. I would have thought ActiveMapper/TurboEntity would only be marginally slower than plain SQLAlchemy. And again, I'm surprised that SA is faster than MySQLdb. How does that work out? I though SA used MySQLdb??? Your use of query cache and best of three sounds sensible, but I've got a feeling we're seeing some kind of measurement effect in the results. If those numbers are correct though, I'd expect fairly simple changes to Elixir could bring the performance close to plain SA. Paul --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Elixir performance
> > I am trying out Elixir 0.3.0 over SQLAlchemy 0.3.10 in a Python 2.5 > > environment. > > > Are there any known performance issues with Elixir for CRUD (Create > > Select Update Delete) commands? > > Not that I know of. There shouldn't be any overhead (over raw > SQLAlchemy) after the initial class initialization phase. > > -- > Gaëtan de Mentenhttp://openhex.org It all depends upon what your performance requirements are. I have a large project consisting of C++ for speed-critical simulations, Python for rapid prototyping, and TurboGears/Pylons for presentation of the data. I did some benchmarks a while back to see how everything stacked up as I was wondering if I was doing everything the hard way (in C++) instead of using SqlAlchemy, etc. TurboEntity is the same as Elixir for these results (I did re-run them with Elixir with the same results). The database being tested was a simple table select of a MySQL database using the MyISAM engine. Results for reading with InnoDB were very similar. I enabled the query cache, so the times reported below are the best out of 3 queries with the query cache enabled. All rows and columns were accessed, but the values discarded. Based upon these tests, I concluded that for rapid prototyping, SqlAlchemy is a great fit for where I previously used hand-crafted SQL queries along with MySQLdb. However, Elixir is still too slow for most of my prototypes, so I'm sticking with straight SqlAlchemy. For the speed-critical sections, C++ wins hands down and makes much better usage of my finite hardware resources. As always, your mileage may vary. I hope that Elixir gets some performance enhancements in the future that will bring it more in line with native SQLAlchemy. Note that you may need to paste the table into a fixed-font editor for everything to line up. = | Language | Library | Time (s) - 1k Rows | Time - 10k Rows | Time - 100k Rows | | =| | Python| SqlAlchemy + ActiveMapper | 1.325 | 13.028 | 139.669 | | Python| TurboEntity | 1.295 | 12.927 | 134.829 | | Python| MySQLdb + db_row| 0.108 | 1.046| 10.768 | | Python| MySQLdb | 0.036 | 0.384| 4.778| | Python| SqlAlchemy | 0.061 | 0.392| 4.062| | C | mysql client| 0.010 | 0.020| 0.170| | C++ | Direct API | 0.001 | 0.010| 0.141| = --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [patch] max() arg is an empty sequence while trying to reflect bugzilla table
Marcin Kasperski wrote: > Attached patch seems to resolve both problems. Thanks, this is fixed in r3464. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: [patch] max() arg is an empty sequence while trying to reflect bugzilla table
hey Marcin -
Seems like Jason Kirtland is out today. Any chance you could add a
simple test case to test/dialect/mysql.py for this ? It might go
quite nicely into test_type_reflection(). We generally like to add
unit tests at the same time bugs like these are fixed.
- mike
On Sep 5, 2007, at 12:48 PM, Marcin Kasperski wrote:
> Attached patch seems to resolve both problems.
>
>
> >
> poprawione
>
> diff --git a/lib/sqlalchemy/databases/mysql.py b/lib/sqlalchemy/
> databases/mysql.py
> --- a/lib/sqlalchemy/databases/mysql.py
> +++ b/lib/sqlalchemy/databases/mysql.py
> @@ -1106,7 +1106,10 @@ class MSEnum(MSString):
>
> self.enums = strip_enums
> self.strict = kw.pop('strict', False)
> -length = max([len(v) for v in strip_enums])
> +if strip_enums:
> +length = max([len(v) for v in strip_enums])
> +else:
> +length = 30 # some heuristics...
> super(MSEnum, self).__init__(length, **kw)
>
> def bind_processor(self, dialect):
> @@ -2192,7 +2195,7 @@ class MySQLSchemaReflector(object):
> r'%(iq)s(?P(?:%(esc_fq)s|[^%(fq)s])+)%(fq)s +'
> r'(?P\w+)'
> r'(?:\((?P(?:\d+|\d+,\d+|'
> - r'(?:\x27(?:\x27\x27|[^\x27])+\x27,?)+))\))?'
> + r'(?:\x27(?:\x27\x27|[^\x27])*\x27,?)+))\))?'
> r'(?: +(?PUNSIGNED))?'
> r'(?: +(?PZEROFILL))?'
> r'(?: +CHARACTER SET +(?P\w+))?'
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: sqlalchemy with turbogears and assign_mapper: select
> > Hi, > > >I am using turbogears with sqlalchemy and assign_mapper. > >How do I select one column only when doing a select statement? > > > > > Can I ask why you want to do that? Unless you're really speed critical, > selecting them all isn't too bad. > > But, if you're set on this, there are two approaches: > 1) Abandon the ORM and do something like > db.execute(select([Useraddress.Address_Sid], Useraddress.User_Sid == 30)) And if I wanted to select a year and group by year? "select User.Year from User group by User.Year" db.execute(select([User.Year]) ??? > 2) Use deferred column loading (see the docs, advanced data mapping). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] [patch] max() arg is an empty sequence while trying to reflect bugzilla table
Attached patch seems to resolve both problems.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
poprawione
diff --git a/lib/sqlalchemy/databases/mysql.py
b/lib/sqlalchemy/databases/mysql.py
--- a/lib/sqlalchemy/databases/mysql.py
+++ b/lib/sqlalchemy/databases/mysql.py
@@ -1106,7 +1106,10 @@ class MSEnum(MSString):
self.enums = strip_enums
self.strict = kw.pop('strict', False)
-length = max([len(v) for v in strip_enums])
+if strip_enums:
+length = max([len(v) for v in strip_enums])
+else:
+length = 30 # some heuristics...
super(MSEnum, self).__init__(length, **kw)
def bind_processor(self, dialect):
@@ -2192,7 +2195,7 @@ class MySQLSchemaReflector(object):
r'%(iq)s(?P(?:%(esc_fq)s|[^%(fq)s])+)%(fq)s +'
r'(?P\w+)'
r'(?:\((?P(?:\d+|\d+,\d+|'
- r'(?:\x27(?:\x27\x27|[^\x27])+\x27,?)+))\))?'
+ r'(?:\x27(?:\x27\x27|[^\x27])*\x27,?)+))\))?'
r'(?: +(?PUNSIGNED))?'
r'(?: +(?PZEROFILL))?'
r'(?: +CHARACTER SET +(?P\w+))?'
[sqlalchemy] Re: sqlalchemy with turbogears and assign_mapper: select
Hi, >I am using turbogears with sqlalchemy and assign_mapper. >How do I select one column only when doing a select statement? > > Can I ask why you want to do that? Unless you're really speed critical, selecting them all isn't too bad. But, if you're set on this, there are two approaches: 1) Abandon the ORM and do something like db.execute(select([Useraddress.Address_Sid], Useraddress.User_Sid == 30)) 2) Use deferred column loading (see the docs, advanced data mapping). Paul --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: sqlalchemy with turbogears and assign_mapper: select
Hello, I am using turbogears with sqlalchemy and assign_mapper. How do I select one column only when doing a select statement? d=model.Useraddress() How do I do: "select Address_Sid from Useraddress where User_Sid='30' " This d.select_by(USER_SID=30) will do select * from Useraddress where User_Sid='30' How do I get Address_Sid columns only? Lucas http://lucasmanual.com/mywiki/TurboGears#head-69d50afbd4ced8ca65202cd8367774fe8e4004cc --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Connection Timeout
My app saves bookmarks to different Postgres servers, and tries to connect at login. But if the server from the bookmark is not reachable or down, it only times out after like 2 minutes. So I could either check if the host is available before connecting (telnetlib connection to the Postgres port) or put the testconnection in a thread and kill it if there is no connection after x seconds. Or is there a nice way to solve this from withing SQLAlchemy? Ciao Koen --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
