[sqlalchemy] Re: any particular reason for creating unused lists?
OK, i added this in 2298. I dont notice any speed enhancement in overall performance of the unit tests even though it builds on set and not sets.Set...this is probably because we dont use OrderedSet too much. Also i had to make some changes so its compatible with 2.3 (no generator expressions, subclasses Set which is set or sets.Set). On Feb 4, 3:42 am, [EMAIL PROTECTED] wrote: - using value.lower() == value instead of value.islower() ha x = 27 x.islower() False x.lower() == x True breaks a lot of tests too okay, forget that. Here an example OrderedSet over set; that seems to give a bigger speedup than anything else, and without touching anything. IMO any bigger speedup would require logical changes, e.g. not doing certain things at all, caching etc. oset.py.bz2 1KDownload --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: any particular reason for creating unused lists?
On Jan 31, 1:28 pm, svilen [EMAIL PROTECTED] wrote: another things i noted: - using value.lower() == value instead of value.islower() ha x = 27 x.islower() False x.lower() == x True breaks a lot of tests too --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: any particular reason for creating unused lists?
- using value.lower() == value instead of value.islower() ha x = 27 x.islower() False x.lower() == x True breaks a lot of tests too okay, forget that. Here an example OrderedSet over set; that seems to give a bigger speedup than anything else, and without touching anything. IMO any bigger speedup would require logical changes, e.g. not doing certain things at all, caching etc. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~--- oset.py.bz2 Description: BZip2 compressed data
[sqlalchemy] Re: any particular reason for creating unused lists?
another things i noted: - using value.lower() == value instead of value.islower() - ansisql.py: in _requires_quotes(): this bool(len([x for x in str(value) if x not in self._legal_characters()])) should be same as bool( s.translate( 256*' ', self._legal_characters() ) ) and that table(256) can be a static/global. i know, speed will not budge... well, profiling some case (5x (simple A,B,C setup + 50x query)), 6% goes in visit_select, 3% of time goes in __generic_obj_format(), and 3% in getattr, and everything else gets less. just habit ...i dont like one liners with :, also makes it easy to tack on conditionals...feel free to submit a patch for all those if theyre really bothering you (i guarantee program speed /mem usage will not budge in any measurable way). On Jan 25, 4:30 pm, [EMAIL PROTECTED] wrote: there are several places of such unused lists being made. i pick a random occurence, in this case InstrumentedAttribute: def _adapt_list(self, data): if self.typecallable is not None: t = self.typecallable() if data is not None: [t.append(x) for x in data] return t else: return data why not just for x in data: t.append(x) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: any particular reason for creating unused lists?
On Jan 31, 1:28 pm, svilen [EMAIL PROTECTED] wrote: another things i noted: - using value.lower() == value instead of value.islower() - ansisql.py: in _requires_quotes(): this bool(len([x for x in str(value) if x not in self._legal_characters()])) should be same as bool( s.translate( 256*' ', self._legal_characters() ) ) and that table(256) can be a static/global. oh no those are great, and should be fixed. the only ones i dont want to use are map() and reduce() since guido has said that list comprehensions should be used instead. i tend to not memorize all the little tweaky functions like these (note that I am frequently coming out against humane interfaces since i am more of a thinker and less of a memorizer). one reason you wont see too much performance gain with those in particuilar is because the quoting system caches all the decisions it makes about identifiers. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: any particular reason for creating unused lists?
another things i noted: - using value.lower() == value instead of value.islower() - ansisql.py: in _requires_quotes(): this bool(len([x for x in str(value) if x not in self._legal_characters()])) should be same as bool( s.translate( 256*' ', self._legal_characters() ) ) and that table(256) can be a static/global. oh no those are great, and should be fixed. the only ones i dont want to use are map() and reduce() since guido has said that list comprehensions should be used instead. i tend to not memorize all the little tweaky functions like these (note that I am frequently coming out against humane interfaces since i am more of a thinker and less of a memorizer). one reason you wont see too much performance gain with those in particuilar is because the quoting system caches all the decisions it makes about identifiers. hmmm. i run 300 times a query, and it gets something like #calls 47026__generic_obj_format called from: ansisql.py:929(format_label)(10530) ansisql.py:935(format_table)(20273) ansisql.py:942(format_column)(16223) which is a lot. Only getattr, isinstance etc are more. Anyway, my attempts at brainless (non-logic) optimisation show this: - fixing a little __generic_obj_format(): ~1% gain - replace OrderedSet( sets.Set+OrderedDict) with one over set: ~3% - little fix of _process_froms() and append_column(): 0.5% This is 1 setup + many times a relatively simple query. -- as of trunk: 2017068 function calls (193 primitive calls) Ordered by: internal time List reduced from 1631 to 15 due to restriction 15 %ncalls tottime filename:lineno(function) 4% 8101.030ansisql.py:343(visit_select) 3%1343680.780:0(getattr) 3% 170330.730ansisql.py:208(visit_column) 3% 470260.690ansisql.py:892(__generic_obj_format) 2% 441250.590util.py:178(__setitem__) 2%1354200.570:0(isinstance) 2% 122730.510sql.py:1514(append_column) 1% 121800.380interfaces.py:66(_get_context_strategy) 1% 8330.370base.py:553(__init__) 1% 162230.370ansisql.py:933(format_column) 1% 117830.330sql.py:1223(__init__) 1% 344600.320sql.py:1263(accept_visitor) 1% 741350.320:0(has_key) 1% 452610.280:0(lower) 1% 208560.260sets.py:519(add) ++ with my changes: 1910888 function calls (1895502 primitive calls) Ordered by: internal time List reduced from 1631 to 15 due to restriction 15 %ncalls tottime filename:lineno(function) 5% 8101.090 ansisql.py:343(visit_select) 4%1343520.870:0(getattr) 3% 170330.770ansisql.py:208(visit_column) 3% 470260.670ansisql.py:892(__generic_obj_format) 3%1321180.640:0(isinstance) 2% 121800.490interfaces.py:66(_get_context_strategy) 2% 162230.420ansisql.py:933(format_column) 1% 122730.380sql.py:1514(append_column) 1% 139590.370sql.py:1551(_process_froms) 1% 344600.300sql.py:1263(accept_visitor) 1% 8330.290base.py:553(__init__) 1% 105300.290ansisql.py:203(visit_label) 1% 202730.280ansisql.py:926(format_table) 1% 426670.260:0(get) 1% 306320.260oset.py:8(add) === definitely, not that big gain (in 3 hours). btw, for the record, comparative time of these (x,y are strings): 9 '%s.%s' % (x,y) 12 str(x)+'.'+str(y) 16 '.'join( (str(x),str(y))) 5 x+'.'+y 9 '.'join( x,y) same but x 3 14 '%s.%s.%s.%s.%s.%s' % (x,y , x,y, x,y) 40 str(x)+'.'+str(y)+... 34 '.'join( (str(x),str(y)...)) 19 x+'.'+y+,,, 15 '.'join( x,y,...) func calls _are_ expensive... ciao svil --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: any particular reason for creating unused lists?
Reasonable people can differ here, but I agree that if what you care about is a side effect, rather than a resulting list, using a for loop is more clear than a list comprehension. (I suspect it is also more performant since you are not allocating and populating a list object for no reason.) But in this case I think it's clear that the best way is to simply write t.extend(data) On 1/25/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: there are several places of such unused lists being made. i pick a random occurence, in this case InstrumentedAttribute: def _adapt_list(self, data): if self.typecallable is not None: t = self.typecallable() if data is not None: [t.append(x) for x in data] return t else: return data why not just for x in data: t.append(x) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: any particular reason for creating unused lists?
just habit ...i dont like one liners with :, also makes it easy to tack on conditionals...feel free to submit a patch for all those if theyre really bothering you (i guarantee program speed /mem usage will not budge in any measurable way). On Jan 25, 4:30 pm, [EMAIL PROTECTED] wrote: there are several places of such unused lists being made. i pick a random occurence, in this case InstrumentedAttribute: def _adapt_list(self, data): if self.typecallable is not None: t = self.typecallable() if data is not None: [t.append(x) for x in data] return t else: return data why not just for x in data: t.append(x) --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
