Re: [Python-Dev] 2.5 slower than 2.4 for some things?
On 6/13/07, ocean [EMAIL PROTECTED] wrote:
Meanwhile I tried to replace the parsing I did with Plex by re.Scanner.
And
again there is a remarkable speed difference. Again python2.5 is slower:
try:
from re import Scanner
except:
from sre import Scanner
pars = {}
order = []
count = 0
def par(scanner,name):
global count, order, pars
if name in ['caller','e','pi']:
return name
if name not in pars.keys():
pars[name] = ('ns', count)
order.append(name)
ret = 'a[%d]'%count
count += 1
else:
ret = 'a[%d]'%(order.index(name))
return ret
scanner = Scanner([
(rx, lambda y,x: x),
(r[a-zA-Z]+\., lambda y,x: x),
(r[a-z]+\(, lambda y,x: x),
(r[a-zA-Z_]\w*, par),
(r\d+\.\d*, lambda y,x: x),
(r\d+, lambda y,x: x),
(r\+|-|\*|/, lambda y,x: x),
(r\s+, None),
(r\)+, lambda y,x: x),
(r\(+, lambda y,x: x),
(r,, lambda y,x: x),
])
import profile
import pstats
def run():
arg = '+amp*exp(-(x-pos)/fwhm)'
for i in range(100):
scanner.scan(arg)
profile.run('run()','profscanner')
p = pstats.Stats('profscanner')
p.strip_dirs()
p.sort_stats('cumulative')
p.print_stats()
Well, I tried this script, there was no big difference.
Python2.4 0.772sec
Python2.5 0.816sec
Probably I found one reason comparation for classic style class is slower on
Python2.5.
Comparation function instance_compare() calls PyErr_GivenExceptionMatches(),
and it was just flag operation on 2.4. But on 2.5, probably related to
introduction of BaseException,
it checks inherited type tuple. (ie: PyExceptionInstance_Check)
I'm curious about the speed of 2.6 (trunk). I think this should have
become faster due to the introduction of fast subtype checks (he says
without looking at the code).
n
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 slower than 2.4 for some things?
Probably I found one reason comparation for classic style class is slower on Python2.5. Comparation function instance_compare() calls PyErr_GivenExceptionMatches(), and it was just flag operation on 2.4. But on 2.5, probably related to introduction of BaseException, it checks inherited type tuple. (ie: PyExceptionInstance_Check) I'm curious about the speed of 2.6 (trunk). I think this should have become faster due to the introduction of fast subtype checks (he says without looking at the code). n Yes, I confirmed trunk is faster than 2.5. /// // Code import timeit t = timeit.Timer( f1 f2 , class Foo: pass f1 = Foo() f2 = Foo() ) print t.timeit(1) /// // Result release-maint24 0.337sec release-maint25 0.625sec trunk 0.494sec // // Result of plex_test2.py release-maint24 2.944sec release-maint25 4.026sec trunk 3.625sec ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] 2.5 slower than 2.4 for some things?
I've had a report from a user that Plex runs about half
as fast in 2.5 as it did in 2.4. In particular, the
NFA-to-DFA conversion phase, which does a lot of
messing about with dicts representing mappings between
sets of states.
Does anyone in the Ministry for Making Python Blazingly
fast happen to know of some change that might have
pessimised things in this area?
--
Greg
---BeginMessage---
Hi,
I have been using Plex now for several years and really like it very much!
Recently I switched from python 2.4 to 2.5 and I noticed that the parser runs
significantly slower with 2.5. I hope you do not mind that I attach an example
script and two profiler logs which show the difference. The difference is almost
a factor of 2. Do you have an idea why that might happen and is there anything
one could do to improve the performance?
Regards, Christian
--
Christian Kristukat ::
Institut fuer Festkoerperphysik, TU Berlin ==
[EMAIL PROTECTED] ||
Tel. +49-30-20896371
from Plex import *
from Plex.Traditional import re as regex
class ParseString:
def __init__(self, parse_str):
self.parse_str = parse_str
self.EOF = 0
def read(self, size):
if self.EOF:
return ''
else:
self.EOF = 1
return self.parse_str
def reset(self):
self.EOF = 0
class SymParser:
def __init__(self, tok):
self.pstr = ParseString(tok)
self.count = 0
self.varlist = {}
self.dummy = []
self.nvars = 0
self.varfunc = self.setvar
def setvar(self,scanner,name):
if name in ['caller','e','pi']:
return name
if name not in self.varlist:
self.varlist[name] = ['ns',self.nvars]
self.dummy.append(name)
ret = 'a[%d]'%self.nvars
self.nvars += 1
else:
ret = 'a[%d]'%(self.dummy.index(name)+self.count)
return ret
def parse(self):
letter = regex('[A-Za-z_]')
digit = Range(09)
dot = Str(.)
rnumber = (Rep1(digit)+dot+Rep1(digit))|Rep1(digit)
expnumber = Rep1(digit)+dot+Rep1(digit)+Str('e')+(Any('-+')|Empty)+Rep1(digit)
cnumber = (Rep1(digit)+dot+Rep1(digit)+Str('j'))|(Rep1(digit)+Str('j'))
number = rnumber|cnumber|expnumber
x = Str(x)
name = Rep1(letter)|(Rep1(letter)+Rep1(digit)+Rep(letter))
inst_member = (name|Str())|digit)+dot+name
parname = Str(r')+name+Str(r')
func = name+Str(()
op = Any(^+-/*(),)
space = Any( \t\n\r)
lex = Lexicon([
(number, TEXT),
(x, TEXT),
(func, TEXT),
(parname, TEXT),
(inst_member, TEXT),
(name, self.varfunc),
(op, TEXT),
(space, IGNORE),
(AnyChar, IGNORE)
])
parsed =
scanner = Scanner(lex, self.pstr, pparse)
while 1:
tok = scanner.read()
if tok[0] is None:
break
parsed += tok[0]
self.count += 1
return self.varlist
def sym():
for x in range(10):
a = SymParser('amp*exp(-(x-pos)**2/fwhm)')
a.parse()
print a
def prof_sym():
import profile
import pstats
profile.run('sym()','modelprof')
p = pstats.Stats('modelprof')
p.strip_dirs()
p.sort_stats('cumulative')
p.print_stats()
if __name__ == '__main__':
prof_sym()
__main__.SymParser instance at 0xb7c2d34c
Sat Jun 9 21:45:53 2007modelprof
106631 function calls (104491 primitive calls) in 1.700 CPU seconds
Ordered by: cumulative time
ncalls tottime percall cumtime percall filename:lineno(function)
10.0000.0001.7001.700 plex_test2.py:81(sym)
10.0000.0001.7001.700 profile:0(sym())
10.0000.0001.7001.700 string:1(?)
100.0000.0001.7000.170 plex_test2.py:42(parse)
100.0000.0001.5600.156 Lexicons.py:113(__init__)
100.1900.0191.2600.126 DFA.py:13(nfa_to_dfa)
13500.0700.0000.3100.000 DFA.py:100(old_to_new)
900.0100.0000.3000.003
Lexicons.py:158(add_token_to_machine)
530/900.0300.0000.2700.003 Regexps.py:362(build_machine)
590/1000.0200.0000.2400.002 Regexps.py:315(build_machine)
26000.0900.0000.2200.000 DFA.py:50(set_epsilon_closure)
28000.1700.0000.2200.000 Transitions.py:91(items)
13500.0500.0000.1900.000 DFA.py:140(make_key)
2900.0200.0000.1800.001 Regexps.py:384(build_machine)
13400.1000.0000.1500.000 Machines.py:180(add_transitions)
26000.0700.0000.1500.000
Re: [Python-Dev] 2.5 slower than 2.4 for some things?
I've had a report from a user that Plex runs about half as fast in 2.5 as it did in 2.4. In particular, the NFA-to-DFA conversion phase, which does a lot of messing about with dicts representing mappings between sets of states. Does anyone in the Ministry for Making Python Blazingly fast happen to know of some change that might have pessimised things in this area? Hello, I investigated. On my environment, consumed time is E:\Plex-1.1.5py24 plex_test2.py 0.71065668 E:\Plex-1.1.5py25 plex_test2.py 0.92131335 And after I applied this patch to Plex/Machines, (make `Node' new style class) 62c62 class Node: --- class Node(object): E:\Plex-1.1.5py24 plex_test2.py 0.40122888 E:\Plex-1.1.5py25 plex_test2.py 0.350999832153 So, probably hash, comparation mechanizm of old/new style class has changed. # improved for new style class, worse for old style class. Maybe optimized for new style class? Try this for minimum test. import timeit init = class Class: pass c1 = Class() c2 = Class() t1 = timeit.Timer( c1 c2 , init) t2 = timeit.Timer( hash(c1) hash(c2) , init) print t1.timeit(1000) print t2.timeit(1000) ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 slower than 2.4 for some things?
ocean wrote: So, probably hash, comparation mechanizm of old/new style class has changed. # improved for new style class, worse for old style class. Maybe optimized for new style class? Thanks -- it looks like there's a simple solution that will make Plex even faster! I'll pass this on to the OP. -- Greg ___ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] 2.5 slower than 2.4 for some things?
ocean wrote:
I've had a report from a user that Plex runs about half
as fast in 2.5 as it did in 2.4. In particular, the
NFA-to-DFA conversion phase, which does a lot of
messing about with dicts representing mappings between
sets of states.
That was me.
Does anyone in the Ministry for Making Python Blazingly
fast happen to know of some change that might have
pessimised things in this area?
Hello, I investigated. On my environment, consumed time is
E:\Plex-1.1.5py24 plex_test2.py
0.71065668
E:\Plex-1.1.5py25 plex_test2.py
0.92131335
And after I applied this patch to Plex/Machines, (make `Node' new style
class)
62c62
class Node:
---
class Node(object):
E:\Plex-1.1.5py24 plex_test2.py
0.40122888
E:\Plex-1.1.5py25 plex_test2.py
0.350999832153
Nice!.
Meanwhile I tried to replace the parsing I did with Plex by re.Scanner. And
again there is a remarkable speed difference. Again python2.5 is slower:
try:
from re import Scanner
except:
from sre import Scanner
pars = {}
order = []
count = 0
def par(scanner,name):
global count, order, pars
if name in ['caller','e','pi']:
return name
if name not in pars.keys():
pars[name] = ('ns', count)
order.append(name)
ret = 'a[%d]'%count
count += 1
else:
ret = 'a[%d]'%(order.index(name))
return ret
scanner = Scanner([
(rx, lambda y,x: x),
(r[a-zA-Z]+\., lambda y,x: x),
(r[a-z]+\(, lambda y,x: x),
(r[a-zA-Z_]\w*, par),
(r\d+\.\d*, lambda y,x: x),
(r\d+, lambda y,x: x),
(r\+|-|\*|/, lambda y,x: x),
(r\s+, None),
(r\)+, lambda y,x: x),
(r\(+, lambda y,x: x),
(r,, lambda y,x: x),
])
import profile
import pstats
def run():
arg = '+amp*exp(-(x-pos)/fwhm)'
for i in range(100):
scanner.scan(arg)
profile.run('run()','profscanner')
p = pstats.Stats('profscanner')
p.strip_dirs()
p.sort_stats('cumulative')
p.print_stats()
Christian
___
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
