Re: so if zodb has no index or search, how run fast query?
visphatesj...@gmail.com writes: ... The only service of the ZODB is storing (persistent) objects persistently and providing (almost) transparent acces to them. Persistent objects have a (usually) implicitely managed attribute which contains the object's object id and the ZODB can (quite) efficiently access the objects based on its object id (it is using caches and indexes for this). The objects managed by the ZODB form a (directed) graph with a distinguished root node - where all nodes are (persistent) Python objects. You essentially work with those persistent objects as with normal Python objects: the ZODB transparently accesses the persistent storage, when necessary, to give you the feeling as if those objects were normal Python objects. There are a few caveats with respect to modifications - but most modifications work as you would expect for normal Python objects (however, you must call transaction.commit(), in order to persist the modifications). That is essentially all, the ZODB does for you. When you need searches, you must use appropriate datatypes -- in the same way, you would need to do this in a normal Python program. The ZODB comes with a collection of datatypes (so called BTrees) which may form the base for searches. BTrees are essentially persistent dicts - however, they are not build from a fixed number of persistent subobjects but the persistent substructure is changed automatically to accommodate efficiently for various tree sizes. Someone else has already mentioned some higher level packages which can provide search capabilities. -- http://mail.python.org/mailman/listinfo/python-list
Diacretical incensitive search
One feature that seems to be missing in the re module (or any tools that I know for searching text) is diacretical incensitive search. I would like to have a match for something like this: re.match(franc, français) in about the same whay we can have a case incensitive search: re.match((?i)fran, Français). Another related and more general problem (in the sense that it could easily be used to solve the first problem) would be to translate a string removing any diacritical mark: nodiac(Français) - Francais The algorithm to write such a function is trivial but there are a lot of mark we can put on a letter. It would be necessary to have the list of a's with something on it. i.e. à,á,ã, etc. and this for every letter. Trying to make such a list by hand would inevitably lead to some symbols forgotten (and would be tedious). Olive -- http://mail.python.org/mailman/listinfo/python-list
Re: Diacretical incensitive search
On May 17, 2013, at 8:57 AM, Olive diolu.remove_this_p...@bigfoot.com wrote: The algorithm to write such a function is trivial but there are a lot of mark we can put on a letter. It would be necessary to have the list of a's with something on it. i.e. à,á,ã, etc. and this for every letter. Trying to make such a list by hand would inevitably lead to some symbols forgotten (and would be tedious). Perhaps of interest… Sean M. Burke Unidecode… There appear to be several python implementations, e.g.: https://pypi.python.org/pypi/Unidecode -- http://mail.python.org/mailman/listinfo/python-list
Re: Diacretical incensitive search
Olive wrote: One feature that seems to be missing in the re module (or any tools that I know for searching text) is diacretical incensitive search. I would like to have a match for something like this: re.match(franc, français) in about the same whay we can have a case incensitive search: re.match((?i)fran, Français). Another related and more general problem (in the sense that it could easily be used to solve the first problem) would be to translate a string removing any diacritical mark: nodiac(Français) - Francais The algorithm to write such a function is trivial but there are a lot of mark we can put on a letter. It would be necessary to have the list of a's with something on it. i.e. à,á,ã, etc. and this for every letter. Trying to make such a list by hand would inevitably lead to some symbols forgotten (and would be tedious). [Python3.3] unicodedata.normalize(NFKD, Français).encode(ascii, ignore).decode() 'Francais' import sys from collections import defaultdict from unicodedata import name, normalize d = defaultdict(list) for i in range(sys.maxunicode): c = chr(i) n = normalize(NFKD, c)[0] if ord(n) 128 and n.isalpha(): # optional d[n].append(c) for k, v in d.items(): if len(v) 1: print(k, .join(v)) See also http://effbot.org/zone/unicode-convert.htm PS: Be warned that experiments on the console may be misleading: unicodedata.normalize(NFKD, ç) 'c' ascii(_) 'c\\u0327' -- http://mail.python.org/mailman/listinfo/python-list
Re: How to write fast into a file in python?
On 05/17/2013 12:35 AM, lokeshkopp...@gmail.com wrote:
On Friday, May 17, 2013 8:50:26 AM UTC+5:30, lokesh...@gmail.com wrote:
I need to write numbers into a file upto 50mb and it should be fast
can any one help me how to do that?
i had written the following code..
SNIP
value = 0
with open(filename, w) as f:
while f.tell() size:
f.write({0}\n.format(value))
SNIP more double-spaced nonsense from googlegroups
If you must use googlegroups, at least read this
http://wiki.python.org/moin/GoogleGroupsPython.
it takes about 20sec i need 5 to 10 times less than that.
size = 50mb
Most of the time is spent figuring out whether the file has reached its
limit size. If you want Python to go fast, just specify the data. On
my Linux system, it takes 11 seconds to write the first 633 values,
which is just under 50mb. If I write the obvious loop, writing that
many values takes .25 seconds.
--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list
RE: How to write fast into a file in python?
I've got the following results on my desktop PC (Win7/Python2.7.5):
C:\src\Pythonpython -m timeit -cvn3 -r3 execfile('fastwrite2.py')
raw times: 123 126 125
3 loops, best of 3: 41 sec per loop
C:\src\Pythonpython -m timeit -cvn3 -r3 execfile('fastwrite5.py')
raw times: 34 34.3 34
3 loops, best of 3: 11.3 sec per loop
C:\src\Pythonpython -m timeit -cvn3 -r3 execfile('fastwrite6.py')
raw times: 0.4 0.447 0.391
3 loops, best of 3: 130 msec per loop
If you can just copy a preexisting file it will surely increase the speed to
the levels you need, but doing the cStringIO operations can reduce the time in
72%.
Strangely I just realised that the time it takes to complete such scripts is
the same no matter what hard drive I choose to run them. The results are the
same for an SSD (main drive) and a HDD.
I think it's very strange to take 11.3s to write 50MB (4.4MB/s) sequentially on
a SSD which is capable of 140MB/s.
Is that a Python problem? Why does it take the same time on the HDD?
### fastwrite2.py ### this is your code
size = 50*1024*1024
value = 0
filename = 'fastwrite2.dat'
with open(filename, w) as f:
while f.tell() size:
f.write({0}\n.format(value))
value += 1
f.close()
### fastwrite5.py ###
import cStringIO
size = 50*1024*1024
value = 0
filename = 'fastwrite5.dat'
x = 0
b = cStringIO.StringIO()
while x size:
line = '{0}\n'.format(value)
b.write(line)
value += 1
x += len(line)+1
f = open(filename, 'w')
f.write(b.getvalue())
f.close()
b.close()
### fastwrite6.py ###
import shutil
src = 'fastwrite.dat'
dst = 'fastwrite6.dat'
shutil.copyfile(src, dst)
Date: Fri, 17 May 2013 07:58:43 -0400
From: da...@davea.name
To: python-list@python.org
Subject: Re: How to write fast into a file in python?
On 05/17/2013 12:35 AM, lokeshkopp...@gmail.com wrote:
On Friday, May 17, 2013 8:50:26 AM UTC+5:30, lokesh...@gmail.com wrote:
I need to write numbers into a file upto 50mb and it should be fast
can any one help me how to do that?
i had written the following code..
SNIP
value = 0
with open(filename, w) as f:
while f.tell() size:
f.write({0}\n.format(value))
SNIP more double-spaced nonsense from googlegroups
If you must use googlegroups, at least read this
http://wiki.python.org/moin/GoogleGroupsPython.
it takes about 20sec i need 5 to 10 times less than that.
size = 50mb
Most of the time is spent figuring out whether the file has reached its
limit size. If you want Python to go fast, just specify the data. On
my Linux system, it takes 11 seconds to write the first 633 values,
which is just under 50mb. If I write the obvious loop, writing that
many values takes .25 seconds.
--
DaveA
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: Diacretical incensitive search
Tanks a lot! -- http://mail.python.org/mailman/listinfo/python-list
Re: How to write fast into a file in python?
On Fri, 17 May 2013 18:20:33 +0300, Carlos Nepomuceno wrote:
I've got the following results on my desktop PC (Win7/Python2.7.5):
C:\src\Pythonpython -m timeit -cvn3 -r3 execfile('fastwrite2.py') raw
times: 123 126 125
3 loops, best of 3: 41 sec per loop
Your times here are increased significantly by using execfile. Using
execfile means that instead of compiling the code once, then executing
many times, it gets compiled over and over and over and over again. In my
experience, using exec, execfile or eval makes your code ten or twenty
times slower:
[steve@ando ~]$ python -m timeit 'x = 100; y = x/3'
100 loops, best of 3: 0.175 usec per loop
[steve@ando ~]$ python -m timeit 'exec(x = 100; y = x/3)'
1 loops, best of 3: 37.8 usec per loop
Strangely I just realised that the time it takes to complete such
scripts is the same no matter what hard drive I choose to run them. The
results are the same for an SSD (main drive) and a HDD.
There's nothing strange here. The time you measure is dominated by three
things, in reducing order of importance:
* the poor choice of execfile dominates the time taken;
* followed by choice of algorithm;
* followed by the time it actually takes to write to the disk, which is
probably insignificant compared to the other two, regardless of whether
you are using a HDD or SSD.
Until you optimize the code, optimizing the media is a waste of time.
I think it's very strange to take 11.3s to write 50MB (4.4MB/s)
sequentially on a SSD which is capable of 140MB/s.
It doesn't. It takes 11.3 seconds to open a file, read it into memory,
parse it, compile it into byte-code, and only then execute it. My
prediction is that the call to f.write() and f.close() probably take a
fraction of a second, and nearly all of the rest of the time is taken by
other calculations.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: any cherypy powred sites I can check out?
On 17/05/2013 01:00, visphatesj...@gmail.com wrote: fuck straight off I assume you're the author of How to win friends and influence people? -- If you're using GoogleCrap™ please read this http://wiki.python.org/moin/GoogleGroupsPython. Mark Lawrence -- http://mail.python.org/mailman/listinfo/python-list
RE: How to write fast into a file in python?
Thank you Steve! You are totally right!
It takes about 0.2s for the f.write() to return. Certainly because it writes to
the system file cache (~250MB/s).
Using a little bit different approach I've got:
C:\src\Pythonpython -m timeit -cvn3 -r3 -sfrom fastwrite5r import run run()
raw times: 24 25.1 24.4
3 loops, best of 3: 8 sec per loop
This time it took 8s to complete from previous 11.3s.
Does those 3.3s are the time to open, read, parse, compile steps you told me?
If so, the execute step is really taking 8s, right?
Why does it take so long to build the string to be written? Can it get faster?
Thanks in advance!
### fastwrite5r.py ###
def run():
import cStringIO
size = 50*1024*1024
value = 0
filename = 'fastwrite5.dat'
x = 0
b = cStringIO.StringIO()
while x size:
line = '{0}\n'.format(value)
b.write(line)
value += 1
x += len(line)+1
f = open(filename, 'w')
f.write(b.getvalue())
f.close()
b.close()
if __name__ == '__main__':
run()
From: steve+comp.lang.pyt...@pearwood.info
Subject: Re: How to write fast into a file in python?
Date: Fri, 17 May 2013 16:42:55 +
To: python-list@python.org
On Fri, 17 May 2013 18:20:33 +0300, Carlos Nepomuceno wrote:
I've got the following results on my desktop PC (Win7/Python2.7.5):
C:\src\Pythonpython -m timeit -cvn3 -r3 execfile('fastwrite2.py') raw
times: 123 126 125
3 loops, best of 3: 41 sec per loop
Your times here are increased significantly by using execfile. Using
execfile means that instead of compiling the code once, then executing
many times, it gets compiled over and over and over and over again. In my
experience, using exec, execfile or eval makes your code ten or twenty
times slower:
[steve@ando ~]$ python -m timeit 'x = 100; y = x/3'
100 loops, best of 3: 0.175 usec per loop
[steve@ando ~]$ python -m timeit 'exec(x = 100; y = x/3)'
1 loops, best of 3: 37.8 usec per loop
Strangely I just realised that the time it takes to complete such
scripts is the same no matter what hard drive I choose to run them. The
results are the same for an SSD (main drive) and a HDD.
There's nothing strange here. The time you measure is dominated by three
things, in reducing order of importance:
* the poor choice of execfile dominates the time taken;
* followed by choice of algorithm;
* followed by the time it actually takes to write to the disk, which is
probably insignificant compared to the other two, regardless of whether
you are using a HDD or SSD.
Until you optimize the code, optimizing the media is a waste of time.
I think it's very strange to take 11.3s to write 50MB (4.4MB/s)
sequentially on a SSD which is capable of 140MB/s.
It doesn't. It takes 11.3 seconds to open a file, read it into memory,
parse it, compile it into byte-code, and only then execute it. My
prediction is that the call to f.write() and f.close() probably take a
fraction of a second, and nearly all of the rest of the time is taken by
other calculations.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
Re: Diacretical incensitive search
The handling of diacriticals is especially a nice case
study. One can use it to toy with some specific features of
Unicode, normalisation, decomposition, ...
... and also to show how Unicode can be badly implemented.
First and quick example that came to my mind (Py325 and Py332):
timeit.repeat(ud.normalize('NFKC', ud.normalize('NFKD', 'ᶑḗḖḕḹ')),
import unicodedata as ud)
[2.929404406789672, 2.923327801150208, 2.923659417064755]
timeit.repeat(ud.normalize('NFKC', ud.normalize('NFKD', 'ᶑḗḖḕḹ')),
import unicodedata as ud)
[3.8437222586746884, 3.829490737203514, 3.819266963414293]
jmf
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to write fast into a file in python?
On Fri, 17 May 2013 18:20:33 +0300, Carlos Nepomuceno wrote:
### fastwrite5.py ###
import cStringIO
size = 50*1024*1024
value = 0
filename = 'fastwrite5.dat'
x = 0
b = cStringIO.StringIO()
while x size:
line = '{0}\n'.format(value)
b.write(line)
value += 1
x += len(line)+1
Oh, I forgot to mention: you have a bug in this function. You're already
including the newline in the len(line), so there is no need to add one.
The result is that you only generate 44MB instead of 50MB.
f = open(filename, 'w')
f.write(b.getvalue())
f.close()
b.close()
Here are the results of profiling the above on my computer. Including the
overhead of the profiler, it takes just over 50 seconds to run your file
on my computer.
[steve@ando ~]$ python -m cProfile fastwrite5.py
17846645 function calls in 53.575 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 30.561 30.561 53.575 53.575 fastwrite5.py:1(module)
10.0000.0000.0000.000 {cStringIO.StringIO}
59488795.5820.0005.5820.000 {len}
10.0040.0040.0040.004 {method 'close' of
'cStringIO.StringO' objects}
10.0000.0000.0000.000 {method 'close' of 'file' objects}
10.0000.0000.0000.000 {method 'disable' of
'_lsprof.Profiler' objects}
59488799.9790.0009.9790.000 {method 'format' of 'str' objects}
10.1030.1030.1030.103 {method 'getvalue' of
'cStringIO.StringO' objects}
59488797.1350.0007.1350.000 {method 'write' of
'cStringIO.StringO' objects}
10.2110.2110.2110.211 {method 'write' of 'file' objects}
10.0000.0000.0000.000 {open}
As you can see, the time is dominated by repeatedly calling len(),
str.format() and StringIO.write() methods. Actually writing the data to
the file is quite a small percentage of the cumulative time.
So, here's another version, this time using a pre-calculated limit. I
cheated and just copied the result from the fastwrite5 output :-)
# fasterwrite.py
filename = 'fasterwrite.dat'
with open(filename, 'w') as f:
for i in xrange(5948879): # Actually only 44MB, not 50MB.
f.write('%d\n' % i)
And the profile results are about twice as fast as fastwrite5 above, with
only 8 seconds in total writing to my HDD.
[steve@ando ~]$ python -m cProfile fasterwrite.py
5948882 function calls in 28.840 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 20.592 20.592 28.840 28.840 fasterwrite.py:1(module)
10.0000.0000.0000.000 {method 'disable' of
'_lsprof.Profiler' objects}
59488798.2290.0008.2290.000 {method 'write' of 'file' objects}
10.0190.0190.0190.019 {open}
Without the overhead of the profiler, it is a little faster:
[steve@ando ~]$ time python fasterwrite.py
real0m16.187s
user0m13.553s
sys 0m0.508s
Although it is still slower than the heavily optimized dd command,
but not unreasonably slow for a high-level language:
[steve@ando ~]$ time dd if=fasterwrite.dat of=copy.dat
90781+1 records in
90781+1 records out
46479922 bytes (46 MB) copied, 0.737009 seconds, 63.1 MB/s
real0m0.786s
user0m0.071s
sys 0m0.595s
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
RE: How to write fast into a file in python?
You've hit the bullseye! ;)
Thanks a lot!!!
Oh, I forgot to mention: you have a bug in this function. You're already
including the newline in the len(line), so there is no need to add one.
The result is that you only generate 44MB instead of 50MB.
That's because I'm running on Windows.
What's the fastest way to check if '\n' translates to 2 bytes on file?
Here are the results of profiling the above on my computer. Including the
overhead of the profiler, it takes just over 50 seconds to run your file
on my computer.
[steve@ando ~]$ python -m cProfile fastwrite5.py
17846645 function calls in 53.575 seconds
Didn't know the cProfile module.Thanks a lot!
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 30.561 30.561 53.575 53.575 fastwrite5.py:1(module)
1 0.000 0.000 0.000 0.000 {cStringIO.StringIO}
5948879 5.582 0.000 5.582 0.000 {len}
1 0.004 0.004 0.004 0.004 {method 'close' of 'cStringIO.StringO' objects}
1 0.000 0.000 0.000 0.000 {method 'close' of 'file' objects}
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
5948879 9.979 0.000 9.979 0.000 {method 'format' of 'str' objects}
1 0.103 0.103 0.103 0.103 {method 'getvalue' of 'cStringIO.StringO' objects}
5948879 7.135 0.000 7.135 0.000 {method 'write' of 'cStringIO.StringO'
objects}
1 0.211 0.211 0.211 0.211 {method 'write' of 'file' objects}
1 0.000 0.000 0.000 0.000 {open}
As you can see, the time is dominated by repeatedly calling len(),
str.format() and StringIO.write() methods. Actually writing the data to
the file is quite a small percentage of the cumulative time.
So, here's another version, this time using a pre-calculated limit. I
cheated and just copied the result from the fastwrite5 output :-)
# fasterwrite.py
filename = 'fasterwrite.dat'
with open(filename, 'w') as f:
for i in xrange(5948879): # Actually only 44MB, not 50MB.
f.write('%d\n' % i)
I had the same idea but kept the original method because I didn't want to waste
time creating a function for calculating the actual number of iterations needed
to deliver 50MB of data. ;)
And the profile results are about twice as fast as fastwrite5 above, with
only 8 seconds in total writing to my HDD.
[steve@ando ~]$ python -m cProfile fasterwrite.py
5948882 function calls in 28.840 seconds
Ordered by: standard name
ncalls tottime percall cumtime percall filename:lineno(function)
1 20.592 20.592 28.840 28.840 fasterwrite.py:1(module)
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
5948879 8.229 0.000 8.229 0.000 {method 'write' of 'file' objects}
1 0.019 0.019 0.019 0.019 {open}
I thought there would be a call to format method by '%d\n' % i. It seems the
% operator is a lot faster than format.
I just stopped using it because I read it was going to be deprecated. :(
Why replace such a great and fast operator by a slow method? I mean, why format
is been preferred over %?
Without the overhead of the profiler, it is a little faster:
[steve@ando ~]$ time python fasterwrite.py
real 0m16.187s
user 0m13.553s
sys 0m0.508s
Although it is still slower than the heavily optimized dd command,
but not unreasonably slow for a high-level language:
[steve@ando ~]$ time dd if=fasterwrite.dat of=copy.dat
90781+1 records in
90781+1 records out
46479922 bytes (46 MB) copied, 0.737009 seconds, 63.1 MB/s
real 0m0.786s
user 0m0.071s
sys 0m0.595s
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
--
http://mail.python.org/mailman/listinfo/python-list
RE: How to write fast into a file in python?
Think the following update will make the code more portable:
x += len(line)+len(os.linesep)-1
Not sure if it's the fastest way to achieve that. :/
On Fri, 17 May 2013 18:20:33 +0300, Carlos Nepomuceno wrote:
### fastwrite5.py ###
import cStringIO
size = 50*1024*1024
value = 0
filename = 'fastwrite5.dat'
x = 0
b = cStringIO.StringIO()
while x size:
line = '{0}\n'.format(value)
b.write(line)
value += 1
x += len(line)+1
Oh, I forgot to mention: you have a bug in this function. You're already
including the newline in the len(line), so there is no need to add one.
The result is that you only generate 44MB instead of 50MB.
--
http://mail.python.org/mailman/listinfo/python-list
Back-end Python Developer Seeking Telecommute Work
I am seeking part/full time work as a back-end Python developer (telecommute or Utah only). I have been maintaining a Debian/Python/Django/Apache/PostgreSQL/PHP/MySql web application for 3 years on my own. I do all the development, database and system management myself. I can setup a complete system from scratch remotely or setup the hosting service. I have experience with Virtualbox and ProxMox virtualization. I have expereince in all areas of IT including telecom, programming, networking, hardware and software. Worked with Retail stores, warehousing systems, accounting software, data interchange, hardware interfacing, etc. Regards, Rob -- http://mail.python.org/mailman/listinfo/python-list
Re: any cherypy powred sites I can check out?
On Fri, 17 May 2013 18:15:38 +0100, Mark Lawrence wrote: On 17/05/2013 01:00, visphatesj...@gmail.com wrote: fuck straight off I assume you're the author of How to win friends and influence people? There are very few posters to this NG in the Hurry bozo bin, but OP is now one. -- http://mail.python.org/mailman/listinfo/python-list
Re: Back-end Python Developer Seeking Telecommute Work
Rob Sutton於 2013年5月18日星期六UTC+8上午2時36分07秒寫道： I am seeking part/full time work as a back-end Python developer (telecommute or Utah only). I have been maintaining a Debian/Python/Django/Apache/PostgreSQL/PHP/MySql web application for 3 years on my own. I do all the development, database and system management myself. I can setup a complete system from scratch remotely or setup the hosting service. I have experience with Virtualbox and ProxMox virtualization. I have expereince in all areas of IT including telecom, programming, networking, hardware and software. Worked with Retail stores, warehousing systems, accounting software, data interchange, hardware interfacing, etc. Regards, Rob Nic,e I am working on several similar projects including EDA automations with expert systems and some VLSI-FPGA IP cores. Anyway I have to get some US patents first. -- http://mail.python.org/mailman/listinfo/python-list
Continuous Deployment Style Build System for Python
Defend Against Fruit is focused on providing a pragmatic, continuous deployment style build system for Python. Current Python build systems do not properly account for the needs of effective continuous deployment. This package extends the Python tooling to add the missing pieces, including integration with Artifactory. With an eye to agile development principles and fast-feedback, we want a build system which satisfies the following goals: * Every SCM change-set committed should result in a potentially shippable release candidate. * When a defect is introduced, we want to immediately detect and isolate the offending SCM change-set. This is true even if the defect was introduced into a library we depend upon. * Library management should be so easy as to never impede code changes, even in multi-component architecture. More details available at: http://teamfruit.github.io/defend_against_fruit/ License: Apache Public License v2 Authors: James Carpenter jcarpenter621 at yahoo.com LinkedIn: http://www.linkedin.com/in/jamescarpenter1 Matthew Tardiff mattrix at gmail.com LinkedIn: http://www.linkedin.com/in/matthewtardiff -- http://mail.python.org/mailman/listinfo/python-list
Re: any cherypy powred sites I can check out?
On 2013-05-17, Mark Lawrence breamore...@yahoo.co.uk wrote: On 17/05/2013 01:00, visphatesj...@gmail.com wrote: fuck straight off I assume you're the author of How to win friends and influence people? 'I recently wrote a book called How to Get Along with Everybody. I didn't write it myself--I wrote it some asshole.' --Steve Martin -- Neil Cerutti -- http://mail.python.org/mailman/listinfo/python-list
Re: How to write fast into a file in python?
On Fri, 17 May 2013 21:18:15 +0300, Carlos Nepomuceno wrote: I thought there would be a call to format method by '%d\n' % i. It seems the % operator is a lot faster than format. I just stopped using it because I read it was going to be deprecated. :( Why replace such a great and fast operator by a slow method? I mean, why format is been preferred over %? That is one of the most annoying, pernicious myths about Python, probably second only to the GIL makes Python slow (it actually makes it fast). String formatting with % is not deprecated. It will not be deprecated, at least not until Python 4000. The string format() method has a few advantages: it is more powerful, consistent and flexible, but it is significantly slower. Probably the biggest disadvantage to % formatting, and probably the main reason why it is discouraged, is that it treats tuples specially. Consider if x is an arbitrary object, and you call %s % x: py %s % 23 # works '23' py %s % [23, 42] # works '[23, 42]' and so on for *almost* any object. But if x is a tuple, strange things happen: py %s % (23,) # tuple with one item looks like an int '23' py %s % (23, 42) # tuple with two items fails Traceback (most recent call last): File stdin, line 1, in module TypeError: not all arguments converted during string formatting So when dealing with arbitrary objects that you cannot predict what they are, it is better to use format. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Two Dictionaries and a Sum!
Confusing subject for a confusing problem (to a novice like me of course!)
Thx for the help in advance folks
I have (2) dictionaries:
prices = {
banana: 4,
apple: 2,
orange: 1.5,
pear: 3
}
stock = {
banana: 6,
apple: 0,
orange: 32,
pear: 15
}
Here's my instructions:
consider this as an inventory and calculate the sum (thats 4*6 = 24 bananas!)
HERES MY CODE:
for key in prices:
print prices[key]*stock[key]
HERES THE OUTPUT:
48.0
45
24
0
ISSUE:
I need to find a way to add all of those together...any pointers?
--
http://mail.python.org/mailman/listinfo/python-list
Re: Two Dictionaries and a Sum!
for key in prices.keys():
print prices[key]*stock[key]
On Sat, May 18, 2013 at 12:19 PM, Bradley Wright
bradley.wright@gmail.com wrote:
Confusing subject for a confusing problem (to a novice like me of course!)
Thx for the help in advance folks
I have (2) dictionaries:
prices = {
banana: 4,
apple: 2,
orange: 1.5,
pear: 3
}
stock = {
banana: 6,
apple: 0,
orange: 32,
pear: 15
}
Here's my instructions:
consider this as an inventory and calculate the sum (thats 4*6 = 24
bananas!)
HERES MY CODE:
for key in prices:
print prices[key]*stock[key]
HERES THE OUTPUT:
48.0
45
24
0
ISSUE:
I need to find a way to add all of those together...any pointers?
--
http://mail.python.org/mailman/listinfo/python-list
--
*Space Lee*
--
http://mail.python.org/mailman/listinfo/python-list
how to run another file inside current file?
hi, I want to run a another file inside a ached.add_cron_job(..). how is it possible, please help me, I have a file otherFile.py for execution inside current file. I know it is very easy question but i m unable to get anything, please help me. example -- import otherFile from apscheduler.scheduler import Scheduler sched = Scheduler() sched.start() def job_function(): # Can I here add that file for execution, Or can i add that file directly inside cron? sched.add_cron_job(job_function, month='1-12', day='1-31', hour='0-23',minute='44-49') -- http://mail.python.org/mailman/listinfo/python-list
Re: Two Dictionaries and a Sum!
On Sat, May 18, 2013 at 2:19 PM, Bradley Wright
bradley.wright@gmail.com wrote:
Confusing subject for a confusing problem (to a novice like me of course!)
Thx for the help in advance folks
I have (2) dictionaries:
prices = {
banana: 4,
apple: 2,
orange: 1.5,
pear: 3
}
stock = {
banana: 6,
apple: 0,
orange: 32,
pear: 15
}
Here's my instructions:
consider this as an inventory and calculate the sum (thats 4*6 = 24 bananas!)
Let me reword your problem a little, maybe it'll be a bit clearer.
You're trying to calculate the total value of all stock on hand, eg
for insurance purposes. That's not 24 bananas, that's $24 of bananas.
And the part you want now is to get the total value of your entire
stock. Great! You're very close to there...
HERES MY CODE:
for key in prices:
print prices[key]*stock[key]
ISSUE:
I need to find a way to add all of those together...any pointers?
... you just need to accumulate a sum. Since this is almost certainly
homework, I won't give you the answer, but here are a few pointers:
* You'll need a single variable (I use the term sloppily, Python
doesn't actually have variables per se) which will collect the final
total.
* Inside your loop, you're currently printing out an int/float with
the value of the current item. Just add it onto your accumulator.
* Python will happily work with integers and floats together, so you
can just do what's obvious and it'll work.
See where that takes you. Have fun! :)
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Two Dictionaries and a Sum!
total,amount=0,0 for key in prices.keys(): price=prices[key]*stock[key] total+=price print %s %s % (price,total) -- http://mail.python.org/mailman/listinfo/python-list
Re: How to write fast into a file in python?
On Sat, May 18, 2013 at 2:01 PM, Steven D'Aprano
steve+comp.lang.pyt...@pearwood.info wrote:
Consider if x is an arbitrary object, and you call %s % x:
py %s % 23 # works
'23'
py %s % [23, 42] # works
'[23, 42]'
and so on for *almost* any object. But if x is a tuple, strange things
happen
Which can be guarded against by wrapping it up in a tuple. All you're
seeing is that the shortcut notation for a single parameter can't
handle tuples.
def show(x):
return %s % (x,)
show(23)
'23'
show((23,))
'(23,)'
show([23,42])
'[23, 42]'
One of the biggest differences between %-formatting and str.format is
that one is an operator and the other a method. The operator is always
going to be faster, but the method can give more flexibility (not that
I've ever needed or wanted to override anything).
def show_format(x):
return {}.format(x) # Same thing using str.format
dis.dis(show)
2 0 LOAD_CONST 1 ('%s')
3 LOAD_FAST0 (x)
6 BUILD_TUPLE 1
9 BINARY_MODULO
10 RETURN_VALUE
dis.dis(show_format)
2 0 LOAD_CONST 1 ('{}')
3 LOAD_ATTR0 (format)
6 LOAD_FAST0 (x)
9 CALL_FUNCTION1 (1 positional, 0 keyword pair)
12 RETURN_VALUE
Attribute lookup and function call versus binary operator. Potentially
a lot of flexibility, versus basically hard-coded functionality. But
has anyone ever actually made use of it?
str.format does have some cleaner features, like naming of parameters:
{foo} vs {bar}.format(foo=1,bar=2)
'1 vs 2'
%(foo)s vs %(bar)s%{'foo':1,'bar':2}
'1 vs 2'
Extremely handy when you're working with hugely complex format
strings, and the syntax feels a bit clunky in % (also, it's not
portable to other languages, which is one of %-formatting's biggest
features). Not a huge deal, but if you're doing a lot with that, it
might be a deciding vote.
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
Re: Two Dictionaries and a Sum!
total,amount=0,0 for key in prices.keys(): amount=prices[key]*stock[key] total+=amount print %s %s % (amount,total) -- http://mail.python.org/mailman/listinfo/python-list
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Changes by Bohuslav Slavek Kabrda bkab...@redhat.com: -- nosy: +bkabrda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17222] py_compile.compile() explicitly sets st_mode for written files
Armin Rigo added the comment: Can someone confirm the answer to Arfrever's original question: a seemingly innocent use case of py_compile.compile(), which works fine until Python 3.3, when executed as root, can in Python 3.4 fundamentally break down a complete Posix system in a non-obvious way? -- nosy: +arigo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17222 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17967] urllib2.open failed to access a url when a perent directory of the url is permission denied
zhaoqifa added the comment: I had run test by python ./urllib.py -t, test steps for accessing /etc/passwd, file:/etc/passwd, file://localhost/etc/passwd passed, steps for ftp://ftp.gnu.org/pub/README not tested because my work env is not free for internet accessing. And, I have not tried version 3.3. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17967 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17993] Missed comma causes unintentional implicit string literal concatenation
Serhiy Storchaka added the comment: I haven't found any similar bugs, but I have found other bug in Tools/scripts/abitype.py. -- assignee: - serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17993 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17993] Missed comma causes unintentional implicit string literal concatenation
Serhiy Storchaka added the comment: Oh, committed in changesets 26531f21bc4c and 27cc0e0b7637. -- resolution: - fixed stage: commit review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17993 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17222] py_compile.compile() explicitly sets st_mode for written files
Antoine Pitrou added the comment: Yes, the switch to renaming can change behaviour in some corner cases (I say corner cases because the expected situation is to have your pyc files be regular files, not symlinks or character devices). Python is certainly not the only application where you can bust /dev/null by specifying it as target location. Mutating a file in place is a source of unpredictable issues with concurrent access of the file being written to, which is why it was changed to renaming. I think the class of issues which was solved (presumably) is much more important than the use case of compiling to /dev/null. As for symlinks, I'd naively expect breaking symlinks to be a feature, but YMMV. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17222 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9566] Compilation warnings under x64 Windows
STINNER Victor added the comment: Oh, I fixed some warnings yesterday, but I forgot this issue. changeset: 83802:ef5dd5bda323 user:Victor Stinner victor.stin...@gmail.com date:Fri May 17 00:12:04 2013 +0200 files: Python/getargs.c Python/traceback.c description: Fix compilater warnings on Windows 64-bit changeset: 83801:16a00dd78cd0 user:Victor Stinner victor.stin...@gmail.com date:Fri May 17 00:04:56 2013 +0200 files: Python/formatter_unicode.c description: Fix a compilater warning on Windows 64-bit changeset: 83800:e5dc97fb304d user:Victor Stinner victor.stin...@gmail.com date:Thu May 16 23:48:01 2013 +0200 files: Python/ceval.c description: Fix a compilater warning on Windows 64-bit idx variable is used for a tuple indexn so use Py_ssize_t (not int). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9566 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17222] py_compile.compile() explicitly sets st_mode for written files
Armin Rigo added the comment: My point is that in five years' time some existing well-tested sysadmin-like program will be run with Python 3.4 for the first time, and suddenly whole systems will break. I don't want to sound negative but this is the worst behavior ever (with the possible exception of programs that do bad things beyond your own system, like viruses). Please at least crash with an exception instead: add one easy check to py_compile.compile() and complain if cfile points to an existing non-regular file. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17222 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14596] struct.unpack memory leak
Roundup Robot added the comment: New changeset 6707637f68ca by Serhiy Storchaka in branch 'default': Issue #14596: The struct.Struct() objects now use more compact implementation. http://hg.python.org/cpython/rev/6707637f68ca -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14596 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
STINNER Victor added the comment: Are multiple wildcards per fragment even specified? I don't know the standard, but it sounds strange to have more than one wildcard per part of an URL. *.*.*.google.com looks valid to me, whereas *a*a*a*.google.com looks very suspicious. Said differently, I expect: assert max(part.count(*) for part in url.split(.)) = 1 * pattern is replace with '[^.]+' regex, so I may not cause the exponential complexity issue. (I didn't check.) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14596] struct.unpack memory leak
Serhiy Storchaka added the comment: We already lost this improvement for 3.3. :( Could we now close the issue? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14596 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Florian Weimer added the comment: * pattern is replace with '[^.]+' regex, so I may not cause the exponential complexity issue. (I didn't check.) A possessive quantifier might also help, that is [^.]+?. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Marc-Andre Lemburg added the comment: SSL certificate hostname matching is defined in RFC 2818: * http://www.ietf.org/rfc/rfc2818.txt It's not very verbose on how exactly matching should be done: Names may contain the wildcard character * which is considered to match any single domain name component or component fragment. E.g., *.a.com matches foo.a.com but not bar.foo.a.com. f*.com matches foo.com but not bar.com. Given that it's underspecified, I doubt that anyone using wildcards in certificates for valid purposes would risk using anything but very simply prefix/suffix matching - most certainly not any matching that would require backtracking to succeed. There are several variants out there of how the matching is done. See e.g. http://search-hadoop.com/c/Hadoop:hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/ssl/SSLHostnameVerifier.java||dns -- nosy: +lemburg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Antoine Pitrou added the comment:
Non-greedy matching actually makes things worse :-)
$ ./python -m timeit -s import re; pat = re.compile('\A*a*a*a\Z'.replace('*',
'[^.]+'), re.IGNORECASE) pat.match('a' * 100 +'z')
100 loops, best of 3: 3.31 msec per loop
$ ./python -m timeit -s import re; pat = re.compile('\A*a*a*a\Z'.replace('*',
'[^.]+?'), re.IGNORECASE) pat.match('a' * 100 +'z')
100 loops, best of 3: 6.91 msec per loop
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17980
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17222] py_compile.compile() explicitly sets st_mode for written files
Arfrever Frehtes Taifersar Arahesis added the comment: I fixed syntax sanity check in Portage (package manager in Gentoo) to avoid breaking system. This sanity check is always run during upgrade of Portage itself. http://git.overlays.gentoo.org/gitweb/?p=proj/portage.git;a=commitdiff;h=e8d1337b5936ee058872d25f21b914e96bcbf677 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17222 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Antoine Pitrou added the comment: Florian, I'm actually surprised by your assertion that OpenSSL supports a single wildcard character. Last I looked, I couldn't find any hostname matching function in OpenSSL (which is why I had to write our own). Could you point me to the relevant piece of code? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Florian Weimer added the comment: Antoine, support for OpenSSL host name matching is quite new: http://www.openssl.org/docs/crypto/X509_check_host.html -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Antoine Pitrou added the comment: libcurl supports a single wildcard for the whole domain name pattern (not even one per fragment), as per lib/hostcheck.c (this is when linked against OpenSSL; when linked against GnuTLS, curl will use the GnuTLS-provided matching function) Based on all the evidence, I think allowing one wildcard per fragment is sufficient. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Antoine Pitrou added the comment: Antoine, support for OpenSSL host name matching is quite new Ah, thanks. I was looking in 1.0.1e. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue14596] struct.unpack memory leak
Antoine Pitrou added the comment: Closing indeed! -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue14596 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17222] py_compile.compile() explicitly sets st_mode for written files
Antoine Pitrou added the comment: I fixed syntax sanity check in Portage (package manager in Gentoo) to avoid breaking system. This sanity check is always run during upgrade of Portage itself. Well, we could add a dedicated function for sanity checking, then. -- title: py_compile.compile() explicitly sets st_mode for written files - py_compile.compile() explicitly sets st_mode for written files ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17222 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17222] py_compile.compile() explicitly sets st_mode for written files
Antoine Pitrou added the comment: Please at least crash with an exception instead: add one easy check to py_compile.compile() and complain if cfile points to an existing non-regular file. That's a good point (except that symlinks should probably be allowed too). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17222 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17222] py_compile.compile() replaces target files, breaking special files and symlinks
Changes by Arfrever Frehtes Taifersar Arahesis arfrever@gmail.com: -- title: py_compile.compile() explicitly sets st_mode for written files - py_compile.compile() replaces target files, breaking special files and symlinks ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17222 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17563] Excessive resizing of dicts when used as a cache
Roundup Robot added the comment: New changeset cd2457463eeb by Raymond Hettinger in branch '3.3': Issue #17563: Fix dict resize performance regression. http://hg.python.org/cpython/rev/cd2457463eeb -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17563] Excessive resizing of dicts when used as a cache
Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- resolution: - fixed status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17563 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17947] Code, test, and doc review for PEP-0435 Enum
Changes by Donald Stufft donald.stu...@gmail.com: -- nosy: +dstufft ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17947 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Marc-Andre Lemburg added the comment: Here's another long discussions about SSL hostname matching that may provide some useful insights: * https://bugzilla.mozilla.org/show_bug.cgi?id=159483 Note how RFC 2595 doesn't even allow sub-string matching. It only allows '*' to be used as component. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17979] Cannot build 2.7 with --enable-unicode=no
Serhiy Storchaka added the comment: Ubuntu 32-bit, gcc 4.6.3. The bug requires 64 bit. This patch should fix it. -- components: +Build, Extension Modules, Regular Expressions, Unicode keywords: +patch nosy: +mrabarnett stage: - patch review Added file: http://bugs.python.org/file30290/re_nounicode.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17979 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17947] Code, test, and doc review for PEP-0435 Enum
Donald Stufft added the comment: Small nitpick, weakref is imported but not used in the latest patch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17947 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17979] Cannot build 2.7 with --enable-unicode=no
Marc-Andre Lemburg added the comment: On 17.05.2013 13:11, Serhiy Storchaka wrote: Serhiy Storchaka added the comment: Ubuntu 32-bit, gcc 4.6.3. The bug requires 64 bit. This patch should fix it. int and long are the same size on Linux 64-bit platforms. You probably want to use short instead. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17979 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17979] Cannot build 2.7 with --enable-unicode=no
Marc-Andre Lemburg added the comment: On 17.05.2013 13:42, Marc-Andre Lemburg wrote: Marc-Andre Lemburg added the comment: On 17.05.2013 13:11, Serhiy Storchaka wrote: Serhiy Storchaka added the comment: Ubuntu 32-bit, gcc 4.6.3. The bug requires 64 bit. This patch should fix it. int and long are the same size on Linux 64-bit platforms. Sorry, scratch that. It's the same on Windows x64, not Linux x64. You probably want to use short instead. This should still be safer. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17979 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17979] Cannot build 2.7 with --enable-unicode=no
Serhiy Storchaka added the comment: SRE_CODE should be at least 32-bit for support of long regexpes. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17979 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15758] FileIO.readall() has worst case O(n^2) complexity
Richard Oudkerk added the comment: Updated patch adressing Antoine's comments. -- Added file: http://bugs.python.org/file30291/readall.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15758 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15758] FileIO.readall() has worst case O(n^2) complexity
Changes by Richard Oudkerk shibt...@gmail.com: Removed file: http://bugs.python.org/file26985/readall-combined.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15758 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15758] FileIO.readall() has worst case O(n^2) complexity
Changes by Richard Oudkerk shibt...@gmail.com: Added file: http://bugs.python.org/file26986/readall-benchmark.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15758 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15758] FileIO.readall() has worst case O(n^2) complexity
Changes by Richard Oudkerk shibt...@gmail.com: Removed file: http://bugs.python.org/file30287/readall.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15758 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Antoine Pitrou added the comment: Attached patch forbidding more than one wildcard per fragment. -- stage: needs patch - patch review Added file: http://bugs.python.org/file30292/ssl_wildcard_dos2.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Christian Heimes added the comment: I still think that sub string wildcard should not match the IDN xn-- prefix. With current code the rules x*.example.de gives a positive match for götter.example.de. ugötter.example.de.encode(idna) 'xn--gtter-jua.example.de' -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Antoine Pitrou added the comment: I still think that sub string wildcard should not match the IDN xn-- prefix. With current code the rules x*.example.de gives a positive match for götter.example.de. You should open a separate issue for this (possibly with a patch). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17997] ssl.match_hostname(): sub string wildcard should not match IDNA prefix
New submission from Christian Heimes: Python's ssl.match_hostname() does sub string matching as specified in RFC 2818: Names may contain the wildcard character * which is considered to match any single domain name component or component fragment. E.g., *.a.com matches foo.a.com but not bar.foo.a.com. f*.com matches foo.com but not bar.com. The RFC doesn't specify how internationalized domain names shoould be handled because it predates RFC 5890 for IDNA by many year. IDNA are prefixed with xn--, e.g. ugötter.example.de.encode(idna) == xn--gtter-jua.example.de. This can result into false positive matches for a rule like x*.example.de. Chrome has special handling for IDN prefix in X509Certificate::VerifyHostname() http://src.chromium.org/viewvc/chrome/trunk/src/net/cert/x509_certificate.cc Also see #17980 -- messages: 189454 nosy: christian.heimes, pitrou priority: critical severity: normal stage: needs patch status: open title: ssl.match_hostname(): sub string wildcard should not match IDNA prefix type: security versions: Python 3.3, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17997 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17980] CVE-2013-2099 ssl.match_hostname() trips over crafted wildcard names
Christian Heimes added the comment: #17997 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17980 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17951] TypeError during gdb backtracing
R. David Murray added the comment: Thanks for the report and patch. The next step required for this issue is for someone to write a unit test. -- keywords: +easy nosy: +r.david.murray stage: - test needed versions: +Python 2.7, Python 3.3, Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17951 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17965] argparse does not dest.replace('-', '_') for positionals
R. David Murray added the comment: This is a duplicate of issue 15125. (FYI there are many many unix commands that use options of the form -XXX-. This is the standard for the gnu 'long option' format.) -- nosy: +r.david.murray resolution: - duplicate stage: - committed/rejected status: open - closed superseder: - argparse: positional arguments containing - in name not handled well ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17965 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15125] argparse: positional arguments containing - in name not handled well
Changes by R. David Murray rdmur...@bitdance.com: -- nosy: +paul.j3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15125 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17973] '+=' on a list inside tuple both succeeds and raises an exception
R. David Murray added the comment: This gets reported periodically. I wonder if it is time for a FAQ entry. There doesn't seem to be one yet. -- nosy: +r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17973 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15758] FileIO.readall() has worst case O(n^2) complexity
Changes by Richard Oudkerk shibt...@gmail.com: Added file: http://bugs.python.org/file30293/readall.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15758 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15758] FileIO.readall() has worst case O(n^2) complexity
Changes by Richard Oudkerk shibt...@gmail.com: Removed file: http://bugs.python.org/file30291/readall.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15758 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15758] FileIO.readall() has worst case O(n^2) complexity
Richard Oudkerk added the comment: New patch. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15758 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15392] Create a unittest framework for IDLE
Changes by Alejandro Rodas alexrd...@gmail.com: -- nosy: +alex.rodas ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15392 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17973] '+=' on a list inside tuple both succeeds and raises an exception
Ronald Oussoren added the comment: You've got a point there. What about this patch (but then with proper english grammer)? BTW. Actually fixing this wart would be possible, but at a significant cost: you'd have to change the implementation of LHS += RHS from: tmp = LHS tmp = tmp.__iadd__(RHS) LHS = tmp to: tmp = LHS LHS = tmp tmp = tmp.__iadd__(RHS) LHS = tmp The odd assignment on the second line would detect that the LHS is immutable in 99+% of use cases before updating the RHS. My gut feeling is that an implementation of this would have too high a cost (both in runtime performance and in a more complicated compiler), although it would be interesting to actually see a patch. -- keywords: +patch Added file: http://bugs.python.org/file30294/faq-update.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17973 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17998] internal error in regular expression engine
New submission from Jeroen Demeyer:
On Linux Ubuntu 13.04, i686:
$ uname -a
Linux arando 3.5.0-26-generic #42-Ubuntu SMP Fri Mar 8 23:20:06 UTC 2013 i686
i686 i686 GNU/Linux
$ python
Python 2.7.5 (default, May 17 2013, 18:43:24)
[GCC 4.7.3] on linux2
Type help, copyright, credits or license for more information.
import re
re.compile('(.*)\.[0-9]*\.[0-9]*$', re.I|re.S).findall('3.0.0')
Traceback (most recent call last):
File stdin, line 1, in module
RuntimeError: internal error in regular expression engine
This is a 2.7.5 regression, 2.7.4 worked fine.
--
components: Library (Lib)
messages: 189461
nosy: jdemeyer
priority: normal
severity: normal
status: open
title: internal error in regular expression engine
versions: Python 2.7
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17998
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15758] FileIO.readall() has worst case O(n^2) complexity
Changes by Richard Oudkerk shibt...@gmail.com: Added file: http://bugs.python.org/file30295/readall.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15758 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue15758] FileIO.readall() has worst case O(n^2) complexity
Changes by Richard Oudkerk shibt...@gmail.com: Removed file: http://bugs.python.org/file30293/readall.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue15758 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17998] internal error in regular expression engine
Changes by Ezio Melotti ezio.melo...@gmail.com: -- components: +Regular Expressions nosy: +benjamin.peterson, ezio.melotti, mrabarnett type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17998 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17965] argparse does not dest.replace('-', '_') for positionals
R. David Murray added the comment: That should have been --XXX-YYY, of course. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17965 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17989] ElementTree.Element broken attribute setting
Joe Stuart added the comment: It looks like it's being called from the c extension. I would think it should still throw an exception though? e = etree.Element e.ham = 1 Traceback (most recent call last): File stdin, line 1, in module TypeError: can't set attributes of built-in/extension type 'xml.etree.ElementTree.Element' -- nosy: +jjstuart ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17989 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17999] test_super fails in refleak runs
New submission from Antoine Pitrou: $ ./python -m test -R 3:3 test_super [1/1] test_super beginning 6 repetitions 123456 test test_super failed -- Traceback (most recent call last): File /home/antoine/cpython/default/Lib/test/test_super.py, line 97, in test_various___class___pathologies self.assertIs(X.x, type(self)) AssertionError: 42 is not class 'test.test_super.TestSuper' -- components: Tests messages: 189464 nosy: benjamin.peterson, pitrou priority: normal severity: normal stage: needs patch status: open title: test_super fails in refleak runs type: behavior versions: Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17999 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17957] remove outdated (and unexcellent) paragraph in whatsnew
Éric Araujo added the comment: I don’t see any benefit at all in changing this paragraph of a whatsnew document. It was useful and accurate for the 2.6 release, it’s still accessible for the record, let’s keep it. -- nosy: +eric.araujo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17957 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17998] internal error in regular expression engine
Benjamin Peterson added the comment: 27162465316f -- assignee: - serhiy.storchaka nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17998 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17998] internal error in regular expression engine
Changes by Christian Heimes li...@cheimes.de: -- nosy: +christian.heimes ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17998 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17998] internal error in regular expression engine
Benjamin Peterson added the comment: Also, note this particular case only reproduces on 32 bit. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17998 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17999] test_super fails in refleak runs
Roundup Robot added the comment: New changeset b4553d22c163 by Benjamin Peterson in branch 'default': reset __class__, so multiple runs don't fail (closes #17999) http://hg.python.org/cpython/rev/b4553d22c163 -- nosy: +python-dev resolution: - fixed stage: needs patch - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17999 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17998] internal error in regular expression engine
Christian Heimes added the comment:
I'm able to confirm Benjamin's notes. The regexp works on 64bit Linux but fails
with a 32bit build:
$ CFLAGS=-m32 LDFLAGS=-m32 ./configure
$ make -j10
$ ./python -c import re; print(re.compile('(.*)\.[0-9]*\.[0-9]*$',
re.I|re.S).findall('3.0.0'))
Traceback (most recent call last):
File string, line 1, in module
RuntimeError: internal error in regular expression engine
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17998
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17995] report，中 高 层 管 理 技 能158766
Changes by Ezio Melotti ezio.melo...@gmail.com: -- Removed message: http://bugs.python.org/msg189390 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17995 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9338] argparse optionals with nargs='?', '*' or '+' can't be followed by positionals
paul j3 added the comment:
This patch implements, I think, the ideas bethard proposed. It is test patch,
not intended for production.
Most of work is in ArgumentParser._get_alt_length() which
- generates a pattern along the lines bethard proposed
- generates a string like arg_strings_pattern, but with optionals strings
('-x') instead of 'O'.
- runs a match
- from groups like '-xAAA', creates dict entries like:
alt_opt_length['x'] = 3
Later, in consume_optionals(), this alternative count replaces arg_count if it
is lower. The next consume_positionals() then takes care of consuming the
unconsumed arguments.
If _get_alt_length() has any problems, it logs an error, and returns an
otherwise empty dict. So it 'fails' quietly without affecting regular parsing.
Reasons for failing include (for now) the use of subparsers, optionals with
explicit args, and special prefix_chars. With exclusions like this,
test_argparse.py runs without errors or failures.
Since this is still a testing vehicle, it writes an issue9338.log file with
debugging entries.
This version works, but is both not sufficiently general and too general. As
bethard notes, the testing pattern could get very large if there are many
optionals. Ideally the pattern will allow the optionals in any order and
combination between positionals. The ambiguities that I discussed in the
previous 2 posts disappear if the patching pattern is sufficiently general.
But I also suspect it is too general. It does not need to match every case,
just those where an optional is consuming arguments that should go to a
positional. But if we come up with something more specific, this could still
be a useful testing tool.
--
Added file: http://bugs.python.org/file30296/issue9338_3.patch
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue9338
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17973] '+=' on a list inside tuple both succeeds and raises an exception
Andy Chugunov added the comment: Thank you for the clarification! The exception is appropriate as tuples have to stay immutable. Got it. Could you please also explain a bit about the append() call? Should it in theory raise an exception as well or is such clean behavior intended? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17973 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9338] argparse optionals with nargs='?', '*' or '+' can't be followed by positionals
paul j3 added the comment: This is a test file for the patch I just submitted. It is not a formal unitttest, but uses print output as much as assert. Cases include the example bethard used, as well as ones from test_argparse.py that initially caused problems. -- Added file: http://bugs.python.org/file30297/test_9338.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9338 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17998] internal error in regular expression engine
Matthew Barnett added the comment:
Here are some simpler examples of the bug:
re.compile('.*yz', re.S).findall('xyz')
re.compile('.?yz', re.S).findall('xyz')
re.compile('.+yz', re.S).findall('xyz')
Unfortunately I find it difficult to see what's happening when single-stepping
through the code because of the macros. :-(
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue17998
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17989] ElementTree.Element broken attribute setting
Joe Stuart added the comment: At the end of ElementTree all of the c accelerators are being imported and it looks like only XMLParser is being used. Here is a patch that only imports XMLParser. -- keywords: +patch Added file: http://bugs.python.org/file30298/ElementTree.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17989 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17998] internal error in regular expression engine
Serhiy Storchaka added the comment: Here is a patch which should fix this bug. I still have to look for similar bugs and write tests. -- keywords: +patch stage: - patch review versions: +Python 3.3, Python 3.4 Added file: http://bugs.python.org/file30299/re_unsigned_ptrdiff.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17998 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17998] internal error in regular expression engine
Serhiy Storchaka added the comment: Thank you Matthew for simpler examples. They helped and I'll use them in the tests. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17998 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17989] ElementTree.Element broken attribute setting
Changes by Joe Stuart joe.stu...@gmail.com: Removed file: http://bugs.python.org/file30298/ElementTree.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17989 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue17989] ElementTree.Element broken attribute setting
Joe Stuart added the comment: This patch should fix the issue of the classes being overwritten by the c accelerated ones. -- Added file: http://bugs.python.org/file30300/ElementTree.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue17989 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
