RSFile 1.1 released
I'm pleased to announce the first bugfix release of the RSFile package. Issues addressed: - rejection of unicode keys in kwargs arguments, in some versions of py2.6 - indentation bug swallowing some errors on file opening RSFile aims at providing python with a cross-platform, reliable, and comprehensive file I/O API. It's actually a partial reimplementation of the io module, as compatible possible (it passes latest stdlib io tests), which offers a set of new - and possibly very useful - features: shared/exclusive file record locking, cache synchronization, advanced opening flags, handy stat getters (size, inode...), shortcut I/O functions etc. Unix users might particularly be interested by the workaround that this library provides, concerning the catastrophic fcntl() lock semantic (when any descriptor to a file is closed, your process loses ALL locks acquired on it through other streams). RSFile has been tested with py2.6, py2.7, and py3.2, on win32, linux and freebsd systems, and should theoretically work with IronPython/Jython/PyPy (on Mac OS X too). The technical documentation of RSFile includes a comprehensive description of concepts and gotchas encountered while setting up this library, which could prove useful to anyone interested in getting aware about gory file I/O details. The implementation is currently pure-python, as integration with the C implementation of io module raises lots of issues. So if you need heavy performances, standard python streams will remain necessary. But for most programs and scripts, which just care about data integrity, RSFile should be a proper choice. Downloads: http://pypi.python.org/pypi/RSFile/1.1 Documentation: http://bytebucket.org/pchambon/python-rock-solid-tools/wiki/index.html Regards, Pascal Chambon PS : Due to miscellaneous bugs of python core and stdlib io modules which have been fixed relatively recently, it's advised to have an up-to-date minor version of python (be it 2.6, 2.7 or 3.2) to benefit from RSFile. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: Egos, heartlessness, and limitations
On Fri, 15 Apr 2011 13:55:58 +1000, Ben Finney wrote: Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes: Save yourself a lot of time and just killfile him now. You'll thank me for it later. You never thanked *me* for it, after you eventually realised that was the right decision :-) It's not later enough yet :-P -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
Chris Angelico ros...@gmail.com writes:
That loop will exit at the first gap in the sequence. If that's what
you want, you could try (untested):
from itertools import takewhile
seq = takewhile(lambda n: ('Keyword%d'%n) in dct, count(1))
lst = map(dct.get, seq)
This does 2 lookups per key, which you could avoid by making the code
uglier (untested):
sentinel = object()
seq = (dct.get('Keyword%d'%i,sentinel) for i in count(1))
lst = list(takewhile(lambda x: x != sentinel, seq))
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
On Fri, Apr 15, 2011 at 5:24 PM, Paul Rubin no.email@nospam.invalid wrote:
This does 2 lookups per key, which you could avoid by making the code
uglier (untested):
sentinel = object()
seq = (dct.get('Keyword%d'%i,sentinel) for i in count(1))
lst = list(takewhile(lambda x: x != sentinel, seq))
If I understand this code correctly, that's creating generators,
right? It won't evaluate past the sentinel at all?
That might well be what I'm looking for. A bit ugly, but efficient and
compact. And I can bury some of the ugliness away.
Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
Chris Angelico ros...@gmail.com writes:
sentinel = object()
seq = (dct.get('Keyword%d'%i,sentinel) for i in count(1))
lst = list(takewhile(lambda x: x != sentinel, seq))
If I understand this code correctly, that's creating generators,
right? It won't evaluate past the sentinel at all?
Right, it should stop after hitting the sentinel once.
That might well be what I'm looking for. A bit ugly, but efficient and
compact. And I can bury some of the ugliness away.
It occurs to me, operator.ne might be a little faster than the
interpreted lambda.
--
http://mail.python.org/mailman/listinfo/python-list
Re: memory usage multi value hash
On Friday 15 April 2011 02:13:51 christian wrote:
Hello,
i'm not very experienced in python. Is there a way doing
below more memory efficient and maybe faster.
I import a 2-column file and then concat for every unique
value in the first column ( key) the value from the second
columns.
So The ouptut is something like that.
A,1,2,3
B,3,4
C,9,10,11,12,90,34,322,21
Thanks for advance regards,
Christian
import csv
import random
import sys
from itertools import groupby
from operator import itemgetter
f=csv.reader(open(sys.argv[1]),delimiter=';')
z=[[i[0],i[1]] for i in f]
z.sort(key=itemgetter(0))
mydict = dict((k,','.join(map(itemgetter(1), it)))
for k, it in groupby(z, itemgetter(0)))
del(z)
f = open(sys.argv[2], 'w')
for k,v in mydict.iteritems():
f.write(v + \n)
f.close()
Two alternative solutions - the second one with generators is
probably the most economical as far as RAM usage is concerned.
For you example data1.txt is taken as follows:
A, 1
B, 3
C, 9
A, 2
B, 4
C, 10
A, 3
C, 11
C, 12
C, 90
C, 34
C, 322
C, 21
The two in one program is:
#!/usr/bin python
'''generate.py - Example of reading long two column csv list and
sorting. Thread memory usage multi value hash
'''
# Determine a set of unique column 1 values
unique_set = set()
with open('data1.txt') as f:
for line in f:
unique_set.add(line.split(',')[0])
print(unique_set)
with open('data1.txt') as f:
for x in unique_set:
ls = [line.split(',')[1].rstrip() for line in f if
line.split(',')[0].rstrip() == x]
print(x.rstrip(), ','.join(ls))
f.seek(0)
print ('\n Alternative solution with generators')
with open('data1.txt') as f:
for x in unique_set:
gs = (line.split(',')[1].rstrip() for line in f if
line.split(',')[0].rstrip() == x)
s = ''
for ds in gs:
s = s + ds
print(x.rstrip(), s)
f.seek(0)
The output is:
{'A', 'C', 'B'}
A 1, 2, 3
C 9, 10, 11, 12, 90, 34, 322, 21
B 3, 4
Alternative solution with generators
A 1 2 3
C 9 10 11 12 90 34 322 21
B 3 4
Notice that data sequence could be different, without any effect
on output.
OldAl.
--
Algis
http://akabaila.pcug.org.au/StructuralAnalysis.pdf
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
Paul Rubin wrote:
Chris Angelico ros...@gmail.com writes:
sentinel = object()
seq = (dct.get('Keyword%d'%i,sentinel) for i in count(1))
lst = list(takewhile(lambda x: x != sentinel, seq))
If I understand this code correctly, that's creating generators,
right? It won't evaluate past the sentinel at all?
Right, it should stop after hitting the sentinel once.
That might well be what I'm looking for. A bit ugly, but efficient and
compact. And I can bury some of the ugliness away.
It occurs to me, operator.ne might be a little faster than the
interpreted lambda.
Or operator.is_not as you are dealing with a singleton. You also need
functools.partial:
$ python -m timeit -s'sentinel = object(); predicate = lambda x: x !=
sentinel' 'predicate(None)'
100 loops, best of 3: 0.369 usec per loop
$ python -m timeit -s'sentinel = object(); predicate = lambda x: x is not
sentinel' 'predicate(None)'
100 loops, best of 3: 0.314 usec per loop
$ python -m timeit -s'from functools import partial; from operator import
ne; sentinel = object(); predicate = partial(ne, sentinel)'
'predicate(None)'
100 loops, best of 3: 0.298 usec per loop
$ python -m timeit -s'from functools import partial; from operator import
is_not; sentinel = object(); predicate = partial(is_not, sentinel)'
'predicate(None)'
100 loops, best of 3: 0.252 usec per loop
--
http://mail.python.org/mailman/listinfo/python-list
Re: memory usage multi value hash
Terry Reedy wrote: On 4/14/2011 12:55 PM, Peter Otten wrote: I don't expect that it matters much, but you don't need to sort your data if you use a dictionary anyway: Which means that one can build the dict line by line, as each is read, instead of reading the entire file into memory. So it does matter for intermediate memory use. Yes, sorry, that was a bit too much handwaving. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
Chris Angelico wrote:
Apologies for interrupting the vital off-topic discussion, but I have
a real Python question to ask.
I'm doing something that needs to scan a dictionary for elements that
have a particular beginning and a numeric tail, and turn them into a
single list with some processing. I have a function parse_kwdlist()
which takes a string (the dictionary's value) and returns the content
I want out of it, so I'm wondering what the most efficient and
Pythonic way to do this is.
My first draft looks something like this. The input dictionary is
called dct, the output list is lst.
lst=[]
for i in xrange(1,1000): # arbitrary top, don't like this
try:
lst.append(parse_kwdlist(dct[Keyword%d%i]))
except KeyError:
break
I'm wondering two things. One, is there a way to make an xrange object
and leave the top off? (Sounds like I'm risking the numbers
evaporating or something.) And two, can the entire thing be turned
into a list comprehension or something? Generally any construct with a
for loop that appends to a list is begging to become a list comp, but
I can't see how to do that when the input comes from a dictionary.
In the words of Adam Savage: Am I about to feel really, really stupid?
Thanks in advance for help... even if it is just hey you idiot, you
forgot about X!
The initial data structure seems less than ideal. You might be able to
replace it with a dictionary like
{Keyword: [value_for_keyword_1, value_for_keyword_2, ...]}
if you try hard enough.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
On Fri, Apr 15, 2011 at 6:25 PM, Peter Otten __pete...@web.de wrote:
The initial data structure seems less than ideal. You might be able to
replace it with a dictionary like
{Keyword: [value_for_keyword_1, value_for_keyword_2, ...]}
if you try hard enough.
The initial data structure comes from a CSV file, and is not under my control.
ChrisA
--
http://mail.python.org/mailman/listinfo/python-list
PYTHONPATH
Hi, An elementary question that is bugging me, regarding sys.path values.sys.path can be altered easily, but the changes last for the current session only. I would like the changes to stay for several sessions. Is PYTHONPATH a system variable that sets the path for several sessions and if so, where in the system is it? Do I need to create one for setting python path for several sessions? Your answers will be greatly appreciated! TIA, OldAl. -- Algis http://akabaila.pcug.org.au/StructuralAnalysis.pdf -- http://mail.python.org/mailman/listinfo/python-list
Re: TextWrangler run command not working properly
In article 382709dd-5e3f-4b07-a642-4ce141ef4...@18g2000prd.googlegroups.com, Jon Clements jon...@googlemail.com wrote: http://www.velocityreviews.com/forums/t570137-textwrangler-and-new-python-vers ion-mac.html Thank you for the reply Jon. I saw the post in velocityreviews. Unfortunately it doesn't solve my problem. Cheers -- http://mail.python.org/mailman/listinfo/python-list
Can you advice a Python library to query a lan subnet with SNMP and collect MAC addresses of nodes?
Hello All, in my specific problem I will be happy of a response where possible to: 1. distinguish different operating systems of answering nodes 2. collect responses of Wyse thin-clients with Thin OS to get node name and MAC address in particular Thanks a lot in advance for any sharing / forward to documentation, products in the area. KR Aldo -- http://mail.python.org/mailman/listinfo/python-list
Re: PYTHONPATH
On Fri, Apr 15, 2011 at 1:33 AM, Algis Kabaila akaba...@pcug.org.au wrote: Hi, An elementary question that is bugging me, regarding sys.path values.sys.path can be altered easily, but the changes last for the current session only. I would like the changes to stay for several sessions. Is PYTHONPATH a system variable that sets the path for several sessions and if so, where in the system is it? It is an environment variable: http://en.wikipedia.org/wiki/Environment_variable Alternatively, you can use a .pth file to add directories to the module search path: http://docs.python.org/library/site.html Cheers, Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
On Thu, Apr 14, 2011 at 1:46 PM, Westley Martínez aniko...@gmail.com wrote: On Thu, 2011-04-14 at 14:02 +, Steven D'Aprano wrote: On Thu, 14 Apr 2011 19:15:05 +1000, Chris Angelico wrote: 4) Assumes people aren't deliberately fiddling the figures. Yeah, that would be correct. We're in the realm of conspiracy theories here... does anyone seriously think that browser stats are THAT important that they'd go to multiple web servers with deceitful hits? Back in the day, not that many years ago, when it looked like Internet Explorer would never dip below 90% market share and web developers coded for IE quirks instead of standards as a matter of course, I used to fantasize of writing a Windows virus that (apart from propagating) did nothing but change the user-agent string on IE. It would have been awesome to witness the consternation among web developers. But thanks to the EU doing what the US DOJ refused to do, and the grass- roots popularity of Firefox (plus a fewer well-known even if not often used browsers like Safari and Opera), and then Google's scarily efficient way they can capture hearts and minds on the Internet, IE's market share has been whittled away to the point that there are places in the world where IE is a minority browser. A large minority, it is true, but still a minority. Now, if only we could convince web users that having your browser execute untrusted code downloaded from the Internet is not such a good idea, supposed sandbox or not. What the world needs is a virus that silently removes Javascript and Flash from browsers... -- Steven Web developers will always use the tool they find to be the most reliable, efficient, and useful, as will consumers. You're kidding. Web developers will usually use what they believe will reach users, without excessive pain or embarrassment - and sometimes with. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
Chris Angelico wrote:
On Fri, Apr 15, 2011 at 6:25 PM, Peter Otten __pete...@web.de wrote:
The initial data structure seems less than ideal. You might be able to
replace it with a dictionary like
{Keyword: [value_for_keyword_1, value_for_keyword_2, ...]}
if you try hard enough.
The initial data structure comes from a CSV file, and is not under my
control.
ChrisA
Here's some code that might give you an idea. You can ignore the chunk
before 'import csv'; it is there to make the demo self-contained.
from contextlib import contextmanager
@contextmanager
def open(filename):
assert filename == example.csv
from StringIO import StringIO
yield StringIO(\
beta3,alpha1,alpha2,beta1,beta2
b31,a11,a21,b11,b21
b32,a12,a22,b12,b22
b33,a13,a23,b13,b23
b34,a14,a24,b14,b24
)
import csv
import re
def parse_name(s):
name, index = re.match(r(.+?)(\d+)$, s).groups()
return name, int(index)-1
with open(example.csv) as instream:
rows = csv.reader(instream)
header = next(rows)
dct = {}
appends = []
for h in header:
name, index = parse_name(h)
outer = dct.setdefault(name, {})
inner = outer.setdefault(index, [])
appends.append(inner.append)
for row in rows:
for value, append in zip(row, appends):
append(value)
print dct
--
http://mail.python.org/mailman/listinfo/python-list
Re: PYTHONPATH
En Fri, 15 Apr 2011 05:33:18 -0300, Algis Kabaila akaba...@pcug.org.au escribió: An elementary question that is bugging me, regarding sys.path values.sys.path can be altered easily, but the changes last for the current session only. I would like the changes to stay for several sessions. Is PYTHONPATH a system variable that sets the path for several sessions and if so, where in the system is it? Do I need to create one for setting python path for several sessions? PYTHONPATH is an environment variable, you set it the same way as any other, the details depend on the operating system/shell you're currently using. But - why do you think you need to set PYTHONPATH? Don't do that. Use the standard places to put your library modules and packages, like site-packages (where third-party libraries are installed by default). From Python 2.6+ the search path includes per-user directories like ~/.local/lib/python2.6/site-packages and %APPDATA%\Python\Python26\site-packages (see PEP370 [1] for details) so you don't even have to mess with the Python installation directories. [1] http://www.python.org/dev/peps/pep-0370/ -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Can you advice a Python library to query a lan subnet with SNMP and collect MAC addresses of nodes?
On 04/15/2011 05:00 PM, Aldo Ceccarelli wrote: Hello All, in my specific problem I will be happy of a response where possible to: 1. distinguish different operating systems of answering nodes 2. collect responses of Wyse thin-clients with Thin OS to get node name and MAC address in particular Thanks a lot in advance for any sharing / forward to documentation, products in the area. KR Aldo I think for your interest, if what you described is not a part of your software you are doing but only a specific task, you could use some network scanning tools like nmap to achieve your goals. there is also a module called pysnmp and you can look into it to see if it meets your need. frank -- http://mail.python.org/mailman/listinfo/python-list
Re: PYTHONPATH
On Friday 15 April 2011 19:21:12 Chris Rebert wrote: On Fri, Apr 15, 2011 at 1:33 AM, Algis Kabaila akaba...@pcug.org.au wrote: Hi, snip.. It is an environment variable: http://en.wikipedia.org/wiki/Environment_variable Alternatively, you can use a .pth file to add directories to the module search path: http://docs.python.org/library/site.html Cheers, Chris From Gabriel Genellina: akaba...@pcug.org.au escribió: An elementary question that is bugging me, regarding PYTHONPATH is an environment variable, you set it the same way as any other, the details depend on the operating system/shell you're currently using. But - why do you think you need to set PYTHONPATH? Don't do that. Use the standard places to put your library modules and packages, like site-packages (where third-party libraries are installed by default). From Python 2.6+ the search path includes per-user directories like ~/.local/lib/python2.6/site-packages and %APPDATA%\Python\Python26\site-packages (see PEP370 [1] for details) so you don't even have to mess with the Python installation directories. [1] http://www.python.org/dev/peps/pep-0370/ Thank you Gabriel and Thank you Chris, for your valuable advice and equally valuable set of references. Greatly appreciated! To answer the question of why consider a local sandbox to test various versions of a set of programs downloaded from the net. It is very handy to be able to change the standard behaviour to a localised users standard. Even more important, of course, is to know what standards there are. Thank you for your prompt and valuable assistance, OldAl. -- Algis http://akabaila.pcug.org.au/StructuralAnalysis.pdf -- http://mail.python.org/mailman/listinfo/python-list
Re: Can you advice a Python library to query a lan subnet with SNMP and collect MAC addresses of nodes?
On 15 Apr, 11:54, frankcui frankcu...@gmail.com wrote: On 04/15/2011 05:00 PM, Aldo Ceccarelli wrote: Hello All, in my specific problem I will be happy of a response where possible to: 1. distinguish different operating systems of answering nodes 2. collect responses of Wyse thin-clients with Thin OS to get node name and MAC address in particular Thanks a lot in advance for any sharing / forward to documentation, products in the area. KR Aldo I think for your interest, if what you described is not a part of your software you are doing but only a specific task, you could use some network scanning tools like nmap to achieve your goals. there is also a module called pysnmp and you can look into it to see if it meets your need. frank Thanks Frank! I've browsed pysnmp as you kindly adviced, now looking also into http://pynetsnmp.sourceforge.net/ KR Aldo -- http://mail.python.org/mailman/listinfo/python-list
http://DuplicateFilesDeleter.com - This software deletes duplicate files in media collection of any type
http://DuplicateFilesDeleter.com - find duplicates http://DuplicateFilesDeleter.com is an innovative tool that can recognize duplicate audio files even if they are stored in different file formats and not marked with ID3 tags. It will find fast all similar or exact duplicate audio files in a folder and its sub folders. Unlike common duplicate file finders it will actually listen to your music and can recognize a song even if it is saved in different file formats. Supports MP3, MP2, MP1, MPA, WAV, OGG, AIFF, AAC, MP4, FLAC, AC3, WavPack (WV), Musepack (MPC) and Windows Media Audio (WMA) file formats, has an intuitive user interface and is well documented. http://DuplicateFilesDeleter.com - find duplicates -- http://mail.python.org/mailman/listinfo/python-list
Re: TextWrangler run command not working properly
Hello Fabio You have two versions of 2.6 on your system. On Apr 15, 2011, at 4:51 AM, Fabio wrote: I have the built-in Python2.5 which comes installed by mother Apple. My OSX comes with 2.3, 2.5, and 2.6. :) These are under: /System/Library/Frameworks/Python.framework/Versions/ ^ the ones you installed are under: /Library/Frameworks/Python.framework/Versions/ I can reproduce this problem on my system, because /usr/bin/python2.6 points to the system version. There is an easy solution: #!/usr/bin/env python will work, or, #!/usr/local/bin/python it's better to use the former, as it will work even as you change versions, etc... You should avoid using the shebang with a *specific* python version. just use #!/usr/bin/env python bb -- Brian Blais bbl...@bryant.edu http://web.bryant.edu/~bblais http://bblais.blogspot.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
On Fri, 15 Apr 2011 18:32:23 +1000, Chris Angelico wrote:
On Fri, Apr 15, 2011 at 6:25 PM, Peter Otten __pete...@web.de wrote:
The initial data structure seems less than ideal. You might be able to
replace it with a dictionary like
{Keyword: [value_for_keyword_1, value_for_keyword_2, ...]}
if you try hard enough.
The initial data structure comes from a CSV file, and is not under my
control.
There's no reason to duplicate the CSV file's design in your own data
structures though.
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
On Fri, 15 Apr 2011 13:58:22 +1000, Chris Angelico wrote: The dictionary is potentially a lot larger than this particular set of values (it's a mapping of header:value for a row of a user-provided CSV file). Does this make a difference to the best option? (Currently I'm looking at likely figures of 60+ keys in the dictionary and 3-8 postage options to pick up, but both of those could increase significantly before production.) SIXTY keys? When you get to sixty thousand keys, it might take a few seconds to process. Efficiency is important, though not king; this whole code will be inside a loop. But readability is important too. I can't just give all of dct.values() to parse_kwdlist; the function parses a particular format of text string, and it's entirely possible that other values would match that format (some of them are pure free-form text). This has to get only the ones starting with Keyword, and in order. Steven, the line you suggested: lst = [parse_kwdlist(dct[Keyword%d%i]) for i in xrange(1, len(dct)+1)] will bomb with KeyError when it hits the first one that isn't present, I assume. Is there an easy way to say and when you reach any exception, not just StopIteration, return the list? (I could wrap the whole lst = in a try/except, but then it won't set lst at all.) No. You could use the list comprehension form at the cost of running over the dict twice: maxkey = 0 for key in dct: if key.startswith(Keyword): maxkey = max(maxkey, int(key[7:])) lst = [parse_kwdlist(dct[Keyword%d%i]) for i in xrange(1, maxkey+1)] but quite frankly, at this point I'd say, forget the one-liner, do it the old-fashioned way with a loop. Or change your data structure: often you can simplify a task drastically just by changing the way you store the data. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
In article 4da83f8f$0$29986$c3e8da3$54964...@news.astraweb.com, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: for key in dct: if key.startswith(Keyword): maxkey = max(maxkey, int(key[7:])) I would make that a little easier to read, and less prone to Did I count correctly? bugs with something like: prefix = Keyword n = len(prefix) for key in dct: name, value = key[:n], key[n:] if name == prefix: maxkey = max(maxkey, int(value)) -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
Chris Angelico wrote:
lst=[]
for i in xrange(1,1000): # arbitrary top, don't like this
try:
lst.append(parse_kwdlist(dct[Keyword%d%i]))
except KeyError:
break
Possibly overkill:
import dbf
table = dbf.from_csv(csvfile) # fields get names f0, f1, f2, ...
table.rename_field('f3', 'key') # or whatever
def keyword_only(record):
if record.key.startswith('keyword'):
return int(record.key[len('keyword'):]))
return dbf.DoNotIndex
keywords = table.create_index(keyword_only)
# keywords is usable as an iterator
for rec in keywords:
# if you only want the first contiguous records
for i, rec in enum(keywords):
if rec.key != 'keyword%d' % enum:
break
...
Disclosure: I am the author of the dbf module.
Depending on your needs for the other fields of the csv file, this might
be an easier way to access them. Field names can also be specified when
opening the csv file, I didn't bother for this example since I don't
know what all your field names are. ;)
Hope this helps!
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
Re: Can you advice a Python library to query a lan subnet with SNMP and collect MAC addresses of nodes?
On Fri, Apr 15, 2011 at 5:00 AM, Aldo Ceccarelli ceccarelli.a...@gmail.comwrote: Hello All, in my specific problem I will be happy of a response where possible to: 1. distinguish different operating systems of answering nodes 2. collect responses of Wyse thin-clients with Thin OS to get node name and MAC address in particular Thanks a lot in advance for any sharing / forward to documentation, products in the area. KR Aldo -- http://mail.python.org/mailman/listinfo/python-list Aldo If you haven't already, have a look at dpkt and scapy as well. Both good tools to have for tasks such as these. Regards Jack -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
On Fri, Apr 15, 2011 at 10:52 PM, Steven D'Aprano steve+comp.lang.pyt...@pearwood.info wrote: On Fri, 15 Apr 2011 13:58:22 +1000, Chris Angelico wrote: The dictionary is potentially a lot larger than this particular set of values (it's a mapping of header:value for a row of a user-provided CSV file). Does this make a difference to the best option? (Currently I'm looking at likely figures of 60+ keys in the dictionary and 3-8 postage options to pick up, but both of those could increase significantly before production.) SIXTY keys? When you get to sixty thousand keys, it might take a few seconds to process. This whole code is inside a loop that we took, in smoke testing, to a couple hundred million rows (I think), with the intention of having no limit at all. So this might only look at 60-100 headers, but it will be doing so in a tight loop. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Pythonic infinite for loop?
Chris Angelico ros...@gmail.com writes: This whole code is inside a loop that we took, in smoke testing, to a couple hundred million rows (I think), with the intention of having no limit at all. So this might only look at 60-100 headers, but it will be doing so in a tight loop. If you're talking about data sets that large, you should rethink the concept of using python dictionaries with a key for every row. Try importing your CSV into an SQL database and working from there instead. -- http://mail.python.org/mailman/listinfo/python-list
Questions about GIL and web services from a n00b
So I'm in a startup where we are considering using python as our primary development language for all the wonderful reasons you would expect. However, I've had a couple of things come up from mentors and other developers that is causing me to rethink whether python is the right choice. I hope this is the right list for this type of discussion (please forgive me if not and point me in the right direction). We are looking to build an e-commerce integration product so the majority of our work will be linking external SOAP and REST based web service API's to our own REST based API and backend database.I have had the following comments/questions come to me: 1. Are you sure you want to use python because threading is not good due to the Global Lock (GIL)? Is this really an issue for multi-threaded web services as seems to be indicated by the articles from a Google search? If not, how do you avoid this issue in a multi-threaded process to take advantage of all the CPU cores available? 2. Are there good web services frameworks available for building a REST based service? I admit I have looked at web2py, Django, pyramid/pylons, and a few others. SOAP seems to be pretty well supported but I'm not finding the same for quick development of REST based services for exchanging JSON or XML formatted data. This is probably just my n00b status, but what tools are best for building a simple REST data exchange API? Thanks, Chris -- http://mail.python.org/mailman/listinfo/python-list
genfromtxt and comment identifier
Hi All,
I have a problem with reading data from a file using genfromtxt of
numpy module.
I have prepared a minimal example similar to the ones presented in
http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html#splitting-the-lines-into-columns
The script is
import numpy as np
from StringIO import StringIO
file = open('esempio.dat')
for line in file.xreadlines() :
if not line : break
print line
np.genfromtxt(StringIO(line), comments=#, delimiter=,)
I have a data file - esempio.dat - like the following:
#
# Skip me !
# Skip me too !
1, 2
3, 4
5, 6 #This is the third line of the data
7, 8
# And here comes the last line
9, 0
The code is breaking at the first line, it looks like it doesn't
recognize # like comment identifier.
The error in the python interpreter is
Traceback (most recent call last):
File stdin, line 1, in module
File esempio.py, line 7, in module
np.genfromtxt(StringIO(line), comments=#, delimiter=,)
File /opt/numpy/1.5.1/lib/python2.6/site-packages/numpy/lib/
npyio.py, line 1174, in genfromtxt
raise IOError('End-of-file reached before encountering data.')
IOError: End-of-file reached before encountering data.
It is clear that I haven't understood something, what am I doing
wrong?
Cheers
--
http://mail.python.org/mailman/listinfo/python-list
Re: Questions about GIL and web services from a n00b
On 4/15/11 1:03 PM, Tim Wintle wrote: On Fri, 2011-04-15 at 12:33 -0400, Chris H wrote: 1. Are you sure you want to use python because threading is not good due to the Global Lock (GIL)? Is this really an issue for multi-threaded web services as seems to be indicated by the articles from a Google search? If not, how do you avoid this issue in a multi-threaded process to take advantage of all the CPU cores available? Is the limiting factor CPU? If it isn't (i.e. you're blocking on IO to/from a web service) then the GIL won't get in your way. If it is, then run as many parallel *processes* as you have cores/CPUs (assuming you're designing an application that can have multiple instances running in parallel so that you can run over multiple servers anyway). Tim Wintle Great question. At this point, there isn't a limiting factor, but yes the concern is around CPU in the future with lots of threads handling many simultaneous transactions. Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Questions about GIL and web services from a n00b
On Fri, 2011-04-15 at 12:33 -0400, Chris H wrote: 1. Are you sure you want to use python because threading is not good due to the Global Lock (GIL)? Is this really an issue for multi-threaded web services as seems to be indicated by the articles from a Google search? If not, how do you avoid this issue in a multi-threaded process to take advantage of all the CPU cores available? Is the limiting factor CPU? If it isn't (i.e. you're blocking on IO to/from a web service) then the GIL won't get in your way. If it is, then run as many parallel *processes* as you have cores/CPUs (assuming you're designing an application that can have multiple instances running in parallel so that you can run over multiple servers anyway). Tim Wintle -- http://mail.python.org/mailman/listinfo/python-list
Re: genfromtxt and comment identifier
simona bellavista wrote:
Hi All,
I have a problem with reading data from a file using genfromtxt of
numpy module.
I have prepared a minimal example similar to the ones presented in
http://docs.scipy.org/doc/numpy/user/basics.io.genfromtxt.html#splitting-
the-lines-into-columns
The script is
import numpy as np
from StringIO import StringIO
file = open('esempio.dat')
for line in file.xreadlines() :
if not line : break
print line
np.genfromtxt(StringIO(line), comments=#, delimiter=,)
I have a data file - esempio.dat - like the following:
#
# Skip me !
# Skip me too !
1, 2
3, 4
5, 6 #This is the third line of the data
7, 8
# And here comes the last line
9, 0
The code is breaking at the first line, it looks like it doesn't
recognize # like comment identifier.
The error in the python interpreter is
Traceback (most recent call last):
File stdin, line 1, in module
File esempio.py, line 7, in module
np.genfromtxt(StringIO(line), comments=#, delimiter=,)
File /opt/numpy/1.5.1/lib/python2.6/site-packages/numpy/lib/
npyio.py, line 1174, in genfromtxt
raise IOError('End-of-file reached before encountering data.')
IOError: End-of-file reached before encountering data.
It is clear that I haven't understood something, what am I doing
wrong?
The examples use StringIO to simulate a file, but you are wrapping every
line of your actual file. The first simulated file is then
StringIO(#\n)
i. e. it contains only a comment, no data -- and that's what genfromtxt()
complains about. Read the file in one fell swoop and you should be OK:
import numpy as np
with open('esempio.dat') as instream:
print np.genfromtxt(instream, comments=#, delimiter=,)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Questions about GIL and web services from a n00b
On Fri, Apr 15, 2011 at 9:33 AM, Chris H chris.humph...@windsorcircle.com wrote: 1. Are you sure you want to use python because threading is not good due to the Global Lock (GIL)? Is this really an issue for multi-threaded web services as seems to be indicated by the articles from a Google search? If not, how do you avoid this issue in a multi-threaded process to take advantage of all the CPU cores available? Concurrency in Python is a largish topic. It's true that CPython's multithreading is poor. In fact, running a multithreaded CPython application on n vs 2n cores can actually take more time on the 2n cores. However: 1) In Jython, and likely IronPython, threading is good. 2) In CPython, there's a module called multiprocessing that's a little slower than a good threading implementation, but gives looser coupling between the discrete components of your software. Programming with multiprocessing feels similar to programming with threads - you have safe queues, safe scalars or simple arrays in shared memory, etc. 3) There's something called stackless and (similar to stackless) greenlets. While stackless allows you to use thousands of threads comfortably, it's still pretty single-core. It's essentially a fork of CPython, and is being made a part of PyPy. I believe greenlets are an attempt to bring what's good about stackless to CPython, in the form of a C extension module. 4) I've heard that in CPython 3.2, the GIL is less troublesome, though I've not yet heard in what way. 5) Even in CPython, I/O-bound processes are not slowed significantly by the GIL. It's really CPU-bound processes that are. 6) PyPy is quite a bit faster than CPython for non-concurrent applications, but also has a GIL. 7) Cython, which compiles a dialect of Python into faster C (especially if you give it a few type declarations), has GIL-implications. I've heard that Cython can selectively release the GIL on request, but I've also heard that C extension modules always release the GIL. This seems contradictory, and merits further investigation. It's important to remember: Python != CPython. CPython remains the reference implementation, and runs the most Python code, but there are other, significant implementations now. This page looks like a summary of many (more) options: http://wiki.python.org/moin/Concurrency/ 2. Are there good web services frameworks available for building a REST based service? I admit I have looked at web2py, Django, pyramid/pylons, and a few others. SOAP seems to be pretty well supported but I'm not finding the same for quick development of REST based services for exchanging JSON or XML formatted data. This is probably just my n00b status, but what tools are best for building a simple REST data exchange API? I have no experience with REST, but Grig of SoCal Piggies did a presentation on restish a while back. You might see if you can find something about that. -- http://mail.python.org/mailman/listinfo/python-list
Re: [Mac OSX] TextWrangler run command not working properly
On Thu, Apr 14, 2011 at 1:52 PM, Fabio oakw...@email.it wrote: Then, I started to use TexWrangler, and I wanted to use the shebang menu, and run command. I have the #! first line pointing to the 2.6 version. It works fine, as long as I don't import the libraries, in which case it casts an error saying: ImportError: No module named scipy Maybe for some reason it points to the old 2.5 version. But I might be wrong and the problem is another... TextWrangler doesn't launch a shell session that sources your typical resource files (i.e. .bashrc, etc.), so any changes you make in an interactive terminal session probably WON'T be loaded in TextWrangler. See this website about setting environment variables for native Mac OS X applications to see them: http://www.astro.washington.edu/users/rowen/AquaEnvVar.html Maybe if you prepend your Python 2.6 (MacPorts?) location to your PATH, it'll find it. --Jason -- http://mail.python.org/mailman/listinfo/python-list
Can Python control Thuderbird?
I want to control Mozilla Thunderbird using Python. Does anyone know if that is that possible? I would like to use Python to save email attachments to a specific directory, depending on the name of the sender, content in the email, etc.--- and to rename the attachment file -- and to save the email to an html file -- and to insert into the email file links to the locally saved copies of the attachments. Is this possible? If the answer is yes, where could I find information about how to use Python to control Thuderbird (and/or examples of saving attachments to locval filenames,and/or saving emails to htmlfiles and/or connecting to the list of contacts) ? While hunting around, I came across Python 18.4. mailbox — Manipulate mailboxes in various formats — Python v2.7.1 documentation. I'm so totally ignorant of MAPI that I don't know if that is what I'm looking for. If it is, does anyone know where I could find some examples of python scripts using the Python 18.4. mailbox module? Thanks, Marceepoo Thanks for your tme and the knowledge you share, Marc -- http://mail.python.org/mailman/listinfo/python-list
Is it possible to execute Python code from C++ without writing to a file?
I'm a Python newbie who's been given a task requiring calls of Python code from a C++ program. I've tried various tutorials and dug into The Python/C API doc and the Extending and Embedding Python doc, but I haven't been able to answer this question: Is it possible in a C++ program to generate Python code, execute it, and get output back (say a tuple of 10 or so items) without doing any file i/o? Clearly it works to write the generated Python to a file and then use PyImport_Import and PyObject_CallObject to call a function returning the output tuple. But it seems like it should be possible to do this without writing the Python code to a file. I tried PyRun_String, but I can't see how it can be used to return a tuple (the Py_file_input option always returns None). Any help will be greatly appreciated. Roger House Software Developer -- http://mail.python.org/mailman/listinfo/python-list
Re: Is it possible to execute Python code from C++ without writing to a file?
On Sat, Apr 16, 2011 at 9:46 AM, Roger House rho...@sonic.net wrote:
I'm a Python newbie who's been given a task requiring calls of Python code
from a C++ program. I've tried various tutorials and dug into The Python/C
API doc and the Extending and Embedding Python doc, but I haven't been able
to answer this question:
Is it possible in a C++ program to generate Python code, execute
it, and get output back (say a tuple of 10 or so items) without
doing any file i/o?
Clearly it works to write the generated Python to a file and then use
PyImport_Import and PyObject_CallObject to call a function returning the
output tuple. But it seems like it should be possible to do this without
writing the Python code to a file. I tried PyRun_String, but I can't see
how it
can be used to return a tuple (the Py_file_input option always returns
None).
What I do for this is have the Python code place its return value into
a particular location. If you're using file input, I think it's
restricted to returning an integer; but you can do something like
this:
const char *python_code=result=('hello','world',42);
PyObject *globals=... (use same as locals if you don't need globals)
PyObject *locals=PyDict_New();
Py_XDECREF(PyRun_StringFlags(python_code,Py_file_input,globals,locals,0);
PyObject *returned_tuple=PyDict_GetItemString(locals,result);
Py_DECREF(locals);
You now own a reference to whatever the Python code put into its
variable result, in the C variable returned_tuple. Of course, it
might not be a tuple at all, and it might not even be present (in
which case returned_tuple will be NULL). This is a fairly effective
way for Python to return hefty amounts of data to C.
Don't forget to decref the return value.
Hope that helps!
Chris Angelico
--
http://mail.python.org/mailman/listinfo/python-list
Re: Questions about GIL and web services from a n00b
Is the limiting factor CPU? If it isn't (i.e. you're blocking on IO to/from a web service) then the GIL won't get in your way. If it is, then run as many parallel *processes* as you have cores/CPUs (assuming you're designing an application that can have multiple instances running in parallel so that you can run over multiple servers anyway). Great question. At this point, there isn't a limiting factor, but yes the concern is around CPU in the future with lots of threads handling many simultaneous transactions. In the Python world, the usual solution to high transaction loads is to use event-driven processing (using an async library such as Twisted) rather than using multi-threading which doesn't scale well in any language. Also, the usual way to take advantage of multiple-cores is to run multiple pythons in separate processes. Threading is really only an answer if you need to share data between threads, if you only have limited scaling needs, and are I/O bound rather than CPU bound Raymond -- http://mail.python.org/mailman/listinfo/python-list
Re: Questions about GIL and web services from a n00b
On Sat, Apr 16, 2011 at 10:05 AM, Raymond Hettinger pyt...@rcn.com wrote: Is the limiting factor CPU? If it isn't (i.e. you're blocking on IO to/from a web service) then the GIL won't get in your way. If it is, then run as many parallel *processes* as you have cores/CPUs (assuming you're designing an application that can have multiple instances running in parallel so that you can run over multiple servers anyway). Great question. At this point, there isn't a limiting factor, but yes the concern is around CPU in the future with lots of threads handling many simultaneous transactions. In the Python world, the usual solution to high transaction loads is to use event-driven processing (using an async library such as Twisted) rather than using multi-threading which doesn't scale well in any language. My experience is that if you are CPU bound, asynchronous programming in python can be more a curse than a blessing, mostly because the need to insert scheduling points at the right points to avoid blocking and because profiling becomes that much harder in something like twisted. It depends of course of the application, but designing from the ground up with the idea of running multiple processes is what seems to be the most natural way of scaling - this does not prevent using async in each process. This has its own issues, though (e.g. in terms of administration and monitoring). Chris, the tornado documention mentions a simple way to get multiple processes on one box: http://www.tornadoweb.org/documentation (section mentiong nginx for load balancing). The principle is quite common and is applicable to most frameworks (the solution is not specific to tornado). cheers. David -- http://mail.python.org/mailman/listinfo/python-list
Python IDE/text-editor
Good Afternoon, I'm looking for an IDE which offers syntax-highlighting, code-completion, tabs, an embedded interpreter and which is portable (for running from USB on Windows). Here's a mockup of the app I'm looking for: http://i52.tinypic.com/2uojswz.png Which would you recommend? Thanks in advance for any suggestions, Alec Taylor -- http://mail.python.org/mailman/listinfo/python-list
Re: Python IDE/text-editor
Alec Taylor alec.tayl...@gmail.com writes: I'm looking for an IDE which offers syntax-highlighting, code-completion, tabs, an embedded interpreter and which is portable (for running from USB on Windows). Either of Emacs URL:http://www.gnu.org/software/emacs/ or Vim URL:http://www.vim.org/ are excellent general-purpose editors that have strong features for programmers of any popular language or text format. They are available for every major OS, including MS Windows, are mature and have enormous community support, and are free software and community developed. Learn either one, and you won't have to keep switching to different editing tools for different programming tasks. -- \ “Geeks like to think that they can ignore politics. You can | `\leave politics alone, but politics won't leave you alone.” | _o__)—Richard Stallman, 2002-07-26 | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Python IDE/text-editor
On Apr 16, 8:20 am, Alec Taylor alec.tayl...@gmail.com wrote: Good Afternoon, I'm looking for an IDE which offers syntax-highlighting, code-completion, tabs, an embedded interpreter and which is portable (for running from USB on Windows). Here's a mockup of the app I'm looking for:http://i52.tinypic.com/2uojswz.png As Ben suggested you can use emacs (Dunno if vi will give an embedded interpreter) but be prepared to have to work a bit -- it does not just work out of the box. In particular emacs will default to using a different python mode (python.el) than the one that python programmers seem prefer https://launchpad.net/python-mode/ You may also want to look at ipython BTW how did you make that mockup? -- http://mail.python.org/mailman/listinfo/python-list
Re: PYTHONPATH
Algis Kabaila wrote: Is PYTHONPATH a system variable that sets the path for several sessions and if so, where in the system is it? Do I need to create one for setting python path for several sessions? It can be, and there are lots of ways to accomplish what you want, some of which depends on the platform you are using. I will show one of the ways that I accomplish this for my linux sessions. This is based on a very common snippet of code usually found in the users .profile which modifies the users path in the even the user has a ~/bin directory--- looks like this: # set PATH so it includes user's private bin if it exists if [ -d $HOME/bin ] ; then PATH=$HOME/bin:$PATH fi When this snippet finds a ~/bin directory in the users ~/ then (and only then) it modifies the users bash session path with the ~/bin folder at the head of the path. Well, you can use this same snippet on your system to modify the PYTHONPATH system variable so that a special folder in your ~/ directory tree is at or near the head of the sys.path--- looks like this: # set PATH so it includes user's private Python if it exists if [ -d $HOME/Python ] ; then export PYTHONPATH=$HOME/Python:$PYTHONPATH fi You will notice that there is a tiny baby change in the second snippet... the export. If you want you IDLE launches from the Desktop to see the path set in .profile provide that with the export. Of course this only works if the Python folder exists in the users ~/ directory tree (or elsewhere, if you code it that way). By default the sys.path always shows the directory python was opened in, usually the users home directory. With .profile you can set the path any way you want... most useful for setting up special test directories ahead of the real code, or for setting up separate directories for versions--- one for Python26, Python27, and of course Python32. (there are other ways of accomplishing the same thing, and of course, this one only really works with *nix systems--- windows is another mess entirely) kind regards, m harris -- http://mail.python.org/mailman/listinfo/python-list
Re: Egos, heartlessness, and limitations
And who pissed in Guido's punch bowl anyway? Why is he such an elitist now? Why can he not come over once and a while and rub shoulders with http://www.youtube.com/watch?v=FMEe7JqBgvg -- http://mail.python.org/mailman/listinfo/python-list
Re: Free software versus software idea patents
CM wrote: What was criticized was your approach, which seemed counter-productive, and so much so that it seemed like you are really advocating FOR software patents by discrediting the position against them. Oh, the thou protesteth too much argument... ... well, I can only say that none of us can afford to protest too little about this issue. Too many have their heads in the sand and time is running out. If we're ever going to see this changed in our life-times then a whole lot of somebodies better get on the stick... -- http://mail.python.org/mailman/listinfo/python-list
Re: Python IDE/text-editor
On Apr 15, 11:20 pm, Alec Taylor alec.tayl...@gmail.com wrote: Good Afternoon, I'm looking for an IDE which offers syntax-highlighting, code-completion, tabs, an embedded interpreter and which is portable (for running from USB on Windows). Here's a mockup of the app I'm looking for:http://i52.tinypic.com/2uojswz.png Which would you recommend? Thanks in advance for any suggestions, Alec Taylor You seem to have drawn PSPad (http://www.pspad.com/). With one preferences change and a little bit of window rearranging, you can make PSPad do what you want. -- http://mail.python.org/mailman/listinfo/python-list
Re: [OT] Free software versus software idea patents
geremy condra wrote: http://www.groklaw.net/articlebasic.php?story=2009151305785 This is not a proof. This is an argument. There's a very big difference. To be clear, this article makes basically the same mistake you do- you assume that a program is exactly equivalent to its computation, Yes, I agree there is a difference between argument and proof. I full well know that what you are requiring here is a formal rigorous mathematical proof. I'm not trying to be coy about that issue, nor the issue of undecidability; because at the moment it is not necessary for the purpose of isolating the problem underlying software idea patents. I'll explain that in a bit... in another post where it better fits... (hold that thought...) The main point here is that *all* that is necessary at the moment is to make the argument crystal clear, demonstrably (but not rigorous by mathematical standards), that software *is* mathematics. But, that Mathematics *is not* necessarily Software... something new to follow... Please consider these statements as pairs, then a small argument: Mathematics *is not* chess. Chess *is* mathematics. Mathematics *is not* Tic-Tac-Toe Tic-Tac-Toe *is* mathematics. Mathematics *is not* The Towers of Hanoi. The Towers of Hanoi *is* mathematics. Mathematics *is not* Fourier Analysis. Fourier Analysis *is* mathematics. Mathematics *is not* computation. Computation *is* mathematics. Mathematics *is not* software. Software *is* mathematics. I am not stating the problem as natural and obvious from the direction of Mathematics == Software. *NOT* That aspect would in fact require a formal rigorous proof (in my opinion). That aspect of the discussion must at this point remain stipulated as merely conjecture--- maybe with resounding circumstantial evidence and demonstration, but conjecture none the less. At least, I have never seen a formal proof. (This is the direction of my work and research, but more on that later...) I am asserting something of a subset in computational theory, even logic, and certainly number theory--- but there are others--- that the discussion at this point is from the other direction, namely, Software == Mathematics. This aspect of the discussion (and the only one I believe matters for software idea patents at the moment) does not *require* a formal rigorous mathematical proof (we are not trying to prove a theorem here, rather to only demonstrate that software is the same 'thought' and 'process' (certainly a subset) as natural and obvious mathematical 'thought' and 'process'. What? Namely--- input, control, arithmetic, logic, and output. These are subset to the broad and rich categories of mathematical thought and process. However, these small categories *are* definitive subsets of mathematics, obvious and natural, for all thinkers all over the globe. This is the reason I presume to apply for why you find software idea patents unfair and detestable. Because down underneath somewhere (like the rest of us), you too know that this aspect is natural and obvious. Its unfair (I really don't like that word) because it violates truth. Truth and freedom are very closely related topics. In this pursuit for freedom we are not violating (nor abusing) mathematics. (quite the opposite) The full rigor of mathematical thought and philosophical reasoning in mathematics are at the pinnacle of human endeavor and at the height of science. This I fully and truly respect. The issue is *not* to pull mathematics down somehow... the issue is to clearly define *what* software *is* and from where it ultimately derives (as a subset)--- what constitutes software? The constituency of software is nothing less than mathematics, from my viewpoint. In a different post (in response to another of your good questions) I will try to lay out a different argument that comes a little closer at what you are looking for in terms of proof... still not rigorous mathematical proof... but perhaps more demonstrable logically. Be patient, I'm trying to limit my posts to *not too many* whatever I mean by that. :) kind regards, m harris -- http://mail.python.org/mailman/listinfo/python-list
Re: Python IDE/text-editor
Ben Finney ben+pyt...@benfinney.id.au writes: Alec Taylor alec.tayl...@gmail.com writes: I'm looking for an IDE which offers syntax-highlighting, code-completion, tabs, an embedded interpreter and which is portable (for running from USB on Windows). Either of Emacs URL:http://www.gnu.org/software/emacs/ or Vim URL:http://www.vim.org/ are excellent general-purpose editors that have strong features for programmers of any popular language or text format. I second Emacs or vim. I currently use Emacs the most, but I think it's good to learn both. -- John Bokma j3b Blog: http://johnbokma.com/Facebook: http://www.facebook.com/j.j.j.bokma Freelance Perl Python Development: http://castleamber.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Python IDE/text-editor
Thanks, but non of the IDEs so far suggested have an embedded python interpreter AND tabs... a few of the editors (such as Editra) have really nice interfaces, however are missing the embedded interpreter... emacs having the opposite problem, missing tabs (also, selecting text with my mouse is something I do often). Please continue your recommendations. Thanks, Alec Taylor On Sat, Apr 16, 2011 at 3:29 PM, John Bokma j...@castleamber.com wrote: Ben Finney ben+pyt...@benfinney.id.au writes: Alec Taylor alec.tayl...@gmail.com writes: I'm looking for an IDE which offers syntax-highlighting, code-completion, tabs, an embedded interpreter and which is portable (for running from USB on Windows). Either of Emacs URL:http://www.gnu.org/software/emacs/ or Vim URL:http://www.vim.org/ are excellent general-purpose editors that have strong features for programmers of any popular language or text format. I second Emacs or vim. I currently use Emacs the most, but I think it's good to learn both. -- John Bokma j3b Blog: http://johnbokma.com/ Facebook: http://www.facebook.com/j.j.j.bokma Freelance Perl Python Development: http://castleamber.com/ -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
[issue11787] File handle leak in TarFile lib
Ezio Melotti ezio.melo...@gmail.com added the comment: Attached trivial patch to fix the issue. Needs tests. -- keywords: +easy, patch nosy: +ezio.melotti stage: - test needed Added file: http://bugs.python.org/file21668/issue11787.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11787 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5057] Unicode-width dependent optimization leads to non-portable pyc file
Marc-Andre Lemburg m...@egenix.com added the comment: Ezio Melotti wrote: Ezio Melotti ezio.melo...@gmail.com added the comment: PEP 3147 says[0]: For backward compatibility, Python will still support pyc-only distributions, however it will only do so when the pyc file lives in the directory where the py file would have been, i.e. not in the __pycache__ directory. pyc file outside of __pycache__ will only be imported if the py source file is missing. Does that mean that there could be cases where untagged pyc files are used in 3.2+? Yes... even though we did discuss using the same tagging support in that scenario as well, at least for 3.3. In that case the patch should be ported to 3.2 and 3.3 too. [0]: http://www.python.org/dev/peps/pep-3147/#rationale -- title: Unicode-width dependent optimization leads to non-portable pyc file - Unicode-width dependentoptimizationleads to non-portable pyc file ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5057 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11849] ElementTree memory leak
New submission from Kaifeng Zhu cafe...@gmail.com: I'm using xml.etree.ElementTree to parse large XML file, while the memory keep increasing consistently. You can run attached test script to reproduce it. From 'top' in Linux or 'Task Manager' in Windows, the memory usage of python is not decreased as expected when 'Done' is printed. Tested with Python 2.5/3.1 in Windows 7, and Python 2.5 in CentOS 5.3. -- components: XML files: test.py messages: 133797 nosy: Kaifeng.Zhu priority: normal severity: normal status: open title: ElementTree memory leak type: resource usage versions: Python 2.5, Python 3.1 Added file: http://bugs.python.org/file21669/test.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11849 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11652] urlib{, 2} returns a pair of integers as the content-length value
Senthil Kumaran sent...@uthcode.com added the comment:
It is better to close this issue as it was a Server Error. Standard just says
that when there two headers with different values, combine them comma separated
as urllib2 does. Making special case exception for 'Content-Length' header
when the server is at fault would be bad idea. We will not know which value to
choose from if the values are different.
Closing this bug as Invalid.
import urllib2
req =
urllib2.urlopen('http://wwwsearch.sourceforge.net/mechanize/src/mechanize-0.1.11.zip')
req.info()['content-length']
'289519'
--
resolution: - invalid
stage: - committed/rejected
status: open - closed
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11652
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11849] ElementTree memory leak
Florent Xicluna florent.xicl...@gmail.com added the comment: Do you experience same issue with current versions of Python? (3.2 or 2.7) The package was upgraded in latest versions. -- nosy: +flox ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11849 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11849] ElementTree memory leak
Kaifeng Zhu cafe...@gmail.com added the comment: Yes. Just tested with Python 2.7 and 3.2 in Windows 7, the memory usage is still unexpected high after 'Done' is printed. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11849 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11467] urlparse.urlsplit() regression for paths consisting of digits
Roundup Robot devnull@devnull added the comment: New changeset 7a693e283c68 by Senthil Kumaran in branch '2.7': Issue #11467: Fix urlparse behavior when handling urls which contains scheme http://hg.python.org/cpython/rev/7a693e283c68 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11467 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11467] urlparse.urlsplit() regression for paths consisting of digits
Roundup Robot devnull@devnull added the comment: New changeset 495d12196487 by Senthil Kumaran in branch '3.1': Issue #11467: Fix urlparse behavior when handling urls which contains scheme specific part only digits. http://hg.python.org/cpython/rev/495d12196487 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11467 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11467] urlparse.urlsplit() regression for paths consisting of digits
Senthil Kumaran sent...@uthcode.com added the comment: Fixed this in all codelines. Thanks Santoso. -- resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11467 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11819] '-m unittest' should not pretend it works on Python 2.5/2.6
anatoly techtonik techto...@gmail.com added the comment: #6514 is to make `-m unittest` work run on 2.5/2.6. This bug is not to fix it, but to stop displaying confusing messages. It will be enough to exit with a message like: `-m unittest` call is not supported in Python 2.5/2.6 - use something else (nose?) instead -- status: closed - open title: 'unittest -m' should not pretend it works on Python 2.5/2.6 - '-m unittest' should not pretend it works on Python 2.5/2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11819 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11819] '-m unittest' should not pretend it works on Python 2.5/2.6
Michael Foord mich...@voidspace.org.uk added the comment: 2.5 / 2.6 are in security fix only mode. So this won't get fixed. Please don't reopen. -- stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11819 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
New submission from JoeKuan kuan@gmail.com: a = (1970, 1, 1, 0, 59, 58, 0, 0, 0) time.mktime(a) -2.0 a = (1970, 1, 1, 0, 59, 59, 0, 0, 0) time.mktime(a) Traceback (most recent call last): File stdin, line 1, in module OverflowError: mktime argument out of range a = (1970, 1, 1, 1, 0, 0, 0, 0, 0) time.mktime(a) 0.0 a = (1970, 1, 1, 0, 59, 60, 0, 0, 0) time.mktime(a) 0.0 -- messages: 133806 nosy: JoeKuan priority: normal severity: normal status: open title: mktime - OverflowError: mktime argument out of range - on very specific time versions: Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
Changes by Ezio Melotti ezio.melo...@gmail.com: -- nosy: +belopolsky ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
Marc-Andre Lemburg m...@egenix.com added the comment: JoeKuan wrote: New submission from JoeKuan kuan@gmail.com: a = (1970, 1, 1, 0, 59, 58, 0, 0, 0) time.mktime(a) -2.0 On Windows, you get an OverflowError for this tuple as well. a = (1970, 1, 1, 0, 59, 59, 0, 0, 0) time.mktime(a) Traceback (most recent call last): File stdin, line 1, in module OverflowError: mktime argument out of range a = (1970, 1, 1, 1, 0, 0, 0, 0, 0) time.mktime(a) 0.0 a = (1970, 1, 1, 0, 59, 60, 0, 0, 0) time.mktime(a) 0.0 Note that time.mktime() is direct interface to the C lib funtion of the same name. As a result, the support for the various values is platform dependent. In general, dates before the epoch tend not to work or give wrong results. Since mktime() works on local time, the time zone in affect on 1970-01-01 matters and that's why you are seeing the OverflowError even for values after 1970-01-01 00:00:00. -- nosy: +lemburg title: mktime - OverflowError: mktime argument out of range - on very specific time - mktime - OverflowError: mktime argument out of range - on very specific time ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11849] ElementTree memory leak
Florent Xicluna florent.xicl...@gmail.com added the comment: I've tested a small variant of your script, on OSX. It seems to behave correctly (with 2.5, 2.6, 2.7 and 3.1). You can force Python to release memory immediately by calling gc.collect(). -- Added file: http://bugs.python.org/file21670/issue11849_test.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11849 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11849] ElementTree memory leak
Florent Xicluna florent.xicl...@gmail.com added the comment: this is the output for 2.7.1: $ python2.7 issue11849_test.py *** Python 2.7.1 final --- PID STAT TIME SL RE PAGEIN VSZRSS LIM TSIZ %CPU %MEM COMMAND 0 2754 S+ 0:00.07 0 0 0 2441472 5372 -0 11,7 0,1 python2.7 issue11849_test.py 1 2754 S+ 0:02.36 0 0 0 2520740 83720 -0 100,0 2,0 python2.7 issue11849_test.py 2 2754 S+ 0:04.89 0 0 0 2596784 15 -0 100,0 3,8 python2.7 issue11849_test.py 3 2754 S+ 0:07.28 0 0 0 2668740 230972 -0 100,0 5,5 python2.7 issue11849_test.py 4 2754 S+ 0:10.11 0 0 0 2740932 303200 -0 100,0 7,2 python2.7 issue11849_test.py 5 2754 S+ 0:12.85 0 0 0 2812876 375276 -0 98,4 8,9 python2.7 issue11849_test.py 6 2754 R+ 0:14.95 0 0 0 2885868 447740 -0 98,9 10,7 python2.7 issue11849_test.py 7 2754 S+ 0:17.91 0 0 0 2962156 522560 -0 99,1 12,5 python2.7 issue11849_test.py 8 2754 S+ 0:21.08 0 0 0 3034092 594620 -0 98,3 14,2 python2.7 issue11849_test.py 9 2754 S+ 0:23.20 0 0 0 3106028 667004 -0 100,0 15,9 python2.7 issue11849_test.py END 2754 S+ 0:27.50 0 0 0 2551160 114480 -0 96,3 2,7 python2.7 issue11849_test.py GC 2754 S+ 0:27.75 0 0 0 2454904 18992 -0 97,2 0,5 python2.7 issue11849_test.py *** 2754 S+ 0:27.75 0 0 0 2454904 18992 -0 3,0 0,5 python2.7 issue11849_test.py -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11849 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5057] Unicode-width dependent optimization leads to non-portable pyc file
Ezio Melotti ezio.melo...@gmail.com added the comment: Do you think this should go in 3.1 too? -- versions: +Python 3.2, Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5057 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5057] Unicode-width dependent optimization leads to non-portable pyc file
Marc-Andre Lemburg m...@egenix.com added the comment: Ezio Melotti wrote: Ezio Melotti ezio.melo...@gmail.com added the comment: Do you think this should go in 3.1 too? If the problem triggers there as well: Yes. Is the problem also visible on Python 2.7 ? -- title: Unicode-width dependent optimizationleads to non-portable pyc file - Unicode-width dependent optimizationleads to non-portable pyc file ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5057 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5057] Unicode-width dependent optimization leads to non-portable pyc file
Ezio Melotti ezio.melo...@gmail.com added the comment: Yes. The original report was for 2.6. I will apply the patch on all the 4 branches then. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5057 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11849] ElementTree memory leak
kaifeng cafe...@gmail.com added the comment: Python 3.2 On Linux (CentOS 5.3) *** Python 3.2.0 final --- PID TTY STAT TIME MAJFL TRS DRS RSS %MEM COMMAND 0 15116 pts/0S+ 0:00 1 1316 11055 6452 0.6 python3.2 issue11849_test.py 1 15116 pts/0S+ 0:02 1 1316 53155 47340 4.5 python3.2 issue11849_test.py 2 15116 pts/0S+ 0:05 1 1316 91051 86364 8.3 python3.2 issue11849_test.py 3 15116 pts/0S+ 0:08 1 1316 129067 124232 12.0 python3.2 issue11849_test.py 4 15116 pts/0S+ 0:10 1 1316 166587 162096 15.6 python3.2 issue11849_test.py 5 15116 pts/0S+ 0:13 1 1316 204483 198824 19.2 python3.2 issue11849_test.py 6 15116 pts/0S+ 0:17 1 1316 242375 236692 22.8 python3.2 issue11849_test.py 7 15116 pts/0S+ 0:19 1 1316 284383 277528 26.8 python3.2 issue11849_test.py 8 15116 pts/0S+ 0:23 1 1316 318371 312452 30.1 python3.2 issue11849_test.py 9 15116 pts/0S+ 0:25 1 1316 360235 353288 34.1 python3.2 issue11849_test.py END 15116 pts/0S+ 0:30 1 1316 393975 388176 37.4 python3.2 issue11849_test.py GC 15116 pts/0S+ 0:30 1 1316 352035 347656 33.5 python3.2 issue11849_test.py *** 15116 pts/0S+ 0:30 1 1316 352035 347656 33.5 python3.2 issue11849_test.py -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11849 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
JoeKuan kuan@gmail.com added the comment:
I don't think it is to do with the underlying C mktime. Because it works fine
with 00:59:58 and 01:00:00, 1, Jan 1970. It is to do with some specific value
-1 in the internal code of time.mktime
Here is the C code.
int main(int argc, char *argv[]) {
struct tm aTime = { 58, 59, 0, 1, 0, 70, 0, 0, 0, 0 };
time_t mTime = mktime(aTime);
printf(%s\n, ctime(mTime));
aTime.tm_sec = 59;
mTime = mktime(aTime);
printf(%s\n, ctime(mTime));
aTime.tm_sec = 0;
aTime.tm_min = 0;
aTime.tm_hour = 1;
mTime = mktime(aTime);
printf(%s\n, ctime(mTime));
}
---
Output from the same machine which gives the python error message
Thu Jan 1 00:59:58 1970
Thu Jan 1 00:59:59 1970
Thu Jan 1 01:00:00 1970
--
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11850
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
Alexander Belopolsky alexander.belopol...@gmail.com added the comment: Isn't this a duplicate of issue1726687? -- nosy: +Alexander.Belopolsky title: mktime - OverflowError: mktime argument out of range - on very specific time - mktime - OverflowError: mktime argument out of range - on very specific time ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
Marc-Andre Lemburg m...@egenix.com added the comment:
JoeKuan wrote:
JoeKuan kuan@gmail.com added the comment:
I don't think it is to do with the underlying C mktime. Because it works fine
with 00:59:58 and 01:00:00, 1, Jan 1970. It is to do with some specific value
-1 in the internal code of time.mktime
Here's the module code:
buf.tm_wday = -1; /* sentinel; original value ignored */
tt = mktime(buf);
/* Return value of -1 does not necessarily mean an error, but tm_wday
* cannot remain set to -1 if mktime succedded. */
if (tt == (time_t)(-1) buf.tm_wday == -1) {
PyErr_SetString(PyExc_OverflowError,
mktime argument out of range);
return NULL;
}
This is correct by the books, since the Linux man-page states (the
POSIX page is similar):
The mktime() function modifies the fields of the tm structure as
follows:
tm_wday and tm_yday are set to values determined from the contents of
the
other fields; if structure members are outside their valid interval,
they
will be normalized (so that, for example, 40 October is changed into 9
Novem-
ber); tm_isdst is set (regardless of its initial value) to a positive
value
or to 0, respectively, to indicate whether DST is or is not in effect at
the
specified time. Calling mktime() also sets the external variable tzname
with
information about the current timezone.
If the specified broken-down time cannot be represented as calendar
time
(seconds since the Epoch), mktime() returns a value of (time_t) -1 and
does
not alter the members of the broken-down time structure.
On which platform are you trying this ?
Here is the C code.
int main(int argc, char *argv[]) {
struct tm aTime = { 58, 59, 0, 1, 0, 70, 0, 0, 0, 0 };
time_t mTime = mktime(aTime);
printf(%s\n, ctime(mTime));
aTime.tm_sec = 59;
mTime = mktime(aTime);
printf(%s\n, ctime(mTime));
aTime.tm_sec = 0;
aTime.tm_min = 0;
aTime.tm_hour = 1;
mTime = mktime(aTime);
printf(%s\n, ctime(mTime));
}
---
Output from the same machine which gives the python error message
Thu Jan 1 00:59:58 1970
Thu Jan 1 00:59:59 1970
Thu Jan 1 01:00:00 1970
On Windows, you get errors for the first two.
--
title: mktime - OverflowError: mktime argument out of range - on very specific
time - mktime - OverflowError: mktime argument out of range - on very
specific time
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11850
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
Marc-Andre Lemburg m...@egenix.com added the comment: Alexander Belopolsky wrote: Alexander Belopolsky alexander.belopol...@gmail.com added the comment: Isn't this a duplicate of issue1726687? Could be, but that patch is not yet in Python 2.7, since Python 2.7.1 was release in Nov 2010. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
Alexander Belopolsky belopol...@users.sourceforge.net added the comment: If we can rely on the versions field, OP is using python 2.6. I don't think this can be classified as a security issue, so it won't be appropriate to backport issue1726687 to 2.6. -- assignee: - belopolsky components: +Extension Modules nosy: -Alexander.Belopolsky stage: - test needed type: - behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
STINNER Victor victor.stin...@haypocalc.com added the comment: This issue is a duplicate of #1726687 which is already fixed in Python 2.7 by 7a89f4a73d1a (Feb 15 2011): it will be part of 2.7.2. Only security vulnerabilities are fixed in Python 2.6, so I change the version field to 2.7 only. -- nosy: +haypo resolution: - duplicate versions: +Python 2.7 -Python 2.6 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
Changes by STINNER Victor victor.stin...@haypocalc.com: -- status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11787] File handle leak in TarFile lib
Éric Araujo mer...@netwok.org added the comment: LGTM. Is an automated test really needed, or just a manual run with a pydebug build? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11787 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11851] Flushing the standard input causes an error
Changes by Edzard Pasma pasm...@concepts.nl: -- components: None nosy: pasm...@concepts.nl priority: normal severity: normal status: open title: Flushing the standard input causes an error type: behavior ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11851 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11851] Flushing the standard input causes an error
New submission from Nadeem Vawda nadeem.va...@gmail.com: Could you provide more details on the problem? What version of Python did you encounter this error under? A short code fragment that triggers the error would also be useful. (I get no errors executing sys.stdin.flush() on 2.6.6 or 3.3) -- nosy: +nadeem.vawda ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11851 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11787] File handle leak in TarFile lib
Ezio Melotti ezio.melo...@gmail.com added the comment: An automated test would be better. It should be enough to create an invalid tar file, do something similar to the snippet in the first message, but checking that an error is raised and that all the files are closed anyway. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11787 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11819] '-m unittest' should not pretend it works on Python 2.5/2.6
anatoly techtonik techto...@gmail.com added the comment: I need a why-python-suxx keyword to point people with dumb questions about why they should not use specific Python versions to a query that lists all sensitive issues for this specific version that won't be fixed due to security fix only mode. -- resolution: duplicate - wont fix ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11819 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5057] Unicode-width dependent optimization leads to non-portable pyc file
Roundup Robot devnull@devnull added the comment: New changeset 3cffa2009a92 by Ezio Melotti in branch '2.7': Issue #5057: fix a bug in the peepholer that led to non-portable pyc files between narrow and wide builds while optimizing BINARY_SUBSCR on non-BMP chars (e.g. u\U00012345[0]). http://hg.python.org/cpython/rev/3cffa2009a92 New changeset 4679d0fef389 by Ezio Melotti in branch '3.1': Issue #5057: fix a bug in the peepholer that led to non-portable pyc files between narrow and wide builds while optimizing BINARY_SUBSCR on non-BMP chars (e.g. \U00012345[0]). http://hg.python.org/cpython/rev/4679d0fef389 New changeset 503578ddf286 by Ezio Melotti in branch '3.2': #5057: Merge with 3.1. http://hg.python.org/cpython/rev/503578ddf286 New changeset 9801e1f78264 by Ezio Melotti in branch 'default': #5057: Merge with 3.2. http://hg.python.org/cpython/rev/9801e1f78264 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5057 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue5057] Unicode-width dependent optimization leads to non-portable pyc file
Changes by Ezio Melotti ezio.melo...@gmail.com: -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue5057 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11819] '-m unittest' should not pretend it works on Python 2.5/2.6
Ezio Melotti ezio.melo...@gmail.com added the comment: At some point we have to draw the line, otherwise we would have to backport things to 2.3 and 2.4 too. We are already maintaining 4 branches (2.7, 3.1, 3.2, 3.3). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11819 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11850] mktime - OverflowError: mktime argument out of range - on very specific time
Marc-Andre Lemburg m...@egenix.com added the comment: Marc-Andre Lemburg wrote: Marc-Andre Lemburg m...@egenix.com added the comment: Alexander Belopolsky wrote: Alexander Belopolsky alexander.belopol...@gmail.com added the comment: Isn't this a duplicate of issue1726687? Could be, but that patch is not yet in Python 2.7, since Python 2.7.1 was release in Nov 2010. FWIW: The fix does work on Linux for the mentioned special case. Aside: mxDateTime will have a similar fix in the next release and that will also be available for Python 2.6. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11850 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11851] Flushing the standard input causes an error
Changes by STINNER Victor victor.stin...@haypocalc.com: -- nosy: +haypo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11851 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11819] '-m unittest' should not pretend it works on Python 2.5/2.6
anatoly techtonik techto...@gmail.com added the comment: I know. But stuff like this is necessary for proper release management and future planning. Using why-python-suxx per module ™ metric, it is possible to pinpoint badly designed parts that should be removed or replaced in Python4. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11819 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11851] Flushing the standard input causes an error
Edzard Pasma pasm...@concepts.nl added the comment: Hello, The error occured in the APSW shell, when using its .output command. Looking at the apsw source, it appears to perform a sys.stdin.flush() at that point. Trying this in the Python interpreto gives the same error: $ python Python 2.7.1 (r271:86882M, Nov 30 2010, 09:39:13) [GCC 4.0.1 (Apple Inc. build 5494)] on darwin Type help, copyright, credits or license for more information. import sys sys.stdin.flush built-in method flush of file object at 0x28020 sys.stdin.flush() Traceback (most recent call last): File stdin, line 1, in module IOError: [Errno 9] Bad file descriptor But in Python3 it no longer occurs. I'd like to report this to the developer of the APSW module as it appears most easy to avoid it there (also there are more frequent releases). I hope it is useful to report it here too. Regards, E. Pasma (sorry that the original post was empty) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11851 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue828450] sdist generates bad MANIFEST on Windows
Éric Araujo mer...@netwok.org added the comment: setuptools sdist uses a wholly different machinery than distutils, so it’s a red herring. Have you tested that your patch does reproduce the bug? From the diff header, I see that you’ve patched your installed Python instead of using a developpers’ environment. The guide at docs.python.org/devguide should help you get set up. If the patch does reproduce the bug, can you also fix it? -- versions: +Python 3.3 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue828450 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10496] import site failed when Python can't find home directory (sysconfig._getuserbase)
Éric Araujo mer...@netwok.org added the comment: It’s not just a try/except, it’s a behavior change: after the patch, paths returned by sysconfig may not be fully expanded paths. I would like Tarek to make a call on this. -- assignee: - tarek ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10496 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11843] distutils doc: duplicate line in table
Éric Araujo mer...@netwok.org added the comment: Thanks for the report; I’ll fix it when I get Internet access without port 22 blocked, or any committer interested in documentation can do it. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11843 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11844] Update json to upstream simplejson latest release
Alexander Belopolsky belopol...@users.sourceforge.net added the comment: I am not sure anyone other that Bob Ippolito can contribute later versions of simplejson (or patches derived from those versions) to python. ISTM that simplejson distribution is covered by MIT license [1] which is not one of the valid initial licenses. [2] I was trying to find what was the plan for maintaining json package in stdlib when it was initially included, but the only discussion I could find was a short thread [3] and issue #2750. Neither seem to address the issue of future maintenance. [1] https://github.com/simplejson/simplejson/blob/master/LICENSE.txt [2] http://www.python.org/psf/contrib/contrib-form/ [3] http://mail.python.org/pipermail/python-3000/2008-March/012583.html -- nosy: +belopolsky, benjamin.peterson, bob.ippolito, brett.cannon, christian.heimes, georg.brandl, pjenvey, rhettinger ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11844 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10932] distutils.core.setup - data_files misbehaviour ?
Changes by Éric Araujo mer...@netwok.org: -- hgrepos: +19 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10932 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue9325] Add an option to pdb/trace/profile to run library module as a script
Nick Coghlan ncogh...@gmail.com added the comment: Good point about the extra parameter just pushing the problem one layer up the stack rather than completely solving the problem. However, on further reflection, I've realised that I really don't like having runpy import the threading module automatically, since that means even single-threaded applications run via -m will end up initialising the thread support, including the GIL. That's something we try reasonably hard to avoid doing in applications that don't actually need it (it does happen in library modules that genuinely need thread-local storage, such as the decimal module). If you look at the way Pdb._runscript currently works, it imports __main__ and then cleans it out ready to let the child script run. So replacing that with a simple module level global that refers to the runpy execution namespace would probably be an improvement. Looking at this use case more closely, though, shows that it isn't as simple as handing the whole task over to the runpy module, as the debugger needs access to the filename before it starts executing code in order to configure the trace function correctly. That means runpy needs to support a two stage execution process that allows a client script like pdb to retrieve details of the code to be executed, and then subsequently request that it be executed in a specific namespace. My first thought is to switch to a more object-oriented API along the lines of the following: - get_path_runner() - get_module_runner() These functions would parallel the current run_module() and run_path() functions, but would return a CodeRunner object instead of directly executing the specified module - CodeRunner.run(module=None) This method would actually execute the code, using the specified namespace if given, or an automatic temporary namespace otherwise. CodeRunner would store sufficient state to support the delayed execution, as well as providing access to key pieces of information (such as the filename) before code execution actually occurs. pdb could then largely be left alone from a semantic point of view (i.e. still execute everything in the true __main__ module), except that its current code for finding the script to execute would be replaced by a call to runpy.get_runner_for_path(), a new -m switch would be added that tweaked that path to invoke runp.get_runner_for_module() instead, the debugger priming step would query the CodeRunner object for the filename, and finally, the actual code execution step would invoke the run() method of the CodeRunner object (passing in __main__ itself as the target module). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue9325 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10932] distutils.core.setup - data_files misbehaviour ?
Éric Araujo mer...@netwok.org added the comment: Looks great, thanks. You haven’t addressed this part of my previous message: “I think the fix may be in the wrong place: You fixed sdist but not bdists. I think the root of the problem is in the manifest (distutils2) / filelist (distutils1) module.” I don’t understand this comment: “Though, inside zip-file we get files without its parent dir, nothing changes in manifest file(should it change?).” This bug should be fixed in packaging and distutils2, but I’m not sure about distutils: it’s an important change in behavior. Tarek, Fred, what do you think? -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10932 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11843] distutils doc: duplicate line in table
Roundup Robot devnull@devnull added the comment: New changeset 9e49f4d81f54 by Ezio Melotti in branch '2.7': #11843: remove duplicate line from table in distutil doc. http://hg.python.org/cpython/rev/9e49f4d81f54 New changeset 1d6e28df2fb7 by Ezio Melotti in branch '3.1': #11843: remove duplicate line from table in distutil doc. http://hg.python.org/cpython/rev/1d6e28df2fb7 New changeset 850a659d9e6f by Ezio Melotti in branch '3.2': #11843: Merge with 3.1. http://hg.python.org/cpython/rev/850a659d9e6f New changeset bf1bf8fb5d55 by Ezio Melotti in branch 'default': #11843: Merge with 3.2. http://hg.python.org/cpython/rev/bf1bf8fb5d55 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11843 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11843] distutils doc: duplicate line in table
Ezio Melotti ezio.melo...@gmail.com added the comment: Done, thanks for the report. -- assignee: docs@python - ezio.melotti nosy: +ezio.melotti resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11843 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11277] test_zlib.test_big_buffer crashes under BSD (Mac OS X and FreeBSD)
Steffen Daode Nurpmeso sdao...@googlemail.com added the comment:
I was able to spend more time on that this afternoon.
'Got an unkillable diff(1) along the way which required me to
force a cold reboot. Well.
I attach a C version (11277.mmap.c) which i've used for testing.
The file 11277.zsum32.c is a quick-and-dirty C program to
calculate CRC-32 and Adler-32 checksums (i had none for the
latter and maybe you want to test some more, so); it requires zlib.
I also attach 11277.1.diff which updates test/test_zlib.py, though
this is rather useless, because that still results in a bus error.
This is the real interesting thing however, because the C version
actually works quite well for the chosen value, and the resulting
files are identical, as zsum32 shows:
Adler-32 14b9018b CRC-32 c6e340bf -- test_python_413/@test_413_tmp
Adler-32 14b9018b CRC-32 c6e340bf -- c-mmap-testfile
I thought
os.fsync(f.fileno())
does the trick because it does it in C (hi, Charles-Francois),
but no.
So what do i come up with?
Nothing. A quick look into 11277.mmap.c will show you this:
/* *Final* sizes (string written after lseek(2): abcd) */
...
/* Tested good */
//0x1 - PAGESIZE - 5,
//0x1 - 4,
//0x1 - 3,
//0x1 - 1,
0x1 + PAGESIZE + 4,
//0x1 + PAGESIZE + 5,
/* Tested bad */
//0x1,
//0x1 + PAGESIZE,
//0x1 + PAGESIZE + 1,
//0x1 + PAGESIZE + 3,
Hm!
Now i have to go but maybe i can do some more testing tomorrow to
answer the question why test_zlib.py fails even though there is
the fsync() and even though the values work in C.
Any comments?
--
Added file: http://bugs.python.org/file21671/11277.1.diff
Added file: http://bugs.python.org/file21672/11277.mmap.c
Added file: http://bugs.python.org/file21673/11277.zsum32.c
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue11277
___diff --git a/Lib/test/test_zlib.py b/Lib/test/test_zlib.py
--- a/Lib/test/test_zlib.py
+++ b/Lib/test/test_zlib.py
@@ -3,6 +3,7 @@
import binascii
import random
import sys
+import os
from test.support import precisionbigmemtest, _1G, _4G
zlib = support.import_module('zlib')
@@ -68,9 +69,10 @@
def setUp(self):
with open(support.TESTFN, wb+) as f:
-f.seek(_4G)
-f.write(basdf)
-with open(support.TESTFN, rb) as f:
+f.seek(_4G + mmap.PAGESIZE)
+f.write(babcd)
+f.flush()
+os.fsync(f.fileno())
self.mapping = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ)
def tearDown(self):
@@ -82,9 +84,8 @@
@unittest.skipUnless(support.is_resource_enabled(largefile),
May use lots of disk space.)
def test_big_buffer(self):
-self.assertEqual(zlib.crc32(self.mapping), 3058686908)
-self.assertEqual(zlib.adler32(self.mapping), 82837919)
-
+self.assertEqual(zlib.crc32(self.mapping), 0xc6e340bf)
+self.assertEqual(zlib.adler32(self.mapping), 0x14b9018b)
class ExceptionTestCase(unittest.TestCase):
# make sure we generate some expected errors
#include errno.h
#include signal.h
#include stdio.h
#include string.h
#include fcntl.h
#include unistd.h
#include sys/mman.h
#include sys/stat.h
#include sys/types.h
#include sys/uio.h
#define PATHc-mmap-testfile
#define PAGESIZE4096
static void sighdl(int);
static void
sighdl(int signo)
{
const char errmsg[] = \nSignal occurred, cleaning up\n;
(void)signo;
(void)signal(SIGSEGV, SIG_DFL);
(void)signal(SIGBUS, SIG_DFL);
write(2, errmsg, sizeof(errmsg)-1);
(void)unlink(PATH);
return;
}
int
main(void) {
int fd, estat = 0;
void *addr;
auto struct stat s;
/* *Final* sizes (string written after lseek(2): abcd) */
const size_t *ct, tests[] = {
/* Tested good */
//0x1 - PAGESIZE - 5,
//0x1 - 4,
//0x1 - 3,
//0x1 - 1,
0x1 + PAGESIZE + 4,
//0x1 + PAGESIZE + 5,
/* Tested bad */
//0x1,
//0x1 + PAGESIZE,
//0x1 + PAGESIZE + 1,
//0x1 + PAGESIZE + 3,
0
};
if (signal(SIGSEGV, sighdl) == SIG_ERR)
goto jerror;
if (signal(SIGBUS, sighdl) == SIG_ERR)
goto jerror;
for (ct = tests; *ct != 0; ++ct) {
fprintf(stderr, Size %lu/0x%lX: open, *ct, *ct);
fd = open(PATH, O_RDWR|O_TRUNC|O_CREAT, 0666);
if (fd 0)
goto jerror;
fprintf(stderr, . );
fprintf(stderr, lseek);
if (lseek(fd, *ct-4, SEEK_END) 0)
goto jerror;
fprintf(stderr, . );
fprintf(stderr, write);
if (write(fd, abcd, 4) != 4)
goto jerror;
