Re: xlrd 0.8.0 released!
On Wednesday, August 1, 2012 11:01:56 AM UTC-4, Chris Withers wrote: Hi All, I'm pleased to announce the release of xlrd 0.8.0: http://pypi.python.org/pypi/xlrd/0.8.0 This release finally lands the support for both .xls and .xlsx files. Many thanks to John Machin for all his work on making this happen. Opening of .xlsx files is seamless, just use xlrd as you did before and it all should just work. xlrd 0.8.0 is also the first release that that targets Python 2.6 and 2.7, but no Python 3 just yet. Python 2.5 and below may work but are not supported. If you need to use Python 2.5 or earlier, please stick to xlrd 0.7.x. Speaking of xlrd 0.7.x, that's now in requested maintenance only mode ;-) That means, if possible, use 0.8.x. If you have a really good reason for sticking with 0.7.x, and you find a bug that you can't work around, then please make this clear on the python-ex...@googlegroups.com and we'll see what we can do. If you find any problems, please ask about them on the list, or submit an issue on GitHub: https://github.com/python-excel/xlrd/issues Full details of all things Python and Excel related can be found here: http://www.python-excel.org/ cheers, Chris -- Simplistix - Content Management, Batch Processing Python Consulting - http://www.simplistix.co.uk Congrats! Being able to read Office 2007 files will be very useful. Looking forward to the Python 3 support. -- http://mail.python.org/mailman/listinfo/python-list
Re: interval arithmetic libraries
On Jun 6, 8:15 am, srepmub mark.duf...@gmail.com wrote: Hi all, I'm looking for libraries that allow one to calculate with sets of (date) intervals. So for example, I'd like to be able to calculate the overlap between two sets of intervals, the union etc. Preferrably, this works with datetime objects, is written in pure Python, and has reasonably good (algorithmic) performance. The neatest library I've found so far is this one: http://members.cox.net/apoco/interval/ From an API standpoint, it looks rather nice, but the performance could be better (for example, calculating an overlap now involves looping quadratically over both interval sets), and it doesn't work fully with datetime objects (Inf doesn't work). Thanks for any pointers, Mark Dufour. (author of Shedskin, an experimental (restricted-)Python-to-C++ compiler,http://shedskin.googlecode.com) For some pointers look at this thread where I posted a similar question: http://groups.google.com/group/comp.lang.python/browse_frm/thread/1a1d2ed9d05d11d0/56684b795fc527cc#56684b795fc527cc I am surprised that libraries to do that are not more common. I guess everybody rolls his own. -- http://mail.python.org/mailman/listinfo/python-list
Re: fastest way to test file for string?
On Jun 5, 7:50 am, kj no.em...@please.post wrote: Hi. I need to implement, within a Python script, the same functionality as that of Unix's grep -rl some_string some_directory I.e. find all the files under some_directory that contain the string some_string. I imagine that I can always resort to the shell for this, but is there an efficient way of doing it within Python? (BTW, portability is not high on my list here; this will run on a Unix system, and non-portable Unix-specific solutions are fine with me.) TIA! -- You can write your own version of grep in python using os.walk, open, read and find. I don't know why one would want to do that unless for portability reasons. It will be pretty hard to beat grep in efficiency and succinctness. The most sensible thing IMHO is a shell script or call grep using os.system (or using subprocess). -- http://mail.python.org/mailman/listinfo/python-list
Re: DUDA !!!!!!!!!!
On Jun 4, 2:51 pm, Ariel Vazquez Riveron avazqu...@grm.uci.cu wrote: Hola: Hoy en día me encuentro iniciandome dentro del python, en estos momentos quiero saber de que forma puedo eliminar un archivo de un compactado, ya sea zip, rar o cualquier otro. Estudie las librerías zipfile pero no tiene ninguna funcion que me permita hacerlo. Trabajo con python 2.5 salu2 Ariel This message was sent using IMP, the Internet Messaging Program. En esta lista la mayoria no entiende castellano sino ingles. Mejor pregunta en esta lista: http://dir.gmane.org/gmane.org.user-groups.python.argentina -- http://mail.python.org/mailman/listinfo/python-list
Re: Illegal seek with os.popen
On Jun 3, 3:36 pm, a...@pythoncraft.com (Aahz) wrote:
In article
7c93031a-235e-4e13-bd37-7c9dbc6e8...@r16g2000vbn.googlegroups.com,
prueba...@latinmail.com wrote:
Should I open a bug report for this?
Python 2.5.1 (r251:54863, Sep 19 2007, 14:58:06) [C] on aix5
Type help, copyright, credits or license for more information.
import os
os.popen('cat','w')
open file 'cat', mode 'w' at 0x110111990
Python 3.1rc1 (r31rc1:73054, Jun 1 2009, 10:49:24) [C] on aix5
Type help, copyright, credits or license for more information.
os.popen('cat','w')
Traceback (most recent call last):
File stdin, line 1, in module
File /Python-3.1rc1/Lib/os.py, line 641, in popen
return _wrap_close(io.TextIOWrapper(proc.stdin), proc)
IOError: [Errno 29] Illegal seek
What happens in 2.6 and 3.0?
--
Aahz (a...@pythoncraft.com) * http://www.pythoncraft.com/
Given that C++ has pointers and typecasts, it's really hard to have a
serious conversation about type safety with a C++ programmer and keep a
straight face. It's kind of like having a guy who juggles chainsaws
wearing body armor arguing with a guy who juggles rubber chickens wearing
a T-shirt about who's in more danger. --Roy Smith, c.l.py, 2004.05.23
Python 2.6.2 (r262:71600, Jun 4 2009, 16:07:26) [C] on aix5
Type help, copyright, credits or license for more information.
import os
os.popen('cat','w')
open file 'cat', mode 'w' at 0x1101ab4f8
Python 3.0.1 (r301:69556, Jun 4 2009, 16:07:22) [C] on aix5
Type help, copyright, credits or license for more information.
import os
os.popen('cat','w')
os._wrap_close object at 0x1103e5748
So it seems to be something in 3.1 that causes it to fail.
BTW it is not like I use os.popen a lot. I found this problem while
trying to use help().
Python 3.1rc1 (r31rc1:73054, Jun 1 2009, 10:49:24) [C] on aix5
Type help, copyright, credits or license for more information.
help(open)
Traceback (most recent call last):
File stdin, line 1, in module
File /Python-3.1rc1/Lib/site.py, line 429, in __call__
return pydoc.help(*args, **kwds)
File /Python-3.1rc1/Lib/pydoc.py, line 1709, in __call__
self.help(request)
File /Python-3.1rc1/Lib/pydoc.py, line 1756, in help
else: doc(request, 'Help on %s:')
File /Python-3.1rc1/Lib/pydoc.py, line 1505, in doc
pager(render_doc(thing, title, forceload))
File /Python-3.1rc1/Lib/pydoc.py, line 1320, in pager
pager(text)
File /Python-3.1rc1/Lib/pydoc.py, line 1340, in lambda
return lambda text: pipepager(text, 'less')
File /Python-3.1rc1/Lib/pydoc.py, line 1359, in pipepager
pipe = os.popen(cmd, 'w')
File /Python-3.1rc1/Lib/os.py, line 641, in popen
return _wrap_close(io.TextIOWrapper(proc.stdin), proc)
IOError: [Errno 29] Illegal seek
--
http://mail.python.org/mailman/listinfo/python-list
Illegal seek with os.popen
Should I open a bug report for this?
Python 2.5.1 (r251:54863, Sep 19 2007, 14:58:06) [C] on aix5
Type help, copyright, credits or license for more information.
import os
os.popen('cat','w')
open file 'cat', mode 'w' at 0x110111990
Python 3.1rc1 (r31rc1:73054, Jun 1 2009, 10:49:24) [C] on aix5
Type help, copyright, credits or license for more information.
os.popen('cat','w')
Traceback (most recent call last):
File stdin, line 1, in module
File /Python-3.1rc1/Lib/os.py, line 641, in popen
return _wrap_close(io.TextIOWrapper(proc.stdin), proc)
IOError: [Errno 29] Illegal seek
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is there a better way to chose a slice of a list?
On May 8, 3:03 pm, walterbyrd walterb...@iname.com wrote:
This works, but it seems like there should be a better way.
--
week = ['sun','mon','tue','wed','thu','fri','sat']
for day in week[week.index('tue'):week.index('fri')]:
print day
---
Depending on the context this style might help:
week = ['sun','mon','tue','wed','thu','fri','sat']
tue_fri = slice(week.index('tue'), week.index('fri'))
for day in week[tue_fri]:
print day
But I don't really see it as an improvement unless you are using those
intervals repeatedly.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.1 beta 1
On May 7, 11:57 am, bearophileh...@lycos.com wrote:
Equality tests between OrderedDict objects are order-sensitive and are
implemented as list(od1.items())==list(od2.items()). Equality tests between
OrderedDict objects and other Mapping objects are order-insensitive
very nice idea.
I don't know if somebody else is interested but I wouldn't mind
support to OrderedDict and namedtuple in the csv module:
Have csv.DictReader return an OrderedDict instead of a regular one
(based on the order of the column names in the header line).
Have csv.DictWriter accept an OrderedDict. That would take care of
this:
Note that unlike the DictReader class, the fieldnames parameter of
the DictWriter is not optional. Since Python’s dict objects are not
ordered, there is not enough information available to deduce the order
in which the row should be written to the csvfile.
http://docs.python.org/dev/py3k/library/csv.html#csv.DictWriter
Add a new csv.NamedTupleReader and csv.NamedTupleWriter that work
similar to the DictReader and DictWriter then this snippet (in
examples
http://docs.python.org/dev/py3k/library/collections.html#collections.namedtuple
):
EmployeeRecord = namedtuple('EmployeeRecord', 'name, age, title,
department, paygrade')
import csv
for emp in map(EmployeeRecord._make, csv.reader(open(employees.csv,
rb))):
print(emp.name, emp.title)
Could be rewritten as:
import csv
for emp in csv.NamedTupleReader(employees.csv):
print(emp.name, emp.title)
Also, is there a way to convert a OrderedDict into a namedtuple? It is
possible to go the other direction using namedtuple._asdict, but I
don't see a _fromdict method.
And thanks Steven and Raymond for the Counter class. Several people
had started to see the common pattern and now we have it in the
stdlib.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.1 beta 1
On May 7, 6:33 pm, Benjamin Peterson benja...@python.org wrote: pruebauno at latinmail.com writes: Congratulations! Thanks! Is it just me or was some nice summary output added to the make process? I get a nice list of modules that didn't compile and the ones where the library could not be found. Are you compiling on a different platform? The nice output has been around for a while, bu only on non-Windows platforms. Not really, I was on AIX. It is probably just me then, probably because for the first time I see it making it through the whole process even with the Tkinter libraries missing. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 2.6, File read() problems in Windows Xp
On May 8, 5:08 am, Li Wang li.wan...@gmail.com wrote:
Hi Dave:
Thank you very much for you explanation:)
Chances are you forgot the b parameter to open(). Unnecessary in Unix, it
tells the library to *not* translate \r\n to \n upon read, or the inverse
on write. In other words, with the b parameter, the file is read in
unchanged.
So, if I am using python in Linux, do open('file', 'r') and
open('file', 'rb') work the same way?
Thanks,
Best regards,
Li
In old Python up to 2.6, YES.
--
http://mail.python.org/mailman/listinfo/python-list
Re: I'm intrigued that Python has some functional constructions in the language.
On May 8, 3:04 pm, Casey Hawthorne caseyhhammer_t...@istar.ca wrote: I'm intrigued that Python has some functional constructions in the language. Would it be possible to more clearly separate the pure code (without side effects) from the impure code (that deals with state changes, I/O, etc.), so that the pure code could be compiled and have aggressive functional transformations applied to it for efficiency. That way, the syntax would be a lot easier to understand, than most functional languages, like Haskell. I gave a presentation at the beginning of last year on Haskell and at the end, someone's comment was, I can see the benefits of functional programming but why does it have to be so cryptic. -- Regards, Casey Don't forget that the Python interpreter is simple. It makes maintenance easier and allows embedding it into other programs. Good optimizing compilers for functional languages are not simple. Your idea would be something that could be added to the PyPy project in the future. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.1 beta 1
On May 6, 9:32 pm, Benjamin Peterson benja...@python.org wrote: On behalf of the Python development team, I'm thrilled to announce the first and only beta release of Python 3.1. Python 3.1 focuses on the stabilization and optimization of features and changes Python 3.0 introduced. For example, the new I/O system has been rewritten in C for speed. File system APIs that use unicode strings now handle paths with undecodable bytes in them. [1] Other features include an ordered dictionary implementation and support for ttk Tile in Tkinter. For a more extensive list of changes in 3.1, seehttp://doc.python.org/dev/py3k/whatsnew/3.1.htmlor Misc/NEWS in the Python distribution. Please note that this is a beta release, and as such is not suitable for production environments. We continue to strive for a high degree of quality, but there are still some known problems and the feature sets have not been finalized. This beta is being released to solicit feedback and hopefully discover bugs, as well as allowing you to determine how changes in 3.1 might impact you. If you find things broken or incorrect, please submit a bug report at http://bugs.python.org For more information and downloadable distributions, see the Python 3.1 website: http://www.python.org/download/releases/3.1/ See PEP 375 for release schedule details: http://www.python.org/dev/peps/pep-0375/ Enjoy, -- Benjamin Benjamin Peterson benjamin at python.org Release Manager (on behalf of the entire python-dev team and 3.1's contributors) Congratulations! Is it just me or was some nice summary output added to the make process? I get a nice list of modules that didn't compile and the ones where the library could not be found. -- http://mail.python.org/mailman/listinfo/python-list
Re: print(f) for files .. and is print % going away?
On Apr 30, 8:30 am, Esmail ebo...@hotmail.com wrote: Matt Nordhoff wrote: Esmail wrote: Hello all, I use the print method with % for formatting my output to the console since I am quite familiar with printf from my C days, and I like it quite well. I am wondering if there is a way to use print to write formatted output to files? Also, it seems like I read that formatting with print is going away eventually and we should start using something else? Is that true and if so, what? Thanks, Esmail Hi Matt, String formatting has nothing to do with the print statement/function. It's an operator, just like doing foo + bar; you can use it wherever you want. Ah .. so this means I could use this formatting with the write method for files .. that is great news (you don't want to see the ugly code I put together there .. :-) See http://docs.python.org/library/stdtypes.html#string-formatting Also see http://docs.python.org/library/string.html#formatstrings for information on the replacement for the old string formatting, Python 2.6's new str.format(). Will do .. so do you think it's good to move to the new format, or will the older one still be around for a while? Thanks again! Esmail There is also the Template class in the stdlib string module, for unix shell/perl style formatting. The old mod (%) formatting will be around in 3.1 and that version not even out yet, so it will be around for a couple more years at the very least. Although the new format is more powerful and slightly less error prone, there is so much old code using % that there is resistance to drop it. The printf format is also well known among C programmers. One thing many Python programmers don't like is that it is a single character operator (%). It makes formatting hard to spot. Maybe printf formatting will be moved to some module in the stdlib in the future as a compromise. I hope that the information will help you decide what to use. -- http://mail.python.org/mailman/listinfo/python-list
Re: dict is really slow for big truck
On Apr 29, 1:05 pm, Scott David Daniels scott.dani...@acm.org wrote:
Bruno Desthuilliers wrote:
d = {}
for line in open(thefile):
arr = line.strip().split()
d[arr[0]] = arr
Sorry, not picking on Bruno in particular, but I keep seeing
this formulation around various places.
When does line.strip().split() ever differ from line.split()?
--Scott David Daniels
scott.dani...@acm.org
They don't.
It is probably out of habit of using the generalized idiom:
line=a,b,c\n
line.strip().split(,)
['a', 'b', 'c']
line.split(,)
['a', 'b', 'c\n']
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help AIX 5.3 build on Python-3.1a2
On Apr 28, 10:07 am, Jeroen Ruigrok van der Werven asmo...@in- nomine.org wrote: -On [20090427 20:31], prueba...@latinmail.com (prueba...@latinmail.com) wrote: ./Modules/ld_so_aix xlc_r -q64 -bI:Modules/python.exp build/ temp.aix-5.3-3.1//ptst/Python-3.1a2/Modules/_tkinter.o build/ temp.aix-5.3-3.1//ptst/Python-3.1a2/Modules/tkappinit.o -L/usr/X11R6/ lib64 -L/usr/X11R6/lib -L/usr/local/lib -ltk8.3 -ltcl8.3 -lX11 -o build/lib.aix-5.3-3.1/_tkinter.so ld: 0706-006 Cannot find or open library file: -l tk8.3 ld:open(): A file or directory in the path name does not exist. Well, do you have TK installed? It seems so, but are the paths passed to the compiler/linker correct (-L) for locating libtk8.3.so? -- Jeroen Ruigrok van der Werven asmodai(-at-)in-nomine.org / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェンhttp://www.in-nomine.org/|http://www.rangaku.org/| GPG: 2EAC625B Time is a twofold teacher, harsh and yet patient like no-one... I don't think I have tk. Since it is the last thing to build it's not a problem. At that point I just call it a successful compilation :-). Python still works fine regardless. -- http://mail.python.org/mailman/listinfo/python-list
Re: Help AIX 5.3 build on Python-3.1a2
On Apr 26, 5:14 am, Jeroen Ruigrok van der Werven asmo...@in- nomine.org wrote: -On [20090425 19:17], Aahz (a...@pythoncraft.com) wrote: In article acf6ccbd-fcdb-4141-b2ef-0c83cae99...@x5g2000yqk.googlegroups.com, prueba...@latinmail.com wrote: Include/token.h, line 42.9: 1506-213 (S) Macro name TILDE cannot be redefined. Include/token.h, line 42.9: 1506-358 (I) TILDE is defined on line 250 of /usr/include/sys/ioctl.h. Can you try trimming down the compilation to a small reproducible case? I thought it was already clear from what was provided? Include/token.h has a #define for TILDE and apparently AIX has a #define for TILDE in /usr/include/sys/ioctl.h as well. So you can a redefinition. One way around it, depending how AIX protects headers might be to change Include/token.h to either: #if defined(_SYS_IOCTL_H_) #undef TILDE #define TILDE 32 #endif or #if defined(aix) #undef TILDE #define TILDE 32 #endif -- Jeroen Ruigrok van der Werven asmodai(-at-)in-nomine.org / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェンhttp://www.in-nomine.org/|http://www.rangaku.org/| GPG: 2EAC625B A rose is a rose is a rose is a rose... Thanks Jeroen. I know that AIX isn't as supported as other platforms, but I thought it wouldn't hurt to ask anyway. At least now everybody can search for that particular problem and find something. I will experiment a little bit and see if I make it compile. -- http://mail.python.org/mailman/listinfo/python-list
Re: Help AIX 5.3 build on Python-3.1a2
On Apr 27, 10:26 am, Jeroen Ruigrok van der Werven asmo...@in- nomine.org wrote: -On [20090427 15:00], prueba...@latinmail.com (prueba...@latinmail.com) wrote: Thanks Jeroen. I know that AIX isn't as supported as other platforms, but I thought it wouldn't hurt to ask anyway. At least now everybody can search for that particular problem and find something. I will experiment a little bit and see if I make it compile. I'll be interested to see if that fixes it. I have had to support a bunch of different Unix systems over the years compile-wise, but I don't think AIX was one of them though, so I might be a bit rusty there. -- Jeroen Ruigrok van der Werven asmodai(-at-)in-nomine.org / asmodai イェルーン ラウフロック ヴァン デル ウェルヴェンhttp://www.in-nomine.org/|http://www.rangaku.org/| GPG: 2EAC625B What is the short meaning of this long speech..? Yes, I tried them and either way fixes it. But this was with version 6 of the xlc compiler and it later gave me this error: xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Python/symtable.o Python/symtable.c Python/symtable.c, line 767.50: 1506-068 (S) Operation between types struct _object* and int is not allowed. Python/symtable.c, line 826.55: 1506-068 (S) Operation between types struct _object* and int is not allowed. make: 1254-004 The error code from the last command is 1. I gave up and found a machine with xlc version 8 installed. Using that version gave me a bunch of warnings instead but it compiled. Just for the curious this is (more or less) what it looks like (long text follows!): (//ptst/Python-3.1a2) make xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Modules/python.o ./Modules/python.c xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/acceler.o Parser/acceler.c Include/token.h, line 42.9: 1506-236 (W) Macro name TILDE has been redefined. Include/token.h, line 42.9: 1506-358 (I) TILDE is defined on line 250 of /usr/include/sys/ioctl.h. xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/grammar1.o Parser/grammar1.c Include/token.h, line 42.9: 1506-236 (W) Macro name TILDE has been redefined. Include/token.h, line 42.9: 1506-358 (I) TILDE is defined on line 250 of /usr/include/sys/ioctl.h. xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/listnode.o Parser/listnode.c Include/token.h, line 42.9: 1506-236 (W) Macro name TILDE has been redefined. Include/token.h, line 42.9: 1506-358 (I) TILDE is defined on line 250 of /usr/include/sys/ioctl.h. xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/node.o Parser/node.c xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/parser.o Parser/parser.c Include/token.h, line 42.9: 1506-236 (W) Macro name TILDE has been redefined. Include/token.h, line 42.9: 1506-358 (I) TILDE is defined on line 250 of /usr/include/sys/ioctl.h. Parser/parser.c, line 32.31: 1506-1298 (W) The subscript 1500 is out of range. The valid range is 0 to 1499. Parser/parser.c, line 293.45: 1506-1298 (W) The subscript 1500 is out of range. The valid range is 0 to 1499. Parser/parser.c, line 314.29: 1506-1298 (W) The subscript 1500 is out of range. The valid range is 0 to 1499. xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/parsetok.o Parser/parsetok.c Include/token.h, line 42.9: 1506-236 (W) Macro name TILDE has been redefined. Include/token.h, line 42.9: 1506-358 (I) TILDE is defined on line 250 of /usr/include/sys/ioctl.h. xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/bitset.o Parser/bitset.c xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/metagrammar.o Parser/metagrammar.c xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/firstsets.o Parser/firstsets.c Include/token.h, line 42.9: 1506-236 (W) Macro name TILDE has been redefined. Include/token.h, line 42.9: 1506-358 (I) TILDE is defined on line 250 of /usr/include/sys/ioctl.h. xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/grammar.o Parser/grammar.c Include/token.h, line 42.9: 1506-236 (W) Macro name TILDE has been redefined. Include/token.h, line 42.9: 1506-358 (I) TILDE is defined on line 250 of /usr/include/sys/ioctl.h. xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/pgen.o Parser/pgen.c Include/token.h, line 42.9: 1506-236 (W) Macro name TILDE has been redefined. Include/token.h, line 42.9: 1506-358 (I) TILDE is defined on line 250 of /usr/include/sys/ioctl.h. xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include - DPy_BUILD_CORE -o Parser/myreadline.o Parser/myreadline.c xlc_r -q64 -c -DNDEBUG -O2 -I. -IInclude -I./Include -
Help AIX 5.3 build on Python-3.1a2
OPT=-O2 LDFLAGS=-s ./configure --prefix=/ptst --with-gcc=xlc_r -q64 --with-cxx=xlC_r -q64 --disable-ipv6 AR=ar -X64 --without-locale -- without-ctypes checking for --with-universal-archs... 32-bit checking MACHDEP... aix5 checking machine type as reported by uname -m... 00023AAA4C00 checking for --without-gcc... xlc_r -q64 checking for gcc... xlc_r -q64 checking for C compiler default output file name... a.out checking whether the C compiler works... yes checking whether we are cross compiling... no checking for suffix of executables... checking for suffix of object files... o checking whether we are using the GNU C compiler... no checking whether xlc_r -q64 accepts -g... yes checking for xlc_r -q64 option to accept ISO C89... none needed checking for --with-cxx-main=compiler... no checking for c++... no checking for g++... no checking for gcc... no checking for CC... no checking for cxx... no checking for cc++... no checking for cl... no checking how to run the C preprocessor... xlc_r -q64 -E checking for grep that handles long lines and -e... /usr/bin/grep checking for egrep... /usr/bin/grep -E checking for AIX... yes checking for --with-suffix... checking for case-insensitive build directory... no checking LIBRARY... libpython$(VERSION).a checking LINKCC... $(srcdir)/Modules/makexp_aix Modules/python.exp . $ (LIBRARY); $(PURIFY) $(MAINCC) checking for --enable-shared... no checking for --enable-profiling... checking LDLIBRARY... libpython$(VERSION).a checking for ranlib... ranlib checking for ar... ar -X64 checking for svnversion... not-found checking for a BSD-compatible install... ./install-sh -c checking for --with-pydebug... no checking whether xlc_r -q64 accepts -OPT:Olimit=0... no checking whether xlc_r -q64 accepts -Olimit 1500... no checking whether pthreads are available without options... yes checking for ANSI C header files... yes checking for sys/types.h... yes checking for sys/stat.h... yes checking for stdlib.h... yes checking for string.h... yes checking for memory.h... yes checking for strings.h... yes checking for inttypes.h... yes checking for stdint.h... yes checking for unistd.h... yes checking asm/types.h usability... no checking asm/types.h presence... no checking for asm/types.h... no checking conio.h usability... no checking conio.h presence... no checking for conio.h... no checking curses.h usability... yes checking curses.h presence... yes checking for curses.h... yes checking direct.h usability... no checking direct.h presence... no checking for direct.h... no checking dlfcn.h usability... yes checking dlfcn.h presence... yes checking for dlfcn.h... yes checking errno.h usability... yes checking errno.h presence... yes checking for errno.h... yes checking fcntl.h usability... yes checking fcntl.h presence... yes checking for fcntl.h... yes checking grp.h usability... yes checking grp.h presence... yes checking for grp.h... yes checking ieeefp.h usability... no checking ieeefp.h presence... no checking for ieeefp.h... no checking io.h usability... no checking io.h presence... no checking for io.h... no checking langinfo.h usability... yes checking langinfo.h presence... yes checking for langinfo.h... yes checking libintl.h usability... yes checking libintl.h presence... yes checking for libintl.h... yes checking ncurses.h usability... no checking ncurses.h presence... no checking for ncurses.h... no checking poll.h usability... yes checking poll.h presence... yes checking for poll.h... yes checking process.h usability... no checking process.h presence... no checking for process.h... no checking pthread.h usability... yes checking pthread.h presence... yes checking for pthread.h... yes checking shadow.h usability... no checking shadow.h presence... no checking for shadow.h... no checking signal.h usability... yes checking signal.h presence... yes checking for signal.h... yes checking for stdint.h... (cached) yes checking stropts.h usability... yes checking stropts.h presence... yes checking for stropts.h... yes checking termios.h usability... yes checking termios.h presence... yes checking for termios.h... yes checking thread.h usability... no checking thread.h presence... no checking for thread.h... no checking for unistd.h... (cached) yes checking utime.h usability... yes checking utime.h presence... yes checking for utime.h... yes checking sys/audioio.h usability... no checking sys/audioio.h presence... no checking for sys/audioio.h... no checking sys/bsdtty.h usability... no checking sys/bsdtty.h presence... no checking for sys/bsdtty.h... no checking sys/epoll.h usability... no checking sys/epoll.h presence... no checking for sys/epoll.h... no checking sys/event.h usability... no checking sys/event.h presence... no checking for sys/event.h... no checking sys/file.h usability... yes checking sys/file.h presence... yes checking for sys/file.h... yes checking sys/loadavg.h usability... no checking sys/loadavg.h presence... no checking for sys/loadavg.h... no checking
Re: Python, MS SQL, and batch inserts
On Apr 21, 5:21 pm, Scott David Daniels scott.dani...@acm.org wrote: ericwoodwo...@gmail.com wrote: On Apr 21, 4:01 pm, ericwoodwo...@gmail.com wrote: On Apr 21, 3:36 pm, Scott David Daniels scott.dani...@acm.org wrote: I forget the name of the SQL Server bulk loader, bcp (bulk copy) was the name of the bulk loader. I just remembered. Sorry, no more exciting advice. it does sound like a good idea to go to a SQL Server group. --Scott David Daniels scott.dani...@acm.org For MS SQL try: BULK INSERT AdventureWorks.Sales.SalesOrderDetail FROM ''drive:\path\filename'' WITH (ROWTERMINATOR = '''+CHAR(10)+''') -- http://mail.python.org/mailman/listinfo/python-list
Re: Help improve program for parsing simple rules
On Apr 17, 5:32 pm, Paul McGuire pt...@austin.rr.com wrote: On Apr 17, 2:40 pm, prueba...@latinmail.com wrote: On Apr 17, 11:26 am, Paul McGuire pt...@austin.rr.com wrote: On Apr 16, 10:57 am, prueba...@latinmail.com wrote: Another interesting task for those that are looking for some interesting problem: I inherited some rule system that checks for programmers program outputs that to be ported: given some simple rules and the values it has to determine if the program is still working correctly and give the details of what the values are. If you have a better idea of how to do this kind of parsing please chime in. I am using tokenize but that might be more complex than it needs to be. This is what I have come up so far: I've been meaning to expand on pyparsing's simpleArith.py example for a while, to include the evaluation of the parsed tokens. Here is the online version,http://pyparsing.wikispaces.com/file/view/eval_arith.py, it will be included in version 1.5.2 (coming shortly). I took the liberty of including your rule set as a list of embedded test cases. -- Paul That is fine with me. I don't know how feasible it is for me to use pyparsing for this project considering I don't have admin access on the box that is eventually going to run this. To add insult to injury Python is in the version 2-3 transition (I really would like to push the admins to install 3.1 by the end of the year before the amount of code written by us gets any bigger) meaning that any third party library is an additional burden on the future upgrade. I can't remember if pyparsing is pure Python. If it is I might be able to include it alongside my code if it is not too big.- Hide quoted text - - Show quoted text - It *is* pure Python, and consists of a single source file for the very purpose of ease-of-inclusion. A number of projects include their own versions of pyparsing for version compatibility management, matplotlib is one that comes to mind. The upcoming version 1.5.2 download includes a pyparsing_py3.py file for Python 3 compatibility, I should have that ready for users to download *VERY SOON NOW*! -- Paul Thanks, I will consider it. I have to admit that although it looks like it is a very good solution, it is also longer and more complex than my current code. Having to explicitly define standard python evaluation and semantics is a bit overkill. -- http://mail.python.org/mailman/listinfo/python-list
Re: when can i expect libraries and third party tools to be updated for python 3 ?
On Apr 20, 9:47 am, Deep_Feelings doctore...@gmail.com wrote: every one is telling dont go with python 3 , 3rd party tools and libraries have no compitability with python 3 so from previous experience : when can i expect libraries and third party tools to be updated for python 3 ? (especially libraries ) I predict: one year. (THIS PREDICTION IS PROVIDED AS IS, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE PREDICTION OR THE USE OR OTHER DEALINGS IN THE PREDICTION.) :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Help improve program for parsing simple rules
On Apr 16, 3:59 pm, Aaron Brady castiro...@gmail.com wrote:
On Apr 16, 10:57 am, prueba...@latinmail.com wrote:
Another interesting task for those that are looking for some
interesting problem:
I inherited some rule system that checks for programmers program
outputs that to be ported: given some simple rules and the values it
has to determine if the program is still working correctly and give
the details of what the values are. If you have a better idea of how
to do this kind of parsing please chime in. I am using tokenize but
that might be more complex than it needs to be. This is what I have
come up so far:
rules=[
'( A - B ) = 0',
'(A + B + C + D + E + F + G + H + I) = J',
'(A + B + C + D + E + F + G + H) = I',
'(A + B + C + D + E + F) = G',
'(A + B + C + D + E) = (F + G + H + I + J)',
'(A + B + C + D + E) = (F + G + H + I)',
'(A + B + C + D + E) = F',
'(A + B + C + D) = (E + F + G + H)',
'(A + B + C) = (D + E + F)',
'(A + B) = (C + D + E + F)',
'(A + B) = (C + D)',
'(A + B) = (C - D + E - F - G + H + I + J)',
'(A + B) = C',
'(A + B) = 0',
'(A+B+C+D+E) = (F+G+H+I+J)',
'(A+B+C+D) = (E+F+G+H)',
'(A+B+C+D)=(E+F+G+H)',
'(A+B+C)=(D+E+F)',
'(A+B)=(C+D)',
'(A+B)=C',
'(A-B)=C',
'(A/(B+C))',
'(G + H) = I',
'-0.99 LE ((A+B+C)-(D+E+F+G)) LE 0.99',
'-0.99 LE (A-(B+C)) LE 0.99',
'-1000.00 LE A LE 0.00',
snip
def main():
for cur_rule in rules[20:26]:
tokens=get_tokens(cur_rule)
normal=replace_comps(tokens,COMP_REPLACERS)
subst=replace_names(normal,vars_)
groups=split_seccions(subst,COMP_REPLACERS.values())
rep=all_seccions(groups)
rep_out=''.join(x[0]+x[1] for x in rep)
calc=calc_seccions(rep)
calc_out=''.join(x[0]+x[1] for x in calc)
deltas=calc_deltas(calc)
result=eval(calc_out,{},{})
snip
You are using 'eval' which isn't safe if its inputs are dangerous. If
you are controlling the inputs, you might be interested in the
optional arguments to 'eval'.
a= '-1000.00 A 0.00'
eval( a, { 'A': -100 } )
True
eval( a, { 'A': -1000 } )
False
The syntax is slightly different for Python 2. For the replacement of
'LE', I assume you require spaces on both sides.
a= '-1000.00 LE A LE 0.00'
b= a.replace( ' LE ', ' = ' )
b
'-1000.00 = A = 0.00' eval( b, { 'A': -1000 } )
True
eval( b, { 'A': -1001 } )
False
If you need something more flexible, the 'ast' module gives you more
options. The string has to be a valid Python module to start with.
FYI, have you checked order of precedence in your custom rule set to
match Python's?
I know about evals implication of safety. Rules are defined by the
programmers so I don't worry too much about it at this point. They
should know better than messing up their application server. Unless
there is some easier way to do it I am willing to take the risk.
Precedence is standard math, we can always add some extra parenthesis
to the rules, I don't thing the old system would mind.
I thought about using eval with a locals dictionary, but they want
output of the intermediate values. I want to avoid the situation where
the intermediate output does not match what eval is doing.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help improve program for parsing simple rules
On Apr 16, 9:29 pm, John Machin sjmac...@lexicon.net wrote:
On Apr 17, 1:57 am, prueba...@latinmail.com wrote:
COMP_REPLACERS={'LT':'', 'GT':'', 'LE':'=', 'GE':'=', '=':'==',
'=':'=', '=':'='}
What do the '=' and '=' represent? Why are you replacing each by
itself?
because of this:
groups=split_seccions(subst,COMP_REPLACERS.values())
I didn't want to create a separate variable. Those are dubious anyway,
I haven't seen one in all of our current rules and it generates a
syntax error in Python so I took them out:
COMP_REPLACERS={'LT':'', 'GT':'', 'LE':'=', 'GE':'=', '=':'=='}
And before somebody else points it out. This line:
(TOK.NUMBER,str(vars_.get(x[1],x[1]))) if x[0]==TOK.NAME
should really be:
(TOK.NUMBER,str(vars_[x[1]])) if (x[0]==TOK.NAME and x[1] in vars_)
--
http://mail.python.org/mailman/listinfo/python-list
Re: Help improve program for parsing simple rules
On Apr 17, 11:26 am, Paul McGuire pt...@austin.rr.com wrote: On Apr 16, 10:57 am, prueba...@latinmail.com wrote: Another interesting task for those that are looking for some interesting problem: I inherited some rule system that checks for programmers program outputs that to be ported: given some simple rules and the values it has to determine if the program is still working correctly and give the details of what the values are. If you have a better idea of how to do this kind of parsing please chime in. I am using tokenize but that might be more complex than it needs to be. This is what I have come up so far: I've been meaning to expand on pyparsing's simpleArith.py example for a while, to include the evaluation of the parsed tokens. Here is the online version,http://pyparsing.wikispaces.com/file/view/eval_arith.py, it will be included in version 1.5.2 (coming shortly). I took the liberty of including your rule set as a list of embedded test cases. -- Paul That is fine with me. I don't know how feasible it is for me to use pyparsing for this project considering I don't have admin access on the box that is eventually going to run this. To add insult to injury Python is in the version 2-3 transition (I really would like to push the admins to install 3.1 by the end of the year before the amount of code written by us gets any bigger) meaning that any third party library is an additional burden on the future upgrade. I can't remember if pyparsing is pure Python. If it is I might be able to include it alongside my code if it is not too big. -- http://mail.python.org/mailman/listinfo/python-list
Help improve program for parsing simple rules
Another interesting task for those that are looking for some
interesting problem:
I inherited some rule system that checks for programmers program
outputs that to be ported: given some simple rules and the values it
has to determine if the program is still working correctly and give
the details of what the values are. If you have a better idea of how
to do this kind of parsing please chime in. I am using tokenize but
that might be more complex than it needs to be. This is what I have
come up so far:
rules=[
'( A - B ) = 0',
'(A + B + C + D + E + F + G + H + I) = J',
'(A + B + C + D + E + F + G + H) = I',
'(A + B + C + D + E + F) = G',
'(A + B + C + D + E) = (F + G + H + I + J)',
'(A + B + C + D + E) = (F + G + H + I)',
'(A + B + C + D + E) = F',
'(A + B + C + D) = (E + F + G + H)',
'(A + B + C) = (D + E + F)',
'(A + B) = (C + D + E + F)',
'(A + B) = (C + D)',
'(A + B) = (C - D + E - F - G + H + I + J)',
'(A + B) = C',
'(A + B) = 0',
'(A+B+C+D+E) = (F+G+H+I+J)',
'(A+B+C+D) = (E+F+G+H)',
'(A+B+C+D)=(E+F+G+H)',
'(A+B+C)=(D+E+F)',
'(A+B)=(C+D)',
'(A+B)=C',
'(A-B)=C',
'(A/(B+C))',
'(G + H) = I',
'-0.99 LE ((A+B+C)-(D+E+F+G)) LE 0.99',
'-0.99 LE (A-(B+C)) LE 0.99',
'-1000.00 LE A LE 0.00',
'-5000.00 LE A LE 0.00',
'A B',
'A 7000',
'A = -(B)',
'A = C',
'A = 0',
'A GT 0',
'A GT 0.00',
'A GT 7.00',
'A LE B',
'A LT -1000.00',
'A LT -5000',
'A LT 0',
'A=(B+C+D)',
'A=B',
'I = (G + H)',
'0.00 LE A LE 4.00',
'4.00 LT A LE 7.00'
]
vars_={'A': 0, 'B': 1.1, 'C': 2.2, 'D': 3.3, 'E': 4.4, 'F': 5.5, 'G':
6.6, 'H':7.7, 'I':8.8, 'J':9.9}
import tokenize as TOK
import StringIO as SIO
COMP_REPLACERS={'LT':'', 'GT':'', 'LE':'=', 'GE':'=', '=':'==',
'=':'=', '=':'='}
def get_tokens(string):
return [x[:2] for x in TOK.generate_tokens(SIO.StringIO
(string).readline)][:-1]
def replace_comps(toks,repl):
return [(TOK.OP, repl[x[1]]) if x[1] in repl else x for x in toks]
def replace_names(norm,vars_):
return [(TOK.NUMBER,str(vars_.get(x[1],x[1]))) if x[0]==TOK.NAME
else x for x in norm]
def split_seccions(subs,comp_split):
l=[]
g=[]
for type_,value in subs:
if value in comp_split:
g.append((l,value))
l=[]
else:
l.append((type_,value))
g.append((l,''))
return g
def all_seccions(grps):
return [(TOK.untokenize(lst),comper) for lst,comper in grps]
def calc_seccions(rep):
return [(str(eval(comp,{},{})),comper) for comp,comper in rep]
def calc_deltas(calc):
return [eval(calc[i][0]+'-'+calc[i+1][0],{},{}) for i in range(len
(calc)-1)]
def main():
for cur_rule in rules[20:26]:
tokens=get_tokens(cur_rule)
normal=replace_comps(tokens,COMP_REPLACERS)
subst=replace_names(normal,vars_)
groups=split_seccions(subst,COMP_REPLACERS.values())
rep=all_seccions(groups)
rep_out=''.join(x[0]+x[1] for x in rep)
calc=calc_seccions(rep)
calc_out=''.join(x[0]+x[1] for x in calc)
deltas=calc_deltas(calc)
result=eval(calc_out,{},{})
print
print 'Values:',', '.join(str(key)+': '+str(val) for key,val
in sorted(vars_.iteritems()))
print 'Read rule: ',cur_rule
print 'Used rule: ',TOK.untokenize(normal)
print 'Substitution:',rep_out
print 'Calculation: ',calc_out
print 'Result: ','Successful' if result else 'Failed'
if not result and '==' in calc_out:
print 'Difference: ',', '.join(map(str,deltas))
print '='*40
if __name__=='__main__': main()
--
http://mail.python.org/mailman/listinfo/python-list
Re: complaints about no replies last week
On Mar 31, 4:07 pm, Arnaud Delobelle arno...@googlemail.com wrote:
prueba...@latinmail.com writes:
[...]
Well since I attracted a couple people's attention I will describe the
problem in more detail. Describing the problem properly is probably as
hard as solving it, so excuse me if I struggle a bit.
The problem is for a health insurance company and involves the period
of time a person is covered. Most insurance companies allow not only
for the main member to be insured but his family: the spouse and the
dependents (children). This additional coverage costs extra but less
than a full new insurance. So for example if Alice buys an insurance
worth at 100 dollars a month, she can insure her husband Bob for an
additional 50 dollars. Under certain circumstances Alice may go off
the insurance and only Bob stays. In that case the price goes back to
100 dollars or maybe there is a deal for 80 or something like that. In
other words the cost of the insurance is dependent on the combination
of family members that participate in it. Additionally not only do we
have different family compositions but also different insurance
products. So you can get medical, dental and vision insurance.
All that data is stored in a database that is not very tidy and looks
something like this:
First Day of Coverage, Last Day of Coverage, Relationship, Product
5/3/2005, 5/3/2005, D, M
9/10/2005, 10/10/2005, S, V
3/15/2005, 7/15/2005, M, M
3/1/2005, 6/1/2005, S, M
5/15/2005, 7/20/2005, D, D
9/10/2005, 1/1/2140, M, V
2/1/2005, 5/3/2005, M, M
Where
Relationship: M=Main Member, S=Spouse, D=Dependent
Product: M=Medical, D=Dental, V=Vision
As far as I know at the present time there are no deals based on
combination of products purchased so we will handle each product
independently. What I want is a simple algorithm that allows me to
calculate something like this out of it (hopefully I didn’t make a
mistake):
Medical:
2/1/2005, 2/28/2005, M
3/1/2005, 5/2/2005, MS
5/3/2005, 5/3/2005, MSD
5/4/2005, 6/1/2005, MS
6/2/2005, 7/15/2005, M
Dental:
5/15/2005, 7/20/2005, D
Vision:
9/10/2005, 10/10/2005, MS
10/11/2005, 1/1/2140, M
OK the approach I describe in my previous email works fine for this
particular problem. I implement it below - the function walk_ivals is at
the heart of it, I've made it as simple as possible and it's pretty
clear that it is O(nlogn).
The function that actually sorts the data is union(), and it's just a
call to walk_ivals with callback the function acc() which is constant
time, therefore union() itself has the same complexity as walk_ivals.
There are no comments - I don't have the time to add any, sorry!
--
import datetime
from collections import defaultdict
def walk_ivals(ivals, callback, endvals=(-1, 1)):
endpoints = [(x, data) for ival, data in ivals for x in zip(ival,
endvals)]
endpoints.sort()
for (endpoint, endval), data in endpoints:
callback(endpoint, endval, data)
def union(ivals):
timelines = defaultdict(list)
mult = defaultdict(lambda: defaultdict(int))
def acc(date, step, data):
rel, prod = data
old_mult = mult[prod][rel]
mult[prod][rel] -= step
if not(old_mult and mult[prod][rel]):
rels = [rel for rel, m in mult[prod].iteritems() if m]
if timelines[prod] and timelines[prod][-1][0] == date:
timelines[prod][-1][1] = rels
else:
timelines[prod].append([date, rels])
walk_ivals(ivals, acc)
return timelines
test_data = 5/3/2005, 5/3/2005, D, M
9/10/2005, 10/10/2005, S, V
3/15/2005, 7/15/2005, M, M
3/1/2005, 6/1/2005, S, M
5/15/2005, 7/20/2005, D, D
9/10/2005, 1/1/2140, M, V
2/1/2005, 5/3/2005, M, M
def parse_date(date_string):
month, day, year = map(int, date_string.split('/'))
return datetime.date(year, month, day)
def parse_data(data_string):
data = []
for line in data_string.split(\n):
start, end, rel, prod = line.split(,)
start, end = map(parse_date, (start + datetime.timedelta(1), end))
rel, prod = rel.strip(), prod.strip()
data.append([(start, end), (rel, prod)])
return data
def test():
ivals = parse_data(test_data)
timelines = union(ivals)
for prod, timeline in timelines.iteritems():
print -*20
print Product, prod
for date, covers in timeline:
print date, ' '.join(covers)
if __name__ == '__main__':
test()
--
Here is what it outputs:
marigold:junk arno$ python intervals2.py
Product M
2005-02-01 M
2005-03-01 S M
2005-05-03 S M D
2005-05-04 S M
2005-06-02 M
2005-07-16
Product D
2005-05-15 D
2005-07-21
Product V
2005-09-10
Re: complaints about no replies last week
On Mar 31, 2:56 am, Arnaud Delobelle arno...@googlemail.com wrote: Arnaud Delobelle wrote: prueba...@latinmail.com writes: [...] I myself asked about how to write a library to efficiently do union and intersection of sets containing time intervals some time ago on this list and got little to no answers. It is a tricky problem. Since I was getting paid I got an O(n*n) solution working. People on this list on the other hand do not get paid and answer whatever strikes their fancy. Sometimes the question is hard or confusing and nobody is motivated enough to answer. I wasn't around when you posted this I guess. Do you mean intervals sets on the (real) number line such as: 1-3, 6-10 meaning all numbers between 1 and 3 and all numbers between 6 and 10. In this case I think you can achieve union and intersection in O(nlogn) where n is the total number of intervals in the interval sets to unify or intersect. There is an implementation below. I have chosen a very simple data structure for interval sets: an interval set is the list of its endpoints. E.g. 1-3, 6-10 is the list [1, 3, 6, 10] This means that I can't specify whether an interval is closed or open. So in the implementation below all intervals are assumed to be open. The method could be made to work for any kind of intervals with the same complexity, there would just be a few more LOC. I'm focusing on the principle - here it is: -- # Implementation of union and intersection of interval sets. from itertools import * def combine(threshold, intsets): endpoints = sorted(chain(*imap(izip, intsets, repeat(cycle([1,-1]) height = 0 compound = [] for x, step in endpoints: old_height = height height += step if max(height, old_height) == threshold: compound.append(x) return compound def union(*intsets): return combine(1, intsets) def intersection(*intsets): return combine(len(intsets), intsets) # tests def pretty(a): a = iter(a) return ', '.join(%s-%s % (a, b) for a, b in izip(a, a)) tests = [ ([1, 5, 10, 15], [3, 11, 13, 20]), ([2, 4, 6, 8], [4, 7, 10, 11]), ([0, 11], [5, 10, 15, 25], [7, 12, 13, 15]), ] for intsets in tests: print sets: , ; .join(imap(pretty, intsets)) print union: , pretty(union(*intsets)) print intersection: , pretty(intersection(*intsets)) print -*20 -- Is this what you were looking for? -- Arnaud I realised after posting last night that I must be (1) solving the wrong problem (2) solving it badly - My implementation of the combine() function above is O(nlogn) (because of the sorted() call) whereas it could be O(n) by iterating over the interval in the parallel manner, hence (2). This would make union() and intersection() O(n). - As the problem was solved by the OP in O(n^2) I must be solving the wrong problem (1). I apologise for this. However it was a nice and compact implementation IMHO :) -- Arnaud I am pretty sure the problem can be solved in O(n log n). I just wasn't feeling overly smart when I was writing the algorithm. N is on average 4 and it had eventually to be implemented inside a framework using C++ anyway, so it is pretty fast. I can’t believe that no programmer has come over the same kind of problem before, yet my Google fu didn’t do anything for me. Well since I attracted a couple people's attention I will describe the problem in more detail. Describing the problem properly is probably as hard as solving it, so excuse me if I struggle a bit. The problem is for a health insurance company and involves the period of time a person is covered. Most insurance companies allow not only for the main member to be insured but his family: the spouse and the dependents (children). This additional coverage costs extra but less than a full new insurance. So for example if Alice buys an insurance worth at 100 dollars a month, she can insure her husband Bob for an additional 50 dollars. Under certain circumstances Alice may go off the insurance and only Bob stays. In that case the price goes back to 100 dollars or maybe there is a deal for 80 or something like that. In other words the cost of the insurance is dependent on the combination of family members that participate in it. Additionally not only do we have different family compositions but also different insurance products. So you can get medical, dental and vision insurance. All that data is stored in a database that is not very tidy and looks something like this: First Day of Coverage, Last Day of Coverage, Relationship, Product 5/3/2005, 5/3/2005, D, M 9/10/2005, 10/10/2005, S, V 3/15/2005, 7/15/2005, M, M 3/1/2005, 6/1/2005, S, M 5/15/2005, 7/20/2005, D, D 9/10/2005, 1/1/2140, M, V 2/1/2005, 5/3/2005, M,
Re: complaints about no replies last week
On Mar 28, 11:07 am, Aaron Brady castiro...@gmail.com wrote: Hi, A week ago, I posted a question and an idea about Python's garbage collector. I got a few replies. Some days later, I posted a mock-up implementation of it, and got *NO* replies. Does this mean: a) It works b) It doesn't work c) It's not particularly applicable to Python at that point (particularly) d) It's not news Thanks and sincerely. P.S. Non-plugging links:http://groups.google.com/group/comp.lang.python/browse_thread/thread/...http://groups.google.com/group/comp.lang.python/browse_thread/thread/... e) It is a hard or complex problem that requires significant investment of time on a problem or approach that few people are interested in at the moment. f) The description is confusing or incomplete or the source code is long and difficult to read. I myself asked about how to write a library to efficiently do union and intersection of sets containing time intervals some time ago on this list and got little to no answers. It is a tricky problem. Since I was getting paid I got an O(n*n) solution working. People on this list on the other hand do not get paid and answer whatever strikes their fancy. Sometimes the question is hard or confusing and nobody is motivated enough to answer. -- http://mail.python.org/mailman/listinfo/python-list
Re: Programming Python 4th Edition?
On Mar 26, 10:08 pm, Esmail ebo...@hotmail.com wrote: Hi, Does anyone know if there is a 4th edition of this book planned and if so, when it might be coming out? It looks like a good and comprehensive book but is getting a bit outdated(?). And I guess since I'm asking this, I might as well be asking what your favorite, comparable Python book might be :-) Thanks, Esmail Make sure that Programming Python is what you want. I was very disappointment with it (First edition) because I was expecting something like the Perl Camel book and it isn't. It isn't a introduction to the Python language like Learning Python, it doesn't work as reference like Python in a Nutshell, it doesn't contain short idiomatic code like Python Cookbook. What you are left with is different application domains and how to apply Python to them. The book is excellent if you want to do Network, GUI, Databases, etc. but poor if you want to learn about Python the core language. The title of the book should be changed from Programming Python to Applied Python so newcomers to the language don't buy it by mistake. Even the preface says that it is about application-level programming in Python. The book is pretty much 4 books in 1: System programming in Python Tkinter programming in Python Internet programming in Python Database and Persistence programming in Python In my opinion an experienced programmer that wants to learn Python the language should buy Nutshell and somebody new to programming in general Learning. You should buy Programming only if you need to develop in one of the Domains covered in the book. -- http://mail.python.org/mailman/listinfo/python-list
Re: Another form of dynamic import
On Mar 25, 10:23 am, Marco Nawijn naw...@gmail.com wrote: Hello, In short I would like to know if somebody knows if it is possible to re-execute a statement that raised an exception? I will explain the reason by providing a small introduction on why this might be nice in my case and some example code. I am using the python bindings to a *very* large C++ library. About 5000 classes divided over approx. 450 different packages are exposed through the Python interface. To reduce the number of import statements that need to be inserted and to limit the number of wildcard imports it would be very helpful if class names could be automatically imported from the proper module. There is no problem in finding out the proper module given a (valid) class name. As an example, look at the following statement aPoint = gp_Pnt(1.0, 0.0, 0.0) # Oops, this will raise a NameError, since # gp_Pnt class is unknown NameError: name 'gp_Pnt' is not defined As indicated, this will raise a NameError exception. What I would like to do is something like the following (pseudo-code): try: aPoint = gp_Pnt(1.0, 0.0, 0.0) [1] except NameError, e: name = e.args[0].split[1] if isValid(name): doImport(name) === Can I go back to statement [1] from this point? else: raise e There is no problem in catching the exception, finding out which name is unknown to python and check if this is a valid name for my library. My question is, is there any possibility of going back to the statement that raised the error, re-execute the statement and continue? Thanks for any thoughts and suggestions. Marco You can always use a loop: recover=True while True: try: aPoint = gp_Pnt(1.0, 0.0, 0.0)[1] except NameError, e: if recover: recover=False name = e.args[0].split[1] if isValid(name): doImport(name) else: raise e else: break -- http://mail.python.org/mailman/listinfo/python-list
Re: Unit testing frameworks
On Mar 24, 8:06 am, grkunt...@gmail.com wrote: I am looking for a unit testing framework for Python. I am aware of nose, but was wondering if there are any others that will automatically find and run all tests under a directory hierarchy. Thanks, Ralph *Nose *Trial *py.test -- http://mail.python.org/mailman/listinfo/python-list
Re: improve this newbie code/nested functions in Python?
On Mar 19, 10:21 pm, Esmail ebo...@gmail.com wrote: Hi, I'm new to writing Python code. This is a simple client I wrote, it works, but I feel it doesn't look as clean as it could. Can anyone make suggestions how to streamline this code? Also, I am using two nested functions, it seems that nested functions aren't used that much in Python - is that correct? And if so, how come? thanks, Esmail ps: I realize there is minimal error checking/exception handling. #!/usr/bin/env python import sys from socket import * from threading import Thread class Client(object): def __init__(self, host=localhost, port=, name = esmail): self._host = host self._port = port self._name = name self._address=(self._host, self._port) self._sock=socket(AF_INET, SOCK_STREAM) self._sock.connect(self._address) self._parent = self self._keepGoing = True def info(self): return self._host, self._port, self._name class Listener(Thread): def __init__(self, parent, tname=listener): Thread.__init__(self,name = tname) self._parent = parent print self._parent._host def run(self): while self._parent._keepGoing: m = self._parent._sock.recvfrom(1024) print m[0] class Speaker(Thread): def __init__(self, parent, tname = speaker): Thread.__init__(self,name = tname) self._parent = parent self._parent._sock.send(self._parent._name + \n) def run(self): while(True): m = raw_input(- ) if m == bye: self._parent._sock.send(self._parent._name + is signing off.\n) self._parent._sock.send(bye\n) self._parent._keepGoing = False break; else: self._parent._sock.send(m + \n) def main(): if len(sys.argv) == 4: # prog name + 3 args host = sys.argv[1] port = int(sys.argv[2]) name = sys.argv[3] c = Client(host, port, name) else: c = Client() print Client connecting to - host=%s port=%d name=%s % c.info () s = Client.Speaker(c) s.start() l = Client.Listener(c) l.start() main() How about renaming the Client to SocketConfig, Listener to ClientListener and Speaker to ClientSpeaker and put all at the same level. The you can do this: c = SocketConfig(host, port, name) s = ClientSpeaker(c) l = ClientListener(c) the other option would be to create a Speaker and Listener factory in Client that returns Speakers and Listeners so you can do: c = Client(host, port, name) s = c.Speaker() l = c.Listener() -- http://mail.python.org/mailman/listinfo/python-list
Re: How complex is complex?
On Mar 19, 1:25 pm, Paul Hildebrandt paul_hildebra...@yahoo.com wrote: On Mar 19, 9:41 am, Kottiyath n.kottiy...@gmail.com wrote: On Mar 19, 9:33 pm, Kottiyath n.kottiy...@gmail.com wrote: On Mar 19, 8:42 pm, Paul McGuire pt...@austin.rr.com wrote: On Mar 19, 4:39 am, Kottiyath n.kottiy...@gmail.com wrote: I understand that my question was foolish, even for a newbie. I will not ask any more such questions in the future. Gaaah! Your question was just fine, a good question on coding style. I wish more people would ask such questions so that bad habits could be avoided. The newbie posts that are annoying are the ones that: - are answered on page 1 of any tutorial (how do I get the second character of a string?) - are obvious homework assignments with no actual effort on the poster's part (how do I write a Python program to find the first 10 prime numbers?) - pontificate on what is wrong with Python, based on 2 hours' experience with the language (often titled What's wrong with Python, with content like Python sucks because it doesn't have a switch statement/has significant whitespace/doesn't check types of arguments/ isn't totally object-oriented like Java/doesn't have interfaces/...) - are so vague as to be just Usenet noise (titled Help me, with no content, or i need to write a program and don't know where to start can someone write it for me?) I think Daniel's joke was on the rest of us, who each had to chime in with our favorite dict processing algorithm. It *would* be good for you as a newbie to get an appreciation of the topics that were covered in these responses, though, especially the distinction between updating the dict in-place vs. creating a new dict. -- Paul Daniel, Sorry for misunderstanding your post. I hope I was not being passive-aggresive - (also because I found that the second mechanism I provided was quite horrible :-), so I was indeed being foolish there. ) Paul/Aahz, I did understand 2 things (1) When using map always consider that the function will be called everytime, so the hit on the performance is more. (2) The second mechanism and the first mechanism provides different solutions (new dict/same dict) both of which I did not think about at all. Also, thank you everyone for all the help. I have been following this thread for the last 4 months (when I started with python) and I have learned a lot. The amount of help provided here is amazing. p.s. - English is indeed not my first language :-) Oops, Forgot to mention the biggest learning. Readability is better than brevity - I rewrote your sentence to be more optimized. Readability brevity ;-) Thanks to Rhodri. This was a question which was bugging me all the time. When I look at code, I am always envious when I see the same code written in much smaller number of lines. Now, I will force myself to ask the questions Rhodri proposed (esp: does it look uglier part) before deciding whether or not to go ahead with brevity. sometimes: brevity==Readability but as in many things the trick is in finding the right tradeoff. I am willing to accept some trivial amount of additional complexity if it means I have to read less lines of code, but I have my limits too. I aim for a comprehension speed of 2-10 lines per minute for somebody proficient in the language. -- http://mail.python.org/mailman/listinfo/python-list
Re: Rough draft: Proposed format specifier for a thousands separator
On Mar 13, 7:06 am, Tim Rowe digi...@gmail.com wrote: 2009/3/12 Raymond Hettinger pyt...@rcn.com: If anyone here is interested, here is a proposal I posted on the python-ideas list. The idea is to make numbering formatting a little easier with the new format() builtin in Py2.6 and Py3.0: http://docs.python.org/library/string.html#formatspec As far as I can see you're proposing an amendment to *encourage* writing code that is not locale aware, with the amendment itself being locale specific, which surely has to be a regressive move in the 21st century. Frankly, I'd sooner see it made /harder/ to write code that is not locale aware (warnings, like FxCop gives on .net code?) tnan /easier/. Perhaps that's because I'm British, not American and I'm sick of having date fields get the date wrong because the programmer thinks the USA is the world. It makes me sympathetic to the problems caused to others by programmers who think the English-speaking world is the world. By the way, to others who think that 123,456.7 and 123.456,7 are the only conventions in common use in the West, no they're not. 123 456.7 is in common use in engineering, at least in Europe, precisely to reduce (though not eliminate) problems caused by dot and comma confusion.. -- Tim Rowe I lived in three different countries and in school used blank for thousand separator to avoid confusion with the multiply operator. I think this proposal is more for debugging big numbers and meant mostly for programmers' eyes. We are already using the dot instead of comma decimal separator in our programming languages that one more Americanism won't kill us. I am leaning towards proposal 1 now just to avoid the thousand variations that will be requested because of this, making the implementation unnecessarily complex. I can always use the 3 replacement hack (conveniently documented in the pep). +1 for Nick's proposal -- http://mail.python.org/mailman/listinfo/python-list
Re: Rough draft: Proposed format specifier for a thousands separator
On Mar 12, 3:30 am, Raymond Hettinger pyt...@rcn.com wrote: If anyone here is interested, here is a proposal I posted on the python-ideas list. The idea is to make numbering formatting a little easier with the new format() builtin in Py2.6 and Py3.0: http://docs.python.org/library/string.html#formatspec - Motivation: Provide a simple, non-locale aware way to format a number with a thousands separator. Adding thousands separators is one of the simplest ways to improve the professional appearance and readability of output exposed to end users. In the finance world, output with commas is the norm. Finance users and non-professional programmers find the locale approach to be frustrating, arcane and non-obvious. It is not the goal to replace locale or to accommodate every possible convention. The goal is to make a common task easier for many users. Research so far: Scanning the web, I've found that thousands separators are usually one of COMMA, PERIOD, SPACE, or UNDERSCORE. The COMMA is used when a PERIOD is the decimal separator. James Knight observed that Indian/Pakistani numbering systems group by hundreds. Ben Finney noted that Chinese group by ten-thousands. Visual Basic and its brethren (like MS Excel) use a completely different style and have ultra-flexible custom format specifiers like: _($* #,##0_). Proposal I (from Nick Coghlan]: A comma will be added to the format() specifier mini-language: [[fill]align][sign][#][0][minimumwidth][,][.precision][type] The ',' option indicates that commas should be included in the output as a thousands separator. As with locales which do not use a period as the decimal point, locales which use a different convention for digit separation will need to use the locale module to obtain appropriate formatting. The proposal works well with floats, ints, and decimals. It also allows easy substitution for other separators. For example: format(n, 6,f).replace(,, _) This technique is completely general but it is awkward in the one case where the commas and periods need to be swapped. format(n, 6,f).replace(,, X).replace(., ,).replace (X, .) Proposal II (to meet Antoine Pitrou's request): Make both the thousands separator and decimal separator user specifiable but not locale aware. For simplicity, limit the choices to a comma, period, space, or underscore.. [[fill]align][sign][#][0][minimumwidth][T[tsep]][dsep precision] [type] Examples: format(1234, 8.1f) -- ' 1234.0' format(1234, 8,1f) -- ' 1234,0' format(1234, 8T.,1f) -- ' 1.234,0' format(1234, 8T .f) -- ' 1 234,0' format(1234, 8d) -- ' 1234' format(1234, 8T,d) -- ' 1,234' This proposal meets mosts needs (except for people wanting grouping for hundreds or ten-thousands), but it comes at the expense of being a little more complicated to learn and remember. Also, it makes it more challenging to write custom __format__ methods that follow the format specification mini-language. For the locale module, just the T is necessary in a formatting string since the tool already has procedures for figuring out the actual separators from the local context. Comments and suggestions are welcome but I draw the line at supporting Mayan numbering conventions ;-) Raymond As far as I am concerned the most simple version plus a way to swap around commas and period is all that is needed. The rest can be done using one replace (because the decimal separator is always one of two options). This should cover everywhere but the far east. 80% of cases for 20% of implementation complexity. For example: [[fill]align][sign][#][0][,|.][minimumwidth][.precision][type] format(1234, .8.1f) -- ' 1.234,0' format(1234, ,8.1f) -- ' 1,234.0' -- http://mail.python.org/mailman/listinfo/python-list
Re: Candidate for a new itertool
On Mar 7, 8:47 pm, Raymond Hettinger pyt...@rcn.com wrote:
The existing groupby() itertool works great when every element in a
group has the same key, but it is not so handy when groups are
determined by boundary conditions.
For edge-triggered events, we need to convert a boundary-event
predicate to groupby-style key function. The code below encapsulates
that process in a new itertool called split_on().
Would love you guys to experiment with it for a bit and confirm that
you find it useful. Suggestions are welcome.
Raymond
-
from itertools import groupby
def split_on(iterable, event, start=True):
'Split iterable on event boundaries (either start events or stop
events).'
# split_on('X1X23X456X', 'X'.__eq__, True) -- X1 X23 X456 X
# split_on('X1X23X456X', 'X'.__eq__, False) -- X 1X 23X 456X
def transition_counter(x, start=start, cnt=[0]):
before = cnt[0]
if event(x):
cnt[0] += 1
after = cnt[0]
return after if start else before
return (g for k, g in groupby(iterable, transition_counter))
if __name__ == '__main__':
for start in True, False:
for g in split_on('X1X23X456X', 'X'.__eq__, start):
print list(g)
print
from pprint import pprint
boundary = '--===2615450625767277916==\n'
email = open('email.txt')
for mime_section in split_on(email, boundary.__eq__):
pprint(list(mime_section, 1, None))
print '= = ' * 30
For me your examples don't justify why you would need such a general
algorithm. A split function that works on iterables instead of just
strings seems straightforward, so maybe we should have that and
another one function with examples of problems where a plain split
does not work.
Something like this should work for the two examples you gave were the
boundaries are a known constants (and therefore there is really no
need to keep them. I can always add them later):
def split_on(iterable, boundary):
l=[]
for el in iterable:
if el!=boundary:
l.append(el)
else:
yield l
l=[]
yield l
def join_on(iterable, boundary):
it=iter(iterable)
firstel=it.next()
for el in it:
yield boundary
for x in el:
yield x
if __name__ == '__main__':
lst=[]
for g in split_on('X1X23X456X', 'X'):
print list(g)
lst.append(g)
print
print list(join_on(lst,'X'))
--
http://mail.python.org/mailman/listinfo/python-list
Re: Candidate for a new itertool
On Mar 9, 6:55 pm, Raymond Hettinger pyt...@rcn.com wrote:
[prueba]
The data often contains objects with attributes instead of tuples, and
I expect the new namedtuple datatype to be used also as elements of
the list to be processed.
But I haven't found a nice generalized way for that kind of pattern
that aggregates from a list of one datatype to a list of key plus
output datatype that would make it practical and suitable for
inclusion in the standard library.
Looks like you've searched the possibilities thoroughly and no one
aggregation function seems to meet all needs. That's usually a cue
to not try to build one and instead let simple python loops do the
work for you (that also saves the awkward itemgetter() calls in your
examples). To my eyes, all three examples look like straight-forward,
easy-to-write, easy-to-read, fast plain python:
d = defaultdict(int)
for color, n, info in data:
... d[color] += n
d.items()
[('blue', 6), ('yellow', 3), ('red', 4)]
d = defaultdict(list)
for color, n, info in data:
... d[color].append(n) d.items()
[('blue', [5, 1]), ('yellow', [3]), ('red', [2, 2])]
d = defaultdict(set)
for color, n, info in data:
... d[color].add(n)
d.items()
[('blue', set([1, 5])), ('yellow', set([3])), ('red', set([2]))]
I don't think you can readily combine all three examples into a single
aggregator without the obfuscation and awkwardness that comes from
parameterizing all of the varying parts:
def aggregator(default_factory, adder, iterable, keyfunc, valuefunc):
d = defaultdict(default_factory)
for record in iterable:
key = keyfunc(record)
value = valuefunc(record)
adder(d[key], value)
return d.items()
aggregator(list, list.append, data, itemgetter(0), itemgetter(1))
[('blue', [5, 1]), ('yellow', [3]), ('red', [2, 2])] aggregator(set,
set.add, data, itemgetter(0), itemgetter(1))
[('blue', set([1, 5])), ('yellow', set([3])), ('red', set([2]))]
Yuck! Plain Python wins.
Raymond
P.S. The aggregator doesn't work so well for:
aggregator(int, operator.iadd, data, itemgetter(0), itemgetter(1))
[('blue', 0), ('yellow', 0), ('red', 0)]
The problem is that operator.iadd() doesn't have a way to both
retrieve
and store back into a dictionary.
Yes thinking about this more, one probably needs to have two code
paths depending if the type returned by default_factory is mutable or
immutable. But you are probably right that the ratio of redundancy/
variability is pretty low for such a function and the plain written
out for loop is not too painful. The only redundancy is the creation
and manipulation of the dictionary and the explicit looping.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Candidate for a new itertool
On Mar 7, 8:47 pm, Raymond Hettinger pyt...@rcn.com wrote:
The existing groupby() itertool works great when every element in a
group has the same key, but it is not so handy when groups are
determined by boundary conditions.
For edge-triggered events, we need to convert a boundary-event
predicate to groupby-style key function. The code below encapsulates
that process in a new itertool called split_on().
Would love you guys to experiment with it for a bit and confirm that
you find it useful. Suggestions are welcome.
Raymond
-
from itertools import groupby
def split_on(iterable, event, start=True):
'Split iterable on event boundaries (either start events or stop
events).'
# split_on('X1X23X456X', 'X'.__eq__, True) -- X1 X23 X456 X
# split_on('X1X23X456X', 'X'.__eq__, False) -- X 1X 23X 456X
def transition_counter(x, start=start, cnt=[0]):
before = cnt[0]
if event(x):
cnt[0] += 1
after = cnt[0]
return after if start else before
return (g for k, g in groupby(iterable, transition_counter))
if __name__ == '__main__':
for start in True, False:
for g in split_on('X1X23X456X', 'X'.__eq__, start):
print list(g)
print
from pprint import pprint
boundary = '--===2615450625767277916==\n'
email = open('email.txt')
for mime_section in split_on(email, boundary.__eq__):
pprint(list(mime_section, 1, None))
print '= = ' * 30
Sorry to hijack the thread but I now that you have a knack for finding
good iterator patterns. I have noticed a pattern lately: Aggregation
using a defaultdict. I quickly found two examples of problems that
could use this:
http://groups.google.com/group/comp.lang.python/browse_frm/thread/c8b3976ec3ceadfd
http://www.willmcgugan.com/blog/tech/2009/1/17/python-coder-test/
To show an example, using data like this:
data=[('red',2,'other data'),('blue',5,'more data'),('yellow',3,'lots of
things'),('blue',1,'data'),('red',2,'random data')]
Then
from itertools import groupby
from operator import itemgetter
from collections import defaultdict
We can use groupby to do this:
[(el[0],sum(x[1] for x in el[1])) for el in
groupby(sorted(data,key=itemgetter(0)),itemgetter(0))]
[('blue', 6), ('red', 4), ('yellow', 3)]
[(el[0],[x[1] for x in el[1]]) for el in
groupby(sorted(data,key=itemgetter(0)),itemgetter(0))]
[('blue', [5, 1]), ('red', [2, 2]), ('yellow', [3])]
[(el[0],set([x[1] for x in el[1]])) for el in
groupby(sorted(data,key=itemgetter(0)),itemgetter(0))]
[('blue', set([1, 5])), ('red', set([2])), ('yellow', set([3]))]
But this way seems to be more efficient:
def aggrsum(data,key,agrcol):
dd=defaultdict(int)
for el in data:
dd[key(el)]+=agrcol(el)
return dd.items()
aggrsum(data,itemgetter(0),itemgetter(1))
[('blue', 6), ('yellow', 3), ('red', 4)]
def aggrlist(data,key,agrcol):
dd=defaultdict(list)
for el in data:
dd[key(el)].append(agrcol(el))
return dd.items()
aggrlist(data,itemgetter(0),itemgetter(1))
[('blue', [5, 1]), ('yellow', [3]), ('red', [2, 2])]
def aggrset(data,key,agrcol):
dd=defaultdict(set)
for el in data:
dd[key(el)].add(agrcol(el))
return dd.items()
aggrset(data,itemgetter(0),itemgetter(1))
[('blue', set([1, 5])), ('yellow', set([3])), ('red', set([2]))]
The data often contains objects with attributes instead of tuples, and
I expect the new namedtuple datatype to be used also as elements of
the list to be processed.
But I haven't found a nice generalized way for that kind of pattern
that aggregates from a list of one datatype to a list of key plus
output datatype that would make it practical and suitable for
inclusion in the standard library.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Mapping 64 bit int from C to Python-2.2
On Mar 9, 11:51 am, Explore_Imagination mr.hassanshab...@gmail.com wrote: Hi I want to map 64 bit integers from C to python. I must use Python 2.2 BUT There is no support for 64 bits integers in Python2.2 (Supported in 2.5). Now the problem is that I have these four variables: unit32_t a,b,c; uint64_t w,x,y,z; I use this funtion to map values: Py_BuildValue( (lll), a,b,c,w,x,y,z ); As I access 32 bit values in Python it works fine BUT 64 bit intergers in Pythong give garbage values . I think there may be a case of overflow when 64 bit values in C are mapped to python. Any Suggestions? You should be able to create PyLong objects from your 64 bit integers. PyLong objects have unlimited integer precision. Alas I don't know enough about the internal C code of Python to know what function to use to do the conversion from an int64 to a PyLong object. -- http://mail.python.org/mailman/listinfo/python-list
Re: Perl python - regex performance comparison
On Mar 3, 12:38 pm, Ivan ivan@@gmail.com wrote: Hello everyone, I know this is not a direct python question, forgive me for that, but maybe some of you will still be able to help me. I've been told that for my application it would be best to learn a scripting language, so I looked around and found perl and python to be the nice. Their syntax and way is not similar, though. So, I was wondering, could any of you please elaborate on the following, as to ease my dilemma: 1. Although it is all relatively similar, there are differences between regexes of these two. Which do you believe is the more powerful variant (maybe an example) ? 2. They are both interpreted languages, and I can't really be sure how they measure in speed. In your opinion, for handling large files, which is better ? (I'm processing files of numerical data of several hundred mb - let's say 200mb - how would python handle file of such size ? As compared to perl ?) 3. This last one is somewhat subjective, but what do you think, in the future, which will be more useful. Which, in your (humble) opinion has a future ? Thank you for all the info you can spare, and expecially grateful for the time in doing so. -- Ivan p.s. Having some trouble posting this. If you see it come out several times, please ignore the copies. 1. They are so similar that, unless they added a lot in Perl lately, both are about equally powerful. I think the difference had mainly to do with how certain multiline constructs were handled. But that doesn't make one more powerful than the other, it just means that you have to make minor changes to port regexes from one to the other. 2. Speed is pretty similar too, Perl probably has a more optimized regex engine because regexes are usually used a more in Perl programs. Python probably has more optimized function calls, string methods, etc. So it probably depends on the mix of functionality you are using. 3. Both languages have been around for a while and too much infrastructure depends on them that they are not going to go away. Perl is extensively used for sysadmin tasks, nagios, etc. and Python is used in Gentoo, Redhat distributions. Of course COBOL is not dying quickly either for the same reason which isn't the same than wanting to program in it. On this list you will find quite a few that have switched from Perl to Python because we like the later better, but there are many programmers that are quite happy with Perl. I myself want to play around with Perl 6 at some point just for fun. Since a lot of the reasons to choose one or another seem to be mostly syntax, I would recommend you to write a couple of short programs in both of them and see what you like more and use that. -- http://mail.python.org/mailman/listinfo/python-list
Re: starting a Python 2.5 executable without the DOS window
On Feb 27, 9:12 am, Greg Miller et1ssgmil...@gmail.com wrote: I am working on a program that controls a piece of equipment. The GUI/ control software is written with Python2.5/wxPython. I would like to know if there is a way of starting the GUI without the DOS window having to launch? I would like only the application itself to appear, no DOS window. Thanks for any information. Greg Miller change the extension of your python program from .py to .pyw. Alternatively make sure that your program gets started by pythonw.exe (insted of python.exe). -- http://mail.python.org/mailman/listinfo/python-list
Re: is this pythonic?
On Jan 21, 4:23 pm, Scott David Daniels scott.dani...@acm.org wrote: prueba...@latinmail.com wrote: ... If you have duplicates this will not work. You will have to do something like this instead: o=[] i=0 ln=len(l) while iln: if l[i]['title']=='ti': o.append(l.pop(i)) ln-=1 else: i+=1 Or the following: indices = [i for i,d in enumerate(l) if d['title']=='ti'] for i in reversed(indices): # so del doesn't affect later positions del l[i] --Scott David Daniels scott.dani...@acm.org Cool. How come I didn't think of that! That means I can create an evil one liner now :-). replacecount=len([o.append(l.pop(i)) for i in reversed(xrange(len(l))) if l[i]['title']=='ti']) -- http://mail.python.org/mailman/listinfo/python-list
Re: is this pythonic?
On Jan 21, 4:23 pm, Scott David Daniels scott.dani...@acm.org wrote: prueba...@latinmail.com wrote: ... If you have duplicates this will not work. You will have to do something like this instead: o=[] i=0 ln=len(l) while iln: if l[i]['title']=='ti': o.append(l.pop(i)) ln-=1 else: i+=1 Or the following: indices = [i for i,d in enumerate(l) if d['title']=='ti'] for i in reversed(indices): # so del doesn't affect later positions del l[i] --Scott David Daniels scott.dani...@acm.org Cool. How come I didn't think of that!. Now I can write my evil one liner :-). o=[l.pop(i) for i in reversed(xrange(len(l))) if l[i]['title']=='ti'] -- http://mail.python.org/mailman/listinfo/python-list
Re: is this pythonic?
On Jan 21, 12:34 pm, TP tribulati...@paralleles.invalid wrote:
alex23 wrote:
Try not to use 'dict' or the name of any of the other built-in types
So my list is rather:
l=[{title:to, color:blue, value:2}
{title:ti, color:red, value:coucou}]
So, I will rather use your solution:
for index, record in enumerate(l):
if record['title'] == 'ti':
to_add_in_another_list = l.pop(index)
another_list.append(to_add_in_another_list )
If you have duplicates this will not work. You will have to do
something like this instead:
o=[]
i=0
ln=len(l)
while iln:
if l[i]['title']=='ti':
o.append(l.pop(i))
ln-=1
else:
i+=1
If you don't have duplicates you can extract the one and exit early:
for index, record in enumerate(l):
if record['title'] == 'ti':
to_add_in_another_list = l.pop(index)
break
I don't know if these are more pythonic, they should be more efficient
for longer lists though.
--
http://mail.python.org/mailman/listinfo/python-list
Re: optimizing large dictionaries
On Jan 15, 4:39 pm, Per Freem perfr...@yahoo.com wrote: hello i have an optimization questions about python. i am iterating through a file and counting the number of repeated elements. the file has on the order of tens of millions elements... i create a dictionary that maps elements of the file that i want to count to their number of occurs. so i iterate through the file and for each line extract the elements (simple text operation) and see if it has an entry in the dict: for line in file: try: elt = MyClass(line)# extract elt from line... my_dict[elt] += 1 except KeyError: my_dict[elt] = 1 i am using try/except since it is supposedly faster (though i am not sure about this? is this really true in Python 2.5?). the only 'twist' is that my elt is an instance of a class (MyClass) with 3 fields, all numeric. the class is hashable, and so my_dict[elt] works well. the __repr__ and __hash__ methods of my class simply return str() representation of self, while __str__ just makes everything numeric field into a concatenated string: class MyClass def __str__(self): return %s-%s-%s %(self.field1, self.field2, self.field3) def __repr__(self): return str(self) def __hash__(self): return hash(str(self)) is there anything that can be done to speed up this simply code? right now it is taking well over 15 minutes to process, on a 3 Ghz machine with lots of RAM (though this is all taking CPU power, not RAM at this point.) any general advice on how to optimize large dicts would be great too thanks for your help. I am willing to bet a beer that the slow performance has nothing to do with the dict but is either because of MyClass or the read from disk. -- http://mail.python.org/mailman/listinfo/python-list
Re: Encrypted Logging in python
On Jan 9, 8:02 am, koranth...@gmail.com wrote: Also, what I am asking is a generic option in logging - which can help the adoption of the logging framework in even closed source systems. It is not just about security - just that a closed source company might be much more comfortable in using the system if crypt is there. Python is an open source project. Many people that read this list don't like closed source code too much and are not willing to invest time to work in features like this. You might get lucky and somebody that is interested in the topic might give you some tips. -- http://mail.python.org/mailman/listinfo/python-list
Re: Implementing file reading in C/Python
On Jan 9, 8:48 am, Johannes Bauer dfnsonfsdu...@gmx.de wrote: No - and I've not known there was a profiler yet have found anything meaningful (there seems to be an profiling C interface, but that won't get me anywhere). Is that a seperate tool or something? Could you provide a link? Thanks, Kind regards, Johannes It is part of the python standard library: http://docs.python.org/library/profile.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Any news on when some libraries will be ported to Python 3.0?
On Jan 7, 3:48 am, Just Another Victim of the Ambient Morality ihates...@hotmail.com wrote: Anyway, I'd love to hear some news about any of these things in particular or even anything in general. Am I the only one who's psyched for this version of Python? Thank you... There are many people psyched about 3.0, but also many older farts that have been using Python since 1.0 and have tons of code that already works fine and feel a little bit like the mechanic that has to switch his tools from imperial to metric; it is overall probably for the better but still a pain to adapt everything to the new system. This is open source so you either have to find a way to motivate the authors of the libraries without pissing them off, or you will have to be patient. -- http://mail.python.org/mailman/listinfo/python-list
Re: f.seek() unwanted output
On Jan 5, 8:52 am, thomasvang...@gmail.com thomasvang...@gmail.com wrote: I'm having trouble with a script that is printing the output of f.seek () whereas in the documentation it is quoted not to have any output: file.seek(offset[, whence])¶ Set the file’s current position, like stdio‘s fseek. The whence argument is optional and defaults to os.SEEK_SET or 0 (absolute file positioning); other values are os.SEEK_CUR or 1 (seek relative to the current position) and os.SEEK_END or 2 (seek relative to the file’s end). There is no return value. -- I have a file in memory. when i try f.seek(0) #or any other value in f.tell() it gives me 0 as output: the following script illustrates my 'problem' for a in range(10): f.seek(a) 0 1 2 3 4 5 6 7 8 9 I don't want python to produce output when setting the file pointer. Any help woul be appreciated. Kind regards, Thomas You can also avoid the output by assigning the output to something: for a in range(10): dummy=f.seek(a) -- http://mail.python.org/mailman/listinfo/python-list
Re: SQL, lite lite lite
On Dec 29, 1:06 pm, Aaron Brady castiro...@gmail.com wrote: Hi all, About a year ago, I posted an idea I was having about thread synchronization to the newsgroup. However, I did not explain it well, and I really erred on the side of brevity. (After some finagling, Mr. Bieber and I decided it wasn't exactly anything groundbreaking.) But I think the brevity cost me some readers, who might have had more interest. The affair was on the whole discouraging. So, I'm going to try another idea, and assume that readers have some time, and will spend it on it. I don't think relational data can be read and written very easily in Python. There are some options, such as 'sqllite3', but they are not easy. 'sqllite3' statements are valid SQL expressions, which afford the entire power of SQL, but contrary to its name, it is not that 'lite'. To me, 'lite' is something you could learn (even make!) in an afternoon, not a semester; something the size of an ActiveState recipe, or a little bigger, maybe a file or two. If you think SQL is a breeze, you probably won't find my idea exciting. I assume that the basics of SQL are creating tables, selecting records, and updating records. My idea is to create a 'Relation' class. The details are basically open, such as whether to back it with 'sqllite3', 'shelve', 'mmap', or just mapping and sequence objects; what the simplest syntax is that can capture and permit all the basics, and how much and what else can fit in at that level; how and whether it can include arbitrary Python objects, and what constraints there are on them if not; how and whether to permit transactions; and what the simplest and coolest thing you can do with a little Python syntax is. This is basically an invitation for everyone to brainstorm. (No hijackings, good humor digression ok.) Lastly, ... **warning, spoiler! here's what I thought of already.** **repeat! spoiler! here's what I thought of already.** #Just the select and update syntax: a= people._select( firstname== 'Joe' ) #select 'key' from 'people' where 'firstname'== 'joe' a [Entry2864, Entry3076, Entry3172] entry1= a[ 0 ] entry1.phone #select 'phone' from 'people' where 'key'==self.key 555-2413 entry1.phone= 555-1234 #update 'people' set 'phone'= '555-1234' where 'key'==self.key entry1.phone 555-1234 #Create table syntax (a-whole-nother beast in itself): classes= db.Relation( 'class_', 'person', Unique( 'class_', 'person' ) ) #create table 'classes' ( 'key', 'class_', 'person' ) unique ( 'class_', 'person' ) classes._unique( 'class_', 'person' ) classes.class_.noneok= False #'class_' cannot be null classes.person.noneok= False classes._insert( 'Physics', 'Dan' ) classes._insert( 'Chem', 'Tim' ) Hoping-good critic-is-self-consistent-ly, hoping-to-hear-it's-too- complicated-already-ly, A. Brady You really do like to reinvent the wheels do you? :-) Nothing wrong with that. Just be aware that most people that really need what you are proposing are probably already using mature feature rich libraries for that. http://wiki.python.org/moin/HigherLevelDatabaseProgramming -- http://mail.python.org/mailman/listinfo/python-list
Re: 2to3 used in the Shootout
On Dec 23, 5:21 pm, Isaac Gouy igo...@yahoo.com wrote:
On Dec 23, 11:51 am, bearophileh...@lycos.com wrote:
They have translated the Python benchmarks of the Shootout site from
Py2 to Py3 using 2to3:
http://shootout.alioth.debian.org/u32/benchmark.php?test=all〈=pyt...
So please re-write those programs to remove problems created by
automatic translation and better take advantage of Python 3
functionality...
http://shootout.alioth.debian.org/u32/faq.php#play
It shows some performance bugs of Python3 itself (especially
regarding the binary-trees benchmark, that was unexpected by me), and
two points where 2to3 may be improved, for example after the
translation this gives error:
table=string.maketrans('ACBDGHK\nMNSRUTWVYacbdghkmnsrutwvy',
'TGVHCDM
\nKNSYAAWBRTGVHCDMKNSYAAWBR')):
Gives:
TypeError: maketrans arguments must be bytes objects
Bye,
bearophile
BTW I am not sure how to submit this or if this is actually valid to
do, but I have a faster version for the pidigits program that uses
basically the same algorithm but removes function calls and unused
terms of the formula.
pre
import time
def pi_digits(n, width):
out = []
wrt = out.append
aq = 1
ar = 0
at = 1
k = 0
f = 1
g = 2
i = 0
while i n:
y = (aq*3+ar)//at
while y != ((aq*4+ar)//at):
k += 1
f += 2
g += 4
ar = aq*g+ar*f
aq = aq*k
at = at*f
y = (aq*3+ar)//at
aq = 10*aq
ar = 10*ar-10*y*at
i += 1
wrt(str(y))
if not i%width:
wrt('\t:%d\n'%i)
wrt(' '*(width-i%width))
wrt('\t:%d\n'%i)
return ''.join(out)
def main():
begin = time.time()
n = 1000
width = 70
print pi_digits(n,width)
print 'Total Time:', time.time()-begin
main()
/pre
--
http://mail.python.org/mailman/listinfo/python-list
Re: Is this pythonic?
On Dec 18, 11:08 am, ipyt...@gmail.com wrote: x.validate_output(x.find_text(x.match_filename (x.determine_filename_pattern(datetime.datetime.now() Is it even good programming form? Lisp and Scheme programmers love that style. You can tell by the number of parentheses :-). In Python people usually use an intermediate variable to break things up a bit but the amount of acceptable nesting is a matter of personal style. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why no lexical scoping for a method within a class?
On Dec 17, 10:19 am, walterbyrd walterb...@iname.com wrote: For a language as well structured as Python, this seems somewhat sloppy, and inconsistant. Or is there some good reason for this? Here is what I mean: def a(): x = 99 print x def b(): print x a() b() # raises an exception because x is not defined. However in the methods are within a class, the scoping seems to work differently. class ab(): def a(self): self.x = 99 print self.x def b(self): print self.x i = ab() i.a() i.b() # this works, why no lexical scoping? If scoping worked as you want, how, pray tell, would you define object attributes? -- http://mail.python.org/mailman/listinfo/python-list
Re: stable algorithm with complexity O(n)
Non-comparison sorts are a useful technique, but it's changing the problem, and they are only useful in very limited circumstances. There's a good reason that most sort routines are based on O(n*log n) comparison sorts instead of O(n) bucket sorts or radix sorts. This is an assumption that I never quite understood. What most people want is to have sorted data, they don't care if I used a sorting or non-sorting comparison to do it. I think it is just that in most cases n is not very big anyway and comparison sorts make it easier on the programmer to create arbitrary types that are sortable. -- http://mail.python.org/mailman/listinfo/python-list
Re: stable algorithm with complexity O(n)
On Dec 15, 11:05 am, prueba...@latinmail.com wrote: Non-comparison sorts are a useful technique, but it's changing the problem, and they are only useful in very limited circumstances. There's a good reason that most sort routines are based on O(n*log n) comparison sorts instead of O(n) bucket sorts or radix sorts. This is an assumption that I never quite understood. What most people want is to have sorted data, they don't care if I used a sorting or non-sorting comparison to do it. I think it is just that in most cases n is not very big anyway and comparison sorts make it easier on the programmer to create arbitrary types that are sortable. I meant they don't care if I use a comparison or non-comparison sort of course. -- http://mail.python.org/mailman/listinfo/python-list
Re: Removing None objects from a sequence
On Dec 12, 8:08 am, Filip Gruszczyński grusz...@gmail.com wrote: I am not doing it, because I need it. I can as well use if not elem is None, but I just wanted to know, if there is some better way of doing this. I like to know :-) And I can't understand why you are becoming so aggressive because of this. Just because I asked for that, doesn't mean, that I will put some obfuscated code into my project. I just wanted to learn something new - I checked itertools, I googled a bit, now I wanted to ask here if someone knew some really cool way of this. All the other assumptions weren't really necessary. Thanks for those ideas, however. I like the last one a lot :) -- Filip Gruszczyński In this case the cool way is the straightforward way: either the list versions: [x fox x in seq if x is not None] filter(lambda x: x is not None, seq) or the generator versions: (x for x in seq if x is not None) itertools.ifilter(lambda x: x is not None, seq) -- http://mail.python.org/mailman/listinfo/python-list
Re: StringIO in 2.6 and beyond
On Dec 10, 6:58 am, Bill McClain
[EMAIL PROTECTED] wrote:
On 2008-12-10, ajaksu [EMAIL PROTECTED] wrote:
On Dec 9, 5:24 pm, Bill McClain [EMAIL PROTECTED]
wrote:
On 2008-12-09, MRAB [EMAIL PROTECTED] wrote:
In Python 2.x unmarked string literals are bytestrings. In Python 3.x
they're Unicode. The intention is to make the transition from 2.x to 3.x
easier by adding some features of 3.x to 2.x, but without breaking
backwards compatibility (not entirely successfully!).
It is a bit ugly. In 2.6 StringIO won't take bytestrings, so I apply
u'x'. But
in 3.0 u'x' will be gone and I'll have to change the code again.
Try:
from __future__ import unicode_literals
That works for:
output.write('First line.\n')
...but not for:
print('Second line.', file=output)
Maybe a combination of this and functools.partial as was suggested before. At
least the necessary edits would be at the top of the program.
-Bill
--
Sattre Press Tales of
Warhttp://sattre-press.com/ by Lord Dunsany
[EMAIL PROTECTED] http://sattre-press.com/tow.html
I think this combination might do the trick (I don't have 2.6 to test
it right now):
from __future__ import print_function
from __future__ import unicode_literals
from functools import partial
import io
print = partial(print, sep= , end=\n)
out = io.StringIO()
print(hello, file=out)
What puzzles me is the documentation in 2.6 and 3.0:
In 2.6 it says: The StringIO object can accept either Unicode or 8-
bit strings. Why does it fail with old str objects then?
Why is there no documentation for StringIO in 3.0?
--
http://mail.python.org/mailman/listinfo/python-list
Re: StringIO in 2.6 and beyond
On Dec 10, 10:06 am, [EMAIL PROTECTED] wrote:
On Dec 10, 6:58 am, Bill McClain
[EMAIL PROTECTED] wrote:
On 2008-12-10, ajaksu [EMAIL PROTECTED] wrote:
On Dec 9, 5:24 pm, Bill McClain [EMAIL PROTECTED]
wrote:
On 2008-12-09, MRAB [EMAIL PROTECTED] wrote:
In Python 2.x unmarked string literals are bytestrings. In Python 3.x
they're Unicode. The intention is to make the transition from 2.x to
3.x
easier by adding some features of 3.x to 2.x, but without breaking
backwards compatibility (not entirely successfully!).
It is a bit ugly. In 2.6 StringIO won't take bytestrings, so I apply
u'x'. But
in 3.0 u'x' will be gone and I'll have to change the code again.
Try:
from __future__ import unicode_literals
That works for:
output.write('First line.\n')
...but not for:
print('Second line.', file=output)
Maybe a combination of this and functools.partial as was suggested before.
At
least the necessary edits would be at the top of the program.
-Bill
--
Sattre Press Tales of
Warhttp://sattre-press.com/ by Lord Dunsany
[EMAIL PROTECTED] http://sattre-press.com/tow.html
I think this combination might do the trick (I don't have 2.6 to test
it right now):
from __future__ import print_function
from __future__ import unicode_literals
from functools import partial
import io
print = partial(print, sep= , end=\n)
out = io.StringIO()
print(hello, file=out)
What puzzles me is the documentation in 2.6 and 3.0:
In 2.6 it says: The StringIO object can accept either Unicode or 8-
bit strings. Why does it fail with old str objects then?
Why is there no documentation for StringIO in 3.0?
OK I found StringIO it is called io.StringIO now.
--
http://mail.python.org/mailman/listinfo/python-list
Re: StringIO in 2.6 and beyond
On Dec 9, 11:28 am, Bill McClain
[EMAIL PROTECTED] wrote:
On 2008-12-08, Bill McClain [EMAIL PROTECTED] wrote:
On 2008-12-08, Christian Heimes [EMAIL PROTECTED] wrote:
In this context 'str' means Python 3.0's str type, which is unicode in
2.x. Please report the misleading error message.
So this is an encoding problem? Can you give me a hint on how to correct in
my
example? I see that io.StringIO() has an encoding parameter, but I'm unclear
what to specify.
I still don't have this working. I've been specifying encodings without
success.
The StringIO example usage in the Python 3.0 documentation here:
http://docs.python.org/3.0/library/io.html#io.StringIO
gives me the same error on 2.6:
#! /usr/bin/env python
from __future__ import print_function
import io
output = io.StringIO()
output.write('First line.\n')
print('Second line.', file=output)
# Retrieve file contents -- this will be
# 'First line.\nSecond line.\n'
contents = output.getvalue()
# Close object and discard memory buffer --
# .getvalue() will now raise an exception.
output.close()
./stringio30.py
Traceback (most recent call last):
File ./stringio30.py, line 7, in module
output.write('First line.\n')
File /usr/local/lib/python2.6/io.py, line 1487, in write
s.__class__.__name__)
TypeError: can't write str to text stream
-Bill
--
Sattre Press History of
Astronomyhttp://sattre-press.com/ During the 19th Century
[EMAIL PROTECTED] by Agnes M. Clerke
http://sattre-press.com/han.html
This puzzles me too. According to the documentation StringIO accepts
both byte strings and unicode strings. Try to replace
output.write('First line.\n')
with
output.write(unicode('First line.\n'))
or
output.write(str('First line.\n'))
and see if one of those works.
--
http://mail.python.org/mailman/listinfo/python-list
Re: When (and why) to use del?
On Dec 9, 11:35 am, Albert Hopkins [EMAIL PROTECTED] wrote:
I'm looking at a person's code and I see a lot of stuff like this:
def myfunction():
# do some stuff stuff
my_string = function_that_returns_string()
# do some stuff with my_string
del my_string
# do some other stuff
return
and also
def otherfunction():
try:
# some stuff
except SomeException, e:
# more stuff
del e
return
I think this looks ugly, but also does it not hurt performance by
preempting the gc? My feeling is that this is a misuse of 'del'. Am I
wrong? Is there any advantage of doing the above?
If this is CPython (the standard Python) the extra del(s) in the
examples you gave are just garbage and good for nothing. The reason
being that they are inside a function; as soon as the program exits
the function the variable gets deleted automatically. The only reason
to use del would be a global variable in long running program that
uses a lot of space, but global variables should be avoided for other
reasons anyway. I never ever used del in my programs only on the
interactive interpreter prompt after doing foolish things like x=open
('1GBfile').read()
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python 3 For Python 2 Users
On Dec 9, 11:58 am, Tim Daneliuk [EMAIL PROTECTED] wrote: I code in Python 2.x intermittently and have only casually watched the 3.x development discussions. Now it's time to get up to speed. Has someone written a tutorial for people in my situation. Yes, I've looked at the release notes, but I'm looking for something that motivates all the major changes to give me a better sense of the Gestalt of the new language. TIA, -- Tim Daneliuk [EMAIL PROTECTED] PGP Key: http://www.tundraware.com/PGP/ For starters this should work: http://docs.python.org/dev/3.0/whatsnew/3.0.html -- http://mail.python.org/mailman/listinfo/python-list
Re: funny generator behaviour
On Dec 4, 8:00 am, Edvin Fuglebakk [EMAIL PROTECTED] wrote: I have written a generator that puzzles me: The generator is supposed to create ordered selections of a set of objects. repetition of objects is allowed and the selections should be of a size determined by a pramter to the generator. Now, if I try to accummulate the generated selections into a list I get some peculiar behaviour that I hope maybe some of you can help me understand: Help much appreciated -Edvin #straightforward acumulation. Does not give the expected result d=[] for f in orderedCombinations([1,2],3): ... d.append(f) ... d [[1], [2], [1], [2], [1], [2], [1], [2]] #accumulating shallow copies of the genereated combinations works: d=[] for f in orderedCombinations([1,2],3): ... d.append(f[:]) ... d [[1, 1, 1], [1, 1, 2], [1, 2, 1], [1, 2, 2], [2, 1, 1], [2, 1, 2], [2, 2, 1], [2, 2, 2]] #The generator: def orderedCombinations(pool, k): Generator yielding ordered selections of size k with repetition from pool. if k == 1: for m in pool: yield [m] if k 1: for m in pool: for combo in orderedCombinations(pool, k-1): #insert and pop to avoid copying entire list combo.insert(0,m) yield combo combo.pop(0) Functional style (like the recursion you are using) and mutable datastructures are hard to reason about. This version works for me: def orderedCombinations(pool, k): Generator yielding ordered selections of size k with repetition from pool. if k == 1: for m in pool: yield [m] if k 1: for m in pool: for combo in orderedCombinations(pool, k-1): yield [m]+combo -- http://mail.python.org/mailman/listinfo/python-list
Re: funny generator behaviour
On Dec 4, 8:00 am, Edvin Fuglebakk [EMAIL PROTECTED] wrote: I have written a generator that puzzles me: The generator is supposed to create ordered selections of a set of objects. repetition of objects is allowed and the selections should be of a size determined by a pramter to the generator. Now, if I try to accummulate the generated selections into a list I get some peculiar behaviour that I hope maybe some of you can help me understand: Help much appreciated -Edvin #straightforward acumulation. Does not give the expected result d=[] for f in orderedCombinations([1,2],3): ... d.append(f) ... d [[1], [2], [1], [2], [1], [2], [1], [2]] #accumulating shallow copies of the genereated combinations works: d=[] for f in orderedCombinations([1,2],3): ... d.append(f[:]) ... d [[1, 1, 1], [1, 1, 2], [1, 2, 1], [1, 2, 2], [2, 1, 1], [2, 1, 2], [2, 2, 1], [2, 2, 2]] #The generator: def orderedCombinations(pool, k): Generator yielding ordered selections of size k with repetition from pool. if k == 1: for m in pool: yield [m] if k 1: for m in pool: for combo in orderedCombinations(pool, k-1): #insert and pop to avoid copying entire list combo.insert(0,m) yield combo combo.pop(0) BTW if you search in the fine manual version 2.6 you will find that a slick implementation is already part of the standard library (hint: itertools): def orderedCombinations(pool, k): result = [[]] for poolel in [pool] * k: result = [x+[y] for x in result for y in poolel] return result -- http://mail.python.org/mailman/listinfo/python-list
Re: funny generator behaviour
On Dec 4, 8:00 am, Edvin Fuglebakk [EMAIL PROTECTED] wrote: I have written a generator that puzzles me: The generator is supposed to create ordered selections of a set of objects. repetition of objects is allowed and the selections should be of a size determined by a pramter to the generator. Now, if I try to accummulate the generated selections into a list I get some peculiar behaviour that I hope maybe some of you can help me understand: Help much appreciated -Edvin #straightforward acumulation. Does not give the expected result d=[] for f in orderedCombinations([1,2],3): ... d.append(f) ... d [[1], [2], [1], [2], [1], [2], [1], [2]] #accumulating shallow copies of the genereated combinations works: d=[] for f in orderedCombinations([1,2],3): ... d.append(f[:]) ... d [[1, 1, 1], [1, 1, 2], [1, 2, 1], [1, 2, 2], [2, 1, 1], [2, 1, 2], [2, 2, 1], [2, 2, 2]] #The generator: def orderedCombinations(pool, k): Generator yielding ordered selections of size k with repetition from pool. if k == 1: for m in pool: yield [m] if k 1: for m in pool: for combo in orderedCombinations(pool, k-1): #insert and pop to avoid copying entire list combo.insert(0,m) yield combo combo.pop(0) def orderedCombinations(pool, k): res=[[]] for i in xrange(k): res=[n+[m] for n in res for m in pool] return res -- http://mail.python.org/mailman/listinfo/python-list
Re: Good introductory book?
On Dec 3, 9:44 am, Ken D'Ambrosio [EMAIL PROTECTED] wrote: Hi, all. I'm getting ready to do some projects in Python, and I've cut my teeth a little bit, but I've found the Learning|Programming Python books from O'Reilly to be more-or-less useless (to my surprise -- I'm usually an O'Reilly fan). I really, really like Python Essential Reference, but it's -- well, more of a reference than an intro. So, an introductory text that actually assumes some previous programming experience (as opposed to Learning Python which must be the most slowly-paced programming book ever) would be terrific. Thanks for your suggestions! -Ken I had the same experience as you had and almost gave up on Python. I had programmed in other languages before and found the Programming Python book very tedious to read. I just wanted to learn the syntax and library to start writing my own stuff. Luckily, I found Guido's 12 page tutorial (http://www.python.org/doc/2.5.2/tut/tut.html) and that plus the online library reference and Google was everything I ever needed since then. That said I looked at O'Reilly's Python in a Nutshell and I thought it was really good. It is a mix between introduction, language and library reference. I should have gotten that instead of Learning/Programming when I started and I probably would have been much happier. -- http://mail.python.org/mailman/listinfo/python-list
Re: Dynamic features used
On Nov 21, 4:17 am, [EMAIL PROTECTED] wrote: What are the dynamic features of Python that you use in your code? The main ones is using configuration files that are plain Python instead of XML and not having to wait 5 minutes to compile larger programs. I also prefer structural typing over nominative typing and duck typing is closer to structural than nominative typing with the difference that compilation doesn’t take forever but the disadvantage that the checking doesn’t happen until runtime. Hopefully they will keep improving pylint to generate warnings for all those. As far as speed goes, there might be 5% of my code that I would be willing to run through a slow compiler to speed it up. For the rest it is probably not worth it. -- http://mail.python.org/mailman/listinfo/python-list
Re: redirecting stdout/err to mysql table
On Nov 18, 2:07 pm, n00b [EMAIL PROTECTED] wrote:
greetings,
i need to log to the db directly and wrote a little script to do so.
since i'm pretty new to python,
i was wondering if a) you could review the enclosed code and b)
provide suggestions to harden to code to turn it into a more general,
broadly reusable class.
thank you very much.
import sys
import MySQLdb
class DBLogger(object):
def __init__(self):
self.db_name = 'xxx'
self.db_host = '127.0.0.1'
self.db_table = 'yyy'
self.db_uname = 'root'
self.db_passwd = ''
self.db_port = 3306
self.db = None
self.cur = None
self.sql = 'INSERT INTO %s' %self.db_table + ' VALUES(null, NOW
(), %s)'
def openDb(self):
try:
self.db = MySQLdb.connect(host = self.db_host,
user = self.db_uname,
passwd = self.db_passwd,
db = self.db_name,
)
self.cur = self.db.cursor()
return self.db, self.cur
except Exception, e:
sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__
print e[0], e[1]
sys.exit(1)
def closeDb(self):
self.cur.close()
self.db.close()
def write(self, string):
s = string.strip('\n')
if not s=='':
self.openDb()
self.cur.execute(self.sql, (s))
self.db.commit()
self.closeDb()
dbl = DBLogger()
sys.stdout = dbl
sys.stderr = dbl
#a = 'test string'
#dbl.write(a)
print 'a b c '
sys.stdout = sys.__stdout__
sys.stderr = sys.__stderr__
thanks again for your time
Looks good to me.
--
http://mail.python.org/mailman/listinfo/python-list
Re: duck-type-checking?
On Nov 14, 12:47 am, George Sakkis [EMAIL PROTECTED] wrote:
On Nov 13, 10:55 pm, Steven D'Aprano [EMAIL PROTECTED]
cybersource.com.au wrote:
Take this example:
def foo(alist):
alist.sort()
alist.append(5)
The argument can be any object with sort and append methods (assumed to
act in place). But what happens if you pass it an object with a sort
method but no append? The exception doesn't occur until *after* the
object is sorted, which leaves it in an inconsistent state. This can be
undesirable: you might need the function foo to be atomic, either the
entire function succeeds, or none of it.
In this example atomicity is not guaranteed even if alist is a builtin
list (if it contains a complex number or other unorderable object),
let alone if not isistance(alist, list). It gets worse: false
positives are less likely for full-spelled methods with well-known
names such as sort and append (i.e. if hasattr(alist, 'append'),
alist.append *probably* does what you think it does), but good luck
with that when testing for __getitem__, __iter__ for more than one
pass, __call__, and other special methods with ambiguous or undefined
semantics.
Let's face it, duck typing is great for small to medium complexity
projects but it doesn't scale without additional support in the form
of ABCs/interfaces, explicit type checking (at compile and/or run
time), design by contract, etc. It must not be a coincidence that both
Zope and Twisted had to come up with interfaces to manage their
massive (for Python at least) complexity.
George
What would be actually interesting would be an switch to the python
interpreter that internally annotated function parameters with how
they are used in the function and raised an exception as soon as the
function is called instead of later. Failing earlier rather than
later. Example:
def sub(x,y):
...run some stuff
...print x[2]
...return y.strip().replace('a','b')
internally python generates:
def sub(x: must have getitem, y: must have strip and replace)
sub([1,2,3,4],5)
Error calling sub(x,y): y has to have strip() method.
--
http://mail.python.org/mailman/listinfo/python-list
Re: duck-type-checking?
On Nov 12, 1:22 pm, Joe Strout [EMAIL PROTECTED] wrote: On Nov 12, 2008, at 10:45 AM, Tim Rowe wrote: What do you actually mean by Quacks like a string? Supports the 'count()' method? Then you find out if it doesn't when you try to apply the 'count()' method. Supports some method that you don't actually use? Then why do you care? Because if I write a method with the intention of treating the arguments like strings in various ways (slicing, combining with other strings, printing to stdout or writing to a file, etc. etc.), and some idiot (i.e. me six weeks later or long after I should have gone to bed) manages to accidentally pass in something else, then I want my program to blow up right away, not plant a roadside bomb and cheerfully wait for me to drive by. This is not hypothetical -- just last week I had a hard-to-track-down abend that ultimately turned out to be an NLTK.Tree object stored someplace that I expected to only contain strings. I found it by littering my code with assertions of the form isinstance(foo,basestring). If I'd had those in there in the first place, not only documenting my assumptions but letting the computer check them for me, it would have saved me a lot of grief. But in the spirit of duck-typing, I shouldn't actually check that foo is a basestring. I should instead check that foo quacks like a basestring. I'd define that is: x quacks like a basestring if it implements all the public methods of basestring, and can be used in pretty much any context that a basestring can. I have to say pretty much since obviously there may be some evil context that actually checks isinstance. But that's the pathological case, and we shouldn't let it prevent us from neatly handling the typical case. The point about duck typing is that something might quack like a duck but not walk like a duck -- one of those duck calls that hunters use, for instance. Quacking like a duck doesn't actually mean it /is/ a duck, it means that it will do instead of a duck if the quack is all you want. Well, that's one point, but it's not the only point. If I have code that expects to be working with strings, and I want to purposely give it something else, then it's reasonable to expect that the something- else will act like a string in every way that a string is likely to be exercised. My string wrapper or doppleganger_string or whatever should implement all the methods, and support all the operators and type conversions, that basestring does. If you need to know that it walks like a duck, mates like a duck and tastes like a duck when roasted, you probably want it to really /be/ a duck and should go back to inheritance. I can't agree; there are times when inheritance just won't do, for example when you don't have control over the object creation, because they come from some factory method you can't change. In that case you may need to make a wrapper instead of a subclass, but if you've faithfully implemented the interface of the original class, you should be able to use it wherever the original class could be used (within reason). So, since it's pretty clear by now that there's no standard idiom for this, I'll try to cook up something myself. For classes, I think we could do a two-stage test: 1. If the given object isinstance of the specified class, then all is good and return immediately. 2. Otherwise, check each of the public attributes of the specified class, and make sure that the given object has corresponding callable attributes. For case 2, we might be able to cache the result so that we don't do all that work again the next time the same type comparison is done. Anyway, I'll evolve something in our shop here and live with it a while, and in a few months I'll either share what we develop for this purpose, or admit it was a horrible idea all along. :) Cheers, - Joe It seems to me that what you are describing is exactly what abcs were added for in 2.6, in particular registration: class AnotherClass(metaclass=ABCMeta): pass AnotherClass.register(basestring) assert isinstance(str, AnotherClass) Please read this first: http://www.python.org/dev/peps/pep-3119/ and tell us why that would not work. -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible read()/readline() bug?
On Oct 23, 9:48 am, Mike Kent [EMAIL PROTECTED] wrote: To followup on this: Terry: Yes, I did in fact miss the 'buffer' parameter to open. Setting the buffer parameter to 0 did in fact fix the test code that I gave above, but oddly, did not fix my actual production code; it continues to get the data as first read, rather than what is currently on the disk. I'm still investigating why. Carl: I tried the above test code, without 'buffer=0' in the open, but with a flush added before reads in the appropriate places. The flush made no difference; readline continued to return the old data rather than what was actually on the disk. So, flush isn't the answer. I suppose that means that, when the document states it flushes the buffer, it's referring to the output buffer, not the input buffer. Something odd is going on for sure. I had a couple of theories but then I tested it on both Windows XP and AIX and could not reproduce the problem even using the default buffer setting. As soon as I do a seek and read it gives me the new data. I wonder if other people can test this out on different operating systems and file systems and detect a pattern. -- http://mail.python.org/mailman/listinfo/python-list
Re: Possible read()/readline() bug?
On Oct 22, 2:54 pm, Mike Kent [EMAIL PROTECTED] wrote:
Before I file a bug report against Python 2.5.2, I want to run this by
the newsgroup to make sure I'm not being stupid.
I have a text file of fixed-length records I want to read in random
order. That file is being changed in real-time by another process,
and my process want to see the changes to the file. What I'm seeing
is that, once I've opened the file and read a record, all subsequent
seeks to and reads of that same record will return the same data as
the first read of the record, so long as I don't close and reopen the
file. This indicates some sort of buffering and caching is going on.
Consider the following:
$ echo hi foo.txt # Create my test file
$ python2.5 # Run Python
Python 2.5.2 (r252:60911, Sep 22 2008, 16:13:07)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-9)] on linux2
Type help, copyright, credits or license for more information.
f = open('foo.txt') # Open my test file
f.seek(0)# Seek to the beginning of the file
f.readline() # Read the line, I get the data I expected
'hi\n'
# At this point, in another shell I execute 'echo bye foo.txt'.
'foo.txt' now has been changed
# on the disk, and now contains 'bye\n'.
f.seek(0)# Seek to the beginning of the still-open file
f.readline() # Read the line, I don't get 'bye\n', I get the
original data, which is no longer there.
'hi\n'
f.close() # Now I close the file...
f = open('foo.txt') # ... and reopen it
f.seek(0) # Seek to the beginning of the file
f.readline()# Read the line, I get the expected 'bye\n'
'bye\n'
It seems pretty clear to me that this is wrong. If there is any
caching going on, it should clearly be discarded if I do a seek. Note
that it's not just readline() that's returning me the wrong, cached
data, as I've also tried this with read(), and I get the same
results. It's not acceptable that I have to close and reopen the file
before every read when I'm doing random record access.
So, is this a bug, or am I being stupid?
This has to do how the OS file-system operates. This is equivalent of
doing:
echo hi foo.txt
vi foo.txt
in another session type: echo bye foo.txt
the text in the vi session doesn't change.
you can even type 'rm foo.txt' and vi will still have the text there.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Possible read()/readline() bug?
On Oct 22, 3:44 pm, [EMAIL PROTECTED] wrote:
On Oct 22, 2:54 pm, Mike Kent [EMAIL PROTECTED] wrote:
Before I file a bug report against Python 2.5.2, I want to run this by
the newsgroup to make sure I'm not being stupid.
I have a text file of fixed-length records I want to read in random
order. That file is being changed in real-time by another process,
and my process want to see the changes to the file. What I'm seeing
is that, once I've opened the file and read a record, all subsequent
seeks to and reads of that same record will return the same data as
the first read of the record, so long as I don't close and reopen the
file. This indicates some sort of buffering and caching is going on.
Consider the following:
$ echo hi foo.txt # Create my test file
$ python2.5 # Run Python
Python 2.5.2 (r252:60911, Sep 22 2008, 16:13:07)
[GCC 3.4.6 20060404 (Red Hat 3.4.6-9)] on linux2
Type help, copyright, credits or license for more information.
f = open('foo.txt') # Open my test file
f.seek(0)# Seek to the beginning of the file
f.readline() # Read the line, I get the data I expected
'hi\n'
# At this point, in another shell I execute 'echo bye foo.txt'.
'foo.txt' now has been changed
# on the disk, and now contains 'bye\n'.
f.seek(0)# Seek to the beginning of the still-open file
f.readline() # Read the line, I don't get 'bye\n', I get
the original data, which is no longer there.
'hi\n'
f.close() # Now I close the file...
f = open('foo.txt') # ... and reopen it
f.seek(0) # Seek to the beginning of the file
f.readline()# Read the line, I get the expected 'bye\n'
'bye\n'
It seems pretty clear to me that this is wrong. If there is any
caching going on, it should clearly be discarded if I do a seek. Note
that it's not just readline() that's returning me the wrong, cached
data, as I've also tried this with read(), and I get the same
results. It's not acceptable that I have to close and reopen the file
before every read when I'm doing random record access.
So, is this a bug, or am I being stupid?
This has to do how the OS file-system operates. This is equivalent of
doing:
echo hi foo.txt
vi foo.txt
in another session type: echo bye foo.txt
the text in the vi session doesn't change.
you can even type 'rm foo.txt' and vi will still have the text there.
Actually disregard what I said. vi loads everything in memory. You
might want to try:
f = open('foo.txt','r',0)
and see if that fixes your problem.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Where/how to propose an addition to a standard module?
On Oct 10, 3:10 pm, Joe Strout [EMAIL PROTECTED] wrote: I would like to propose a new method for the string.Template class. What's the proper procedure for doing this? I've joined the python- ideas list, but that seems to be only for proposed language changes, and my idea doesn't require any change to the language at all. From http://www.python.org/dev/peps/pep-0001/, it sounds like the PEP process is appropriate here, though other PEPs (like http://www.python.org/dev/peps/pep-3001/ make it sound as though these are meant for proposals for Python 3000, which is not necessarily my intent. Here's a brief sketch of my proposal, in case it helps: Add a match function to string.Template, which takes a text string as a parameter. If this text string can be matched to the template, by substituting some portion of the given string for each field of the template, then .match returns a dictionary, where each key is a field name and the value is the corresponding text from the input. If the text string cannot be matched to the template, then .match returns None. I understand that if I'm to write a PEP, I'll need to flesh this out considerably as per PEP 0001. But that document also suggests first discussing it here. I'm still a newbie (or actually, oldbie-turned- nonbie-turned-newbie-again), so I could use some advice. What's the next step in advocating for this idea? Thanks, - Joe Whenever I needed such functionality I used the re module. The benefit is that it uses unix style regular expression syntax and an egrep/awk/ perl/ruby user can understand it. You should show a few examples where your proposal looks better than just using RE. -- http://mail.python.org/mailman/listinfo/python-list
Re: Append a new value to dict
On Oct 13, 9:41 am, Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote:
On Mon, 13 Oct 2008 14:10:43 +0200, Mathias Frey wrote:
However incrementing a non-existing key throws an exception. So you
either have to use a workaround:
try:
... counter['B'] += 1
... except KeyError:
... counter['B'] = 1
Since this looks ugly somebody invented the setdefault method:
counter['B'] = counter.setdefault('B',0) + 1
Nope, for this use case there is the `dict.get()` method:
counter['B'] = counter.get('B', 0) + 1
This assigns only *once* to ``counter['B']`` in every case.
`dict.setdefault()` is for situations where you really want to actually
put the initial value into the dictionary, like with the list example by
the OP.
Ciao,
Marc 'BlackJack' Rintsch
...and if you are using Python 2.5 or later you can use the
collections module with collections.defaultdict(list) or
collections.defaultdict(int) to do the same thing. I personally find
it easier to read.
--
http://mail.python.org/mailman/listinfo/python-list
Re: python 3: sorting with a comparison function
On Oct 10, 8:35 am, Kay Schluehr [EMAIL PROTECTED] wrote: On 9 Okt., 22:36, [EMAIL PROTECTED] wrote: Yes, that's a wonderful thing, because from the code I see around 99.9% of people see the cmp and just use it, totally ignoring the presence of the 'key' argument, that allows better and shorter solutions of the sorting problem. Me too because I don't get this: key specifies a function of one argument that is used to extract a comparison key from each list element: key=str.lower. The default value is None. Kay Don't know if further explanation is needed, but here is the deal: cmp is a function that receives two values and you return -1, 0 or 1 depending if the first is smaller, equal or bigger. 99% of the time you will do some operation on the values that come in and then do a if statement with or and return -1,0,1. key is a function that receives one value and you return the value that you would normally compare against. Let me show an example: data=[(4,'v'),(2,'x'),(1,'a')] sorted(data) [(1, 'a'), (2, 'x'), (4, 'v')] OK, we sorted the data, but What if we want to sort by the letter instead of the number? Let's use cmp: def comp(x, y): key_of_x=x[1] key_of_y=y[1] if key_of_x key_of_y: return -1 elif key_of_x key_of_y: return 1 else: return 0 #key_of_x == key_of_y sorted(data,cmp=comp) [(1, 'a'), (4, 'v'), (2, 'x')] Very well, so how do we do this using key? def keyfunc(x): key_of_x=x[1] return key_of_x sorted(data,key=keyfunc) [(1, 'a'), (4, 'v'), (2, 'x')] Same output. Very good. (Of course a smart python developer would use the operator module so he doesn't even have to write keyfunc but this was just an example) In summary to transform most cmp functions to a key function you just take the code that calculates the first value to be compared and leave out the rest of the logic. Hope that was helpful. -- http://mail.python.org/mailman/listinfo/python-list
Re: Dict Comprehension ?
On Oct 6, 8:59 am, Ernst-Ludwig Brust [EMAIL PROTECTED] wrote:
Given 2 Number-Lists say l0 and l1,
count the various positiv differences between the 2 lists
the following part works:
dif=[abs(x-y) for x in l0 for y in l1]
da={}
for d in dif: da[d]=da.get(d,0)+1
i wonder, if there is a way, to avoid the list dif
Ernst-Ludwig Brust
from collections import defaultdict
da=defaultdict(int)
for x in [10]:
for y in [11]:
da[abs(x-y)]+=1
--
http://mail.python.org/mailman/listinfo/python-list
Re: Builing Python 2.6 on AIX 5.2
On Oct 6, 4:16 am, brasse [EMAIL PROTECTED] wrote: Hello! I am having some trouble building Python 2.6 on AIX. The steps I have taken are: export PATH=/usr/bin/:/usr/vacpp/bin/ ./configure --with-gcc=xlc_r --with-cxx=xlC_r --disable-ipv6 make This is the error message I'm seeing: ./Modules/ld_so_aix xlc_r -bI:Modules/python.exp build/ temp.aix-5.2-2.6/home/mabr/Python-2.6/Modules/_multiprocessing/ multiprocessing.o build/temp.aix-5.2-2.6/home/mabr/Python-2.6/Modules/ _multiprocessing/socket_connection.o build/temp.aix-5.2-2.6/home/mabr/ Python-2.6/Modules/_multiprocessing/semaphore.o -L/usr/local/lib -o build/lib.aix-5.2-2.6/_multiprocessing.so ld: 0711-317 ERROR: Undefined symbol: .sem_timedwait ld: 0711-317 ERROR: Undefined symbol: .CMSG_SPACE ld: 0711-317 ERROR: Undefined symbol: .CMSG_LEN ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information. *** WARNING: renaming _multiprocessing since importing it failed: No such file or directory error: No such file or directory make: The error code from the last command is 1. Have someone on this list had similar problems? Am I missing some libraries? The configure script runs without errors, I would have expected some kind of error there if I was missing something. Regards, Mattias I don't know if it has anything to do with it. But I got that error one time on AIX and it was a library compiled to 32 bit that could not be linked to 64 bit python. Maybe in your case it is the other way around. I haven't tried compiling Py2.6 yet. I compile 64 bit with: OPT=-O2 ./configure --with-gcc=xlc_r -q64 --with-cxx=xlC_r -q64 -- disable-ipv6 AR=ar -X64 -- http://mail.python.org/mailman/listinfo/python-list
Re: Builing Python 2.6 on AIX 5.2
On Oct 6, 11:03 am, Jesse Noller [EMAIL PROTECTED] wrote: Looks like AIX is missing sem_timedwait - see:http://bugs.python.org/issue3876 Please add your error to the bug report just so I can track it. -jesse On Mon, Oct 6, 2008 at 4:16 AM, brasse [EMAIL PROTECTED] wrote: Hello! I am having some trouble building Python 2.6 on AIX. The steps I have taken are: export PATH=/usr/bin/:/usr/vacpp/bin/ ./configure --with-gcc=xlc_r --with-cxx=xlC_r --disable-ipv6 make This is the error message I'm seeing: ./Modules/ld_so_aix xlc_r -bI:Modules/python.exp build/ temp.aix-5.2-2.6/home/mabr/Python-2.6/Modules/_multiprocessing/ multiprocessing.o build/temp.aix-5.2-2.6/home/mabr/Python-2.6/Modules/ _multiprocessing/socket_connection.o build/temp.aix-5.2-2.6/home/mabr/ Python-2.6/Modules/_multiprocessing/semaphore.o -L/usr/local/lib -o build/lib.aix-5.2-2.6/_multiprocessing.so ld: 0711-317 ERROR: Undefined symbol: .sem_timedwait ld: 0711-317 ERROR: Undefined symbol: .CMSG_SPACE ld: 0711-317 ERROR: Undefined symbol: .CMSG_LEN ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information. *** WARNING: renaming _multiprocessing since importing it failed: No such file or directory error: No such file or directory make: The error code from the last command is 1. Have someone on this list had similar problems? Am I missing some libraries? The configure script runs without errors, I would have expected some kind of error there if I was missing something. Regards, Mattias -- http://mail.python.org/mailman/listinfo/python-list FYI I read that some people use: HAVE_BROKEN_POSIX_SEMAPHORES in pyconfig.h. for AIX, although I never do. Maybe I am not doing anything advanced enough for problems related to semaphores to show up. -- http://mail.python.org/mailman/listinfo/python-list
Re: improving a huge double-for cycle
On Sep 18, 7:42 pm, Steven D'Aprano [EMAIL PROTECTED] cybersource.com.au wrote: On Thu, 18 Sep 2008 20:43:00 +0200, Bruno Desthuilliers wrote: Now the obvious winner is pruebono - even unoptimized, using sets seems to be *way* faster than even the most optimized corrected version of your algorithm. I'm not being snarky about losing priority here, but I submitted essentially the same solution two hours earlier than pruebono. Are people not seeing my posts? Have I been kill-filed by everyone except Mensator? I also asked a question about HTTPError and I haven't seen any responses at all. -- Steven I see your post now, but I didn't notice it at the time I posted. Could be because I am using the noticeable bad Google groups interface that is known for being unreliable. Duplicate solutions on Usenet are almost a given and I consider duplicate solutions a good thing, it means that other people will be able to understand that code. In any case I am not here for glory I am posting under a pseudonym so nobody discovers that I am slacking off at work reading Usen[carrier lost] -- http://mail.python.org/mailman/listinfo/python-list
Re: improving a huge double-for cycle
On Sep 18, 8:25 am, Alexzive [EMAIL PROTECTED] wrote: Hello there :) , I am a python newbie and need to run following code for a task in an external simulation programm called Abaqus which makes use of python to access the mesh (ensamble of nodes with xy coordinates) of a certain geometrical model. [IN is the starting input containing the nodes to be check, there are some double nodes with the same x and y coordinates which need to be removed. SN is the output containing such double nodes] Code: Select all for i in range(len(IN)): #scan all elements of the list IN for j in range(len(IN)): if i j: if IN[i].coordinates[0] == IN[j].coordinates[0]: if IN[i].coordinates[1] == IN[j].coordinates[1]: SN.append(IN[i].label) Unfortunately my len(IN) is about 100.000 and the running time about 15h :( Any idea to improve it? I have already tried to group the if statements in a single one: Code: Select all if i j and if IN[i].coordinates[0] == IN[j].coordinates[0] and if IN[i].coordinates[1] == IN[j].coordinates[1]: but no improvements. Many thanks, Alex dup=set() SN=[] for item in IN: c=item.coordinates[0], item.coordinates[1] if c in dup: SN.append(item.label) else: dup.add(c) -- http://mail.python.org/mailman/listinfo/python-list
Re: dict generator question
On Sep 18, 10:54 am, Simon Mullis [EMAIL PROTECTED] wrote:
Hi,
Let's say I have an arbitrary list of minor software versions of an
imaginary software product:
l = [ 1.1.1.1, 1.2.2.2, 1.2.2.3, 1.3.1.2, 1.3.4.5]
I'd like to create a dict with major_version : count.
(So, in this case:
dict_of_counts = { 1.1 : 1,
1.2 : 2,
1.3 : 2 }
Something like:
dict_of_counts = dict([(v[0:3], count) for v in l])
I can't seem to figure out how to get count, as I cannot do x += 1
or x++ as x may or may not yet exist, and I haven't found a way to
create default values.
I'm most probably not thinking pythonically enough... (I know I could
do this pretty easily with a couple more lines, but I'd like to
understand if there's a way to use a dict generator for this).
Thanks in advance
SM
--
Simon Mullis
3 lines:
from collections import defaultdict
dd=defaultdict(int)
for x in l: dd[x[0:3]]+=1
--
http://mail.python.org/mailman/listinfo/python-list
Re: dict generator question
On Sep 18, 10:54 am, Simon Mullis [EMAIL PROTECTED] wrote:
Hi,
Let's say I have an arbitrary list of minor software versions of an
imaginary software product:
l = [ 1.1.1.1, 1.2.2.2, 1.2.2.3, 1.3.1.2, 1.3.4.5]
I'd like to create a dict with major_version : count.
(So, in this case:
dict_of_counts = { 1.1 : 1,
1.2 : 2,
1.3 : 2 }
Something like:
dict_of_counts = dict([(v[0:3], count) for v in l])
I can't seem to figure out how to get count, as I cannot do x += 1
or x++ as x may or may not yet exist, and I haven't found a way to
create default values.
I'm most probably not thinking pythonically enough... (I know I could
do this pretty easily with a couple more lines, but I'd like to
understand if there's a way to use a dict generator for this).
Thanks in advance
SM
--
Simon Mullis
Considering 3 identical simultpost solutions I'd say:
one obvious way to do it FTW :-)
--
http://mail.python.org/mailman/listinfo/python-list
Re: List of modules available for import inside Python?
On Aug 28, 12:21 am, ssecorp [EMAIL PROTECTED] wrote: Is there a way to view all the modules I have available for import from within Python? Like writing in the interpreter: import.modules Also, is there anything like Cpan for Python? Isn't the most obvious answer to the first question this link? http://docs.python.org/modindex.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Native Code vs. Python code for modules
On Jul 29, 11:56 pm, koblas [EMAIL PROTECTED] wrote: better if there was a way that if I have an interface compatible native (aka C) module that has better performance that there could be a way that python would give it preference. e.g. import random(version=1.2, lang=c) or import random(version=1.2, lang=py) # use the python version by default or import random # use the latest version in the fastest code (C given preference) where there could be a nice set of standard key value pairs that could provide addtional hints as to what language and version of a library was to be used. I will only make a comment on automatic cModule importing. Some of what you are suggesting already happens (e.g. Elementree). In Python 3.0 more of the legacy modules (e.g. StringIO) should also work that way. I don't know what the current status for Pickle is because cPickle had some additional internal functionality not available in regular Pickle that Zope was dependent on. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why does list have no 'get' method?
On Feb 7, 4:36 pm, George Sakkis [EMAIL PROTECTED] wrote: On Feb 7, 4:25 pm, Steven D'Aprano [EMAIL PROTECTED] cybersource.com.au wrote: On Thu, 07 Feb 2008 15:59:13 +0100, Wildemar Wildenburger wrote: Arnaud Delobelle wrote: Personally, between * foo if foo else bar * foo or bar I prefer the second. Maybe it could be spelt * foo else bar ? How about val = foo rather than bar If that is not clear and obvios, I don't know what is. ;) In English, the expression foo rather than bar means bar is never chosen, foo is always chosen. Ok, the fix is easy: val = BETTER foo THAN bar ;-) Cobol-strikes-back-ly yours, George I use a ETL language/tool that actually has a function for this kind of thing: NulltoValue(value,defaultValue) it returns defaultValue if value is null otherwise value. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why does list have no 'get' method?
On Feb 7, 12:15 pm, [EMAIL PROTECTED] wrote: On Feb 7, 11:01 am, Denis Bilenko [EMAIL PROTECTED] wrote: Steve Holden wrote: These versions differ with respect to treatment of blank lines, which indicates how easy it is to go astray in this kind of semantic optimization. Your example simply wouldn't work (though you could patch it up using if line is None. (despite the use of short-circuiting predicates). both examples show reduction by 3 lines. Perhaps so, but you have to realise that Python has never valued code compactness over clarity. I'm not sure that your democratic wish to ensure fairness to sequences will garner much support, interesting though it is in an academic sense. Thank you for the patch. My incentives are not academic though. I convinced that this line = self._buffer.get(self._bufindex) if line is None: self._bufindex += 1 self._lineno += 1 self._filelineno += 1 return line line = self.readline() is more clear than try: line = self._buffer[self._bufindex] except IndexError: pass else: self._bufindex += 1 self._lineno += 1 self._filelineno += 1 return line line = self.readline() I mentioned 3 lines reduction just because it is objective, while 'more clear' is subjective. Code snippets are again not equivalent, e.g. in the case when self._buffer[self._bufindex] exists and is equal to None. Although the author could probably make a guarantee that is never the case. This has been a pet peeve of mine too. So you are not completely crazy: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/269484 There is a discussion about precisely this by Alex Martelli in the printed version of the Python cookbook (2nd edition). I don't share the opinion but I am not going to fight the core Python developers over it. I don't think it is worth the effort. Python is so smooth already, there has to be some excuse to add some cute hack to your program once in a while :-). dict(enumerate(lst)).get(i,default) for example although not very efficient, looks cute. I think I like it better than my own solution. Actually to clarify, I think this is more useful for tuples (or tuple like structures) than lists per se. On homogeneous lists I usually iterate. It is on heterogeneous that I want to extract specific fields that might be optional. -- http://mail.python.org/mailman/listinfo/python-list
Re: Why does list have no 'get' method?
On Feb 7, 11:01 am, Denis Bilenko [EMAIL PROTECTED] wrote: Steve Holden wrote: These versions differ with respect to treatment of blank lines, which indicates how easy it is to go astray in this kind of semantic optimization. Your example simply wouldn't work (though you could patch it up using if line is None. (despite the use of short-circuiting predicates). both examples show reduction by 3 lines. Perhaps so, but you have to realise that Python has never valued code compactness over clarity. I'm not sure that your democratic wish to ensure fairness to sequences will garner much support, interesting though it is in an academic sense. Thank you for the patch. My incentives are not academic though. I convinced that this line = self._buffer.get(self._bufindex) if line is None: self._bufindex += 1 self._lineno += 1 self._filelineno += 1 return line line = self.readline() is more clear than try: line = self._buffer[self._bufindex] except IndexError: pass else: self._bufindex += 1 self._lineno += 1 self._filelineno += 1 return line line = self.readline() I mentioned 3 lines reduction just because it is objective, while 'more clear' is subjective. Code snippets are again not equivalent, e.g. in the case when self._buffer[self._bufindex] exists and is equal to None. Although the author could probably make a guarantee that is never the case. This has been a pet peeve of mine too. So you are not completely crazy: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/269484 There is a discussion about precisely this by Alex Martelli in the printed version of the Python cookbook (2nd edition). I don't share the opinion but I am not going to fight the core Python developers over it. I don't think it is worth the effort. Python is so smooth already, there has to be some excuse to add some cute hack to your program once in a while :-). dict(enumerate(lst)).get(i,default) for example although not very efficient, looks cute. I think I like it better than my own solution. -- http://mail.python.org/mailman/listinfo/python-list
Re: Compiling python2.5 on IBM AIX
On Jul 17, 3:58 am, [EMAIL PROTECTED] wrote:
I haven't compiled it myself, but I'm told that the installation I
work with was compiled with:
export PATH=$PATH:/usr/vacpp/bin:/usr/vacpp/lib
./configure --with-gcc=xlc_r -q64 --with-cxx=xlC_r -q64 --disable-
ipv6 AR=ar -X64
make
make install
I've tried with the followong configuration :
--
export PATH=$PATH:/usr/vacpp/bin:/usr/vacpp/lib
./configure --prefix=${BASE} --with-gcc=xlc_r -q64 --with-cxx=xlC_r
-q64 --disable-ipv6 AR=ar -X64 LDFLAGS=-L\${BASE}/lib/ PPFLAGS=-I
\${BASE}/include/
make
-
but it doesn't compile either :
make
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Modules/python.o ./Modules/python.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Modules/_typesmodule.o Modules/_typesmodule.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/acceler.o Parser/acceler.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/grammar1.o Parser/grammar1.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/listnode.o Parser/listnode.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/node.o Parser/node.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/parser.o Parser/parser.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/parsetok.o Parser/parsetok.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/bitset.o Parser/bitset.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/metagrammar.o Parser/metagrammar.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/firstsets.o Parser/firstsets.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/grammar.o Parser/grammar.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/pgen.o Parser/pgen.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/myreadline.o Parser/myreadline.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Parser/tokenizer.o Parser/tokenizer.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Objects/abstract.o Objects/abstract.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Objects/boolobject.o Objects/boolobject.c
xlc_r -q64 -c -DNDEBUG -O -I. -I./Include -DPy_BUILD_CORE -
o Objects/bufferobject.o Objects/bufferobject.c
Objects/bufferobject.c, line 22.15: 1506-275 (S) Unexpected text ','
encountered.
make: 1254-004 The error code from the last command is 1.
any idea ?
thanks
--
BL
Remove the extra comma at the end of line 22 of bufferobject.c
That should help you fix it.
I sent a bug report for that:
http://sourceforge.net/tracker/index.php?func=detailaid=1557983group_id=5470atid=105470
--
http://mail.python.org/mailman/listinfo/python-list
Re: Dict Help
On Jul 17, 5:57 am, Robert Rawlins - Think Blue
[EMAIL PROTECTED] wrote:
Morning Gabriel,
I'm looking for a little more advice on this dictionary/list to
defaultdict/set conversion that we were talking about, there were a few
things I was looking to clarify. Firstly, what is the difference between a
standard dict and a default dict? Is it purely a performance issue?
This dict is likely to grow pretty large and is read/written on a very
regular basis so the better performing one is going to work best for me.
Also, am I still able to iterate over a set in the same way I can a list?
Here is an example of my add function at the moment, how can that be
converted to a defaultdict/set instead of the dict/list approach?
self.History = {}
def addHistory(self, address, media):
if address not in self.History:
self.History[address] = []
self.History[address].append(media)
Thanks Gabriel,
Rob
1. defaultdict just sets a dictionary entry to a default if the key
does not exist. In other words if you replace the first line with:
self.History=defaultdict(list) #replace list by set if you want
you can get rid of:
if address not in self.History: self.History[address]=[]
because that then happens automatically. Otherwise it works exactly
like a regular dictionary.
2. You can iterate over a set the same way you iterate over a list.
3. Change the following lines:
line 5 self.History[address] = set() #instead of [] (list)
line 7 self.History[address].add(media) #set uses add instead of
append
--
http://mail.python.org/mailman/listinfo/python-list
Re: compiling on AIX 5.3 with vacpp
[EMAIL PROTECTED] wrote:
[EMAIL PROTECTED] wrote:
I am trying to compile Python 2.5 on AIX 5.3. I used
building '_locale' extension
./Modules/ld_so_aix xlc_r -q64 -bI:Modules/python.exp
build/temp.aix-5.3-2.5/home/pxadm/.test/Python-2.5/Modules/_localemodule.o
-L/usr/local/lib -o build/lib.aix-5.3-2.5/_locale.so
ld: 0711-317 ERROR: Undefined symbol: .bindtextdomain
ld: 0711-317 ERROR: Undefined symbol: .textdomain
ld: 0711-317 ERROR: Undefined symbol: .dcgettext
ld: 0711-317 ERROR: Undefined symbol: .dgettext
ld: 0711-317 ERROR: Undefined symbol: .gettext
The problem is that an additional library is needed to link the locale
module. You will first need to determine what library is needed.
Maybe libintl? To find out, you can do a man on any of those symbols,
for example, man textdomain.
That should tell you want library is required. You will then need to
modify setup.py to add the extra library for the locale module.
There's already some support for this in setup.py around line 390:
# access to ISO C locale support
data = open('pyconfig.h').read()
m = re.search(r#s*define\s+WITH_LIBINTL\s+1\s*, data)
if m is not None:
locale_libs = ['intl']
else:
locale_libs = []
if platform == 'darwin':
locale_extra_link_args = ['-framework', 'CoreFoundation']
else:
locale_extra_link_args = []
exts.append( Extension('_locale', ['_localemodule.c'],
libraries=locale_libs,
extra_link_args=locale_extra_link_args)
)
Once you get something working, please post a patch.
n
Thanks for the info Neil,
sorry for taking long to answer. This is skunkwork, I am working on it
whenever I get some time.
indeed as you say. libintl.h is required so I hard coded it into
setup.py to no effect. At what point does setup.py get called? by make?
my code
# access to ISO C locale support
data = open('pyconfig.h').read()
m = re.search(r#s*define\s+WITH_LIBINTL\s+1\s*, data)
if m is not None:
locale_libs = ['intl']
else:
locale_libs = []
if platform == 'darwin':
locale_extra_link_args = ['-framework', 'CoreFoundation']
else:
locale_extra_link_args = []
locale_libs=['intl']
exts.append( Extension('_locale', ['_localemodule.c'],
libraries=locale_libs,
extra_link_args=locale_extra_link_args) )
--
http://mail.python.org/mailman/listinfo/python-list
compiling on AIX 5.3 with vacpp
I am trying to compile Python 2.5 on AIX 5.3. I used export PATH=/usr/bin:/usr/vacpp/bin OPT=-O2 ./configure --with-gcc=xlc_r -q64 --with-cxx=xlC_r -q64 --disable-ipv6 AR=ar -X64 make The following error stops make in its track: building '_locale' extension ./Modules/ld_so_aix xlc_r -q64 -bI:Modules/python.exp build/temp.aix-5.3-2.5/home/pxadm/.test/Python-2.5/Modules/_localemodule.o -L/usr/local/lib -o build/lib.aix-5.3-2.5/_locale.so ld: 0711-317 ERROR: Undefined symbol: .bindtextdomain ld: 0711-317 ERROR: Undefined symbol: .textdomain ld: 0711-317 ERROR: Undefined symbol: .dcgettext ld: 0711-317 ERROR: Undefined symbol: .dgettext ld: 0711-317 ERROR: Undefined symbol: .gettext ld: 0711-345 Use the -bloadmap or -bnoquiet option to obtain more information. *** WARNING: renaming _locale since importing it failed: 0509-022 Cannot load module build/lib.aix-5.3-2.5. 0509-026 System error: A file or directory in the path name does not exist. error: No such file or directory make: 1254-004 The error code from the last command is 1. Stop. Any help appreciated as to explain what that error could mean and possible ways of fixing it. -- http://mail.python.org/mailman/listinfo/python-list
Re: definitive source on advanced python?
vdrab wrote: Hi all, Is there some sort of coherent source (dead tree format, maybe?) on some of the more advanced features of python (decorators, metaclasses, etc)? I'm sort of looking for a If you just want a good book in feature description I recomend Python in a Nutshell. It will explain everything you ever did (or didn't) want to know without much fuzz. It will not have a long list of when you should use them. For the later the Python Cookbook is probably more appropiate. It is the book with the most amount and kind of advanced stuff IMHO. The printed version is much better than the online because it not only contains the examples but explains how they work. The problem is that the cookbook is organized by problem domain and not by implementation technique. To learn techniques you have to read it like the Bible: you read the stories and try to deduce what it tells you about Python. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to debug python code?
[EMAIL PROTECTED] wrote: hi, I am new to Python programming.I am not getting exactly pdb.Can anyone tell me effective way to debug python code? Please give me any example. Looking for responce. Thank You. Sushant If you are having issues you also might want to try: http://pychecker.sourceforge.net/ and see if it finds your problem. You might get lucky. Or just make the bigger install: http://stani.be/python/spe/blog/ that one already includes pychecker and a python debugger. Sorry for the noise if you were just looking for instructions for pdb. -- http://mail.python.org/mailman/listinfo/python-list
Re: tricky time interval billing problem
First of all, you need to use ordering to ensure that the database gives you the most convenient order for processing, as this will make your computations much easier. So I'd suggest sorting by clientNumber, ServiceNumber, Begindate and Enddate. That way you can consider each service separately. Then all you need to do is accumulate the entries where the clientNumber and ServiceNumber are the same and the start date of the second is less than or equal to the end date of the first, repeatedly until either there's no date overlap or a new service and/or client is started. This shouldn't need any intermediate storage of results: if the row you've just read can be used to extend the current range then extend it, otherwise emit the current range and replace it with the new row. Or is there something I've failed to understand about how you want to process the data? regards Steve Hi Steve, (btw I am the one that sent you the quotes at Pycon) Hm, that would be an algorithm for requirement number 2. I do not understand how it would help with respect to requirement 1. Notice that by coincidence, in my examples the input data is already sorted as you specified. The real data is of course more messy than my analogy and I will post it here so that nobody can accuse me of feature creep. I hope I don't totally confuse you now. Feel free to ignore it. The real output is not 100% correct either (that is why I am rewriting the program). Some of the database in all its gory, ..eh glory: INPUT 43756352|D|01/01/1999|09/30/2003|DCUD2B00|DCUD2B00|Y|A|43756350|D|83516 |00374|9048327561|0001| 43756353|D|01/01/1999|09/30/2003|DCUD2B00|DCUD2B00|Y|A|43756350|D|83516 |00374|9048327561|0001| 43756351|D|01/01/1999|09/02/2002|DCUD2B00|DCUD2B00|Y|A|43756350|M|83516 |00374|9048327561|0001| 43756354|D|04/02/1999|09/30/2003|DCUD2B00|DCUD2B00|Y|A|43756350|W|83516 |00374|9048327561|0001| 43756351|M|01/01/1999|03/31/1999|MARTPPG2|MARTPPG2|Y|A|43756350|M|83516 |00374|9048327561|0001| 43756352|M|01/01/1999|03/31/1999|MARTPPG2|MARTPPG2|Y|A|43756350|D|83516 |00374|9048327561|0001| 43756353|M|01/01/1999|03/31/1999|MARTPPG2|MARTPPG2|Y|A|43756350|D|83516 |00374|9048327561|0001| 43756351|M|04/01/1999|07/31/2002|MBCPG002|MBCPG002|Y|A|43756350|M|83516 |00374|9048327561|0001| 43756352|M|04/01/1999|07/31/2002|MBCPG002|MBCPG002|Y|A|43756350|D|83516 |00374|9048327561|0001| 43756353|M|04/01/1999|07/31/2002|MBCPG002|MBCPG002|Y|A|43756350|D|83516 |00374|9048327561|0001| 43756354|M|04/02/1999|07/31/2002|MBCPG002|MBCPG002|Y|A|43756350|W|83516 |00374|9048327561|0001| 43756352|M|08/01/2002|09/30/2003|MBP07305|MBP07305|Y|A|43756350|D|83516 |00374|9048327561|0001| 43756353|M|08/01/2002|09/30/2003|MBP07305|MBP07305|Y|A|43756350|D|83516 |00374|9048327561|0001| 43756354|M|08/01/2002|09/30/2003|MBP07305|MBP07305|Y|A|43756350|W|83516 |00374|9048327561|0001| 43756351|M|08/01/2002|09/02/2002|MBP07305|MBP07305|Y|A|43756350|M|83516 |00374|9048327561|0001| OUTPUT 43756350|9048327561|DCUD2B00|D|A|01/01/1999|04/01/1999|83516 |00374 |0001|Y|A|DCUD2B00| 43756350|9048327561|DCUD2B00|D|E|04/02/1999|09/30/2003|83516 |00374 |0001|Y|A|DCUD2B00| 43756350|9048327561|MARTPPG2|M|A|01/01/1999|03/31/1999|83516 |00374 |0001|Y|A|MARTPPG2| 43756350|9048327561|MBCPG002|M|A|04/01/1999|07/31/2002|83516 |00374 |0001|Y|A|MBCPG002| 43756350|9048327561|MBP07305|M|A|08/01/2002|09/02/2002|83516 |00374 |0001|Y|A|MBP07305| 43756350|9048327561|MBP07305|M|E|09/03/2002|09/30/2003|83516 |00374 |0001|Y|A|MBP07305| CHEAT SHEET: | (M) | (H,W) | (D,O,S) || +== | -|-| - ||O| +-+ | -|-| X ||G| +-+ | -|X| - ||F| +-+ | -|X| X ||E| +-+ | X|-| - ||C| +-+ | X|-| X ||D| +-+ | X|X| - ||B| +-+ | X|X| X ||A| +-+ regards, Nes -- http://mail.python.org/mailman/listinfo/python-list
tricky time interval billing problem
I am currently working on a tricky problem at work. I googled around a bit, but time intervals did not come up with anything useful. Although I have some rough idea of how I could solve it, I still would value some input. I have information of (It has only couple dozen entries.) ServiceNum, DollarCost and a input database table in the form (several GBytes): ClientNumber (CN), BeginDate(BD), EndDate(ED), ServiceNumber_Purchased(SN) --Date intervals are always [closed, closed] The output is: ClientNumber(CN), BeginDate(BD), Enddate(ED), DollarCost(DC) With the following requirements: 1) The input dates can be overlapping dates. The output has to be non-overlapping broken up dates Example: (assuming SN 1=$10 and SN 2=$20) input (CN, BD, ED ,SN): 10, 1/1/2001,1/1/2005,1 10, 1/1/2002,1/1/2004,2 should result in: output (CN, BD, ED, DC): 10, 1/1/2001, 12/31/2001, $10 10, 1/1/2002, 1/1/2004, $30 10, 1/2,2004, 1/1/2005, $10 2) if the same service is purchased twice for an interval it should be billed only once Example: input: 10, 1/1/2001, 1/1/2005, 1 10, 1/1/2004, 1/1/2007, 1 output: 10, 1/1/2001, 1/1/2007, $10 Here are my thoughts about a solution so far: 1. fetch the input data sorted by clientNumber, Begindate, Enddate and ServiceNumber 2. read the row, store as temporary good interval, then read another row 3. if new begin-date is bigger than previous good interval end-date (or previously saved end-date), output previous good interval, start new good interval 4. else create new good interval entry with good interval begin-date and current row begin-date, store good interval end-date into a list with bisect module or something (so we always compare against smallest end-date). 5. Use bitwise or on a service bitfield to add services and caluate the total As you can see my algorithm is a bit scetchy right now, but at this point I am more interested to determine if the bisect module would be the best way to approach the date interval problem or if I should use some other data structure instead (stack, queue, set,...). I am working on a Python proof of concept now. It is likely that the company will want a C++ version of it later (yikes). nes -- http://mail.python.org/mailman/listinfo/python-list
Re: anaconda.real in RH7.1
Allan Adler wrote: I'm using the book, Programming Python, by Mark Lutz, as a reference. No offence to Mark Lutz or O'Reilly but I consider Programming Python one of the worst books I have read (in my case an old first edition). It overwhelms the beginning programmer (Learning Python is probably better for that), it bores the experienced programmer to death with introductory details and does not work as a reference book. It is a nice survey of application areas for python, but the book lacks a clear goal, purpose and a problem it tries to address. It needs more focus. It needs to be marketed correctly also. I believe too many people new to python buy that book and get discouraged about Python, O'Reilly and programming when they should buy something else instead. That said what you probably want is Python in a Nutshell by O'Reilly which is a good reference book it has a concise introduction of Python in the beginning and after that, documentation for the most usefull libraries out there. Personally I use the online documentation a lot: If I know the the name of the module (for example it starts with sys.zz) I use: http://www.python.org/doc/2.4.1/modindex.html If I know what I want but not the name I use: http://www.python.org/doc/2.4.1/lib/lib.html -- http://mail.python.org/mailman/listinfo/python-list
Re: Improving my text processing script
Paul McGuire wrote:
match...), this program has quite a few holes.
What if the word Identifier is inside one of the quoted strings?
What if the actual value is tablename10? This will match your
tablename1 string search, but it is certainly not what you want.
Did you know there are trailing blanks on your table names, which could
prevent any program name from matching?
Good point. I did not think about that. I got lucky because none of the
table names had trailing blanks (google groups seems to add those) the
word identifier is not used inside of quoted strings anywhere and I do
not have tablename10, but I do have dba.tablename1 and that one has
to match with tablename1 (and magically did).
So here is an alternative approach using, as many have probably
predicted by now if they've spent any time on this list, the pyparsing
module. You may ask, isn't a parser overkill for this problem? and
You had to plug pyparsing! :-). Thanks for the info I did not know
something like pyparsing existed. Thanks for the code too, because
looking at the module it was not totally obvious to me how to use it. I
tried run it though and it is not working for me. The following code
runs but prints nothing at all:
import pyparsing as prs
f=file('tlst'); tlst=[ln.strip() for ln in f if ln]; f.close()
f=file('plst'); plst=f.read() ; f.close()
prs.quotedString.setParseAction(prs.removeQuotes)
identLine=(prs.LineStart()
+ 'Identifier'
+ prs.quotedString
+ prs.LineEnd()
).setResultsName('prog')
tableLine=(prs.LineStart()
+ 'Value'
+ prs.quotedString
+ prs.LineEnd()
).setResultsName('table')
interestingLines=(identLine | tableLine)
for toks,start,end in interestingLines.scanString(plst):
print toks,start,end
--
http://mail.python.org/mailman/listinfo/python-list
Re: Improving my text processing script
Miki Tebeka wrote: Look at re.findall, I think it'll be easier. Minor changes aside the interesting thing, as you pointed out, would be using re.findall. I could not figure out how to. -- http://mail.python.org/mailman/listinfo/python-list
Re: Improving my text processing script
[EMAIL PROTECTED] wrote: Paul McGuire wrote: match...), this program has quite a few holes. tried run it though and it is not working for me. The following code runs but prints nothing at all: import pyparsing as prs And this is the point where I have to post the real stuff because your code works with the example i posted and not with the real thing. The identifier I am interested in is (if I understood the the requirements correctly) the one after the title with the stars So here is the real data for tlst some info replaced with z to protect privacy: * Identifier zzz0main * Identifier zz501 Value zzz_CLCL_,zz_ID Name z Name zz * Identifier 3main * Identifier zzz505 Value dba.zzz_CKPY__SUM Name xxx_xxx_xxx_DT -- Value zzz__zzz_zzz Name zzz_zz_zzz -- Value zzz_zzz_zzz_HIST Name zzz_zzz -- -- http://mail.python.org/mailman/listinfo/python-list
Improving my text processing script
I am sure there is a better way of writing this, but how?
import re
f=file('tlst')
tlst=f.read().split('\n')
f.close()
f=file('plst')
sep=re.compile('Identifier (.*?)')
plst=[]
for elem in f.read().split('Identifier'):
content='Identifier'+elem
match=sep.search(content)
if match:
plst.append((match.group(1),content))
f.close()
flst=[]
for table in tlst:
for prog,content in plst:
if content.find(table)0:
flst.append('%s,%s'%(prog,table))
flst.sort()
for elem in flst:
print elem
What would be the best way of writing this program. BTW find0 to check
in case table=='' (empty line) so I do not include everything.
tlst is of the form:
tablename1
tablename2
...
plst is of the form:
Identifier Program1
Name Random Stuff
Value tablename2
...other random properties
Name More Random Stuff
Identifier Program 2
Name Yet more stuff
Value tablename2
...
I want to know in what programs are the tables in tlst (and only those)
used.
--
http://mail.python.org/mailman/listinfo/python-list
