Re: Read-Write Lock vs primitive Lock()
Note that the thread acquires the lock ONCE, repeats several thousand times an assignment to a *local* variable called GLOBAL_VAR (!), finally releases the lock and exits. As every thread does the same, they just run one after another, they never have a significant overlap. Ok. Forget about the GLOBAL_VAR. See the below code: import threading import locks import time GLOBAL_VAR = [] #GLOBAL_LOCK = locks.ReadWriteLock() GLOBAL_LOCK = threading.Lock() #GLOBAL_LOCK = mylock() GLOBAL_LOOP_COUNT = 1000 GLOBAL_READER_COUNT = 100 GLOBAL_WRITER_COUNT = 1 global GLOBAL_LEN class wthread(threading.Thread): def run(self): try: GLOBAL_LOCK.acquireWrite() #GLOBAL_LOCK.acquire_write() #GLOBAL_LOCK.acquire() for i in range(GLOBAL_LOOP_COUNT): pass finally: #GLOBAL_LOCK.release_write() GLOBAL_LOCK.release() class rthread(threading.Thread): def run(self): try: GLOBAL_LOCK.acquireRead() #GLOBAL_LOCK.acquire_read() print 'ENTER:'+str(threading.currentThread()) #GLOBAL_LOCK.acquire() print 'ACQUIRE:'+str(threading.currentThread()) for i in range(GLOBAL_LOOP_COUNT): pass finally: #GLOBAL_LOCK.release_read() GLOBAL_LOCK.release() print 'RELEASE:'+str(threading.currentThread()) # module executed? if __name__ == __main__: starttime = time.clock() threads = [] for i in range(GLOBAL_READER_COUNT): rt = rthread() threads.append(rt) for i in range(GLOBAL_WRITER_COUNT): wt = wthread() threads.append(wt) for thread in threads: thread.start() for thread in threads: thread.join() print All operations took + str(time.clock() - starttime) + msecs As GLOBAL_LOOP_COUNT is 1000, now this is making a bottleneck on the readers. I had assumed that as everythread is given only 100 bytecodes to execute, that it will be enough to have a 1 value for this number to let other rthread start() and try to acquire the lock. But, Python undocumentedly, does not grab the GIL easily if a Lock is held. This is strange for me. If I put the for loop outside, in that case, readers will not overlap at all, and you would be amazed by the numbers for that test. They indicate primitive lock is faster than read-write lock, as it requires the lock, executes only one bytecode operation and releases the lock. So, in order to create a bottlenneck on readers, we need to somehow do not release the lock immediately. Again, what is your specific use case? why do you think readers will see a bottleneck? With that, primitive locks perform 10 times better than Read-Write lock. See above. So what? You don't like such results? The read-write lock in the recipe you posted is rather complex, and isn't obvious whether it is correct or not, and the author doesn't prove it nor provide a reference. I'd stick to the standard Lock object unless I have specific needs. Note that threading.Lock objects are implemented in C and don't have significant overhead (other than the wait itself); even threading.RLock objects are a lot slower. So depending on what you really have to do, the overhead of using a class like ReadWriteLock may be worse than using a standard Lock. That may be the point. That is why I am trying to test this. When will ReadWrite() lock permforms better over the primitive lock. By the way, for the reference, the code in the recipe is used in CherryPy and based on the Tanenbaum's book Operating Systems Design. Hmmm, it's a reader but attempts to modify the value? You don't have to protect a read operation on a global variable - so a lock isn't required here. This is just for testing. Suppose that I am actually reading the value. I don't understand why a lock is not required? Are you saying lock is not necesary beacuse GLOBAL_VALUE is an immutable object, if then, suppose it is not. This is just a test. Suppose GLOBAl_VAR is a list and we are calling append() on it which is nt an atomic operation. It doesn't matter whether it is mutable or immutable. Remember that, in Python, a global variable is just a *name* that refers to an object; you either get the old object referred, or the new one. If it is a list and it is being modified in place, you always get the same list -- its contents may differ from one instant to another so iterating over it isn't safe. That's why I insist on knowing your use case: you may me doing things more complicated than they should. I understand your point completely, but let's change anything inside the loop. Just assume that we a thread that is supposed to read something, and a thread that is supposed to write. If I create
Re: Read-Write Lock vs primitive Lock()
Note that the thread acquires the lock ONCE, repeats several thousand times an assignment to a *local* variable called GLOBAL_VAR (!), finally releases the lock and exits. As every thread does the same, they just run one after another, they never have a significant overlap. Ok. Forget about the GLOBAL_VAR. See the below code: import threading import locks import time GLOBAL_VAR = [] #GLOBAL_LOCK = locks.ReadWriteLock() GLOBAL_LOCK = threading.Lock() #GLOBAL_LOCK = mylock() GLOBAL_LOOP_COUNT = 1000 GLOBAL_READER_COUNT = 100 GLOBAL_WRITER_COUNT = 1 global GLOBAL_LEN class wthread(threading.Thread): def run(self): try: GLOBAL_LOCK.acquireWrite() #GLOBAL_LOCK.acquire_write() #GLOBAL_LOCK.acquire() for i in range(GLOBAL_LOOP_COUNT): pass finally: #GLOBAL_LOCK.release_write() GLOBAL_LOCK.release() class rthread(threading.Thread): def run(self): try: GLOBAL_LOCK.acquireRead() #GLOBAL_LOCK.acquire_read() print 'ENTER:'+str(threading.currentThread()) #GLOBAL_LOCK.acquire() print 'ACQUIRE:'+str(threading.currentThread()) for i in range(GLOBAL_LOOP_COUNT): pass finally: #GLOBAL_LOCK.release_read() GLOBAL_LOCK.release() print 'RELEASE:'+str(threading.currentThread()) # module executed? if __name__ == __main__: starttime = time.clock() threads = [] for i in range(GLOBAL_READER_COUNT): rt = rthread() threads.append(rt) for i in range(GLOBAL_WRITER_COUNT): wt = wthread() threads.append(wt) for thread in threads: thread.start() for thread in threads: thread.join() print All operations took + str(time.clock() - starttime) + msecs As GLOBAL_LOOP_COUNT is 1000, now this is making a bottleneck on the readers. I had assumed that as everythread is given only 100 bytecodes to execute, that it will be enough to have a 1 value for this number to let other rthread start() and try to acquire the lock. But, Python undocumentedly, does not grab the GIL easily if a Lock is held. This is strange for me. If I put the for loop outside, in that case, readers will not overlap at all, and you would be amazed by the numbers for that test. They indicate primitive lock is faster than read-write lock, as it requires the lock, executes only one bytecode operation and releases the lock. So, in order to create a bottlenneck on readers, we need to somehow do not release the lock immediately. Again, what is your specific use case? why do you think readers will see a bottleneck? With that, primitive locks perform 10 times better than Read-Write lock. See above. So what? You don't like such results? The read-write lock in the recipe you posted is rather complex, and isn't obvious whether it is correct or not, and the author doesn't prove it nor provide a reference. I'd stick to the standard Lock object unless I have specific needs. Note that threading.Lock objects are implemented in C and don't have significant overhead (other than the wait itself); even threading.RLock objects are a lot slower. So depending on what you really have to do, the overhead of using a class like ReadWriteLock may be worse than using a standard Lock. That may be the point. That is why I am trying to test this. When will ReadWrite() lock permforms better over the primitive lock. By the way, for the reference, the code in the recipe is used in CherryPy and based on the Tanenbaum's book Operating Systems Design. Hmmm, it's a reader but attempts to modify the value? You don't have to protect a read operation on a global variable - so a lock isn't required here. This is just for testing. Suppose that I am actually reading the value. I don't understand why a lock is not required? Are you saying lock is not necesary beacuse GLOBAL_VALUE is an immutable object, if then, suppose it is not. This is just a test. Suppose GLOBAl_VAR is a list and we are calling append() on it which is nt an atomic operation. It doesn't matter whether it is mutable or immutable. Remember that, in Python, a global variable is just a *name* that refers to an object; you either get the old object referred, or the new one. If it is a list and it is being modified in place, you always get the same list -- its contents may differ from one instant to another so iterating over it isn't safe. That's why I insist on knowing your use case: you may me doing things more complicated than they should. I understand your point completely, but let's change anything inside the loop. Just assume that we a thread that is supposed to read something, and a thread that is supposed to write. If I create
Re: How do I DRY the following code?
On Mon, 29 Dec 2008 21:13:55 -0500, R. Bernstein wrote:
How do I DRY the following code?
class C():
[snip code]
Move the common stuff into methods (or possibly external functions). If
you really need to, make them private by appending an underscore to the
front of the name.
class C():
def common1(self, *args):
return common1
def common2(self, *args):
return common2
def _more(self, *args): # Private, don't touch!
return more stuff
def f1(self, arg1, arg2=None, globals=None, locals=None):
... unique stuff #1 ...
self.common1()
ret = eval(args, globals, locals)
self._more()
return retval
def f2(self, arg1, arg2=None, *args, **kwds):
... unique stuff #2 ...
self.common1()
ret = arg1(args, *args, **kwds)
self.common2
return retval
def f3(self, arg1, globals=None, locals=None):
... unique stuff #3 ...
self.common1()
exec cmd in globals, locals
self.common2()
return None # unneeded
def f4(self, arg1, globals=None, locals=None):
... unique stuff #4 ...
self.common1()
execfile(args, globals, locals)
self._more()
An explicit return None is probably not needed. Python functions and
methods automatically return None if they reach the end of the function.
Above there are two kinds of duplication: that within class C and that
outside which creates an instance of the class C and calls the
corresponding method.
Do you really need them? If so, you're repeating yourself by definition.
That's not necessarily a problem, the stand-alone functions are just
wrappers of methods. You can decrease (but not eliminate) the amount of
repeated code with a factory function:
def build_standalone(methodname, docstring=None):
def function(arg, arg2=None, globals=None, locals=None):
c = C()
return c.getattr(methodname)(arg, arg2, globals, locals)
function.__name__ = methodname
function.__doc__ = docstring
return function
f1 = build_standalone('f1', Docstring f1)
f2 = build_standalone('f2', Docstring f2)
f3 = build_standalone('f3', Docstring f3)
There's no way to avoid the repeated f1 etc.
But honestly, with only three such functions, I'd consider that overkill.
Lest the above be too abstract, those who want to look at the full
(and redundant) code:
http://code.google.com/p/pydbg/source/browse/trunk/api/pydbg/api/
debugger.py
You have parameters called Globals and Locals. It's the usual Python
convention that names starting with a leading uppercase letter is a
class. To avoid shadowing the built-ins, it would be more conventional to
write them as globals_ and locals_. You may or may not care about
following the convention :)
I notice you have code like the following:
if Globals is None:
import __main__
Globals = __main__.__dict__
I would have written that like:
if Globals is None:
Globals = globals()
or even
if Globals is None:
from __main__ import __dict__ as Globals
You also have at least one unnecessary pass statement:
if not isinstance(expr, types.CodeType):
expr = expr+'\n'
pass
The pass isn't needed.
In your runcall() method, you say:
res = None
self.start(start_opts)
try:
res = func(*args, **kwds)
except DebuggerQuit:
pass
finally:
self.stop()
return res
This is probably better written as:
self.start(start_opts)
try:
return func(*args, **kwds)
except DebuggerQuit:
return None
finally:
self.stop()
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: wxPython.button.disabled still catching clicks
On Dec 23, 6:12 pm, Mike Driscoll kyoso...@gmail.com wrote:
On Dec 23, 7:27 am,mynthonmynth...@gmail.com wrote:
On Dec 23, 11:58 am, Aaron Brady castiro...@gmail.com wrote:
On Dec 23, 4:50 am,mynthonmynth...@gmail.com wrote:
Hello! (sorry for my english)
I have a problem with buttons in wxPython. When button is disabled
(by .Disable() or .Enable(False)) it is grayed out but still receive
clicks.
Eg. i have button that disable itself, runs long action and enable
itself:
def onClick(self, evt):
self.btn.Enable(False)
for i in range (1000):
print i
self.btn.Enable(True)
when for loop is running button is greyed out and when i click on it
nothing happens but when loop ends another one is started because
button remebered thad i click on it when was diabled. My only idea
is to reposition button outside frame instead of disabling it but this
solution is...not good.
thanks for any help. Ive searched groups, google and it looks that
only i have this problem :)
No, it is very common. During your for loop, the loop is dominating
the process completely. Events are just building up in the app's
message queue, and don't get handled until after you yield on control.
If you need to run a long task, look into threading, the OnIdle
method, the 'multiprocessing' module, or pump messages during your
long task.
ok, maybe someone will need it. I dont know how it works because i
didnt have time to read docs and i cannot explain everything. I used
google and wxPython demo (in tree: wxpython overview / process and
events / process)
class leftPanel(wx.Panel):
def __init__(self, parent, id):
wx.Panel.__init__(self, parent, id, style=wx.BORDER_SUNKEN)
# here you have to define new process, IDLE event, and
onPRocessEnd event
self.process = None
self.GetParent().Bind(wx.EVT_IDLE, self.onIdle)
self.Bind(wx.EVT_END_PROCESS, self.onProcessEnd)
# create button and bind event to it
self.runScriptBtn = wx.Button(self, -1, 'RUN ME!', (10,220))
self.runScriptBtn.Bind(wx.EVT_BUTTON, self.onClick,
self.runScriptBtn)
def onClick(self, evt):
# disable button
self.runScriptBtn.Enable(False)
# here you have to enter command to run
# previusly i heve here exec('pythonmyScript.py')
# but now it will be a subprocess
cmd = 'pythonxxx1.py'
#create new process
self.process = wx.Process(self)
# dont know what it is for
self.process.Redirect()
# execute cmd command
pid = wx.Execute(cmd, wx.EXEC_ASYNC, self.process)
def onIdle(self, evt):
# beacuse this method is called only when app enters idle mode
# the line below is nedded to simulate entering idle mode
# dont know how it works but it works
evt.RequestMore(True)
# here is some code to catch subprocess output
if self.process is not None:
stream = self.process.GetInputStream()
if stream.CanRead():
text = stream.read()
print text
def onProcessEnd(self, evt):
# here is some code to catch subprocess output
# when it is destroyed
stream = self.process.GetInputStream()
if stream.CanRead():
text = stream.read()
print text
# dont know it is necessary
self.process.CloseOutput()
# remove (clear) process object
self.process.Destroy()
self.process = None
# show button again
self.runScriptBtn.Enable()
I'm pretty sure there's a better way to do this. If you disable the
button and click on it, you'll notice that it isn't firing events, so
something else is going on here. It seems like the wx.Frame or
wx.Application is queuing the mouse button clicks or something. I
would post to the wxPython mailing list:http://wxpython.org/maillist.php
They'll be better able to address this and they'll probably have a
simpler solution too.
Mike
I changed it. Now i'm running separate thread instead of process.
First example at:
http://wiki.wxpython.org/LongRunningTasks
--
http://mail.python.org/mailman/listinfo/python-list
Re: get method
James Mills schreef:
Ross, the others have informed you that you are not
actually incrementing the count. I'll assume you've
fixed your function now :) ... I want to show you a far
simpler way to do this which takes advantage of
Python's list comprehensions and mappings (which are
really what dictionaries are):
s = James Mills and Danielle Van Sprang
dict([(k, len([x for x in s if x == k])) for k in s])
{'a': 5, ' ': 5, 'e': 3, 'd': 1, 'g': 1, 'i': 2, 'M': 1, 'J': 1, 'm':
1, 'l': 4, 'n': 4, 'p': 1, 's': 2, 'r': 1, 'V': 1, 'S': 1, 'D': 1}
Hm, you just changed an O(n) algorithm to an O(n**2) algorithm. No big
deal for short strings, but try your solution on a string with length
1 and see the difference. On my computer the O(n) version takes
0.008 seconds, while your version takes 8.6 seconds. That's 1000 times
slower.
--
The saddest aspect of life right now is that science gathers knowledge
faster than society gathers wisdom.
-- Isaac Asimov
Roel Schroeven
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to debug embeding Python?
Using udp to send out message is a convenient way, you may define a
log function like following, and start a udp server to lisen.
#
from socket import *
udpSock = socket(AF_INET,SOCK_DGRAM)
def log(s):
udpSock.sendto(s, ('127.0.0.1', 514))
log('hello')
--
http://mail.python.org/mailman/listinfo/python-list
Re: SQL, lite lite lite
Johannes Bauer a écrit : (snip) Even if it took (as you mentioned) a semester of SQL studies - which it does not - why do you think your syntax is easier? The only person your proposed syntax is easier for is you. Get over it, learn SQL, and enjoy the benefits of one unified standard - not everyone cooking their own soup. You'll be able to learn PostgreSQL, Oracle SQL, mySQL, SQlite all at once! Oh what a beautiful dream... We all wish we'd live in such a perfect world. Sadly, it's not the case. First because each SQL vendor cooks its own soup, so you still have to learn each SQL implementation, limitations and gory details. Also, because handling SQL queries as raw strings is more than painfull. It just doesn't play well with the host language. So any non trivial project ends up reinventing its own half-backed abstraction layer, that you have to learn too. Now I don't mean that learning SQL (and the relational theory behind SQL) is a waste of time. You obviously need a good understanding of SQL to use a SQL database - directly or thru any abstraction layer. We definitively agree on this. But a smart abstraction layer (like SQLAlchemy, or even the less powerfull but still pretty good Django ORM) at least avoids reinventing a new one for each project. My 2 cents... -- http://mail.python.org/mailman/listinfo/python-list
Re: How do I DRY the following code?
Steven D'Aprano st...@remove-this-cybersource.com.au writes:
On Mon, 29 Dec 2008 21:13:55 -0500, R. Bernstein wrote:
How do I DRY the following code?
class C():
[snip code]
Move the common stuff into methods (or possibly external functions). If
you really need to, make them private by appending an underscore to the
front of the name.
class C():
def common1(self, *args):
return common1
def common2(self, *args):
return common2
def _more(self, *args): # Private, don't touch!
return more stuff
def f1(self, arg1, arg2=None, globals=None, locals=None):
... unique stuff #1 ...
self.common1()
ret = eval(args, globals, locals)
self._more()
return retval
def f2(self, arg1, arg2=None, *args, **kwds):
... unique stuff #2 ...
self.common1()
ret = arg1(args, *args, **kwds)
self.common2
return retval
def f3(self, arg1, globals=None, locals=None):
... unique stuff #3 ...
self.common1()
exec cmd in globals, locals
self.common2()
return None # unneeded
def f4(self, arg1, globals=None, locals=None):
... unique stuff #4 ...
self.common1()
execfile(args, globals, locals)
self._more()
I realize I didn't mention this, but common1 contains try: and more contains
except ... finally.
So for example there is
self.start()
try:
res = func(*args, **kwds) # this line is different
except
...
finally:
...
Perhaps related is the fact that common1 and common2 are really
related and therefore breaking into the function into 3 parts sort of
breaks up the flow of reading one individually.
I had also considered say passing an extra parameter and having a case
statement around the one line that changes, but that's ugly and
complicates things too.
An explicit return None is probably not needed. Python functions and
methods automatically return None if they reach the end of the function.
return None is a perhaps a stylistic idiom. I like to put returns
at the end of a function because it helps (and Emacs) me keep
indentation straight. Generally, I'll put return None if the
function otherwise returns a value and just return if I don't think
there is a useable return value.
I've also noticed that using pass at the end of blocks also helps me
and Emacs keep indntation straight. For example:
if foo():
bar()
else
baz()
pass
while True:
something
pass
Above there are two kinds of duplication: that within class C and that
outside which creates an instance of the class C and calls the
corresponding method.
Do you really need them?
Perhaps, because they may be the more common idioms. And by having
that function there a temporary complex object can be garbage
collected and doesn't pollute the parent namespace. Is this a big
deal? I don't know but it doesn't hurt.
If so, you're repeating yourself by definition.
That's not necessarily a problem, the stand-alone functions are just
wrappers of methods. You can decrease (but not eliminate) the amount of
repeated code with a factory function:
def build_standalone(methodname, docstring=None):
def function(arg, arg2=None, globals=None, locals=None):
c = C()
return c.getattr(methodname)(arg, arg2, globals, locals)
function.__name__ = methodname
function.__doc__ = docstring
return function
f1 = build_standalone('f1', Docstring f1)
f2 = build_standalone('f2', Docstring f2)
f3 = build_standalone('f3', Docstring f3)
Yes, this is better than the lambda. Thanks!
There's no way to avoid the repeated f1 etc.
But honestly, with only three such functions, I'd consider that overkill.
Yeah, I think so too.
Lest the above be too abstract, those who want to look at the full
(and redundant) code:
http://code.google.com/p/pydbg/source/browse/trunk/api/pydbg/api/
debugger.py
You have parameters called Globals and Locals. It's the usual Python
convention that names starting with a leading uppercase letter is a
class. To avoid shadowing the built-ins, it would be more conventional to
write them as globals_ and locals_. You may or may not care about
following the convention :)
Ok. Yes, some earlier code used globals_ and locals_. Thanks.
I notice you have code like the following:
if Globals is None:
import __main__
Globals = __main__.__dict__
I would have written that like:
if Globals is None:
Globals = globals()
Yes, that's better. Thanks.
or even
if Globals is None:
from __main__ import __dict__ as Globals
You also have at least one unnecessary pass statement:
if not isinstance(expr, types.CodeType):
expr = expr+'\n'
pass
The pass isn't needed.
In your runcall() method, you say:
res = None
self.start(start_opts)
try:
res = func(*args, **kwds)
except DebuggerQuit:
pass
finally:
self.stop()
return res
This is probably better
Re: How do I DRY the following code?
Patrick Mullen saluk64...@gmail.com writes: f1(...): Docstring f1 c = C() return c.f1(...) f2(...): Docstring f2 c = C() return c.f2(...) Why not just do either this: C().f2(..) where you need f2 Yes, this is a little better. Thanks! -- http://mail.python.org/mailman/listinfo/python-list
string in files
guys i need info on how to call up different words in a line of a file using python example : file = 'this is a python coding group' i want to assign a xter to this, is, a, python , coding and group thanks -- http://mail.python.org/mailman/listinfo/python-list
RE: string in files
Simple solution: us result=yourString.split( ) and you get a list with all the words. -Original Message- From: python-list-bounces+rbqg84=motorola@python.org [mailto:python-list-bounces+rbqg84=motorola@python.org] On Behalf Of ibpe...@gmail.com Sent: Tuesday, December 30, 2008 3:43 PM To: python-list@python.org Subject: string in files guys i need info on how to call up different words in a line of a file using python example : file = 'this is a python coding group' i want to assign a xter to this, is, a, python , coding and group thanks -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: string in files
On Dec 30, 11:17 am, Narasimhan Raghu-RBQG84 rbq...@motorola.com wrote: Simple solution: us result=yourString.split( ) and you get a list with all the words. -Original Message- From: python-list-bounces+rbqg84=motorola@python.org [mailto:python-list-bounces+rbqg84=motorola@python.org] On Behalf Of ibpe...@gmail.com Sent: Tuesday, December 30, 2008 3:43 PM To: python-l...@python.org Subject: string in files guys i need info on how to call up different words in a line of a file using python example : file = 'this is a python coding group' i want to assign a xter to this, is, a, python , coding and group thanks --http://mail.python.org/mailman/listinfo/python-list thanks brother i mean how do i particularly assign (u = this) (y = is) in the strings up there. i have been able to split strings with any character sign. -- http://mail.python.org/mailman/listinfo/python-list
using def in pythons
hi, i want to have a broad knowledge on the use of def in python as i know i might need it to my string handling and for a lot of things in general. I will really appreciate anybody who can support my mission of becoming a python Programmer as per constant chatting and support or manuals to read. thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: string in files
thanks brother
i mean how do i particularly assign (u = this)
(y = is)
in the strings up there. i have been able to split strings with any
character sign.
If i'm not wrong this is simple with RE:
In [1]: st = 'this is a python coding group'
In [2]: import re
In [3]: re.compile( (?Pfirst.*) (?Psecond.*) (?Pt.*) (?Pfo.*)
(?Pfi.*) (?Psi.*) )
Out[3]: _sre.SRE_Pattern object at 0x9e93ac0
In [4]: rule = re.compile( (?Pfirst.*) (?Psecond.*) (?Pt.*)
(?Pfo.*) (?Pfi.*) (?Psi.*) )
In [5]: m = rule.match( st )
In [6]: dir(m)
Out[6]:
['__copy__', '__deepcopy__',
'end', 'expand',
'group', 'groupdict',
'groups', 'span', 'start']
In [7]: m.groupdict().items()
Out[7]:
[('si', 'group'),
('second', 'is'),
('t', 'a'),
('fi', 'coding'),
('fo', 'python'),
('first', 'this')]
In [8]: dict(m.groupdict().items())
Out[8]:
{'fi': 'coding',
'first': 'this',
'fo': 'python',
'second': 'is',
'si': 'group',
't': 'a'}
Glauco
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python in C
aki Although this is not what you are asking but I'm wondering why you aki need to read CPython implementation. A couple reasons come to mind: * education * want to make it better (extend it, fix bugs, etc) * want to see how it relates to the implementation of other languages (Ruby, Perl, Tcl, etc) aki CPython worked great for me. I don't want to read a large piece of aki software, like CPython, unless it is really really necessary. Sure, but not everyone works at the Python layer. Which is a good thing, because if everyone confined themselves to Python code nobody would fix problems in the language implementation or enhance it. -- Skip Montanaro - s...@pobox.com - http://smontanaro.dyndns.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: using def in pythons
On Dec 30, 3:37 pm, ibpe...@gmail.com wrote: hi, i want to have a broad knowledge on the use of def in python as i know i might need it to my string handling and for a lot of things in general. I will really appreciate anybody who can support my mission of becoming a python Programmer as per constant chatting and support or manuals to read. thanks go through this link http://www.python.org/doc/2.5.1/ref/function.html -- http://mail.python.org/mailman/listinfo/python-list
Re: string in files
ibpe...@gmail.com wrote: On Dec 30, 11:17 am, Narasimhan Raghu-RBQG84 rbq...@motorola.com wrote: Simple solution: us result=yourString.split( ) and you get a list with all the words. -Original Message- From: python-list-bounces+rbqg84=motorola@python.org [mailto:python-list-bounces+rbqg84=motorola@python.org] On Behalf Of ibpe...@gmail.com Sent: Tuesday, December 30, 2008 3:43 PM To: python-l...@python.org Subject: string in files guys i need info on how to call up different words in a line of a file using python example : file = 'this is a python coding group' i want to assign a xter to this, is, a, python , coding and group [...] thanks brother i mean how do i particularly assign (u = this) (y = is) in the strings up there. i have been able to split strings with any character sign. Well, if you *know* you have a particular number of words in the string you can say u, v, w, x, y, z = 'this is a python coding group'.split() But if you have a variable number of words this isn't practical in Python 2, though Python 3 has features that make it easier. The real question is what is the larger goal you are trying to achieve. Where a programmer is trying to create names dynamically there are usually better ways to proceed. Could you tell us a little more about what you are trying to do? regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: string in files
On Tue, 30 Dec 2008 11:53:17 +0100, Glauco wrote: thanks brother i mean how do i particularly assign (u = this) (y = is) in the strings up there. i have been able to split strings with any character sign. If i'm not wrong this is simple with RE: Using Regular Expression for this is an overkill, you'd better use the str.split: longstring = 'this is a python string' splitted_string = longstring.split() result = ', '.join(splitted_string[:-1]) + ' and ' + splitted_string[-1] -- http://mail.python.org/mailman/listinfo/python-list
Re: Any equivalent to Ruby's 'hpricot' html/xpath/css selector package?
Mark Thomas wrote: The main difference is that lxml doesn't have CSS selector syntax Feel free to read the docs: http://codespeak.net/lxml/cssselect.html Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Any equivalent to Ruby's 'hpricot' html/xpath/css selector package?
Bruno Desthuilliers wrote: However, what makes it really useful is that it does a good job of handling the broken html that is so commonly found on the web. BeautifulSoup ? http://pypi.python.org/pypi/BeautifulSoup/3.0.7a possibly with ElementSoup ? http://pypi.python.org/pypi/ElementSoup/rev452 It's actually debatable if BS is any better than lxml/libxml2 when parsing broken HTML, as lxml tends to tidy things up pretty well. The only major difference is in encoding detection, for which you can also use a separate tool like chardet: http://chardet.feedparser.org/ Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Any equivalent to Ruby's 'hpricot' html/xpath/css selector package?
Kenneth McDonald wrote: Ruby has a package called 'hpricot' which can perform limited xpath queries, and CSS selector queries. However, what makes it really useful is that it does a good job of handling the broken html that is so commonly found on the web. Does Python have anything similar, i.e. something that will not only do XPath queries, but will do so on imperfect HTML? lxml.html is your friend. http://codespeak.net/lxml/lxmlhtml.html Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: How to debug embeding Python?
Hongtian schrieb: Hi Friends, My application is written in C/C++ and Python is embed to extend some functions (several .py files are invoked). But I am confused how to debug these embed Python? Can I use 'print-debuging'? and where can I capture the output string? Or Python can support 'break' debug for such scenario? You should be able to use the python debugger (pdb) by manually putting import pdb; pdb.set_trace() on a place that is of interest to you. Then if your program is started inside a shell, you should be able to step-debug it. Additinoally, there is the very flexible powerful logging-module that you can use to write logfiles. This is better than simple print-statements, as you can keep the logging statements around and just turn them on or off on demand. Additionally, printing might not work reliably in cases where the stdout/err-streams aren't visible or otherwise in use. But the logging-module can log to files (or even system event logs *me thinks*) Diez -- http://mail.python.org/mailman/listinfo/python-list
Re: Any equivalent to Ruby's 'hpricot' html/xpath/css selector package?
On Dec 30, 8:20 am, Stefan Behnel stefan...@behnel.de wrote: Mark Thomas wrote: The main difference is that lxml doesn't have CSS selector syntax Feel free to read the docs: http://codespeak.net/lxml/cssselect.html Don't know how I missed that... So lxml is pretty much an exact equivalent to what Ruby has to offer (Hpricot or Nokogiri). Nice. -- http://mail.python.org/mailman/listinfo/python-list
Triple quoted string in exec function ?
hello, I'm running scripts, with the execute function (Python 2.5), and it seems that triple quoted strings are not allowed. Is there a workaround, or is this a fundamental problem of the exec-function ? thanks, Stef Mientki -- http://mail.python.org/mailman/listinfo/python-list
select.select and socket.setblocking
I'm using this method to read from a socket: def read_data(self,size): Read data from connection until a given size. res = fd = self.socket.fileno() while not self.stop_requested.isSet(): remaining = size - len(res) if remaining=0: break # Give one second for an incoming connection so we can stop the # server in seconds when needed ready = select.select([fd], [], [], 1) if fd in ready[0]: data = self.socket.recv(min(remaining,8192)) # 8192 is recommended by socket.socket manual. if not data: # select returns the fd but there is no data to read - connection closed! raise TransportError(Connection closed.) else: res += data else: pass if self.stop_requested.isSet(): raise SystemExit(0) return res This works: if I close the socket on the other side, then I see this in the traceback: File /usr/home/gandalf/Python/Projects/OrbToy/orb/endpoint.py, line 233, in read_data raise TransportError(Connection closed.) TransportError: Connection closed. Also when I call stop_requested.set() then the thread stops within one seconds. Then I switch to non blocking mode, my code works exactly the same way, or at least I see no difference. I have read the socket programming howto ( http://docs.python.org/howto/sockets.html#sockets ) but it does not explain how a blocking socket + select is different from a non blocking socket + select. Is there any difference? Thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: Triple quoted string in exec function ?
Stef Mientki wrote: hello, I'm running scripts, with the execute function (Python 2.5), and it seems that triple quoted strings are not allowed. Is there a workaround, or is this a fundamental problem of the exec-function ? If you think about it, it should be obvious that you can't surround a string to be compiled with any of the quotes that appear inside the string to be compiled. That's about the only limitation I am aware of. And, by the way, exec is a *statement*, not a function! exec print '''This is ... a long string''' ... This is a long string regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Python list's mail server in DNSBL ?
I got this message when I tried to send something to this list, through my ISP's SMTP server: snip This Message was undeliverable due to the following reason: Each of the following recipients was rejected by a remote mail server. The reasons given by the server are included to help you determine why each recipient was rejected. Recipient: python-list@python.org Reason:5.7.1 python-list@python.org: Recipient address rejected: policyd-weight Mail appears to be spam or forged. Ask your Mail-/DNS-Administrator to correct HELO and DNS MX settings and to get removed from DNSBLs; in bogusmx.rfc-ignorant.org Please reply to postmas...@chello.at if you feel this message to be in error. Reporting-MTA: dns; viefep19.chello.at Arrival-Date: Tue, 30 Dec 2008 14:12:36 +0100 Received-From-MTA: dns; edge04.upc.biz (192.168.13.239) Final-Recipient: RFC822; python-list@python.org Action: failed Status: 5.1.1 Remote-MTA: dns; mail.python.org (194.109.207.14) Diagnostic-Code: smtp; 550 5.7.1 python-list@python.org: Recipient address rejected: policyd-weight Mail appears to be spam or forged. Ask your Mail-/DNS-Administrator to correct HELO and DNS MX settings and to get removed from DNSBLs; in bogusmx.rfc-ignorant.org /snip Is python.org really blacklisted? Any admin please, try to remove it. Thanks Laszlo -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Laszlo Nagy wrote: I'm using this method to read from a socket: def read_data(self,size): Read data from connection until a given size. res = fd = self.socket.fileno() while not self.stop_requested.isSet(): remaining = size - len(res) if remaining=0: break # Give one second for an incoming connection so we can stop the # server in seconds when needed ready = select.select([fd], [], [], 1) if fd in ready[0]: data = self.socket.recv(min(remaining,8192)) # 8192 is recommended by socket.socket manual. if not data: # select returns the fd but there is no data to read - connection closed! raise TransportError(Connection closed.) else: res += data else: pass if self.stop_requested.isSet(): raise SystemExit(0) return res This works: if I close the socket on the other side, then I see this in the traceback: File /usr/home/gandalf/Python/Projects/OrbToy/orb/endpoint.py, line 233, in read_data raise TransportError(Connection closed.) TransportError: Connection closed. Also when I call stop_requested.set() then the thread stops within one seconds. Then I switch to non blocking mode, my code works exactly the same way, or at least I see no difference. I have read the socket programming howto ( http://docs.python.org/howto/sockets.html#sockets ) but it does not explain how a blocking socket + select is different from a non blocking socket + select. Is there any difference? Well, ignoring your question for the moment, what's wrong with the following (untested) code? def read_data(self, size): data = while len(data) size: d = self.socket.read(size-len(data)) if not d: raise TransportError(Socket closed while reading data) data += d return data I feel the raw socket operations are easier to understand and less confusing. Since you don't want to return until you've read the data there really doesn't seem to be any point using select(), which is mainly useful in asynchronous operations (in which case you would have to use non-blocking sockets). It's not obvious from your code how self.stop_requested is supposed to change the result of its is_set() method. Maybe that's where the select() comes in? regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Python list's mail server in DNSBL ?
Laszlo: Read the message again. There's nothing the list admins can do about this, you'll have to contact postmas...@chello.at to have them remove the blacklisting, since it's their server that's imposing it. regards Steve Laszlo Nagy wrote: I got this message when I tried to send something to this list, through my ISP's SMTP server: snip This Message was undeliverable due to the following reason: Each of the following recipients was rejected by a remote mail server. The reasons given by the server are included to help you determine why each recipient was rejected. Recipient: python-list@python.org Reason:5.7.1 python-list@python.org: Recipient address rejected: policyd-weight Mail appears to be spam or forged. Ask your Mail-/DNS-Administrator to correct HELO and DNS MX settings and to get removed from DNSBLs; in bogusmx.rfc-ignorant.org Please reply to postmas...@chello.at if you feel this message to be in error. Reporting-MTA: dns; viefep19.chello.at Arrival-Date: Tue, 30 Dec 2008 14:12:36 +0100 Received-From-MTA: dns; edge04.upc.biz (192.168.13.239) Final-Recipient: RFC822; python-list@python.org Action: failed Status: 5.1.1 Remote-MTA: dns; mail.python.org (194.109.207.14) Diagnostic-Code: smtp; 550 5.7.1 python-list@python.org: Recipient address rejected: policyd-weight Mail appears to be spam or forged. Ask your Mail-/DNS-Administrator to correct HELO and DNS MX settings and to get removed from DNSBLs; in bogusmx.rfc-ignorant.org /snip Is python.org really blacklisted? Any admin please, try to remove it. Thanks Laszlo -- http://mail.python.org/mailman/listinfo/python-list -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
[ANN] release of CubicWeb 3.0.0
The development team is pleased to announce the release of CubicWeb 3.0.0 (nicknamed ShowTime). What is CubicWeb? - With CubicWeb, the Semantic Web is a construction game! CubicWeb_ is a semantic web application framework, licensed under the LGPL, that empowers developers to efficiently build web applications by reusing components (called cubes) and following the well known object-oriented design principles. Its main features are: * an engine driven by the explicit data model of the application, * a query language named RQL similar to W3C’s SPARQL, * a selection+view mechanism for semi-automatic XHTML/XML/JSON/ text generation, * a library of reusable components (data model and views) called cubes that fulfill common needs, * the power and flexibility of the Python programming language, * the reliability of SQL databases, LDAP directories, Subversion and Mercurial for storage backends. Being built since 2000 by an RD project still going on today, supporting 100,000s of daily visits at some production sites, CubicWeb is a proven end to end solution for semantic web application development that promotes quality, reusability and efficiency. The unbeliever will read the quick overview_ of CubicWeb. The hacker will join development at the forge_. The impatient will move right away to installation_ and set-up of a CubicWeb environment. .. _cubicweb: http://www.cubicweb.org/ .. _overview: http://www.cubicweb.org/doc/en/A020-tutorial.en.html#overview .. _forge: http://www.cubicweb.org/project?vtitle=All%20cubicweb%20projects .. _installation: http://www.cubicweb.org/doc/en/C010-setup.en.html#miseenplaceenv Home page - http://www.cubicweb.org/ Download http://ftp.logilab.org/pub/cubicweb/ Mailing list http://lists.cubicweb.org/mailman/listinfo/cubicweb Licence --- LGPL - Lesser General Public Licence -- http://mail.python.org/mailman/listinfo/python-list
Re: Python module import loop issue
En Tue, 30 Dec 2008 01:32:48 -0200, Carl Banks pavlovevide...@gmail.com escribió: Gabriel Genellina wrote: A problem with metaclasses is when you have intermediate subclasses that are not meant to be registered, but the metaclass applies equally to all of them. Not the way I wrote it. If you'll note, the metaclass only added the class to the factory map if a tag attribute was defined in the class dict. Ah, I didn't notice that, sorry! -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: why cannot assign to function call
On Tue, 30 Dec 2008, Aaron Brady wrote: [...] On a technicality, to avert a flaming, change the value of 'b' is an ambiguous phrase. There are two interpretations of change what 'b' refers to and change what 'b' refers to. Even in spoken language, I don't think that emphasis can resolve them either. One means, 'make a change in the world, in the actual configuration of such and such actual matter.' The other means, 'update the axioms the speaker is using to communicate to the listeners. (Such and such will no longer refer to such and such; it will refer to such and such; accept this and reject that.)' To make an observation, reference is a purely linguistic phenomenon. I, for one, am at a loss for how to disambiguate it. I'm open to suggestions. I think you've already done so quite clearly. But I suspect the ambiguity centres on the word change - in the first interpretation it means alter (the object in place), in the second, it more precisely means exchange (one object for another). I'd be interested to know to what extent this ambiguity exists in languages other than English - as a somewhat relevant example, ambiguities in English around the verb to be (I am Fred, I am tall, I am hungry; i.e., identity vs. attributes vs. state) disappear in other languages which use different verbs in each case. On a (philosophical) side-note, I wonder if this is a real ambiguity: while the statement that the name b now refers to a different object is clear-cut, the other interpretation, that the object itself has changed but is somehow still the same object, is a bit of an ontological minefield. Fortunately, unlike the murky world of philosophy, Python (AIUI) simplifies this question by simply declaring that yes, in the case of mutable objects, we may say that we are still referring to the same object although we've changed it, and no, in the case of immutable objects, we may not, and must exchange it if we want a different value (a word too fraught with ambiguity in this context to use unquoted!). The sometimes useful fuzziness of human languages means we don't need to ask, for example, how tall is the object currently referred to by the name Fred?, but in Python, it helps to understand that this is what's going on. Regards, John -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing vs thread performance
On Dec 29, 9:08 pm, James Mills prolo...@shortcircuit.net.au wrote: On Tue, Dec 30, 2008 at 12:52 PM, Aaron Brady castiro...@gmail.com wrote: On Dec 29, 7:40 pm, James Mills prolo...@shortcircuit.net.au wrote: On Tue, Dec 30, 2008 at 11:34 AM, Aaron Brady castiro...@gmail.com wrote: The OP may be interested in Erlang, which Wikipedia (end-all, be-all) claims is a 'distribution oriented language'. snip I'm presently looking at Virtual Synchrony and other distributed processing architectures - but circuits is meant to be general purpose enough to fit event-driven applications/systems. I noticed a while ago that threads can be used to simulate generators. 'next', 'send', and 'yield' are merely replaced by synchronizor calls (coining the term). Not the other way around, though, unless the generator guarantees a yield frequently. 'settrace' anyone? Aaron, circuits doesn't use generators :) What did your comment have to do with this ? I have often seen generators used to facilitate coroutine and coooperative programming though. cheers James James, Hi. I'm glad you asked; I never know how out there my comments are (but surmise that feedback is always a good thing). What I was thinking was, I didn't know Virtual Synchrony, and I've never used Erlang, but I'm interested in concurrency especially as it pertains to units of work, division of labor, and division of context; and generators are another way to divide context. So: I wanted to put more of my background and interests on the table. What I said wasn't directly relevant, I see. But it's not like I dissertated (discussed) the Tibettan-Austrian spice trade. I think I just want to say stuff about threading! Maybe I'm just excited to meet people who share my interests... not unheard of. In Economics, you can divide a market into vertical and horizontal dimensions, vertical being the chain from raw resources to finished products, and horizontal being market coverage. With a similar division in tasks, horizontal units would handle incoming events, prepare them, then pass them on to a next unit, which processes a little, then passes it on, like an assembly line (bucket brigade/ alternating current); and vertical units would take one incoming event, and see it through start to finish (direct current). You don't have to use that depiction. The terminology is transposed from that of a distributed matrix multiplication task depiction, where horizontal works start-to-finish, and vertical takes handoffs from its predecessor. 'Circuits' doesn't use generators. I think generators are an underexplored technique. Is 'circuits' assembly line or start-to- finish, if my analogy makes any sense? 'Circuits' is event-driven, but I don't see any difference between 'event-driven' and multithreaded in general. (I think contrast can create a good picture and a clear understanding.) What is special about an 'event-driven' architecture? Are you distinguishing blocking from polling? -- http://mail.python.org/mailman/listinfo/python-list
Re: Python list's mail server in DNSBL ?
Steve Holden wrote: Laszlo: Read the message again. There's nothing the list admins can do about this, you'll have to contact postmas...@chello.at to have them remove the blacklisting, since it's their server that's imposing it. Maybe it is my bad English but this part: Ask your Mail-/DNS-Administrator to correct HELO and DNS MX settings and to get removed from DNSBLs; in bogusmx.rfc-ignorant.org Please reply to postmas...@chello.at if you feel this message to be in error. tells that the problem is with the MX record of python.org and/or the HELO message of the python.org smtp server. At least one of them is said to be RFC ignorant, and this is why the recipient is on the blacklist. Without making those changes, the python.org domain will not be removed from the blacklist. Changing the MX record / configuring the SMTP server needs to be done by the DNS or SMTP admin of python.org. At least I think if I ask chello.at to remove the entry, the first thing they will do is check if the changes has been made. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python list's mail server in DNSBL ?
Ask your Mail-/DNS-Administrator to correct HELO and DNS MX settings and to get removed from DNSBLs; in bogusmx.rfc-ignorant.org Please reply to postmas...@chello.at if you feel this message to be in error. I went to rfc-ignorant.org and looked up python.org. python.org was in their database from 2004-08-27 to 2005-11-09. It's not been there since. My guess is that chello.at is having problems interpreting rfc-ignorant.org records. -- Skip Montanaro - s...@pobox.com - http://smontanaro.dyndns.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Triple quoted string in exec function ?
On Dec 30, 2:48 pm, Steve Holden st...@holdenweb.com wrote: Stef Mientki wrote: hello, I'm running scripts, with the execute function (Python 2.5), and it seems that triple quoted strings are not allowed. Is there a workaround, or is this a fundamental problem of the exec-function ? If you think about it, it should be obvious that you can't surround a string to be compiled with any of the quotes that appear inside the string to be compiled. That's about the only limitation I am aware of. And, by the way, exec is a *statement*, not a function! exec print '''This is ... a long string''' ... This is a long string regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ hello, the message Steven sent you is ok to explain how to work with triple quote Cheers. -- http://mail.python.org/mailman/listinfo/python-list
thread, multiprocessing: communication overhead
Hello everyone, This time I decided to test communication overhead in multithreaded / multiprocess communication. The results are rather disappointing, that is, communication overhead seems to be very high. In each of the following functions, I send 10,000 numbers to the function / 10 threads / 10 processes, which simply returns it in its respective way. Function: notfunBest: 0.00622 sec Average: 0.00633 sec (simple function) Function: threadsemfun Best: 0.64428 sec Average: 0.64791 sec (10 threads synchronizing using semaphore) Function: threadlockfun Best: 0.66288 sec Average: 0.66453 sec (10 threads synchronizing using locks) Function: procqueuefun Best: 1.16291 sec Average: 1.17217 sec (10 processes communicating with main process using queues) Function: procpoolfun Best: 1.18648 sec Average: 1.19577 sec (a pool of 10 processes) If I'm doing smth wrong in the code below (smth that would result in performance suffering), please point it out. Code: import threading import multiprocessing import time import timeit def time_fun(fun): t = timeit.Timer(stmt = fun, setup = from __main__ import + fun.__name__) results = t.repeat(repeat=10, number=1) best_result = min(results) avg = sum(results) / len(results) print Function: %-15s Best: %5.5f sec Average: %5.5f sec % (fun.__name__, best_result, avg) def notfun(): inputlist = range(0,1) reslist = [] for x in range(len(inputlist)): reslist.append(inputlist.pop()) def threadsemfun(): def tcalc(sem, inputlist, reslist, tid, activitylist): while len(inputlist) 0: sem.acquire() try: x = inputlist.pop() except IndexError: sem.release() return #activitylist[tid] += 1 reslist.append(x) sem.release() inputlist = range(0,1) #print before: , sum(inputlist) reslist = [] tlist = [] activitylist = [ 0 for x in range(0,10) ] sem = threading.Semaphore() for t in range(0,10): tlist.append(threading.Thread(target=tcalc, args=(sem, inputlist, reslist, t, activitylist))) for t in tlist: t.start() for t in tlist: t.join() #print after: , sum(reslist) #print thread action count:, activitylist def threadlockfun(): def tcalc(lock, inputlist, reslist, tid, activitylist): while True: lock.acquire() if len(inputlist) == 0: lock.release() return x = inputlist.pop() reslist.append(x) #activitylist[tid] += 1 lock.release() inputlist = range(0,1) #print before: , sum(inputlist) reslist = [] tlist = [] activitylist = [ 0 for x in range(0,10) ] sem = threading.Semaphore() for t in range(0,10): tlist.append(threading.Thread(target=tcalc, args=(sem, inputlist, reslist, t, activitylist))) for t in tlist: t.start() for t in tlist: t.join() #print after: , sum(reslist) #print thread action count:, activitylist def pf(x): return x def procpoolfun(): pool = multiprocessing.Pool(processes=10) inputlist = range(0,1) reslist = [] i, j, jmax = 0, 10, len(inputlist) #print before: , sum(inputlist) while j = jmax: res = pool.map_async(pf, inputlist[i:j]) reslist.extend(res.get()) i += 10 j += 10 #print after: , sum(reslist) def procqueuefun(): def pqf(qin, qout): pid = multiprocessing.current_process().pid while True: x = qin.get() if x == 'STOP': return qout.put((pid, x)) qin = multiprocessing.Queue() qout = multiprocessing.Queue() plist = [] activity = dict() for i in range(0,10): p = multiprocessing.Process(target = pqf, args=(qin, qout)) p.start() plist.append(p) activity[p.pid] = 0 inputlist = range(0,1) reslist = [] #print before:, sum(inputlist) ilen = len(inputlist) x = 0 while x != ilen: for i in range(0,10): qin.put(inputlist[x+i]) for i in range(0,10): pid, res =
Re: why cannot assign to function call
On Dec 30, 8:21 am, John O'Hagan m...@johnohagan.com wrote:
On Tue, 30 Dec 2008, Aaron Brady wrote:
[...]
On a technicality, to avert a flaming, change the value of 'b' is an
ambiguous phrase. There are two interpretations of change what 'b'
refers to and change what 'b' refers to. Even in spoken language,
I don't think that emphasis can resolve them either.
One means, 'make a change in the world, in the actual configuration of
such and such actual matter.' The other means, 'update the axioms the
speaker is using to communicate to the listeners. (Such and such will
no longer refer to such and such; it will refer to such and such;
accept this and reject that.)' To make an observation, reference is a
purely linguistic phenomenon.
I, for one, am at a loss for how to disambiguate it. I'm open to
suggestions.
I think you've already done so quite clearly. But I suspect the ambiguity
centres on the word change - in the first interpretation it means alter
(the object in place), in the second, it more precisely means exchange (one
object for another).
I'd be interested to know to what extent this ambiguity exists in languages
other than English - as a somewhat relevant example, ambiguities in English
around the verb to be (I am Fred, I am tall, I am hungry; i.e.,
identity vs. attributes vs. state) disappear in other languages which use
different verbs in each case.
On a (philosophical) side-note, I wonder if this is a real ambiguity: while
the statement that the name b now refers to a different object is
clear-cut, the other interpretation, that the object itself has changed but
is somehow still the same object, is a bit of an ontological minefield.
Fortunately, unlike the murky world of philosophy, Python (AIUI) simplifies
this question by simply declaring that yes, in the case of mutable objects,
we may say that we are still referring to the same object although we've
changed it, and no, in the case of immutable objects, we may not, and must
exchange it if we want a different value (a word too fraught with ambiguity
in this context to use unquoted!).
The sometimes useful fuzziness of human languages means we don't need to ask,
for example, how tall is the object currently referred to by the name
Fred?, but in Python, it helps to understand that this is what's going on.
Regards,
John
There are a few problems going on here. (My ultimate goal is a
unambiguous, non-arbitrary way to talk about Python; it's just that
unambiguous and non-arbitrary are really tall orders.) One is the
dependence (supervenience) of meaning on grammar (semantics on syntax,
function on form). One is the absence of form in computing and math.
Another is the composite identity.
I think the problem goes deeper than just English. In any language
that has a plural, the propositions in question come out as, 'one
thing is two things' or 'two things are one thing'. According to some
rules, these are ungrammatical sentences, due to plurality
disagreement. Ex:
The Morning Star is ...
The Evening Star is ...
*The Morning Star and The Evening Star is...
*The Morning Star and The Evening Star are...
Neither of the latter two is correct. (* marks ungrammatical.) As
such, the listener isn't sure what meaning to take.
Accepting that, I'll adopt the terms John proposed, 'change' vs.
'exchange', the former when the material configuration changes, the
latter when the communication axioms change.
b= [2, 3]
b= [3, 4]
'b' has exchanged. (Somewhat ungrammatical.)
b= [2, 3]
b.append( 4 )
'b' has changed.
According to this, when you replace every floorboard on a porch, one
at a time, it's still the same porch-- it's changed, it's different,
and it's the same. However, if you haul off the porch and put a new
one in its place, it's not. ('porch[:]=' vs. 'porch='.) You can't
just look at the initial and final configurations of matter to
determine so. Therefore, 'id' is an informal function.
Identity isn't defined on math objects, only on Python objects; there
is no notion of 'is' in math. It doesn't make sense to ask if '2 is
2', only if '2=2'. In Python, they are both defined, and not the
same. Unfortunately, in the language involved, 'same' and 'different'
aren't opposites, so the communication is horrendous and belabored.
Lastly, equality is a tolerance concept on the computer, not in math.
The computer looks at voltage thresholds in the underlying gates; the
actual voltages, beyond a certain precision, of two representations of
'0110' aren't the same. Further, the actual voltages of one
representation of '0110' over time aren't equal either.
However, that means, that in order to answer some questions about
equality and identity, we have to address the implementation of the
compiler. You might get the same answers in many or practically all
implementations of Python, but the specification doesn't make the leap
onto a real computer, where the equivalents of continuity of
Parsing Excel spreadsheets
Hi, Can anybody recommend an approach for loading and parsing Excel spreadsheets in Python. Any well known/recommended libraries for this? The only thing I found in a brief search was http://www.lexicon.net/sjmachin/xlrd.htm, but I'd rather get some more input before going with something I don't know. Thanks, Andy. -- http://mail.python.org/mailman/listinfo/python-list
Re: thread, multiprocessing: communication overhead
On Dec 30, 9:46 am, mk mrk...@gmail.com wrote: Hello everyone, This time I decided to test communication overhead in multithreaded / multiprocess communication. The results are rather disappointing, that is, communication overhead seems to be very high. In each of the following functions, I send 10,000 numbers to the function / 10 threads / 10 processes, which simply returns it in its respective way. Function: notfun Best: 0.00622 sec Average: 0.00633 sec (simple function) Function: threadsemfun Best: 0.64428 sec Average: 0.64791 sec (10 threads synchronizing using semaphore) Function: threadlockfun Best: 0.66288 sec Average: 0.66453 sec (10 threads synchronizing using locks) Function: procqueuefun Best: 1.16291 sec Average: 1.17217 sec (10 processes communicating with main process using queues) Function: procpoolfun Best: 1.18648 sec Average: 1.19577 sec (a pool of 10 processes) If I'm doing smth wrong in the code below (smth that would result in performance suffering), please point it out. snips def threadsemfun(): sem = threading.Semaphore() def threadlockfun(): sem = threading.Semaphore() You used a Semaphore for both lock objects here. 'multiprocessing' is a really high level layer that makes a lot of decisions about trade-offs, has highly redundant communication, and is really easy to use. If you want to save a byte, you'll have to make your own decisions about trade-offs and redundancies (possibly even looking at real result data to make them). I actually think 'multiprocessing' is really good, and even if I hand- wrote my own IPC, it would be slower! CMIIW, but I believe your timing function includes the time to launch the actual processes and threads, create the synch. objects, etc. You might try it again, creating them first, starting the timer, then loading them. -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Excel spreadsheets
On Dec 30, 10:07 am, andyh...@gmail.com andyh...@gmail.com wrote: Hi, Can anybody recommend an approach for loading and parsing Excel spreadsheets in Python. Any well known/recommended libraries for this? The only thing I found in a brief search washttp://www.lexicon.net/sjmachin/xlrd.htm, but I'd rather get some more input before going with something I don't know. Thanks, Andy. xlrd(read only) also see pyExcelerator -- http://mail.python.org/mailman/listinfo/python-list
Re: thread, multiprocessing: communication overhead
Aaron Brady wrote: snips def threadsemfun(): sem = threading.Semaphore() def threadlockfun(): sem = threading.Semaphore() You used a Semaphore for both lock objects here. Right... I corrected that (simply changed to threading.Lock() in threadlockfun) and the result is much better, though still an order of magnitude worse than plain function: Function: threadlockfun Best: 0.08665 sec Average: 0.08910 sec Function: notfunBest: 0.00987 sec Average: 0.01003 sec 'multiprocessing' is a really high level layer that makes a lot of decisions about trade-offs, has highly redundant communication, and is really easy to use. If you want to save a byte, you'll have to make your own decisions about trade-offs and redundancies (possibly even looking at real result data to make them). Hmm, do you think that lower-level 'thread' module might work more efficiently? I actually think 'multiprocessing' is really good, and even if I hand- wrote my own IPC, it would be slower! CMIIW, but I believe your timing function includes the time to launch the actual processes and threads, create the synch. objects, etc. You might try it again, creating them first, starting the timer, then loading them. Except I don't know how to do that using timeit.Timer. :-/ -- http://mail.python.org/mailman/listinfo/python-list
Re: thread, multiprocessing: communication overhead
mk mrk...@gmail.com wrote: This time I decided to test communication overhead in multithreaded / multiprocess communication. The results are rather disappointing, that is, communication overhead seems to be very high. In each of the following functions, I send 10,000 numbers to the function / 10 threads / 10 processes, which simply returns it in its respective way. Function: notfunBest: 0.00622 sec Average: 0.00633 sec (simple function) Function: threadsemfun Best: 0.64428 sec Average: 0.64791 sec (10 threads synchronizing using semaphore) Function: threadlockfun Best: 0.66288 sec Average: 0.66453 sec (10 threads synchronizing using locks) Function: procqueuefun Best: 1.16291 sec Average: 1.17217 sec (10 processes communicating with main process using queues) Function: procpoolfun Best: 1.18648 sec Average: 1.19577 sec (a pool of 10 processes) If I'm doing smth wrong in the code below (smth that would result in performance suffering), please point it out. You aren't just timing the communication overhead: each of your functions creates a pool of 10 threads or 10 processes on every run, so your times include the startup and shutdown times. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python in C
Hello Skip, Thank you for your response. Your posting reminds me that we, Python community as a whole, owe a great deal to Python developers. The problem is ... The more you work on Python, the harder you can go back to C or C++ world. I use SWIG, instead. I think SWIG is a good way to mix two worlds. Aki- On Dec 30, 4:10 am, s...@pobox.com wrote: aki Although this is not what you are asking but I'm wondering why you aki need to read CPython implementation. A couple reasons come to mind: * education * want to make it better (extend it, fix bugs, etc) * want to see how it relates to the implementation of other languages (Ruby, Perl, Tcl, etc) aki CPython worked great for me. I don't want to read a large piece of aki software, like CPython, unless it is really really necessary. Sure, but not everyone works at the Python layer. Which is a good thing, because if everyone confined themselves to Python code nobody would fix problems in the language implementation or enhance it. -- Skip Montanaro - s...@pobox.com -http://smontanaro.dyndns.org/ -- http://mail.python.org/mailman/listinfo/python-list
Re: SQL, lite lite lite
On Dec 29, 1:06 pm, Aaron Brady castiro...@gmail.com wrote: Hi all, About a year ago, I posted an idea I was having about thread synchronization to the newsgroup. However, I did not explain it well, and I really erred on the side of brevity. (After some finagling, Mr. Bieber and I decided it wasn't exactly anything groundbreaking.) But I think the brevity cost me some readers, who might have had more interest. The affair was on the whole discouraging. So, I'm going to try another idea, and assume that readers have some time, and will spend it on it. I don't think relational data can be read and written very easily in Python. There are some options, such as 'sqllite3', but they are not easy. 'sqllite3' statements are valid SQL expressions, which afford the entire power of SQL, but contrary to its name, it is not that 'lite'. To me, 'lite' is something you could learn (even make!) in an afternoon, not a semester; something the size of an ActiveState recipe, or a little bigger, maybe a file or two. If you think SQL is a breeze, you probably won't find my idea exciting. I assume that the basics of SQL are creating tables, selecting records, and updating records. My idea is to create a 'Relation' class. The details are basically open, such as whether to back it with 'sqllite3', 'shelve', 'mmap', or just mapping and sequence objects; what the simplest syntax is that can capture and permit all the basics, and how much and what else can fit in at that level; how and whether it can include arbitrary Python objects, and what constraints there are on them if not; how and whether to permit transactions; and what the simplest and coolest thing you can do with a little Python syntax is. This is basically an invitation for everyone to brainstorm. (No hijackings, good humor digression ok.) Lastly, ... **warning, spoiler! here's what I thought of already.** **repeat! spoiler! here's what I thought of already.** #Just the select and update syntax: a= people._select( firstname== 'Joe' ) #select 'key' from 'people' where 'firstname'== 'joe' a [Entry2864, Entry3076, Entry3172] entry1= a[ 0 ] entry1.phone #select 'phone' from 'people' where 'key'==self.key 555-2413 entry1.phone= 555-1234 #update 'people' set 'phone'= '555-1234' where 'key'==self.key entry1.phone 555-1234 #Create table syntax (a-whole-nother beast in itself): classes= db.Relation( 'class_', 'person', Unique( 'class_', 'person' ) ) #create table 'classes' ( 'key', 'class_', 'person' ) unique ( 'class_', 'person' ) classes._unique( 'class_', 'person' ) classes.class_.noneok= False #'class_' cannot be null classes.person.noneok= False classes._insert( 'Physics', 'Dan' ) classes._insert( 'Chem', 'Tim' ) Hoping-good critic-is-self-consistent-ly, hoping-to-hear-it's-too- complicated-already-ly, A. Brady You really do like to reinvent the wheels do you? :-) Nothing wrong with that. Just be aware that most people that really need what you are proposing are probably already using mature feature rich libraries for that. http://wiki.python.org/moin/HigherLevelDatabaseProgramming -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Excel spreadsheets
r wrote: On Dec 30, 10:07 am, andyh...@gmail.com andyh...@gmail.com wrote: Hi, Can anybody recommend an approach for loading and parsing Excel spreadsheets in Python. Any well known/recommended libraries for this? The only thing I found in a brief search washttp://www.lexicon.net/sjmachin/xlrd.htm, but I'd rather get some more input before going with something I don't know. Thanks, Andy. xlrd(read only) also see pyExcelerator which is now replaced by xlwt :-) Cheers Tino smime.p7s Description: S/MIME Cryptographic Signature -- http://mail.python.org/mailman/listinfo/python-list
python import sys.path
I have both 2.4 and 2.5 interpreters installed on a linux box. The PythonPath is set to : PYTHONPATH=/usr/lib64/portage/pym:/prod/bacula/local/lib64/python2.4/site-pa ckages:/prod/bacula/local/lib/python2.4/site-packages My main script is getting called like so: python2.4 cleanup.py wrkstnbs The imports statements in cleanup.py are as follows: import os,sys print sys.path from datetime import datetime from optparse import OptionParser# used for parsing parameters from bacula_conf import *# used for connecting to our databases, etc. from registration_cleanup \ import RegistrationCleanup # used for interacting w/ registration db (sql1) # and configuration database (genunix) import directory_cleanup as fclean # file cleanup. One of the scripts being imported from bacula_conf is called purge_client.py. It has the following imports: import sys import MySQLdb Everytime I run python2.4 cleanup.py wrkstnbs I get the following error: Traceback (most recent call last): File purge_client.py, line 22, in module import MySQLdb File /prod/bacula/local/lib64/python2.4/site-packages/MySQLdb/__init__.py, line 27, in module import _mysql ImportError: /prod/bacula/local/lib64/python2.4/site-packages/_mysql.so: undefined symbol: Py_InitModule4 If I print sys.path of both the calling script and the imported module I get the following sys.path from cleanup.py: ['/prod/bacula/local/tools/bacula_conf/maintenance', '/usr/lib64/portage/pym', '/prod/bacula/local/lib64/python2.4/site-packages', '/prod/bacula/local/lib/python2.4/site-packages', '/usr/lib/python24.zip', '/usr/lib/python2.4', '/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib-tk', '/usr/lib64/python2.4/lib-dynload', '/usr/lib/portage/pym', '/usr/lib64/python2.4/site-packages', '/usr/lib/python2.4/site-packages'] However the sys.path printed by the called module purge_client.py displays as such: ['/prod/bacula/local/tools/bacula_conf/maintenance', '/usr/lib64/portage/pym', '/prod/bacula/local/lib64/python2.4/site-packages', '/prod/bacula/local/lib/python2.4/site-packages', '/usr/lib64/python25.zip', '/usr/lib64/python2.5', '/usr/lib64/python2.5/plat-linux2', '/usr/lib64/python2.5/lib-tk', '/usr/lib64/python2.5/lib-dynload', '/usr/lib64/python2.5/site-packages'] If I call purge_client.py like so: python2.4 purge_client.py wrkstnbs the sys.path is: ['/prod/bacula/local/tools/bacula_conf/maintenance', '/usr/lib64/portage/pym', '/prod/bacula/local/lib64/python2.4/site-packages', '/prod/bacula/local/lib/python2.4/site-packages', '/usr/lib/python24.zip', '/usr/lib/python2.4', '/usr/lib/python2.4/plat-linux2', '/usr/lib/python2.4/lib-tk', '/usr/lib64/python2.4/lib-dynload', '/usr/lib/portage/pym', '/usr/lib64/python2.4/site-packages', '/usr/lib/python2.4/site-packages'] Can anyone explain why purge_client.py has a different sys.path when imported from another script? At this point I'm quite puzzled and would like to continue using the 2.4 interpreter for the time being. Thanks, Brian Kelly smime.p7s Description: S/MIME cryptographic signature -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
Laszlo Nagy ha scritto: I'm using this method to read from a socket: def read_data(self,size): Read data from connection until a given size. res = fd = self.socket.fileno() while not self.stop_requested.isSet(): remaining = size - len(res) if remaining=0: break # Give one second for an incoming connection so we can stop the # server in seconds when needed ready = select.select([fd], [], [], 1) if fd in ready[0]: data = self.socket.recv(min(remaining,8192)) # 8192 is recommended by socket.socket manual. if not data: # select returns the fd but there is no data to read - connection closed! raise TransportError(Connection closed.) else: res += data else: pass if self.stop_requested.isSet(): raise SystemExit(0) return res This works: if I close the socket on the other side, then I see this in the traceback: File /usr/home/gandalf/Python/Projects/OrbToy/orb/endpoint.py, line 233, in read_data raise TransportError(Connection closed.) TransportError: Connection closed. Also when I call stop_requested.set() then the thread stops within one seconds. Then I switch to non blocking mode, my code works exactly the same way, or at least I see no difference. I have read the socket programming howto ( http://docs.python.org/howto/sockets.html#sockets ) but it does not explain how a blocking socket + select is different from a non blocking socket + select. Is there any difference? Thanks Couple of remarks: 1. AFAIK, select in python accepts also socket objects, or anything which has a fileno() method returning an integer. So you don't need to extract the fileno from the socket (python will do for you) although it does no harm. 2. IMO, the behaviour of your code is correct: with TCP protocol, when the remote ends disconnects, your end receives a 'read event' without data; you should just handle the fact that recv returns nothing as normal, not as error, and close your end of the connection. If you are interested in socket errors, you should also fill the third 'fd-set' in the select call, and after select returns check that fd is not in it anymore: ready = select.select( [fd],[], [fd] ) if fd in ready[2]: # raise your error here 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, or the disconnection, in the other case would return immediately (and you should check the number of returned bytes, and when you read the remaining bytes of the message put the pieces together). I myself tend to avoid using non-blocking sockets, since blocking sockets are much easier to handle... HTH Ciao -- FB -- http://mail.python.org/mailman/listinfo/python-list
duck typing at will
Duck typing is called that way because If it looks like a duck and
quacks like a duck, it must be a duck. I think it would be good to
have also If the programmer wants to deal with it like a duck, it
must be a duck
I mean, some tasks are rather boring in python when compared with php,
for example, let's imagine we have a dictionary that contains
dictionaries that contain the times that a key appears. We, or at
least I, would like to write something as short as:
dict[k1][k2] += 1
However we will have to do a longer code (this is the smallest code I
could come up with):
dict = {}
if not k1 in dict:
dict[k1] = {}
if not k2 in dict[k1]:
dict[k1][k2] = 0
dict[k1][k2] += 1
I know it is not the Apocalypse nor something really important when
compared with other things, but maybe it would be better if it wasn't
necessary to declare the variables when they are built-in types or to
initialize the keys in a dictionary, having special values (the
identity element of the operation that causes the initialization) to
initialize when it has not been done, initializing with the most
general type that supports the operation that causes the
initialization, casting to other type if necessary after that.
This is just an idea, maybe I'm not the first one suggesting it, or
maybe it is completely impossible to implement it because it would
require deep changes in python, but I wanted to discuss it.
Best regards.
--
http://mail.python.org/mailman/listinfo/python-list
Re: wxPython.button.disabled still catching clicks
On Dec 30, 3:04 am, mynthon mynth...@gmail.com wrote:
On Dec 23, 6:12 pm, Mike Driscoll kyoso...@gmail.com wrote:
On Dec 23, 7:27 am,mynthonmynth...@gmail.com wrote:
On Dec 23, 11:58 am, Aaron Brady castiro...@gmail.com wrote:
On Dec 23, 4:50 am,mynthonmynth...@gmail.com wrote:
Hello! (sorry for my english)
I have a problem with buttons in wxPython. When button is disabled
(by .Disable() or .Enable(False)) it is grayed out but still receive
clicks.
Eg. i have button that disable itself, runs long action and enable
itself:
def onClick(self, evt):
self.btn.Enable(False)
for i in range (1000):
print i
self.btn.Enable(True)
when for loop is running button is greyed out and when i click on it
nothing happens but when loop ends another one is started because
button remebered thad i click on it when was diabled. My only idea
is to reposition button outside frame instead of disabling it but this
solution is...not good.
thanks for any help. Ive searched groups, google and it looks that
only i have this problem :)
No, it is very common. During your for loop, the loop is dominating
the process completely. Events are just building up in the app's
message queue, and don't get handled until after you yield on control.
If you need to run a long task, look into threading, the OnIdle
method, the 'multiprocessing' module, or pump messages during your
long task.
ok, maybe someone will need it. I dont know how it works because i
didnt have time to read docs and i cannot explain everything. I used
google and wxPython demo (in tree: wxpython overview / process and
events / process)
class leftPanel(wx.Panel):
def __init__(self, parent, id):
wx.Panel.__init__(self, parent, id, style=wx.BORDER_SUNKEN)
# here you have to define new process, IDLE event, and
onPRocessEnd event
self.process = None
self.GetParent().Bind(wx.EVT_IDLE, self.onIdle)
self.Bind(wx.EVT_END_PROCESS, self.onProcessEnd)
# create button and bind event to it
self.runScriptBtn = wx.Button(self, -1, 'RUN ME!', (10,220))
self.runScriptBtn.Bind(wx.EVT_BUTTON, self.onClick,
self.runScriptBtn)
def onClick(self, evt):
# disable button
self.runScriptBtn.Enable(False)
# here you have to enter command to run
# previusly i heve here exec('pythonmyScript.py')
# but now it will be a subprocess
cmd = 'pythonxxx1.py'
#create new process
self.process = wx.Process(self)
# dont know what it is for
self.process.Redirect()
# execute cmd command
pid = wx.Execute(cmd, wx.EXEC_ASYNC, self.process)
def onIdle(self, evt):
# beacuse this method is called only when app enters idle mode
# the line below is nedded to simulate entering idle mode
# dont know how it works but it works
evt.RequestMore(True)
# here is some code to catch subprocess output
if self.process is not None:
stream = self.process.GetInputStream()
if stream.CanRead():
text = stream.read()
print text
def onProcessEnd(self, evt):
# here is some code to catch subprocess output
# when it is destroyed
stream = self.process.GetInputStream()
if stream.CanRead():
text = stream.read()
print text
# dont know it is necessary
self.process.CloseOutput()
# remove (clear) process object
self.process.Destroy()
self.process = None
# show button again
self.runScriptBtn.Enable()
I'm pretty sure there's a better way to do this. If you disable the
button and click on it, you'll notice that it isn't firing events, so
something else is going on here. It seems like the wx.Frame or
wx.Application is queuing the mouse button clicks or something. I
would post to the wxPython mailing list:http://wxpython.org/maillist.php
They'll be better able to address this and they'll probably have a
simpler solution too.
Mike
I changed it. Now i'm running separate thread instead of process.
First example at:http://wiki.wxpython.org/LongRunningTasks
I should have posted that link when I posted originally. Oh well. Glad
you found a workaround.
Mike
--
http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Excel spreadsheets
Tino Wildenhain wrote: r wrote: On Dec 30, 10:07 am, andyh...@gmail.com andyh...@gmail.com wrote: Hi, Can anybody recommend an approach for loading and parsing Excel spreadsheets in Python. Any well known/recommended libraries for this? The only thing I found in a brief search washttp://www.lexicon.net/sjmachin/xlrd.htm, but I'd rather get some more input before going with something I don't know. Thanks, Andy. xlrd(read only) also see pyExcelerator which is now replaced by xlwt :-) I've had very good results with xlrd, having used it to extract data from spreadsheets for storage into a relational database. It's also cross-platform, which means you don't need to run under Windows. I don't know anything at all about xlwt. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: folder extraction
On Dec 30, 9:30 am, ibpe...@gmail.com wrote: how do i get along with this task of extracting multiples folder and generating their names individually in a their respective files as they were generated. Are you talking about unzipping an archive or walking a directory? If the former, see the zip module or use the subprocess module to control an external archiving application. If the latter, see os.walk and the glob module. Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Excel spreadsheets
On Dec 30, 10:07 am, andyh...@gmail.com andyh...@gmail.com wrote: Hi, Can anybody recommend an approach for loading and parsing Excel spreadsheets in Python. Any well known/recommended libraries for this? The only thing I found in a brief search washttp://www.lexicon.net/sjmachin/xlrd.htm, but I'd rather get some more input before going with something I don't know. Thanks, Andy. If you need complete control of Excel, then you'll probably have to use COM (see Mark Hammond's PyWin32 package). Otherwise, the suggestions by the others will work. Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: duck typing at will
Jose Mora wrote:
Duck typing is called that way because If it looks like a duck and
quacks like a duck, it must be a duck. I think it would be good to
have also If the programmer wants to deal with it like a duck, it
must be a duck
I mean, some tasks are rather boring in python when compared with php,
for example, let's imagine we have a dictionary that contains
dictionaries that contain the times that a key appears. We, or at
least I, would like to write something as short as:
dict[k1][k2] += 1
However we will have to do a longer code (this is the smallest code I
could come up with):
dict = {}
if not k1 in dict:
dict[k1] = {}
if not k2 in dict[k1]:
dict[k1][k2] = 0
dict[k1][k2] += 1
I know it is not the Apocalypse nor something really important when
compared with other things, but maybe it would be better if it wasn't
necessary to declare the variables when they are built-in types or to
initialize the keys in a dictionary, having special values (the
identity element of the operation that causes the initialization) to
initialize when it has not been done, initializing with the most
general type that supports the operation that causes the
initialization, casting to other type if necessary after that.
This is just an idea, maybe I'm not the first one suggesting it, or
maybe it is completely impossible to implement it because it would
require deep changes in python, but I wanted to discuss it.
You could use defaultdict like this:
from collections import defaultdict
my_dict = defaultdict(lambda: defaultdict(int))
my_dict[k1][k2] += 1
The disadvantage is that my_dict will then return values even when you
don't want it to:
print my_dict[unknown_key]
defaultdict(type 'int', {})
so after you've filled it you might want to turn it into a 'normal' dict
of dict:
my_dict = dict((key, dict(inner.iteritems())) for key, inner in
my_dict.iteritems())
--
http://mail.python.org/mailman/listinfo/python-list
Re: duck typing at will
In article
6e1bfdea0812301042x70ab57capf99ce73d364d5...@mail.gmail.com,
Jose Mora try...@gmail.com wrote:
[...]
I mean, some tasks are rather boring in python when compared with php,
for example, let's imagine we have a dictionary that contains
dictionaries that contain the times that a key appears. We, or at
least I, would like to write something as short as:
dict[k1][k2] += 1
However we will have to do a longer code (this is the smallest code I
could come up with):
dict = {}
if not k1 in dict:
dict[k1] = {}
if not k2 in dict[k1]:
dict[k1][k2] = 0
dict[k1][k2] += 1
How about one line plus an import?
$ python2.5
Python 2.5.3c1 (release25-maint, Dec 17 2008, 21:50:37)
[GCC 4.0.1 (Apple Computer, Inc. build 5363)] on darwin
Type help, copyright, credits or license for more information.
from collections import defaultdict
dic = defaultdict(lambda : defaultdict(int))
dic[spam][eggs] += 1
dic[42][3] += 1
dic[42][3] += 1
dic[42][3]
2
dic
defaultdict(function lambda at 0x661f0, {42: defaultdict(type
'int', {3: 2}), 'spam': defaultdict(type 'int', {'eggs': 1})})
--
Ned Deily,
n...@acm.org
--
http://mail.python.org/mailman/listinfo/python-list
Re: SQL, lite lite lite
On Dec 30, 11:16 am, prueba...@latinmail.com wrote: On Dec 29, 1:06 pm, Aaron Brady castiro...@gmail.com wrote: snip My idea is to create a 'Relation' class. The details are basically open, such as whether to back it with 'sqllite3', 'shelve', 'mmap', or just mapping and sequence objects; what the simplest syntax is that can capture and permit all the basics, and how much and what else can fit in at that level; how and whether it can include arbitrary Python objects, and what constraints there are on them if not; how and whether to permit transactions; and what the simplest and coolest thing you can do with a little Python syntax is. snip You really do like to reinvent the wheels do you? :-) Nothing wrong with that. Just be aware that most people that really need what you are proposing are probably already using mature feature rich libraries for that. http://wiki.python.org/moin/HigherLevelDatabaseProgramming Look at these options! Who invents the wheel? Naturally, I've had time to master every one. The page on Dee reminded me that there was some relational algebra early on in the databases course. I'd forgotten. Maybe I wouldn't feel so sheepish if I'd tried starting there, to see exactly what I need. It's possible that if I had a real goal, it could drive the requirements for my Relation class. I don't. What I have is very vague, somewhat second-hand, and also inspired by a tangent in a recent thread here on c-l-databases, er, -py. You remember the days of parallel arrays before records were invented. The bare minimum I can think of for a relation is just a set of tuples. TakingClass( Arthur, Science ) #TakingClass.add( ( Arthur, Science ) ) It's just that 'what classes is Arthur taking?' is an O( total ) op. Next is a parallel mapping. TakingClass( Arthur, Science ) # science.add( arthur ) # arthur.add( science ) O( key ) read and O( 1 ) add, but it's repetitious. I feel it's really error prone, but maybe it's the best balance. The actual abstraction is something more like this: # Emps( Name, EmpID, Department ) Emps( KnightsOfNi, NiNiNi, Gardening ) # tupA= EmpsTuple( KnightsOfNi, NiNiNi, Gardening ) # KnightsOfNi.add( tupA ) # NiNiNi.add( tupA ) # Gardening.add( tupA ) (If KnightsOfNi will appear in more than one relation, it will need to distinguish: 'KnightsOfNi.emps.add( tupA )'.) A single predicate is just a set. A relation is just a set of tuples. Dee lets you write lambda expressions for filters/selects. Django lets you chain selects with a double underscore. -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On Tue, 30 Dec 2008 19:19:08 +0100, Francesco Bochicchio bock...@virgilio.it wrote: [snip] If you are interested in socket errors, you should also fill the third 'fd-set' in the select call, and after select returns check that fd is not in it anymore: ready = select.select( [fd],[], [fd] ) if fd in ready[2]: # raise your error here The third argument to select() isn't for monitoring sockets for errors. Its behavior is also rather platform sensitive. In general, you don't need it at all on POSIX, but on Windows you should pass the same list for it as you pass for the write-set, merge the results, and treat them all as writeable. Or use a higher-level library that deals with all the asinine details for you. ;) Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
need help with list/variables
I have a list and would like to parse the list appending each list item to the end of a variable on a new line. for instance mylist = ['something\n', 'another something\n', 'something again\n'] then parse mylist to make it appear in my variable in this format: myvar = something another something something again how would i go about setting a variable like this? -- http://mail.python.org/mailman/listinfo/python-list
Re: thread, multiprocessing: communication overhead
mk mrk...@gmail.com wrote: CMIIW, but I believe your timing function includes the time to launch the actual processes and threads, create the synch. objects, etc. You might try it again, creating them first, starting the timer, then loading them. Except I don't know how to do that using timeit.Timer. :-/ Easy enough: put the code into a class which creates the worker pool when instantiated and has a method to execute the code you want to time. Then you create an instance from the setup argument and time a call to the method. -- http://mail.python.org/mailman/listinfo/python-list
Re: need help with list/variables
On Dec 30, 11:31 am, wx1...@gmail.com wrote: I have a list and would like to parse the list appending each list item to the end of a variable on a new line. for instance mylist = ['something\n', 'another something\n', 'something again\n'] then parse mylist to make it appear in my variable in this format: myvar = something another something something again how would i go about setting a variable like this? I think you want to concatenate the three elements in your list and then assign the resulting string to myvar. Strings that are defined with tripple quotes still get the newline character as though they were defined with double or single quotes and \n in them. conc = lambda x,y: x[:] + y # concatenate 2 lists without side effects mylist = ['something\n', 'another something\n', 'something again\n'] myvar = reduce(conc, mylist) print myvar something another something something again Hope this helps michael -- http://mail.python.org/mailman/listinfo/python-list
Re: need help with list/variables
I have a list and would like to parse the list appending each list
item to the end of a variable on a new line.
for instance
mylist = ['something\n', 'another something\n', 'something again\n']
then parse mylist to make it appear in my variable in this format:
myvar =
something
another something
something again
Just use the join() method on an empty string:
myvar = .join(mylist)
(though your example would need to be tweaked to make the
newlines match the newlines in your result, given the newlines in
your list)
myvar = something
another something
something again
Or you can do something like
myvar = '\n'.join(s.rstrip('\n') for s in mylist)
Mix and match to get your desired output.
-tkc
--
http://mail.python.org/mailman/listinfo/python-list
embedding python in wxpython
Hi, I've looked around for a way to allow a python console from within a wxPython application, but have only found stuff on embedded/ extending python with C/C++ or wxWidgets in C++, but not wxPython. Is this easy to do? Can someone point me in the right direction? Also, typically when you embed a scripting language into a larger application, how do you get the console environment to share data with the larger application? For instance, if the application has some gui stuff (for example clicking on a object and dragging it around), how do you get object.select(x,y) to print out on the console, and vice-versa: the object gets selected if the user types object.select(x,y)? I'd like the console to be a bidirectional representation of what's going on in the gui, plus a general purpose evaluation environment where you can manipulate application data via some api which is automatically exposed to the console when the application opens up. I'm looking for high-level hints/strategies/directions. Thank you Michael -- http://mail.python.org/mailman/listinfo/python-list
Re: duck typing at will
Jose Mora a écrit :
Duck typing is called that way because If it looks like a duck and
quacks like a duck, it must be a duck.
or at least something close enough...
I think it would be good to
have also If the programmer wants to deal with it like a duck, it
must be a duck
DWIM[1] just doesn't work. No parser can read your mind, and the harder
they try, the more bugs they are likely to cause. IOW : cows neither
look nor quack like ducks, and asking a cow to quack just won't turn
your cow into a duck.
[1] Do What I Mean
I mean, some tasks are rather boring in python when compared with php,
Possibly. OTHO, and from years of experience with both languages, a lot
of tasks are way more boring in php when compared to Python.
for example, let's imagine we have a dictionary that contains
dictionaries that contain the times that a key appears. We, or at
least I, would like to write something as short as:
dict[k1][k2] += 1
However we will have to do a longer code (this is the smallest code I
could come up with):
dict = {}
bad identifier - it shadows the builtin dict type.
if not k1 in dict:
dict[k1] = {}
if not k2 in dict[k1]:
dict[k1][k2] = 0
dict[k1][k2] += 1
What about:
from collections import defaultdict
d = defaultdict(lambda: defaultdict(int))
for k1, k2 in whatever:
d[k1][k2] += 1
NB : this requires a recent Python version (works on 2.5.1 at least).
Else, yes, this will be a bit more boring:
d = dict()
for k1, k2 in whatever:
d2 = d.setdefault(k1, dict())
d2[k2] = d2.get(k2, 0) + 1
HTH
--
http://mail.python.org/mailman/listinfo/python-list
Re: need help with list/variables
On Tue, 2008-12-30 at 11:31 -0800, wx1...@gmail.com wrote: I have a list and would like to parse the list appending each list item to the end of a variable on a new line. for instance mylist = ['something\n', 'another something\n', 'something again\n'] then parse mylist to make it appear in my variable in this format: myvar = something another something something again how would i go about setting a variable like this? myvar = ''.join(mylist) Though if you really want an blank line in the beginning you'll need to prepend a '\n' -a -- http://mail.python.org/mailman/listinfo/python-list
Re: embedding python in wxpython
On Dec 30, 1:52 pm, 5lvqbw...@sneakemail.com wrote: Hi, I've looked around for a way to allow a python console from within a wxPython application, but have only found stuff on embedded/ extending python with C/C++ or wxWidgets in C++, but not wxPython. Is this easy to do? Can someone point me in the right direction? Also, typically when you embed a scripting language into a larger application, how do you get the console environment to share data with the larger application? For instance, if the application has some gui stuff (for example clicking on a object and dragging it around), how do you get object.select(x,y) to print out on the console, and vice-versa: the object gets selected if the user types object.select(x,y)? I'd like the console to be a bidirectional representation of what's going on in the gui, plus a general purpose evaluation environment where you can manipulate application data via some api which is automatically exposed to the console when the application opens up. I'm looking for high-level hints/strategies/directions. Thank you Michael You should ask the guys on the wxPython list for pointers: http://wxpython.org/maillist.php I'm pretty sure I've seen it discussed there before. They already have PyShell and PyCrumb that you could probably use within your program. The wxPython includes demos of them and lots of other widgets. Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: Triple quoted string in exec function ?
ibpe...@gmail.com wrote: On Dec 30, 2:48 pm, Steve Holden st...@holdenweb.com wrote: Stef Mientki wrote: hello, I'm running scripts, with the execute function (Python 2.5), and it seems that triple quoted strings are not allowed. Is there a workaround, or is this a fundamental problem of the exec-function ? If you think about it, it should be obvious that you can't surround a string to be compiled with any of the quotes that appear inside the string to be compiled. That's about the only limitation I am aware of. And, by the way, exec is a *statement*, not a function! exec ( Init_Code, PG.P_Globals ) I've really doubt that this is a statement, unless I don't understand what a statement is. exec print '''This is ... a long string''' ... This is a long string regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ hello, the message Steven sent you is ok to explain how to work with triple quote Yes, but not to work around my problem. I guess I've to remove all triple quoted strings from my code. anyway thanks, Stef -- http://mail.python.org/mailman/listinfo/python-list
Re: embedding python in wxpython
5lvqbw...@sneakemail.com wrote: Hi, I've looked around for a way to allow a python console from within a wxPython application, but have only found stuff on embedded/ extending python with C/C++ or wxWidgets in C++, but not wxPython. Is this easy to do? Can someone point me in the right direction? Also, typically when you embed a scripting language into a larger application, how do you get the console environment to share data with the larger application? For instance, if the application has some gui stuff (for example clicking on a object and dragging it around), how do you get object.select(x,y) to print out on the console, and vice-versa: the object gets selected if the user types object.select(x,y)? I'd like the console to be a bidirectional representation of what's going on in the gui, plus a general purpose evaluation environment where you can manipulate application data via some api which is automatically exposed to the console when the application opens up. I'm looking for high-level hints/strategies/directions. If you're looking for high-level, I don't understand why you want to see what's going on in the gui, as I see the gui just a tool to control the real data. For a pretty high level data manipulation, something like PyLab_Works ? http://code.google.com/p/pylab-works/ cheers, Stef -- http://mail.python.org/mailman/listinfo/python-list
Re: Triple quoted string in exec function ?
On Tue, 30 Dec 2008 21:16:39 +0100, Stef Mientki stef.mien...@gmail.com wrote: ibpe...@gmail.com wrote: On Dec 30, 2:48 pm, Steve Holden st...@holdenweb.com wrote: Stef Mientki wrote: hello, I'm running scripts, with the execute function (Python 2.5), and it seems that triple quoted strings are not allowed. Is there a workaround, or is this a fundamental problem of the exec-function ? If you think about it, it should be obvious that you can't surround a string to be compiled with any of the quotes that appear inside the string to be compiled. That's about the only limitation I am aware of. And, by the way, exec is a *statement*, not a function! exec ( Init_Code, PG.P_Globals ) I've really doubt that this is a statement, unless I don't understand what a statement is. What do you think a statement is? According to the Python grammar, exec is this: exec_stmt: 'exec' expr ['in' test [',' test]] stmt is short for statement. :) A more satisfying demonstration of the statementness of exec is this, though: x = exec foo File stdin, line 1 x = exec foo ^ SyntaxError: invalid syntax And no, putting parenthesis around the expression given to exec doesn't make a difference: x = exec(foo) File stdin, line 1 x = exec(foo) ^ SyntaxError: invalid syntax Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Understanding search queries, semantics, and Meaning ...aren't we all looking for meaning?
I have Section 4.4.1 of SICP rattling around in my head (database
queries), and I'm trying to come up with a simple dictionary-based
database in Python to represent circuit diagrams. My main confusion
isn't one of implementation, but a matter of big thinking,
fundamentally, about the problem. Please don't suggest using a SQL
library, as I'm looking to learn to fish, so to speak, and to learn a
bit about the biology of fish.
I've subclassed dict to hdict (hashable dict) and rewritten the
__hash__ function so I can include a dict into a set. Thanks to a
previous poster here for providing that suggestion.
A circuit component looks like this for example:
comp1 = hdict(value=1e-6, footprint='0402', vendor='digikey')
comp2 = hdict(value=10e3, footprint='0603', vendor='mouser')
etc, etc.
The database holds the values like this:
db = dict() # normal dict
db['value'] = ([1e-6, 10e3], [comp1, comp2]) #ordered set for fast
lookup/insertion
db['footprint'] = {'0402':set[comp1], '0603':comp2} # unordered uses
normal dict for fast lookup
db['vendor'] = {'digikey':[comp1], 'mouser':[comp2]}
So basically the keys are the component parameters, and the values is
the list of components with that value. Stuff that is comparable is
ordered; stuff that is discrete is not ordered, using either 2-tuples
or dicts, respectively.
This allows extremely fast lookup of components based on their
properties, with O(1) performance for non-ordered stuff, and O(log n)
performance for ordered stuff (using bisect methods). The set
operations are extremely fast, so I can do AND and OR operations on
compound queries of this nature without worry.
I need this speed not so much for selecting components when the user
types in a query, but for when the mouse is hovering over the
schematic and I need things to light up underneath, if the GUI is
generating hundreds of mouse messages per second, and for inspector
windows to appear quickly when you click, etc. If you have ever used
Altium, you can see how effective this is in terms of creating a good
interactive user experience.
My question is what happens when I choose to search for NOT
footprint='0402'.
Should this return a blank list? This happens by default., and is in
fact true: a blank list satisfies not anything actually.
Should this return everything that is NOT footprint 0402 (ie returns
0603 components)? This *strongly implies* a pre-selection of *all*
objects before performing the negating function, or of looking at the
ordering of other queries in the overall search term, and then
applying NOT to the results of another query.
But I'm suspicious of a brute force preselection of all objects
whenever I see a NOT, and anyway it messes up the clean query/combiner
method of search I'm doing, and it requires an implied sequence of
search, where I'm pretty sure it should not rely on sequencing. Even
though this is single threaded, etc., the semantics of the query
should not rely on ordering of the search term:
footprint='0402' and NOT vendor='mouser' should return the same as
NOT vendor='mouser' and footprint='0402'.
So this is my philosophical quandary. I'm not sure what the correct
thing is. In SICP they are using nondeterministic stuff which I don't
quite get, so it's hard to follow. Also they are not using
dictionaries and hashes, so I'm not sure if their generate-and-test
method would work here anyway. Generate-and-test seems extremely
inefficient.
Can a wise guru please enlighten me?
thanks
Michael
--
http://mail.python.org/mailman/listinfo/python-list
Re: PIL - font kerning
On Dec 23, 9:51 pm, Ivan Illarionov ivan.illario...@gmail.com wrote:
On Dec 23, 11:22 pm, Ivan Illarionov ivan.illario...@gmail.com
wrote:
On 23 дек, 16:44, carsn carsten.kr...@gmail.com wrote:
Hey all,
anybody know, if there´s a way to specify the kerning of a font, when
you draw text withPIL?
I´d like to achieve the same effect that you get, when you set a
negative kerning in Gimp/Photshop - ie. reduce the spacing between
glyphs.
CanPILdo that or do I use another lib for that?
Thx for any pointers some nice xmas days to U all!
carsten
No.PILcan't do that. I suggest combination of cairo/pango/pangocairo
(pycairo and pygtk packages).
Ivan
I found a little helper function that does what you want (and more)
import cairo
import pango
import pangocairo
def draw_text(surface, context, text, font=sans 14, position=None,
color=None,
box_width=None,
alignment=pango.ALIGN_CENTER,
line_spacing=None, letter_spacing=None,
extra_kerning=None):
if color is None:
color = (0.0, 0.0, 0.0)
context.set_source_rgb(*color)
pc = pangocairo.CairoContext(context)
layout = pc.create_layout()
layout.set_text(text)
layout.set_font_description(pango.FontDescription(font))
if box_width: layout.set_width(box_width)
layout.set_alignment(alignment)
if line_spacing: layout.set_spacing(spacing)
alist = pango.AttrList()
if letter_spacing:
alist.insert(pango.AttrLetterSpacing(letter_spacing, 0, len
(text)))
if extra_kerning:
for pos, kern in extra_kerning.iteritems():
alist.insert(pango.AttrLetterSpacing(kern, pos, pos
+1))
layout.set_attributes(alist)
if position is None:
width, height = surface.get_width(), surface.get_height()
w, h = layout.get_pixel_size()
position = (width/2.0 - w/2.0, height/2.0 - h/2.0)
context.move_to(*position)
pc.show_layout(layout)
And example usage:
surface = cairo.ImageSurface(cairo.FORMAT_ARGB32, width, height)
context = cairo.Context(surface)
draw_text(surface, context, 'Hello world!',
font=sans 52, color=(.25,.28,.33),
letter_spacing=-6000,
extra_kerning={0:-9000, 1:-1000, 6:6000, 7:-15000, 8:5000,
9:-7000})
surface.write_to_png(hello.png)
--
Ivan
Works great, thanks a lot!!
--
http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On 2008-12-30, Francesco Bochicchio bock...@virgilio.it wrote: 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. In non-blocking mode, it will always return immediately, either with some data, no data (other end closed), or an EAGAIN or EWOULDBLOCK error (I forget which). [...] I myself tend to avoid using non-blocking sockets, since blocking sockets are much easier to handle... That depends on whether you can tolerate blocking or not. In an event-loop, blocking is generally not allowed. -- Grant -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On Tue, 30 Dec 2008 14:41:17 -0600, Grant Edwards gra...@visi.com wrote: On 2008-12-30, Francesco Bochicchio bock...@virgilio.it wrote: 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. Hi Grant, I don't think you read Francesco's message carefully enough. :) He said: there is no difference in select between blocking and non-blocking mode. You're describing a difference in recv, not select. In non-blocking mode, it will always return immediately, either with some data, no data (other end closed), or an EAGAIN or EWOULDBLOCK error (I forget which). [...] I myself tend to avoid using non-blocking sockets, since blocking sockets are much easier to handle... That depends on whether you can tolerate blocking or not. In an event-loop, blocking is generally not allowed. If you're careful, it's possible to avoid blocking, even when using a blocking socket, at least for AF_INET, SOCK_STREAM sockets. Of course, it's easier to avoid blocking by using a non-blocking socket. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: select.select and socket.setblocking
On Tue, 30 Dec 2008 15:55:51 -0500, Jean-Paul Calderone exar...@divmod.com wrote: On Tue, 30 Dec 2008 14:41:17 -0600, Grant Edwards gra...@visi.com wrote: On 2008-12-30, Francesco Bochicchio bock...@virgilio.it wrote: 3. AFAIK (sorry, I feel acronym-ly today ;), there is no difference in select between blocking and non-blocking mode. The difference is in the recv (again, assuming that you use TCP as protocol, that is AF_INET, SOCK_STREAM), which in the blocking case would wait to receive all the bytes that you requested, No, in blocking mode it will wait to receive _some_ data (1 or more bytes). The requested amount is strictly an upper limit: recv won't return more than the requested number of bytes, but it might return less. Hi Grant, I don't think you read Francesco's message carefully enough. :) Ah, no, it was I who didn't read carefully enough. Nevermind. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: SQL, lite lite lite
Bruno Desthuilliers wrote: Aaron Brady a écrit : Hi all, (snip) I don't think relational data can be read and written very easily in Python. Did you try SQLAlchemy or Django's ORM ? [...] Using an ORM when you don't grasp the relational model and/or the SQL query language is futile. That's probably the case for many other abstraction layers, too. -- Gerhard -- http://mail.python.org/mailman/listinfo/python-list
Re: embedding python in wxpython
On Wed, Dec 31, 2008 at 7:21 AM, Stef Mientki stef.mien...@gmail.comwrote: 5lvqbw...@sneakemail.com wrote: Hi, I've looked around for a way to allow a python console from within a wxPython application, but have only found stuff on embedded/ extending python with C/C++ or wxWidgets in C++, but not wxPython. Is this easy to do? Can someone point me in the right direction? Also, typically when you embed a scripting language into a larger application, how do you get the console environment to share data with the larger application? For instance, if the application has some gui stuff (for example clicking on a object and dragging it around), how do you get object.select(x,y) to print out on the console, and vice-versa: the object gets selected if the user types object.select(x,y)? I'd like the console to be a bidirectional representation of what's going on in the gui, plus a general purpose evaluation environment where you can manipulate application data via some api which is automatically exposed to the console when the application opens up. I'm looking for high-level hints/strategies/directions. If you're looking for high-level, I don't understand why you want to see what's going on in the gui, as I see the gui just a tool to control the real data. For a pretty high level data manipulation, something like PyLab_Works ? http://code.google.com/p/pylab-works/ cheers, Stef -- http://mail.python.org/mailman/listinfo/python-list Try checking out the wxPython demo package. There is an example of a widget inspection tool in the help menu. This tool has an embedded python console and allows introspection of live widgets and sizers in an application. Cheers, Jervis -- http://mail.python.org/mailman/listinfo/python-list
Re: Triple quoted string in exec function ?
Stef Mientki wrote: ibpe...@gmail.com wrote: On Dec 30, 2:48 pm, Steve Holden st...@holdenweb.com wrote: Stef Mientki wrote: hello, I'm running scripts, with the execute function (Python 2.5), and it seems that triple quoted strings are not allowed. Is there a workaround, or is this a fundamental problem of the exec-function ? If you think about it, it should be obvious that you can't surround a string to be compiled with any of the quotes that appear inside the string to be compiled. That's about the only limitation I am aware of. And, by the way, exec is a *statement*, not a function! exec ( Init_Code, PG.P_Globals ) I've really doubt that this is a statement, unless I don't understand what a statement is. We'll have to conclude you don't, then, won't we ;-) regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Triple quoted string in exec function ?
Stef Mientki wrote in news:mailman.6399.1230668197.3487.python- l...@python.org in comp.lang.python: And, by the way, exec is a *statement*, not a function! exec ( Init_Code, PG.P_Globals ) I've really doubt that this is a statement, unless I don't understand what a statement is. In python 2.x the above is a statement that is passed a tuple: http://docs.python.org/reference/simple_stmts.html#exec its a statement like print is: print ( 1,2 ) (1, 2) In 3.x it is a function: http://docs.python.org/3.0/library/functions.html#exec print is also a function in python 3.x, so: print(1, 2) 1 2 Rob. -- http://www.victim-prime.dsl.pipex.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: embedding python in wxpython
5lvqbw...@sneakemail.com wrote: Hi, I've looked around for a way to allow a python console from within a wxPython application, but have only found stuff on embedded/ extending python with C/C++ or wxWidgets in C++, but not wxPython. Is this easy to do? Can someone point me in the right direction? Also, typically when you embed a scripting language into a larger application, how do you get the console environment to share data with the larger application? For instance, if the application has some gui stuff (for example clicking on a object and dragging it around), how do you get object.select(x,y) to print out on the console, and vice-versa: the object gets selected if the user types object.select(x,y)? I'd like the console to be a bidirectional representation of what's going on in the gui, plus a general purpose evaluation environment where you can manipulate application data via some api which is automatically exposed to the console when the application opens up. I'm looking for high-level hints/strategies/directions. I seem to remember you can create your wxApp with an argument of True or False. One of those settings creates a window containing any output to sys.stderr, if I remember rightly. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: win32gui
On Dec 30, 3:22 pm, Gandalf goldn...@gmail.com wrote: I'm searching the win32gui hooks for a function to get the windowClass position any idea? thanks! Try looking in the docs: http://docs.activestate.com/activepython/2.4/pywin32/win32gui.html I think the GetWindowPlacement() might be what you're looking for, although it's kind of hard to tell from what little info you gave. Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: SQL, lite lite lite
Gerhard Häring a écrit : Bruno Desthuilliers wrote: Aaron Brady a écrit : Hi all, (snip) I don't think relational data can be read and written very easily in Python. Did you try SQLAlchemy or Django's ORM ? [...] Using an ORM when you don't grasp the relational model and/or the SQL query language is futile. Yes, indeed. And ? Aaron's post was mostly about a better integration of the relational model in Python - which obviously requires some knowledge of the topic. -- http://mail.python.org/mailman/listinfo/python-list
Re: Understanding search queries, semantics, and Meaning ...aren't we all looking for meaning?
On Dec 30, 12:35 pm, 5lvqbw...@sneakemail.com wrote:
I have Section 4.4.1 of SICP rattling around in my head (database
queries), and I'm trying to come up with a simple dictionary-based
database in Python to represent circuit diagrams. My main confusion
isn't one of implementation, but a matter of big thinking,
fundamentally, about the problem. Please don't suggest using a SQL
library, as I'm looking to learn to fish, so to speak, and to learn a
bit about the biology of fish.
I'm going to break rule #1 of your requirements but in an unexpected
way. Rather than studying PostgreSQL, MySQL, or Oracle, why don't you
crack open the topic of relational database theory and relational
algebra? That should help with the big thinking bit in the same way
understanding 1+1 helps you understand how to add any two numbers
together. No, SQL is not the be-all-end-all of relational theory. Yes,
it does a pretty dang good job at it, though, which is why it is still
around.
I've subclassed dict to hdict (hashable dict) and rewritten the
__hash__ function so I can include a dict into a set. Thanks to a
previous poster here for providing that suggestion.
A circuit component looks like this for example:
comp1 = hdict(value=1e-6, footprint='0402', vendor='digikey')
comp2 = hdict(value=10e3, footprint='0603', vendor='mouser')
etc, etc.
The database holds the values like this:
db = dict() # normal dict
db['value'] = ([1e-6, 10e3], [comp1, comp2]) #ordered set for fast
lookup/insertion
db['footprint'] = {'0402':set[comp1], '0603':comp2} # unordered uses
normal dict for fast lookup
db['vendor'] = {'digikey':[comp1], 'mouser':[comp2]}
So basically the keys are the component parameters, and the values is
the list of components with that value. Stuff that is comparable is
ordered; stuff that is discrete is not ordered, using either 2-tuples
or dicts, respectively.
This allows extremely fast lookup of components based on their
properties, with O(1) performance for non-ordered stuff, and O(log n)
performance for ordered stuff (using bisect methods). The set
operations are extremely fast, so I can do AND and OR operations on
compound queries of this nature without worry.
I need this speed not so much for selecting components when the user
types in a query, but for when the mouse is hovering over the
schematic and I need things to light up underneath, if the GUI is
generating hundreds of mouse messages per second, and for inspector
windows to appear quickly when you click, etc. If you have ever used
Altium, you can see how effective this is in terms of creating a good
interactive user experience.
OK, turn it around in your head. Consider the indexes you built above
as yet another table. Consider what a table or a database really is.
When you see how they are all really the same thing, either you've
mastered relational algebra of you've seen the big picture. Kind of
like the cons cell being enough to describe any data structure in the
universe, a table is enough to describe everything in the relational
world.
My question is what happens when I choose to search for NOT
footprint='0402'.
Should this return a blank list? This happens by default., and is in
fact true: a blank list satisfies not anything actually.
Should this return everything that is NOT footprint 0402 (ie returns
0603 components)? This *strongly implies* a pre-selection of *all*
objects before performing the negating function, or of looking at the
ordering of other queries in the overall search term, and then
applying NOT to the results of another query.
But I'm suspicious of a brute force preselection of all objects
whenever I see a NOT, and anyway it messes up the clean query/combiner
method of search I'm doing, and it requires an implied sequence of
search, where I'm pretty sure it should not rely on sequencing. Even
though this is single threaded, etc., the semantics of the query
should not rely on ordering of the search term:
footprint='0402' and NOT vendor='mouser' should return the same as
NOT vendor='mouser' and footprint='0402'.
So this is my philosophical quandary. I'm not sure what the correct
thing is. In SICP they are using nondeterministic stuff which I don't
quite get, so it's hard to follow. Also they are not using
dictionaries and hashes, so I'm not sure if their generate-and-test
method would work here anyway. Generate-and-test seems extremely
inefficient.
Can a wise guru please enlighten me?
(I don't think I qualify as a guru, but I think I see how I can help.)
In a typical SQL database, when you type in SELECT foo FROM bar WHERE
baz='bo', you are not writing a program, at least not in the sense of
Python or C or Java or Perl where you give instructions on HOW to run
the program. You are writing a program in the sense of Lisp or Scheme
or Haskell in that you are giving instructions on WHAT the program is.
The database doesn't, at least shouldn't, read in all the
Re: SQL, lite lite lite
Aaron Brady a écrit : On Dec 30, 11:16 am, prueba...@latinmail.com wrote: (snip) You really do like to reinvent the wheels do you? :-) Nothing wrong with that. Just be aware that most people that really need what you are proposing are probably already using mature feature rich libraries for that. http://wiki.python.org/moin/HigherLevelDatabaseProgramming Look at these options! Who invents the wheel? Naturally, I've had time to master every one. Oh, so that's why you propose to add yet another item to the list ? -- http://mail.python.org/mailman/listinfo/python-list
Re: Parsing Excel spreadsheets
On Dec 31, 5:48 am, Mike Driscoll kyoso...@gmail.com wrote: On Dec 30, 10:07 am, andyh...@gmail.com andyh...@gmail.com wrote: Hi, Can anybody recommend an approach for loading and parsing Excel spreadsheets in Python. Any well known/recommended libraries for this? The only thing I found in a brief search was http://www.lexicon.net/sjmachin/xlrd.htm, but I'd rather get some more input before going with something I don't know. Thanks, Andy. If you need complete control of Excel, then you'll probably have to use COM (see Mark Hammond's PyWin32 package). Otherwise, the suggestions by the others will work. For avoidance of doubt, the OP should presume that you mean the *nett* suggestions by the others i.e. after Tino struck out pyExcelerator. -- http://mail.python.org/mailman/listinfo/python-list
Re: embedding python in wxpython
Steve Holden wrote: I'd like the console to be a bidirectional representation of what's going on in the gui, plus a general purpose evaluation environment where you can manipulate application data via some api which is automatically exposed to the console when the application opens up. I'm looking for high-level hints/strategies/directions. I seem to remember you can create your wxApp with an argument of True or False. One of those settings creates a window containing any output to sys.stderr, if I remember rightly. You do -- True (the default) redirects standard output to a window, while False does not redirect it. However, neither setting will create a bidirectional console or evaluation environment as the OP was asking for. (But other wx widgets do provide that, as other replies have pointed out.) Best, - Joe -- http://mail.python.org/mailman/listinfo/python-list
[ANN] PyYAML-3.08: Now with Python 3 support
Announcing PyYAML-3.08
A new release of PyYAML is now available:
http://pyyaml.org/wiki/PyYAML
This release features complete support for Python 3. For
compatibility notes between Python 2 and Python 3 versions,
please see
http://pyyaml.org/wiki/PyYAMLDocumentation#Python3support
Changes
===
* Python 3 support (Thank to Erick Tryzelaar).
* Use Cython instead of Pyrex to build LibYAML bindings. Note
that the source package is distributed with a pre-generated
'_yaml.c' file so you don't need Cython installed to build
LibYAML bindings.
* Refactored support for unicode and byte input/output streams.
Resources
=
PyYAML homepage: http://pyyaml.org/wiki/PyYAML
PyYAML documentation: http://pyyaml.org/wiki/PyYAMLDocumentation
TAR.GZ package: http://pyyaml.org/download/pyyaml/PyYAML-3.08.tar.gz
ZIP package: http://pyyaml.org/download/pyyaml/PyYAML-3.08.zip
Windows installers:
http://pyyaml.org/download/pyyaml/PyYAML-3.08.win32-py2.3.exe
http://pyyaml.org/download/pyyaml/PyYAML-3.08.win32-py2.4.exe
http://pyyaml.org/download/pyyaml/PyYAML-3.08.win32-py2.5.exe
http://pyyaml.org/download/pyyaml/PyYAML-3.08.win32-py2.6.exe
http://pyyaml.org/download/pyyaml/PyYAML-3.08.win32-py3.0.exe
PyYAML SVN repository: http://svn.pyyaml.org/pyyaml
Submit a bug report: http://pyyaml.org/newticket?component=pyyaml
YAML homepage: http://yaml.org/
YAML-core mailing list:
http://lists.sourceforge.net/lists/listinfo/yaml-core
About PyYAML
YAML is a data serialization format designed for human readability and
interaction with scripting languages. PyYAML is a YAML parser and
emitter for Python.
PyYAML features a complete YAML 1.1 parser, Unicode support, pickle
support, capable extension API, and sensible error messages. PyYAML
supports standard YAML tags and provides Python-specific tags that allow
to represent an arbitrary Python object.
PyYAML is applicable for a broad range of tasks from complex
configuration files to object serialization and persistance.
Example
===
import yaml
yaml.load(
... name: PyYAML
... description: YAML parser and emitter for Python
... homepage: http://pyyaml.org/wiki/PyYAML
... keywords: [YAML, serialization, configuration, persistance, pickle]
... )
{'keywords': ['YAML', 'serialization', 'configuration', 'persistance',
'pickle'], 'homepage': 'http://pyyaml.org/wiki/PyYAML', 'description':
'YAML parser and emitter for Python', 'name': 'PyYAML'}
print yaml.dump(_)
name: PyYAML
homepage: http://pyyaml.org/wiki/PyYAML
description: YAML parser and emitter for Python
keywords: [YAML, serialization, configuration, persistance, pickle]
Copyright
=
The PyYAML module is written by Kirill Simonov x...@resolvent.net.
PyYAML is released under the MIT license.
--
http://mail.python.org/mailman/listinfo/python-list
Re: need help with list/variables
On Dec 30, 11:41 am, 5lvqbw...@sneakemail.com wrote: conc = lambda x,y: x[:] + y # concatenate 2 lists without side effects mylist = ['something\n', 'another something\n', 'something again\n'] myvar = reduce(conc, mylist) print myvar conc? side effects? Missing Lisp much? ;-) Let's try to Pythonize your lisp. One: Assigning a lambda to a variable? Just use def. It's pretty much the same thing. def conc(x,y): return x[:]+y Two: I don't think x+y affects x at all when x and y are lists. (Is that a lispism?) So x[:]+y has an unnecessary array copy. def conc(x,y): return x+y Three: Python may still be slow at string concatenation. (Perl is fast.) Rather than concatenating one at a time, it's better, I understand, to build up a list and then join() them together. The other posters got the right answer in the pythonic way. -- http://mail.python.org/mailman/listinfo/python-list
Re: embedding python in wxpython
On Dec 30, 3:41 pm, Steve Holden st...@holdenweb.com wrote: 5lvqbw...@sneakemail.com wrote: Hi, I've looked around for a way to allow a python console from within a wxPython application, but have only found stuff on embedded/ extending python with C/C++ or wxWidgets in C++, but not wxPython. Is this easy to do? Can someone point me in the right direction? Also, typically when you embed a scripting language into a larger application, how do you get the console environment to share data with the larger application? For instance, if the application has some gui stuff (for example clicking on a object and dragging it around), how do you get object.select(x,y) to print out on the console, and vice-versa: the object gets selected if the user types object.select(x,y)? I'd like the console to be a bidirectional representation of what's going on in the gui, plus a general purpose evaluation environment where you can manipulate application data via some api which is automatically exposed to the console when the application opens up. I'm looking for high-level hints/strategies/directions. I seem to remember you can create your wxApp with an argument of True or False. One of those settings creates a window containing any output to sys.stderr, if I remember rightly. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ That's true...or you can just redirect stdout, which is what the demo does. Here's one way to do it: code import sys import wx class RedirectText: def __init__(self,aWxTextCtrl): self.out=aWxTextCtrl def write(self,string): self.out.WriteText(string) class MyForm(wx.Frame): def __init__(self): wx.Frame.__init__(self, None, wx.ID_ANY, wxPython Redirect Tutorial) # Add a panel so it looks the correct on all platforms panel = wx.Panel(self, wx.ID_ANY) log = wx.TextCtrl(panel, wx.ID_ANY, size=(300,100), style = wx.TE_MULTILINE|wx.TE_READONLY| wx.HSCROLL) btn = wx.Button(panel, wx.ID_ANY, 'Push me!') self.Bind(wx.EVT_BUTTON, self.onButton, btn) # Add widgets to a sizer sizer = wx.BoxSizer(wx.VERTICAL) sizer.Add(log, 1, wx.ALL|wx.EXPAND, 5) sizer.Add(btn, 0, wx.ALL|wx.CENTER, 5) panel.SetSizer(sizer) # redirect text here redir=RedirectText(log) sys.stdout=redir def onButton(self, event): print You pressed the button! # Run the program if __name__ == __main__: app = wx.PySimpleApp() frame = MyForm().Show() app.MainLoop() /code - Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: return in def
Steven D'Aprano a écrit :
(snip)
Avoiding early exits is an over-reaction to the Bad Old Days of spaghetti
code.
Mostly, yes. It can also be a way to help avoiding resource leaks
(memory or whatever) - just like try/finally blocks or the 'with'
statement in Python.
But used wisely, early exists can simplify, not complicate, code.
Consider the following:
def find_ham(alist):
for item in alist:
if isinstance(item, Ham):
return item
raise ValueError('no ham found')
def find_spam(alist):
found_item = None
for item in alist:
if found_item is not None:
if isinstance(item, Spam):
found_item = item
if found_item is None:
raise ValueError('no spam found')
else:
return found_item
The second version has double the number of lines of code of the first.
And doesn't do the same thing (hint: you forgot a break statement). Also
and FWIW, the single exit golden rule (hem) comes from languages
without exception handling. So a fair comparison would be:
def find_egg(alist):
for item in alist:
if isinstance(item, Egg):
return item
return None
vs:
def find_sausage(alist):
found = None
for item in alist:
if isinstance(item, Sausage):
found = item
break
return found
Granted, this still doesn't make the second version better than the
first !-)
But:
I don't think the claim that the version with an early exit is more
complicated than the version without can justified.
Certainly not in this simple (simplistic ?) example. Now in C or Pascal,
functions tend to be much longer and complicated, thanks to all the gory
details one has to take care of.
Not that I'm advocating no early return as a golden rule - specially
in Python where functions tend to be short and readable -, but there
really are cases (depending on the language *and* concrete use case)
where avoiding early returns makes for more readable and robust code.
My 2 cents...
--
http://mail.python.org/mailman/listinfo/python-list
Re: Read-Write Lock vs primitive Lock()
En Tue, 30 Dec 2008 06:16:23 -0200, k3xji sum...@gmail.com escribió: As GLOBAL_LOOP_COUNT is 1000, now this is making a bottleneck on the readers. I had assumed that as everythread is given only 100 bytecodes to execute, that it will be enough to have a 1 value for this number to let other rthread start() and try to acquire the lock. But, Python undocumentedly, does not grab the GIL easily if a Lock is held. This is strange for me. This has nothing to do with the GIL, nor undocumented behavior. If all threads are blocked waiting for the same lock, yielding the CPU has no effect, because nobody can progress until the lock is released. I'll ask you again: what is your use case? what do you want to do? what is a reader in your problem? what is a writer? what is the resource you want to protect using a lock? Different problems have different behaviors; for example, for reading/writing files, your model is basically wrong. That may be the point. That is why I am trying to test this. When will ReadWrite() lock permforms better over the primitive lock. By the way, for the reference, the code in the recipe is used in CherryPy and based on the Tanenbaum's book Operating Systems Design. Ah, good to know at least it has some background. Anyway, isn't a guarantee of correctness. By exampe, Tanenbaum's algorithm for the dining philosophers problem were (is?) flawed (according to W. Stallings). I understand your point completely, but let's change anything inside the loop. Just assume that we a thread that is supposed to read something, and a thread that is supposed to write. If I create thousands of readers, and if the operation is enormously huge calculation(as Python does not grab GIL easily inside a Lock), then this will create a bottlencek on readers. But, with ReadWrite Lock this problem *SHOULD* be fixed and in my tests I see it is fixed, when I increase the time spent in the loop in huge amounts. But why would you protect the whole long calculation with a lock? And why do you say Python does not grab GIL easily inside a Lock, whatever that means? If you hold a scarce resource for a long time, this is likely to cause a bottleneck, and the language (be it Python or other) is mostly irrelevant here. I had assumed that Python will release GIL for other threads after 100 bytecodes are executed by default. Yes. You can alter how often this occurs, using sys.setcheckinterval (the code I posted does that, just to see how results vary) However, as I stated, when a Lock() is held this is changing. I think this is because to avoid simple MT problems for new programmers. Not sure. I will read threads.c for information. No, it's always the same thing, locks don't affect this behavior. But as I said above, if no other thread can progress because all of them are waiting to acquire the same lock, you gain nothing by releasing the GIL. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: get method
On Tue, Dec 30, 2008 at 7:10 PM, Roel Schroeven rschroev_nospam...@fastmail.fm wrote: Hm, you just changed an O(n) algorithm to an O(n**2) algorithm. No big deal for short strings, but try your solution on a string with length 1 and see the difference. On my computer the O(n) version takes 0.008 seconds, while your version takes 8.6 seconds. That's 1000 times slower. Yes you are right :) Sadly :/ I wonder if there is a way to implement the same thing with close to O(n) complexity using list/dict comprehensions. cheers James -- http://mail.python.org/mailman/listinfo/python-list
Re: Python in C
akineko wrote: The more you work on Python, the harder you can go back to C or C++ world. I use SWIG, instead. I think SWIG is a good way to mix two worlds. If you find it hard to go from Python back to C, you should have a look at Cython. http://cython.org/ Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: multiprocessing vs thread performance
On Wed, Dec 31, 2008 at 12:29 AM, Aaron Brady castiro...@gmail.com wrote: James, Hi. I'm glad you asked; I never know how out there my comments are (but surmise that feedback is always a good thing). What I was thinking was, I didn't know Virtual Synchrony, and I've never used Erlang, but I'm interested in concurrency especially as it pertains to units of work, division of labor, and division of context; and generators are another way to divide context. So: I wanted to put more of my background and interests on the table. What I said wasn't directly relevant, I see. But it's not like I dissertated (discussed) the Tibettan-Austrian spice trade. I think I just want to say stuff about threading! Maybe I'm just excited to meet people who share my interests... not unheard of. Glad to see others also interested in these topics :) (snip) 'Circuits' doesn't use generators. I think generators are an underexplored technique. Is 'circuits' assembly line or start-to- finish, if my analogy makes any sense? 'Circuits' is event-driven, but I don't see any difference between 'event-driven' and multithreaded in general. (I think contrast can create a good picture and a clear understanding.) What is special about an 'event-driven' architecture? Are you distinguishing blocking from polling? I'll shortly be releasing circuits-1.0 today hopefully. To answer your question, circuits is inspired by a software architecture that my most favoured lecturer a few years back was teaching. That is: * Behaviour Trees (design) and consequently: * The concept of everything is a Component * Systems and Sub-Systems are built upon Components and * Everything is an event. and * An emergent property of such systems are Behaviour. That being said, circuits employs both an event-driven approach as well as a Component architecture. In your analogy it is both horizontal and vertical. As I continue to develop circuits and improve it's core design as well as building it's ever growing set of Components, I try to keep it as general as possible - my main aim though is distributed processing and architectures. (See the primes example). Thanks for sharing your interest :) cheers James -- http://mail.python.org/mailman/listinfo/python-list
Re: python import sys.path
On Dec 31, 5:05 am, Kelly, Brian brian.ke...@uwsp.edu wrote: I have both 2.4 and 2.5 interpreters installed on a linux box. The PythonPath is set to : PYTHONPATH=/usr/lib64/portage/pym:/prod/bacula/local/lib64/python2.4/site-pa ckages:/prod/bacula/local/lib/python2.4/site-packages My main script is getting called like so: python2.4 cleanup.py wrkstnbs The imports statements in cleanup.py are as follows: import os,sys print sys.path from datetime import datetime from optparse import OptionParser # used for parsing parameters from bacula_conf import * # used for connecting to our databases, etc. from registration_cleanup \ import RegistrationCleanup # used for interacting w/ registration db (sql1) # and configuration database (genunix) import directory_cleanup as fclean # file cleanup. One of the scripts being imported from bacula_conf is called purge_client.py. It has the following imports: import sys import MySQLdb Everytime I run python2.4 cleanup.py wrkstnbs I get the following error: Traceback (most recent call last): Have you snipped any traceback entries here? You say you are running cleanup.py but the first traceback entry is from purge_client.py!! You should first fix that, so that you've got the full story. My guess is that something along that trail is invoking another instance of the Python interpreter and somehow that instance is 2.5, not 2.4. I can't imagine how those 2.5-related entries would otherwise get into sys.path Suggestions: (1) where you are debug-printing sys.path, also print sys.version and sys.argv[0] ... and do debug-printing at more places between start and error. (2) run python2.4 -vv cleanup.py wrkstnbs and look for the first mention of 2.5 (indicating something has manipulated sys.path), or sudden cessation of -vv output (indicating a new instance of python is running without -vv), or some other phenomenon ... File purge_client.py, line 22, in module import MySQLdb File /prod/bacula/local/lib64/python2.4/site-packages/MySQLdb/__init__.py, line 27, in module import _mysql ImportError: /prod/bacula/local/lib64/python2.4/site-packages/_mysql.so: undefined symbol: Py_InitModule4 What happens when you run python2.4 and at the interactive prompt do import _mysql ? Is there a possibility that you installed 2.5 MySQLdb into the 2.4 hierarchy? [could be a stupid question from a windows guy; this may be a non-problem on linux] Is this the first thing that you've tried after installing 2.5, or the only problem found in an exhaustive regression test of all your apps using 2.4, or somewhere in the middle? [big snip] HTH, John -- http://mail.python.org/mailman/listinfo/python-list
Re: need help with list/variables
On Dec 31, 6:41 am, 5lvqbw...@sneakemail.com wrote: On Dec 30, 11:31 am, wx1...@gmail.com wrote: I have a list and would like to parse the list appending each list item to the end of a variable on a new line. for instance mylist = ['something\n', 'another something\n', 'something again\n'] then parse mylist to make it appear in my variable in this format: myvar = something another something something again how would i go about setting a variable like this? I think you want to concatenate the three elements in your list and then assign the resulting string to myvar. Strings that are defined with tripple quotes still get the newline character as though they were defined with double or single quotes and \n in them. conc = lambda x,y: x[:] + y # concatenate 2 lists without side effects mylist = ['something\n', 'another something\n', 'something again\n'] myvar = reduce(conc, mylist) print myvar OTOH escapees from FORTRAN would prefer something like: TOT = '' for I in xrange(len(MYLIST)): TOT = TOT + MYLIST[I] FTNWRITE(6, '*', TOT) SNOpy, anyone? -- http://mail.python.org/mailman/listinfo/python-list
Re: get method
James Mills wrote:
On Tue, Dec 30, 2008 at 7:10 PM, Roel Schroeven
rschroev_nospam...@fastmail.fm wrote:
Hm, you just changed an O(n) algorithm to an O(n**2) algorithm. No big
deal for short strings, but try your solution on a string with length
1 and see the difference. On my computer the O(n) version takes
0.008 seconds, while your version takes 8.6 seconds. That's 1000 times
slower.
Yes you are right :) Sadly :/
I wonder if there is a way to implement
the same thing with close to O(n)
complexity using list/dict comprehensions.
A while back I posted a Python implementation of 'bag' (also called a
multiset). The code would then become something like:
s = James Mills and Danielle Van Sprang
dict(bag(s).iteritems())
{'a': 5, ' ': 5, 'e': 3, 'd': 1, 'g': 1, 'i': 2, 'm': 1, 'J': 1, 'M': 1,
'l': 4, 'n': 4, 'p': 1, 's': 2, 'r': 1, 'V': 1, 'S': 1, 'D': 1}
--
http://mail.python.org/mailman/listinfo/python-list
Re: Understanding search queries, semantics, and Meaning ...aren't we all looking for meaning?
library, as I'm looking to learn to fish, so to speak, and to learn a
bit about the biology of fish.
I'm going to break rule #1 of your requirements but in an unexpected
way. Rather than studying PostgreSQL, MySQL, or Oracle, why don't you
crack open the topic of relational database theory and relational
algebra? That should help with the big thinking bit in the same way
understanding 1+1 helps you understand how to add any two numbers
Ah, exactly, thanks for understanding my original post :) I guess
'relational database theory' is what I should start looking around
for... otherwise googling on sql, queries, etc., just returns how to
execute a query, but says nothing about how a fish works.
In a typical SQL database, when you type in SELECT foo FROM bar WHERE
baz='bo', you are not writing a program, at least not in the sense of
Python or C or Java or Perl where you give instructions on HOW to run
the program. You are writing a program in the sense of Lisp or Scheme
or Haskell in that you are giving instructions on WHAT the program is.
I've gotten a strong inkling that parsing a query yields new code,
(lambdas) that are created on the fly to do the search.
table and then start going through the process of elimination removing
rows it doesn't want to think about. That's highly inefficient, of
Exactly what I'm trying to avoid.
course. Instead, it transforms the query (the WHAT) into a set of
procedures that describe HOW to get the result.
For now I'm not parsing actual text queries... my real search query
is coded directly in python like this:
p1 = lambda: db.search_leaf('x location', 'lte', 5)
p2 = lambda: db.search_leaf('footprint', 'eq', '0603')
p3 = lambda: db.search(db.AND, p1, p2)
p4 = lambda: db.search_leaf('x location', 'gte', 19)
p5 = lambda: db.search_leaf('footprint', 'eq', '0402')
p6 = lambda: db.search(db.AND, p1, p2)
fc = db.search(db.OR, p3, p4)
this particular example doesn't necessarily make any sense, but in
effect I'm trying to string together lambda functions which are
created explicitly for the individual query, then strung together with
combiner functions.
Oh, by the way, this step is nondeterministic. Why? Well, no one can
really say what the BEST way to run any sufficiently complicated
program is. We can point out good ways and bad ways, but not the best
way. It's like the travelling salesman problem in a way.
The nondeterministic stuff... wow, I've come across (call/cc...),
(require...), and different variants of this, both in sicp, teach
yourself scheme, the plt docs, other places, etc., but it still eludes
me. I'm afraid that unless I understand it, I'll wind up creating
some type of cargo-cult mimcry of a database without doing it right
(http://en.wikipedia.org/wiki/Cargo_cult)
programmer, will be infinitely better for it. When you understand it,
you will really unlock the full potential of Python and Scheme and
whatever other language is out there because you will see how to go
from HOW languages to WHAT languages.
I'm under the impression python doesn't have the internal guts for
nondeterministic programming, specifcially that its lambdas are
limited to single-line expressions, but I may be wrong here.
Still, at the begining of the nondeterministic section of SICP
(section 4.3), it says nondeterministic computing... is useful for
'generate and test' applications, which almost categorically sounds
like an element-by-element search for what you're looking for. This
was my initial intention behind (prematurely?) optimizing the database
by using parameter keys instead of something like [x for x in stuff if
pred(x, val)] filtering.
Programmers who can write compilers are GOOD programmers. Programmers
who can understand someone else's compilers are even better.
Does incorporating a search capability in an application necessarily
mean I'm writing a search compiler? That seems overkill for the
specific case, but may be true generally. For instance, if a word
processing app allows you to search for characters with a certain
font, is that incorporating a search compiler and query language? Or
is it just brute-force filtering?
That's my semi-guru advice.
Thanks :)
Michael
--
http://mail.python.org/mailman/listinfo/python-list
[ANN]: circuits-1.0b1 released!
Hi all, I'm pleased to announce the release of circuits-1.0b1 Overview == circuits is an event-driven framework with a focus on Component Software Architectures where System Functionality is defined in Components. Components communicate with one another by propagating events throughout the system. Each Component can react to events and expose events to other parts of the system Components are able to manage their own events and can also be linked to other Components. circuits has a clean architecture and has no external dependencies on any other library. It's simplistic design is unmatchable but yet delivers a powerful framework for building large, scalable, maintainable applications and systems. circuits was a core integral part of the pymills library developed in 2006 and was partly inspired by the Trac architecture. Quick Examples Hello World! from circuits.core import listener, Component, Event, Manager class Hello(Component): ... @listener(hello) ... def onHELLO(self): ... print Hello World! manager = Manager() manager += Hello() manager.push(Event(), hello) manager.flush() Hello World! Hello Web! ~~~ from circuits.lib.web import Server, Controller class HelloWorld(Controller): def index(self): return Hello World! server = Server(8000) server += HelloWorld() server.run() Hello Web! (WSGI) from circuits.lib.web import Application, Controller class HelloWorld(Controller): def index(self): return Hello World! application = Application() application += HelloWorld() Download circuits using easy_install or from here: http://trac.softcircuit.com.au/circuits/downloads or from the Python Package Index. Please visit the circuits website for more information about circuits, or to file bug reports or enhancements. http://trac.softcircuit.com.au/circuits/ --JamesMills -- -- -- Problems are solved by method -- http://mail.python.org/mailman/listinfo/python-list
Re: get method
On Wed, Dec 31, 2008 at 9:15 AM, MRAB goo...@mrabarnett.plus.com wrote: (snip) A while back I posted a Python implementation of 'bag' (also called a multiset). The code would then become something like: What complexity is this ? cheers James -- http://mail.python.org/mailman/listinfo/python-list
Re: Triple quoted string in exec function ?
En Tue, 30 Dec 2008 18:16:39 -0200, Stef Mientki stef.mien...@gmail.com escribió: ibpe...@gmail.com wrote: the message Steven sent you is ok to explain how to work with triple quote Yes, but not to work around my problem. I guess I've to remove all triple quoted strings from my code. Why so? You only have to avoid repeating the *same* kind of triple quotes inside a triple quoted string literal. But you are free to use any other combination of ' and , even include the surrounding triple quotes escape inside (escaping them with \) x = This is line 'one'. ... ''line two starts '''here ... and this is line three'' ... this last line contains \\\ too print x This is line 'one'. ''line two starts '''here and this is line three'' this last line contains too With so much freedom I can't imagine what prevents you from using triple quoted strings. -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Python 3.0 Curses Unicode
Just resolved the issue (turned out to be an issue with linked ncurses
libraries). If others run into this discussion of the solution can be found
at:
http://bugs.python.org/issue4787
Cheers! -Damian
On Mon, Dec 29, 2008 at 10:30 PM, Damian Johnson atag...@gmail.com wrote:
It seems as if the curses module in Python 3.0 isn't respecting the
system's preferred encoding (utf-8) which was set via:
locale.setlocale(locale.LC_ALL, '')
The purpose of this was described at the top of '
http://docs.python.org/dev/3.0/library/curses.html#module-curses'. The
getlocale function is reporting the proper values ('en_US', 'UTF8') but
addstr is clearly not treating it as Unicode - is this a bug? -Damian
2008/12/28 Damian Johnson atag...@gmail.com
Hi, I've switched to Python 3.0 for a new Japanese vocab quizzing
application due to its much improved Unicode support. However, I'm running
into an issue with displaying Unicode characters via curses. In Python 2.x a
simple hello-world looks like:
#!/usr/bin/python
# coding=UTF-8
import curses
import locale
locale.setlocale(locale.LC_ALL,)
def doStuff(stdscr):
message = uhello わたし!
stdscr.addstr(0, 0, message.encode(utf-8), curses.A_BLINK)
stdscr.getch() # pauses until a key's hit
curses.wrapper(doStuff)
This works. However, when I try to come up with an equivalent for Python
3.0:
#!/usr/bin/python
import curses
import locale
locale.setlocale(locale.LC_ALL,)
def doStuff(stdscr):
message = hello わたし!
stdscr.addstr(0, 0, message, curses.A_BLINK)
stdscr.getch() # pauses until a key's hit
curses.wrapper(doStuff)
It fails (printing gibberish to the console). Anyone have a clue what I'm
doing wrong? Thanks! -Damian
PS. Is the # coding=UTF-8 header meaningless in Python 3.0? Also, is
locale.setlocale(locale.LC_ALL,) still necessary for getting curses to
provide Unicode support?
--
http://mail.python.org/mailman/listinfo/python-list
