En Mon, 29 Dec 2008 08:17:42 -0200, k3xji <sum...@gmail.com> escribió:
On 29 Aralık, 11:52, "Gabriel Genellina" <gagsl-...@yahoo.com.ar>
wrote:
En Mon, 29 Dec 2008 05:56:10 -0200, k3xji <sum...@gmail.com> escribió:
> I am trying to see on which situations does the Read-Write Lock
> performs better on primitive Lock() itself. Below is the code I am
> using to test the performance:
> import threading
> import locks
> import time
> class mylock(object):
(I'm not convinced your lock is correct)
No problem.:)
> GLOBAL_VAR = 1
> #GLOBAL_LOCK = locks.ReadWriteLock()
> GLOBAL_LOCK = threading.Lock()
> #GLOBAL_LOCK = mylock()
> GLOBAL_LOOP_COUNT = 100000
> GLOBAL_READER_COUNT = 1000
> GLOBAL_WRITER_COUNT = 1
Only one writer? If this is always the case, you don't need a lock at
all.
No just used for testing. It does not matter, what I want to see is
that I want to create a botleneck on readers.
Is this a theoretical question? What do your readers/writers actually do?
Just reading a global variable does not require any locking. If it is
being modified, you either get the previous value, or the new value.
Note that the thread acquires the lock ONCE, repeats several thousand
times an assignment to a *local* variable called GLOBAL_VAR (!), finally
releases the lock and exits. As every thread does the same, they just
run
one after another, they never have a significant overlap.
If I put the for loop outside, in that case, readers will not overlap
at all, and you would be amazed by the numbers for that test. They
indicate primitive lock is faster than read-write lock, as it requires
the lock, executes only one bytecode operation and releases the lock.
So, in order to create a bottlenneck on readers, we need to somehow do
not release the lock immediately.
Again, what is your specific use case? why do you think readers will see a
bottleneck?
With that, primitive locks perform 10 times better than Read-Write
lock. See above.
So what? You don't like such results? The read-write lock in the recipe
you posted is rather complex, and isn't obvious whether it is correct or
not, and the author doesn't prove it nor provide a reference. I'd stick to
the standard Lock object unless I have specific needs.
Note that threading.Lock objects are implemented in C and don't have
significant overhead (other than the wait itself); even threading.RLock
objects are a lot slower. So depending on what you really have to do, the
overhead of using a class like ReadWriteLock may be worse than using a
standard Lock.
Hmmm, it's a reader but attempts to modify the value?
You don't have to protect a read operation on a global variable - so a
lock isn't required here.
This is just for testing. Suppose that I am actually reading the
value. I don't understand why a lock is not required? Are you saying
lock is not necesary beacuse GLOBAL_VALUE is an immutable object, if
then, suppose it is not. This is just a test. Suppose GLOBAl_VAR is a
list and we are calling append() on it which is nt an atomic
operation.
It doesn't matter whether it is mutable or immutable. Remember that, in
Python, a global variable is just a *name* that refers to an object; you
either get the old object referred, or the new one. If it is a list and it
is being modified "in place", you always get the same list -- its contents
may differ from one instant to another so iterating over it isn't safe.
That's why I insist on knowing your use case: you may me doing things more
complicated than they should.
Attached there is a modified test using a shared list. A group of writers
append elements; another group of writers pop elements. Readers
continuously check the list sanity. The list starts empty, should be empty
at the end, and should be equal to range(len(global_list)) at any time.
Run the code as-is, with all locks enabled, and it should work fine.
Disable the locking in each place, and things start going wrong.
> What I am doing is: I am creating multiple readers and try to do
> something. I had assumed that with using primitive Lock() on the above
> situation, it will create a bottleneck on the rthreads. But the
> numbers indicate that there are no difference at all. I had
> implemented my own READ-WRIET lock as can be seen above mylock and
> also used the one here: code.activestate.com/recipes/502283/.
I hope you now understand why you got the same numbers always.
Unfortunately, I do not understand anyhing.
Each thread in your original code were like this:
begin thread; wait for lock; execute something locally; release lock; exit
thread;
There was no shared state between threads. Even if you fix them to share
some variable, they were a single, sequential, piece of code (the inner
loop is irrelevant here). Once a thread gets the lock, it just runs to its
end. No one waits more than once for the lock. So all of them run
sequentially, in arbitrary order, but just one after the other. They never
overlap.
--
Gabriel Genellina
import sys
import threading
import time
import random
GLOBAL_VAR = []
GLOBAL_LOCK = threading.Lock()
STOP = False
GLOBAL_LOOP_COUNT = 1000
class wthread(threading.Thread): pass
class wthread_append(wthread):
def run(self):
for i in xrange(GLOBAL_LOOP_COUNT):
GLOBAL_LOCK.acquire()
try:
GLOBAL_VAR.append(len(GLOBAL_VAR))
finally:
GLOBAL_LOCK.release()
pass
class wthread_pop(wthread):
def run(self):
# a bit more complex, because you can't pop from an empty list
i = 0
while i<GLOBAL_LOOP_COUNT:
GLOBAL_LOCK.acquire()
try:
ok = False
try: GLOBAL_VAR.pop()
except IndexError: pass
else: ok = True
finally:
GLOBAL_LOCK.release()
pass
if ok: i += 1
else: time.sleep(0) # give other threads a chance to run
class rthread(threading.Thread):
def run(self):
while not STOP:
GLOBAL_LOCK.acquire()
try:
l1 = len(GLOBAL_VAR)
for i, item in enumerate(GLOBAL_VAR):
if i!=item:
sys.stdout.write("inconsistent state: GLOBAL_VAR[%d]=%d\n" % (i, item))
break
l2 = len(GLOBAL_VAR)
if l1!=l2:
sys.stdout.write("list changed length: %d -> %d\n" % (l1, l2))
finally:
GLOBAL_LOCK.release()
pass
time.sleep(0) # give other threads a chance to run
#sys.setcheckinterval(1)
starttime = time.clock()
threads = []
for i in range(100):
wt = wthread_append()
threads.append(wt)
wt = wthread_pop()
threads.append(wt)
for i in range(3):
rt = rthread()
threads.append(rt)
random.shuffle(threads)
for thread in threads:
thread.start()
# wait first until all writers have finished
for thread in threads:
if isinstance(thread, wthread):
thread.join()
STOP = True
for thread in threads:
thread.join() # only rthreads should remain active
print "len=", len(GLOBAL_VAR), GLOBAL_VAR[:10]
print "time=", time.clock() - starttime
--
http://mail.python.org/mailman/listinfo/python-list
