Re: Infinite loops and synchronization

2009-07-14 Thread Aahz 
In article mailman.3048.1247462046.8015.python-l...@python.org,
Vincent Gulinao  vincent.guli...@gmail.com wrote:

lst = list()

while True:
   if len(lst) == SOME_NUMBER:
   return lst

Q2: operating on list from threads (mostly appends) must be safe,
right (synchronization)?

What do you mean by safe?  Python certainly won't crash, but there's
no guarantee that the list will be consistent from *your* perspective.
Consider what happens if len(lst) == SOME_NUMBER - 1 and some other part
of your code adds two elements to lst.  You'll skip right over your if
condition.
-- 
Aahz (a...@pythoncraft.com)   * http://www.pythoncraft.com/

If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur.  --Red Adair
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Infinite loops and synchronization

2009-07-13 Thread Piet van Oostrum 
 Vincent Gulinao vincent.guli...@gmail.com (VG) wrote:

VG lst = list()
VG (lst populated by async twisted deferred callbacks)

VG while True:
VGif len(lst) == SOME_NUMBER:
VGreturn lst

VG Q1: is this a common OK practice? I'm worried infinite loops hogs memory.
VG Q2: operating on list from threads (mostly appends) must be safe,
VG right (synchronization)?

I am not familiar enough with twisted, but I think the principle is
independent from twisted.

This loop will not hog memory but it will hog CPU time.

You should use a synchronisation construct like threading.Condition or a
Semaphore. Here is my suggestion with a Condition:

Global somewhere:
   lst_cond = Condition()

In your loop:

lst = list()  # Why not lst = []?

while True: # strange while/if combo
if len(lst) == SOME_NUMBER:
return lst

Make that:

with lst_cond:
while len(lst)  SOME_NUMBER:
lst_cond.wait()
return lst

In the callback:

with lst_cond:
lst.append(new_value)
lst_cond.notify()

In case you don't have a python that supports the with statement (not
even `from future') you should use:

lst_cond.acquire()
try:
.
finally:
lst_cond.release()

I think the solution with a semaphore is less elegant.

global: sem = Semaphore()

loop:
for i in range(SOME_NUMBER): 
sem.acquire()
return lst

In callback:

lst.append(new_value)
sem.release()

*Be careful: I haven't tested this code (not even syntax checked). So
consider it pseudo code.*
-- 
Piet van Oostrum p...@cs.uu.nl
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Infinite loops and synchronization

2009-07-13 Thread pdpi 
On Jul 13, 6:06 am, Vincent Gulinao vincent.guli...@gmail.com wrote:
 lst = list()

 (lst populated by async twisted deferred callbacks)

 while True:
         if len(lst) == SOME_NUMBER:
                 return lst

 Q1: is this a common OK practice? I'm worried infinite loops hogs memory.
 Q2: operating on list from threads (mostly appends) must be safe,
 right (synchronization)?

Q1: I'll answer your question with another. What's so fundamentally
different between your infinite loop and this one:

while len(lst) != SOME_NUMBER:
  pass
return lst

which is not an infinite loop[1]. Why would yours be any worse in
terms of memory than mine? Are you allocating anything that would hog
memory? Of course, like Piet said, it *will* hog your CPU, so you want
a time.sleep(.1) in there, at the least. Of course, the question is:
why aren't you using a semaphore to let you know you can proceed, and
make the provider increment the semaphore?

[1] -- well, it can be, if len(lst) == SOME_NUMBER never comes about,
and I'd hazard a guess that that's pretty much where your fear of
memory hogging comes from: it's easy to allocate stuff and not
deallocate it within a cycle, only to find the bounds on that cycle
going awry.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Infinite loops and synchronization

2009-07-13 Thread Lawrence D'Oliveiro 
In message mailman.3048.1247462046.8015.python-l...@python.org, Vincent 
Gulinao wrote:

 Q1: is this a common OK practice? I'm worried infinite loops hogs memory.

The problem is not that the loop is infinite, but that it busy-waits, 
hogging CPU.

-- 
http://mail.python.org/mailman/listinfo/python-list

Infinite loops and synchronization

2009-07-12 Thread Vincent Gulinao 
lst = list()

(lst populated by async twisted deferred callbacks)

while True:
if len(lst) == SOME_NUMBER:
return lst

Q1: is this a common OK practice? I'm worried infinite loops hogs memory.
Q2: operating on list from threads (mostly appends) must be safe,
right (synchronization)?
-- 
http://mail.python.org/mailman/listinfo/python-list