Re: Infinite loops and synchronization

[Aahz]

In article ,
Vincent Gulinao   wrote:
>
>lst = list()
>
>while True:
>   if len(lst) == SOME_NUMBER:
>   return lst
>
>Q2: operating on list from threads (mostly appends) must be safe,
>right (synchronization)?

What do you mean by "safe"?  Python certainly won't crash, but there's
no guarantee that the list will be consistent from *your* perspective.
Consider what happens if len(lst) == SOME_NUMBER - 1 and some other part
of your code adds two elements to lst.  You'll skip right over your if
condition.
-- 
Aahz (a...@pythoncraft.com)   <*> http://www.pythoncraft.com/

"If you think it's expensive to hire a professional to do the job, wait
until you hire an amateur."  --Red Adair
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Infinite loops and synchronization

[Lawrence D'Oliveiro]

In message , Vincent 
Gulinao wrote:

> Q1: is this a common OK practice? I'm worried infinite loops hogs memory.

The problem is not that the loop is infinite, but that it busy-waits, 
hogging CPU.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Infinite loops and synchronization

[pdpi]

On Jul 13, 6:06 am, Vincent Gulinao  wrote:
> lst = list()
>
> (lst populated by async twisted deferred callbacks)
>
> while True:
>         if len(lst) == SOME_NUMBER:
>                 return lst
>
> Q1: is this a common OK practice? I'm worried infinite loops hogs memory.
> Q2: operating on list from threads (mostly appends) must be safe,
> right (synchronization)?

Q1: I'll answer your question with another. What's so fundamentally
different between your infinite loop and this one:

while len(lst) != SOME_NUMBER:
  pass
return lst

which is not an "infinite loop"[1]. Why would yours be any worse in
terms of memory than mine? Are you allocating anything that would hog
memory? Of course, like Piet said, it *will* hog your CPU, so you want
a time.sleep(.1) in there, at the least. Of course, the question is:
why aren't you using a semaphore to let you know you can proceed, and
make the provider increment the semaphore?

[1] -- well, it can be, if len(lst) == SOME_NUMBER never comes about,
and I'd hazard a guess that that's pretty much where your fear of
memory hogging comes from: it's easy to allocate stuff and not
deallocate it within a cycle, only to find the bounds on that cycle
going awry.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Infinite loops and synchronization

[Piet van Oostrum]

> Vincent Gulinao  (VG) wrote:

>VG> lst = list()
>VG> (lst populated by async twisted deferred callbacks)

>VG> while True:
>VG>if len(lst) == SOME_NUMBER:
>VG>return lst

>VG> Q1: is this a common OK practice? I'm worried infinite loops hogs memory.
>VG> Q2: operating on list from threads (mostly appends) must be safe,
>VG> right (synchronization)?

I am not familiar enough with twisted, but I think the principle is
independent from twisted.

This loop will not hog memory but it will hog CPU time.

You should use a synchronisation construct like threading.Condition or a
Semaphore. Here is my suggestion with a Condition:

Global somewhere:
   lst_cond = Condition()

In your loop:

lst = list()  # Why not lst = []?

while True: # strange while/if combo
if len(lst) == SOME_NUMBER:
return lst

Make that:

with lst_cond:
while len(lst) < SOME_NUMBER:
lst_cond.wait()
return lst

In the callback:

with lst_cond:
lst.append(new_value)
lst_cond.notify()

In case you don't have a python that supports the with statement (not
even `from future') you should use:

lst_cond.acquire()
try:
.
finally:
lst_cond.release()

I think the solution with a semaphore is less elegant.

global: sem = Semaphore()

loop:
for i in range(SOME_NUMBER): 
sem.acquire()
return lst

In callback:

lst.append(new_value)
sem.release()

*Be careful: I haven't tested this code (not even syntax checked). So
consider it pseudo code.*
-- 
Piet van Oostrum 
URL: http://pietvanoostrum.com [PGP 8DAE142BE17999C4]
Private email: p...@vanoostrum.org
-- 
http://mail.python.org/mailman/listinfo/python-list

Infinite loops and synchronization

[Vincent Gulinao]

lst = list()

(lst populated by async twisted deferred callbacks)

while True:
if len(lst) == SOME_NUMBER:
return lst

Q1: is this a common OK practice? I'm worried infinite loops hogs memory.
Q2: operating on list from threads (mostly appends) must be safe,
right (synchronization)?
-- 
http://mail.python.org/mailman/listinfo/python-list