Re: is_iterable function.

2007-07-26 Thread Bruno Desthuilliers
Neil Cerutti a écrit :
 def is_iterable(obj):
 try:
 iter(obj)
 return True
 except TypeError:
 return False
 
 Is there a better way?

The only other alternative I see is worse:

def iterable(obj):
   # strings are iterable and don't have an __iter__ method...
   for name in ('__iter__', '__getitem__'):
 try:
   getattr(obj, name)
   return True
 except AttributeError:
   pass
  else:
return False


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-26 Thread Neil Cerutti
On 2007-07-26, George Sakkis [EMAIL PROTECTED] wrote:
 That's not the only problem; try a string element to see it
 break too. More importantly, do you *always* want to handle
 strings as iterables ?

 The best general way to do what you're trying to is pass
 is_iterable() as an optional argument with a sensible default,
 but allow the user to pass a different one that is more
 appropriate for the task at hand:

 def is_iterable(obj):
 try: iter(obj)
 except: return False
 else: return True

 def flatten(obj, is_iterable=is_iterable):

That makes good sense.

Plus the subtly different way you composed is_iterable is clearer
than what I originally wrote. I haven't ever used a try with an
else.

 if is_iterable(obj):
 for item in obj:
 for flattened in flatten(item, is_iterable):
 yield flattened
 else:
 yield obj

 By the way, it's bad design to couple two distinct tasks:
 flattening a (possibly nested) iterable and applying a function
 to its elements. Once you have a flatten() function,
 deeply_mapped is reduced down to itertools.imap.

I chose to implement deeply_mapped because it illustrated the
problem of trying to catch a TypeError exception when one might
be thrown by some other code. I agree with your opinion that it's
a design flaw, and most of my problems with the code were caused
by that flaw.

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-26 Thread Marc 'BlackJack' Rintsch
On Thu, 26 Jul 2007 15:02:39 +, Neil Cerutti wrote:

 Based on the discussions in this thread (thanks all for your
 thoughts), I'm settling for:
 
 def is_iterable(obj):
   try:
 iter(obj).next()
 return True
   except TypeError:
 return False
   except KeyError:
 return False
 
 The call to iter will fail for objects that don't support the
 iterator protocol, and the call to next will fail for a
 (hopefully large) subset of the objects that don't support the
 sequence protocol.

And the `next()` consumes an element if `obj` is not re-iterable.

Ciao,
Marc 'BlackJack' Rintsch
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-26 Thread Neil Cerutti
Based on the discussions in this thread (thanks all for your
thoughts), I'm settling for:

def is_iterable(obj):
  try:
iter(obj).next()
return True
  except TypeError:
return False
  except KeyError:
return False

The call to iter will fail for objects that don't support the
iterator protocol, and the call to next will fail for a
(hopefully large) subset of the objects that don't support the
sequence protocol.

This seems preferable to cluttering code with exception handling
and inspecting tracebacks. But it's still basically wrong, I
guess.

To repost my use case:

def deeply_mapped(func, iterable):
 Recursively apply a function to every item in a iterable object,
recursively descending into items that are iterable. The result is an
iterator over the mapped values. Similar to the builtin map function, func
may be None, causing the items to returned unchanged.

 import functools
 flattened = functools.partial(deeply_mapped, None)
 list(flattened([[1], [2, 3, []], 4]))
[1, 2, 3, 4]

 list(flattened(((1), (2, 3, ()), 4)))
[1, 2, 3, 4]

 list(flattened(]]], 1, 2, 3, 4]))
[1, 2, 3, 4]

 list(flattened([1, [[[2, 3]], 4]]))
[1, 2, 3, 4]


for item in iterable:
if is_iterable(item):
for it in deeply_mapped(func, item):
if func is None:
yield it
else:
yield func(it)
else:
if func is None:
yield item
else:
yield func(item)
-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-26 Thread Neil Cerutti
On 2007-07-26, Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote:
 On Thu, 26 Jul 2007 15:02:39 +, Neil Cerutti wrote:

 Based on the discussions in this thread (thanks all for your
 thoughts), I'm settling for:
 
 def is_iterable(obj):
   try:
 iter(obj).next()
 return True
   except TypeError:
 return False
   except KeyError:
 return False
 
 The call to iter will fail for objects that don't support the
 iterator protocol, and the call to next will fail for a
 (hopefully large) subset of the objects that don't support the
 sequence protocol.

 And the `next()` consumes an element if `obj` is not
 re-iterable.

Crap.

So how best to imlement deeply_mapped?

The following still breaks for objects that don't provide
__iter__, do provide __getitem__, but don't support the sequence
protocol.

def deeply_mapped(func, iterable):
 Recursively apply a function to every item in a nested container,
recursively descending into items that are iterable. The result is an
iterator over the mapped values. Similar to the builtin map function, func
may be None, causing the items to returned unchanged.

 import functools
 flattened = functools.partial(deeply_mapped, None)
 list(flattened([[1], [2, 3, []], 4]))
[1, 2, 3, 4]

 list(flattened(((1), (2, 3, ()), 4)))
[1, 2, 3, 4]

 list(flattened(]]], 1, 2, 3, 4]))
[1, 2, 3, 4]

 list(flattened([1, [[[2, 3]], 4]]))
[1, 2, 3, 4]

 def magic(o):
... if o == 3:
... raise TypeError('Three is a magic number')
... return o

 list(deeply_mapped(magic, [1, [[[2, 3]], 4]]))
Traceback (most recent call last):
...
TypeError: Three is a magic number


frame = inspect.currentframe()
info = inspect.getframeinfo(frame)
filename = info[0]
funcname = info[2]
for item in iterable:
try:
for it in deeply_mapped(func, item):
if func is None:
yield it
else:
yield func(it)
except TypeError:
eframe = inspect.trace()[-1]
efilename = eframe[1]
efuncname = eframe[3]
if efilename != filename or efuncname != funcname:
raise
if func is None:
yield item
else:
yield func(item)

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-26 Thread George Sakkis
On Jul 26, 2:56 pm, Neil Cerutti [EMAIL PROTECTED] wrote:
 On 2007-07-26, Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote:



  On Thu, 26 Jul 2007 15:02:39 +, Neil Cerutti wrote:

  Based on the discussions in this thread (thanks all for your
  thoughts), I'm settling for:

  def is_iterable(obj):
try:
  iter(obj).next()
  return True
except TypeError:
  return False
except KeyError:
  return False

  The call to iter will fail for objects that don't support the
  iterator protocol, and the call to next will fail for a
  (hopefully large) subset of the objects that don't support the
  sequence protocol.

  And the `next()` consumes an element if `obj` is not
  re-iterable.

 Crap.

 So how best to imlement deeply_mapped?

 The following still breaks for objects that don't provide
 __iter__, do provide __getitem__, but don't support the sequence
 protocol.

That's not the only problem; try a string element to see it break too.
More importantly, do you *always* want to handle strings as
iterables ?

The best general way to do what you're trying to is pass is_iterable()
as an optional argument with a
sensible default, but allow the user to pass a different one that is
more appropriate for the task at hand:

def is_iterable(obj):
try: iter(obj)
except: return False
else: return True

def flatten(obj, is_iterable=is_iterable):
if is_iterable(obj):
for item in obj:
for flattened in flatten(item, is_iterable):
yield flattened
else:
yield obj

from functools import partial
flatten_nostr = partial(flatten, is_iterable=lambda obj: not
isinstance(obj,basestring)
 and
is_iterable(obj))

print list(flatten_nostr([1, [[[2, 'hello']], (4, u'world')]]))


By the way, it's bad design to couple two distinct tasks: flattening a
(possibly nested) iterable and applying a function to its elements.
Once you have a flatten() function, deeply_mapped is reduced down to
itertools.imap.

HTH,
George

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread Carsten Haese
On Wed, 2007-07-25 at 11:58 -0700, [EMAIL PROTECTED] wrote:
 You can use the built-in dir() function to determine whether or not
 the __iter__ method exists:
 
 class Iterable(object):
 def __iter__(self):
 pass
 
 class NotIterable(object):
 pass
 
 def is_iterable(thing):
 return '__iter__' in dir(thing)
 
 print 'list is iterable = ', is_iterable(list())
 print 'int is iterable = ', is_iterable(10)
 print 'float is iterable = ', is_iterable(1.2)
 print 'dict is iterable = ', is_iterable(dict())
 print 'Iterable is iterable = ', is_iterable(Iterable())
 print 'NotIterable is iterable = ', is_iterable(NotIterable())
 
 Results:
 list is iterable =  True
 int is iterable =  False
 float is iterable =  False
 dict is iterable =  True
 Iterable is iterable =  True
 NotIterable is iterable =  False

Testing for __iter__ alone is not enough:

 class X(object):
...   def __getitem__(self,i):
... if i10: return i
... else: raise IndexError, i
... 
 x = X()
 is_iterable(x)
False
 iter(x)
iterator object at 0xb7f0182c
 for i in x: print i
... 
0
1
2
3
4
5
6
7
8
9

No __iter__ in sight, but the object is iterable.

-- 
Carsten Haese
http://informixdb.sourceforge.net


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread Marc 'BlackJack' Rintsch
On Wed, 25 Jul 2007 11:58:40 -0700, [EMAIL PROTECTED] wrote:

 You can use the built-in dir() function to determine whether or not
 the __iter__ method exists:

Doesn't work:

In [58]: is_iterable('hello')
Out[58]: False

But strings *are* iterable.

And just calling `iter()` doesn't work either:

In [72]: class A:
   : def __getitem__(self, key):
   : if key == 42:
   : return 'answer'
   : raise KeyError
   :

In [73]: iter(A())
Out[73]: iterator object at 0xb7829b2c

In [74]: a = iter(A())

In [75]: a.next()
---
type 'exceptions.KeyError'Traceback (most recent call last)

/home/bj/ipython console in module()

/home/bj/ipython console in __getitem__(self, key)

type 'exceptions.KeyError':


So there's no reliable way to test for iterables other than actually
iterate over the object.

Ciao,
Marc 'BlackJack' Rintsch
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread Neil Cerutti
On 2007-07-25, Carsten Haese [EMAIL PROTECTED] wrote:
 On Wed, 2007-07-25 at 19:26 +, Neil Cerutti wrote:
 Speaking of the iter builtin function, is there an example of the
 use of the optional sentinel object somewhere I could see?

 Example 1: If you use a DB-API module that doesn't support direct cursor
 iteration with for row in cursor, you can simulate it this way:

 for row in iter(cursor.fetchone, None):
# do something

 Example 2: Reading a web page in chunks of 8kB:

 f = urllib.urlopen(url)
 for chunk in iter(lambda:f.read(8192), ):
   # do something

Ah! Thanks for the examples. That's much simpler than I was
imagining. It's also somewhat evil, but I suppose it conserves a
global name to do it that way.

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread Neil Cerutti
On 2007-07-25, Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote:
 So there's no reliable way to test for iterables other than
 actually iterate over the object.

A TypeError exception is perhaps too generic for comfort in this
use case:

def deeply_mapped(func, iterable):
  for item in iterable:
try:
  for it in flattened(item):
yield func(it)
except TypeError:
  yield func(item)

I'd be more confortable excepting some sort of IterationError (or
using an is_iterable function, of course). I guess there's always
itertools. ;)

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread Marc 'BlackJack' Rintsch
On Wed, 25 Jul 2007 15:46:14 -0400, Carsten Haese wrote:

 On Wed, 2007-07-25 at 19:11 +, Marc 'BlackJack' Rintsch wrote:
 And just calling `iter()` doesn't work either:
 
 In [72]: class A:
: def __getitem__(self, key):
: if key == 42:
: return 'answer'
: raise KeyError
:
 
 In [73]: iter(A())
 Out[73]: iterator object at 0xb7829b2c
 
 In [74]: a = iter(A())
 
 In [75]: a.next()
 ---
 type 'exceptions.KeyError'Traceback (most recent call last)
 
 /home/bj/ipython console in module()
 
 /home/bj/ipython console in __getitem__(self, key)
 
 type 'exceptions.KeyError':
 
 
 So there's no reliable way to test for iterables other than actually
 iterate over the object.
 
 You seem to say that your 'a' object is not iterable. I disagree. While
 it's true that it raises an exception upon retrieval of the zeroth
 iteration, that situation is quite different from attempting to iterate
 over the number 10, where you can't even ask for a zeroth iteration.

But it raises that exception on every attempt to iterate and was clearly
not meant to be iterable.  The docs say objects that offer no `__iter__()`
but a `__getitem__()` are iterable if this `__getitem__()` can be called
with consecutive integers from zero up and if there is an upper limit it
must be signaled by an `IndexError`.  The object above doesn't follow this
protocol.  For me an iterable is about behavior.  If it doesn't quack
like a duck…

 To illustrate this point further, imagine you write an object that
 iterates over the lines of text read from a socket. If the connection is
 faulty and closes the socket before you read the first line, the zeroth
 iteration raises an exception. Does that mean the object is iterable or
 not depending on the reliability of the socket connection? I find that
 notion hard to swallow.

It is an iterable as there is at least the chance or the intent of the
programmer that it will behave like an iterable.  The duck tries hard to
quack but may be hoarse.  :-)

Ciao,
Marc 'BlackJack' Rintsch
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: is_iterable function.

2007-07-25 Thread Duncan Booth
Neil Cerutti [EMAIL PROTECTED] wrote:

 Speaking of the iter builtin function, is there an example of the
 use of the optional sentinel object somewhere I could see?

for line in iter(open('somefile.txt', 'r').readline, ''):
print line

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread Carsten Haese
On Wed, 2007-07-25 at 19:26 +, Neil Cerutti wrote:
 Speaking of the iter builtin function, is there an example of the
 use of the optional sentinel object somewhere I could see?

Example 1: If you use a DB-API module that doesn't support direct cursor
iteration with for row in cursor, you can simulate it this way:

for row in iter(cursor.fetchone, None):
   # do something

Example 2: Reading a web page in chunks of 8kB:

f = urllib.urlopen(url)
for chunk in iter(lambda:f.read(8192), ):
  # do something

HTH,

-- 
Carsten Haese
http://informixdb.sourceforge.net


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread Steve Holden
Carsten Haese wrote:
 On Wed, 2007-07-25 at 19:26 +, Neil Cerutti wrote:
 Speaking of the iter builtin function, is there an example of the
 use of the optional sentinel object somewhere I could see?
 
 Example 1: If you use a DB-API module that doesn't support direct cursor
 iteration with for row in cursor, you can simulate it this way:
 
 for row in iter(cursor.fetchone, None):
# do something
 
[...]
This would, of course, be a horribly inefficient way to handle a 
database result with 1,500,000 rows. Calling fetchall() might also have 
its issues. The happy medium is to use a series of calls to fetchmany(N) 
with an appropriate value of N.

regards
  Steve
-- 
Steve Holden+1 571 484 6266   +1 800 494 3119
Holden Web LLC/Ltd   http://www.holdenweb.com
Skype: holdenweb  http://del.icio.us/steve.holden
--- Asciimercial --
Get on the web: Blog, lens and tag the Internet
Many services currently offer free registration
--- Thank You for Reading -

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread Jeff McNeil
That's not going to hold true for generator functions or iterators
that are implemented on sequences without '__iter__.'   You might also
want to check for __getitem__.  I'm not sure if there's a way to tell
if a function is a generator without actually calling it.

-Jeff

On 7/25/07, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
 You can use the built-in dir() function to determine whether or not
 the __iter__ method exists:

 class Iterable(object):
 def __iter__(self):
 pass

 class NotIterable(object):
 pass

 def is_iterable(thing):
 return '__iter__' in dir(thing)

 print 'list is iterable = ', is_iterable(list())
 print 'int is iterable = ', is_iterable(10)
 print 'float is iterable = ', is_iterable(1.2)
 print 'dict is iterable = ', is_iterable(dict())
 print 'Iterable is iterable = ', is_iterable(Iterable())
 print 'NotIterable is iterable = ', is_iterable(NotIterable())

 Results:
 list is iterable =  True
 int is iterable =  False
 float is iterable =  False
 dict is iterable =  True
 Iterable is iterable =  True
 NotIterable is iterable =  False

 On Jul 25, 12:24 pm, Neil Cerutti [EMAIL PROTECTED] wrote:
  def is_iterable(obj):
  try:
  iter(obj)
  return True
  except TypeError:
  return False
 
  Is there a better way?
 
  --
  Neil Cerutti


 --
 http://mail.python.org/mailman/listinfo/python-list

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread Neil Cerutti
On 2007-07-25, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
 You can use the built-in dir() function to determine whether or not
 the __iter__ method exists:

I think you might have to check for __getitem__, also, which may
support the sequence protocol.

 def is_iterable(thing):
 return '__iter__' in dir(thing)

So then:

def is_iterable(thing):
  return '__iter__' in dir(thing) or '__getitem__' in dir(thing)

Speaking of the iter builtin function, is there an example of the
use of the optional sentinel object somewhere I could see?

-- 
Neil Cerutti
-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread [EMAIL PROTECTED]
You can use the built-in dir() function to determine whether or not
the __iter__ method exists:

class Iterable(object):
def __iter__(self):
pass

class NotIterable(object):
pass

def is_iterable(thing):
return '__iter__' in dir(thing)

print 'list is iterable = ', is_iterable(list())
print 'int is iterable = ', is_iterable(10)
print 'float is iterable = ', is_iterable(1.2)
print 'dict is iterable = ', is_iterable(dict())
print 'Iterable is iterable = ', is_iterable(Iterable())
print 'NotIterable is iterable = ', is_iterable(NotIterable())

Results:
list is iterable =  True
int is iterable =  False
float is iterable =  False
dict is iterable =  True
Iterable is iterable =  True
NotIterable is iterable =  False

On Jul 25, 12:24 pm, Neil Cerutti [EMAIL PROTECTED] wrote:
 def is_iterable(obj):
 try:
 iter(obj)
 return True
 except TypeError:
 return False

 Is there a better way?

 --
 Neil Cerutti


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread George Sakkis
On Jul 25, 3:26 pm, Neil Cerutti [EMAIL PROTECTED] wrote:

 Speaking of the iter builtin function, is there an example of the
 use of the optional sentinel object somewhere I could see?

# iterate over random numbers from 1 to 10; use 0 as a sentinel to
stop the iteration
for n in iter(lambda:random.randrange(10), 0):
print n

More generally, iter(callable, sentinel) is just a convenience
function for the following generator:

def iter(callable, sentinel):
while True:
c = callable()
if c == sentinel: break
yield c


George

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: is_iterable function.

2007-07-25 Thread Carsten Haese
On Wed, 2007-07-25 at 19:11 +, Marc 'BlackJack' Rintsch wrote:
 And just calling `iter()` doesn't work either:
 
 In [72]: class A:
: def __getitem__(self, key):
: if key == 42:
: return 'answer'
: raise KeyError
:
 
 In [73]: iter(A())
 Out[73]: iterator object at 0xb7829b2c
 
 In [74]: a = iter(A())
 
 In [75]: a.next()
 ---
 type 'exceptions.KeyError'Traceback (most recent call last)
 
 /home/bj/ipython console in module()
 
 /home/bj/ipython console in __getitem__(self, key)
 
 type 'exceptions.KeyError':
 
 
 So there's no reliable way to test for iterables other than actually
 iterate over the object.

You seem to say that your 'a' object is not iterable. I disagree. While
it's true that it raises an exception upon retrieval of the zeroth
iteration, that situation is quite different from attempting to iterate
over the number 10, where you can't even ask for a zeroth iteration.

To illustrate this point further, imagine you write an object that
iterates over the lines of text read from a socket. If the connection is
faulty and closes the socket before you read the first line, the zeroth
iteration raises an exception. Does that mean the object is iterable or
not depending on the reliability of the socket connection? I find that
notion hard to swallow.

-- 
Carsten Haese
http://informixdb.sourceforge.net


-- 
http://mail.python.org/mailman/listinfo/python-list