Re: Record seperator
On Aug 27, 10:45 am, Roy Smith wrote:
> In article <4e592852$0$29965$c3e8da3$54964...@news.astraweb.com>,
> Steven D'Aprano wrote:
>
> > open("file.txt") # opens the file
> > .read() # reads the contents of the file
> > .split("\n\n") # splits the text on double-newlines.
>
> The biggest problem with this code is that read() slurps the entire file
> into a string. That's fine for moderately sized files, but will fail
> (or at least be grossly inefficient) for very large files.
>
> It's always annoyed me a little that while it's easy to iterate over the
> lines of a file, it's more complicated to iterate over a file character
> by character. You could write your own generator to do that:
>
> for c in getchar(open("file.txt")):
> whatever
>
> def getchar(f):
> for line in f:
> for c in line:
> yield c
>
> but that's annoyingly verbose (and probably not hugely efficient).
read() takes an optional size parameter; so f.read(1) is another
option...
>
> Of course, the next problem for the specific problem at hand is that
> even with an iterator over the characters of a file, split() only works
> on strings. It would be nice to have a version of split which took an
> iterable and returned an iterator over the split components. Maybe
> there is such a thing and I'm just missing it?
I don't know if there is such a thing; but for the OP's problem you
could read the file in chunks, e.g.:
def readgroup(f, delim, buffsize=8192):
tail=''
while True:
s = f.read(buffsize)
if not s:
yield tail
break
groups = (tail + s).split(delim)
tail = groups[-1]
for group in groups[:-1]:
yield group
for group in readgroup(open('file.txt'), '\n\n'):
# do something
Cheers - Chas
--
http://mail.python.org/mailman/listinfo/python-list
Re: testing if a list contains a sublist
On Aug 16, 1:37 am, Steven D'Aprano wrote: > On Tue, 16 Aug 2011 04:14 pm ChasBrown wrote: > > > > > On Aug 15, 4:26 pm, Johannes wrote: > >> hi list, > >> what is the best way to check if a given list (lets call it l1) is > >> totally contained in a second list (l2)? > > >> for example: > >> l1 = [1,2], l2 = [1,2,3,4,5] -> l1 is contained in l2 > >> l1 = [1,2,2,], l2 = [1,2,3,4,5] -> l1 is not contained in l2 > >> l1 = [1,2,3], l2 = [1,3,5,7] -> l1 is not contained in l2 > > >> my problem is the second example, which makes it impossible to work with > >> sets insteads of lists. But something like set.issubset for lists would > >> be nice. > > >> greatz Johannes > > > My best guess: > > > from collections import Counter > > There's no reason to think that the Original Poster wants a multiset based > solution. He asked about lists and sublists. That's a standard term, like > substring: > > "12" is a substring of "01234". > "21" and "13" are not. > > [1, 2] is a sublist of [0, 1, 2, 3, 4]. > [2, 1] and [1, 3] are not. > > Since lists are ordered, so are sublists. > That's reasonable; although except in the subject, the OP never uses the term 'sublist'; instead using more ambiguous terms like 'contains', 'is totally contained', etc., with definition by limited example. So it was a bit of a guess on my part of what was wanted. > If the OP does want a solution that ignores order, then he needs to describe > his problem better. As it turns out, in another response the OP says he wants [2,1,2] to be 'contained' by [1,2,2]. But in any case he always has sorted lists, in which case, interestingly, the multiset approach and your more canonical sublist approach yield the same results. Cheers - Chas -- http://mail.python.org/mailman/listinfo/python-list
Re: testing if a list contains a sublist
On Aug 15, 11:51 pm, Laszlo Nagy wrote: > >> hi list, > >> what is the best way to check if a given list (lets call it l1) is > >> totally contained in a second list (l2)? > > >> for example: > >> l1 = [1,2], l2 = [1,2,3,4,5] -> l1 is contained in l2 > >> l1 = [1,2,2,], l2 = [1,2,3,4,5] -> l1 is not contained in l2 > >> l1 = [1,2,3], l2 = [1,3,5,7] -> l1 is not contained in l2 > > >> my problem is the second example, which makes it impossible to work with > >> sets insteads of lists. But something like set.issubset for lists would > >> be nice. > > >> greatz Johannes > > Fastest, error-free and simplest solution is to use sets: > > >>> l1 = [1,2] > >>> l2 = [1,2,3,4,5] > >>> set(l1)-set(l2) > set([]) > >>> set(l2)-set(l1) > set([3, 4, 5]) > >>> > This approach fails the OP's desires when >>> l1 = [1,2,2,] >>> l2 = [1,2,3,4,5] The OP explicitly desires 'l2 contains l1' to be false in that case. Cheers - Chas -- http://mail.python.org/mailman/listinfo/python-list
Re: testing if a list contains a sublist
On Aug 15, 4:26 pm, Johannes wrote: > hi list, > what is the best way to check if a given list (lets call it l1) is > totally contained in a second list (l2)? > > for example: > l1 = [1,2], l2 = [1,2,3,4,5] -> l1 is contained in l2 > l1 = [1,2,2,], l2 = [1,2,3,4,5] -> l1 is not contained in l2 > l1 = [1,2,3], l2 = [1,3,5,7] -> l1 is not contained in l2 > > my problem is the second example, which makes it impossible to work with > sets insteads of lists. But something like set.issubset for lists would > be nice. > > greatz Johannes My best guess: from collections import Counter def contains(alist, sublist): alist = Counter(alist) for k in sublist: try: alist[k] -= 1 if alist[k] < 0: return False except KeyError: return False return True 'Counter' being better understood as 'Multiset'. If m = len(alist) and n = len(sublist), looks to be roughly O(m+n). Cheers - Chas -- http://mail.python.org/mailman/listinfo/python-list
Re: testing if a list contains a sublist
On Aug 15, 4:26 pm, Johannes wrote: > hi list, > what is the best way to check if a given list (lets call it l1) is > totally contained in a second list (l2)? > > for example: > l1 = [1,2], l2 = [1,2,3,4,5] -> l1 is contained in l2 > l1 = [1,2,2,], l2 = [1,2,3,4,5] -> l1 is not contained in l2 > l1 = [1,2,3], l2 = [1,3,5,7] -> l1 is not contained in l2 > > my problem is the second example, which makes it impossible to work with > sets insteads of lists. But something like set.issubset for lists would > be nice. > > greatz Johannes My best guess: from collections import Counter def contains(alist, sublist): alist = Counter(alist) for k in sublist: try: alist[k] -= 1 if alist[k] < 0: return False except KeyError: return False return True 'Counter' being better understood as 'Multiset'. If m = len(alist) and n = len(sublist), looks to be roughly O(m+n). Cheers - Chas -- http://mail.python.org/mailman/listinfo/python-list
Re: testing if a list contains a sublist
On Aug 15, 4:26 pm, Johannes wrote: > hi list, > what is the best way to check if a given list (lets call it l1) is > totally contained in a second list (l2)? > > for example: > l1 = [1,2], l2 = [1,2,3,4,5] -> l1 is contained in l2 > l1 = [1,2,2,], l2 = [1,2,3,4,5] -> l1 is not contained in l2 > l1 = [1,2,3], l2 = [1,3,5,7] -> l1 is not contained in l2 > > my problem is the second example, which makes it impossible to work with > sets insteads of lists. But something like set.issubset for lists would > be nice. > > greatz Johannes My best guess: from collections import Counter def contains(alist, sublist): alist = Counter(alist) for k in sublist: try: alist[k] -= 1 if alist[k] < 0: return False except KeyError: return False return True 'Counter' being better understood as 'Multiset'. If m = len(alist) and n = len(sublist), looks to be roughly O(m+n). Cheers - Chas -- http://mail.python.org/mailman/listinfo/python-list
Re: Why won't this decorator work?
On Jul 2, 11:08 am, John Salerno wrote:
> On Jul 2, 12:33 pm, MRAB wrote:
>
>
>
> > On 02/07/2011 17:56, John Salerno wrote:
>
> > > I thought I had finally grasped decorators, but the error I'm getting
> > > ('str' type is not callable) is confusing me.
> > > def move(roll):
> > > return 'You moved to space {0}.'.format(roll)
>
> > A decorator should return a callable.
Well, should typically return a callable, but there are exceptions...
>
> > This:
>
> > @move
> > def roll_die():
> > return random.randint(1, 6)
>
> > is equivalent to this:
>
> > def roll_die():
> > return random.randint(1, 6)
>
> > roll_die = move(roll_die)
>
> > You should be defining a function (a callable) and then passing it to a
> > decorator which returns a callable.
>
> > As it is, you're defining a function and then passing it to a decorator
> > which is returning a string. Strings aren't callable.
>
> But why does the documentation say "The return value of the decorator
> need not be callable"?
Well, because it need not be callable: i.e., if the return value is
not callable, that is perfectly legal. You very rarely want to return
something else, but an example of this is the @property decorator: it
returns a property object, which is not callable.
> And why, if I remove the decorator and just
> leave the two functions as if, does the call to move(roll_die()) work?
> Isn't that what the decorator syntax is essentially doing?
No; instead it's doing the similar looking but quite different
move(roll_die)()
As you wrote it, move(roll_die) returns the string 'You moved to space
.' which is not callable. You typically want
instead something like:
def move(roll):
# note that roll is a function, not a number
def wrapper():
result = roll()
print 'You moved to space {0}.'.format(result)
return result
return wrapper # which is a function
Now move returns a callable (the function 'wrapper') so move(roll_die)
is callable, and move(roll_die)() is legal.
Cheers - Chas
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why won't this decorator work?
On Jul 2, 11:08 am, John Salerno wrote:
> On Jul 2, 12:33 pm, MRAB wrote:
>
>
>
> > On 02/07/2011 17:56, John Salerno wrote:
>
> > > I thought I had finally grasped decorators, but the error I'm getting
> > > ('str' type is not callable) is confusing me.
> > > def move(roll):
> > > return 'You moved to space {0}.'.format(roll)
>
> > A decorator should return a callable.
Well, should typically return a callable, but there are exceptions...
>
> > This:
>
> > @move
> > def roll_die():
> > return random.randint(1, 6)
>
> > is equivalent to this:
>
> > def roll_die():
> > return random.randint(1, 6)
>
> > roll_die = move(roll_die)
>
> > You should be defining a function (a callable) and then passing it to a
> > decorator which returns a callable.
>
> > As it is, you're defining a function and then passing it to a decorator
> > which is returning a string. Strings aren't callable.
>
> But why does the documentation say "The return value of the decorator
> need not be callable"?
Well, because it need not be callable: i.e., if the return value is
not callable, that is perfectly legal. You very rarely want to return
something else, but an example of this is the @property decorator: it
returns a property object, which is not callable.
> And why, if I remove the decorator and just
> leave the two functions as if, does the call to move(roll_die()) work?
> Isn't that what the decorator syntax is essentially doing?
No; instead it's doing the similar looking but quite different
move(roll_die)()
As you wrote it, move(roll_die) returns the string 'You moved to space
.' which is not callable. You typically want
instead something like:
def move(roll):
# note that roll is a function, not a number
def wrapper():
result = roll()
print 'You moved to space {0}.'.format(result)
return result
return wrapper # which is a function
Now move returns a callable (the function 'wrapper') so move(roll_die)
is callable, and move(roll_die)() is legal.
Cheers - Chas
--
http://mail.python.org/mailman/listinfo/python-list
Re: Why won't this decorator work?
On Jul 2, 11:08 am, John Salerno wrote:
> On Jul 2, 12:33 pm, MRAB wrote:
>
>
>
> > On 02/07/2011 17:56, John Salerno wrote:
>
> > > I thought I had finally grasped decorators, but the error I'm getting
> > > ('str' type is not callable) is confusing me.
> > > def move(roll):
> > > return 'You moved to space {0}.'.format(roll)
>
> > A decorator should return a callable.
Well, should typically return a callable, but there are exceptions...
>
> > This:
>
> > @move
> > def roll_die():
> > return random.randint(1, 6)
>
> > is equivalent to this:
>
> > def roll_die():
> > return random.randint(1, 6)
>
> > roll_die = move(roll_die)
>
> > You should be defining a function (a callable) and then passing it to a
> > decorator which returns a callable.
>
> > As it is, you're defining a function and then passing it to a decorator
> > which is returning a string. Strings aren't callable.
>
> But why does the documentation say "The return value of the decorator
> need not be callable"?
Well, because it need not be callable: i.e., if the return value is
not callable, that is perfectly legal. You very rarely want to return
something else, but an example of this is the @property decorator: it
returns a property object, which is not callable.
> And why, if I remove the decorator and just
> leave the two functions as if, does the call to move(roll_die()) work?
> Isn't that what the decorator syntax is essentially doing?
No; instead it's doing the similar looking but quite different
move(roll_die)()
As you wrote it, move(roll_die) returns the string 'You moved to space
.' which is not callable. You typically want
instead something like:
def move(roll):
# note that roll is a function, not a number
def wrapper():
result = roll()
print 'You moved to space {0}.'.format(result)
return result
return wrapper # which is a function
Now move returns a callable (the function 'wrapper') so move(roll_die)
is callable, and move(roll_die)() is legal.
Cheers - Chas
--
http://mail.python.org/mailman/listinfo/python-list
Re: Multiple instances and wrong parental links
On Jan 1, 5:59 pm, Josh English wrote:
> I have hit yet another wall. I am dynamically creating a class and then
> creating instances of that class. The class relies on a second class to store
> a list of objects. (This is simplified from the the original by a factor of
> about 20. The real program is trying to create a Python object around an XML
> definition object.)
>
> Here's the code:
>
> ## OPTION ONE for class: ElementList
> ### Not really a list, but a wrapper that behaves like a list
> class ElementList(object):
> def __init__(self, parent, name):
> self._parent = parent
> self._name = name
>
> def MakeWrapper(checker, _addNameAsAttribute = False ):
>
> ## OPTION TWO for class: ElementList
> class Wrap(object):
>
> ## OPTION THREE for class: Elementlist
>
> def __init__(self, name):
> self._name = name
> setattr(Wrap, 'stuff', ElementList(self, 'test'))
>
> Wrap.__name__= checker.title()
>
> return Wrap
>
> if __name__ == '__main__':
>
> Dude = MakeWrapper('Dude')
> print Dude
> d1 = Dude('Josh')
> print d1, d1.stuff
>
> # creating the second instance changes the behavior of the subclass
> d2 = Dude('Ben')
> print d2, d2.stuff
> print d1.stuff._parent
> print d2.stuff._parent
>
> #creating a third instance changes the behavior of all the subclasses
> d3 = Dude('David')
> print d3, d3.stuff
> print d1.stuff._parent, d2.stuff._parent, d3.stuff._parent
>
> ## END CODE
>
> And here is the output:
>
>
>
>
> <__main__.Dude object at 0x00DFB930> <__main__.ElementList object at
> 0x00DFB950>
> <__main__.Dude object at 0x00DFB730> <__main__.ElementList object at
> 0x00DFB770>
> <__main__.Dude object at 0x00DFB730>
> <__main__.Dude object at 0x00DFB730>
> <__main__.Dude object at 0x00DFB870> <__main__.ElementList object at
> 0x00DFB9B0>
> <__main__.Dude object at 0x00DFB870> <__main__.Dude object at 0x00DFB870>
> <__main__.Dude object at 0x00DFB870>
>
> The 'stuff' attribute is an ElementList object linked back to the parent
> instance, but every time I create an instance, every instance's 'stuff' links
> back to the last instance created.
>
If every instance's is the same, one guesses that the
has a class scope, rather than an instance scope.
> I'm not sure why this is happening, or how to prevent it.
>
It's perfectly predictable; to understand what is happening, compare:
>>> def foo():
class Bar(object):
def __init__(self):
setattr(Bar, 'stuff', {})
return Bar
>>> Dude = foo()
>>> a = Dude()
>>> b = Dude()
>>> a.stuff['foo'] = 2
>>> b.stuff
{'foo': 2}
>>> c = Dude()
>>> a.stuff
{}
with the behavior (which I think you expected) of:
>>> def foo():
class Bar(object):
def __init__(self):
setattr(self, 'stuff', {})
return Bar
>>> Dude = foo()
>>> a = Dude()
>>> b = Dude()
>>> a.stuff['foo'] = 2
>>> b.stuff
{}
>>> c = Dude()
>>> a.stuff
{'foo': 2}
Cheers - Chas
--
http://mail.python.org/mailman/listinfo/python-list
Re: How can a function find the function that called it?
On Dec 24, 8:24 am, kj wrote:
> I want to implement a frozen and ordered dict.
>
> I thought I'd implement it as a subclass of collections.OrderedDict
> that prohibits all modifications to the dictionary after it has
> been initialized.
>
> In particular, calling this frozen subclass's update method should,
> in general, trigger an exception ("object is not mutable").
>
> But OrderedDict's functionality *requires* that its __init__ be
> run, and this __init__, in turn, does part of its initialization
> by calling the update method.
>
Rather than trying to identify the caller, I'd do something like:
class FrozenODict(OrderedDict):
def __init__(self, *args, **kwargs):
OrderedDict.__init__(self, *args, **kwargs)
self.update = self._update # init is complete, so override
# update method for this instance
def _update(self, dict2):
raise Exception("object is immutable!!")
After the __init__, calls to the instance's 'update' function will be
mapped to _update. It's essentially overriding the inherited function
on the fly.
Cheers - Chas
--
http://mail.python.org/mailman/listinfo/python-list
