Re: [Tutor] subtyping builtin type
On 01/02/2014 03:21 AM, Steven D'Aprano wrote: On Wed, Jan 01, 2014 at 02:49:17PM +0100, spir wrote: On 01/01/2014 01:26 AM, Steven D'Aprano wrote: On Tue, Dec 31, 2013 at 03:35:55PM +0100, spir wrote: [...] I take the opportunity to add a few features, but would do without Source altogether if it were not for 'i'. The reason is: it is for parsing library, or hand-made parsers. Every matching func, representing a pattern (or rule), advances in source whenever mathc is ok, right? Thus in addition to return the form (of what was matched), they must return the new match index: return (form, i) The usual way to do this is to make the matching index an attribute of the parser, not the text being parsed. In OOP code, you make the parser an object: class Parser: def __init__(self, source): self.current_position = 0 # Always start at the beginning self.source = source def parse(self): ... parser = Parser(some text to be parsed) for token in parser.parse(): handle(token) The index is not an attribute of the source text, because the source text doesn't care about the index. Only the parser cares about the index, so it should be the responsibility of the parser to manage. There is (no need for) a distinct Parser class or evne notion of parser. A parser is a top-level pattern (rule, object, or match func if one designs more like FP than OOP). Symmetrically, every pattern is a parser for what it matches. Think at branches in a tree: the tree is a top-level branch and every branch is a sub-tree. This conception is not semantically meaningful but highly useful, if not necessary, in practice: it permits using every pattern on its own ,for what it matches; it permits trying and testing patterns indicidually. (One could always find and implement workarounds, they would be needless complications.) However, I see and partially share what you say -- se below. Symmetrically, every match func using another (meaning nearly all) receive this pair. (Less annoyingly, every math func also takes i as input, in addition to the src str.) (There are also a handful of other annoying points, consequences of those ones.) The match functions are a property of the parser, not the source text. So they should be methods on a Parser object. Since they need to track the index (or indexes), the index ought to be an attribute on the parser, not the source text. This does not hold for me. Think eg at 2-phase parsing (like when using lex yacc): the lexer (lex) provides the parser (yacc) with a stream of lexemes completely opaquely for the parser, which does not know about indexes (neither in original source sting, nore in the stream of lexemes). Since I parse (usually) in a single phase, the source string is in the position lex above: it feeds the parser with a stream of ucodes, advancing in coherent manner; the parser does not need, nore want (lol!) to manage the source's index That's the point. The index is a given for the parser, that it just uses to try match at the correct position. If I have a string that stores its index, all of this mess is gone. What you are describing is covered by Martin Fowler's book Refactoring. He describes the problem: A field is, or will be, used by another class more than the class on which it is defined. and the solution is to move the field from that class to the class where it is actually used. (Refactoring - Ruby Edition, by Jay Fields, Shane Harvie and Martin Fowler.) Having a class (in your case, Source) carry around state which is only used by *other functions* is a code-smell. That means that Source is responsible for things it has no need of. That's poor design. I don't share this. Just like an open file currently beeing read conceptually (if not in practice) has a current index. It's also sane simple, and thread-safe, even if two Source objects happened to share (refs to) the same underlying actual source string (eg read form the same source file): each has its own current index. I don't see how Fowler's views apply to this case. Whether a Source or a Parser holds the index does not change attribute access or its wishable properties. By making the parser a class, instead of a bunch of functions, they can share state -- the *parser state*. That state includes: - the text being parsed; - the tokens that can be found; and - the position in the text. The caller can create as many parsers as they need: parse_this = Parser(some text) parse_that = Parser(different text) without them interfering, and then run the parsers independently of each other. The implementer, that is you, can change the algorithm used by the Parser without the caller needing to know. With your current design, you start with this: # caller is responsible for tracking the index source = Source(some text) assert source.i = 0 parse(source) Maybe we just don't have the same experience or practice of parsing, but
Re: [Tutor] Shelve immutable objects
Thanks for all this Eryksun (and Mark!), but... I don't understand why you brought gdbm in? Is it something underlying shelve, or a better approach, or something else? That last part really puts me in a pickle, and I don't understand why. Separately, I'm also curious about how to process big files. For example, I was trying to play 100 million games of chutes ladders, and I crashed my machine, I believe: the game results, including 4 ints 2 short lists of ints per game, are gathered into a list, so it can become a pretty big list. I need to do stats and other analyses on it in the end (okay, I really don't NEED to play 100 million games of chutes ladders, but as long as I have...): I suppose I could break it into manageable (maybe 1 million games each), but that will make some of the stats either clunky or wrong (I haven't really attacked that part yet). And since I'm not REALLY ready to ask this question, I'll tack it on to the end... I'm also beginning to think about how to speed it up: I'm imagining my two options are going to be to code some sections in a faster language (i.e. C), or maybe to introduce multi-threading since I'm working on a multicore machine generally (core I7), and I'm doing a lot of iterations of the same thing with no important order... seems like a good candidate. Now, I'm probably pretty far from that piece (in my learning process), but this is moving along pretty well so I'm open to suggestions about how to proceed. I've started switching up my code a fair bit to try to make it more OOP, though I'm still rough on that part. K ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] what's your name? (to a class)
Hello tutorians, Am I missing something or don't classes know how they're called (unlike funcs, which have a __name__ attribute, very practicle)? Is there a way to get it otherwise? The point is to have a super-type define a general __repr__ like eg: class SuperType: # ... def __repr__ (sef): return %s(stuff) % (self.__class__.__name__, stuff) But I need each class to know its name. Subtypes are actually defined by users (of a lib), presently they are forced to write the name explicitely, which is stupid since they already give it as var name: class SubType (SuperType): __name__ = SubType # Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] what's your name? (to a class)
Hi, Am I missing something or don't classes know how they're called (unlike funcs, which have a __name__ attribute, very practicle)? Is there a way to get it otherwise? The class has it, the instance doesn't. That said, you are looking for self.__class__.__name__ ... class SuperType: # ... def __repr__ (sef): return %s(stuff) % (self.__class__.__name__, stuff) ... which you do use here. But I need each class to know its name. Subtypes are actually defined by users (of a lib), presently they are forced to write the name explicitely, which is stupid since they already give it as var name: class SubType (SuperType): __name__ = SubType # I do not get it. The __class__.__name__ thing works great for me: class Foo(): ... def whoami(self): ... return self.__class__.__name__ ... f = Foo() f.whoami() 'Foo' class Bar(Foo): ... pass ... b = Bar() b.whoami() 'Bar' Please be a bit more specific about your problem, because the problem you described obviously does not exist ;). -nik -- * mirabilos is handling my post-1990 smartphone * mirabilos Aaah, it vibrates! Wherefore art thou, demonic device?? PGP-Fingerprint: 3C9D 54A4 7575 C026 FB17 FD26 B79A 3C16 A0C4 F296 signature.asc Description: Digital signature ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] what's your name? (to a class)
On Thu, Jan 02, 2014 at 11:12:30AM +0100, spir wrote: Hello tutorians, Am I missing something or don't classes know how they're called (unlike funcs, which have a __name__ attribute, very practicle)? Is there a way to get it otherwise? py type(42).__name__ 'int' py class Spam: ... pass ... py Spam.__name__ 'Spam' The point is to have a super-type define a general __repr__ like eg: class SuperType: # ... def __repr__ (sef): return %s(stuff) % (self.__class__.__name__, stuff) That works for me. Is there some reason you think it doesn't work? But I need each class to know its name. Subtypes are actually defined by users (of a lib), presently they are forced to write the name explicitely, which is stupid since they already give it as var name: class SubType (SuperType): __name__ = SubType Completely unnecessary. py class Ham(Spam): ... pass ... py Ham.__name__ 'Ham' I think maybe you've made an error somewhere and are misinterpreting what you are seeing. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Shelve immutable objects
On Thu, Jan 02, 2014 at 04:15:06AM -0500, Keith Winston wrote: Separately, I'm also curious about how to process big files. For example, I was trying to play 100 million games of chutes ladders, and I crashed my machine, I believe: the game results, including 4 ints 2 short lists of ints per game, are gathered into a list, so it can become a pretty big list. How to process big files, in order of best to worst (in my opinion): (1) Get more memory. (2) Find a way to process them in small chunks, e.g. a line at a time, rather than try to hold the entire file in memory at once. (3) Split them into small files, then process each file in turn. (4) Pass them to external tools which are optimized for dealing with huge files. (5) Find a way to process them using mmap. (6) Write your own tool for handling huge files. I need to do stats and other analyses on it in the end (okay, I really don't NEED to play 100 million games of chutes ladders, but as long as I have...): I suppose I could break it into manageable (maybe 1 million games each), but that will make some of the stats either clunky or wrong (I haven't really attacked that part yet). It shouldn't. Well, it might, but only if you're doing some pretty sophisticated statistical analysis. Most common statistics can be calculated on a single pass through the data. If you need to calculate (say) the mean, variance and standard deviation, it is moderately straight forward to calculate all three in a single pass without needing to have all the data in memory at once. (Feel free to ask for more detail if you wish.) Even statistics like median, which normally requires the entire data set to be in memory so it can be sorted, can be calculated with a couple of passes through the data. I think there is even a single pass algorithm for it. And since I'm not REALLY ready to ask this question, I'll tack it on to the end... I'm also beginning to think about how to speed it up: I'm imagining my two options are going to be to code some sections in a faster language (i.e. C), or maybe to introduce multi-threading since I'm working on a multicore machine generally (core I7), and I'm doing a lot of iterations of the same thing with no important order... seems like a good candidate. A game of Chutes and Ladders (or as we call it in Australia, Snakes and Ladders) surely doesn't need to be optimized. It will spend most of its time waiting for the human user, who is probably a hundred thousand or million times slower than the computer. But let's say you want to make it faster regardless. How to make it faster: (1) Use a smarter algorithm or less wasteful implementation. (2) Get more memory. (3) Get a faster computer. (4) I think what you are trying to do is a good candidate for PyPy, the optimizing Python JIT compiler. Especially if you are playing multiple millions of games. (5) Profile, profile, profile, profile. Only once you have profiled your code can you possibly hope to guess where the bottlenecks are. After 15 or 20 years of programming with Python, my guesses for the bottlenecks are still wrong more often than they are right. Assuming you identify the actual bottlenecks: (6) Try writing them in Cython, which will often give you good speedups. (7) Or write a C extension. Should you use threads? Probably not. - Unless the bottleneck in your code is disk or network I/O, threads will not help, they'll just make your program slower. - Unless you use IronPython or Jython instead, in which case treads *might* help. But probably not as much as you think. - If your bottleneck is CPU processing, you can use the multiprocessing module instead of threads. - You did profile your code first didn't you? Now, I'm probably pretty far from that piece (in my learning process), but this is moving along pretty well so I'm open to suggestions about how to proceed. I've started switching up my code a fair bit to try to make it more OOP, though I'm still rough on that part. Advice about optimization from some experts: http://en.wikipedia.org/wiki/Program_optimization#Quotes My favourite quote is: The First Rule of Program Optimization: Don't do it. The Second Rule of Program Optimization (for experts only!): Don't do it yet. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] what's your name? (to a class)
On Thu, Jan 2, 2014 at 5:12 AM, spir denis.s...@gmail.com wrote: Am I missing something or don't classes know how they're called (unlike funcs, which have a __name__ attribute, very practicle)? Is there a way to get it otherwise? What are you smoking, and where can I get some? ;) Actually, I think I understand the source of your confusion. `dir` doesn't show attributes from the metaclass because that would be too confusing. __class__ is a descriptor in the dict of `type`: name = vars(type)['__name__'] type(name) class 'getset_descriptor' class C: pass ... name.__get__(C) 'C' For a heap type, the getter can just return the Python object `ht_name`. For a built-in type it has to construct a Unicode string (assuming 3.x) from the `tp_name` slot. The setter (`type_set_name`) won't let you change the name of a built-in type. It also has to ensure that the name doesn't contain a null: name.__set__(C, 'GoodName') C.__name__ 'GoodName' name.__set__(int, 'float') Traceback (most recent call last): File stdin, line 1, in module TypeError: can't set int.__name__ name.__set__(C, 'Bad\x00Name') Traceback (most recent call last): File stdin, line 1, in module ValueError: __name__ must not contain null bytes name.__set__(C, 'Really Weird Name') C.__name__ 'Really Weird Name' ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] what's your name? (to a class)
On Thu, Jan 2, 2014 at 10:48 AM, eryksun eryk...@gmail.com wrote: __class__ is a descriptor in the dict of `type`: I think I must be smoking something, too. That should be __name__. Sorry. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Shelve immutable objects
On Thu, Jan 2, 2014 at 4:15 AM, Keith Winston keithw...@gmail.com wrote:
Thanks for all this Eryksun (and Mark!), but... I don't understand why you
brought gdbm in? Is it something underlying shelve, or a better approach, or
something else? That last part really puts me in a pickle, and I don't
understand why.
A Shelf is backed by a container with the following mapping methods:
keys
__contains__
__getitem__
__setitem__
__delitem__
__len__
Shelf will also try to call `close` and `sync` on the container if
available. For some reason no one has made Shelf into a context
manager (i.e. __enter__ and __exit__), so remember to close() it.
For demonstration purposes, you can use a dict with Shelf:
sh = shelve.Shelf(dict={})
sh['alist'] = [1,2,3]
The mapping is referenced in the (badly named) `dict` attribute:
sh.dict
{b'alist': b'\x80\x03]q\x00(K\x01K\x02K\x03e.'}
Keys are encoded as bytes (UTF-8 default) and the value is serialized
using pickle. This is to support using a database from the dbm module.
shelve.open returns an instance of shelve.DbfilenameShelf, which is a
subclass of Shelf specialized to open a dbm database.
Here's an overview of Unix dbm databases that Google turned up:
http://www.unixpapa.com/incnote/dbm.html
Note the size restrictions for keys and values in ndbm, which gdbm
doesn't have. Using gdbm lifts the restriction on the size of pickled
objects (the docs vaguely suggest to keep them fairly small).
Unfortunately gdbm isn't always available.
On my system, dbm defaults to creating a _gdbm.gdbm database, where
_gdbm is an extension module that wraps the GNU gdbm library (e.g.
libgdbm.so.3).
You can use a different database with Shelf (or a subclass), so long
as it has the required methods. For example, shelve.BsdDbShelf is
available for use with pybsddb (Debian package python3-bsddb3). It
exposes the bsddb3 database methods `first`, `next`, `previous`,
`last` and `set_location`.
Separately, I'm also curious about how to process big files.
...
I'm also beginning to think about how to speed it up:
I defer to Steven's sage advice.
Look into using databases such as sqlite3, numpy, and also add the
multiprocessing and concurrent.futures modules to your todo list. Even
if you know C/C++, I suggest you use Cython to create CPython
extension modules.
http://www.cython.org
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] what's your name? (to a class)
On 01/02/2014 11:18 AM, Dominik George wrote: Hi, Am I missing something or don't classes know how they're called (unlike funcs, which have a __name__ attribute, very practicle)? Is there a way to get it otherwise? The class has it, the instance doesn't. That said, you are looking for self.__class__.__name__ ... class SuperType: # ... def __repr__ (sef): return %s(stuff) % (self.__class__.__name__, stuff) ... which you do use here. Well, this was not real code, but an example of what I wished to be able to write! But I need each class to know its name. Subtypes are actually defined by users (of a lib), presently they are forced to write the name explicitely, which is stupid since they already give it as var name: class SubType (SuperType): __name__ = SubType # I do not get it. The __class__.__name__ thing works great for me: class Foo(): ... def whoami(self): ... return self.__class__.__name__ ... f = Foo() f.whoami() 'Foo' class Bar(Foo): ... pass ... b = Bar() b.whoami() 'Bar' Please be a bit more specific about your problem, because the problem you described obviously does not exist ;). I just missed it. Sorry! In fact, I did: class C: pass ... dir(C) ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__'] No __name__, or am I blind? But: C.__name__ 'C' Still, there is a __name__. ??? Compare with: def f(): pass ... dir(f) ['__annotations__', '__call__', '__class__', '__closure__', '__code__', '__defaults__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__get__', '__getattribute__', '__globals__', '__gt__', '__hash__', '__init__', '__kwdefaults__', '__le__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__'] Here I can see __name__ (thus, I'm probably not that blind yet ;-). Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] what's your name? (to a class)
On 01/02/2014 02:40 PM, Steven D'Aprano wrote: On Thu, Jan 02, 2014 at 11:12:30AM +0100, spir wrote: Hello tutorians, Am I missing something or don't classes know how they're called (unlike funcs, which have a __name__ attribute, very practicle)? Is there a way to get it otherwise? py type(42).__name__ 'int' py class Spam: ... pass ... py Spam.__name__ 'Spam' The point is to have a super-type define a general __repr__ like eg: class SuperType: # ... def __repr__ (sef): return %s(stuff) % (self.__class__.__name__, stuff) That works for me. Is there some reason you think it doesn't work? Sorry again, the reason is dir() does not show __name__ for classes, while it does for funcs. See answer to Dominik. I guess I've now found it: class C: pass ... dir(C) # note __dir__ below: ['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__'] dir(C.__dir__) # here is __name__ : ['__call__', '__class__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__get__', '__getattribute__', '__gt__', '__hash__', '__init__', '__le__', '__lt__', '__name__', '__ne__', '__new__', '__objclass__', '__qualname__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__'] Denis ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] what's your name? (to a class)
On 01/02/2014 04:48 PM, eryksun wrote: On Thu, Jan 2, 2014 at 5:12 AM, spir denis.s...@gmail.com wrote: Am I missing something or don't classes know how they're called (unlike funcs, which have a __name__ attribute, very practicle)? Is there a way to get it otherwise? What are you smoking, and where can I get some? ;) I hadn't this morminig ;-) Must be getting along with wonderful people these times, makes similar effects on mood, without side-effects (as fonctional programmers say ;-) ... Actually, I think I understand the source of your confusion. `dir` doesn't show attributes from the metaclass because that would be too confusing. That's it, exactly! (see other replies) ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Shelve immutable objects
Separately, I'm also curious about how to process big files. For example, I was trying to play 100 million games of chutes ladders, and I crashed my machine, I believe: the game results, including 4 ints 2 short lists of ints per game, are gathered into a list, so it can become a pretty big list. I need to do stats and other analyses on it in the end (okay, I really don't NEED to play 100 million games of chutes ladders, but as long as I have...): I suppose I could break it into manageable (maybe 1 million games each), but that will make some of the stats either clunky or wrong (I haven't really attacked that part yet). This is probably one of those cases where you don't want to persist this in active memory, but rather store it in some kind of long-term store. We can do a back of the envelope calculation to see why. An int's native size is 4 bytes on 32-bit machines. This is Python, so there's a bit more overhead. Let's roughly say that each record about 10 ints, so about 40 bytes. 100 * 10^6 records * 40 bytes per record = 4 * 10^9 bytes. ### import humanize humanize.intword(4*10**9) '4 billion' ### So that's about 4 billion bytes, as a very rough guess. Unfortunately, when we get to those magnitudes, that's way too large to fit into a standard computer's memory. 32-bit machines can only address up to about 4 billion bytes: humanize.intword(2**32) '4.3 billion' So trying to juggle all those records in short-term RAM is a non-starter: it won't work. We'll want to do something different instead, such as saving each record into a persistent on-disk, external database. If you're interested, bring this up as a separate topic on python-tutor, and I'm sure folks here will be happy to talk about it. Good luck! ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] what's your name? (to a class)
On Thu, Jan 2, 2014 at 11:21 AM, spir denis.s...@gmail.com wrote: dir(C.__dir__) # here is __name__ : It's good that you've brought up the special method __dir__, because that's at the heart of the issue. In 3.3, objects use object.__dir__ unless the type overrides it: vars(object)['__dir__'] method '__dir__' of 'object' objects vars(type)['__dir__'] method '__dir__' of 'type' objects Your class C is an instance of `type`. Here's the comment for `type_dir` in typeobject.c: __dir__ for type objects: returns __dict__ and __bases__. We deliberately don't suck up its __class__, as methods belonging to the metaclass would probably be more confusing than helpful. Also, there's a note about this in the docs: http://docs.python.org/3/library/functions.html#dir Because dir() is supplied primarily as a convenience for use at an interactive prompt, it tries to supply an interesting set of names more than it tries to supply a rigorously or consistently defined set of names, and its detailed behavior may change across releases. For example, metaclass attributes are not in the result list when the argument is a class. Prior to 3.3 (see issue 12166), there's no type.__dir__ or object.__dir__ built-in methods, so this behavior is baked in. http://bugs.python.org/issue12166 Back to __name__, for a built-in type the getter decodes the UTF-8 byte string in the `tp_name` slot. It's an example of why you should rarely depend on identity checks as opposed to equality. You might be lulled into complacency by how it works for heap types: name1 = C.__name__ name2 = C.__name__ name1 is name2 True But there's a built-in surprise: name1 = type.__name__ name2 = type.__name__ name1 is name2 False For non-class objects, the rules vary with the type. For a built-in method, the name is created on the fly from the `ml_name` field of its `PyMethodDef`. There's no setter, so it's readonly. For a function object, you can only set the name to a string and you can't delete it. But you can set it to a name with a null in it since it doesn't need compatibility with a C null-terminated string: f = lambda:0 f.__name__ = '\x00' f.__name__ '\x00' ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Shelve immutable objects
Separately, I'm also curious about how to process big files. For example, I was trying to play 100 million games of chutes ladders Without doing the 100,000,000, you could try either researching the nums, or trying an algorithm that tried intervals, and narrowed down the best , and numerically decisive amount of largest intervals of chute and ladders.. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] sqlite3 import problem
im having problems importing sqlite3 on ubuntu python3. File /usr/local/lib/python3.3/sqlite3/__init__.py, line 23, in module from sqlite3.dbapi2 import * File /usr/local/lib/python3.3/sqlite3/dbapi2.py, line 26, in module from _sqlite3 import * ImportError: No module named '_sqlite3' i have that path and file (dbapi2.py). the problem is an import statement for _sqlite3. i don't see this file but shouldn't python3.3 be able to import sqlite without errors? any suggestions? does anyone have _sqlite3.py in their /usr/local/lib/python3.3/sqlite3/ directory? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sqlite3 import problem
Hi Matthew, This might be an Ubuntu bug or deficiency. The file you're looking for is for the underlying low-level C module that bridges the world of SQLite and Python. By all rights, this would have been provided by something like the python-pysqlite2 package, but that package is for Python 2. I have not been able yet to find the equivalent binary package for Python 3, out of the list in: http://packages.debian.org/search?keywords=python3 and that seems unfortunate. Does anyone with Ubuntu experience know more information about this? From discussion on Stack Overflow, I see that people have been able to compile Python 3 from scratch and things work: http://stackoverflow.com/questions/8628774/python-3-2-cant-import-sqlite3-module http://superuser.com/questions/557152/sqlite-not-available-with-python-3-3-0-on-debian Frankly, that seems somewhat overkill for the situation, but if you really have to, that's one workaround. It certainly sounds a bit funky: is there something that stops the package developer from having some 'python3-pysqlite2' package? I'm very confused. I'd recommend asking on an Ubuntu-related forum to confirm that this is an Ubuntu or Debian problem, and then hopefully they'll be able to point you in the right direction or get the gears moving to fix this. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] What's in a name?
I've got the beginner's version of a question I think Denis asked recently... If I'm iterating a variable through a series of list names, for future processing, I would like to print the name of the list the variable is set to in a given moment... i.e. for i in [alist, blist, clist] i[3] = okey dokey print(i[3], i.name) # this is definitely not the way to do it... output: okey dokey alist okey dokey blist okey dokey clist Is there any way to do that? I'm thinking there may not be, since the variable is actually bound to the content of the list, not the other name... which would be good, in the sense that I'm beginning to understand, but bad in the sense that I need to rethink a piece (or two) of code. -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What's in a name?
Hmm, maybe I stumbled upon at least one approach, turning the problem around. Make it something like: for i in [alist, blist, clist] i[3] = okey dokey print(eval(i)[3], i) Of course I've been staring at this for a while, but as soon as I post I find a way... this is my first use of eval(), so it took me a while to stumble upon it. If there are more elegant solutions, I'm all ears. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What's in a name?
On 03/01/2014 05:22, Keith Winston wrote: I've got the beginner's version of a question I think Denis asked recently... If I'm iterating a variable through a series of list names, for future processing, I would like to print the name of the list the variable is set to in a given moment... i.e. for i in [alist, blist, clist] i[3] = okey dokey print(i[3], i.name http://i.name) # this is definitely not the way to do it... output: okey dokey alist okey dokey blist okey dokey clist Is there any way to do that? I'm thinking there may not be, since the variable is actually bound to the content of the list, not the other name... which would be good, in the sense that I'm beginning to understand, but bad in the sense that I need to rethink a piece (or two) of code. -- Keith I don't understand what you're asking for. It doesn't help that the code you've posted isn't even valid. If you add the missing colon at the end of the for loop, you've still got a problem in that standard lists don't have a name attribute. -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What's in a name?
On 03/01/2014 05:29, Keith Winston wrote: Hmm, maybe I stumbled upon at least one approach, turning the problem around. Make it something like: for i in [alist, blist, clist] i[3] = okey dokey print(eval(i)[3], i) Of course I've been staring at this for a while, but as soon as I post I find a way... this is my first use of eval(), so it took me a while to stumble upon it. If there are more elegant solutions, I'm all ears. Your code is still invalid. I still don't understand what you're asking for. I've never used eval in over 10 years of using Python, here's why http://nedbatchelder.com/blog/201206/eval_really_is_dangerous.html -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What's in a name?
On Thu, Jan 2, 2014 at 9:22 PM, Keith Winston keithw...@gmail.com wrote:
I've got the beginner's version of a question I think Denis asked
recently...
If I'm iterating a variable through a series of list names, for future
processing, I would like to print the name of the list the variable is set
to in a given moment... i.e.
for i in [alist, blist, clist]
i[3] = okey dokey
print(i[3], i.name) # this is definitely not the way to do it...
One of the things you can do is make the following idea more concrete:
You want some _name_ associated with a _value_.
In that case, you want the name 'alist' to be closely associated with
the list that goes by that name.
You can do this kind of thing without being specific to any
particularities about Python. Directly: put that information in a
data structure, such as a dictionary. Put the name _in_ the value.
For example:
##
class Person(object):
def __init__(self, name):
self.name = name
def sayHi(self):
print(this is + self.name + ; hello!)
people = {'frank' : Person('frank'),
'sally' : Person('sally'),
'timothy' : Person('timothy')}
for nick, p in people.items():
print(repr(nick) + says: )
p.sayHi()
##
There is a reason why you might want to save the name as part of the
value, rather than to try to get that from the name of the variable.
Here's one reason: sometimes people go by different names.
##
tim = people['timothy']
tiny_tim = tim
tym = tim
# ... see http://en.wikipedia.org/wiki/Timothy_(name)
##
in which case, if we try to print out this person later on, which name
would you like to see? Rather than guess, we might as well use the
name that timothy identifies himself with.
Good luck!
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What's in a name?
Sorry, I forgot to add. You titled the subject of the thread as: What's in a name? A good answer to your question is to permute the words a little. Name is in a... ... where ... is what you want to put the name in. :P ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sqlite3 import problem
Matthew Ngaha chigga...@gmail.com writes: im having problems importing sqlite3 on ubuntu python3. File /usr/local/lib/python3.3/sqlite3/__init__.py, line 23, in module from sqlite3.dbapi2 import * File /usr/local/lib/python3.3/sqlite3/dbapi2.py, line 26, in module from _sqlite3 import * ImportError: No module named '_sqlite3' i have that path and file (dbapi2.py). the problem is an import statement for _sqlite3. i don't see this file but shouldn't python3.3 be able to import sqlite without errors? any suggestions? does anyone have _sqlite3.py in their /usr/local/lib/python3.3/sqlite3/ directory? The /usr/local heirarchy is not used by official debian/ubuntu packages, so other people will not have the same contents in there as you. You must have a locally built python in /usr/local. Check the build logs if it has complained about not being able to build the sqlite3 module. If so, you can install all build dependencies for the offical python3 package by running this command: # apt-get build-dep python3 And then rebuilding your local python3 installation. -- regards, kushal pgpqnOrb9Kbe6.pgp Description: PGP signature ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Fwd: What's in a name?
Shoot: I sent this response directly to Mark, without even trimming. Here
it is to the list...
Hi Mark: sorry for unclarity. I am probably going to make a hash of
explaining this, but here goes:
I want to iterate a variable across a list of objects, and print both the
outputs (wrong word) of said objects, and the name of the objects. Those
objects might be lists, or functions, as examples.
As a non-iterative example, something like this:
a = max
print(eval(a)(3,4), a) # output: 4 max
That's the only way I can figure out how to make it work. Here's an actual
code snippet, watch for stype:
for func in [mean, max, min, variance, stdev]:
print({moves:9.2f} {chutes:12.2f} {ladders:13.2f}
{stype}.format(
moves=eval(func)(tgset[1] for tgset in garray),
chutes=eval(func)(tgset[2] for tgset in garray),
ladders=eval(func)(tgset[3] for tgset in garray),
stype=func
))
Output:
4.67 0.21 0.79 mean
28.00 1.00 1.00 max
1.00 0.00 0.00 min
23.69 0.17 0.17 variance
4.87 0.41 0.41 stdev
I appreciate the point about eval being dangerous, though the second line
in your reference does say if you accept strings to evaluate from
untrusted input. Still, I can appreciate how eval() could go off the
rails. Is there another way I can do what I want? Sorry for not testing the
code I posted earlier.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What's in a name?
Danny: I appreciate your point, but these are just for little code loops,
nothing I need to hold on to, like the snippet above: I'm just trying to
wrap a few things into one loop, which gives me flexibility about
expanding/contracting the stats, for example, that I print (by changing the
range list for func). Here's another code example, from my recent
masterpiece:
def print_candl_info(garray):
game_array, chute_nums, ladder_nums = {}, chutes, ladders
list_index = 3
for i in chute_nums.keys(): chute_nums[i] = 0 # zero all values
for i in ladder_nums.keys(): ladder_nums[i] = 0
for clkeys in [chute_nums, ladder_nums]: # increment candl values
list_index += 1 # horrible kluge: must be 4 for chutes, 5 for
ladders below
for corl in eval(clkeys):
for game in garray:
if corl in game[list_index]:
eval(clkeys)[corl] += 1
print(total , clkeys, = , sum(list(eval(clkeys).values(
For the purposes of answering this question, ignore the horrible kluge.
Though if you have any suggestions on that point, I'm still all ears. Since
I couldn't figure out the right way to do that, I left it as more or less
the most unforgivable kluge I could find, so I won't be able to sleep until
I improve on it. Suggestions are welcome. If it's not obvious, it relies on
the fact that the clkeys iterates precisely twice. It might be that the
answer lies in rearranging my data structure... I'm about to post the
entire program again for any comments, once I clean it up just a BIT more,
so if it's not obvious don't worry about it.
game = [int, int, int, int, [], []] # game[4] = chutes, game[5] = ladders
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: What's in a name?
On 03/01/2014 06:18, Keith Winston wrote:
Shoot: I sent this response directly to Mark, without even trimming.
Here it is to the list...
Hi Mark: sorry for unclarity. I am probably going to make a hash of
explaining this, but here goes:
I want to iterate a variable across a list of objects, and print both
the outputs (wrong word) of said objects, and the name of the objects.
Those objects might be lists, or functions, as examples.
As a non-iterative example, something like this:
a = max
print(eval(a)(3,4), a) # output: 4 max
That's the only way I can figure out how to make it work. Here's an
actual code snippet, watch for stype:
for func in [mean, max, min, variance, stdev]:
print({moves:9.2f} {chutes:12.2f} {ladders:13.2f}
{stype}.format(
moves=eval(func)(tgset[1] for tgset in garray),
chutes=eval(func)(tgset[2] for tgset in garray),
ladders=eval(func)(tgset[3] for tgset in garray),
stype=func
))
You enjoy making life difficult for yourself :) You've assigned strings
to the name func, just assign the functions themselves? Like.
for func in max, min:
print(func.__name__, func(range(5)))
Output.
max 4
min 0
Output:
4.67 0.21 0.79 mean
28.00 1.00 1.00 max
1.00 0.00 0.00 min
23.69 0.17 0.17 variance
4.87 0.41 0.41 stdev
I appreciate the point about eval being dangerous, though the second
line in your reference does say if you accept strings to evaluate from
untrusted input. Still, I can appreciate how eval() could go off the
rails. Is there another way I can do what I want? Sorry for not testing
the code I posted earlier.
--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.
Mark Lawrence
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: What's in a name?
Mark wrote: You enjoy making life difficult for yourself :) You've assigned strings to the name func, just assign the functions themselves? Like. for func in max, min: print(func.__name__, func(range(5))) Output. max 4 min 0 I wouldn't say I enjoy making life difficult for myself, but it is one of my strengths ;) That would work, I think, for the function example I gave you, but the example I gave in response to Danny used the same trick on lists: that is, I want to iterate through a bunch of lists, subsequently printing both list members (via indexing, for example) and the name of the list I'm on. I might even want to do the same of a dict, or some random data structure. Unless it's just really a bad idea to code like this. But certainly when the .__name__ attribute is available, that makes more sense. I'll change that part. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: What's in a name?
On 03/01/2014 06:55, Keith Winston wrote: Mark wrote: You enjoy making life difficult for yourself :) You've assigned strings to the name func, just assign the functions themselves? Like. for func in max, min: print(func.__name__, func(range(5))) Output. max 4 min 0 I wouldn't say I enjoy making life difficult for myself, but it is one of my strengths ;) That would work, I think, for the function example I gave you, but the example I gave in response to Danny used the same trick on lists: that is, I want to iterate through a bunch of lists, subsequently printing both list members (via indexing, for example) and the name of the list I'm on. I might even want to do the same of a dict, or some random data structure. Unless it's just really a bad idea to code like this. But certainly when the .__name__ attribute is available, that makes more sense. I'll change that part. lista = list(range(5)) listb = list(reversed(range(5))) for alist in lista, listb: print(alist.__class__.__name__, alist) list [0, 1, 2, 3, 4] list [4, 3, 2, 1, 0] -- My fellow Pythonistas, ask not what our language can do for you, ask what you can do for our language. Mark Lawrence ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Fwd: What's in a name?
I spoke about iterating through a bunch of lists in my last post, but in fact I'm iterating through a bunch of dicts in the example I gave. Sorry. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
[Tutor] Fwd: Fwd: What's in a name?
OMG, another one to Mark and not the list. I'll see if there's something I can adjust in email... On Fri, Jan 3, 2014 at 2:23 AM, Mark Lawrence breamore...@yahoo.co.ukwrote: lista = list(range(5)) listb = list(reversed(range(5))) for alist in lista, listb: print(alist.__class__.__name__, alist) list [0, 1, 2, 3, 4] list [4, 3, 2, 1, 0] Thank you Mark for your unreasonable patience. But the output I'd like to see from you example would be: lista [0, 1, 2, 3, 4] listb [4, 3, 2, 1, 0] Which I think is not possible since the list names are lost by the time the print statement is executing. Unless, of course, I'm wrong. I need the instance name, I guess, not the object name, of an object that includes no __name__ method (I'm stretching on this description) -- Keith ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] What's in a name?
On Thu, Jan 2, 2014 at 10:28 PM, Keith Winston keithw...@gmail.com wrote:
Danny: I appreciate your point, but these are just for little code loops,
nothing I need to hold on to, like the snippet above:
I hope you don't take offense. But I actually do not understand
print_candl_info(). I thought I did! But then I realized I was
deluding myself. I don't not understand it after all, and that's
after staring at it for more than about thirty minutes.
Take that a canary in the coalmine kind of comment. eval() can
encourages habits that hurt program readability in a deep way.
[Note: the rest of this message is a relentless refactoring of the
original code, step by step.]
If you don't mind, I'd like to iteratively strip away the eval() and
see if we can't make it a little easier to understand. From a first
treatment, I think this might look something like the following:
def print_candl_info(garray):
game_array, chute_nums, ladder_nums = {}, chutes, ladders
for i in chute_nums.keys(): chute_nums[i] = 0
for i in ladder_nums.keys(): ladder_nums[i] = 0
for corl in chute_nums:
for game in garray:
if corl in game[4]:
chute_nums[corl] += 1
print(total , chute_nums, = , sum(chute_nums.values()))
for corl in ladder_nums:
for game in garray:
if corl in game[5]:
ladder_nums[corl] += 1
print(total , ladder_nums, = , sum(ladder_nums.values()))
where we first unroll the original's outer loop out so that there are
no eval()s anywhere. This hurts program length a little, but we can
deal with that. Do you agree so far that the program above preserves
the meaning of what you had before?
If we can assume that the rewritten code means the same thing, then
we can eliminate the duplication by doing something like this:
def print_candl_info(garray):
game_array, chute_nums, ladder_nums = {}, chutes, ladders
for i in chute_nums.keys(): chute_nums[i] = 0
for i in ladder_nums.keys(): ladder_nums[i] = 0
summarize_game(chutes, chute_nums, 4, garray)
summarize_game(ladders, ladder_nums, 5, garray)
def summarize_game(game_name, game_nums, game_index, garray):
for corl in game_nums:
for game in garray:
if corl in game[game_index]:
game_nums[corl] += 1
print(total , game_name, = , sum(game_nums.values()))
where we put the the duplicated part in a function, and then call that
function twice, once for chutes, and the other for ladders.
Hmm... it's a little easier to see from this point that the zero all
values part also appears to be duplication.
Maybe that can be absorbed into the front of summarize_game? Let's
iterate again. Here's what it looks like afterwards:
def print_candl_info(garray):
game_array, chute_nums, ladder_nums = {}, chutes, ladders
summarize_game(chutes, chute_nums, 4, garray)
summarize_game(ladders, ladder_nums, 5, garray)
def summarize_game(game_name, game_nums, game_index, garray):
for i in game_nums.keys(): game_nums[i] = 0
for corl in game_nums:
for game in garray:
if corl in game[game_index]:
game_nums[corl] += 1
print(total , game_name, = , sum(game_nums.values()))
Reading...
Hmmm. I don't understand why chute_nums is assigned to chutes, nor
ladder_nums to ladders, and I don't see game_array being used here
yet. We can absorb that:
def print_candl_info(garray):
summarize_game(chutes, chutes, 4, garray)
summarize_game(ladders, ladders, 5, garray)
def summarize_game(game_name, game_nums, game_index, garray):
for i in game_nums.keys(): game_nums[i] = 0
for corl in game_nums:
for game in garray:
if corl in game[game_index]:
game_nums[corl] += 1
print(total , game_name, = , sum(game_nums.values()))
Hmmm... I think that's about as far as I can go with zero domain knowledge. :P
At this point, the code makes a little more sense to me, though the
data structure still feels like a black box. That is, I don't know
the shape of the data for 'garray' or 'corl'. If we knew more about
the structure of the data, maybe this could be further improved.
As a side note: the code above has more latitude than the original
because it can print out a friendlier game name. That is, for
example, you can do this:
##
summarize_game( Chutes! , chutes, 4, garray)
summarize_game( Ladders! , ladders, 5,
Re: [Tutor] Fwd: What's in a name?
And to beat that poor horse in the same example, my current way of doing that would be: for alist in lista, listb: print(alist, eval(alist)) ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
