Re: Data cleaning workouts
Thanks. I will try the SciPy list. It was a bit of a hail mary anyway. Pretty sure elevated Python types don't actually get their hands dirty with data. ;) - Original Message - From: rusi To: python-list@python.org Cc: Sent: Thursday, August 23, 2012 11:01 PM Subject: Re: Data cleaning workouts On Aug 23, 12:52 pm, Fg Nu wrote: > List folk, > > I am a newbie trying to get used to Python. I was wondering if anyone knows > of web resources that teach good practices in data cleaning and management > for statistics/analytics/machine learning, particularly using Python. > > Ideally, these would be exercises of the form: here is some horrible raw data > --> here is what it should look like after it has been cleaned. Guidelines > about steps that should always be taken, practices that should be avoided; > basically, workflow of data analysis in Python with special emphasis on the > cleaning part. Since no one has answered, I suggest you narrow your searching from 'python' to 'scipy' (or 'numpy'). Also perhaps ipython. And then perhaps try those specific mailing lists/fora. Since I dont know this area much, not saying more. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: Data cleaning workouts
On Aug 23, 12:52 pm, Fg Nu wrote: > List folk, > > I am a newbie trying to get used to Python. I was wondering if anyone knows > of web resources that teach good practices in data cleaning and management > for statistics/analytics/machine learning, particularly using Python. > > Ideally, these would be exercises of the form: here is some horrible raw data > --> here is what it should look like after it has been cleaned. Guidelines > about steps that should always be taken, practices that should be avoided; > basically, workflow of data analysis in Python with special emphasis on the > cleaning part. Since no one has answered, I suggest you narrow your searching from 'python' to 'scipy' (or 'numpy'). Also perhaps ipython. And then perhaps try those specific mailing lists/fora. Since I dont know this area much, not saying more. -- http://mail.python.org/mailman/listinfo/python-list
Does a wxPython program not run on 64bit Windows?
Does a wxPython program not run on 64bit Windows? I saw this “ wxPython is a cross-platform toolkit. This means that the same program will run on multiple platforms without modification. Currently supported platforms are 32-bit Microsoft Windows, most Unix or unix-like systems, and Macintosh OS X. ” -- http://mail.python.org/mailman/listinfo/python-list
Re: Built-in open() with buffering > 1
On 08/24/2012 06:35 AM, Marco wrote: Please, can anyone explain me the meaning of the "buffering > 1" in the built-in open()? The doc says: "...and an integer > 1 to indicate the size of a fixed-size chunk buffer." Sorry, I get it: >>> f = open('myfile', 'w', buffering=2) >>> f._CHUNK_SIZE = 5 >>> for i in range(6): ... n = f.write(str(i)) ... print(i, open('myfile').read(), sep=':') ... 0: 1: 2: 3: 4: 5:012345 -- http://mail.python.org/mailman/listinfo/python-list
Built-in open() with buffering > 1
Please, can anyone explain me the meaning of the "buffering > 1" in the built-in open()? The doc says: "...and an integer > 1 to indicate the size of a fixed-size chunk buffer." So I thought this size was the number of bytes or chars, but it is not: >>> f = open('myfile', 'w', buffering=2) >>> f.write('a') 1 >>> open('myfile').read() '' >>> f.write('b') 1 >>> open('myfile').read() '' >>> f.write('cdefghi\n') 8 >>> open('myfile').read() '' >>> f.flush() >>> open('myfile').read() 'abcdefghi\n' Regards, Marco -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On Aug 24, 11:34 am, Chris Angelico wrote: > On Fri, Aug 24, 2012 at 10:36 AM, Roy Smith wrote: > > Even id() thinks they're the same thing: > id(a) > > 1755402140 > id(globals()["a"]) > > 1755402140 > > Ah, no. What you have there is actually id(4) and nothing to do with a at all. Well, nothing expect for the fact that he's demonstrating Python references and how they bind to objects. Sure, id() isn't doing any lookup, but that's missing the point. > > But, notice what happens if I now assign something new to a: > > a = 123 > id(a) > > 1755403176 > > > The id has changed! Now, we all know that the id of an object is its > > memory address (that's not guaranteed, but in the standard C > > implementation of Python, that's what it is). > > And you now have id(123) - of course, it's possible for there to be > two integer objects with the value 123, but what I'm emphasizing is > that you're not looking at a here. But Roy's point was that referring to 'a' as a 'variable' makes no sense, as it's not an allocated piece of memory. In fact, he even said "the id of an object" and not "the id of 'a'" so I'm not entirely sure what you're objecting to here. You don't need to emphasise anything as _that was the point_: you're _not_ looking at 'a' _ever_, you're only ever looking at the object to which it refers. -- http://mail.python.org/mailman/listinfo/python-list
Re: Books?
I agree with Madison, think Python is a good choice if you have no programming experience. It'll teach you Python while also covering programming fundamentals. If you have prior programming experience, the Python docs/tutorials should be enough to get you started. On Thu, Aug 23, 2012 at 5:56 PM, Madison May < worldpeaceagentforcha...@gmail.com> wrote: > I would definitely take a look at Think Python by Allen Downey if you have > minimal prior experience with computer programming. It's succinct yet > thorough. > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On Fri, Aug 24, 2012 at 10:36 AM, Roy Smith wrote: > In fact, I can even write it that way and everything works: > globals()["a"] = 42 a > 42 > > Even id() thinks they're the same thing: > id(a) > 1755402140 id(globals()["a"]) > 1755402140 Ah, no. What you have there is actually id(4) and nothing to do with a at all. > But, notice what happens if I now assign something new to a: > a = 123 id(a) > 1755403176 > > The id has changed! Now, we all know that the id of an object is its > memory address (that's not guaranteed, but in the standard C > implementation of Python, that's what it is). And you now have id(123) - of course, it's possible for there to be two integer objects with the value 123, but what I'm emphasizing is that you're not looking at a here. > Now, what if I do something similar in C: > > #include > > main() { > int a = 40; > printf("a = %d, &a = %p\n", a, &a); > a = 99; > printf("a = %d, &a = %p\n", a, &a); > } > > When I compile and run this, it prints: > > a = 40, &a = 0x7fff1911f5bc > a = 99, &a = 0x7fff1911f5bc > > Notice that the address of the variable "a" didn't change when I > assigned it a new value. That's what people mean when they say C has > variables and Python doesn't; it just binds names to values. Try this instead. It's C++ not C but a much closer match. You could instead play with malloc if you want it to be C. #include main() { int *a=new int(40); printf("a = %d, id(a) = %p\n",*a,a); a=new int(99); printf("a = %d, id(a) = %p\n",*a,a); } I've not tested the code and may have a syntax issue with "new int(40)" (who ever allocates a single int on the heap??) but you get the idea. At no point do you ever look at, or need to look at, &a. That's utterly irrelevant. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
In article , Evan Driscoll wrote: > > In fact, Python doesn't have variables not as C or Java programmers > > would understand the term. What it has instead are references to objects > > (with names as one kind of reference). > > OK, I've seen this said a few times, and I have to ask: what do you mean > by this? I consider myself pretty decent at Python and other languages, > and I really don't get it. I'll take a shot at this. When I execute: a = 4 I'm doing two things. The first is to create an object of type int with a value of 4. I think everybody is OK with that part. The confusing part comes with the LHS. In C, or Java, there's a container called "a" which holds a value. In C, that value is the integer 4, in Java it's an Integer object (well, at least I think it is, I've never fully groked how Java handles integers). In Python, there is no container named "a". There is, however, a dict which exists somewhere in python-space. You can get a reference to this dict by calling globals(). What the assignment does is effectively: globals()["a"] = 4 In fact, I can even write it that way and everything works: >>> globals()["a"] = 42 >>> a 42 Even id() thinks they're the same thing: >>> id(a) 1755402140 >>> id(globals()["a"]) 1755402140 But, notice what happens if I now assign something new to a: >>> a = 123 >>> id(a) 1755403176 The id has changed! Now, we all know that the id of an object is its memory address (that's not guaranteed, but in the standard C implementation of Python, that's what it is). Now, what if I do something similar in C: #include main() { int a = 40; printf("a = %d, &a = %p\n", a, &a); a = 99; printf("a = %d, &a = %p\n", a, &a); } When I compile and run this, it prints: a = 40, &a = 0x7fff1911f5bc a = 99, &a = 0x7fff1911f5bc Notice that the address of the variable "a" didn't change when I assigned it a new value. That's what people mean when they say C has variables and Python doesn't; it just binds names to values. -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On Fri, Aug 24, 2012 at 9:54 AM, Dennis Lee Bieber wrote: > On Fri, 24 Aug 2012 08:00:59 +1000, Chris Angelico > declaimed the following in gmane.comp.python.general: > >> >> But again, that probably doesn't help explain the variables. Unless >> you've come from (or can at least imagine) an environment in which you >> use pointers for *everything*. >> > ... but can not manipulate the pointer directly Right, obviously pointer arithmetic doesn't make sense in Python. But that's (almost) guaranteed by the fact that there is NOTHING you can do with bare integers. foo q = new q; /* note that 'foo' is a typedef that hides the asterisk */ foo w = q +1; /* Pointer arith! Impossible. */ But! foo e = q + one; /* one is the object representing the integer 1 */ This isn't pointer arithmetic. No C compiler will let you add two pointers. You can subtract one from another, but you get back a bare integer, which won't go into a 'foo', so the only thing you could do that would break stuff is: foo r = q + (w - e); /* Syntactically valid */ So you just won't do pointer arith if everything's a pointer. There, I think I just broke a few minds. My task here is done. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
Steven D'Aprano writes: > No offence to Ben Finney, but I think sometimes he's a bit too eager > to emphasise the subtle differences between Python and other > languages, rather than the similarities. No offense taken. > Again, context is important: Indeed. Note that my assertion was in response to someone *already* confused by pre-conceived notions about what “variable” means, and misguided attempts to apply those to Python. If it's over-eager to attempt to head off such confusion in others who might be reading, then I deny the charge. Others have said it helps, so I'll keep doing it. > I don't think the differences are important enough to *prohibit* use of > the word "variable" to describe name bindings. Only to discourage it. I wholly agree. -- \ “Facts do not cease to exist because they are ignored.” —Aldous | `\Huxley | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Variables vs names [was: Objects in Python]
On Fri, Aug 24, 2012 at 5:22 AM, Evan Driscoll wrote: > In Python--, any time you use a name, you have to prefix it with the > word 'variable': > variable x = 4 > print(variable x) That gets really unwieldy. You should shorten it to a single symbol. And your language could be called Python Hypertext Preprocessor. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On Fri, Aug 24, 2012 at 3:56 AM, Steven D'Aprano wrote: > But name bindings are a kind of variable. Named memory locations are a > *different* kind of variable. The behaviour of C variables and Python > variables do not follow the Liskov Substitution Principle -- you can't > rip out the entire "names bound to objects" machinery of Python and > replace it with C-like named memory locations without changing the high- > level behaviour of Python. And so by some ways of thinking it is *wrong* > to treat name bindings and memory locations as "the same sort of entity". > Hence, if C variables are variables, then Python name bindings can't be. Yet Python's variables are extremely close to C's pointer variables. If you allocate all your "real data" on the heap and do everything with pointers, you'll have semantics very similar to Python's (except that things aren't refcounted, so you have massive memory leaks). In fact, that's how the Python-C API handles things - a PyObject* basically _is_ a Python object, as far as C's concerned. But again, that probably doesn't help explain the variables. Unless you've come from (or can at least imagine) an environment in which you use pointers for *everything*. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On Fri, Aug 24, 2012 at 4:49 AM, Dennis Lee Bieber wrote: > By the time you wrap the equation with a lambda all, named, terms, > AND supply the named terms after the lambda, you might as well just wrap > the equation in a plain try/except block. But closures spare you that hassle. >>> a=1 >>> b=0 >>> print(safe(lambda: a/b)) 42 >>> b=2 >>> print(safe(lambda: a/b)) 0.5 (Example done in Python 3 so semantics are a little different) ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On 8/23/12 20:17 , Ian Kelly wrote: ... Well, there you go. There *is* something wrong with having six variables called 'q'. Sometimes you don't want only six variables called 'q' but a hundred of them :-) def fac(q): if q < 1 : return 1 else: return q * fac(q-1) print(fac(100)) That's only one variable called 'q', instantiated 100 times simultaneously. Bare with me, i come from a C world, and think of each variable, whatever its name or scope, as a piece of memory and therefore different. btw. I like the idea of simultaneously instantiation :-) Jan Kuiken -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 23/08/2012 19:33, wxjmfa...@gmail.com wrote: Le jeudi 23 août 2012 15:57:50 UTC+2, Neil Hodgson a écrit : wxjmfa...@gmail.com: Small illustration. Take an a4 page containing 50 lines of 80 ascii characters, add a single 'EM DASH' or an 'BULLET' (code points> 0x2000), and you will see all the optimization efforts destroyed. sys.getsizeof('a' * 80 * 50) 4025 sys.getsizeof('a' * 80 * 50 + '•') 8040 This example is still benefiting from shrinking the number of bytes in half over using 32 bits per character as was the case with Python 3.2: >>> sys.getsizeof('a' * 80 * 50) 16032 >>> sys.getsizeof('a' * 80 * 50 + '•') 16036 Correct, but how many times does it happen? Practically never. In this unicode stuff, I'm fascinated by the obsession to solve a problem which is, due to the nature of Unicode, unsolvable. For every optimization algorithm, for every code point range you can optimize, it is always possible to find a case breaking that optimization. This follows quasi the mathematical logic. To proof a law is valid, you have to proof all the cases are valid. To proof a law is invalid, just find one case showing it. Sure, it is possible to optimize the unicode usage by not using French characters, punctuation, mathematical symbols, currency symbols, CJK characters... (select undesired characters here: http://www.unicode.org/charts/). In that case, why using unicode? (A problematic not specific to Python) jmf What do you propose should be used instead, as you appear to be the resident expert in the field? -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On Thu, Aug 23, 2012 at 12:33 PM, wrote: >> >>> sys.getsizeof('a' * 80 * 50) >> >> > 4025 >> >> sys.getsizeof('a' * 80 * 50 + '•') >> >> > 8040 >> >> >> >> This example is still benefiting from shrinking the number of bytes >> >> in half over using 32 bits per character as was the case with Python 3.2: >> >> >> >> >>> sys.getsizeof('a' * 80 * 50) >> >> 16032 >> >> >>> sys.getsizeof('a' * 80 * 50 + '•') >> >> 16036 >> > Correct, but how many times does it happen? > Practically never. What are you talking about? Surely it happens the same number of times that your example happens, since it's the same example. By dismissing this example as being too infrequent to be of any importance, you dismiss the validity of your own example as well. > In this unicode stuff, I'm fascinated by the obsession > to solve a problem which is, due to the nature of > Unicode, unsolvable. > > For every optimization algorithm, for every code > point range you can optimize, it is always possible > to find a case breaking that optimization. So what? Similarly, for any generalized data compression algorithm, it is possible to engineer inputs for which the "compressed" output is as large as or larger than the original input (this is easy to prove). Does this mean that compression algorithms are useless? I hardly think so, as evidenced by the widespread popularity of tools like gzip and WinZip. You seem to be saying that because we cannot pack all Unicode strings into 1-byte or 2-byte per character representations, we should just give up and force everybody to use maximum-width representations for all strings. That is absurd. > Sure, it is possible to optimize the unicode usage > by not using French characters, punctuation, mathematical > symbols, currency symbols, CJK characters... > (select undesired characters here: http://www.unicode.org/charts/). > > In that case, why using unicode? > (A problematic not specific to Python) Obviously, it is because I want to have the *ability* to represent all those characters in my strings, even if I am not necessarily going to take advantage of that ability in every single string that I produce. Not all of the strings I use are going to fit into the 1-byte or 2-byte per character representation. Fine, whatever -- that's part of the cost of internationalization. However, *most* of the strings that I work with (this entire email message, for instance) -- and, I think, most of the strings that any developer works with (identifiers in the standard library, for instance) -- will fit into at least the 2-byte per character representation. Why shackle every string everywhere to 4 bytes per character when for a majority of them we can do much better than that? -- http://mail.python.org/mailman/listinfo/python-list
Variables vs names [was: Objects in Python]
On 08/23/2012 12:56 PM, Steven D'Aprano wrote: > On Thu, 23 Aug 2012 12:17:03 -0500, Evan Driscoll wrote: > >> I definitely *wouldn't* say "Python >> classes aren't really classes" -- even though (I assert) Python classes >> are *far* further from Simula-style (/Java/C++) classes than Python >> variables are from Java variables. > > Well, Python classes are first-class (pun not intended) objects in their > own right, and hence are data. Java and C++ classes are not, they are > instructions to the compiler rather than data. > > But other than that, what do you see as the difference between Python > classes and Simula-style classes? So first, to some extent that's like saying "these two different things are different in this important way, but other than that, what's the difference?" :-) But there are some other differences. (Not all of these are strictly with classes per se, but I would say they all have strong interactions with the object system.) * Explicit 'self'. OK, this sounds just like a minor syntactic difference, and in some respect it is. (For some time I considered it an annoying piece of syntactic salt.) But it has significant interactions with other things which you can do, such as using a method as just a normal function ('type(c).f(c)' ~= 'c.f()') or attaching other functions to an instance or a class (more on that later). This is definitely my weakest argument. :-) * Fields. In Simula-style classes, you can tell easily what fields a class and its objects contain. In Python, that question from some point of view doesn't even make sense (in the absence of __slots__). Fields are a property of the *objects* rather than the class, and two objects of the same class don't necessarily have the same fields. Related to this point we have... * What it means for an object to have a particular class type. With Simula-style classes, if I have an object 'o' of class 'c', then I know that 'o' has the functions and fields defined by 'c'. Now, the virtual functions may have been overriden in base classes and stuff, and maydbe they'll always fail, but I at least know they're *there*. In Python, I know... well, nothing basically. As far as I know, it's possible to make it so that o's only relation to 'c' is what 'type(o)' and 'instanceof' say. (And maybe you can even override those, I dunno!) You can go through and add/remove/replace functions. Two different objects of the same class may have completely disjoint sets of attributes. > Given: > > x = some_object() > y = x > > I could say that x and y are the same object, rather than x and y are > references to the same object. Huh, fair enough. >> To me, saying "here's an alternative way to look at variables" is great, >> but saying "Python doesn't have variables" is, IMO, at least as silly as >> what Jussi said. To me, dancing around the issue just leads to more >> confusing terminology and makes things worse. >> >> (And this is reinforced by the fact that neither I nor Google seems to >> have really seen "Python doesn't have classes" ever used, when that >> statement is at least as true as "Python doesn't have variables".) > > I think you are utterly wrong here. > > Python has classes. They are created by the "class" keyword. Whether > those classes are identical to Java classes is irrelevant -- in Python, > these whatever-they-are-things are called "classes", and so Python has > classes. > > But Python does not have things called "variables". There is no > "variable" keyword to create a variable. OK, let's make a new language. I'll call it 'Python--' because at least *I* wouldn't want to program in it. :-) In Python--, any time you use a name, you have to prefix it with the word 'variable': variable x = 4 print(variable x) Does Python-- have variables? Does Python? To me, the answer to those questions basically has to be the same -- after all, the new 'variable' keyword didn't really change the language, just have it a slightly different concrete syntax. Heck, if I wanted to implement Python--, the only thing I'd have to change in a Python implementation is the lexer! And if you say "no, Python-- doesn't have variables, it just has something that it wrongly calls variables", then it's no contradiction to say "Python doesn't have classes, it just has something that it wrongly calls classes." Think of it as duck-typing the term "variable". :-) To me, Python locals and globals look and quack like a variable. Incidentally, I also realized another reason I don't like the 'names' description. Take 'foo.bar'. (That is, the 'bar' attribute in object 'foo'.) Is 'foo.bar' a name? I'm not sure what the 'names' proponents would say, but to me both answers are problematic. I *really* dislike a 'no' answer because to me, 'foo.bar' absolutely *is* a name for the corresponding object. (This terminology has precedent.) But a 'yes' answer, if you also reject 'variable', means you no longer hav
Re: Unittest - testing for filenames and filesize
On Thursday, August 23, 2012 1:29:19 PM UTC-4, Terry Reedy wrote: > One can start with a set rather than tuple of file names. > filenames = {"this.txt", "that.txt", "the_other.txt"} Yeah, that's even cleaner. Just be aware, the set notation above is only available in (IIRC), 2.7 or above. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
Le jeudi 23 août 2012 15:57:50 UTC+2, Neil Hodgson a écrit : > wxjmfa...@gmail.com: > > > > > Small illustration. Take an a4 page containing 50 lines of 80 ascii > > > characters, add a single 'EM DASH' or an 'BULLET' (code points> 0x2000), > > > and you will see all the optimization efforts destroyed. > > > > > >>> sys.getsizeof('a' * 80 * 50) > > > 4025 > > sys.getsizeof('a' * 80 * 50 + '•') > > > 8040 > > > > This example is still benefiting from shrinking the number of bytes > > in half over using 32 bits per character as was the case with Python 3.2: > > > > >>> sys.getsizeof('a' * 80 * 50) > > 16032 > > >>> sys.getsizeof('a' * 80 * 50 + '•') > > 16036 > Correct, but how many times does it happen? Practically never. In this unicode stuff, I'm fascinated by the obsession to solve a problem which is, due to the nature of Unicode, unsolvable. For every optimization algorithm, for every code point range you can optimize, it is always possible to find a case breaking that optimization. This follows quasi the mathematical logic. To proof a law is valid, you have to proof all the cases are valid. To proof a law is invalid, just find one case showing it. Sure, it is possible to optimize the unicode usage by not using French characters, punctuation, mathematical symbols, currency symbols, CJK characters... (select undesired characters here: http://www.unicode.org/charts/). In that case, why using unicode? (A problematic not specific to Python) jmf -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On Thu, Aug 23, 2012 at 12:02 PM, Jan Kuiken wrote: > On 8/23/12 06:11 , Steven D'Aprano wrote: > >>> 2) Related to the above, you can infinitely nest scopes. There's nothing >>> wrong with having six variables called 'q'; you always use the innermost >>> one. Yes, this can hurt readability >> >> >> Well, there you go. There *is* something wrong with having six variables >> called 'q'. > > > Sometimes you don't want only six variables called 'q' but a hundred > of them :-) > > def fac(q): > if q < 1 : > return 1 > else: > return q * fac(q-1) > > print(fac(100)) That's only one variable called 'q', instantiated 100 times simultaneously. -- http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On Thu, 23 Aug 2012 09:49:41 -0700, Aaron Brady wrote: [...] > The patch for the above is only 40-60 lines. However it introduces two > new concepts. > > The first is a "linked list", a classic dynamic data structure, first > developed in 1955, cf. http://en.wikipedia.org/wiki/Linked_list . > Linked lists are absent in Python They certainly are not. There's merely no named "linked list" class. Linked lists are used by collections.ChainMap, tracebacks, xml.dom, Abstract Syntax Trees, and probably many other places. (Well, technically some of these are trees rather than lists.) You can trivially create a linked list: x = [a, [b, [c, [d, [e, None] is equivalent to a singly-linked list with five nodes. Only less efficient. > The second is "uncounted references". The uncounted references are > references to "set iterators" exclusively, exist only internally to > "set" objects, and are invisible to the rest of the program. The reason > for the exception is that iterators are unique in the Python Data Model; > iterators consist of a single immutable reference, unlike both immutable > types such as strings and numbers, as well as container types. Counted > references could be used instead, but would be consistently wasted work > for the garbage collector, though the benefit to programmers' peace of > mind could be significant. The usual way to implement "uncounted references" is by using weakrefs. Why invent yet another form of weakref? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On 8/23/12 06:11 , Steven D'Aprano wrote: 2) Related to the above, you can infinitely nest scopes. There's nothing wrong with having six variables called 'q'; you always use the innermost one. Yes, this can hurt readability Well, there you go. There *is* something wrong with having six variables called 'q'. Sometimes you don't want only six variables called 'q' but a hundred of them :-) def fac(q): if q < 1 : return 1 else: return q * fac(q-1) print(fac(100)) Jan Kuiken -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On 23/08/12 17:44, Evan Driscoll wrote: On 08/23/2012 04:19 AM, lipska the kat wrote: Well we don't want to turn this into a language comparison thread do we, that might upset too many people but I can't remember ever writing a method that took an Object as argument, you just can't do that much with an Object. In the pre-Java-1.5 days, functions that took Objects were *very* common; Well the Collections framework does expose methods that take Objects but generally you override certain methods in class Object when you want to compare Objects, in fact String comparison is a really interesting area due to the way Java internalizes Strings at compile time... but that is way to off topic for here. regards lipska -- Lipska the Kat©: Troll hunter, sandbox destroyer and farscape dreamer of Aeryn Sun -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On Thu, 23 Aug 2012 12:17:03 -0500, Evan Driscoll wrote: > I definitely *wouldn't* say "Python > classes aren't really classes" -- even though (I assert) Python classes > are *far* further from Simula-style (/Java/C++) classes than Python > variables are from Java variables. Well, Python classes are first-class (pun not intended) objects in their own right, and hence are data. Java and C++ classes are not, they are instructions to the compiler rather than data. But other than that, what do you see as the difference between Python classes and Simula-style classes? > On 08/23/2012 03:59 AM, Jussi Piitulainen wrote: >> I would also avoid any distinction between an object and a "reference" >> to an object, except as an implementation detail. It's not helpful. > > Wh? > > How would you describe it then? To me, that distinction is absolutely > *fundamental* to understanding how languages like Python/Scheme/Java > work, because it tells you how to reason about aliasing behavior in an > unconvoluted way (which is essential to understanding how they work). > > How would *you* suggest dealing with that issue? Given: x = some_object() y = x I could say that x and y are the same object, rather than x and y are references to the same object. Sometimes, when I say "x", I mean the *name* x, which is a reference to some object. Much more often though, when I say "x", I use it as a convenient label for some object. Rather than saying "the object which is referenced to by the name x", I just say "x". Nearly always, the meaning is obvious in context. [...] > *However*, this thread wasn't really prompted by someone just trying to > explain variables in different terms -- it was prompted by one of the > many comments you see from time-to-time that "Python doesn't have > variables – not as C or Java programmers would understand the term". No offence to Ben Finney, but I think sometimes he's a bit too eager to emphasise the subtle differences between Python and other languages, rather than the similarities. (I used to be like Ben in this regard, but I got better -- or worse, depending on your perspective.) Again, context is important: sometimes I will choose to gloss over the differences by calling x a variable, and sometimes I will emphasise the differences to C or Pascal by referring to name binding. > To me, saying "here's an alternative way to look at variables" is great, > but saying "Python doesn't have variables" is, IMO, at least as silly as > what Jussi said. To me, dancing around the issue just leads to more > confusing terminology and makes things worse. > > (And this is reinforced by the fact that neither I nor Google seems to > have really seen "Python doesn't have classes" ever used, when that > statement is at least as true as "Python doesn't have variables".) I think you are utterly wrong here. Python has classes. They are created by the "class" keyword. Whether those classes are identical to Java classes is irrelevant -- in Python, these whatever-they-are-things are called "classes", and so Python has classes. But Python does not have things called "variables". There is no "variable" keyword to create a variable. It is absolutely fundamental to the programming model of Python that it has objects which are bound to names in namespaces (and other entities, such as list items). That is *critical* -- Python uses name bindings. But name bindings are a kind of variable. Named memory locations are a *different* kind of variable. The behaviour of C variables and Python variables do not follow the Liskov Substitution Principle -- you can't rip out the entire "names bound to objects" machinery of Python and replace it with C-like named memory locations without changing the high- level behaviour of Python. And so by some ways of thinking it is *wrong* to treat name bindings and memory locations as "the same sort of entity". Hence, if C variables are variables, then Python name bindings can't be. I used to agree with that reasoning. I no longer do, not entirely. While I see the differences between them -- for instance, C variables exist before they have a value assigned to them, Python name bindings do not -- I don't think the differences are important enough to *prohibit* use of the word "variable" to describe name bindings. Only to discourage it. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: Unittest - testing for filenames and filesize
On 8/23/2012 8:28 AM, Roy Smith wrote: I think you want to end up with something like: def test_1(self): "Verify creation of files is possible" filenames = ("this.txt", "that.txt", "the_other.txt") for filename in filenames: f = open(filename, "w") f.write("Some text\n") f.close() self.assertTrue(f.closed) dir_names = os.listdir() self.assertEqual(set(dir_names), set(filenames)) The above code isn't tested, but it should give you the gist of what you need to do. One can start with a set rather than tuple of file names. def test_1(self): "Verify creation of files is possible" filenames = {"this.txt", "that.txt", "the_other.txt"} for filename in filenames: f = open(filename, "w") f.write("Some text\n") f.close() self.assertTrue(f.closed) dir_names = set(os.listdir()) self.assertEqual(dir_names, filenames) -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On Aug 23, 3:11 pm, Peter Otten <__pete...@web.de> wrote: > Mark Carter wrote: > > Suppose I want to define a function "safe", which returns the argument > > passed if there is no error, and 42 if there is one. So the setup is > > something like: > > > def safe(x): > > # WHAT WOULD DEFINE HERE? > > > print safe(666) # prints 666 > > print safe(1/0) # prints 42 > > > I don't see how such a function could be defined. Is it possible? > > 1/0 is evaluated before safe() is called. Therefore safe() has no chance to > catch the exception. You have to move the evaluation into the safe() > function: > > >>> def safe(deferred, default=42, exception=Exception): > > ... try: > ... return deferred() > ... except exception: > ... return default > ...>>> print safe(lambda: 666) > 666 > >>> print safe(lambda: 1/0) > > 42 Nice! Functional programmers know that once you have a lazy language, you can simulate all control constructs within that framework. eg in Haskell one could define an 'if-function' as iffunc (True, a, b) = a iffunc (False, a, b) = b In most other languages such a definition would not work. What Peter has demonstrated is that for an eager language like python, a bare-lambda is a lazy-fying construct -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On 8/23/2012 10:43 AM, Jerry Hill wrote: Personally, when I was learning python I found the idea of python having names and values (rather than variables and references) to clear up a lot of my misconceptions of the python object model. I think it's done the same for other people too, especially those who come from the C world, where a variable is a named and typed location in memory. There are two important points about C and assembler. First, the named locations (and un-named internal locations like function return addresses) are *contiguous*. Giving a user access to one block may give a user access to other blocks if one is not careful. The other is that the typing is in the code and compiler, but not in the runtime memory. So text input can be read as code and a return jump address to the bytes interpreted as code. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
[I have some things to say to a few people so I'm just putting it all in one rather than clutter up your email box even more.] On 08/23/2012 12:33 AM, Chris Angelico wrote: > [Snip discussion on names vs variables] > > Does that sort things out, or just make things more confusing? > > ChrisA I... am not sure. I don't think it really told me anything new. I think it's just that I don't think that any of the distinctions (nor all of them put together) makes it worthwhile to reject a perfectly good term in common use just because the behavior of Python variables is a bit different from the behavior of variables in *some* other languages. For instance, I don't get annoyed that both Java and Python use the terms "class" and "object" for similar but still very different things. And while I think it's worth saying "Python classes and objects are rather different from Java's", I definitely *wouldn't* say "Python classes aren't really classes" -- even though (I assert) Python classes are *far* further from Simula-style (/Java/C++) classes than Python variables are from Java variables. On 08/23/2012 03:59 AM, Jussi Piitulainen wrote: > I would also avoid any distinction between an object and a > "reference" to an object, except as an implementation detail. It's not > helpful. Wh? How would you describe it then? To me, that distinction is absolutely *fundamental* to understanding how languages like Python/Scheme/Java work, because it tells you how to reason about aliasing behavior in an unconvoluted way (which is essential to understanding how they work). How would *you* suggest dealing with that issue? On 08/23/2012 09:43 AM, Jerry Hill wrote: > On Thu, Aug 23, 2012 at 4:59 AM, Jussi Piitulainen wrote: >> I don't get it either. To me the python-has-no-variables campaigners >> seem confused. As far as I can see, Python can be seen in terms of >> variables bound to (locations containing) values perfectly well, in a >> way that should be quite familiar to people who come from Java (or >> Lisp, Scheme like me). > > ... > > Perhaps those that come from the Java and Lisp world don't find the > name/value paradigm as helpful. Still, it seems silly to excoriate > people on the list for trying to explain python fundamentals in > several different ways. Sometimes explaining the same idea in > different words helps people understand the concept better. I agree with this, and I'm happy that people found it useful. *However*, this thread wasn't really prompted by someone just trying to explain variables in different terms -- it was prompted by one of the many comments you see from time-to-time that "Python doesn't have variables – not as C or Java programmers would understand the term". That's a different than saying "here's another way of looking at Python variables", and that instance is even toned down compared to a lot of the instances you'll find (which will often omit the qualification at the end). To me, saying "here's an alternative way to look at variables" is great, but saying "Python doesn't have variables" is, IMO, at least as silly as what Jussi said. To me, dancing around the issue just leads to more confusing terminology and makes things worse. (And this is reinforced by the fact that neither I nor Google seems to have really seen "Python doesn't have classes" ever used, when that statement is at least as true as "Python doesn't have variables".) Evan signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On Aug 23, 9:34 am, Steven D'Aprano wrote: > On Wed, 22 Aug 2012 18:46:43 +0100, lipska the kat wrote: > > We need to separate out the 'view' from the 'implementation' here. Most > > developers I know, if looking at the code and without the possibly > > dubious benefit of knowing that in Python 'everything is an object' > > would not call this 'strong typing' > > Most developers are wrong :) > > A very useful resource to read is "What To Know Before Debating Type > Systems", which was put on the internet, lost, then found and put back up > here: > > http://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ Thanks for that article. There was one by Peter Wegner -- exact one I cant find. Here's a good approximation: http://analysis-of-persistence-tools.googlecode.com/files/p7-wegner.pdf Section 5.2.1 in particular has 7 different things that the word 'type' could mean to different types(!) of programmers. -- http://mail.python.org/mailman/listinfo/python-list
Re: set and dict iteration
On Saturday, August 18, 2012 9:28:32 PM UTC-5, Aaron Brady wrote: > On Saturday, August 18, 2012 5:14:05 PM UTC-5, MRAB wrote: > > > On 18/08/2012 21:29, Aaron Brady wrote: > > > > On Friday, August 17, 2012 4:57:41 PM UTC-5, Chris Angelico wrote: > > > >> On Sat, Aug 18, 2012 at 4:37 AM, Aaron Brady > > >> wrote: > > > >> > Is there a problem with hacking on the Beta? > > > >> Nope. Hack on the beta, then when the release arrives, rebase your > > > >> work onto it. I doubt that anything of this nature will be changed > > > >> between now and then. > > > >> ChrisA > > > > Thanks Chris, your post was encouraging. > > > > I have a question about involving the 'tp_clear' field of the types. > > > > http://docs.python.org/dev/c-api/typeobj.html#PyTypeObject.tp_clear > > > > ''' > > > > ...The tuple type does not implement a tp_clear function, because it’s > > > possible to prove that no reference cycle can be composed entirely of > > > tuples. > > > > ''' > > > > I didn't follow the reasoning in the proof; the premise is necessary but > > > IMHO not obviously sufficient. Nevertheless, the earlier diagram > > > contains an overt homogeneous reference cycle. > > > > Reposting: > > > http://home.comcast.net/~castironpi-misc/clpy-0062%20set%20iterators.png > > > > In my estimate, the 'tp_traverse' and 'tp_clear' fields of the set > > > doesn't need to visit the auxiliary collection; the same fields of the > > > iterators don't need to visit the primary set or other iterators; and > > > references in the linked list don't need to be included in the iterators' > > > reference counts. > > > > Can someone who is more familiar with the cycle detector and cycle > > > breaker, help prove or disprove the above? > > > In simple terms, when you create an immutable object it can contain > > > only references to pre-existing objects, but in order to create a cycle > > > you need to make an object refer to another which is created later, so > > > it's not possible to create a cycle out of immutable objects. > > > However, using Python's C API it _is_ possible to create such a cycle, > > > by mutating an otherwise-immutable tuple (see PyTuple_SetItem and > > > PyTuple_SET_ITEM). > > Are there any precedents for storing uncounted references to PyObject's? > > One apparent problematic case is creating an iterator to a set, then adding > it to the set. However the operation is a modification, and causes the > iterator to be removed from the secondary list before the set is examined for > collection. > > Otherwise, the iterator keeps a counted reference to the set, but the set > does not keep a counted reference to the iterator, so the iterator will > always be freed first. Therefore, the set's secondary list will be empty > when the set is freed. > > Concurrent addition and deletion of iterators should be disabled, and the > iterators should remove themselves from the set's secondary list before they > decrement their references to the set. > > Please refresh the earlier diagram; counted references are distinguished > separately. Reposting: > http://home.comcast.net/~castironpi-misc/clpy-0062%20set%20iterators.png The patch for the above is only 40-60 lines. However it introduces two new concepts. The first is a "linked list", a classic dynamic data structure, first developed in 1955, cf. http://en.wikipedia.org/wiki/Linked_list . Linked lists are absent in Python, including the standard library and CPython implementation, beyond the weak reference mechanism and garbage collector. The "collections.deque" structure shares some of the linked list interface but uses arrays. The second is "uncounted references". The uncounted references are references to "set iterators" exclusively, exist only internally to "set" objects, and are invisible to the rest of the program. The reason for the exception is that iterators are unique in the Python Data Model; iterators consist of a single immutable reference, unlike both immutable types such as strings and numbers, as well as container types. Counted references could be used instead, but would be consistently wasted work for the garbage collector, though the benefit to programmers' peace of mind could be significant. Please share your opinion! Do you agree that the internal list resolves the inconsistency? Do you agree with the strategy? Do you agree that uncounted references are justified to introduce, or are counted references preferable? -- http://mail.python.org/mailman/listinfo/python-list
Re: Re: Objects in Python
On 08/23/2012 04:19 AM, lipska the kat wrote: > Well we don't want to turn this into a language comparison thread do we, > that might upset too many people but I can't remember ever writing a > method that took an Object as argument, you just can't do that much with > an Object. In the pre-Java-1.5 days, functions that took Objects were *very* common; e.g. every collections class. Even now there are probably lingering cases where either there's some code that hasn't been genericized or is too much work to genericize to make it worthwhile. (I do very little Java programming so can't point to any concrete cases if you would (reasonably) reject the example of java.util.collections being used in their non-generic form.) Anyway, the point wasn't that 'foo(Object o)' is common, just that you've probably seen something which is somewhat comparable. Evan signature.asc Description: OpenPGP digital signature -- http://mail.python.org/mailman/listinfo/python-list
Looking for duplicate modules
We got burned yesterday by a scenario which has burned us before. We had multiple copies of a module in sys.path. One (the one we wanted) was in our deployed code tree, the other was in /usr/local/lib/python/ or some such. It was a particularly confusing situation because we *knew* we had uninstalled the copy in /usr/local/lib. What we didn't know was that puppet was set up to check to make sure it was there and reinstalled it automagically if it ever disappeared! What often adds to the confusion in cases like this is that if you use a package manager (such as apt-get) to manage your module deployments, when you uninstall, the .py files get removed, but the .pyc's stay behind. I'm working on a tool which scans all the directories in sys.path and finds any modules which appear multiple times in the path. It'll also call out any .pyc's it finds without matching py's. I know those are not strictly errors, but both of them are, for lack of a better term, "deployment smells" and something to be warned about. Before I invest too much effort into building it, does such a tool already exist? We're slowly moving more of our deployment to using virtualenv (with --no-site-packages), but we're not fully there yet. And even when we get there, there will still be opportunities to make these sorts of mistakes. --- Roy Smith r...@panix.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On Thu, Aug 23, 2012 at 9:11 AM, MRAB wrote: > Perhaps the solution should've been to just switch between 2/4 bytes instead > of 1/2/4 bytes. :-) Why? You don't lose any complexity by doing that. I can see arguments for 1/2/4 or for just 4, but I can't see any advantage of 2/4 over either of those. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 23/08/2012 14:57, Neil Hodgson wrote: wxjmfa...@gmail.com: Small illustration. Take an a4 page containing 50 lines of 80 ascii characters, add a single 'EM DASH' or an 'BULLET' (code points> 0x2000), and you will see all the optimization efforts destroyed. sys.getsizeof('a' * 80 * 50) 4025 sys.getsizeof('a' * 80 * 50 + '•') 8040 This example is still benefiting from shrinking the number of bytes in half over using 32 bits per character as was the case with Python 3.2: >>> sys.getsizeof('a' * 80 * 50) 16032 >>> sys.getsizeof('a' * 80 * 50 + '•') 16036 >>> Perhaps the solution should've been to just switch between 2/4 bytes instead of 1/2/4 bytes. :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On Thu, Aug 23, 2012 at 4:59 AM, Jussi Piitulainen wrote: > I don't get it either. To me the python-has-no-variables campaigners > seem confused. As far as I can see, Python can be seen in terms of > variables bound to (locations containing) values perfectly well, in a > way that should be quite familiar to people who come from Java (or > Lisp, Scheme like me). Personally, when I was learning python I found the idea of python having names and values (rather than variables and references) to clear up a lot of my misconceptions of the python object model. I think it's done the same for other people too, especially those who come from the C world, where a variable is a named and typed location in memory. Perhaps those that come from the Java and Lisp world don't find the name/value paradigm as helpful. Still, it seems silly to excoriate people on the list for trying to explain python fundamentals in several different ways. Sometimes explaining the same idea in different words helps people understand the concept better. -- Jerry -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On 23/08/12 14:59, Ben Finney wrote: lipska the kat writes: On 23/08/12 05:14, Steven D'Aprano wrote: I think that's uncalled for. […] Excellent advice as usual, but I'm more than capable of looking after myself thank you. As is usual, it's not all about you; Steven is demonstrating that we require civil behaviour here, for anyone who may be watching but not saying anything. You 'require civil behaviour here' do you. Well so far I have been very civil. This is a USENET newsgroup, it's a public forum, not your personal domain. Please stop all this 'it's not all about you' meaningless nonsense. It's pointless and wastes mine and no doubt others time. If you want to carry this on please contact me off list. lipska -- Lipska the Kat©: Troll hunter, sandbox destroyer and farscape dreamer of Aeryn Sun -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
lipska the kat writes: > On 23/08/12 14:59, Ben Finney wrote: > > lipska the kat writes: > > > >> On 23/08/12 05:14, Steven D'Aprano wrote: > >>> I think that's uncalled for. > > […] > > > >> Excellent advice as usual, but I'm more than capable of looking after > >> myself thank you. > > > > As is usual, it's not all about you; Steven is demonstrating that we > > require civil behaviour here, for anyone who may be watching but not > > saying anything. > > You 'require civil behaviour here' do you. Well so far I have been > very civil. And I say again: it's not all about you. Steven's respons was to a different person in this thread. Please stop reacting as if everything said around you is directed at you. -- \“I don't accept the currently fashionable assertion that any | `\ view is automatically as worthy of respect as any equal and | _o__) opposite view.” —Douglas Adams | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
On 23/08/2012 13:47, wxjmfa...@gmail.com wrote: This is neither a complaint nor a question, just a comment. In the previous discussion related to the flexible string representation, Roy Smith added this comment: http://groups.google.com/group/comp.lang.python/browse_thread/thread/2645504f459bab50/eda342573381ff42 Not only I agree with his sentence: "Clearly, the world has moved to a 32-bit character set." he used in his comment a very intersting word: "punctuation". There is a point which is, in my mind, not very well understood, "digested", underestimated or neglected by many developers: the relation between the coding of the characters and the typography. Unicode (the consortium), does not only deal with the coding of the characters, it also worked on the characters *classification*. A deliberatly simplistic representation: "letters" in the bottom of the table, lower code points/integers; "typographic characters" like punctuation, common symbols, ... high in the table, high code points/integers. The conclusion is inescapable, if one wish to work in a "unicode mode", one is forced to use the whole palette of the unicode code points, this is the *nature* of Unicode. Technically, believing that it possible to optimize only a subrange of the unicode code points range is simply an illusion. A lot of work, probably quite complicate, which finally solves nothing. Python, in my mind, fell in this trap. "Simple is better than complex." -> hard to maintained "Flat is better than nested." -> code points range "Special cases aren't special enough to break the rules." -> special unicode code points? "Although practicality beats purity." -> or the opposite? "In the face of ambiguity, refuse the temptation to guess." -> guessing a user will only work with the "optimmized" char subrange. ... Small illustration. Take an a4 page containing 50 lines of 80 ascii characters, add a single 'EM DASH' or an 'BULLET' (code points > 0x2000), and you will see all the optimization efforts destroyed. sys.getsizeof('a' * 80 * 50) 4025 sys.getsizeof('a' * 80 * 50 + '•') 8040 Just my 2 € (code point 0x20ac) cents. jmf I'm looking forward to all the patches you are going to provide to correct all these (presumably) cPython deficiencies. When do they start arriving on the bug tracker? -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On 23/08/2012 10:05, Mark Carter wrote: Suppose I want to define a function "safe", which returns the argument passed if there is no error, and 42 if there is one. So the setup is something like: def safe(x): # WHAT WOULD DEFINE HERE? print safe(666) # prints 666 print safe(1/0) # prints 42 I don't see how such a function could be defined. Is it possible? Well you've already got lots of answers but I'm not certain about what you're trying to achieve. If you could explicitly state your requirements I'm sure that the numerous MVPs (Most Valuable Pythonistas) here would come up with The Best Solution ™ to your problem. -- Cheers. Mark Lawrence. -- http://mail.python.org/mailman/listinfo/python-list
Re: Flexible string representation, unicode, typography, ...
wxjmfa...@gmail.com: Small illustration. Take an a4 page containing 50 lines of 80 ascii characters, add a single 'EM DASH' or an 'BULLET' (code points> 0x2000), and you will see all the optimization efforts destroyed. sys.getsizeof('a' * 80 * 50) 4025 sys.getsizeof('a' * 80 * 50 + '•') 8040 This example is still benefiting from shrinking the number of bytes in half over using 32 bits per character as was the case with Python 3.2: >>> sys.getsizeof('a' * 80 * 50) 16032 >>> sys.getsizeof('a' * 80 * 50 + '•') 16036 >>> Neil -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
lipska the kat writes: > On 23/08/12 05:14, Steven D'Aprano wrote: > > I think that's uncalled for. […] > Excellent advice as usual, but I'm more than capable of looking after > myself thank you. As is usual, it's not all about you; Steven is demonstrating that we require civil behaviour here, for anyone who may be watching but not saying anything. -- \ “Skepticism is the highest duty and blind faith the one | `\ unpardonable sin.” —Thomas Henry Huxley, _Essays on | _o__) Controversial Questions_, 1889 | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: How to set the socket type and the protocol of a socket using create_connection?
On 2012-08-22, Dennis Lee Bieber wrote: > On Wed, 22 Aug 2012 01:43:19 -0700 (PDT), Guillaume Comte > declaimed the following in > gmane.comp.python.general: > >> I've managed to build the IP header. I've put the source and destination >> addresses in this header but it doesn't change the real source address... > > For all I know (I've done very little network programming, and that > was years ago using plain TCP and UDP -- worse, on a VMS system so it > wasn't the "UNIX style" socket interface), your network stack may still > be overriding the packet at some lower level and inserting the IP > associated with the interface the packet went out on... I've only been intermittently following this thread, but back when I added Python's raw packet support for Unix, the socket module was a _very_ thin wrapper for the underlying OS network socket API. The behavior of various types of sockets was defined entirely by the underlying OS. So, if you're trying to do something obscure (which it seems you are), asking people who know how to do it in C on the relevent OS is probably the best approach. Below are examples of sending and receiving a completely raw packet on Linux (where you provide _all_ the bytes: the MAC addreses, the Ethernet type, everything). --send-- #!/usr/bin/python import sys,os,socket,struct from optparse import OptionParser p = OptionParser() p.add_option("-i","--interface",dest="interface",metavar="",type='str',default="eth0") options,args = p.parse_args() if len(args) != 1: sys.stderr.write("you must provide a destination MAC address\n") sys.exit(1) def toHex(s): return " ".join([("%02x" % ord(c)) for c in s]) ethProto = 0x5678 dstMacStr = args[0] dstMacAddr = "".join(map(chr,[int(x,16) for x in dstMacStr.split(":")])) s = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, ethProto) s.bind((options.interface,ethProto)) ifName,ifProto,pktType,hwType,hwAddr = s.getsockname() srcMacAddr = hwAddr ethHeader = struct.pack("!6s6sh",dstMacAddr,srcMacAddr,ethProto) packet = ethHeader + "some ASCII data here" sys.stdout.write("tx: %s\n" % toHex(packet)) s.send(packet) s.close() - --recv-- #!/usr/bin/python import sys,os,socket,struct from optparse import OptionParser p = OptionParser() p.add_option("-i","--interface",dest="interface",metavar="",type='str',default="eth0") options,args = p.parse_args() if len(args) != 0: sys.stderr.write("no arguments accepted\n") sys.exit(1) def toHex(s): return " ".join([("%02x" % ord(c)) for c in s]) ethProto = 0x5678 s = socket.socket(socket.AF_PACKET, socket.SOCK_RAW, ethProto) s.bind((options.interface,ethProto)) packet = s.recv(4096) sys.stdout.write("rx: %s\n" % toHex(packet)) s.close() - -- Grant Edwards grant.b.edwardsYow! I'm changing the at CHANNEL ... But all I get gmail.comis commercials for "RONCO MIRACLE BAMBOO STEAMERS"! -- http://mail.python.org/mailman/listinfo/python-list
Flexible string representation, unicode, typography, ...
This is neither a complaint nor a question, just a comment. In the previous discussion related to the flexible string representation, Roy Smith added this comment: http://groups.google.com/group/comp.lang.python/browse_thread/thread/2645504f459bab50/eda342573381ff42 Not only I agree with his sentence: "Clearly, the world has moved to a 32-bit character set." he used in his comment a very intersting word: "punctuation". There is a point which is, in my mind, not very well understood, "digested", underestimated or neglected by many developers: the relation between the coding of the characters and the typography. Unicode (the consortium), does not only deal with the coding of the characters, it also worked on the characters *classification*. A deliberatly simplistic representation: "letters" in the bottom of the table, lower code points/integers; "typographic characters" like punctuation, common symbols, ... high in the table, high code points/integers. The conclusion is inescapable, if one wish to work in a "unicode mode", one is forced to use the whole palette of the unicode code points, this is the *nature* of Unicode. Technically, believing that it possible to optimize only a subrange of the unicode code points range is simply an illusion. A lot of work, probably quite complicate, which finally solves nothing. Python, in my mind, fell in this trap. "Simple is better than complex." -> hard to maintained "Flat is better than nested." -> code points range "Special cases aren't special enough to break the rules." -> special unicode code points? "Although practicality beats purity." -> or the opposite? "In the face of ambiguity, refuse the temptation to guess." -> guessing a user will only work with the "optimmized" char subrange. ... Small illustration. Take an a4 page containing 50 lines of 80 ascii characters, add a single 'EM DASH' or an 'BULLET' (code points > 0x2000), and you will see all the optimization efforts destroyed. >> sys.getsizeof('a' * 80 * 50) 4025 >>> sys.getsizeof('a' * 80 * 50 + '•') 8040 Just my 2 € (code point 0x20ac) cents. jmf -- http://mail.python.org/mailman/listinfo/python-list
Re: Python3.3 email policy date field
On Thu, 23 Aug 2012 12:36:01 +0100, MRAB wrote: > From what I've tried, it looks like the date can't be a string: > > >>> m['Date'] = datetime.datetime.utcnow() > >>> m['Date'] > 'Thu, 23 Aug 2012 11:33:20 -' Many thanks - it's even easier! Waiting for Python 3.3 to become standard! Helmut. -- http://mail.python.org/mailman/listinfo/python-list
Re: Unittest - testing for filenames and filesize
In article <6b0299df-bc24-406b-8d69-489e990d8...@googlegroups.com>, Tigerstyle wrote: > Hi. > > I need help with an assignment and I hope you guys can guide me in the right > direction. > [code elided] > 1. The test_1() method includes code to verify that the test directory > contains only the files created by the for loop. Hint: You might create a set > containing the list of three filenames, and then create a set from the > os.listdir() method. I'm not sure what your question is. The hint you give above pretty much tells you what to do. The basic issue here is that you started out with a list (well, tuple) of filenames. You can use os.listdir() to get a list of filenames that exist in the current directory. The problem is that you can't compare these two lists directly, because lists are ordered. Converting both lists to sets eliminates the ordering and lets you compare them. > I'm new to Python programming so I don't know where to put the set in point > 1. Before the test or under test1. I think you want to end up with something like: def test_1(self): "Verify creation of files is possible" filenames = ("this.txt", "that.txt", "the_other.txt") for filename in filenames: f = open(filename, "w") f.write("Some text\n") f.close() self.assertTrue(f.closed) dir_names = os.listdir() self.assertEqual(set(dir_names), set(filenames)) The above code isn't tested, but it should give you the gist of what you need to do. -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On 23 August 2012 10:05, Mark Carter wrote: > Suppose I want to define a function "safe", which returns the argument > passed if there is no error, and 42 if there is one. So the setup is > something like: > > def safe(x): ># WHAT WOULD DEFINE HERE? > > print safe(666) # prints 666 > print safe(1/0) # prints 42 > > I don't see how such a function could be defined. Is it possible? > It isn't possible to define a function that will do this as the function will never be called if an exception is raised while evaluating its arguments. Depending on your real problem is a context-manager might do what you want: >>> from contextlib import contextmanager >>> @contextmanager ... def safe(default): ... try: ... yield default ... except: ... pass ... >>> with safe(42) as x: ... x = 1/0 ... >>> x 42 >>> with safe(42) as x: ... x = 'qwe' ... >>> x 'qwe' Oscar -- http://mail.python.org/mailman/listinfo/python-list
Unittest - testing for filenames and filesize
Hi. I need help with an assignment and I hope you guys can guide me in the right direction. This is the code: -- """ Demostration of setUp and tearDown. The tests do not actually test anything - this is a demo. """ import unittest import tempfile import shutil import glob import os class FileTest(unittest.TestCase): def setUp(self): self.origdir = os.getcwd() self.dirname = tempfile.mkdtemp("testdir") os.chdir(self.dirname) def test_1(self): "Verify creation of files is possible" for filename in ("this.txt", "that.txt", "the_other.txt"): f = open(filename, "w") f.write("Some text\n") f.close() self.assertTrue(f.closed) def test_2(self): "Verify that current directory is empty" self.assertEqual(glob.glob("*"), [], "Directory not empty") def tearDown(self): os.chdir(self.origdir) shutil.rmtree(self.dirname) - I need to modify this code as following: 1. The test_1() method includes code to verify that the test directory contains only the files created by the for loop. Hint: You might create a set containing the list of three filenames, and then create a set from the os.listdir() method. 2. A test_3() method creates a binary file that contains exactly a million bytes, closes it and then uses os.stat to verify that the file on disk is of the correct length (with os.stat, statinfo.st_size returns the size in bytes). I'm new to Python programming so I don't know where to put the set in point 1. Before the test or under test1. Would appreciate pointers and solutions (with explanation)for both point 1 and 2. Thank you in advance. T -- http://mail.python.org/mailman/listinfo/python-list
Re: Python3.3 email policy date field
On 23/08/2012 09:30, Helmut Jarausch wrote: Hi, in response to a bug report I got the follow helpful comments from R. David Murray. Many thanks to him. (Unfortunately, I don't know his email, so I can write him directly) To generate an email (with non-ascii letters) R. David Murray wrote: But even better, so will this: m = Message(policy=policy.SMTP) m['From'] = "Günter Weiße " This works, but now I cannot add a date field Trying m['Date'] = datetime.datetime.utcnow().strftime('%m/%d/%Y %I:%M:%S %p') I get Traceback (most recent call last): File "Test_EMail_Py3_4.py", line 23, in msg['Date'] = datetime.datetime.utcnow().strftime('%m/%d/%Y %I:%M:%S %p') File "/usr/lib64/python3.3/email/message.py", line 359, in __setitem__ self._headers.append(self.policy.header_store_parse(name, val)) File "/usr/lib64/python3.3/email/policy.py", line 119, in header_store_parse return (name, self.header_factory(name, value)) File "/usr/lib64/python3.3/email/headerregistry.py", line 583, in __call__ return self[name](name, value) File "/usr/lib64/python3.3/email/headerregistry.py", line 194, in __new__ cls.parse(value, kwds) File "/usr/lib64/python3.3/email/headerregistry.py", line 300, in parse value = utils.parsedate_to_datetime(value) File "/usr/lib64/python3.3/email/utils.py", line 243, in parsedate_to_datetime *dtuple, tz = __parsedate_tz(data) TypeError: 'NoneType' object is not iterable From what I've tried, it looks like the date can't be a string: >>> m['Date'] = datetime.datetime.utcnow() >>> m['Date'] 'Thu, 23 Aug 2012 11:33:20 -' -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On 23/08/2012 09:59, Jussi Piitulainen wrote: Steven D'Aprano writes: On Wed, 22 Aug 2012 23:49:17 -0500, Evan Driscoll wrote: > On 8/22/2012 18:58, Ben Finney wrote: >> You haven't discovered anything about types; what you have >> discovered is that Python name bindings are not variables. >> >> In fact, Python doesn't have variables – not as C or Java >> programmers would understand the term. What it has instead are >> references to objects (with names as one kind of reference). > > OK, I've seen this said a few times, and I have to ask: what do > you mean by this? I consider myself pretty decent at Python and > other languages, and I really don't get it. I think the point that Ben would like to make is that while "name binding" is a specific kind of "variable", the word "variable" comes with too much baggage from the old-school C, Pascal etc. style of variables- are-named-memory-locations. Most of the time, the differences are unimportant, but when they are important, if your mental image is that Python "variables" (name bindings) are like C or Pascal "variables" (memory locations), you're going to get confused. I don't get it either. To me the python-has-no-variables campaigners seem confused. As far as I can see, Python can be seen in terms of variables bound to (locations containing) values perfectly well, in a way that should be quite familiar to people who come from Java (or Lisp, Scheme like me). [snip] In Java a variable exists even when it has not been assigned a value. In Python, on the other hand, the basic model is that a 'variable' doesn't exist until it has been bound to an value (although, for efficiency reasons, that's not entirely true, because at compile time CPython will identify the local variables in a function and allocate a 'slot' for it). -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
Laszlo Nagy wrote: > def safe(deferred, default=42, exception=Exception): >> ... try: >> ... return deferred() >> ... except exception: >> ... return default > > What a beautiful solution! I was wondering if the following would be > possible: > > > def test(thing, default, *exc_classes): > try: > thing() > except *exc_classes: > return default > > > But it is syntactically invalid. The except clause allows a tuple of exceptions: >>> def safe(deferred, default, *exceptions): ... try: ... return deferred() ... except exceptions: ... return default ... >>> safe(lambda: 1/0, -1, ValueError, ZeroDivisionError) -1 -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On 23/08/2012 12:01, Laszlo Nagy wrote: def safe(deferred, default=42, exception=Exception): ... try: ... return deferred() ... except exception: ... return default What a beautiful solution! I was wondering if the following would be possible: def test(thing, default, *exc_classes): try: thing() except *exc_classes: return default But it is syntactically invalid. Here is a workaround that is not so beautiful: def test(thing, default, *exc_classes): try: thing() except Exception, e: for cls in exc_classes: if isinstance(e,cls): return default raise print test( (lambda: 1/0), -1, ValueError, ZeroDivisionError) # prints -1 The 'except' clause accepts a tuple of exception classes, so this works: def test(thing, default, *exc_classes): try: thing() except exc_classes: return default -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
def safe(deferred, default=42, exception=Exception): ... try: ... return deferred() ... except exception: ... return default What a beautiful solution! I was wondering if the following would be possible: def test(thing, default, *exc_classes): try: thing() except *exc_classes: return default But it is syntactically invalid. Here is a workaround that is not so beautiful: def test(thing, default, *exc_classes): try: thing() except Exception, e: for cls in exc_classes: if isinstance(e,cls): return default raise print test( (lambda: 1/0), -1, ValueError, ZeroDivisionError) # prints -1 -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
Mark Carter wrote: > Suppose I want to define a function "safe", which returns the argument > passed if there is no error, and 42 if there is one. So the setup is > something like: > > def safe(x): ># WHAT WOULD DEFINE HERE? > > print safe(666) # prints 666 > print safe(1/0) # prints 42 > > I don't see how such a function could be defined. Is it possible? 1/0 is evaluated before safe() is called. Therefore safe() has no chance to catch the exception. You have to move the evaluation into the safe() function: >>> def safe(deferred, default=42, exception=Exception): ... try: ... return deferred() ... except exception: ... return default ... >>> print safe(lambda: 666) 666 >>> print safe(lambda: 1/0) 42 -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On Thursday, 23 August 2012 10:23:20 UTC+1, Laszlo Nagy wrote: > On 2012-08-23 11:05, Mark Carter wrote: > You are very vague. "There is an error" - but what kind of error? Assume that it doesn't matter. > In some special cases, this can be a good idea to do. Those are the cases that I'm interested in. -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On Thu, Aug 23, 2012 at 7:28 PM, Laszlo Nagy wrote: >> That can work ONLY if the division of 1/0 doesn't raise an exception. >> This is why the concept of NaN exists; I'm not sure if there's a way >> to tell Python to return NaN instead of bombing, but it's most likely >> only possible with floating point, not integer. > > For integers, Python will always raise an exception when you try to divide > by zero. And integers has nothing to do with NaN. Because NaN is meaningful > for floating point numbers only. Python can be compiled to raise floating > point exceptions. (On Python 2, this is a compile time option: FPECTL. On > Python 3, this can be configured runtime: > http://docs.python.org/library/fpectl.html ) Thanks, that's the sort of thing I meant. I'm not familiar enough with Python's floating point handler to know those details. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On Thu, Aug 23, 2012 at 7:22 PM, Mark Carter wrote: > OK, so it looks like a solution doesn't exist to the problem as specified. I > guess it's something that only a language with macros could accommodate. You're asking for a function to prevent the evaluation of its arguments from throwing an error. That's fundamentally not possible with a function call. Exception handling is a much more normal way of handling it; though if you prefer, you could wrap it up in a function like this: def safe_div(num,denom): try: return num/denom except ZeroDivisionError: return 42 print safe_div(1,0) # still a poor name though ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On 2012-08-23 11:05, Mark Carter wrote: Suppose I want to define a function "safe", which returns the argument passed if there is no error, and 42 if there is one. So the setup is something like: def safe(x): # WHAT WOULD DEFINE HERE? print safe(666) # prints 666 print safe(1/0) # prints 42 I don't see how such a function could be defined. Is it possible? You are very vague. "There is an error" - but what kind of error? To catch all possible exceptions you could do: def unsafe(x): # put your code here... def safe(x): try: return unsafe(x) except: return 42 Generally, it is a bad idea. Exception handlers were invented because they give you a way to handle any error in the call chain. When an exception occurs, the interpreter will start searching for an appropriate exception handler traversing up in the call chain. By converting exceptions into return values, you are bypassing this search. Then you will have to write conditions instead of exception handlers inside ALL methods in the call chain, creating a "manual" search for the handler of the exception. In most cases, this will make your code difficult, error prone and hard to read. In some special cases, this can be a good idea to do. Can you please let us know when and how would you like to use it? -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
That can work ONLY if the division of 1/0 doesn't raise an exception. This is why the concept of NaN exists; I'm not sure if there's a way to tell Python to return NaN instead of bombing, but it's most likely only possible with floating point, not integer. For integers, Python will always raise an exception when you try to divide by zero. And integers has nothing to do with NaN. Because NaN is meaningful for floating point numbers only. Python can be compiled to raise floating point exceptions. (On Python 2, this is a compile time option: FPECTL. On Python 3, this can be configured runtime: http://docs.python.org/library/fpectl.html ) -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On Thursday, 23 August 2012 10:16:08 UTC+1, Chris Angelico wrote: > On Thu, Aug 23, 2012 at 7:05 PM, Mark Carter <> wrote: > > Suppose I want to define a function "safe", which returns the argument > > passed if there is no error, and 42 if there is one. > only possible with floating point, not integer. > > try: > print 1/0 > except ZeroDivisionError: > print 42 OK, so it looks like a solution doesn't exist to the problem as specified. I guess it's something that only a language with macros could accommodate. -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On 22/08/12 22:31, Evan Driscoll wrote: On 08/22/2012 02:45 PM, lipska the kat wrote: On 22/08/12 20:03, Evan Driscoll wrote: Second, this concept isn't *so* unfamiliar to you. If I give you the following Java code: void foo(Object o) { ... } looking at this method declaration I can see that the method takes an argument of type Object (and just FYI class Object is not abstract and you can do Object o = new Object()) and does not return a value. I know that for the lifetime of this JVM, whatever o turns out to be it will always be an Object. I can't assign a primitive to o as ints chars floats etc are certainly not Objects. There are certain invariants that give me a warm and comfortable feeling inside. I'm not saying it's nothing, but "can't assign a primitive" isn't much of an invariant in the broad scheme of things Well we don't want to turn this into a language comparison thread do we, that might upset too many people but I can't remember ever writing a method that took an Object as argument, you just can't do that much with an Object. I do however often write methods that take an interface as argument knowing that in future, any classes I write that implement this interface would just work thanks to subtype polymorphism A method 'declaration' such as this in an interface Product getProductByBarcode(Barcode b) throws CrappyProductException; tells me a whole lot about what the 'definition' in an implementing class might do, in fact I might well get away with just reading the interface and using the method without having to delve into the code. And I think this is the nub of the problem at the moment. I'm in a particular mindset, almost 'locked in' you might say and when I see a Python function that doesn't give me what I need straight away I get annoyed. I will get over it. when you can pass items as diverse as lists, GUI buttons, files, etc. I would say it's more like if you see 'int x' then *that* imposes a pretty big invariant, but passing 'Object' imposes almost nothing. Well you may be able to pass them in but you couldn't really do anything meaningful with them as you are restricted to operations on Object, I suppose you could pepper your code with tests to check the runtime type of a reference but it all gets a bit messy. [snip] Thus *all* Python variables are essentially references.) That makes sense Thanks for taking the time to reply. It really is most valuable to me. lipska -- Lipska the Kat©: Troll hunter, sandbox destroyer and farscape dreamer of Aeryn Sun -- http://mail.python.org/mailman/listinfo/python-list
Re: Guarding arithmetic
On Thu, Aug 23, 2012 at 7:05 PM, Mark Carter wrote: > Suppose I want to define a function "safe", which returns the argument passed > if there is no error, and 42 if there is one. So the setup is something like: > > def safe(x): ># WHAT WOULD DEFINE HERE? > > print safe(666) # prints 666 > print safe(1/0) # prints 42 > > I don't see how such a function could be defined. Is it possible? That can work ONLY if the division of 1/0 doesn't raise an exception. This is why the concept of NaN exists; I'm not sure if there's a way to tell Python to return NaN instead of bombing, but it's most likely only possible with floating point, not integer. However, there's an easier way. try: print 1/0 except ZeroDivisionError: print 42 Catch the exception and do with it what you will. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Guarding arithmetic
Suppose I want to define a function "safe", which returns the argument passed if there is no error, and 42 if there is one. So the setup is something like: def safe(x): # WHAT WOULD DEFINE HERE? print safe(666) # prints 666 print safe(1/0) # prints 42 I don't see how such a function could be defined. Is it possible? -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
Steven D'Aprano writes: > On Wed, 22 Aug 2012 23:49:17 -0500, Evan Driscoll wrote: > > > On 8/22/2012 18:58, Ben Finney wrote: > >> You haven't discovered anything about types; what you have > >> discovered is that Python name bindings are not variables. > >> > >> In fact, Python doesn't have variables – not as C or Java > >> programmers would understand the term. What it has instead are > >> references to objects (with names as one kind of reference). > > > > OK, I've seen this said a few times, and I have to ask: what do > > you mean by this? I consider myself pretty decent at Python and > > other languages, and I really don't get it. > > I think the point that Ben would like to make is that while "name > binding" is a specific kind of "variable", the word "variable" comes > with too much baggage from the old-school C, Pascal etc. style of > variables- are-named-memory-locations. Most of the time, the > differences are unimportant, but when they are important, if your > mental image is that Python "variables" (name bindings) are like C > or Pascal "variables" (memory locations), you're going to get > confused. I don't get it either. To me the python-has-no-variables campaigners seem confused. As far as I can see, Python can be seen in terms of variables bound to (locations containing) values perfectly well, in a way that should be quite familiar to people who come from Java (or Lisp, Scheme like me). It is very bad that people campaign here against the terminology that is used in the official Python language reference and the Python tutorial. Best would be to simply explain how the variables, values, assignment, and calls work in Python, the way the language reference and tutorial actually do. If the no-variables campaign is to continue, the language reference and the tutorial should be changed to match. I would consider it a mad move, but the current practice of badmouthing the official wording is not healthy either. > [...] > > And many other languages have reference behavior and still call > > their bindings "variables", e.g. Scheme. > > Yes, well in my opinion, a great deal of the terminology in use is > absolutely dire. E.g. both Ruby and Java have the exact same > parameter binding strategy as Python, only the Ruby people call > theirs "call by reference" and the Java people call theirs "call by > value", *both of which are identical*, and NEITHER of which are the > same thing that C and Pascal programmers will understand by call by > value *or* call by reference. That's indeed such a mess that those call-by-* terms may be now best avoided. I would also avoid any distinction between an object and a "reference" to an object, except as an implementation detail. It's not helpful. It only leads to the confusion where it seems that Java (or Python) does not actually have objects at all, only references to objects, which in turn don't exist, so, er, what. The swap function is helpful. Why doesn't it work? Because it assigns to different variables that are local to the function. If I pass it a list, why can it then swap the elements? Because that is the same list, not a copy. Get used to it. Works for me. -- http://mail.python.org/mailman/listinfo/python-list
Python3.3 email policy date field
Hi, in response to a bug report I got the follow helpful comments from R. David Murray. Many thanks to him. (Unfortunately, I don't know his email, so I can write him directly) To generate an email (with non-ascii letters) R. David Murray wrote: >>> But even better, so will this: >>> m = Message(policy=policy.SMTP) >>> m['From'] = "Günter Weiße " This works, but now I cannot add a date field Trying m['Date'] = datetime.datetime.utcnow().strftime('%m/%d/%Y %I:%M:%S %p') I get Traceback (most recent call last): File "Test_EMail_Py3_4.py", line 23, in msg['Date'] = datetime.datetime.utcnow().strftime('%m/%d/%Y %I:%M:%S %p') File "/usr/lib64/python3.3/email/message.py", line 359, in __setitem__ self._headers.append(self.policy.header_store_parse(name, val)) File "/usr/lib64/python3.3/email/policy.py", line 119, in header_store_parse return (name, self.header_factory(name, value)) File "/usr/lib64/python3.3/email/headerregistry.py", line 583, in __call__ return self[name](name, value) File "/usr/lib64/python3.3/email/headerregistry.py", line 194, in __new__ cls.parse(value, kwds) File "/usr/lib64/python3.3/email/headerregistry.py", line 300, in parse value = utils.parsedate_to_datetime(value) File "/usr/lib64/python3.3/email/utils.py", line 243, in parsedate_to_datetime *dtuple, tz = __parsedate_tz(data) TypeError: 'NoneType' object is not iterable Many thanks for a hint, Helmut. -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On 23/08/12 05:14, Steven D'Aprano wrote: On Thu, 23 Aug 2012 01:19:49 +, Walter Hurry wrote: On Wed, 22 Aug 2012 18:46:43 +0100, lipska the kat wrote: Well I'm a beginner Then maybe you should read more and write less. I think that's uncalled for. Lipska isn't trolling. He's making observations as he sees them. The fact that they're sometimes wrong is not a reason to effectively tell him to STFU. Better the misconception which is spoken allowed and corrected, then the one which is kept quiet and festers. Excellent advice as usual, but I'm more than capable of looking after myself thank you. Nobody has ever succeeded in making me STFU yet :-) When that day comes I'll retire to the garden lipska -- Lipska the Kat©: Troll hunter, sandbox destroyer and farscape dreamer of Aeryn Sun -- http://mail.python.org/mailman/listinfo/python-list
Re: Books?
[ Virgil Stokes wrote on Wed 22.Aug'12 at 16:34:40 +0200 ] > On 22-Aug-2012 16:04, Steven D'Aprano wrote: > > On Tue, 21 Aug 2012 18:36:50 -0700, Anonymous Group wrote: > > > >> What books do you recomend for learning python? Preferably free and/or > >> online. > > Completely by coincidence, I have just discovered, and I mean *literally* > > just a few minutes ago, this book: > > > > http://www.springer.com/mathematics/computational+science+%26+engineering/book/978-3-642-30292-3 > > > > http://codingcat.com/knjige/python/A%20Primer%20on%20Scientific%20Programming%20with%20Python.pdf > > > > > > I wish it had existed when I was a beginner! I haven't read the whole > > thing, but dammit it looks like exactly the sort of book I would have > > adored as a newbie. (Your mileage may vary.) > > > > > > > I second this --- this is a very good book IMHO. I have the first edition > (2009) > and have found it very useful. > > Good tip! absolutely. I've been reading it most of last night, certainly a great learning resource. -- http://mail.python.org/mailman/listinfo/python-list
Re: New image and color management library for Python 2+3
On 20.08.2012 20:34, Christian Heimes wrote: > Am 19.08.2012 19:35, schrieb Jan Riechers: > > Hello Jan, > > we decided against ImageMagick and pgmagick for several reasons. For one > we were already using FreeImage in other projects (Delphi projects and > through ctypes binding with FreeImagePy). > [SNIP] > > The Python bindings for ImageMagick weren't that good and today they are > still buggy. For example I'm not able to set the resize filter to a high > quality setting like Catmull-Rom-Spline with the most recent version of > pgmagick. filterType() doesn't do anything. The output image still has > the same MD5 hash. > > ImageMagick and PIL were missing important features, too. For example > both libraries don't support color management with lcms2 nor cached > color transformations nor introspection of ICC profiles. They use lcms1. > The color profiles explains - perhaps it would be interesting still to test out your library with regular images (without color profiles) against ImageMagick, just to compare the raw speed of both solutions. In my case I really would love to see if there is a even higher performance solution available as I plan to serve a lot of users a time and especially keeping the Input/Output during file writing low would be very helpful. > >> Can you perhaps test your solution with ImageMagick (as it is used >> widely) it would be interesting so. > > I've added some more benchmark results to the README.txt (PIL with > non-standard libjpeg-turbo and pgmagick). The results are available at > https://bitbucket.org/tiran/smc.freeimage > > Spoiler: pgmagick isn't faster and its resize filter settings don't work. > Thank you for the additional benchmarks, nice to read! I will try your library out to see how it compares to old-pendant ImageMagick with Pypy 1.9 | Wand / Pillow (Pil fork for pypy) and Python 2.7.3 PIL / ImageMagick - perhaps I can deliver you those numbers too - for those folks not completely in need of a color profile aware version but to compare raw numbers Perhaps it would also be nice to see if and how it works with Pypy too Jan -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On 23/08/12 02:19, Walter Hurry wrote: On Wed, 22 Aug 2012 18:46:43 +0100, lipska the kat wrote: Well I'm a beginner Then maybe you should read more and write less. Really ? I read all the responses to my posts and learn more from them in less time than I ever have from reading the 'documentation' If you don't like it then don't read it. lipska -- Lipska the Kat©: Troll hunter, sandbox destroyer and farscape dreamer of Aeryn Sun -- http://mail.python.org/mailman/listinfo/python-list
Data cleaning workouts
List folk, I am a newbie trying to get used to Python. I was wondering if anyone knows of web resources that teach good practices in data cleaning and management for statistics/analytics/machine learning, particularly using Python. Ideally, these would be exercises of the form: here is some horrible raw data --> here is what it should look like after it has been cleaned. Guidelines about steps that should always be taken, practices that should be avoided; basically, workflow of data analysis in Python with special emphasis on the cleaning part. -- http://mail.python.org/mailman/listinfo/python-list
Re: help me debug my "word capitalizer" script
On Wednesday, August 22, 2012 3:28:18 AM UTC-7, Kamil Kuduk wrote: > less file.txt | sed -e "s/\b\([a-z]\{4,\}\)/\u\1/g" Say what? Yes, you could do a crazy regex at the Linux prompt. But... will you be able to retain that insane syntax in your head until the NEXT time you need to write something like that? Probably not, unless you write awk all day. That, ladies and gentlemen, is why there's Python. I stopped programming for about 18 months a while back. When I came back to Python and needed to do it again, I picked up right where I left off. -- http://mail.python.org/mailman/listinfo/python-list
Re: Objects in Python
On Wed, 22 Aug 2012 23:49:17 -0500, Evan Driscoll wrote: > On 8/22/2012 18:58, Ben Finney wrote: >> You haven't discovered anything about types; what you have discovered >> is that Python name bindings are not variables. >> >> In fact, Python doesn't have variables – not as C or Java programmers >> would understand the term. What it has instead are references to >> objects (with names as one kind of reference). > > OK, I've seen this said a few times, and I have to ask: what do you mean > by this? I consider myself pretty decent at Python and other languages, > and I really don't get it. I think the point that Ben would like to make is that while "name binding" is a specific kind of "variable", the word "variable" comes with too much baggage from the old-school C, Pascal etc. style of variables- are-named-memory-locations. Most of the time, the differences are unimportant, but when they are important, if your mental image is that Python "variables" (name bindings) are like C or Pascal "variables" (memory locations), you're going to get confused. [...] > And many other languages have reference behavior and still call their > bindings "variables", e.g. Scheme. Yes, well in my opinion, a great deal of the terminology in use is absolutely dire. E.g. both Ruby and Java have the exact same parameter binding strategy as Python, only the Ruby people call theirs "call by reference" and the Java people call theirs "call by value", *both of which are identical*, and NEITHER of which are the same thing that C and Pascal programmers will understand by call by value *or* call by reference. http://mail.python.org/pipermail/tutor/2010-December/080505.html -- Steven -- http://mail.python.org/mailman/listinfo/python-list