luban 1.0.0
Luban (http://lubanui.org) is a compact, generic UI language. It is a python package for building (web) user interface. It is NOT yet-another web framework. Features: * Dynamic, ajax-based web user interface can be created using pure python (no knowledge of javascript/ajax/etc is required) * A compact set of simple concepts and rules * Simple, natural syntax for describing UI * A set of basic and advanced widgets --- see demo and API of widgets at http://lubanui.org/aokuang * Write bindings of your favorite js widgets to use them in python. Python versions: * Luban 1.0.0 mainly supports python 3.1+ * Experimental support for python 2 exists. Tested and working under python 2.7 Installation: $ pip install luban Bug reports and any comments are appreciated! -- Jiao Lin linj...@caltech.edu -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations/
Re: .format vs. %
alex23於 2012年1月4日星期三UTC+8上午10時26分35秒寫道： 8 Dihedral dihedr...@googlemail.com wrote: This is a good evolution in Python. It is 2012 now and the text I/O part is not as important as 10 years ago. The next move of Python could be easy integration of C++ libraries. You mean like with Py++? http://pypi.python.org/pypi/pyplusplus/ The auto code generation part for low-paid tedious GUI tasks is still missing in the standard Python. Wait, what? How is it the Python community's responsibility to relieve talentless code-monkeys of their tedium? Why can't they write their own damn code generators? I think a code generator to assist the programmer for common tedious boring GUI jobs such as BOA is more important in Python. But Python is not VB. Boa and wxpython are still not good enough. And what are you contributing to the situation other than misinformation and markov-generated spam? Do you know what can attract newbies to support python? -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
Honestly, is this list really what this is all about? I'm bored already... Enough? On Tue, Jan 3, 2012 at 9:42 PM, Ben Finney ben+pyt...@benfinney.id.auwrote: Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes: On Wed, 04 Jan 2012 12:54:09 +1100, Ben Finney wrote: It objectifies women. So you claim. I'm sure you have a hundred ready rationalisations for why a joke that has “girlfriend” as a fungible object, together with “car” and “house” as things to mechanically import into one's life, is somehow not objectifying women. But, while those rationalisations may satisfy you, I'm not interested in hearing them. If you don't see that the joke objectifies women, that tells me quite a lot about how blinkered you are to the problem. For what it's worth, I'm very familiar with such rationalisations, having employed them many times myself. Fortunately there are women who will speak up against it and encourage men to do the same URL:http://tldp.org/HOWTO/Encourage-Women-Linux-HOWTO/x168.html. Since you can't or won't persuade me (and anyone else reading) that this sort of joke is harmful, does that mean that you will stop claiming that it is harmful? I have no idea what it would take to persuade you in particular. I do know that the combined privileges of being white, male, not-poor, and English-fluent (and many more privileges, I'm sure) grant both of us the luxury of barely even perceiving the harm done by a pervasive atmosphere of even low-level prejudice against any given group of people. Women (to return to the people in question), on the other hand, do not have that luxury. In this community they have no option but to be aware of the privileges we males have here. We have very effective cognitive blinders, merely because we have never needed to know what it's like being a woman in this prevalently-male field. The onus is on us to try hard to see, despite those blinders and easy rationalisations, that there is a lot we allow from our fellows which is perpetuating a hostile environment. I'm not making a fuss about one sexist joke, which has already been retracted by its author. I'm making a fuss about allowing and, worse, defending such jokes as a tacitly-accepted norm of our community. And I hope those of us who prefer to think of ourselves as not-sexist will act to clean up our house more. -- \ “A hundred times every day I remind myself that […] I must | `\ exert myself in order to give in the same measure as I have | _o__)received and am still receiving” —Albert Einstein | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: python philosophical question - strong vs duck typing
Since Python does not 'silently convert types' as I understand those 3
words, you lose me here. Can you give a code example of what you mean?
I mean the reasoning behind the arguments like
'X isn't strongly typed because 2 + 3 = 5 but 3 + 2 = 32'.
OCaml considers this a problem and bans all implicit conversions whatsoever.
(Or maybe that's more to do with making their type inference better?)
I actually stole the classification of continuums from a Python wiki
page, and my terminology was influenced by the reading. See:
http://wiki.python.org/moin/StrongVsWeakTyping
I am aware that typed names lead to typed function signatures and that some
languages in the ML family even type operators, so that different operators
are required for 1+2 and 1.0+2.0 and mixed operations like 1+2.0 are
prohibited. But we are talking about the typing of objects here.
I should state up-front that I'm not sure we can only talk about the
typing of objects.
There's a bit of a problem in that dynamically typed and statically
typed aren't exactly things that can be compared -- they're almost
orthogonal. Almost. Dynamically typed languages like Python keep, by
necessity, information about what they are and what they came from (at
run-time). (The alternative is to be untyped, which is bad).
Statically typed languages do not always have to keep them around. If
you do enough at compile-time, you can forget everything at run-time.
Run-time and compile-time are completely different, and a language can
do any combination of things at both. It's weird that we've decided
that there's only two options rather than four, and that we really
want to compare these two options.
I'm not sure how that fits into what I've said. Maybe I've already
contradicted myself.
Anyway, in answer, OCaml forbids both at run-time _and_ compile-time
the coercion of floats to ints via the + operator, and in doing so
enforces its notion of strong typing. You can't add a float to an int.
So in this notion, it allows less implicit type conversion -- and
run-time object conversion -- than Python, which freely converts
floats to ints. This philosophy extends elsewhere in OCaml.
Using induction, I can prove, for instance, that these two functions
def f_tr(n, val=base): # tail recursive
if n:
return f_tr(n-1, rec(n, val))
else:
return val
def f_wi(n, val = base): # while iterative
while n:
n, val = n-1, rec(n, val)
return val
are equivalent, assuming enough stack and normal procedural Python
semantics. (And assuming no typos ;-).
f_tr(0) == base == f_wi(0)
f_tr(n+1) == f_tr(n+1, base) == f_tr(n, rec(n+1, base))
== by inductive hypothsis
f_wi(n, rec(n+1, base)) == f_wi(n+1, base) == f_wi(n+1)
So it is not clear to me what such proofs have to do with types as I
understand the term.
Ah, I messed up and didn't explain that. It doesn't help that I've
been too lazy to actually work with ATS proofs directly, so I might
say some things that are wrong/vague.
In some languages -- particularly dependently-typed languages -- a
type can represent a computation. That is, the return type of a
function might essentially be the computation that we wish to do, and
therefore if the function typechecks then we know it does that
computation. (This has all the caveats of formal methods as a whole).
Many (all?) of the automated theorem provers are dependently typed
languages in this fashion.
In particular, if you pick some well-defined decidable computational
language, you can try to make your type system powerful enough to
encode it. (For example, the primitive recursive functions are a nice
subset.)
For an example, in ATS the following is a compiler-type which can be
used as part of the type declaration for a function:
dataprop FIB (int, int) =
| FIB0 (0, 0) | FIB1 (1, 1)
| {n:nat} {r0,r1:int} FIB2 (n+2, r0+r1) of (FIB (n, r0), FIB (n+1, r1))
The predicate FIB(X, Y) is true when Fibonacci(X) = Y, so this is
essentially the same as defining fibonacci recursively:
Fibonacci(0) = 0
Fibonacci(1) = 1
Fibonacci(A+2) = Fibonacci(A+1) + Fibonacci(A)
It encodes the recursive definition of fibonacci, and if we declare a
function as taking a value m, returning a value n, such that FIB(m,
n), then a successful typecheck is equivalent to a successful proof
that the implementation is correct.
How does that typecheck work? I don't really know, I haven't at all
investigated the proof structure for ATS. Sorry. :(
Some cursory reading says that a function can return a proof that is
typechecked against dataprop, as well as the value. Maybe that's it.
:)
I can imagine that if one overloads 'type' with enough extra information,
the procedural reasoning involved in such proofs might reduce to a more
calculational demonstration. The question is whether the months and years of
intellectual investment required on the front end pay off in reduced
intellectual effort across several applications.
Yes. Well, the compiler writers
Re: Locale bug?
=?ISO-8859-1?Q?Trond_Endrest=F8l?= trond.endres...@ximalas.info:
Marko Rauhamaa ma...@pacujo.net writes:
$ python3
locale.setlocale(locale.LC_TIME, ('fi_FI', 'UTF-8'))
time.strftime(%a, %d %b %Y %H:%M:%S %z (%Z))
'ti, 03 tammi\xa0 2012 14:51:57 +0200 (EET)'
It may be OS-specific.
You're right. It's a generic linux problem (in the coreutils rpm of
Fedora). An analogous C program demonstrates the same issue.
Your sample code runs fine on FreeBSD/i386
trond@enterprise:~python3.2
locale.setlocale(locale.LC_TIME, ('fi_FI', 'UTF-8'))
time.strftime(%a, %d %b %Y %H:%M:%S %z (%Z))
'Ti, 03 Tam 2012 15:03:15 +0100 (CET)'
Well, Ti and Tam probably shouldn't be capitalized...
Marko
--
http://mail.python.org/mailman/listinfo/python-list
Re: Locale bug?
Marko Rauhamaa ma...@pacujo.net:
=?ISO-8859-1?Q?Trond_Endrest=F8l?= trond.endres...@ximalas.info:
Marko Rauhamaa ma...@pacujo.net writes:
$ python3
locale.setlocale(locale.LC_TIME, ('fi_FI', 'UTF-8'))
time.strftime(%a, %d %b %Y %H:%M:%S %z (%Z))
'ti, 03 tammi\xa0 2012 14:51:57 +0200 (EET)'
It may be OS-specific.
You're right. It's a generic linux problem (in the coreutils rpm of
Fedora). An analogous C program demonstrates the same issue.
Actually, it's working as designed, no bug at all.
It turns out the garbage character is the nonbreaking space:
print(time.strftime(%a, %d %b %Y %H:%M:%S %z (%Z)))
ke, 04 tammi 2012 15:03:21 +0200 (EET)
The apparent intent is to make all month abbreviations equally long (six
characters).
Marko
--
http://mail.python.org/mailman/listinfo/python-list
Re: python philosophical question - strong vs duck typing
On Tue, Jan 3, 2012 at 7:28 PM, Ben Finney ben+pyt...@benfinney.id.au wrote: Sean Wolfe ether@gmail.com writes: Hello everybody, I'm a happy pythonista newly subscribed to the group. Welcome! Thanks! and thanks to all, hjaha. I have a theoretical / philosophical question regarding strong vs duck typing in Python. Let's say we wanted to type strongly in Python There may be an unstated assumption there, and your wording confuses me. yep, probably. I am throwing around terminology a bit. Here's another attempt -- If I am willing to create some python code, when needed, where when I create a variable, let's say an integer, that it will be for all time an integer, and also that a method that returns say a Sprite custom object, and will for all time return only a Sprite object ... , does this get me significantly closer to being able to compile to C++? I am just thinking in my brain about the differences between cpp and python, and if there is a way to compromise a bit on the python-ness to get closer to cpp, but still be able to keep a lot of the goodness, then put in a translator or converter to cpp and gain performance by using cpp code. Sounds like Rpython, cython, shedskin are doing a lot or all of this, so lots to study up on. “Strongly-typed” is one end of a spectrum whose opposite end is “weakly-typed”. Weakly-typed objects are in languages like e.g. PHP, where an integer object can be added to a string object. Ah ok, I didn't realize this distinction. Now I grok it a bit better. Python does not have variables in the sense of languages like C; rather, Python has references bound to objects. A reference (e.g. a name, or a list index, etc.) never has a type. An object always has a type. yeah I've been learning a lot about this ... at times I have to 're-create' a variable to avoid modifying the original value as well. For example, when I pass a 'screen' object in my game, at times I have to duplicate the screen in the new method, then work on the duplicate, otherwise I will be using the original screen by reference. You may be thinking of “static typing” (identifiers have types, and won't allow themselves to refer to an object of a different type), versus “dynamic typing” (identifiers are ignorant of types – this is what you have in Python). Yep I think so. Thanks for the info all! -- http://mail.python.org/mailman/listinfo/python-list
pipe into preallocated buffer?
is there a way to pipe directly into a preallocated buffer? (subprocessing.pipe.stdout) thanks, --mihai -- http://mail.python.org/mailman/listinfo/python-list
Re: Large list in memory slows Python
On Tue, Jan 3, 2012 at 5:59 PM, Peter Otten __pete...@web.de wrote: Benoit Thiell wrote: I am experiencing a puzzling problem with both Python 2.4 and Python 2.6 on CentOS 5. I'm looking for an explanation of the problem and possible solutions. Here is what I did: Python 2.4.3 (#1, Sep 21 2011, 19:55:41) IPython 0.8.4 -- An enhanced Interactive Python. In [1]: def test(): ...: return [(i,) for i in range(10**6)] In [2]: %time x = test() CPU times: user 0.82 s, sys: 0.04 s, total: 0.86 s Wall time: 0.86 s In [4]: big_list = range(50 * 10**6) In [5]: %time y = test() CPU times: user 9.11 s, sys: 0.03 s, total: 9.14 s Wall time: 9.15 s As you can see, after creating a list of 50 million integers, creating the same list of 1 million tuples takes about 10 times longer than the first time. I ran these tests on a machine with 144GB of memory and it is not swapping. Before creating the big list of integers, IPython used 111MB of memory; After the creation, it used 1664MB of memory. In older Pythons the heuristic used to decide when to run the cyclic garbage collection is not well suited for the creation of many objects in a row. Try switching it off temporarily with import gc gc.disable() # create many objects that are here to stay gc.enable() You may also encorporate that into your test function: def test(): gc.disable() try: return [...] finally: gc.enable() Thanks Peter, this is very helpful. Modifying my test according to your directions produced much more consistent results. Benoit. -- Benoit Thiell The SAO/NASA Astrophysics Data System http://adswww.harvard.edu/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
On Wed, Jan 4, 2012 at 02:42, Ben Finney ben+pyt...@benfinney.id.au wrote: I'm sure you have a hundred ready rationalisations for why a joke that has “girlfriend” as a fungible object, together with “car” and “house” as things to mechanically import into one's life, is somehow not objectifying women. But, while those rationalisations may satisfy you, I'm not interested in hearing them. If you don't see that the joke objectifies women, that tells me quite a lot about how blinkered you are to the problem. For what it's worth, I'm very familiar with such rationalisations, having employed them many times myself. Fortunately there are women who will speak up against it and encourage men to do the same URL:http://tldp.org/HOWTO/Encourage-Women-Linux-HOWTO/x168.html. Steven D'Aprano steve+comp.lang.pyt...@pearwood.info writes: Since you can't or won't persuade me (and anyone else reading) that this sort of joke is harmful, does that mean that you will stop claiming that it is harmful? I have no idea what it would take to persuade you in particular. I do know that the combined privileges of being white, male, not-poor, and English-fluent (and many more privileges, I'm sure) grant both of us the luxury of barely even perceiving the harm done by a pervasive atmosphere of even low-level prejudice against any given group of people. Women (to return to the people in question), on the other hand, do not have that luxury. In this community they have no option but to be aware of the privileges we males have here. We have very effective cognitive blinders, merely because we have never needed to know what it's like being a woman in this prevalently-male field. The onus is on us to try hard to see, despite those blinders and easy rationalisations, that there is a lot we allow from our fellows which is perpetuating a hostile environment. I'm not making a fuss about one sexist joke, which has already been retracted by its author. I'm making a fuss about allowing and, worse, defending such jokes as a tacitly-accepted norm of our community. And I hope those of us who prefer to think of ourselves as not-sexist will act to clean up our house more. A hearty +1 to this! -- Rami Chowdhury A mind all logic is like a knife all blade - it makes the hand bleed that uses it. -- Rabindranath Tagore +44-7581-430-517 / +1-408-597-7068 / +88-0189-245544 -- http://mail.python.org/mailman/listinfo/python-list
Program blocked in Queue.Queue.get and Queue.Queue.put
I have a program that is blocked and all threads are blocked on a Queue.Queue.get or Queue.Queue.put method (on the same Queue.Queue object). 1 thread shows the below as its last entry in the stack: File: c:\python27\lib\Queue.py, line 161, in get self.not_empty.acquire() 2 threads show the below as its last entry in the stack: File: c:\python27\lib\Queue.py, line 118, in put self.not_full.acquire() According to me, this means both the Queue.Queue.not_full and Queue.Queue.not_empty locks are taken, but no other thread seems to have it. Of course, I don't access the locks my self directly. I did send an KeyboardInterrupt to the main thread however. Could it be that it was at that moment doing a Queue.Queue.put and it got interrupted while it has the lock, but before it entered the try block with the finally that releases the lock (so between line 118 and 119 in the Queue.py file)? If this is the case, how do I avoid that? Or is it a bug in the Queue.Queue class? If this is not the case, any clue what else could have happened? Thanks -- http://mail.python.org/mailman/listinfo/python-list
help me get excited about python 3
I am still living in the 2.x world because all the things I want to do right now in python are in 2 (django, pygame). But I want to be excited about the future of the language. I understand the concept of needing to break backwards compatibility. But it's not particularly exciting to think about. What are the cool new bits I should be reading up on? Related reading ... http://lucumr.pocoo.org/2011/12/7/thoughts-on-python3 [Zaphod] Hey, Marvin! We've got a job for you. [Marvin] I won't enjoy it. [Zaphod] Yes, you will. There's a whole new life stretching out in front of you! [Marvin] Oh, not another one! [Zaphod] Shut up and listen! There'll be excitement and adventure and really wild things! [Marvin] Sounds awful. [Zaphod] But, Marvin ... [Marvin] I suppose you want me to help you to get into this spaceship ... [Zaphod] Marvin, will you just listen! [Marvin] ... and open the door for you. [Zaphod] What? Er ... Yeah. [Marvin] Well, I wish you'd just tell me, rather than try to engage my enthusiasm, because I haven't got one. -- A musician must make music, an artist must paint, a poet must write, if he is to be ultimately at peace with himself. - Abraham Maslow -- http://mail.python.org/mailman/listinfo/python-list
Re: python philosophical question - strong vs duck typing
On 1/4/2012 12:37 AM, Terry Reedy wrote: Using induction, I can prove, for instance, that these two functions [snip] are equivalent, assuming enough stack and normal procedural Python semantics. (And assuming no typos ;-). YOU proved that; your type system didn't. With a powerful enough type system, those two functions would have the same type, while if you had made a typo they wouldn't. The extreme example of a powerful type system is something like Coq or Elf. In a language like that, a mathematical sentence is encoded in a type, and objects of a certain type represent a proof that that the sentence can be proved. Evan -- http://mail.python.org/mailman/listinfo/python-list
Re: Newbie Help
On Tue, 03 Jan 2012 17:13:17 -0600, mixolydian wrote: I want to get into Python progamming for both local database applications and dynamic web pages. Maybe some QD scripts. I am new to Python too, and recently completed my first real cross- platform GUI application with local/remote database access. I concur with the other replies. I see no need for an IDE and wholeheartedly recommend PostgreSQL (which integrates with Python very well indeed using Psycopg 2). I have found wxPython best for GUI. There is even a source code generator available for the graphical bits of the application - wxGlade. I was able to get up to speed very easily and quickly indeed (though of course there is a great deal more to learn). These tools are all FOSS. -- http://mail.python.org/mailman/listinfo/python-list
Re: python philosophical question - strong vs duck typing
On Wed, 2012-01-04 at 11:30 -0300, Sean Wolfe wrote:
On Tue, Jan 3, 2012 at 7:28 PM, Ben Finney ben+pyt...@benfinney.id.au wrote:
Sean Wolfe ether@gmail.com writes:
Hello everybody, I'm a happy pythonista newly subscribed to the group.
Welcome!
Thanks! and thanks to all, hjaha.
I have a theoretical / philosophical question regarding strong vs duck
typing in Python. Let's say we wanted to type strongly in Python
There may be an unstated assumption there, and your wording confuses me.
yep, probably. I am throwing around terminology a bit. Here's another
attempt --
If I am willing to create some python code, when needed, where when I
create a variable, let's say an integer, that it will be for all time
an integer, and also that a method that returns say a Sprite custom
object, and will for all time return only a Sprite object ... , does
this get me significantly closer to being able to compile to C++?
I'd really recommend looking at Cython - which has optional static
typing and does compile into C / C++ (as a python extension)
More generally, a compiler can perform static analysis on code which
will re-order AST nodes into single constant assignments. I've forgotten
the name but it's something like single static assignment form. When the
return type of functions is known it can lead to known types for
variables.
It's being used heavily in the newest generation of javascript JITs to
speed up generated native code.
However, when a function has multiple return types (e.g. {}.get returns
None if there is no result) then you can't imply the type of the
variable even in this form.
A JIT (such as pypy) can generate the native code for all seen return
types - which is why JITs can in general be more useful to dynamically
typed languages such as Python than compilers.
Another issue is where types can be modified (e.g. in python you can
modify the class of an object at runtime) - dynamic language features
such as this make what counts as a type fairly flexible. JITs are
getting around this using hidden classes (there are lots of other
names for the same thing) - again it would be very difficult to
statically compile this kind of thing to native code.
I am just thinking in my brain about the differences between cpp and
python, and if there is a way to compromise a bit on the python-ness
to get closer to cpp, but still be able to keep a lot of the goodness,
then put in a translator or converter to cpp and gain performance by
using cpp code. Sounds like Rpython, cython, shedskin are doing a lot
or all of this, so lots to study up on.
Yup
Tim Wintle
--
http://mail.python.org/mailman/listinfo/python-list
Re: help me get excited about python 3
On Wed, Jan 4, 2012 at 8:56 AM, Sean Wolfe ether@gmail.com wrote: I am still living in the 2.x world because all the things I want to do right now in python are in 2 (django, pygame). But I want to be excited about the future of the language. I understand the concept of needing to break backwards compatibility. But it's not particularly exciting to think about. What are the cool new bits I should be reading up on? pygame is available for Python 3. I've used it a small amount, and it appears to work just fine. There is also a fork of Django that supports Python 2 and Python 3 in a single codebase: https://bitbucket.org/vinay.sajip/django/ It's still experimental right now, but that doesn't mean you couldn't try it out. The expectation is that it will be merged for the Django 1.5 release. -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
On Tue, Jan 3, 2012 at 7:50 PM, Tony Pelletier tony.pellet...@gmail.com wrote: Honestly, is this list really what this is all about? I'm bored already... Sorry, this list does not exist for your personal entertainment. Maybe you should try YouTube. And no, it's not really about sexism either, but there is no harm in the occasional educational rant. Kudos to Ben for fighting the good fight! -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
On Wed, Jan 4, 2012 at 11:45 AM, Tony Pelletier tony.pellet...@gmail.com wrote: That's a rather ironic comment. Idiot. Really? Which part was ironic? Sorry, this list does not exist for your personal entertainment. Not this one, that's just a statement of fact. Maybe you should try YouTube. Kinda. What I really meant here was Maybe you should just go away, but that is not really in opposition with what I actually said, so it's not technically irony either, just mild understatement. And no, it's not really about sexism either, Literal statement of fact. but there is no harm in the occasional educational rant. Literal statement of opinion. Kudos to Ben for fighting the good fight! And I meant that 100%. Also not ironic. ... Oh, wait, I get it now. Your post was ironically self-referential. Clever! By the way, take a constructive suggestion and stop top-posting. Bottom-posting is the commonly preferred style around these parts. http://en.wikipedia.org/wiki/Posting_style#Top-posting Cheers [irony!], Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
I have zero desire to follow the rules of a Python(here's the ironic part. Get it now clever boy?) list when it'd riddled with childish banter that has nothing to do with wait for it. Python? Do I need to explain it any further? I'm done with you and this list that pollutes my inbox with globs of worthlessness... On Wed, Jan 4, 2012 at 2:28 PM, Ian Kelly ian.g.ke...@gmail.com wrote: On Wed, Jan 4, 2012 at 11:45 AM, Tony Pelletier tony.pellet...@gmail.com wrote: That's a rather ironic comment. Idiot. Really? Which part was ironic? Sorry, this list does not exist for your personal entertainment. Not this one, that's just a statement of fact. Maybe you should try YouTube. Kinda. What I really meant here was Maybe you should just go away, but that is not really in opposition with what I actually said, so it's not technically irony either, just mild understatement. And no, it's not really about sexism either, Literal statement of fact. but there is no harm in the occasional educational rant. Literal statement of opinion. Kudos to Ben for fighting the good fight! And I meant that 100%. Also not ironic. ... Oh, wait, I get it now. Your post was ironically self-referential. Clever! By the way, take a constructive suggestion and stop top-posting. Bottom-posting is the commonly preferred style around these parts. http://en.wikipedia.org/wiki/Posting_style#Top-posting Cheers [irony!], Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
On Wed, Jan 4, 2012 at 12:36 PM, Tony Pelletier tony.pellet...@gmail.com wrote: I have zero desire to follow the rules of a Python(here's the ironic part. Get it now clever boy?) list when it'd riddled with childish banter that has nothing to do with wait for it. Python? Fair enough. If this thread and this exchange were actually representative of the list, I wouldn't want to waste my time with it either. -- http://mail.python.org/mailman/listinfo/python-list
Typed python comparison / code analysis questions
Hello, I'm relatively new to Python. I come from C/C++ so I love the flexibility but I am slightly irked by the lack of compilation time checking. I've got two questions 1) Are there any tools that do an analysis of code and attempt to catch potential issues such as undefined variables, etc? I use xdebug in php for this purpose and it's a life saver. Currently for Python I use module test cases and simulate situations but this doesn't catch everything and on a larger scale of development becomes difficult. 2) Is there a way to error when comparing variables of different types? I was up late last night coding and caused a few bugs with the below code, which turns out to compare a typestr to a typeint. I know good programming I should know what my variable types are, but it would be even better if I could always abide by a rule of using _ (or something) which would spit an exception when comparing non-equal types/instances. rows = pamss.conn.fetchRows(results) count = rows[0]['cases']// should be int(rows[0]['cases']) return count MAX_CONCURRENT_IVR thanks! Lucas -- http://mail.python.org/mailman/listinfo/python-list
Re: python philosophical question - strong vs duck typing
On 1/4/2012 1:37 AM, Terry Reedy wrote:
On 1/3/2012 8:04 PM, Devin Jeanpierre wrote:
[ An example of a simple dependently typed program:
http://codepad.org/eLr7lLJd ]
Just got it after a minute delay.
A followup now that I have read it. Removing the 40 line comment, the
function itself is
fun getitem{n,m:nat}(arr : array(int, n) ,
length : int(n), index : int m) : int =
if index length then
arr[index]
else
~1 (* -1, error *)
where n,m are compiler variables used to define the dependent
(paramaterized) types array(int,n) and int(n)/ The double use of n means
that the compiler checks that length n of the array equals the length
passed.
My response: in Python, there is no need to pass concrete collection
sizes because they are packaged with the collection at runtime as an
attribute. So:
1) In Python, there is no need for such checking. In addition, the
for-loop construct, 'for item in iterable:', removes the possibility of
indexing errors.
2) Python classes are, in a sense, or in effect, runtime dependent
types. While the formal implementation type of a 'list' is just 'list',
the effective computation type is 'mutable sequence of length n'. The
type of an iterator is 'read-only sequence of indefinite length'. I find
this an interesting way to look at Python.
--
Terry Jan Reedy
--
http://mail.python.org/mailman/listinfo/python-list
Re: Newbie Help
Ben Finney ben+pyt...@benfinney.id.au writes:
* Cross-platform, so that you're not denied the use of any popular
workstation OS.
For my purposes, either { GNU Screen + Bash + Emacs } or { GNU Screen +
Bash + Vim } are good choices satisfying all the above criteria. There
may be other good combinations.
I slightly mis-spoke. Obviously GNU Screen and Bash are only truly
available on Unix-like operating systems. But that's pretty much all of
the popular workstation OSes except one.
GNU Emacs and Vim are truly available on every popular workstation OS.
Learn one of them well, and you will be able to take on the diversity of
programming tasks much easier.
--
\ “A lot of people are afraid of heights. Not me, I'm afraid of |
`\ widths.” —Steven Wright |
_o__) |
Ben Finney
--
http://mail.python.org/mailman/listinfo/python-list
Re: Typed python comparison / code analysis questions
On Wed, Jan 4, 2012 at 1:09 PM, Lucas Vickers lucasvick...@gmail.com wrote: Hello, I'm relatively new to Python. I come from C/C++ so I love the flexibility but I am slightly irked by the lack of compilation time checking. I've got two questions 1) Are there any tools that do an analysis of code and attempt to catch potential issues such as undefined variables, etc? I use xdebug in php for this purpose and it's a life saver. Currently for Python I use module test cases and simulate situations but this doesn't catch everything and on a larger scale of development becomes difficult. Yes, try pylint, pychecker, or pyflakes. 2) Is there a way to error when comparing variables of different types? I was up late last night coding and caused a few bugs with the below code, which turns out to compare a typestr to a typeint. I know good programming I should know what my variable types are, but it would be even better if I could always abide by a rule of using _ (or something) which would spit an exception when comparing non-equal types/instances. This is a well-known wart in Python 2. The recommended solution would be to upgrade to Python 3, which no longer allows ordered comparisons between unrelated built-in types. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Typed python comparison / code analysis questions
On Thu, Jan 5, 2012 at 7:09 AM, Lucas Vickers lucasvick...@gmail.com wrote: 2) Is there a way to error when comparing variables of different types? Yep. Use Python version 3. 11 Traceback (most recent call last): File pyshell#88, line 1, in module 11 TypeError: unorderable types: int() str() Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: Typed python comparison / code analysis questions
Thank you! At the moment python3 isn't an option. There's a variety of dependencies I'm working around. Is there any type of 2.x add-on? either way thanks for the info L On Wed, Jan 4, 2012 at 3:34 PM, Ian Kelly ian.g.ke...@gmail.com wrote: On Wed, Jan 4, 2012 at 1:09 PM, Lucas Vickers lucasvick...@gmail.com wrote: Hello, I'm relatively new to Python. I come from C/C++ so I love the flexibility but I am slightly irked by the lack of compilation time checking. I've got two questions 1) Are there any tools that do an analysis of code and attempt to catch potential issues such as undefined variables, etc? I use xdebug in php for this purpose and it's a life saver. Currently for Python I use module test cases and simulate situations but this doesn't catch everything and on a larger scale of development becomes difficult. Yes, try pylint, pychecker, or pyflakes. 2) Is there a way to error when comparing variables of different types? I was up late last night coding and caused a few bugs with the below code, which turns out to compare a typestr to a typeint. I know good programming I should know what my variable types are, but it would be even better if I could always abide by a rule of using _ (or something) which would spit an exception when comparing non-equal types/instances. This is a well-known wart in Python 2. The recommended solution would be to upgrade to Python 3, which no longer allows ordered comparisons between unrelated built-in types. Cheers, Ian -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
Le mar, 03 jan 2012 20:28:59, Steven D'Aprano a ploppé: On Tue, 03 Jan 2012 05:08:47 -0800, Ethan Furman wrote: [...] maybe policing uploads is worse than cluttering PyPI's disk space and RSS feed with dumb 1 KB packages. (Matt Chaput) I'd drop the maybe. It's hard enough finding what one wants without having to wade through crap. PyPI is not the place for it. I agree. But who gets to choose which packages are serious enough to deserve to be on PyPI? Or good enough? The cost of allowing anyone the freedom to publish their work is that some published work will be crap. Besides, I find it hard to believe that the search facilities on PyPI are so bad that there would be any searches that come up with girlfriend.py or car.py as false positives. Seriously, the over-reaction here is something that has to be seen to be believed. The original complaint is over a couple of entries in an RSS feed and showing up on the front page of PyPI, perhaps a dozen words in total. The reaction has been thousands of words arguing back and forth. +1 The it objectifies wymyn argument (back and forth) was particularly funny (in a insignificant nonsensical hollier-than-thou nitpicking way). I really was on the verge of pushing the plonk button a couple times (between spurts of uncontrollable laughter). If anything, I see the joke as derogatory to Python programmers; sadly, this very thread proves this view. Also, I find it disturbing that the wymyn-power knights in shiny armour identify a python module with a woman. If I nuke a Karmic Koala, will they rat me out to the WWF, to the UNODA, or to both? -- PiLS -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
On 4 January 2012 20:08, Ian Kelly ian.g.ke...@gmail.com wrote: On Wed, Jan 4, 2012 at 12:36 PM, Tony Pelletier tony.pellet...@gmail.com wrote: I have zero desire to follow the rules of a Python(here's the ironic part. Get it now clever boy?) list when it'd riddled with childish banter that has nothing to do with wait for it. Python? Fair enough. If this thread and this exchange were actually representative of the list, I wouldn't want to waste my time with it either. Sadly, my feeling is that the amount of vacuous discussions and hair-splitting debates is steadily increasing. Just to see, I had a look at the archives for July 2001 (because that's around the time I took up Python) and the list definitely had a more interesting feel to it. It also made me realise that many of the best contributors have gone or only make rare appearances. Anyway, there was little spam, but already some trolls, e.g. Is Python Dead? http://mail.python.org/pipermail/python-list/2001-July/091178.html Not everyone was convinced about PEP 238 (new semantics for /): http://mail.python.org/pipermail/python-list/2001-July/090659.html Did you change your mind, Terry? I too used to be against it :) I'll stop now. Cheers, -- Arnaud -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
On Thu, Jan 5, 2012 at 7:48 AM, PiLS p...@invalid.ca wrote: If I nuke a Karmic Koala, will they rat me out to the WWF, to the UNODA, or to both? Neither, actually. We'll be so glad you didn't call it a Karmic Koala Bear that we'll send you three American tourists for free. (They're actually quite delicious when properly prepared.) ChrisA -- http://mail.python.org/mailman/listinfo/python-list
UnicodeEncodeError when piping stdout, but not when printing directly to the console
(I'm using Python 2.7.2+ on Ubuntu.) When I'm running my program in an xterm, the print command with an argument containing unicode works fine (it correctly detects my UTF-8 environment). But when I run it with a pipe or redirect to a file (| or ), unicode strings fail with the following (for example): UnicodeEncodeError: 'ascii' codec can't encode character u'\u0107' in position 21: ordinal not in range(128) How can I force python (preferably within my python program, rather than having to set something externally) to treat stdout as UTF-8? Thanks, Adam -- Nam Sibbyllam quidem Cumis ego ipse oculis meis vidi in ampulla pendere, et cum illi pueri dicerent: beable beable beable; respondebat illa: doidy doidy doidy. [plorkwort] -- http://mail.python.org/mailman/listinfo/python-list
Re: UnicodeEncodeError when piping stdout, but not when printing directly to the console
Adam Funk wrote: (I'm using Python 2.7.2+ on Ubuntu.) When I'm running my program in an xterm, the print command with an argument containing unicode works fine (it correctly detects my UTF-8 environment). But when I run it with a pipe or redirect to a file (| or ), unicode strings fail with the following (for example): UnicodeEncodeError: 'ascii' codec can't encode character u'\u0107' in position 21: ordinal not in range(128) How can I force python (preferably within my python program, rather than having to set something externally) to treat stdout as UTF-8? $ cat force_utf8.py # -*- coding: utf-8 -*- import sys if sys.stdout.encoding is None: import codecs writer = codecs.getwriter(utf-8) sys.stdout = writer(sys.stdout) print uÄhnlich üblich nötig $ python force_utf8.py Ähnlich üblich nötig $ python force_utf8.py | cat Ähnlich üblich nötig -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
On Wed, 04 Jan 2012 20:48:36 +, PiLS wrote: If I nuke a Karmic Koala, will they rat me out to the WWF, to the UNODA, or to both? Personally I'd be cheering for you, provided you also took out all the warthogs, hedgehogs, badgers, drakes, efts, fawns, gibbons, herons, ibexes, jackalopes, lynxes, meerkats, narwhals, ocelots and pangolins. -- http://mail.python.org/mailman/listinfo/python-list
Re: help me get excited about python 3
On 1/4/2012 9:56 AM, Sean Wolfe wrote: I am still living in the 2.x world because all the things I want to do right now in python are in 2 (django, pygame). But I want to be excited about the future of the language. I understand the concept of needing to break backwards compatibility. But it's not particularly exciting to think about. What are the cool new bits I should be reading up on? All the cool new language features, of course. I've only started learning Python, so I haven't watched the what's new stuff for older versions (and therefore can't pull anything off the top of my head), but I am pretty excited for 3.3. The new exceptions, the LZMA module, support for cp65001 in Windows... http://docs.python.org/release/3.0.1/whatsnew/3.0.html http://docs.python.org/release/3.1.3/whatsnew/3.1.html http://docs.python.org/py3k/whatsnew/3.2.html http://docs.python.org/dev/whatsnew/3.3.html -- CPython 3.2.2 | Windows NT 6.1.7601.17640 -- http://mail.python.org/mailman/listinfo/python-list
Expert Advice
Hi Everyone, A number of friends in the community recommended i email this group regarding some obstacles im running into regarding a Python/Django dilemma :). Im currently representing a e-Plushing firm who has built an amazing custom ebook publishing platform in Django and Python. We are now looking for a Python engineer who has dabbled in NoSQL databases. This person will understand programming languages both static and dynamic typing. They offer a great work environment as they have a nice loft in Midtown in addition to challenging/custom work. Below i have included additional details if you know of anyone that might be interested. Ps. We have a great referral program as well. Our ebook publishing platform is built on Django and LAMP (for P in ['Python']) - and the demand is growing! We're looking for a senior software engineer who's passionate about building and shipping products that are great inside and out. As a senior team member, you'll ensure our product enjoys high-availability, scalability, and the other expected non-functional requirements through peer review and leveraging cloud infrastructure. You'll also support our journey towards continuous delivery inclusive of iterative development, automated testing (CI), modern configuration management using chef, and push-button deployments with confidence. About you: You love technology and good engineering as much as shipping great products - and deftly balance these concerns. You're a Python expert and code for fun. You've probably dabbled in NoSQL databases, know multiple programming languages both static and dynamic typing (and pros/cons of each), have tried a variety of web servers and frameworks, understand the power and challenges of different caching strategies, are a go-to for CS fundamentals, and contribute to open source projects. You constructively initiate improvements. You also don't shy away from Linux systems operations where it butts up against development concerns such as redundancy with auto-failover, efficient resource utilization, performance optimization, data management, apache configuration, application monitoring, log file aggregation (e.g., using scribe), security, capacity and scaling concerns - i.e., approaching devops from the development side. You enjoy being a hands-on individual contributor and also have great skills mentoring peers/colleagues - a team player who's well respected due to your technical abilities and willingness to help other team members without the ego. You thrive in a fast-paced yet thoughtful startup environment. Requirements 8+ years in software development in Linux - years of experience are necessary to design robust architectures Python expertise required - (open source) samples will be requested Experience building and maintaining distributed, scale-out, high performant web-based systems on Linux required Have solid CS and OO fundamentals Strong and effective communication skills including with remote team members Have created and consumed RESTful APIs Cloud experience with AWS or comparable required Iterative (agile) development experience required Must have a passion for automated testing Python web framework and MVC pattern usage required; Django a plus; built your own web framework a major plus Push-button deployment experience a plus Experience with Celery and RabbitMQ a plus Knowledge of ePub and HTML5 standards a plus Perks Fun coworkers, great mentors Convenient mid-town Manhattan loft Lots of fun with Apple products and ereaders luis perez | principal winter wyman companies - technology staffing tel: 212.616.3582| fax: 212.616.3592 | aim: lpcnn23 I Please consider the environment before printing this email -- http://mail.python.org/mailman/listinfo/python-list
can a subclass method determine if called by superclass?
Situation: I am subclassing a class which has methods that call other class methods (and without reading the code of the superclass I am discovering these by trial and error as I build the subclass - this is probably why I may have approached the problem from the wrong viewpoint :-)). Problem: when overriding one of these indirectly called superclass methods I would like to take differing actions (in the subclass instance) depending on whether it is the superclass or the subclass instance performing the call. Question: Is there any way to determine in a method whether it is being called by the superclass or by a method of the subclass instance? Now I suspect that what I am doing is actually very muddy thinking :-) and I don't want to attempt to explain why I am approaching the design this way as an explanation would require too much work - I will consider an alternative inheritance approach while waiting an answer, but the answer to the question interested me (even if I do a redesign and come up with a more elegant approach to the problem). Thanks Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: can a subclass method determine if called by superclass?
On Wed, Jan 4, 2012 at 3:42 PM, Peter peter.milli...@gmail.com wrote: Situation: I am subclassing a class which has methods that call other class methods (and without reading the code of the superclass I am discovering these by trial and error as I build the subclass - this is probably why I may have approached the problem from the wrong viewpoint :-)). Problem: when overriding one of these indirectly called superclass methods I would like to take differing actions (in the subclass instance) depending on whether it is the superclass or the subclass instance performing the call. Question: Is there any way to determine in a method whether it is being called by the superclass or by a method of the subclass instance? Well, you could get the previous stack level using traceback.extract_stack() and check the filename. But it sounds like what you actually have are two different methods -- one that is used by the superclass, and one that only the subclass knows about and uses. Why not implement it as such? -- http://mail.python.org/mailman/listinfo/python-list
Re: Spamming PyPI with stupid packages
On Tuesday 2012 January 03 17:28, Steven D'Aprano wrote: Besides, I find it hard to believe that the search facilities on PyPI are so bad that there would be any searches that come up with girlfriend.py or car.py as false positives. Try an author search for D'Aprano. -- I have seen the future and I am not in it. -- http://mail.python.org/mailman/listinfo/python-list
a little help
Hi, I am new using Python, although I have experience using other programming languages like Pascal, FORTRAN, C, Prolog, etc. I am using IDLE Editor for Python in coordination with the command line interface. My situation is the following: I am developing some code. I use the IDLE Editor to write it down. Then, I save it and import it from the command line interface, so it is already available from the prompt. Then I load (read) some data from files using that code. Let suppose that after that I make some changes in the code using again the IDLE Editor, save the program code, and…what else? The updated code is not already available from the command line interface. If I run the module, I lose the data already loaded (and it is a big amount). If I re-import it, the new code is not available Any suggestion? Thanks Andres -- http://mail.python.org/mailman/listinfo/python-list
Re: a little help
On Thu, Jan 5, 2012 at 10:25 AM, Andres Soto soto_and...@yahoo.com wrote: My situation is the following: I am developing some code. I use the IDLE Editor to write it down. Then, I save it and import it from the command line interface, so it is already available from the prompt. Then I load (read) some data from files using that code. Let suppose that after that I make some changes in the code using again the IDLE Editor, save the program code, and…what else? The updated code is not already available from the command line interface. If I run the module, I lose the data already loaded (and it is a big amount). If I re-import it, the new code is not available Re-importing modules is a bit messy. The usual way to do this sort of thing would be to run the program directly from the command line, and terminate it when you're done. Is there a particular reason for wanting to import it that way? Chris Angelico -- http://mail.python.org/mailman/listinfo/python-list
Re: can a subclass method determine if called by superclass?
On Jan 5, 10:09 am, Ian Kelly ian.g.ke...@gmail.com wrote: Well, you could get the previous stack level using traceback.extract_stack() and check the filename. But it sounds like what you actually have are two different methods -- one that is used by the superclass, and one that only the subclass knows about and uses. Why not implement it as such? Thanks Ian - that is one possibility. I am trying to create a subclass with slightly different functionality and use it with an existing code base i.e. there is already one or more modules that instantiate the current superclass and I want to just drop in this new class to replace it with no ripples up the line (so to speak). The new class implements some interface changes that can safely be hidden from the rest of the application. -- http://mail.python.org/mailman/listinfo/python-list
Re: help me get excited about python 3
On Jan 4, 9:56 am, Sean Wolfe ether@gmail.com wrote: I am still living in the 2.x world because all the things I want to do right now in python are in 2 (django, pygame). But I want to be excited about the future of the language. Okay. So why not enjoy the best of both worlds (almost) and use version 2.7.2 along with from __future__ import EVERYTHING. Your best course of action would be to write all code that can be written in Python 3000 IN Python 3000. If you must use Python 2.x because of dependencies, well then, you must. -- http://mail.python.org/mailman/listinfo/python-list
Re: a little help
I think you meant to send that to the list; hope you don't mind my replying on-list. On Thu, Jan 5, 2012 at 10:56 AM, Andres Soto soto_and...@yahoo.com wrote: the problem is that if I re-run the program, every time I change some instructions, I have to read (load) again the data and that is what I want to avoid. Is it possible? That's normal with Python, yes. Usually you'll find that it's more hassle than it's worth to try to modify code live like that; even in languages specifically designed with this feature in mind, there's a lot to keep track of. It may be worth writing your program to take a snapshot of current state (eg with the pickle module); this might be easier than re-parsing a complicated input data set. But that can be a lot of bother too, and usually in the end it's just not worthwhile. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: .format vs. %
On Jan 4, 6:25 pm, 8 Dihedral dihedral88...@googlemail.com wrote: And what are you contributing to the situation other than misinformation and markov-generated spam? Do you know what can attract newbies to support python? I'm sure other people doing all the work for them would be a great attractor. Are you volunteering to work on this code generator? -- http://mail.python.org/mailman/listinfo/python-list
Re: a little help
On Thu, Jan 5, 2012 at 11:29 AM, Andres Soto soto_and...@yahoo.com wrote: my mistake is because I have no problem to do that using Prolog which use an interpreter as Python. I thought that the variables in the main global memory space (associated with the command line environment) were kept, although the code that use it could change. As you explain me, Python behave like a compiled language: any time I make a change in the code, I have to compile it again, and re-run (and re-load the data). There is nothing to do. Yeah. There are ways around it (there's a 'reload' function in Python 2, or 'imp.reload' in Python 3), but it's usually simpler and safer to fully restart. ChrisA -- http://mail.python.org/mailman/listinfo/python-list
Re: a little help
my mistake is because I have no problem to do that using Prolog which use an interpreter as Python. I thought that the variables in the main global memory space (associated with the command line environment) were kept, although the code that use it could change. As you explain me, Python behave like a compiled language: any time I make a change in the code, I have to compile it again, and re-run (and re-load the data). There is nothing to do. Thank you! Regards Prof. Dr. Andrés Soto DES DACI UNACAR From: Chris Angelico ros...@gmail.com To: python-list@python.org Sent: Wednesday, January 4, 2012 6:02 PM Subject: Re: a little help I think you meant to send that to the list; hope you don't mind my replying on-list. On Thu, Jan 5, 2012 at 10:56 AM, Andres Soto soto_and...@yahoo.com wrote: the problem is that if I re-run the program, every time I change some instructions, I have to read (load) again the data and that is what I want to avoid. Is it possible? That's normal with Python, yes. Usually you'll find that it's more hassle than it's worth to try to modify code live like that; even in languages specifically designed with this feature in mind, there's a lot to keep track of. It may be worth writing your program to take a snapshot of current state (eg with the pickle module); this might be easier than re-parsing a complicated input data set. But that can be a lot of bother too, and usually in the end it's just not worthwhile. ChrisA -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Trouble getting Python 2.7.2 to recognize Tk in Scientific Linux Release 6
Hi, I've been a python user for a long time - on Windows, but now I'm working on a Linux system. I'm having trouble getting python to include Tk in it's build. My Tcl/Tk is in a non-standard location (I don't want to interfere with the Python 2.6 installation - that does include Tk until I have 2.7.2 running fully. I've read everything I can find about installing python in Linux, but to no avail. Can someone give me some advice on which file(s) to edit, configure arguments to add, etc.? Thanks! ...Jeff -- http://mail.python.org/mailman/listinfo/python-list
Re: .format vs. %
alex23於 2012年1月5日星期四UTC+8上午8時23分06秒寫道： On Jan 4, 6:25 pm, 8 Dihedral dihedr...@googlemail.com wrote: And what are you contributing to the situation other than misinformation and markov-generated spam? Do you know what can attract newbies to support python? I'm sure other people doing all the work for them would be a great attractor. Are you volunteering to work on this code generator? I am working on the code generation problem in other applications. I could contribute some opinions about this problem. I think the BOA project is in the right direction. -- http://mail.python.org/mailman/listinfo/python-list
Re: can a subclass method determine if called by superclass?
On Wed, 04 Jan 2012 15:37:55 -0800, Peter wrote: I am trying to create a subclass with slightly different functionality and use it with an existing code base i.e. there is already one or more modules that instantiate the current superclass and I want to just drop in this new class to replace it with no ripples up the line (so to speak). The new class implements some interface changes that can safely be hidden from the rest of the application. This is *exactly* the idea behind subclassing. I don't understand your problem, can you explain more? If you want to change behaviour of an object, you subclass it, then override or overload the methods you want to change. You certainly shouldn't be changing the superclass to recognise when it is being called from a subclass! That's completely the wrong approach -- you should put all the new behaviour in the new class. # WRONG! Don't do this. class Parent(object): def method(self, arg): if type(self) is not Parent: # Must be a subclass. print(Called from a subclass. But which one?) print(Doing method stuff.) class Child(Parent): pass # RIGHT! Do this instead. class Parent(object): def method(self, arg): print(Doing method stuff.) class Child(Parent): def method(self, arg): # Overload an existing method. print(Called from Child subclass.) super().method(arg) # Python 3 only # Python 2 use: super(Child, self).method(arg) If method() returns a result, you can capture the result of calling the superclass method and then modify it as needed. Or you can override the method completely, and not call the parent method() at all. Now you can use Child() anywhere that you can use Parent and the caller shouldn't even notice. Or at least that is the idea behind subclassing, although it is possible to break it. You will be okay if the caller uses duck-typing, or isinstance checks, but not if they do exact type checks (which is almost always the wrong thing to do). c = Child() # Duck-typing works: hasattr(c, 'method') # returns True # So do isinstance checks: isinstance(c, Parent) # returns True # but this doesn't and defeats the purpose of having subclasses: type(c) is Parent # returns False, you shouldn't do this! If the caller does do exact type checks, then almost certainly it should be considered a bug in their code and they should be beaten with a clue- bat and told to use isinstance (good) or duck-typing (better still). If I have misunderstood your problem, can you give a concrete (but simple) example of what you are trying to do? -- Steven -- http://mail.python.org/mailman/listinfo/python-list
Re: help me get excited about python 3
On 1/4/2012 9:56 AM, Sean Wolfe wrote: I am still living in the 2.x world because all the things I want to do right now in python are in 2 (django, pygame). But I want to be excited about the future of the language. I understand the concept of needing to break backwards compatibility. But it's not particularly exciting to think about. What are the cool new bits I should be reading up on? This should be enough to convince you: ~ : python Python 2.7.1 (r271:86832, May 3 2011, 10:31:28) 1 1 True ~ : python3 Python 3.2 (r32:88445, May 3 2011, 13:26:55) 1 1 Traceback (most recent call last): File stdin, line 1, in module TypeError: unorderable types: int() str() Maybe with Python 4, '1 True' will give a TypeError too ;-). Or if that's not enough, ~ : python Python 2.7.1 (r271:86832, May 3 2011, 10:31:28) True, False = False, True True is + (True if True else False) 'True is False' ~ : python3 Python 3.2 (r32:88445, May 3 2011, 13:26:55) True, False = False, True File stdin, line 1 SyntaxError: assignment to keyword Evan -- http://mail.python.org/mailman/listinfo/python-list
Re: a little help
On 1/4/2012 7:29 PM, Andres Soto wrote: As you explain me, Python behave like a compiled language: any time I make a change in the code, I have to compile it again, and re-run (and re-load the data). While you are developing a program and expect to make changes, you can try working with a small subset of a large dataset, so reloading is not too painful. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Typed python comparison / code analysis questions
On 1/4/2012 3:42 PM, Lucas Vickers wrote: At the moment python3 isn't an option. There's a variety of dependencies I'm working around. Please consider telling the authors of libraries you need that you would like a Python 3 version and say why. One reason given for not upgrading packages is 'lack of demand'. -- Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
[issue13707] Clarify hash() constancy period
Martin v. Löwis mar...@v.loewis.de added the comment: Martin, I do not understand. The default hash is based on id (as is default equality comparison), not value. In the default implementation, the id *is* the object's value (i.e. objects, by default, only compare equal if they are identical). So the default implementation is just a special case of the more general rule that hashes need to be consistent with equality. Are you OK with hash values changing if the 'value' changes? An object that can change its value (i.e. a mutable object) should fail to hash. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13707 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13707] Clarify hash() constancy period
Marc-Andre Lemburg m...@egenix.com added the comment: Terry J. Reedy wrote: Terry J. Reedy tjre...@udel.edu added the comment: Martin, I do not understand. The default hash is based on id (as is default equality comparison), not value. Are you OK with hash values changing if the 'value' changes? My understanding is that changing hash values for objects in sets and dicts is bad, which is why mutable builtins with value-based equality do not have hash values. Hash values are based on the object values, not their id(). See the various type implementations as reference. The id() is only used as hash for objects which don't have a value (and thus cannot be compared). Given that we have the invariant a==b = hash(a)==hash(b) in Python, it immediately follows that hash values for objects with comparison method cannot have a lifetime - at least not within the same process and, depending how you look at it, also not in multi-process applications. -- nosy: +lemburg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13707 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue8416] python 2.6.5 documentation can't search
Georg Brandl ge...@python.org added the comment: The continually updated docs are built from the stable branches, whose version remains at (e.g.) 2.7.2 until 2.7.3a1 is released, at which point the continuous updating stops until 2.7.3 is final. I don't think presenting docs with an alpha version on the http://docs.python.org/ frontpage is useful. On the other hand, I do think it is important to have doc fixed reflected (more or less) instantly somewhere, so that e.g. people reporting typos can see the fixes. The status quo is a compromise between these two needs. When we do make backwards incompatible changes or additions during a stable cycle, they need to be marked with new/changed in version 2.7.X+1 anyway. So the SequenceMatcher change would alert users itself. If not, that's a bug. About the obsolete snapshots: I don't know what you're referring to there: if it's the released docs for specific versions, then I think that's standard practice to have a doc version released for a specific Python version; and I wouldn't change that. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue8416 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13704] Random number generator in Python core
Christian Heimes li...@cheimes.de added the comment: Release blocker: I was following the example in #13703. A RNG (PRNG or CSPRNG) is required for randomized hashing function. The patch contains more than just the RNG changes. Only Include/pyrandom.h, Modules/_randommodule.c, Modules/posixmodule.c, Python/hash.c and parts of Makefile.pre.in are relevant for this tracker item. Sorry for the inconvenience! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13704 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13704] Random number generator in Python core
Changes by Raymond Hettinger raymond.hettin...@gmail.com: -- assignee: rhettinger - christian.heimes ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13704 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13707] Clarify hash() constancy period
Raymond Hettinger raymond.hettin...@gmail.com added the comment: [Antoine] Suggest closing as invalid/rajected. [Martin] -1. The hash has nothing to do with the lifetime, but with the value of an object. -- resolution: - invalid status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13707 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Changes by Mark Shannon m...@hotpy.org: -- nosy: +Mark.Shannon ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13697] python RLock implementation unsafe with signals
Antoine Pitrou pit...@free.fr added the comment: That sounds like a good solution in the middle-term. Are there any drawbacks? (apart from launching a thread) Just to be clear: the approach I was suggesting is to have a resident thread dedicated to signal management, not to spawn a new one when needed. Another advantage is that we could mask signals in all threads except this one, and have a consistent cross-platform behavior with respect to signals+threads. Hmm, but that would break single-threaded programs which expect their select() (or other) to return EINTR when a signal is received (which is a perfectly valid expectation in that case). However I see two drawbacks: - it seems that we want to allow building Python without threads support. In that case, this wouldn't work, or we would need the current implementation as a fallback, but this would complicate the code somewhat. I don't know if that's still useful to build Python without threads. I would expect most platforms to have a compatible threads implementation (and Python probably can't run on very small embedded platforms). Perhaps you can ask on python-dev. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13697 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Antoine Pitrou pit...@free.fr added the comment: Using a fairly small value (4k) should not make the results much worse from a security perspective, but might be problematic from a collision/distribution standpoint. Keep in mind the average L1 data cache size is between 16KB and 64KB. 4KB is already a significant chunk of that. Given a hash function's typical loop is to feed back the current result into the next computation, I don't see why a small value (e.g. 256 bytes) would be detrimental. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13699] test_gdb has recently started failing
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset dfffb293f4b3 by Vinay Sajip in branch 'default': Closes #13699. Skipped two tests if Python is optimised. http://hg.python.org/cpython/rev/dfffb293f4b3 -- nosy: +python-dev resolution: - fixed stage: - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13699 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13711] html.parser.HTMLParser doesn't parse tags in comments in scripts correctly
New submission from Manuel Bärenz man...@enigmage.de: I've attached a script which demonstrates the bug. When feeding a script that contains a comment tag with the actual script and the script containing tags itself (e.g. a 'document.write(td/td)'), the parser doesn't call handle_comment and handle_starttag. -- components: Library (Lib) files: htmlparserbug.py messages: 150603 nosy: turion priority: normal severity: normal status: open title: html.parser.HTMLParser doesn't parse tags in comments in scripts correctly type: behavior versions: Python 3.2 Added file: http://bugs.python.org/file24137/htmlparserbug.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13711] html.parser.HTMLParser doesn't parse tags in comments in scripts correctly
Manuel Bärenz man...@enigmage.de added the comment: I forgot to say, I'm using python version 3.2.2. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13711] html.parser.HTMLParser doesn't parse tags in comments in scripts correctly
R. David Murray rdmur...@bitdance.com added the comment: The content of a script tag is CDATA. Why would you expect it to be parsed? -- nosy: +ezio.melotti, r.david.murray ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13711] html.parser.HTMLParser doesn't parse tags in comments in scripts correctly
Manuel Bärenz man...@enigmage.de added the comment: Oh, I wasn't aware of that. Then, the bug is actually calling handle_endtag. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13711] html.parser.HTMLParser doesn't parse tags in comments in scripts correctly
Manuel Bärenz man...@enigmage.de added the comment: To clarify this even further: Consider parser_instance.feed(scripttd/td/script) It should call: parser_instance.handle_starttag(script, []) parser_instance.handle_data(td/td) parser_instance.handle_endtag(script, []) Instead, it calls: parser_instance.handle_starttag(script, []) parser_instance.handle_data(td) parser_instance.handle_endtag(td, []) parser_instance.handle_endtag(script, []) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13711] html.parser.HTMLParser doesn't parse tags in comments in scripts correctly
R. David Murray rdmur...@bitdance.com added the comment: I believe this was fixed recently as part of issue 670664. Ezio will know for sure. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Éric Araujo mer...@netwok.org added the comment: If test_packaging fails because it relies on dict order / hash details, that’s a bug. Can you copy the full tb (possibly in another report, I can fix it independently of this issue)? -- nosy: +eric.araujo ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13712] test_packaging depends on hash order
New submission from Christian Heimes li...@cheimes.de:
As requested in http://bugs.python.org/issue13703#msg150609
./python Lib/test/regrtest.py test_packaging
[1/1] test_packaging
Warning -- threading._dangling was modified by test_packaging
Warning -- sysconfig._SCHEMES was modified by test_packaging
test test_packaging failed -- Traceback (most recent call last):
File /home/heimes/dev/python/randomhash/Lib/packaging/tests/test_create.py,
line 168, in test_convert_setup_py_to_cfg
))
AssertionError: '[metadata]\nname = pyxfoil\nversion = 0.2\nsummary = Python
bindings for the Xf [truncated]... != '[metadata]\nname = pyxfoil\nversion =
0.2\nsummary = Python bindings for the Xf [truncated]...
[metadata]
name = pyxfoil
version = 0.2
summary = Python bindings for the Xfoil engine
download_url = UNKNOWN
home_page = http://www.python-science.org/project/pyxfoil
maintainer = André Espaze
maintainer_email = andre.esp...@logilab.fr
description = My super Death-scription
|barbar is now on the public domain,
|ho, baby !
[files]
packages = pyxfoil
babar
me
modules = my_lib
mymodule
scripts = my_script
bin/run
- extra_files = setup.py
+ extra_files = Martinique/Lamentin/dady
+ Martinique/Lamentin/mumy
+ Martinique/Lamentin/sys
+ Martinique/Lamentin/bro
+ setup.py
README
- pyxfoil/fengine.so
Pom
Flora
Alexander
+ pyxfoil/fengine.so
- Martinique/Lamentin/dady
- Martinique/Lamentin/mumy
- Martinique/Lamentin/sys
- Martinique/Lamentin/bro
resources =
README.rst = {doc}
pyxfoil.1 = {man}
1 test failed:
test_packaging
--
assignee: eric.araujo
components: Distutils2
messages: 150610
nosy: alexis, christian.heimes, eric.araujo
priority: normal
severity: normal
status: open
title: test_packaging depends on hash order
type: behavior
versions: Python 3.3
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13712
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13711] html.parser.HTMLParser doesn't parse tags in comments in scripts correctly
Ezio Melotti ezio.melo...@gmail.com added the comment:
Yep, this was fixed in #670664.
With the development version of Python (AFAIK the fix has not be released yet)
and the example parser found in the doc[0] I get this:
parser = MyHTMLParser()
parser.feed('scripttd/td/script')
Encountered a start tag: script
Encountered some data: td/td
Encountered an end tag: script
[0]:
http://docs.python.org/dev/library/html.parser.html#example-html-parser-application
--
assignee: - ezio.melotti
resolution: - duplicate
stage: - committed/rejected
status: open - closed
superseder: - HTMLParser.py - more robust SCRIPT tag parsing
___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue13711
___
___
Python-bugs-list mailing list
Unsubscribe:
http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13704] Random number generator in Python core
Barry A. Warsaw ba...@python.org added the comment: On Jan 04, 2012, at 07:30 AM, Raymond Hettinger wrote: Why is this listed as a release blocker? It is questionable whether it should be done at all? It is a very aggressive change. It's a release blocker so that the issue won't get ignored before the next release. That doesn't necessarily mean it must be fixed. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13704 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Barry A. Warsaw ba...@python.org added the comment: On Jan 04, 2012, at 06:00 AM, Paul McMillan wrote: Developers would be startled to find that ordering stays consistent on a 64 bit build but varies on 32 bit builds. Well, one positive outcome of this issue is that users will finally viscerally understand that dictionary (and set) order should never be relied upon, even between successive runs of the same Python executable. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13711] html.parser.HTMLParser doesn't parse tags in comments in scripts correctly
Manuel Bärenz man...@enigmage.de added the comment: Great! Thank you! -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13711 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13713] Regression for http.client read()
New submission from Ross Lagerwall rosslagerw...@gmail.com: 806cfe39f729 introduced a regression for http.client read(len). To see this: $ ./python test.py $ wget http://archives.fedoraproject.org/pub/archive/fedora/linux/core/1/SRPMS/ $ diff index.html index2.html This is a difference in the files (which there shouldn't be). The change which introduced the problem was: changeset: 73875:806cfe39f729 user:Antoine Pitrou solip...@pitrou.net date:Tue Dec 06 22:33:57 2011 +0100 summary: Issue #13464: Add a readinto() method to http.client.HTTPResponse. -- components: Library (Lib) files: test.py messages: 150615 nosy: orsenthil, pitrou, rosslagerwall priority: normal severity: normal stage: needs patch status: open title: Regression for http.client read() type: behavior versions: Python 3.3 Added file: http://bugs.python.org/file24138/test.py ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13713 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment: Some comments: 1. The security implications in all this is being somewhat overemphasized. There are many ways you can do a DoS attack on web servers. It's the responsibility of the used web frameworks and servers to deal with the possible cases. It's a good idea to provide some way to protect against hash collision attacks, but that will only solve one possible way of causing a resource attack on a server. There are other ways you can generate lots of CPU overhead with little data input (e.g. think of targeting the search feature on many Zope/Plone sites). In order to protect against such attacks in general, we'd have to provide a way to control CPU time and e.g. raise an exception if too much time is being spent on a simple operation such as a key insertion. This can be done using timers, signals or even under OS control. The easiest way to protect against the hash collision attack is by limiting the POST/GET/HEAD request size. The second best way would be to limit the number of parameters that a web framework accepts for POST/GET/HEAD request. 2. Changing the semantics of hashing in a dot release is not allowed. If randomization of the hash start vector or some other method is enabled by default in a dot release, this will change the semantics of any application switching to that dot release. The hash values of Python objects are not only used by the Python dictionary implementation, but also by other storage mechanisms such as on-disk dictionaries, inter-process object exchange via share memory, memcache, etc. Hence, if changed, the hash change should be disabled per default for dot releases and enabled for 3.3. 3. Changing the way strings are hashed doesn't solve the problem. Hash values of other types can easily be guessed as well, e.g. take integers which use a trivial hash function. We'd have to adapt all hash functions of the basic types in Python or come up with a generic solution using e.g. double-hashing in the dictionary/set implementations. 4. By just using a random start vector you change the absolute hash values for specific objects, but not the overall hash sequence or its period. An attacker only needs to create many hash collisions, not specific ones. It's the period of the hash function that's important in such attacks and that doesn't change when moving to a different start vector. 5. Hashing needs to be fast. It's one of the most used operations in Python. Please get experts into the boat like Tim Peters and Christian Tismer, who both have worked on the dict implementation and the hash functions, before experimenting with ad-hoc fixes. 6. Counting collisions could solve the issue without having to change hashing. Another idea would be counting the collisions and raising an exception if the number of collisions exceed a certain threshold. Such a change would work for all hashable Python objects and protect against the attack without changing any hash function. Thanks, -- Marc-Andre Lemburg eGenix.com ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ -- nosy: +lemburg ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue12660] test_gdb fails when installed
Vinay Sajip vinay_sa...@yahoo.co.uk added the comment: Pending the real fix, I've attached a patch to skip the test if it's not a source build. -- keywords: +patch nosy: +vinay.sajip stage: needs patch - patch review Added file: http://bugs.python.org/file24139/test-gdb-patch.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue12660 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13712] test_packaging depends on hash order
Éric Araujo mer...@netwok.org added the comment: Thanks, I will check this. -- versions: +3rd party ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13712 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13713] Regression for http.client read()
Changes by Antoine Pitrou pit...@free.fr: -- nosy: +Jon.Kuhn priority: normal - critical ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13713 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment: Marc-Andre Lemburg wrote: 3. Changing the way strings are hashed doesn't solve the problem. Hash values of other types can easily be guessed as well, e.g. take integers which use a trivial hash function. Here's an example for integers on a 64-bit machine: g = ((x*(2**64 - 1), hash(x*(2**64 - 1))) for x in xrange(1, 100)) d = dict(g) This takes ages to complete and only uses very little memory. The input data has some 32MB if written down in decimal numbers - not all that much data either. 32397634 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment: The email interface ate part of my reply: g = ((x*(2**64 - 1), hash(x*(2**64 - 1))) for x in xrange(1, 100)) s = ''.join(str(x) for x in g) len(s) 32397634 g = ((x*(2**64 - 1), hash(x*(2**64 - 1))) for x in xrange(1, 100)) d = dict(g) ... lots of time for coffee, pizza, taking a walk, etc. :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Terry J. Reedy tjre...@udel.edu added the comment: To expand on Marc-Andre's point 1: the DOS attack on web servers is possible because servers are generally dumb at the first stage. Upon receiving a post request, all key=value pairs are mindlessly packaged into a hash table that is then passed on to a page handler that typically ignores the invalid keys. However, most pages do not need any key,value pairs and forms that do have a pre-defined set of expected and recognized keys. If there were a possibly empty set of keys associated with each page, and the set were checked against posted keys, then a DOS post with thousands of effectively random keys could quickly (in O(1) time) be rejected as erroneous. In Python, the same effect could be accomplished by associating a class with slots with each page and having the server create an instance of the class. Attempts to create an undefined attribute would then raise an exception. Either way, checking input data for face validity before processing it in a time-consuming way is one possible solution for nearly all web pages and at least some other applications. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Alex Gaynor alex.gay...@gmail.com added the comment: Except, it's a totally non-scalable approach. People have vulnerabilities all over their sites which they don't realize. Some examples: django-taggit (an application I wrote for handling tags) parses tags out an input, it stores these in a set to check for duplicates. It's vulnerable. Another site I'm writing accepts JSON POSTs, you can put arbitrary keys in the JSON. It's vulnerable. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13713] Regression for http.client read()
Antoine Pitrou pit...@free.fr added the comment: The fix is quite trivial. Here is a patch + tests. -- keywords: +patch stage: needs patch - patch review Added file: http://bugs.python.org/file24140/readinto_chunked.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13713 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13713] Regression for http.client read()
Ross Lagerwall rosslagerw...@gmail.com added the comment: The patch looks right and seems to fix the issue. Thanks :-) -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13713 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Marc-Andre Lemburg m...@egenix.com added the comment: Marc-Andre Lemburg wrote: 1. The security implications in all this is being somewhat overemphasized. There are many ways you can do a DoS attack on web servers. It's the responsibility of the used web frameworks and servers to deal with the possible cases. It's a good idea to provide some way to protect against hash collision attacks, but that will only solve one possible way of causing a resource attack on a server. There are other ways you can generate lots of CPU overhead with little data input (e.g. think of targeting the search feature on many Zope/Plone sites). In order to protect against such attacks in general, we'd have to provide a way to control CPU time and e.g. raise an exception if too much time is being spent on a simple operation such as a key insertion. This can be done using timers, signals or even under OS control. The easiest way to protect against the hash collision attack is by limiting the POST/GET/HEAD request size. For GET and HEAD, web servers normally already apply such limitations at rather low levels: http://stackoverflow.com/questions/686217/maximum-on-http-header-values So only HTTP methods which carry data in the body part of the HTTP request are effected, e.g. POST and various WebDAV methods. The second best way would be to limit the number of parameters that a web framework accepts for POST/GET/HEAD request. Depending on how parsers are implemented, applications taking XML/JSON/XML-RPC/etc. as data input may also be vulnerable, e.g. non validating XML parsers which place element attributes into a dictionary or a JSON parser that has to read the JSON version of the dict I generated earlier on. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13464] HTTPResponse is missing an implementation of readinto
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 4b21f651 by Antoine Pitrou in branch 'default': Issue #13713: fix a regression in HTTP chunked reading after 806cfe39f729 http://hg.python.org/cpython/rev/4b21f651 -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13464 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13713] Regression for http.client read()
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 4b21f651 by Antoine Pitrou in branch 'default': Issue #13713: fix a regression in HTTP chunked reading after 806cfe39f729 http://hg.python.org/cpython/rev/4b21f651 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13713 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13713] Regression for http.client read()
Antoine Pitrou pit...@free.fr added the comment: Ok, committed! (Jon, don't worry, such things happen :-)) -- resolution: - fixed stage: patch review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13713 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue7098] g formatting for decimal types should always strip trailing zeros.
Stefan Krah stefan-use...@bytereef.org added the comment: [Mark] So I think the current code is correct. I agree with this. Currently the 'g' format is like to_sci_string() with the added possibility of adjusting the number of significant digits. It's probably hard to come up with a better way to handle this for Decimal. -- nosy: +skrah ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue7098 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue11648] openlog()s 'logopt' keyword broken in syslog module
Sandro Tosi sandro.t...@gmail.com added the comment: This has already been fixed with 71f7175e2b34 friends. -- nosy: +sandro.tosi resolution: - fixed stage: - committed/rejected status: open - closed versions: -Python 3.4 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue11648 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10772] Several actions for argparse arguments missing from docs
Roundup Robot devn...@psf.upfronthosting.co.za added the comment: New changeset 278fbd7b9608 by Sandro Tosi in branch '2.7': Issue #10772: add count and help argparse action; patch by Marc Sibson http://hg.python.org/cpython/rev/278fbd7b9608 New changeset 326f755962e3 by Sandro Tosi in branch '3.2': Issue #10772: add count and help argparse action; patch by Marc Sibson http://hg.python.org/cpython/rev/326f755962e3 -- nosy: +python-dev ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10772 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue10772] Several actions for argparse arguments missing from docs
Sandro Tosi sandro.t...@gmail.com added the comment: Thanks Marc for the patch, I've just committed it. -- resolution: - fixed stage: commit review - committed/rejected status: open - closed ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue10772 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13641] decoding functions in the base64 module could accept unicode strings
Berker Peksag berker.pek...@gmail.com added the comment: Hi Antoine, I added some tests for b64decode function. Also, I wrote some tests for b32decode and b16decode functions and failed. I think my patch is not working for b32decode and b16decode functions. I'll dig into code and try to find a way. Thanks! -- Added file: http://bugs.python.org/file24141/issue13641_v2_with_tests.diff ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13641 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
STINNER Victor victor.stin...@haypocalc.com added the comment: Work-in-progress patch implementing my randomized hash function (random.patch): - add PyOS_URandom() using CryptoGen, SSL (only on VMS!!) or /dev/urandom, will a fallback on a dummy LCG if the OS urandom failed - posix.urandom() is always defined and reuses PyOS_URandom() - hash(str) is now randomized using two random Py_hash_t values: don't touch the critical loop, only add a prefix and a suffix Notes: - PyOS_URandom() reuses mostly code from Modules/posixmodule.c, except dev_urandom() and fallback_urandom() which are new - I removed memset(PyBytes_AS_STRING(result), 0, howMany); from win32_urandom() because it doesn't really change anything because the LCG is used if win32_urandom() fails - Python refuses to start if the OS urandom is missing. - Python/random.c code may be moved into Python/pythonrun.c if it is an issue to add a new file in old Python versions. - If the OS urandom fails to generate the unicode hash secret, no warning is emitted (because the LCG is used). I don't know if a warning is needed in this case. - os.urandom() argument is now a Py_ssize_t instead of an int TODO: - add an environment option to ignore the OS urandom and only uses the LCG - fix all tests broken because of the randomized hash(str) - PyOS_URandom() raises exceptions whereas it is called before creating the interpreter state. I suppose that it cannot work like this. - review and test PyOS_URandom() - review and test the new randomized hash(str) -- keywords: +patch Added file: http://bugs.python.org/file24142/random.patch ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
STINNER Victor victor.stin...@haypocalc.com added the comment: add PyOS_URandom() using CryptoGen, SSL (only on VMS!!) or /dev/urandom Oh, OpenSSL (RAND_pseudo_bytes) should be used on Windows, Linux, Mac OS X, etc. if OpenSSL is available. I was just too lazy to add a define or pyconfig.h option to indicate if OpenSSL is available or not. FYI RAND_pseudo_bytes() is now exposed in the ssl module of Python 3.3. will a fallback on a dummy LCG It's the Linear congruent generator (LCG) used by Microsoft Visual C++ and PHP: x(n+1) = (x(n) * 214013 + 2531011) % 2^32 I only use bits 23..16 (bits 15..0 are not really random). -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Antoine Pitrou pit...@free.fr added the comment: add PyOS_URandom() using CryptoGen, SSL (only on VMS!!) or /dev/urandom Oh, OpenSSL (RAND_pseudo_bytes) should be used on Windows, Linux, Mac OS X, etc. if OpenSSL is available. Apart from the large dependency, the OpenSSL license is not GPL-compatible which may be a problem for some Python-embedding applications: http://en.wikipedia.org/wiki/OpenSSL#Licensing will a fallback on a dummy LCG It's the Linear congruent generator (LCG) used by Microsoft Visual C++ and PHP: x(n+1) = (x(n) * 214013 + 2531011) % 2^32 I only use bits 23..16 (bits 15..0 are not really random). If PHP uses it, I'm confident it is secure. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
STINNER Victor victor.stin...@haypocalc.com added the comment: +printf(read %i bytes\n, size); Oops, I forgot a debug message. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
STINNER Victor victor.stin...@haypocalc.com added the comment: If PHP uses it, I'm confident it is secure. If I remember correctly, it is only used for the Windows version of PHP, but PHP doesn't implement it correctly because it uses all bits. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
Paul McMillan p...@mcmillan.ws added the comment: This is not something that can be fixed by limiting the size of POST/GET. Parsing documents (even offline) can generate these problems. I can create books that calibre (a Python-based ebook format shifting tool) can't convert, but are otherwise perfectly valid for non-python devices. If I'm allowed to insert usernames into a database and you ever retrieve those in a dict, you're vulnerable. If I can post things one at a time that eventually get parsed into a dict (like the tag example), you're vulnerable. I can generate web traffic that creates log files that are unparsable (even offline) in Python if dicts are used anywhere. Any application that accepts data from users needs to be considered. Even if the web framework has a dictionary implementation that randomizes the hashes so it's not vulnerable, the entire python standard library uses dicts all over the place. If this is a problem which must be fixed by the framework, they must reinvent every standard library function they hope to use. Any non-trivial python application which parses data needs the fix. The entire standard library needs the fix if is to be relied upon by applications which accept data. It makes sense to fix Python. Of course we must fix all the basic hashing functions in python, not just the string hash. There aren't that many. Marc-Andre: If you look at my proposed code, you'll notice that we do more than simply shift the period of the hash. It's not trivial for an attacker to create colliding hash functions without knowing the key. Since speed is a concern, I think that the proposal to avoid using the random hash for short strings is a good idea. Additionally, randomizing only some of the characters in longer strings will allow us to improve security without compromising speed significantly. I suggest that we don't randomize strings shorter than 6 characters. For longer strings, we randomize the first and last 5 characters. This means we're only adding additional work to a max of 10 rounds of the hash, and only for longer strings. Collisions with the hash from short strings should be minimal. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue13703] Hash collision security issue
STINNER Victor victor.stin...@haypocalc.com added the comment: Since speed is a concern, I think that the proposal to avoid using the random hash for short strings is a good idea. My proposition only adds two XOR to hash(str) (outside the loop on Unicode characters), so I expect a ridiculous overhead. I don't know yet how hard it is to guess the secret from hash(str) output. -- ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue13703 ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
