Re: User input masks - Access Style
On Dec 27, 6:01 pm, Tim Harig wrote:
> On 2010-12-27, flebber wrote:
>
> > Is there anyay to use input masks in python? Similar to the function
> > found in access where a users input is limited to a type, length and
> > format.
>
> > So in my case I want to ensure that numbers are saved in a basic
> > format.
> > 1) Currency so input limited to 000.00 eg 1.00, 2.50, 13.80 etc
>
> Some GUIs provide this functionality or provide callbacks for validation
> functions that can determine the validity of the input. I don't know of
> any modules that provide "formatted input" in a terminal. Most terminal
> input functions just read from stdin (in this case a buffered line)
> and output that as a string. It is easy enough to validate whether
> terminal input is in the proper.
>
> Your example time code might look like:
>
> ... import re
> ... import sys
> ...
> ... # get the input
> ... print("Please enter time in the format 'MM:SS:HH': ", end="")
> ... timeInput = input()
> ...
> ... # validate the input is in the correct format (usually this would be in
> ... # loop that continues until the user enters acceptable data)
> ... if re.match(r'''^[0-9]{2}:[0-9]{2}:[0-9]{2}$''', timeInput) == None:
> ... print("I'm sorry, your input is improperly formated.")
> ... sys.exit(1)
> ...
> ... # break the input into its componets
> ... componets = timeInput.split(":")
> ... minutes = int(componets[0])
> ... seconds = int(componets[1])
> ... microseconds = int(componets[2])
> ...
> ... # output the time
> ... print("Your time is: " + "%02d" % minutes + ":" + "%02d" % seconds + ":" +
> ... "%02d" % microseconds)
>
> Currency works the same way using validating it against:
> r'''[0-9]+\.[0-9]{2}'''
>
> > For sports times that is time duration not a system or date times
> > should I assume that I would need to calculate a user input to a
> > decimal number and then recalculate it to present it to user?
>
> I am not sure what you are trying to do or asking. Python provides time,
> date, datetime, and timedelta objects that can be used for date/time
> calculations, locale based formatting, etc. What you use, if any, will
> depend on what you are actually tring to accomplish. Your example doesn't
> really show you doing much with the time so it is difficult giving you any
> concrete recommendations.
yes you are right I should have clarified. The time is a duration over
distance, so its a speed measure. Ultimately I will need to store the
times so I may need to use something likw sqlAlchemy but I am nowehere
near the advanced but I know that most Db's mysql, postgre etc don't
support time as a duration as such and i will probably need to store
it as a decimal and convert it back for the user.
--
http://mail.python.org/mailman/listinfo/python-list
Re: User input masks - Access Style
On 2010-12-27, Tim Harig wrote:
> ... if re.match(r'''^[0-9]{2}:[0-9]{2}:[0-9]{2}$''', timeInput) == None:
[SNIP]
> Currency works the same way using validating it against:
> r'''[0-9]+\.[0-9]{2}'''
Sorry, you need to check to make sure that there are no trailing characters
as in the example above. Checking the beginning is not actually necessary
with match().
r'''^[0-9]+\.[0-9]{2}$'''
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
Steven D'Aprano wrote:
...
>
I think critics of my position have forgotten what it's like to learning
the language. One of the most valuable skills to learn is to *ignore
parts of the traceback* -- a skill that takes practice and familiarity
with the library or function in use. To those who are less familiar with
the function, it can be very difficult to determine which parts of the
traceback are relevant and which aren't. In this case, the caller has
nothing to do with _compile, and the traceback looks like it's an
internal bug in a subroutine, when in fact it is actually due to bad
input. The experienced developer learns (by trial and error, possibly) to
ignore nearly half of the error message in this case.
And it can still be some work to figure out which parts of the traceback
are relevant, even after a couple years...
...
It need not be that way. This could, in principle, be left up to the
developer of the public function to specify (somehow!) that some specific
exceptions are expected, and should be treated as coming from public()
rather than from some internal subroutine. I don't have a concrete
proposal for such, although I did suggest a work-around. I expected
disinterest ("I don't see the point"). I didn't expect the level of
hostility to the idea that exceptions should (if and when possible) point
to the source of the error rather than some accidental implementation-
specific subroutine. Go figure.
My objection is not to the idea, but to the ad-hoc methods that would
currently be required. Resorting to passing exceptions in-band is a
step backwards. If python had a way to specify what level an exception
should be reported from, I might be interested. At this point, if
sparing the user one level of traceback was that high a priority to me,
I would make the validation be either a decorator, or have the
validation *be* the main routine, and the *real work* routine be the
private one.
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
Re: User input masks - Access Style
On 2010-12-27, flebber wrote:
> Is there anyay to use input masks in python? Similar to the function
> found in access where a users input is limited to a type, length and
> format.
>
> So in my case I want to ensure that numbers are saved in a basic
> format.
> 1) Currency so input limited to 000.00 eg 1.00, 2.50, 13.80 etc
Some GUIs provide this functionality or provide callbacks for validation
functions that can determine the validity of the input. I don't know of
any modules that provide "formatted input" in a terminal. Most terminal
input functions just read from stdin (in this case a buffered line)
and output that as a string. It is easy enough to validate whether
terminal input is in the proper.
Your example time code might look like:
... import re
... import sys
...
... # get the input
... print("Please enter time in the format 'MM:SS:HH': ", end="")
... timeInput = input()
...
... # validate the input is in the correct format (usually this would be in
... # loop that continues until the user enters acceptable data)
... if re.match(r'''^[0-9]{2}:[0-9]{2}:[0-9]{2}$''', timeInput) == None:
... print("I'm sorry, your input is improperly formated.")
... sys.exit(1)
...
... # break the input into its componets
... componets = timeInput.split(":")
... minutes = int(componets[0])
... seconds = int(componets[1])
... microseconds = int(componets[2])
...
... # output the time
... print("Your time is: " + "%02d" % minutes + ":" + "%02d" % seconds + ":" +
... "%02d" % microseconds)
Currency works the same way using validating it against:
r'''[0-9]+\.[0-9]{2}'''
> For sports times that is time duration not a system or date times
> should I assume that I would need to calculate a user input to a
> decimal number and then recalculate it to present it to user?
I am not sure what you are trying to do or asking. Python provides time,
date, datetime, and timedelta objects that can be used for date/time
calculations, locale based formatting, etc. What you use, if any, will
depend on what you are actually tring to accomplish. Your example doesn't
really show you doing much with the time so it is difficult giving you any
concrete recommendations.
--
http://mail.python.org/mailman/listinfo/python-list
Re: string u'hyv\xe4' to file as 'hyvä'
"gintare" wrote in message
news:83dc3076-9ddc-42bd-8c33-6af96b263...@l32g2000yqc.googlegroups.com...
Hello,
STILL do not work. WHAT to be done.
import codecs
item=u'hyv\xe4'
F=codecs.open('/opt/finnish.txt', 'w+', 'utf8')
F.writelines(item.encode('utf8'))
F.close()
In file i find 'hyv\xe4' instead of hyvä.
When you open a file with codecs.open(), it expects Unicode strings to be
written to the file. Don't encode them again. Also, .writelines() expects
a list of strings. Use .write():
import codecs
item=u'hyv\xe4'
F=codecs.open('/opt/finnish.txt', 'w+', 'utf8')
F.write(item)
F.close()
An additional comment, if you save the script in UTF8, you can inform Python
of that fact with a special comment, and actually use the correct characters
in your string constants (ä instead of \xe4). Make sure to use a text
editor that can save in UTF8, or use the correct coding comment for whatever
encoding in which you save the file.
# coding: utf8
import codecs
item=u'hyvä'
F=codecs.open('finnish.txt', 'w+', 'utf8')
F.write(item)
F.close()
-Mark
--
http://mail.python.org/mailman/listinfo/python-list
Re: string u'hyv\xe4' to file as 'hyvä'
Hello,
STILL do not work. WHAT to be done.
import codecs
item=u'hyv\xe4'
F=codecs.open('/opt/finnish.txt', 'w+', 'utf8')
F.writelines(item.encode('utf8'))
F.close()
In file i find 'hyv\xe4' instead of hyvä.
(Sorry for mistyping in previous letter about 'latin-1'. I was making
all possible combinations, when normal example syntax did not work,
before writting to this forum.)
regards,
gintare
On 27 Gruo, 00:43, gintare wrote:
> Could you please help me with special characters saving to file.
>
> I need to write the string u'hyv\xe4' to file.
> I would like to open file and to have line 'hyvä'
>
> import codecs
> word= u'hyv\xe4'
> F=codecs.open(/opt/finnish.txt, 'w+','Latin-1')
>
> F.writelines(item.encode('Latin-1'))
> F.writelines(item.encode('utf8'))
> F.writelines(item)
>
> F.close()
>
> All three writelines gives the same result in finnish.txt: hyv\xe4
> i would like to find 'hyvä'.
>
> regards,
> gintare
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
On Sun, 26 Dec 2010 17:12:50 -0800, misno...@gmail.com wrote:
> On Dec 23, 3:22 am, Steven D'Aprano +comp.lang.pyt...@pearwood.info> wrote:
>> You seem to have completely missed that there will be no bug report,
>> because this isn't a bug. (Or if it is a bug, the bug is elsewhere,
>> external to the function that raises the exception.) It is part of the
>> promised API. The fact that the exception is generated deep down some
>> chain of function calls is irrelevant.
>
> While the idea of being able to do this (create a general validation
> exception) sounds mildly appealing at first, what happens when the
> module implementing this documented API and documented error, has a bug?
> It seems that the user, or even module developer in the midst of
> writing, would now have no tools to easily tackle the problem, and no
> useful information to submit in the required bug report.
That's very true, but the same applies to *any* use of encapsulation. Any
time you hide information, you hide information (duh!). This doesn't stop
us from doing this:
def public(x):
if condition: return _private(x)
elif other_condition: return _more_private(x+1)
else: return _yet_another_private(x-1)
If you call public(42), and get the wrong answer, it's a bug, but the
source of the bug is hidden from the caller. If you have access to the
source code, you can work out where the bug lies (which of the three
private functions is buggy?) given the argument, but the return result
itself does not expose any information about where the bug lies. This is
considered an unavoidable but acceptable side-effect of an otherwise
desirable state of affairs: information hiding and encapsulation. The
caller being unaware of where and how the result is calculated is
considered a good thing, and the fact that it occasionally adds to the
debugging effort is considered such a trivial cost that it normally isn't
remarked upon, except by lunatics and die-hard fans of spaghetti code
using GOTO. But I repeat myself.
Why should exceptions *necessarily* be different? As I've repeatedly
acknowledged, for an unexpected exception (a bug), the developer needs
all the help he can get, and the current behaviour is the right way to do
it. You won't hear me argue differently. But for a documented, explicit,
expected, deliberate exception, Python breaks encapsulation by exposing
the internal details of any internal subroutines used to generate that
exception. This leads to messy tracebacks that obscure the source of bugs
in the caller's code:
>>> import re
>>> re.compile(r"(")
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.6/re.py", line 190, in compile
return _compile(pattern, flags)
File "/usr/lib/python2.6/re.py", line 245, in _compile
raise error, v # invalid expression
sre_constants.error: unbalanced parenthesis
I think critics of my position have forgotten what it's like to learning
the language. One of the most valuable skills to learn is to *ignore
parts of the traceback* -- a skill that takes practice and familiarity
with the library or function in use. To those who are less familiar with
the function, it can be very difficult to determine which parts of the
traceback are relevant and which aren't. In this case, the caller has
nothing to do with _compile, and the traceback looks like it's an
internal bug in a subroutine, when in fact it is actually due to bad
input. The experienced developer learns (by trial and error, possibly) to
ignore nearly half of the error message in this case.
In principle, the traceback could be roughly half as big, which means the
caller would find it half as difficult to read and understand:
>>> re.compile(r"(")
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.6/re.py", line 190, in compile
raise error, v # invalid expression
sre_constants.error: unbalanced parenthesis
With a one-to-one correspondence between the function called, and the
function reporting an error, it is easier to recognise that the error
lies in the input rather than some internal error in some subroutine you
have nothing to do with. Unfortunately there's no straightforward way to
consistently get this in Python without giving up the advantages of
delegating work to subroutines.
It need not be that way. This could, in principle, be left up to the
developer of the public function to specify (somehow!) that some specific
exceptions are expected, and should be treated as coming from public()
rather than from some internal subroutine. I don't have a concrete
proposal for such, although I did suggest a work-around. I expected
disinterest ("I don't see the point"). I didn't expect the level of
hostility to the idea that exceptions should (if and when possible) point
to the source of the error rather than some accidental implementation-
specific subroutine. Go figure.
--
Steven
--
http://mail.python.org/mailman/listinfo/
Re: How to pop the interpreter's stack?
On Sun, 26 Dec 2010 09:15:32 -0800, Ethan Furman wrote:
> Steven D'Aprano wrote:
>> Right. But I have thought of a clever trick to get the result KJ was
>> asking for, with the minimum of boilerplate code. Instead of this:
>>
>>
>> def _pre_spam(args):
>> if condition(args):
>> raise SomeException("message")
>> if another_condition(args):
>> raise AnotherException("message")
>> if third_condition(args):
>> raise ThirdException("message")
>>
>> def spam(args):
>> _pre_spam(args)
>> do_useful_work()
>>
>>
>> you can return the exceptions instead of raising them (exceptions are
>> just objects, like everything else!), and then add one small piece of
>> boilerplate to the spam() function:
>>
>>
>> def _pre_spam(args):
>> if condition(args):
>> return SomeException("message")
>> if another_condition(args):
>> return AnotherException("message")
>> if third_condition(args):
>> return ThirdException("message")
>>
>> def spam(args):
>> exc = _pre_spam(args)
>> if exc: raise exc
>> do_useful_work()
>
> -1
>
> You failed to mention that cleverness is not a prime requisite of the
> python programmer -- in fact, it's usually frowned upon. The big
> problem with the above code is you are back to passing errors in-band,
> pretty much completely defeating the point of have an out-of-band
> channel.
How is that any worse than making _pre_spam() a validation function that
returns a bool?
def spam(args):
flag = _pre_spam(args)
if flag: raise SomeException()
do_useful_work()
Is that also frowned upon for being too clever?
--
Steven
--
http://mail.python.org/mailman/listinfo/python-list
Re: __delitem__ "feature"
On Sun, 26 Dec 2010 18:49:55 +, kj wrote: > In Ian Kelly > writes: > >>On 12/26/2010 10:53 AM, kj wrote: >>> P.S. If you uncomment the commented-out line, and comment out the last >>> line of the __init__ method (which installs self._delitem as >>> self.__delitem__) then *all* the deletion attempts invoke the >>> __delitem__ method, and are therefore blocked. FWIW. > >>Because subclasses of builtins only check the class __dict__ for special >>method overrides, not the instance __dict__. > > > How do you know this? Is this documented? Or is this a case of > Monday-night quarterbacking? We know it because it explains the observable facts. It also happens to be documented, but documentation can be wrong or incomplete. The facts are never wrong, since by definition they are the facts. -- Steven -- http://mail.python.org/mailman/listinfo/python-list
User input masks - Access Style
Is there anyay to use input masks in python? Similar to the function
found in access where a users input is limited to a type, length and
format.
So in my case I want to ensure that numbers are saved in a basic
format.
1) Currency so input limited to 000.00 eg 1.00, 2.50, 13.80 etc
For sports times that is time duration not a system or date times
should I assume that I would need to calculate a user input to a
decimal number and then recalculate it to present it to user?
So an example, sorry.
import time #not sure if this is any use
minute = input("How many minutes: ")
seconds = input("How many seconds: ")
Hundredths = input("how many Hundredths: ")
# convert user input
MyTime = (minute/60)+(seconds)+(Hundredths/1800)
#Display to user assuming i had written a name and user
# had retrieved it
print("[User], your time was"), (MyTime/60:MyTime(MyTime-((MyTime/
60)*60).(MyTime-(MyTime>0))) )
--
http://mail.python.org/mailman/listinfo/python-list
Re: IDLE GUI not working
On Sun, Dec 26, 2010 at 4:33 PM, Python Programming wrote:
> Hello all,
>
> Newbie here so go easy on me. I've been trying to get the IDLE GUI to
> work on my machine, but have been unsuccessful so far. I have an IBM
> Thinkpad running Windows XP and it has an older version of Python
> running (2.2, I believe). When I try to use the shortcut to open the
> IDLE GUI nothing happens (not even a process running in task
> manager). When I use the cmd line to try to open the IDLE GUI, I get
> this error message:
>
>
> C:\>C:\python27\python C:\python27\Lib\idlelib\idle.py
> Traceback (most recent call last):
> File "C:\python27\Lib\idlelib\idle.py", line 11, in
>idlelib.PyShell.main()
> File "C:\python27\Lib\idlelib\PyShell.py", line 1389, in main
>root = Tk(className="Idle")
> File "C:\python27\lib\lib-tk\Tkinter.py", line 1685, in __init__
>self.tk = _tkinter.create(screenName, baseName, className,
> interactive, want
> objects, useTk, sync, use)
> _tkinter.TclError: Can't find a usable init.tcl in the following
> directories:
>{C:\IBMTOOLS\Python22\tcl\tcl8.4} C:/IBMTOOLS/Python22/tcl/tcl8.5
> C:/python2
> 7/lib/tcl8.5 C:/lib/tcl8.5 C:/lib/tcl8.5 C:/library C:/library C:/
> tcl8.5.2/libra
> ry C:/tcl8.5.2/library
>
> C:/IBMTOOLS/Python22/tcl/tcl8.4/init.tcl: version conflict for package
> "Tcl": ha
> ve 8.5.2, need exactly 8.4
> version conflict for package "Tcl": have 8.5.2, need exactly 8.4
>while executing
> "package require -exact Tcl 8.4"
>(file "C:/IBMTOOLS/Python22/tcl/tcl8.4/init.tcl" line 19)
>invoked from within
> "source C:/IBMTOOLS/Python22/tcl/tcl8.4/init.tcl"
>("uplevel" body line 1)
>invoked from within
> "uplevel #0 [list source $tclfile]"
>
>
> This probably means that Tcl wasn't installed properly.
>
>
> As I understand this, python 2.7 is looking in a directory or path
> specified by the older, python 2.2 for Tcl. I have tried to unset
> this path by entering this into the cmd line:
>
> ---
> C:\>set TCL_LIBRARY=
> C:\>set TK_LIBRARY=
> C:\>C:\Python27\python.exe C:\Python27\Lib\idlelib\idle.py
> ---
>
> No dice.
>
> I tried to enter this information as a new pythonpath in the
> environment variables. No dice.
>
> I've tried, in vain, to get this thing working, but most of the other
> explanations are way over my head (I'm pretty new to programming and
> digging around the programming guts of computers)
>
> It seems like a fairly common problem, but haven't gotten a good
> answer from either the official python help boards or elsewhere. I
> was hoping that someone here could give me a easy to understand way to
> make the IDLE GUI work.
>
> Thanks in advance.
> --
> http://mail.python.org/mailman/listinfo/python-list
Wow, I wrote a note to this list so closely resembling yours I had to read
carefully to ensure it wasn't my own.
You have a PATH problem - these machines ship with Tcl installed and that's
creating this issue.
There is the standard Path variable, but if you'll notice, there is also a
scrollbar to the right. Scroll down and find additional variables that need
fixed - PythonPath, TCL_Library, and TK Library all need to be redirected to
your current install. If you fix all of these, your problem should be
solved and you'll be on your way, at least that was my exact issue on a
Thinkpad running XP.
Best of luck!
Grant
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
On Dec 25, 2:49 pm, Steven D'Aprano wrote:
> On Sat, 25 Dec 2010 09:17:27 -0500, Robert Kern wrote:
> > On 12/24/10 5:14 PM, Ethan Furman wrote:
>
> >> There are also times when I change the exception being raised to match
> >> what python expects from that type of object -- for example, from
> >> WhatEverException to KeyError for a dict-like object. So in this regard
> >> I agree with Steven.
>
> > Steven isn't arguing that particular point here, nor is anyone arguing
> > against it.
>
> Emphasis on *here*.
>
> You will note that in Python 3, if you raise an exception inside an
> except block, both the *original* and the new exception are printed. This
> is great for fixing bugs inside except blocks, but terribly disruptive
> for catching one error and raising another error in it's place, e.g.:
>
> try:
> x+0
> except ValueError, TypeError as e:
> # x is not a numeric value, e.g. a string or a NAN.
> raise MyError('x is not a number')
>
> The explicit raise is assumed to indicate a bug in the except block, and
> the original exception is printed as well.
>
> But that's a separate issue from what is being discussed here. What we're
> discussing here is the idea that a function should be able to delegate
> work to private subroutines without the caller being aware of that fact.
> When you return a value, the caller doesn't see the internal details of
> how you calculated the value, but if you deliberately raise an exception,
> the caller does. Often this is the right thing to do, but sometimes it
> isn't. E.g. you can't delegate input validation to a subroutine and raise
> inside the subroutine without obfuscating the traceback.
>>> import re
>>> re.compile(r"(")
Traceback (most recent call last):
File "", line 1, in
File "/usr/lib/python2.6/re.py", line 190, in compile
return _compile(pattern, flags)
File "/usr/lib/python2.6/re.py", line 245, in _compile
raise error, v # invalid expression
sre_constants.error: unbalanced parenthesis
OHMYGOD HOW DARE the standard library allow the traceback list an
internal function that does input valididation!
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
On Dec 26, 11:09 am, kj wrote: > In writes: > > >Except that the *caller* never gets the traceback (unless if it deliberately > >inspects the stack for some metaprogramming reason). It gets the exception, > >and > >that is the same no matter what you do. The developer/user gets the > >traceback, > >and those implementation details *are* often important to them. > > Just look at what Python shows you if you pass the wrong number of > arguments to a function: > > >>> def spam(x, y, z): pass > ... > >>> spam(1, 2) > > Traceback (most recent call last): > File "", line 1, in > TypeError: spam() takes exactly 3 arguments (2 given) > > That's it. The traceback stops at the point of the error. Python > doesn't show you all the underlying C-coded machinery that went > into detecting the error and emitting the error message. *No one* > needs this information at this point. All I'm saying is that I > want to do the same thing with my argument validation code as Python > does with its argument validation code: keep it out of sight. When > my argument validation code fires an exception ***there's no bug > in **my** code***. It's doing exactly what it's supposed to do. > Therefore, there's no need for me to debug anything, and certainly > no need for me to inspect the traceback all the way to the exception. > The bug is in the code that called my function with the wrong > arguments. The developer of that code has no more use for seeing > the traceback all the way to where my code raises the exception > than I have for seeing the traceback of Python's underlying C code > when I get an error like the one shown above. Python makes no attempt to hide its machinery in tracebacks (that I'm aware of); in fact stack points from internal Python functions, classes, and modules appear in tracebacks all the time. The reason you don't see traceback lines for Python's argument validation is it's written in C. If it bothers you that much, you're welcome to write you own argument validation in C, too. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
On Dec 23, 3:22 am, Steven D'Aprano wrote: > You seem to have completely missed that there will be no bug report, > because this isn't a bug. (Or if it is a bug, the bug is elsewhere, > external to the function that raises the exception.) It is part of the > promised API. The fact that the exception is generated deep down some > chain of function calls is irrelevant. While the idea of being able to do this (create a general validation exception) sounds mildly appealing at first, what happens when the module implementing this documented API and documented error, has a bug? It seems that the user, or even module developer in the midst of writing, would now have no tools to easily tackle the problem, and no useful information to submit in the required bug report. -- http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
On 12/26/2010 2:14 PM, kj wrote: > In Ethan Furman > writes: > >> You failed to mention that cleverness is not a prime requisite of the >> python programmer -- in fact, it's usually frowned upon. > > That's the party line, anyway. I no longer believe it. I've been > crashing against one bit of cleverness after another in Python's > unification of types and classes... Well if you can find a way to implement a class system that doesn't use clever tricks *in its implementation* please let me know. regards Steve -- Steve Holden +1 571 484 6266 +1 800 494 3119 PyCon 2011 Atlanta March 9-17 http://us.pycon.org/ See Python Video! http://python.mirocommunity.org/ Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
IDLE GUI not working
Hello all,
Newbie here so go easy on me. I've been trying to get the IDLE GUI to
work on my machine, but have been unsuccessful so far. I have an IBM
Thinkpad running Windows XP and it has an older version of Python
running (2.2, I believe). When I try to use the shortcut to open the
IDLE GUI nothing happens (not even a process running in task
manager). When I use the cmd line to try to open the IDLE GUI, I get
this error message:
C:\>C:\python27\python C:\python27\Lib\idlelib\idle.py
Traceback (most recent call last):
File "C:\python27\Lib\idlelib\idle.py", line 11, in
idlelib.PyShell.main()
File "C:\python27\Lib\idlelib\PyShell.py", line 1389, in main
root = Tk(className="Idle")
File "C:\python27\lib\lib-tk\Tkinter.py", line 1685, in __init__
self.tk = _tkinter.create(screenName, baseName, className,
interactive, want
objects, useTk, sync, use)
_tkinter.TclError: Can't find a usable init.tcl in the following
directories:
{C:\IBMTOOLS\Python22\tcl\tcl8.4} C:/IBMTOOLS/Python22/tcl/tcl8.5
C:/python2
7/lib/tcl8.5 C:/lib/tcl8.5 C:/lib/tcl8.5 C:/library C:/library C:/
tcl8.5.2/libra
ry C:/tcl8.5.2/library
C:/IBMTOOLS/Python22/tcl/tcl8.4/init.tcl: version conflict for package
"Tcl": ha
ve 8.5.2, need exactly 8.4
version conflict for package "Tcl": have 8.5.2, need exactly 8.4
while executing
"package require -exact Tcl 8.4"
(file "C:/IBMTOOLS/Python22/tcl/tcl8.4/init.tcl" line 19)
invoked from within
"source C:/IBMTOOLS/Python22/tcl/tcl8.4/init.tcl"
("uplevel" body line 1)
invoked from within
"uplevel #0 [list source $tclfile]"
This probably means that Tcl wasn't installed properly.
As I understand this, python 2.7 is looking in a directory or path
specified by the older, python 2.2 for Tcl. I have tried to unset
this path by entering this into the cmd line:
---
C:\>set TCL_LIBRARY=
C:\>set TK_LIBRARY=
C:\>C:\Python27\python.exe C:\Python27\Lib\idlelib\idle.py
---
No dice.
I tried to enter this information as a new pythonpath in the
environment variables. No dice.
I've tried, in vain, to get this thing working, but most of the other
explanations are way over my head (I'm pretty new to programming and
digging around the programming guts of computers)
It seems like a fairly common problem, but haven't gotten a good
answer from either the official python help boards or elsewhere. I
was hoping that someone here could give me a easy to understand way to
make the IDLE GUI work.
Thanks in advance.
--
http://mail.python.org/mailman/listinfo/python-list
Re: string u'hyv\xe4' to file as 'hyvä'
On 26/12/2010 22:43, gintare wrote:
Could you please help me with special characters saving to file.
I need to write the string u'hyv\xe4' to file.
I would like to open file and to have line 'hyvä'
import codecs
word= u'hyv\xe4'
F=codecs.open(/opt/finnish.txt, 'w+','Latin-1')
This opens the file using the Latin-1 encoding (although only if you
put the filename in quotes).
F.writelines(item.encode('Latin-1'))
This encodes the Unicode item (did you mean 'word'?) to a bytestring
using the Latin-1 encoding. You opened the file using Latin-1 encoding,
so this is pointless. You should pass a Unicode string; it will encode
it for you.
You're also passing a bytestring to the .writelines method, which
expects a list of strings.
What you should be doing is this:
F.write(word)
F.writelines(item.encode('utf8'))
This encodes the Unicode item to a bytestring using the UTF-8 encoding.
This is also pointless. You shouldn't be encoding to UTF-8 and then
trying to write it to a file which was opened using Latin-1 encoding!
F.writelines(item)
F.close()
All three writelines gives the same result in finnish.txt: hyv\xe4
i would like to find 'hyvä'.
--
http://mail.python.org/mailman/listinfo/python-list
Need urllib.urlretrieve and urllib2.OpenerDirector together
Hello everyone. I'm writing a script in Python 2.7 which uses a urllib2.OpenerDirector instance via urllib2.build_opener() to take advantage of the urllib2.HTTPCookieProcessor class, because I need to store and re-send the cookies I get: opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookielib.CookieJar())) However, after making several requests and moving the cookies around, eventually I need to retrieve a list of URLs. I wanted to use urllib.urlretrieve() because I read it downloads the file in chunks, but I cannot because I need to carry my cookies on the request and urllib.urlretrieve() uses a urllib.URLOpener, which doesn't have support for cookie handlers like OpenerDirector has. I want to know the reason of this strange way of splitting functionality, and how can I achieve my goal. Thank you in advance! -- http://mail.python.org/mailman/listinfo/python-list
Re: inspect.getsource bug?
On 26 Dic, 19:24, Ciccio wrote: > Try this: > 1) define a function 'foo' in a script > 2) runfile the script from a shell > 3) do 'inspect.getsource(foo)' > 4) change the source of 'foo' > 5) runfile the script from the same shell > 6) do 3 again > > On my 2.6.6 getsource returns twice the same code. I couldn't find > very much about this, is there any known workaround? > > thanks found this in the meantime: http://bugs.python.org/issue993580 -- http://mail.python.org/mailman/listinfo/python-list
string u'hyv\xe4' to file as 'hyvä'
Could you please help me with special characters saving to file.
I need to write the string u'hyv\xe4' to file.
I would like to open file and to have line 'hyvä'
import codecs
word= u'hyv\xe4'
F=codecs.open(/opt/finnish.txt, 'w+','Latin-1')
F.writelines(item.encode('Latin-1'))
F.writelines(item.encode('utf8'))
F.writelines(item)
F.close()
All three writelines gives the same result in finnish.txt: hyv\xe4
i would like to find 'hyvä'.
regards,
gintare
--
http://mail.python.org/mailman/listinfo/python-list
Re: lxml etree question
Jim, 26.12.2010 00:32:
On Dec 25, 5:33 am, Stefan Behnel wrote:
lxml knows about this special case, so you can write
{http://www.w3.org/XML/1998/namespace}lang
and lxml will take care of using the right prefix.
Stefan, thank you for the software, which has helped me a great deal.
I tried that exact thing, among a number of others, and it didn't work
for me (I got ns0).
Works for me, at least with a recent SVN version:
Python 2.7.1rc1+ (trunk:86636, Nov 21 2010, 09:18:37)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import lxml.etree as ET
>>> el = ET.Element('test',
...{'{http://www.w3.org/XML/1998/namespace}lang': 'de'})
>>> ET.tostring(el)
''
Anyway, I applied a patch that makes sure it will always use the 'xml'
prefix for this namespace. Will be in 2.3 final.
Stefan
--
http://mail.python.org/mailman/listinfo/python-list
Re: __delitem__ "feature"
Am 26.12.2010 19:49, schrieb kj: > How do you know this? Is this documented? Or is this a case of > Monday-night quarterbacking? Please stop bitching around. You know that by carefully reading the documentation: http://docs.python.org/reference/datamodel.html#special-method-lookup-for-new-style-classes -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
On 2010-12-26, Stefan Behnel wrote: > Tim Harig, 26.12.2010 10:22: >> On 2010-12-26, Stefan Behnel wrote: >>> Tim Harig, 26.12.2010 02:05: On 2010-12-25, Nobody wrote: > On Sat, 25 Dec 2010 14:41:29 -0500, Roy Smith wrote: >> Of course, one advantage of XML is that with so much redundant text, it >> compresses well. We typically see gzip compression ratios of 20:1. >> But, that just means you can archive them efficiently; you can't do >> anything useful until you unzip them. > > XML is typically processed sequentially, so you don't need to create a > decompressed copy of the file before you start processing it. Sometimes XML is processed sequentially. When the markup footprint is large enough it must be. Quite often, as in the case of the OP, you only want to extract a small piece out of the total data. In those cases, being forced to read all of the data sequentially is both inconvenient and and a performance penalty unless there is some way to address the data you want directly. >>> [...] >>> If you do it a lot, you will have to find a way to make the access >>> efficient for your specific use case. So the file format doesn't matter >>> either, because the data will most likely end up in a fast data base after >>> reading it in sequentially *once*, just as in the case above. >> >> If the data is just going to end up in a database anyway; then why not >> send it as a database to begin with and save the trouble of having to >> convert it? > > I don't think anyone would object to using a native format when copying > data from one database 1:1 to another one. But if the database formats are > different on both sides, it's a lot easier to map XML formatted data to a > given schema than to map a SQL dump, for example. Matter of use cases, not > of data size. Your assumption keeps hinging on the fact that I should want to dump the data into a database in the first place. Very often I don't. I just want to rip out the small portion of information that happens to be important to me. I may not even want to archive my little piece of the information once I have processed it. Even assuming that I want to dump all the data into a database, walking through a bunch of database records to translate them into the schema for another database is no more difficult then walking through a bunch of XML elements. In fact, it is even easier since I can use the relational model to reconstruct the information in an organization that better fits how the data is actually structured in my database instead of being constrained by how somebody else wanted to organize their XML. There is no need to "map a[sic] SQL dump." XML is great when the data is set is small enough that parsing the whole tree has negligable costs. I can choose whether I want to parse it sequentially or use XPath/DOM/Etree etc to make it appear as though I am making random accesses. When the data set grows so that parsing it is expensive I loose that choice even if my use case would otherwise prefer a random access paradigm. When that happens, there are better ways of communicating that data that doesn't force me into using a high overhead method of extracting my data. The problem is that XML has become such a defacto standard that it used automatically, without thought, even when there are much better alternatives available. -- http://mail.python.org/mailman/listinfo/python-list
Re: Python Web App
@Katie Thank you I considered this option until I realized it wouldn't let me do anything other than ping from the command line. The rest of you all make valid points after doing a little more research on my own I found some really nice web based text editors but they didn't have any testing abilities which meant learning in that environment wasn't feasible in my opinion. I am inclined to agree that chrome OS will probably not do as well as they want it to but with the kind of capital Google has they could easily flood the market. In the end I wound up giving the notebook to my mom because all she really does is check her email and Facebook so it was perfect for her. Thank You for all the responses they were a great help with me testing the notebook. On Dec 25, 9:02 pm, Katie T wrote: > On Wed, Dec 22, 2010 at 9:43 PM, Sean wrote: > > Anybody know where I can find a Python Development Environment in the > > form of a web app for use with Chrome OS. I have been looking for a > > few days and all i have been able to find is some old discussions with > > python developers talking about they will want one for the OS to be a > > success with them. > > Your best bet is probably just to SSH to a *nix box and use something > like vim or emacs. None of the web solutions are anywhere near acceptable. > > Katie > -- > CoderStackhttp://www.coderstack.co.uk/python-jobs > The Software Developer Job Board -- http://mail.python.org/mailman/listinfo/python-list
A Brief Illustrated Guide To Understanding Islam Home
A Brief Illustrated Guide To Understanding Islam Home http://www.islam-guide.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
On 12/25/2010 2:50 PM, Steven D'Aprano wrote:
On Fri, 24 Dec 2010 10:51:32 -0800, John Nagle wrote:
On 12/24/2010 3:24 AM, Carl Banks wrote:
On Dec 24, 1:24 am, Steven D'Aprano wrote:
All I'm
suggesting is that there should be a way of reducing the boilerplate
needed for this idiom:
def _validate_arg(x):
if x == 'bad input': return False
return True
def f(arg):
if not _validate_arg(arg):
raise ValueError
process(arg)
to something more natural that doesn't needlessly expose
implementation details that are completely irrelevant to the caller.
How about
raise ValueError("Bad input %s to validate_arg" % (repr(arg),))
You can pass arguments to most exceptions, and the content of the
exception is determined entirely by the code raising it.
I know that exceptions can take arguments (usually string error
messages). I was writing in short-hand. My apologies, I thought that
would have been obvious :(
Perhaps you have missed the context of the discussion. The context is
that the called function delegates the job of validating input to a
private function, which should be hidden from the caller (it's private,
not part of the public API, subject to change, hidden, etc.) but
tracebacks expose that information, obscuring the cause of the fault.
(The fault being bad input to the public function, not an accidental bug
in the private function.)
If end users are seeing uncaught tracebacks, the program is broken.
Well, perhaps, but that's a separate issue. We're talking about the
caller of the function seeing internal details, not the end use.
No, that is the issue, unless the program itself is examining the
stack traceback data. Python exception-catching has no notion of
what code raised the exception. Only the contents of the exception
object are normally available. So the "private function" is always
"hidden", unless you're debugging, in which case it shouldn't be
hidden.
Traceback is purely a debugging feature. In some Python
implementations, such as Shed Skin, you don't get tracebacks unless
you're running under a debugger.
John Nagle
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
In Ethan Furman writes: >You failed to mention that cleverness is not a prime requisite of the >python programmer -- in fact, it's usually frowned upon. That's the party line, anyway. I no longer believe it. I've been crashing against one bit of cleverness after another in Python's unification of types and classes... -- http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
In writes: >Except that the *caller* never gets the traceback (unless if it deliberately >inspects the stack for some metaprogramming reason). It gets the exception, >and >that is the same no matter what you do. The developer/user gets the traceback, >and those implementation details *are* often important to them. Just look at what Python shows you if you pass the wrong number of arguments to a function: >>> def spam(x, y, z): pass ... >>> spam(1, 2) Traceback (most recent call last): File "", line 1, in TypeError: spam() takes exactly 3 arguments (2 given) That's it. The traceback stops at the point of the error. Python doesn't show you all the underlying C-coded machinery that went into detecting the error and emitting the error message. *No one* needs this information at this point. All I'm saying is that I want to do the same thing with my argument validation code as Python does with its argument validation code: keep it out of sight. When my argument validation code fires an exception ***there's no bug in **my** code***. It's doing exactly what it's supposed to do. Therefore, there's no need for me to debug anything, and certainly no need for me to inspect the traceback all the way to the exception. The bug is in the code that called my function with the wrong arguments. The developer of that code has no more use for seeing the traceback all the way to where my code raises the exception than I have for seeing the traceback of Python's underlying C code when I get an error like the one shown above. ~kj -- http://mail.python.org/mailman/listinfo/python-list
Re: Keeping track of the N largest values
Am 25.12.2010 16:42, schrieb Roy Smith: I'm processing a stream of N numbers and want to keep track of the K largest. There's too many numbers in the stream (i.e. N is too large) to keep in memory at once. K is small (100 would be typical). > From a theoretical point of view, I should be able to do this in N log K time. What I'm doing now is essentially: top = [-1]# Assume all x are>= 0 for x in input(): if x<= top[0]: continue top.append(x) if len(top)> K: top.sort() top.pop(0) I can see pathological cases (say, all input values the same) where running time would be N K log K, but on average (N>> K and random distribution of values), this should be pretty close to N. Is there a better way to do this, either from a theoretical running time point of view, or just a nicer way to code this in Python? Here is my version: l = [] K = 10 while 1: a = input() if len(l) == K: l.remove(min(l)) l=[x for x in l if x < a] + [a] + [x for x in l if x > a] print l -- http://mail.python.org/mailman/listinfo/python-list
Re: __delitem__ "feature"
In Ian Kelly writes: >On 12/26/2010 10:53 AM, kj wrote: >> P.S. If you uncomment the commented-out line, and comment out the >> last line of the __init__ method (which installs self._delitem as >> self.__delitem__) then *all* the deletion attempts invoke the >> __delitem__ method, and are therefore blocked. FWIW. >Because subclasses of builtins only check the class __dict__ for special >method overrides, not the instance __dict__. How do you know this? Is this documented? Or is this a case of Monday-night quarterbacking? ~kj -- http://mail.python.org/mailman/listinfo/python-list
Re: type(d) != type(d.copy()) when type(d).issubclass(dict)
In Duncan Booth writes: >kj wrote: >> Watch this: >> > class neodict(dict): pass >> ... > d = neodict() > type(d) >> > type(d.copy()) >> >> >> >> Bug? Feature? Genius beyond the grasp of schlubs like me? >Feature. >In (almost?) all cases any objects constructed by a subclass of a builtin >class will be of the original builtin class. What I *really* would like to know is: how do *you* know this (and the same question goes for the other responders who see this behavior of dict as par for the course). Can you show me where it is in the documentation? I'd really appreciate it. TIA! ~kj -- http://mail.python.org/mailman/listinfo/python-list
Re: type(d) != type(d.copy()) when type(d).issubclass(dict)
"John O'Hagan" writes: >IMO one of the benefits of subclassing is that you can just "bolt on" >additional behaviour without having to know all the inner workings of the >superclass, a benefit that is somewhat defeated by this behaviour of builtins. I agree. I've read the old post/articles by GvR and other over how great it will be now that one can subclass Python builtin types like any other class (GvR even gives explicit examples of this luscious possibility in his paper on type/class unification). But now I'm discovering so many caveats, exceptions, and gotchas about subclassing builtins that I have to conclude that this much celebrated new capability is basically useless... Just like "readability counts", it is also true that "conceptual clarity" counts, and treating builtins as classes in Python is the most obfuscated design I've ever seen. UserDict, come back, all is forgotten! ~kj -- http://mail.python.org/mailman/listinfo/python-list
Re: __delitem__ "feature"
Am 26.12.2010 18:53, schrieb kj: > It means that, for both subclasses, del fails to trigger the > dynamically installed instance method __delitem__. Magic methods like __delitem__ are looked up on the type, not on the instance. You can't change the behaviour on instances. > If I replace dict with UserDict, *both* deletion attempts lead to > a call to the dynamic __delitem__ method, and are thus blocked. > This is the behavior I expected of dict (and will help me hold on > to my belief that I'm not going insane when inevitably I'm told > that there's no bug in dict or list). UserDict is an old style class (not a subclass of object). Old style classes behave differently. Christian -- http://mail.python.org/mailman/listinfo/python-list
inspect.getsource bug?
Try this: 1) define a function 'foo' in a script 2) runfile the script from a shell 3) do 'inspect.getsource(foo)' 4) change the source of 'foo' 5) runfile the script from the same shell 6) do 3 again On my 2.6.6 getsource returns twice the same code. I couldn't find very much about this, is there any known workaround? thanks -- http://mail.python.org/mailman/listinfo/python-list
Re: __delitem__ "feature"
On 12/26/2010 10:53 AM, kj wrote: P.S. If you uncomment the commented-out line, and comment out the last line of the __init__ method (which installs self._delitem as self.__delitem__) then *all* the deletion attempts invoke the __delitem__ method, and are therefore blocked. FWIW. Because subclasses of builtins only check the class __dict__ for special method overrides, not the instance __dict__. -- http://mail.python.org/mailman/listinfo/python-list
__delitem__ "feature"
When I execute this file:
#--
def nodelfactory(klass):
class nodel(klass):
def _delitem(self, _):
raise TypeError("can't delete")
# __delitem__ = _delitem
def __init__(self, *a, **k):
klass.__init__(self, *a, **k)
self.__delitem__ = self._delitem
nodel.__name__ = 'nodel%s' % klass.__name__
return nodel
if __name__ == '__main__':
import traceback as tb
d = nodelfactory(dict)([('k1', 'v1'), ('k2', 'v2')])
try: d.__delitem__('k1')
except TypeError: tb.print_exc()
print d
try: del d['k1']
except TypeError: tb.print_exc()
print d
l = nodelfactory(list)([1, 2, 3, 4])
try: l.__delitem__(0)
except TypeError: tb.print_exc()
print l
try: del l[0]
except TypeError: tb.print_exc()
print l
#--
...the output I get is:
Traceback (most recent call last):
File "/tmp/delbug.py", line 20, in
try: d.__delitem__('k1')
File "/tmp/delbug.py", line 4, in _delitem
raise TypeError("can't delete")
TypeError: can't delete
{'k2': 'v2', 'k1': 'v1'}
{'k2': 'v2'}
Traceback (most recent call last):
File "/tmp/delbug.py", line 30, in
try: l.__delitem__(0)
File "/tmp/delbug.py", line 4, in _delitem
raise TypeError("can't delete")
TypeError: can't delete
[1, 2, 3, 4]
[2, 3, 4]
It means that, for both subclasses, del fails to trigger the
dynamically installed instance method __delitem__.
If I replace dict with UserDict, *both* deletion attempts lead to
a call to the dynamic __delitem__ method, and are thus blocked.
This is the behavior I expected of dict (and will help me hold on
to my belief that I'm not going insane when inevitably I'm told
that there's no bug in dict or list).
Interestingly enough, if I replace list with UserList, I see no
change in behavior. So maybe I am going insane after all.
~kj
P.S. If you uncomment the commented-out line, and comment out the
last line of the __init__ method (which installs self._delitem as
self.__delitem__) then *all* the deletion attempts invoke the
__delitem__ method, and are therefore blocked. FWIW.
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
Steven D'Aprano wrote:
Right. But I have thought of a clever trick to get the result KJ was
asking for, with the minimum of boilerplate code. Instead of this:
def _pre_spam(args):
if condition(args):
raise SomeException("message")
if another_condition(args):
raise AnotherException("message")
if third_condition(args):
raise ThirdException("message")
def spam(args):
_pre_spam(args)
do_useful_work()
you can return the exceptions instead of raising them (exceptions are
just objects, like everything else!), and then add one small piece of
boilerplate to the spam() function:
def _pre_spam(args):
if condition(args):
return SomeException("message")
if another_condition(args):
return AnotherException("message")
if third_condition(args):
return ThirdException("message")
def spam(args):
exc = _pre_spam(args)
if exc: raise exc
do_useful_work()
-1
You failed to mention that cleverness is not a prime requisite of the
python programmer -- in fact, it's usually frowned upon. The big
problem with the above code is you are back to passing errors in-band,
pretty much completely defeating the point of have an out-of-band channel.
~Ethan~
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
Steven D'Aprano wrote: On Sat, 25 Dec 2010 09:17:27 -0500, Robert Kern wrote: On 12/24/10 5:14 PM, Ethan Furman wrote: There are also times when I change the exception being raised to match what python expects from that type of object -- for example, from WhatEverException to KeyError for a dict-like object. So in this regard I agree with Steven. Steven isn't arguing that particular point here, nor is anyone arguing against it. Emphasis on *here*. You will note that in Python 3, if you raise an exception inside an except block, both the *original* and the new exception are printed. This is great for fixing bugs inside except blocks, but terribly disruptive for catching one error and raising another error in it's place... Yes, this is where I was agreeing with Steven. While I love python3, the current nested exception behavior is horrible. ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
Carl Banks wrote: Python is not, and never has been, about hiding internal details. It's about openness, and there's no reason a traceback should hide internal details any more than a class should--in fact hiding information in the traceback is far worse, because you're suppressing information that could be crucial for debugging. +100 ~Ethan~ -- http://mail.python.org/mailman/listinfo/python-list
Re: Keeping track of the N largest values
In article , n00m wrote: > from bisect import insort_left > > K = 5 > top = [] > while 1: > x = input() > if len(top) < K: > insort_left(top, x) > elif x > top[0]: > del top[0] > insort_left(top, x) > print top > > > will be enough Hmmm, that's an interesting idea. Thanks. -- http://mail.python.org/mailman/listinfo/python-list
Re: Design Ideals Goals Python 3 - Forest for the trees
On Dec 26, 5:34 am, Alice Bevan–McGregor wrote: > > I've never really attempted to use JIT optimizers due to the fact that > all of my development and production environments are 64-bit, and I > haven't found one yet that supports 64-bit properly. Relying on dead > projects, however, is an issue of larger scope than mere portability. > ;) > The PyPy JIT supports x86_64. It's still being improved, but it does provide real speedups in some cases already. Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Keeping track of the N largest values
from bisect import insort_left K = 5 top = [] while 1: x = input() if len(top) < K: insort_left(top, x) elif x > top[0]: del top[0] insort_left(top, x) print top will be enough -- http://mail.python.org/mailman/listinfo/python-list
Re: Keeping track of the N largest values
Roy Smith wrote: > In article , > Duncan Booth wrote: > >> Roy Smith wrote: >> >> > >> > I'm processing a stream of N numbers and want to keep track of the K >> > largest. There's too many numbers in the stream (i.e. N is too large) >> > to keep in memory at once. K is small (100 would be typical). >> > ... >> > Is there a better way to do this, either from a theoretical running >> > time point of view, or just a nicer way to code this in Python? >> >> How about: >> >> from heapq import nlargest >> top = nlargest(K, input()) >> >> It uses a heap so avoids completely resorting each time. > > Hmmm, that looks like it would solve the problem as stated, but there's > another twist. In addition to finding the K largest values, I *also* > need to do some other processing on all the values (which I didn't show > in the original example, incorrectly thinking it not germane to my > question). The problem with nlargest() is that it doesn't give me a > hook to do that. If Paul's solution doesn't suffice -- the heapq module has the building blocks for a custom solution, e. g.: import heapq from functools import partial class NLargest(object): def __init__(self, n): self.n = n self.heap = [] def tally(self, item): heap = self.heap if len(heap) >= self.n: heapq.heappushpop(heap, item) self.tally = partial(heapq.heappushpop, heap) else: heapq.heappush(heap, item) def __str__(self): return str(sorted(self.heap, reverse=True)) if __name__ == "__main__": import random items = range(100) random.shuffle(items) accu = NLargest(10) for item in items: accu.tally(item) print item, accu > PS: I'm assuming heapq.nlargest(n, iterable) operates with memory > proportional to n, and not to the iterator length. That's the only > reasonable conclusion, but the docs don't actually come out and say it. I would hope so. -- http://mail.python.org/mailman/listinfo/python-list
Re: GUI Tools for Python 3.1
On Friday 24 December 2010, 03:58:15 Randy Given wrote: > Lots of stuff for 2.6 and 2.7 -- what GUI tools are there for 3.1? PyQt4 of course. http://www.riverbankcomputing.com Pete -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
Tim Harig, 26.12.2010 10:22: On 2010-12-26, Stefan Behnel wrote: Tim Harig, 26.12.2010 02:05: On 2010-12-25, Nobody wrote: On Sat, 25 Dec 2010 14:41:29 -0500, Roy Smith wrote: Of course, one advantage of XML is that with so much redundant text, it compresses well. We typically see gzip compression ratios of 20:1. But, that just means you can archive them efficiently; you can't do anything useful until you unzip them. XML is typically processed sequentially, so you don't need to create a decompressed copy of the file before you start processing it. Sometimes XML is processed sequentially. When the markup footprint is large enough it must be. Quite often, as in the case of the OP, you only want to extract a small piece out of the total data. In those cases, being forced to read all of the data sequentially is both inconvenient and and a performance penalty unless there is some way to address the data you want directly. [...] If you do it a lot, you will have to find a way to make the access efficient for your specific use case. So the file format doesn't matter either, because the data will most likely end up in a fast data base after reading it in sequentially *once*, just as in the case above. If the data is just going to end up in a database anyway; then why not send it as a database to begin with and save the trouble of having to convert it? I don't think anyone would object to using a native format when copying data from one database 1:1 to another one. But if the database formats are different on both sides, it's a lot easier to map XML formatted data to a given schema than to map a SQL dump, for example. Matter of use cases, not of data size. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: Design Ideals Goals Python 3 - Forest for the trees
So do the new changes(to the GIL) nullify concerns raised by David Beazely here http://dabeaz.com/python/UnderstandingGIL.pdf Ah, good catch. I had watched the recorded presentation some time ago. Yes, the rewritten GIL should alleviate those problems. Instead of simply releasing and re-acquiring the GIL, it releases, then determines thread scheduling using the brainfuck algorithm instead of leaving it up to the kernel in the brief instant the GIL is unassigned (which often doesn't context switch to another thread, thus the performance penalty). (I beleive that algorithm, whose name -is- accurate, was the winner of the long, long discussion on the Python ticket.) Some projects have been using and requiring psyco to gain speed improvements in python http://psyco.sourceforge.net/introduction.html however it seems that the developer is not active on this project anymore and is more active on PyPy http://codespeak.net/pypy/dist/pypy/doc/ I've never really attempted to use JIT optimizers due to the fact that all of my development and production environments are 64-bit, and I haven't found one yet that supports 64-bit properly. Relying on dead projects, however, is an issue of larger scope than mere portability. ;) A program such as AVSP http://avisynth.org/qwerpoi/ which relies on psyco what would be a good proposition to use when taking the project to python 3.0 if psyco will remain unavailable? I'd take the same approach Python 3 itself did; rewrite it for Python 3 and take the opportunity to remove excessive backwards compatibility cruft, streamline algorithms, etc. With a suite of existing unit/functional tests, that possibility is the ultimate in test-driven development. ;) It would also follow the write clean code, then profile and optimize where actually needed philosophy. Obviously that recommendation won't be the "best" solution for every project. With all of the FOSS projects I really, really care about I'm writing from near-scratch (the code, if not the algorithms) with 2.6+ and 3.1+ polygot (no conversion tools like 2to3, and no split packaging) compatibility as a primary design goal. So far it's working out quite well. - Alice -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
On 2010-12-26, Stefan Behnel wrote: > Tim Harig, 26.12.2010 02:05: >> On 2010-12-25, Nobody wrote: >>> On Sat, 25 Dec 2010 14:41:29 -0500, Roy Smith wrote: Of course, one advantage of XML is that with so much redundant text, it compresses well. We typically see gzip compression ratios of 20:1. But, that just means you can archive them efficiently; you can't do anything useful until you unzip them. >>> >>> XML is typically processed sequentially, so you don't need to create a >>> decompressed copy of the file before you start processing it. >> >> Sometimes XML is processed sequentially. When the markup footprint is >> large enough it must be. Quite often, as in the case of the OP, you only >> want to extract a small piece out of the total data. In those cases, being >> forced to read all of the data sequentially is both inconvenient and and a >> performance penalty unless there is some way to address the data you want >> directly. > > So what? If you only have to do that once, it doesn't matter if you have to > read the whole file or just a part of it. Should make a difference of a > couple of minutes. Much agreed. I assume that the process needs to be repeated or it probably would be simpler just to rip out what I wanted using regular expressions with shell utilities. > If you do it a lot, you will have to find a way to make the access > efficient for your specific use case. So the file format doesn't matter > either, because the data will most likely end up in a fast data base after > reading it in sequentially *once*, just as in the case above. If the data is just going to end up in a database anyway; then why not send it as a database to begin with and save the trouble of having to convert it? -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
On 2010-12-26, Nobody wrote: > On Sun, 26 Dec 2010 01:05:53 +, Tim Harig wrote: > >>> XML is typically processed sequentially, so you don't need to create a >>> decompressed copy of the file before you start processing it. >> >> Sometimes XML is processed sequentially. When the markup footprint is >> large enough it must be. Quite often, as in the case of the OP, you only >> want to extract a small piece out of the total data. In those cases, >> being forced to read all of the data sequentially is both inconvenient and >> and a performance penalty unless there is some way to address the data you >> want directly. > > OTOH, formats designed for random access tend to be more limited in their > utility. You can only perform random access based upon criteria which > match the format's indexing. Once you step outside that, you often have to > walk the entire file anyhow. That may be true and it may not. Even assuming that you have to walk through a large number of top level elements there may be an advantage to being able to directly access the next element as opposed to having to parse through the entire current element once you have determined it isn't one which you are looking for. To be fair, this may be invalid preoptimization without taking into account how the hard drive buffers; but, I would suspect that there is a threshold where the amount of data skipped starts to outweigh the penalty of overreaching the hard drives buffers. -- http://mail.python.org/mailman/listinfo/python-list
Re: Trying to parse a HUGE(1gb) xml file
Tim Harig, 26.12.2010 02:05: On 2010-12-25, Nobody wrote: On Sat, 25 Dec 2010 14:41:29 -0500, Roy Smith wrote: Of course, one advantage of XML is that with so much redundant text, it compresses well. We typically see gzip compression ratios of 20:1. But, that just means you can archive them efficiently; you can't do anything useful until you unzip them. XML is typically processed sequentially, so you don't need to create a decompressed copy of the file before you start processing it. Sometimes XML is processed sequentially. When the markup footprint is large enough it must be. Quite often, as in the case of the OP, you only want to extract a small piece out of the total data. In those cases, being forced to read all of the data sequentially is both inconvenient and and a performance penalty unless there is some way to address the data you want directly. So what? If you only have to do that once, it doesn't matter if you have to read the whole file or just a part of it. Should make a difference of a couple of minutes. If you do it a lot, you will have to find a way to make the access efficient for your specific use case. So the file format doesn't matter either, because the data will most likely end up in a fast data base after reading it in sequentially *once*, just as in the case above. I really don't think there are many important use cases where you need fast random access to large data sets and cannot afford to adapt the storage layout before hand. Stefan -- http://mail.python.org/mailman/listinfo/python-list
Re: lxml etree question
On Dec 24, 10:17 am, Adam Tauno Williams
wrote:
> On Fri, 2010-12-24 at 20:48 +0530, Nitin Pawar wrote:
> > On Fri, Dec 24, 2010 at 8:40 PM, Jim wrote:
> > Hello, I wonder if someone knows about lxml.etree and
> > namespaces?
>
> Yes, and don't.
He's using lxml.etree (which is a third-party library that mimics
ElementTree's interface), not ElementTree. Were you aware of this?
> > I want to build an ElementTree where some of the sub-elements have
> > attributes that serialize this way.
> > ..
> > I've tried just comment_elet.set('xml:lang','de') and it didn't like
> > that at all (although it takes comment_elet.set('auth:id','jones')
> > just fine). I've also spelunked the docs and googled but have not hit
> > on the right invocation. If someone knows, I'd be grateful.
>
> I'd *strongly* recommend using ElementFlow for building XML documents
> (over ElementTree), especially if namespaces are involved. ElementFlow
> is far more intuitive.
>
I'd have to disagree with the use of strong recommendation here. The
library you recommended isn't a general replacement for lxml (or
ElementTree), and you didn't qualify the conditions for when it is a
suitable alternative.
A. What if he needed to keep the tree in memory?
B. This library builds the tags with "with" statements, which could be
convenient for xml files with rigid structure, but I would think it'd
be inconvenient if the format were relatively loose.
If you're going to recommend a more specialized solution, you should
also give the conditions for which it is suitable.
Carl Banks
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
On Dec 25, 6:21 am, Robert Kern wrote: > On 12/24/10 4:24 AM, Steven D'Aprano wrote: > > > On Thu, 23 Dec 2010 22:38:05 -0800, Carl Banks wrote: > >> OTOH, going the extra mile to hide useful information from a user is > >> asinine. As a user, I will decide for myself how I want to use > >> implementation-defined information, and I don't want the implementor to > >> decide this for me. It's bad enough if an implementor fails to provide > >> information out of laziness, but when they deliberately do extra work to > >> hide information, that's self-importance and arrogance. > > > But that of course is nonsense, because as the user you don't decide > > anything of the sort. The developer responsible for writing the function > > decides what information he provides you, starting with whether you get > > an exception at all, where it comes from, the type of exception, and the > > error message (if any). > > Carl isn't arguing that the user is or should be responsible for this sort of > thing. He is arguing that developers should be responsible for doing this in > such a way that is beneficial for the developer/user down the road. I'm not even arguing that; I think I would be content if the developer merely doesn't actively work to harm the user. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: How to pop the interpreter's stack?
On Dec 25, 2:49 pm, Steven D'Aprano wrote: > But that's a separate issue from what is being discussed here. What we're > discussing here is the idea that a function should be able to delegate > work to private subroutines without the caller being aware of that fact. I can't fathom any possible reason why this could be considered a good thing, especially in Python where the culture is, "We're all adults here". By the same logic, it would be a good idea to prevent the user from accessing private members of a class, internal variables of a module, or importing an internal module directly. There is a convention in Python: internal objects are preceded by underscore. The fact that your internal function is marked this way in the traceback is more than enough information to the user that this is an internal function. Python is not, and never has been, about hiding internal details. It's about openness, and there's no reason a traceback should hide internal details any more than a class should--in fact hiding information in the traceback is far worse, because you're suppressing information that could be crucial for debugging. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
