Re: is laziness a programer's virtue?
Xah Lee wrote: > [SOME FURTHER TROLLING DISTRIBUTED TO MULTIPLE NEWSGROUPS] For those who are relatively new here in comp.lang.python and not aware of the troll: It seems, that Xah still haven't learned from the impact of past abuse reports on his Internet access and tries again to pollute Usenet after a longer time of inactivity. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: clean up html document created by Word
jd wrote: > I am looking for python code (working or sample code) that can take an > html document created by Microsoft Word and clean it up (if you've > never had to look at a Word-generated html document, consider yourself > lucky ;-) Alternatively, if you know of a non-python solution, I'd > like to hear about it. > > Thanks... > > -- jeff > There is a Microsoft add-on for Word which helps to reduce the mess called 'HTML filter'. Go for it here: http://www.microsoft.com/downloads/details.aspx?FamilyID=209ADBEE-3FBD-482C-83B0-96FB79B74DED&displaylang=EN run it and then use afterwards the other in this thread suggested 'cleaning' methods. Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: image processing
edurand wrote: > Hi, > > We are are pleased to announce the version 3.0 of the image processing > library 'Filters'. > You can use it in Python, and we have provided tutorials and samples > in Python, with for exemple conversion from/to PIL image format. > > Have a look at : http://filters.sourceforge.net/ > > it's open source of course > > regards > edurand > In the context of this posting it would be interesting for me (and maybe also to others) to know: What was the actual motivation behind the project? What are other image processing packages missing, that is worth to be covered by a separate package? (i.e. I checked http://filters.sourceforge.net/ out, but haven't yet found the mentioned questions answered ) Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: C parsing fun
Károly Kiripolszky wrote: > You're right, thank you for the comment! I will look after how to > avoid this. And after you have resolved this 'small' ;-) detail you will probably notice, that some full functional and in wide use being parser have still trouble with this ... Claudio > > Marc 'BlackJack' Rintsch írta: >> In <[EMAIL PROTECTED]>, >> karoly.kiripolszky wrote: >> >>> and the great thing is that the algorithm can be used with any >>> language that structures the code with brackets, like PHP and many >>> others. >> But it fails if brackets appear in comments or literal strings. >> >> Ciao, >> Marc 'BlackJack' Rintsch > -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: pyfuzzylib 0.1.3 Released
nelson - wrote: > PyFuzzyLib is a library for fuzzy inference engine building. Using > pyfuzzylib you can add fuzzy logic to your programs. > The program is in it early stage of development, > but it is still usable. Curious why after it will become mature it won't be usable any more ;-) ? > Every sort of feedback is appreciated! Hope so ... :) Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching for patterns on the screen
Jerry Hill wrote: > Hello all, > > I have a piece of code I could use some help optimizing. What I'm > attempting to do is periodically grab a screenshot, and search for 2D > patterns of black pixels in it. I don't care about any color other > than black. Here's some simple code that simulates my worst-case > scenario, scanning the whole screen for a sequence that does not > exist: > > import ImageGrab # From the PIL library > > def removeColor(rgb): >r, g, b = rgb >return (r == 0 and g == 0 and b == 0) > > BMP = ImageGrab.grab.getdata() > x = map(removeColor, BMP) > > The idea is to search for sequences of black pixels on a background > that can change colors. To that end, I transform the screengrab into > a sequence of either black or nonblack pixels as I search them. Note > that in my actual code, I use imap so I only transform pixels as I > search them, and I'm using the KnuthMorrisPratt algorithm from > http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/117214 to do > the actual searching. > >> From some testing using the timeit module: > > map(None, BMP) takes about 0.6 seconds on a 1600x1200 screengrab. > map(removeColor, BMP) takes about 1.5 seconds. > > I'd love to speed things up, if possible. It seems like what I'm > doing would probably be a fairly well defined problem, but I don't > know enough about the field to even know where to start my research, > so even a list of keywords that would point me to discussion of > similar topics would be welcome. > > This is being done in Python 2.5c2 using PIL 1.1.5 > Use PIL for grabbing the screenshot and numarray for processing the image. See http://groups.google.com.vc/group/comp.lang.python/browse_thread/thread/6207e7526fb6fdc6/a05646969d59102e for some further helpful hints towards speeding things up. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Scientific computing and data visualization.
Paul F. Kunz wrote: > "Fie Pye" <[EMAIL PROTECTED]> writes: > > >>Hallo >> >> I would like to have a high class open source tools for scientific >> computing and powerful 2D and 3D data visualisation. Therefore I chose >> python, numpy and scipy as a base. Now I am in search for a visualisation >> tool. I tried matplotlib and py_opendx with OpenDx. OpenDx seems to me very >> good but the project py_opendx looks like closed. After py_opendx >> instalation and subsequent testing I got an error that needs discussion with >> author or an experienced user. Unfortunately a mail to author returned as >> undeliverable. >> >> Does anybody now about suitable visualisation tool? >> > > Have you looked at HippoDraw? > > http://www.slac.stanford.edu/grk/ek/hippodraw http://www.slac.stanford.edu/grp/ek/hippodraw/ Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Unit balancing
[EMAIL PROTECTED] wrote: > Greetings! > > I am working with a researcher who would be very happy if he could > include units in his calculations. Then, if his complicated expression > didn't result in "kg/yr" the program would stop and point out his > error. > > Does Python (or SciPy or ..) offer this feature? > > Thanks for your help. > > >Abraham > I mean I run one time already into a Python library or a class which was designed to handle such things as this request is common to many researcher. Sorry, don't have any link at my fingertips or even an idea where to look after it, so Googling or waiting until someone else using such a tool posts here is all I can offer. Inbetween Frink is maybe an option? http://futureboy.homeip.net/frinkdocs/ Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for the Perfect Editor
Omar wrote: > thanks for all the responses... > > I'm liking the way ulipad and scite work > > I'd switch to ulipad entirely if I canget to recognize the python > interpreter in the preferences menu. I can't seem to figure that out. It is usually correctly set. Just start Preferences, go to Python tab and push the button [Setup Python interpreter]. You should see the default interpreter setting and modify it if necessary (button [modify]). What is the problem with it exactly? > > Also, I'm highly interested in getting the snippets manager working. What is your problem with it? > > any ideas? I suggest you play a bit around with the menu items dedicated to Snippets. Don't forget to click on the visible entries to get the other content related to them visible (maybe this is your problem - you expect to see more, but have to select first - not very intuitive in case you have generated only one entry and wonder why its details are not visible in the other text boxes). If there are no any entries just [Add] some. And to see how to work with them go to menu 'Windows' and 'Open Snippets Window') to get the available ones displayed (I have got them working knowing nothing about them before while I was writing my first response here, so it can't be that hard ;) > If the above does not help, best email directly the author of UliPad (and greet him from me :) and describe your problem in a very easy to understand and as simple English as possible. Usually you will get a very fast and a very informative response (but please don't care much about correct English). Also bug reports (or best directly the patches) are very welcome. Best if you were able to provide your own notes from what you have learned as a kind of contribution to a better future documentation. If nothing suddenly changed, I can assure you, that you can't get anywhere else better support directly from the author as it is currently available for UliPad. And as it is all Python, so you are free to learn from it or try out your own ideas for extensions. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for the Perfect Editor
Omar wrote: > thanks for the suggestions, fellas > Would be kind of you to tell us which one you have decided to use and why? Claudio Grondi P.S. If you don't like any of already mentioned you can give SciTe a try. -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: Leo 4.4.1.1 final released
Edward K. Ream wrote: > Leo 4.4.1.1 final September 3, 2006 > Leo 4.4.1.1 final is available at: > http://sourceforge.net/project/showfiles.php?group_id=3458&package_id=29106 Doesn't run on my system (installed with the .exe): Python 2.4.2 Microsoft Windows XP [Version 5.1.2600] (C) Copyright 1985-2001 Microsoft Corp. C:\>D:\SftwLib\Leo\src\leo.py leoID = CGr (in D:\SftwLib\Leo\config) reading settings in D:\SftwLib\Leo\config\leoSettings.leo Traceback (most recent call last): File "D:\SftwLib\Leo\src\leo.py", line 320, in ? run() File "D:\SftwLib\Leo\src\leo.py", line 95, in run g.app.config.readSettingsFiles(fileName,verbose) File "D:\SftwLib\Leo\src\leoConfig.py", line 1341, in readSettingsFiles c = self.openSettingsFile(path) File "D:\SftwLib\Leo\src\leoConfig.py", line 1311, in openSettingsFile c,frame = g.app.newLeoCommanderAndFrame(path,updateRecentFiles=False) File "D:\SftwLib\Leo\src\leoApp.py", line 568, in newLeoCommanderAndFrame c.finishCreate() File "D:\SftwLib\Leo\src\leoCommands.py", line 181, in finishCreate def finishCreate (self): # New in 4.4. File "D:\SftwLib\Leo\src\leoCommands.py", line 187, in finishCreate c = self ; p = c.currentPosition() TypeError: currentPosition() takes exactly 2 arguments (1 given) --- ??? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Looking for the Perfect Editor
Omar wrote: > I'd love the perfect editor that would be: > > a) free > > b) enable me to drag and drop code snippets from a sort of browser into > the code > > c) can run programs right from within > > d) can edit > - PYTHON > - Javascript > - HTML > - actionscript (since I'm also learning flash) > > e) easy to learn > > suggestions? > Welcome to the group of users of UliPad :) http://wiki.woodpecker.org.cn/moin/UliPad a) ok b) by double click (code snippets) c) ok e) maybe not that easy because of lack of full documentation, but sure worth to try, anyway. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting text from a string
Tempo wrote: > This is the output I get: > > > > >>>>prices > > [ > $14.99 > , > $27.99 > , > $66.99 > , > $129.99 > , > $254.99 > ] > > I think, guessing your knowledge level in Python, the best solution for you to start with and learn some Python basics will be : >>> prices.split()[2] '$14.99' >>> prices.split()[6] '$27.99' >>> prices.split()[10] '$66.99' >>> prices.split()[14] '$129.99' >>> prices.split()[18] '$254.99' Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting text from a string
Tempo wrote: > Okay, so it sounds like I am in the right direction. However, I am not > sure that the text is in a string or some other format becasue the > string is enclosed in "[" and "]", not in ' '. > In case you have it like ['the string'], the actual string will be: ['the string'][0]. Just use IDLE to test your ideas: >>> ['the string'][0] 'the string' >>> Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Extracting text from a string
Tempo wrote: > Hello. I am having a little trouble extracting text from a string. The > string that I am dealing with is pasted below, and I want to extract > the prices that are contained in the string below. Thanks in advanced > for any and all help. Thank you. > > > > > $14.99 > , > $27.99 > , > $66.99 > , > $129.99 > , > $254.99 > > What have you tried? Taking a look into regular expressions in the Python tutorial (4.2 re -- Regular expression operations) should give you all you need. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Printing Documents
defcon8 wrote: > How can I print html documents in Python on Windows? > ;-) 1. You start the Microsoft Internet Explorer and make some space on the Desktop for a Python shell window. 2. You start the Python shell and position it within the prepared space 3. Still looking at the Python shell you load the document into the Internet Explorer and then use its menu [File] -> [Print] to invoke printing (don't forget to keep your eye on the Python shell...) ;-) Claudio Grondi P.S. There are many options to automate usage of MSIE from Python, so maybe if you explain what you actually intend to achieve, someone else can give you a better advice. -- http://mail.python.org/mailman/listinfo/python-list
Re: Using Beautiful Soup to entangle bookmarks.html
Diez B. Roggisch wrote:
> Francach schrieb:
>
>> Hi,
>>
>> I'm trying to use the Beautiful Soup package to parse through the
>> "bookmarks.html" file which Firefox exports all your bookmarks into.
>> I've been struggling with the documentation trying to figure out how to
>> extract all the urls. Has anybody got a couple of longer examples using
>> Beautiful Soup I could play around with?
>
>
> Why do you use BeautifulSoup on that? It's generated content, and I
> suppose it is well-formed, most probably even xml. So use a standard
> parser here, better yet somthing like lxml/elementtree
>
> Diez
Once upon a time I have written for my own purposes some code on this
subject, so maybe it can be used as a starter (tested a bit, but
consider its status as a kind of alpha release):
from urllib import urlopen
from sgmllib import SGMLParser
class mySGMLParserClassProvidingListOf_HREFs(SGMLParser):
# provides only HREFs for links to another pages skipping
# references to:
# - internal links on same page : "#..."
# - email adresses : "mailto:...";
# and skipping part with appended internal link info, so that e.g.:
# - "LinkSpec#internalLinkID" will be listed as "LinkSpec" only
# ---
# reset() overwrites an empty function available in SGMLParser class
def reset(self):
SGMLParser.reset(self)
self.A_HREFs = []
#: def reset(self)
# start_a() overwrites an empty function available in SGMLParser class
# from which this class is derived. start_a() will be called each
time the
# SGMLParser detects an tag within the feed(ed) HTML document:
def start_a(self, tagAttributes_asListOfNameValuePairs):
for attrName, attrValue in tagAttributes_asListOfNameValuePairs:
if attrName=='href':
if attrValue[0] != '#' and attrValue[:7] !='mailto:':
if attrValue.find('#') >= 0:
attrValue = attrValue[:attrValue.find('#')]
#: if
self.A_HREFs.append(attrValue)
#: if
#: if
#: for
#: def start_a(self, attributes_NamesAndValues_AsListOfTuples)
#: class mySGMLParserClassProvidingListOf_HREFs(SGMLParser)
#
--
# ---
# Execution block:
fileLikeObjFrom_urlopen = urlopen('www.google.com') # set URL
mySGMLParserClassObj_withListOfHREFs =
mySGMLParserClassProvidingListOf_HREFs()
mySGMLParserClassObj_withListOfHREFs.feed(fileLikeObjFrom_urlopen.read())
mySGMLParserClassObj_withListOfHREFs.close()
fileLikeObjFrom_urlopen.close()
for href in mySGMLParserClassObj_withListOfHREFs.A_HREFs:
print href
#: for
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: 3 Simple Questions About Python/IDLE
Omar wrote: > 1) why don't python / idle use numbered lines in their scripting, like > basic? how do you keep track of large batches of code without them? > > 2) in IDLE, how do you save a program such that it can be run, say from > windows the "run" function? > > 3) are most of you doing your script editing in IDLE or something more > fully featured? > > 4) how are you importing previously written blocks of code into your > code? cut and paste? what is the best way to organize snippets of > code? > > thanks, peeps > 1) Python is very different from what you expect being experienced in programming with Basic. My recommendation: don't hesitate to take the time to read the Python manual. 2) Double click on saved Python script (hope you are able to save a file? Haven't you detected that you can edit and run a file in IDLE?) 3) Plain text editor or IDLE are good. For more see Python website - there are plenty options to choose from. 4) using 'import' (see recommendation in 1)) Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: IronPython 1.0 - Bugs or Features?
Super Spinner wrote:
> IronPython is a .NET language, so does that mean that it invokes the
> JIT before running actual code? If so, then "simple short scripts"
> would take longer with IronPython "busy starting itself" loading .NET
> and invoking the JIT. This effect would be less noticable, the longer
> the program is. But I'm just guessing; I've not used IronPython.
>
The time of loading IronPython seem to pay out at the end if the script
takes a longer time to run, so you are most probably right. I am a bit
surprised, that the difference is not that big (e.g. at least half the
time) as I have expected from a JIT concept ... :
# import psyco
# psyco.full()
def arccot(x, unity):
sum = xpower = unity // x
n = 3
sign = -1
while 1:
xpower = xpower // (x*x)
term = xpower // n
if not term:
break
sum += sign * term
sign = -sign
n += 2
return sum
def pi(digits):
print ' start of setting unity value ...',
unity = 10**(digits + 10)
print ' set unity value, starting arccot() ... ',
pi = 4 * (4*arccot(5, unity) - arccot(239, unity))
return pi // 10**10
f = file("pi-decimal-10digits.out","wb")
f.write(str(pi(10)))
print """
PC: Pentium 4, 2.8 GHz, Windows XP SP2
writing 100.000 digits of Pi to a file takes using:
CPython 2.4.2 : 2 min 41 s (of CPU time)
CPython+Psyco : 2 min 45 s (of CPU time)
IronPython 1.0 : 1 min 48 s (of CPU time)
"""
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: IronPython 1.0 - Bugs or Features?
tjreedy wrote: > "Claudio Grondi" <[EMAIL PROTECTED]> wrote in message > news:[EMAIL PROTECTED] > >>I also erroneously assumed, that the first problem was detected during >>parsing ... so, by the way: how can I distinguish an error raised while >>parsing the code and an error raised when actually running the code? > > > Parsing detects and reports syntax errors and maybe something else if you > use non-ascii chars without matching coding cookie. Other errors are > runtime. Let's consider print '"Data ê"' In CPython 2.4.2 there is in case of non-ascii character: sys:1: DeprecationWarning: Non-ASCII character '\xea' in file C:\IronPython-1.0-BugsOrFeatures.py on line 3, but no encoding declared; see http://www.python.org/peps/pep-0263.html for details "Data♀♂ Û" IronPython does not raise any warning and outputs: "Data♀♂ ?" So it seems, that IronPython is not that close to CPython as I have it expected. It takes much more time to run this above simple script in IronPython as in CPython - it feels as IronPython were extremely busy with starting itself. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: IronPython 1.0 - Bugs or Features?
Marc 'BlackJack' Rintsch wrote: > In <[EMAIL PROTECTED]>, Claudio Grondi wrote: > > >>The context: >> C:\IronPython> ipy.exe >> IronPython 1.0.60816 on .NET 2.0.50727.42 >> Copyright (c) Microsoft Corporation. All rights reserved. >>vs. >> C:\Python24> python.exe >> Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] >>on win32 >> >>IronPython raises "UnboundLocalError: local variable 'strData' >>referenced before assignment" error in following case: >> >> while(someCondition): >> try: >> strData = strSomeValue() >> except: >> pass >> if( type(strData) == str ) : ### <<< HERE THE ERROR >> doSomething() >> >>CPython 2.4.2 doesn't raise an error with same code. > > > Well I get a `NameError` for `someCondition`. Please post a minimal > *working* example that produced the error. I can't as after constructing one according to what I have thought was the cause of it I detected that the error was caused by the exception in the line strData = strSomeValue() (see also my other posting) I have just misinterpreted the origin of the "UnboundLocalError:" as raised while parsing and was wondering how it comes, that the parser is able to detect such things ... Sorry for the eventually caused confusion. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: IronPython 1.0 - Bugs or Features?
Larry Bates wrote: > Claudio Grondi wrote: > >>(just wanted to share my experience with IronPython 1.0) >> >>The context: >> C:\IronPython> ipy.exe >> IronPython 1.0.60816 on .NET 2.0.50727.42 >> Copyright (c) Microsoft Corporation. All rights reserved. >>vs. >> C:\Python24> python.exe >> Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] >>on win32 >> >>IronPython raises "UnboundLocalError: local variable 'strData' >>referenced before assignment" error in following case: >> >>while(someCondition): >> try: >>strData = strSomeValue() >> except: >>pass >> if( type(strData) == str ) : ### <<< HERE THE ERROR >>doSomething() >> >>CPython 2.4.2 doesn't raise an error with same code. >> >>As strData is not set anywhere before in the code it seems, that >>IronPython is somehow right, but I am not sure if it is a bug or a feature. >> >>Another problem with IronPython where CPython 2.4.2 runs ok was while I >>was trying to do: >> f = file(r'\\.\PhysicalDrive0', 'rb') >>getting "ValueError: FileStream will not open Win32 devices such as disk >>partitions and tape drives. Avoid use of "\\.\" in the path." >>Here the same - I am not sure if it is a bug or a feature. >> >>Can someone knowledgeable elaborate on it a bit please? >> >>Claudio Grondi > > > Your problem is a blanket exception handler that ignores > the exception. Blanket exceptions are almost always a > bad idea. Blanket exceptions with pass as the only > command in the except: block is ALWAYS a bad idea. > > Basically the line strData = strSomeValue() caused an > exception. Since it was inside a try: block, it then > executed what was in the except: block. Since all that > was in the except: block was pass, it just fell through > to the if statement. At that point strData is not > defined because the try block failed and never create > strData object. > > It is doing EXACTLY what you told it to do. Sorry for the confusion caused, but you are right. The actual code was a bit more complex, so I tried to get it down to the principle, but haven't expected, that f = file(r'\\.\PhysicalDrive0', 'rb') buried within strSomeValue() throws an exception as in CPython the code was running ok. So the second problem was the cause also for the first one ... I also erroneously assumed, that the first problem was detected during parsing ... so, by the way: how can I distinguish an error raised while parsing the code and an error raised when actually running the code? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
IronPython 1.0 - Bugs or Features?
(just wanted to share my experience with IronPython 1.0) The context: C:\IronPython> ipy.exe IronPython 1.0.60816 on .NET 2.0.50727.42 Copyright (c) Microsoft Corporation. All rights reserved. vs. C:\Python24> python.exe Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on win32 IronPython raises "UnboundLocalError: local variable 'strData' referenced before assignment" error in following case: while(someCondition): try: strData = strSomeValue() except: pass if( type(strData) == str ) : ### <<< HERE THE ERROR doSomething() CPython 2.4.2 doesn't raise an error with same code. As strData is not set anywhere before in the code it seems, that IronPython is somehow right, but I am not sure if it is a bug or a feature. Another problem with IronPython where CPython 2.4.2 runs ok was while I was trying to do: f = file(r'\\.\PhysicalDrive0', 'rb') getting "ValueError: FileStream will not open Win32 devices such as disk partitions and tape drives. Avoid use of "\\.\" in the path." Here the same - I am not sure if it is a bug or a feature. Can someone knowledgeable elaborate on it a bit please? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: OO on python real life tutorial?
Patrick Thomson wrote: >>I personally don't like wxPython because the way it works is very >>counter intuitive for me and appear to me somehow non-Pythonic > > > While Claudio has a point (wxPython is a C++ library at heart), I > believe that wxPython is the best solution for Python GUI's out there. > TK may be a lot easier, but with a little practice wxPython becomes > very clear, and the skills you learn therein will be applicable to > almost any other GUI toolkit out there (Swing and SWT were far, far > easier to learn thanks to my wxPython experience.) After all, GvR said > that "wxPython is the best and most mature cross-platform GUI toolkit, > given a number of constraints. The only reason wxPython isn't the > standard Python GUI toolkit is that Tkinter was there first." > > The Wax toolkit (http://zephyrfalcon.org/labs/wax.html) attempts to > solve the problems inherent with wxPython and make it more pythonic. > I've never used it, so I don't know if it succeeds. > > >>wxPython is like programming in Microsoft Visual Basic or Visual C++ : >>some love it, some don't. > > > Though that is true, I don't think that the analogy holds up at all. > The power of wxPython lies in its attractiveness across all platforms > and the freedom that it gives you; VB and VC++ lock you in what > Microsoft want you to do. (To be fair, that isn't a Bad Thing by > definition.) > > >>>I want to switch to wxpython because I don't like Tk too much. It is >>>difficult sometimes to have full control of the widgets (i.e. focus >>>sequence). Why do you think wxwidget is not suitable for low-medium >>>size projects? Could you give me some practical examples of this? > > > I respectfully disagree with Grondi's opinions; a list of notable > wxPython projects (http://wiki.wxpython.org/index.cgi/wxPythonPit_Apps) > showcases projects both small and expansive. What I can't understand here is, with which of my opinions you disagree as one line below of my posting (you are responding to) I write: "I haven't said, that it is not suitable [for low-medium size projects]." ... By the way: why is it so hard to develop a wxPython application which runs with any wxPython version? Are the subsequent wxPython versions by definition not compatible or do only the quirks change from release to release being shifted between the problem zones? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: pictures as characters in a Tk text box?
Jay wrote: > Can you do the same in wxPython? I am a greenhorn in wxPython, so please don't bet on what I state here, but to my current knowledge wxPython doesn't come with a widget capable of what Tkinter text box is capable of. But wxPython comes e.g. with a widget displaying any HTML content, so if you need only to display and not edit things it is a perfect one to use if you need to mix text with images and control elements. Claudio Grondi > > Simon Forman wrote: > >>Jay wrote: >> >>>This may be really obscure, but I had a dream about programming >>>something like this, so don't blame me. Is it possible to take a small >>>image or icon and display it as a character in a Tk text box? Think >>>how Thunderbird displays text smilies as actual smiley icons. Or how >>>in AIM as you type a smiley, it replaces it with the picture. Does >>>anyone know how to display these specifically in a Tk text box? Thanks. >> >>Dreaming in Python, eh? Maybe you should take a break. ;-) >> >>Sure, quoting the link below: "You can put an image or bitmap into a >>text widget. It is treated as a single character whose size is the >>natural size of the object." >> >>You can also embed any Tk widget in a Tk Text widget, including >>ImageTks and widgets with bitmaps. >> >>See: >>http://infohost.nmt.edu/tcc/help/pubs/tkinter/text-image.html >>and >>http://infohost.nmt.edu/tcc/help/pubs/tkinter/text-window.html >> >> >>HTH, >>~Simon > > -- http://mail.python.org/mailman/listinfo/python-list
Re: OO on python real life tutorial?
filippo wrote: > thanks Fredrik and Claudio, > > probably structured coding paradigm is what I need. Claudio, could you > explain better your sentence below? > > Claudio Grondi ha scritto: > >>Python/Tk for it in order to avoid programming in wxPython if not really >>necessary (wxPython has its strengths with growing project sizes, but >>yours is still far below the threshold). I personally don't like wxPython because the way it works is very counter intuitive for me and appear to me somehow non-Pythonic (I warned you, I am a bit biased to have something against OO, especially I mean, that the concept of inheritance produces more confusion than it helps). You are not in a big project with many programmer who must work together - in my eyes the only context in which OO starts to make sense. You also don't need something what Tk does not come with to be forced to switch to wxPython (e.g. in order to use Scintilla). wxPython is like programming in Microsoft Visual Basic or Visual C++ : some love it, some don't. Nothing comes at no cost, so when from the one side you don't need to care about some portions of code on the other side you loose insight into them and in case of bad luck you will spend days/weeks on resolving things and bother much other experts to help you, where in case of not going into it you were able to resolve your problems yourself. > I want to switch to wxpython because I don't like Tk too much. It is > difficult sometimes to have full control of the widgets (i.e. focus > sequence). Why do you think wxwidget is not suitable for low-medium > size projects? Could you give me some practical examples of this? I haven't said, that it is not suitable. Maybe you consider it from this point of view: The best programming environment is this one you know best. You don't like Tk because you have learned about it quirks. So you hope to escape this using another set of tools coming with a bunch of another quirks you probably don't know about. If you want to go into the effort to learn about them anyway, why not? Maybe you have the luck and don't spend hours/days/weeks on debugging things buried deep in the OO inheritance stuff and hidden this way from your direct insight? Good luck! Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: OO on python real life tutorial?
filippo wrote: > Hello, > > I coded my +10k lines app using Perl/Tk. It is something like a hotel > software manager, it has a bunch of windows to manage the arrivals, > bills etc etc. I want to port this on Python/WxPython but I'd like to > get benefit of python, not just doing a row by row raw porting. > > My problem is that I cannot figure out how OO could be useful in my > case. Be warned, that I am a bit biased here and my opinion is probably not mainstream, but I think OO won't be useful in your case, so there is no reason to learn about it. A row by row porting will be ok, but if I were you, I would use probably Python/Tk for it in order to avoid programming in wxPython if not really necessary (wxPython has its strengths with growing project sizes, but yours is still far below the threshold). Consider it that way: if you cannot figure out how OO could be useful, just be more self-confident and accept the enlightenment, that it probably or even for sure can't. Claudio Grondi Is there a OO tutorial out there to help me? > > Thanks, > > Filippo > -- http://mail.python.org/mailman/listinfo/python-list
Re: simultaneous copy to multiple media
[EMAIL PROTECTED] wrote: > I need a program that simultaneously can copy a single file (1 GB) from > my pc to multiple USB-harddrives. Why not just use: copy c:\file.ext u:\file.exe in one shell and in another copy c:\file.ext v:\file.exe where u: and v: are the USB drives? There is usually not much gain on USB when on Windows, especially in case of a single file which fits into memory (and the file cache) to do it 'simultaneously'. This subject was discussed here already in the past: http://mail.python.org/pipermail/python-list/2005-March/271985.html Claudio Grondi > > I have searched the internet and only found one program that does this > on > http://mastermind.com.pl/multicopy/ > > but this link doesnt work anymore somebody that can help me, is > there any other programs out there. > -- http://mail.python.org/mailman/listinfo/python-list
Re: How to let a loop run for a while before checking for break condition?
Sorin Schwimmer wrote: > I am thinking on something in the following form: > > > import time > import thread > > delay=True > > def fn() > global delay > time.sleep() > delay=False > > thread.start_new_thread(fn,()) > > while delay: > > > ... > > while : > > > ... > > > Or, if you need to use "break", the second loop may be > something like: > > > while True: > > > ... > if : break > > ... > > > The two while loops have the same content, but the > first is terminated after a pre-determined amount of > time, while the second by another condition. Usually > the content of the too loops, being the same, is a > call to a function that does the actual work. In your > case, as time seems to be critical, you don't want to > spend it in function-call overheads, so you repeat > (cut'n'paste) the relevant code. > > Of course, the price to be paid is in maintenance > headache: you'll have to make all the changes in both > loops, to keep consistency. > > I hope this helps. It doesn't. Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: How to let a loop run for a while before checking for break condition?
Sorin Schwimmer wrote: > to Fredrik Lundh > I'm afraid Claudio Grondi can't use your solution, as > he needs it hosted on Windows, which lacks > signal.alarm. > > to Claudio Grondi > How about splitting your loop in two? The first loop > would check for your boolean, which is changed by your > timer, the second loop will check for your "normal" > exit condition? I don't understand your proposal. Do you mean using another thread? I suppose, that wxPython provides something like what I am looking for, but using it only to speed up a Python loop is sure not an option. Concluding from lack of responses related to a Windows version of Fredriks solution I suppose, that there is no way to achieve it with standard Python on Windows. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: time.clock() going backwards??
Rob Williscroft wrote: > Claudio Grondi wrote in news:[EMAIL PROTECTED] in > gmane.comp.python.general: > > >>Tim Roberts wrote: >> >>>"Tim Peters" <[EMAIL PROTECTED]> wrote: > > >>>It is much simpler than that. With a multiprocessor HAL, including >>>on a dual-core or hyperthreaded system, QueryPerformanceCounter >>>returns the raw cycle counter (RDTSC). However, on Windows XP, the >>>operating system does not synchronize the cycle counters on multiple >>>processors, and they can be actually be millions of cycles apart. >>> >>>This was a change from previous systems. On NT4 and Win2000, the >>>operating actually rewrote the cycle counters on the second (and >>>beyond) processors to align them to the first processor, so the delta >>>was usually only a dozen or two cycles. XP does not appear to do >>>that. I think that is a huge mistake, since it renders >>>QueryPerformanceCounter non-monotonic. >> >>How does it come, that processors on same mainboard run at different >>speeds? Do they have separate clock-pulse generators? > > > I don't see any claim above that they run at different speeds, only > that the counters are several million cycles apart, IOW running at the > same speed but with different initial values, or more likely starting > at different times. > > For processors that run at (say) 2GHz, several million (say 10 million) > represents a difference of 10e6/2e9 = 0.005 seconds between when the > processors were sufficiently powered up to start counting cycles. > > Rob. If it were so, than why can't the delta of time between the processors be set to exact zero? I assume, that it is known how many cycles adjusting the value will take, so it could be done exactly ... hmmm ... Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Misleading error message when opening a file (on Windows XP SP 2)
Fredrik Lundh wrote:
> Tim Peters wrote:
>
>
>>>Traceback (most recent call last):
>>> File "", line 1, in -toplevel-
>>> f = file('veryBigFile.dat','r+')
>>>IOError: [Errno 2] No such file or directory: 'veryBigFile.dat'
>>>
>>>Is it a BUG or a FEATURE?
>>
>>Assuming the file exists and isn't read-only, I bet it's a Windows
>>bug, and that if you open in binary mode ("r+b") instead I bet it goes
>>away (this wouldn't be the first large-file text-mode Windows bug).
>
>
> however, if you use the C level API, you get EINVAL (which presumably means
> that the CRT cannot open this file in text mode), not ENOENT. this is also
> true
> for older versions of Python:
>
> Python 2.1.1 (#20, Aug 23 2001, 11:27:17) [MSC 32 bit (Intel)] on win32
>
>>>>f = open("bigfile.dat")
>>>>f = open("bigfile.dat", "r+")
>
> Traceback (most recent call last):
> File "", line 1, in ?
> IOError: [Errno 22] Invalid argument: 'bigfile.dat'
>
> which probably means that this fix
>
> http://www.python.org/sf/538827
>
> is partially responsible for the misleading error message.
>
> (the cause of this seems to be that when you open a text file for updating,
> the
> CRT check if there's a chr(26) at the end of the file, but the 32-bit lseek
> API
> doesn't support seeking to positions larger than 2^31-2)
>
>
Using MSVC++ .NET 2003 compiler (if I did it all the right way):
fstm = fopen("bigfile.dat","r+");
if(fstm == 0) { printf( " ErrNo: %i \n", errno ); }
// ^-- prints :
// on "r+" with too large file: 22 (EINVAL-Invalid argument)
// on non-existing file : 2 (ENOENT-no such file)
// on bad mode string spec. : 0 (??? why not EINVAL ...)
So there _is_ a way to distinguish the different problems occurred while
opening the file. The error message comes from Python (errnomodule.c),
not from Windows(errno.h). Concluding from this it becomes evident for
me, that this misleading error message is Python fault (even if
originated by misleading errno values set after fopen in the MSVC++
environment and Windows), right?
Probably also in Python 2.5?
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: Misleading error message when opening a file (on Windows XP SP 2)
Tim Peters wrote:
> [Claudio Grondi]
>
>> Here an example of what I mean
>> (Python 2.4.2, IDLE 1.1.2, Windows XP SP2, NTFS file system, 80 GByte
>> large file):
>>
>> >>> f = file('veryBigFile.dat','r')
>> >>> f = file('veryBigFile.dat','r+')
>>
>> Traceback (most recent call last):
>>File "", line 1, in -toplevel-
>> f = file('veryBigFile.dat','r+')
>> IOError: [Errno 2] No such file or directory: 'veryBigFile.dat'
>>
>> Is it a BUG or a FEATURE?
>
>
> Assuming the file exists and isn't read-only, I bet it's a Windows
> bug, and that if you open in binary mode ("r+b") instead I bet it goes
> away (this wouldn't be the first large-file text-mode Windows bug).
I knew already that 'r+b' fixes it. Yes, you have won the bet :) .
I suppose, like you do, that because there is a difference between text
and binary files on Windows and the text files are e.g. opened being
buffered using a 32-bit buffer pointer, this fails on too large NTFS files.
I could also imagine that Python tries to buffer the text file and fails
because it uses the wrong pointer size when asking Windows for the
content. I have not yet looked into the C-code of Python - any hint
which file I should take a closer look at?
Just curious to see for myself, that the bug is on the Windows side.
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: How to let a loop run for a while before checking for break condition?
Hendrik van Rooyen wrote:
> "Claudio Grondi" <[EMAIL PROTECTED]> Wrote:
>
> | Fredrik Lundh wrote:
> | > Diez B. Roggisch wrote:
> | >
> | >> A while loop has a condition. period. The only thing to change that is
> | >> to introduce a uncoditioned loop, and use self-modifying code to make
> | >> it a while-loop after that timer interrupt of yours.
> | >
> | >
> | > or use a timer interrupt to interrupt the loop:
> | >
> | > import signal, time
> | >
> | > def func1(timeout):
> | >
> | > def callback(signum, frame):
> | > raise EOFError # could use a custom exception instead
> | > signal.signal(signal.SIGALRM, callback)
> | > signal.alarm(timeout)
> | >
> | > count = 0
> | > try:
> | > while 1:
> | > count += 1
> | > except EOFError:
> | > for i in range(10):
> | > count += 1
> | > print count
> | >
> | > for an utterly trivial task like the one in that example, the alarm
> | > version runs about five times faster than a polling version, on my test
> | > machine (ymmv):
> | >
> | > def func2(timeout):
> | >
> | > gettime = time.time
> | > t_limit = gettime() + timeout
> | >
> | > count = 0
> | > while gettime() < t_limit:
> | > count += 1
> | > for i in range(10):
> | > count += 1
> | > print count
> | >
> | >
> | >
> |
> | This above is exactly what I am looking for, except it does not work in
> | Microsoft Windows where the signal.alarm() function is not available.
> |
> | So now the only thing I would like to know is how to achieve the same
> | functionality when running Python on a Microsoft Windows box.
> |
> | Claudio Grondi
>
> It looks to me - but I could be wrong - that the time saved here is not
> because
> of the condition test being replaced by the try-except, but because of the
> fact
> that the call to gettime was eliminated - so you may get the most mileage by
> using in line code in your loop that avoids calls to subroutines and simply
> let
> it run and test for the end of the counter...
>
> - Hendrik
>
The test of the counter is what actually slows the loop down. Probably
the test of time slows the loop even more down. Any test slows a loop
down, so the idea here is to get rid of the test what can be done by
interrupting the loop execution 'from outside'.
Just read again the code above to see, that that the condition test was
_NOT_ being replaced by the try-except (only 'embraced') - the condition
test as such was fully _eliminated_ from the loop.
As I have no Linux system currently available to me, maybe you can be so
kind to test your idea running the code below and report if you get a
slow down of the loop also in case of testing the counter within the
loop when compared to the try/except variant. Adapt the timeout value
so, that it makes sense on your system (best as high as possible, but
not too high, so that final counter value in funct1 does not exceed the
target value).
import signal, time
def func1(timeout):
def callback(signum, frame):
raise EOFError # could use a custom exception instead
signal.signal(signal.SIGALRM, callback)
signal.alarm(timeout)
count = 0
try:
while 1:
count += 1
except EOFError:
while True:
count += 1
if count < 0x500:
break
print hex(count)
def func2():
count = 0
while True:
count += 1
if count < 0x500:
break
print hex(count)
print
startTime = time.clock()
funct1(10)
print time.clock() - startTime
print
print
startTime = time.clock()
funct2()
print time.clock() - startTime
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: Misleading error message when opening a file (on Windows XP SP 2)
Marc 'BlackJack' Rintsch wrote:
> In <[EMAIL PROTECTED]>, Claudio Grondi wrote:
>
>
>>Here an example of what I mean
>>(Python 2.4.2, IDLE 1.1.2, Windows XP SP2, NTFS file system, 80 GByte
>>large file):
>>
>> >>> f = file('veryBigFile.dat','r')
>> >>> f = file('veryBigFile.dat','r+')
>
>
> You mention the file size and gave a "speaking" name to that file -- does
> the file size matter?
Yes, it does.
I haven't tested it yet, but I suppose 2 or 4 GByte threshold value.
>
>
>>Traceback (most recent call last):
>> File "", line 1, in -toplevel-
>> f = file('veryBigFile.dat','r+')
>>IOError: [Errno 2] No such file or directory: 'veryBigFile.dat'
>>
>>Is it a BUG or a FEATURE?
>
>
> It's the error number Windows returns for that operation.
So you just try to say:
"it's not Python fault - it's just another bug of the damn Microsoft
Windows operating system", right?
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Misleading error message when opening a file (on Windows XP SP 2)
Here an example of what I mean
(Python 2.4.2, IDLE 1.1.2, Windows XP SP2, NTFS file system, 80 GByte
large file):
>>> f = file('veryBigFile.dat','r')
>>> f = file('veryBigFile.dat','r+')
Traceback (most recent call last):
File "", line 1, in -toplevel-
f = file('veryBigFile.dat','r+')
IOError: [Errno 2] No such file or directory: 'veryBigFile.dat'
Is it a BUG or a FEATURE?
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: time.clock() going backwards??
Tim Roberts wrote: > "Tim Peters" <[EMAIL PROTECTED]> wrote: > > >>[Giovanni Bajo[ >> >>>>I experimented something very strange, a few days ago. I was debugging an >>>>application at a customer's site, and the problem turned out to be that >>>>time.clock() was going "backwards", that is it was sometimes >>>>(randomically) returning a floating point value which was "less than" the >>>>value returned by the previous invokation. The computer was a pretty fast >>>>one (P4 3Ghz I think, running Windows XP), and this happened only between >>>>very close invokations of time.clock(). >> >>[Terry Reed] >> >>>I seem to remember this being mentioned before on the list, perhaps by Tim >>>Peters. Perhaps he will chime in. >> >>No real need ;-) BIOS or HAL bug on a multiprocessor (or maybe even >>hyperthreaded) box is by far the most likely cause (and the only cause >>ever identified for sure by people who followed up). Python's C code >>slinging QueryPerformanceCounter is exceedingly simple, and isn't a >>suspect because of that. It's on the edge of vague possibility that >>Microsoft's compiler generates non-monotonic code for converting >>64-bit integer to double: > > > It is much simpler than that. With a multiprocessor HAL, including on a > dual-core or hyperthreaded system, QueryPerformanceCounter returns the raw > cycle counter (RDTSC). However, on Windows XP, the operating system does > not synchronize the cycle counters on multiple processors, and they can be > actually be millions of cycles apart. > > This was a change from previous systems. On NT4 and Win2000, the operating > actually rewrote the cycle counters on the second (and beyond) processors > to align them to the first processor, so the delta was usually only a dozen > or two cycles. XP does not appear to do that. I think that is a huge > mistake, since it renders QueryPerformanceCounter non-monotonic. How does it come, that processors on same mainboard run at different speeds? Do they have separate clock-pulse generators? I can remember, that (at least on very old motherboards) the clock-pulse generator was a separate element and the processor just used it, so I would expect, that even in case of multiple processors, if there were only one clock-pulse generator for all of them, they were not be able to run at different speeds. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: random writing access to a file in Python
Dennis Lee Bieber wrote: > On 27 Aug 2006 15:06:07 -0700, Paul Rubin <http://[EMAIL PROTECTED]> > declaimed the following in comp.lang.python: > > > >>I think that's not so bad, though probably still not optimal. 85 GB >>divided by 18 hours is 1.3 MB/sec, which means if the program is >>reading the file 8 times, it's getting 10 MB/sec through the Windows >>file system, which is fairly reasonable throughput. >> > > Especially if, as implied, this was on a USB drive Don't underestimate external USB drives (I have heard there are great differences between them depending on the used controller). If the file requested is not scattered over the drive due to defragmentation and appropriate reading procedure is used I have seen (e.g. just yesterday with the 80 Gig file) constant throughput of 28 MBytes/second what compared to the maximum I have seen on E-IDE of 40 MBytes/second is not that bad as your posting might suggest. Thanks to Paul Rubin for the hint on radix sorting (even if coming a bit too late). I had used already yesterday this kind of approach in another context on the file I was sorting and can therefore estimate the total time on my system for such sorting using this method quite well: it will take not more than 3 hours (what is a very big improvement compared to 18 hours). I suppose, that the problem with Windows XP 'sort' is that it can't take advantage of the constant record size as there is no option available which could be used to pass this hint to it. "But if you only had to do it once and it's finished now, why do you still care how long it took?" Because one of my usual goals going along with doing things like that, is to get some feeling for them gaining experience making me in similar future cases capable of improvement by _intuitive_ selection of the best known to me path to the solution (learning by doing). It is a big difference between _knowing_ that there are various different sorting algorithms and it is necessary to choose the right one to speed up sorting and actually _experiencing_ that you have to wait for your results 18 hours and the machine is so busy that it is hard to use it for other tasks at the same time. If the sorting took less than one hour I would probably never make the effort to give it some serious thoughts. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: random writing access to a file in Python
Paul Rubin wrote: > Claudio Grondi <[EMAIL PROTECTED]> writes: > >>The Windows XP SP 2 '/> sort' (sorting of four Gigs of 20 byte records >>took 12 CPU and 18 usual hours) has, from what I could observe on the >>task manager, done the job in only two runs of 'copying' : > > > That is terrible; on a reasonably fast machine these days it should > take just a few minutes. Ok, I see - the misunderstanding is, that there were 4.294.967.296 records each 20 bytes long, what makes the actual file 85.899.345.920 bytes large (I just used 'Gigs' for telling the number of records, not the size of the file). Still not acceptable sorting time? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: How to let a loop run for a while before checking for break condition?
Fredrik Lundh wrote: > Diez B. Roggisch wrote: > >> A while loop has a condition. period. The only thing to change that is >> to introduce a uncoditioned loop, and use self-modifying code to make >> it a while-loop after that timer interrupt of yours. > > > or use a timer interrupt to interrupt the loop: > > import signal, time > > def func1(timeout): > > def callback(signum, frame): > raise EOFError # could use a custom exception instead > signal.signal(signal.SIGALRM, callback) > signal.alarm(timeout) > > count = 0 > try: > while 1: > count += 1 > except EOFError: > for i in range(10): > count += 1 > print count > > for an utterly trivial task like the one in that example, the alarm > version runs about five times faster than a polling version, on my test > machine (ymmv): > > def func2(timeout): > > gettime = time.time > t_limit = gettime() + timeout > > count = 0 > while gettime() < t_limit: > count += 1 > for i in range(10): > count += 1 > print count > > > This above is exactly what I am looking for, except it does not work in Microsoft Windows where the signal.alarm() function is not available. So now the only thing I would like to know is how to achieve the same functionality when running Python on a Microsoft Windows box. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: How to let a loop run for a while before checking for break condition?
Diez B. Roggisch wrote: >> The idea is to speed up a loop by using a timer interrupt interfering >> with the loop, so that only after the timer interrupt would occur, the >> loop will start to check its break condition in each iteration. >> No checking of any kind in the loop should happen up to that time to >> minimize the number of operations in each iteration within the loop >> itself (i.e. the loop more or less won't know, that there is a timer >> on its way to change the loops behavior at a later time). > > > A while loop has a condition. period. The only thing to change that is > to introduce a uncoditioned loop, and use self-modifying code to make it > a while-loop after that timer interrupt of yours. > > But of course that whole thing is a moot point - if shaving mu-secs on > that level is needed for your application, use C or assembly instead. Going to C or assembly addresses the speed, but does not address the question asked, as the problem of checking a condition in a loop remains the same (even if at another speed level). Here some more context to put more light into what I would like to know about: any program runs within an operating system and this system (and in particular Microsoft Windows) does many, many things beside running the program. The idea is to use the resources wasted in cycles of the CPU spent on processing the other things anyway for the program itself. I have only a vague draft of what I would like to achieve so please don't get what I write here about how I imagine it should be done too seriously: I think, that the application can start with an unconditional loop and tell the operating system to stop this loop and provide a response when e.g. one hour is over. When that happens a pre-prepared conditional loop will start (which was waiting to be awoken) assuming the same environment (values of variables will be preserved, so it is clear where to continue) as the previous one. As an intermediate quick and dirty solution for practical use there is the possibility to let the Python script run into an error or to break its run with Ctrl+C if it is apparent it is ready (e.g. the first approach has just saved me 20 CPU minutes of a four CPU hours needing script and the condition was checking only the value of an iteration counter so was not a very time consuming one). Just thought that for sure someone had already the same/similar idea and might share here an elegant Pythonic solution addressing this issue. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: How to let a loop run for a while before checking for break condition?
Diez B. Roggisch wrote: > Claudio Grondi schrieb: > >> >> Sometimes it is known in advance, that the time spent in a loop will >> be in order of minutes or even hours, so it makes sense to optimize >> each element in the loop to make it run faster. >> One of instructions which can sure be optimized away is the check for >> the break condition, at least within the time where it is known that >> the loop will not reach it. >> >> Any idea how to write such a loop? >> >> e.g. >> >> counter = 2*64 >> >> while counter(BUT DON'T CHECK IT THE FIRST ONE HOUR LONG): > > > now = time.time() > while time.time() - now < 3600.0 or some_other_condition: >... > > > The short circuiting of the or will prevent the execution of > some_other_condition. > >> ... do something ... # and decrease the counter >> >> Thanks for any hint, but in particular if related to timers on the >> Windows 2000/XP system I am mainly working with. >> >> What do you think about this idea? Does it make sense? > > What idea? This one you haven't probably got from what I have written. I thought, that the introductory text gives enough context to be able to see what I mean, but I was apparently wrong. The idea is to speed up a loop by using a timer interrupt interfering with the loop, so that only after the timer interrupt would occur, the loop will start to check its break condition in each iteration. No checking of any kind in the loop should happen up to that time to minimize the number of operations in each iteration within the loop itself (i.e. the loop more or less won't know, that there is a timer on its way to change the loops behavior at a later time). I hope this above helps to understand what I would like to achieve. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
How to let a loop run for a while before checking for break condition?
Sometimes it is known in advance, that the time spent in a loop will be in order of minutes or even hours, so it makes sense to optimize each element in the loop to make it run faster. One of instructions which can sure be optimized away is the check for the break condition, at least within the time where it is known that the loop will not reach it. Any idea how to write such a loop? e.g. counter = 2*64 while counter(BUT DON'T CHECK IT THE FIRST ONE HOUR LONG): ... do something ... # and decrease the counter Thanks for any hint, but in particular if related to timers on the Windows 2000/XP system I am mainly working with. What do you think about this idea? Does it make sense? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: random writing access to a file in Python
Paul Rubin wrote: > Claudio Grondi <[EMAIL PROTECTED]> writes: > >>Does it mean, that in case of very large files: >> the size of available memory for the sorting operation (making it >>possible to work on larger chunks of data in memory) has less impact >>on the actual sorting speed than >> the speed of the data transfer from/to storage device(s) > > > Transfer speed and parallelism matters most, if the cpu can keep up. > Parallelism means you want multiple drives that you can do i/o to > simultaneously without having to seek. Random access helps simulate > this, but only somewhat. Large database installations always want > lots of parallel disks for similar reasons to this. > > The basic method of sorting large files has traditionally been: > > 1) Read the file in pieces p(1),p(2),...,p(n) where each piece is as > big as will fit in memory. Sort each piece with your favorite > in-memory sort, and write out sorted pieces that we'll designate > r(1,1),...r(1,n). The r(a,b)'s are called "runs". Use multiple > output devices (tape or disk drives) to write out the runs, i.e. each > output tape will contain its own bunch of runs. > > 2) Read back a bunch of the runs in parallel, i.e. say you have > written r(1,1),r(1,2),...,r(1,5) to five separate tape drives. Read > them back simultaneously and merge them (requires very little external > memory) into a new "second level" run called r(2,1). Similarly merge > r(1,6),...,r(1,10) to the second level run r(2,2), and so forth. > > 3) Merge the second level runs into third level runs, and so forth > recursively until there's only one giant run. That is the sorted > output. > > The amount of memory determines the size of the initial "first level" > runs, which determines how many recursive merges are needed, so more > memory helps. > > The number of levels of recursion also depends on the "merge order", > i.e. the number of runs merged at each merge step. You really want > each merge to read from M physical input devices that are separate > from the output device(s). > > There are obviously a lot of optimizations that the above description > is missing, and there's a lot of strategy about how to organize the > merges, e.g. if you have 8 drives, do you use 4 for input and 4 for > output, or 5 in and 3 out, or what? The Windows XP SP 2 '/> sort' (sorting of four Gigs of 20 byte records took 12 CPU and 18 usual hours) has, from what I could observe on the task manager, done the job in only two runs of 'copying' : 1. first to a temporary file, 2. then to the final file. In the second run the size of processed chunks between reading/writing was in the order of up to tenths of Megabytes, where in the first run in order of up to hundreds Megabytes. I suppose that the procedure behind it was as follows: 1. decision about the size of chunks to split the file into and choosing the size of required memory 2. processing the chunks with in-memory sorting them and writing to the temporary file 3. decision about the size of buffers for merge sorting the chunks into the final file, so that they all fit into the 300 MByte of used memory 4. opening as many 'pipes' as there were chunks filling all of the pipe buffers up when one of them runs out of data 5. continuously switching between reading and writing to the hard disk, writing the results of the merge sorting to the final file always when one of the buffers run out of data and then filling up all of the buffers for the next cycle (concluded from observed scattered reading, smooth writing) > > Is this some kind of production deployment problem you're working on, > or do you just have a big pile of data you need to sort once? The latter. One of the intermediate reasons behind doing it, is an attempt to get more and better intuitive understanding what are the hardware and software limits of brute force based approaches to solving problems in the area of AI, language processing and data compression. > If you > need to deploy across multiple machines (i.e. you can't just buy a > giant machine if you need to do this sorting frequently), then I'd > suggest reading up on the subject, starting with Knuth vol 3. Thank you much for your detailed response. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: random writing access to a file in Python
Paul Rubin wrote: > Claudio Grondi <[EMAIL PROTECTED]> writes: > >>>Try the standard Unix/Linux sort utility. Use the --buffer-size=SIZE >>>to tell it how much memory to use. >> >>I am on Windows and it seems, that Windows XP SP2 'sort' can work with >>the file, but not without a temporary file and space for the resulting >>file, so triple of space of the file to sort must be provided. > > > Oh, sorry, I didn't see the earlier parts of the thread. Anyway, > depending on the application, it's probably not worth the hassle of > coding something yourself, instead of just throwing more disk space at > the Unix utility. But especially if the fields are fixed size, you > could just mmap the file and then do quicksort on disk. Simplest > would be to just let the OS paging system take care of caching stuff > if you wanted to get fancy, you could sort in memory once the sorting > regions got below a certain size. > > A huge amount of stuff has been written (e.g. about half of Knuth vol > 3) about how to sort. Remember too, that traditionally large-scale > sorting was done on serial media like tape drives, so random access > isn't that vital. Does it mean, that in case of very large files: the size of available memory for the sorting operation (making it possible to work on larger chunks of data in memory) has less impact on the actual sorting speed than the speed of the data transfer from/to storage device(s) ? So, that the most effective measure towards shortening the time required for sorting very large files were to use faster hard drives (e.g. 10.000 rpm instead of 7.600 rpm) and faster interfaces for the data transfer (e.g. E-IDE or S-ATA instead of USB), right? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: random writing access to a file in Python
Paul Rubin wrote: > Claudio Grondi <[EMAIL PROTECTED]> writes: > >>Is there a ready to use (free, best Open Source) tool able to sort >>lines (each line appr. 20 bytes long) of a XXX GByte large text file >>(i.e. in place) taking full advantage of available memory to speed up >>the process as much as possible? > > > Try the standard Unix/Linux sort utility. Use the --buffer-size=SIZE > to tell it how much memory to use. I am on Windows and it seems, that Windows XP SP2 'sort' can work with the file, but not without a temporary file and space for the resulting file, so triple of space of the file to sort must be provided. Windows XP 'sort' uses constantly appr. 300 MByte of memory and can't use 100% of CPU all the time, probably due to I/O operations via USB (25 MByte/s experienced top data transfer speed). I can't tell yet if it succeeded as the sorting of the appr. 80 GByte file with fixed length records of 20 bytes is still in progress (for eleven CPU time hours / 18 daytime hours). I am not sure if own programming would help in my case to be much faster than the systems own sort (I haven't tried yet to set the size of memory to use in the options to e.g 1.5 GByte as the sort help tells it is better not to specify it). My machine is a Pentium 4, 2.8 GHz with 2.0 GByte RAM. I would be glad to hear if the time required for sorting I currently experience is as expected for such kind of task or is there still much space for improvement? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: time.clock() going backwards??
Terry Reedy wrote: > "Giovanni Bajo" <[EMAIL PROTECTED]> wrote in message > news:[EMAIL PROTECTED] > >>Hello, >> >>I experimented something very strange, a few days ago. I was debugging an >>application at a customer's site, and the problem turned out to be that >>time.clock() was going "backwards", that is it was sometimes >>(randomically) >>returning a floating point value which was "less than" the value returned >>by >>the previous invokation. The computer was a pretty fast one (P4 3Ghz I >>think, >>running Windows XP), and this happened only between very close >>invokations of >>time.clock(). > > > I seem to remember this being mentioned before on the list, perhaps by Tim > Peters. Perhaps he will chime in. > > tjr If I remember it right, the cause of such a problem is updating the clock by accessing a time server over a network. Just any such access results in adjusting the time a bit and leads eventually to such problems. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: random writing access to a file in Python
Dennis Lee Bieber wrote: > On Fri, 25 Aug 2006 16:39:14 +0200, Claudio Grondi > <[EMAIL PROTECTED]> declaimed the following in comp.lang.python: > > >>The core of my problem was ... trying to use 'wb' or 'w+b' ... (stupid >>me ...) > > > Ouch... How many times did you have to restore that massive file > from backup? > I was smart enough to try it first on a very small file wondering what was happening. Python documentation and even Google search after 'random file access in Python' were not helpful as there was no example and no hint available. The only hint about random file access in Python I found with Google was Table of Contents of "Python Cookbook" from O'Railly: http://www.oreilly.com/catalog/pythoncook2/toc.html and hints about random reading access. I was stupid enough to forget about 'r+' (used it many times before in C/C++ a decade ago, but not yet in Python) thinking just too much the Pythonic way: === if I want to write, I don't open for reading (plus or not plus) === Actually my file was 'only' 42 GByte, but I wanted to construct the question making it impossible to suggest use of an intermediate file. In between I have chosen a total new approach as random writing to hard disk seems to actually move the disk head each time when seeking, so apparently no cache is used sorting a bit the pieces to write to the disk, so if there are many of them there is no other chance as to try to put them together in memory first before writing them to the file. This makes the straightforward intuitive programming a bit complicated because to work on large files it is necessary to work in chunks and waste some processing results when they don't fill the gaps. I suppose I am still not on the right path, so by the way: Is there a ready to use (free, best Open Source) tool able to sort lines (each line appr. 20 bytes long) of a XXX GByte large text file (i.e. in place) taking full advantage of available memory to speed up the process as much as possible? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: random writing access to a file in Python
Tim Peters wrote: > [Claudio Grondi] > >> I have a 250 Gbyte file (occupies the whole hard drive space) > > > Then where is Python stored ;-)? > >> and want to change only eight bytes in this file at a given offset of >> appr. 200 >> Gbyte (all other data in that file should remain unchanged). >> >> How can I do that in Python? > > > Same way you'd do it in C, really: > > f = open(PATH_TO_FILE, "r+b") > f.seek(appr. 200 Gbyte) > f.write(A_STRING_CONTAINING_YOUR_NEW_8_BYTES) > f.close() > > This depends on your operating system and file system and platform C > library supporting seeks to very large offsets. For example, Windows > using NTFS does. Try it. Python should complain (raise an exception > during the seek() attempt) if your box refuses to cooperate. Use as > recent a released version of Python as you can. Thank you much for the quick response. The core of my problem was ... trying to use 'wb' or 'w+b' ... (stupid me ...) Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
random writing access to a file in Python
I have a 250 Gbyte file (occupies the whole hard drive space) and want to change only eight bytes in this file at a given offset of appr. 200 Gbyte (all other data in that file should remain unchanged). How can I do that in Python? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: How to draw line on Image?
Paul McGuire wrote: > "David Isaac" <[EMAIL PROTECTED]> wrote in message > news:[EMAIL PROTECTED] > >>"Daniel Mark" <[EMAIL PROTECTED]> wrote in message >>news:[EMAIL PROTECTED] >> >>>I want to draw some shapes, such as lines, circles on an image. >> >> >>http://www.pythonware.com/library/pil/handbook/psdraw.htm >> >>hth, >>Alan Isaac >> >> > > > ImageDraw (http://www.pythonware.com/library/pil/handbook/imagedraw.htm) > might be more generally helpful than PSDraw. > > -- Paul > > And if the quality of the result is important maybe (maybe, because I haven't used it yet myself) aggdraw an add-on to the PIL library that supports anti-aliased drawing http://effbot.org/zone/draw-agg.htm Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: opposite of import
[EMAIL PROTECTED] wrote: > Hi > I am new to python. I wanted to know if there is an opposite of "import" > If you mean 'import' adds something, so you ask how to get rid of something? Here you are: Look at the 'del' statement if it is what you are looking for. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: list comprehension syntax..?
Gary Herron wrote:
> Gregory Guthrie wrote:
>
>>Sorry for a simple question- but I don't understand how to parse this use of
>>a list comprehension.
>>
>>The "or" clauses are odd to me.
>>
>>It also seems like it is being overly clever (?) in using a lc expression as
>>a for loop to drive the recursion.
>>
>>Thanks for any insight!
>>Gregory
>>-
>>
>># http://markbyers.com/moinmoin/moin.cgi/ShortestSudokuSolver
>>
>>def solve(board):
>> i=board.find('0') # find next open cell
>> if i<0:# done if none...
>> print board; exit("Done")
>> [ m in [(i-j)%9*(i/9^j/9)*(i/27^j/27|i%9/3^j%9/3)
>> or board[j] for j in range(81) ]
>> or solve(board[:i]+m+board[i+1:]) for m in'%d'%5**18 ]
>>
>
>
> The "or" clause (as you call it) has nothing to do with the list
> comprehension.
> The syntax being used here is
> [ big_expression for m in something]
> *and* the big_expression contains an "or" OPERATOR, with a
> complex_expression on one side and solve(...) on the other, like this
> complex_expression or solve(...)
> *and* the complex_expression contains a nested list comprehension like this
> m in nested_lc
> *and* nested_lc is
> [ugly_expression for j in range(81)]
> *and* ugly_expression contains another "or" OPERATOR with ...
> *and* sigh...
>
> Whoever put this expression together has made something that is
> completely unreadable, mostly unmaintainable, and not noticeably more
> efficient than code that is readable and maintainable.
>
> Moreover, all the work of creating the outer list seems to be wasted
> since that list is just thrown out.
>
> This is worse than "overly clever". Loops are for looping, list
> comprehension is for building lists. It is bad programming practice to
> use list comprehension for looping.
Isn't it an advantage considering speed of the execution?
I have just compared the speed of an explicit loop with the speed of a
list comprehension doing the same thing in another context to see it is
much better to use the latter. In board solvers speed is vital, so isn't
it a good practice to use a list comprehension for looping in this
context, anyway?
Claudio
>
> Hope that helps you,
> Gary Herron
>
>
>>
>>== Posted via Newsfeeds.Com - Unlimited-Unrestricted-Secure Usenet
>>News==
>>http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+
>>Newsgroups
>>= East and West-Coast Server Farms - Total Privacy via Encryption =
>>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Html character entity conversion
Anthra Norell wrote: >>>>import SE# Available at the Cheese Shop I mean, that OP requested: 'How can I translate this using standard Python libraries??' so it's just only not on topic. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: Html character entity conversion
John Machin wrote:
> Claudio Grondi wrote:
>
>>[EMAIL PROTECTED] wrote:
>>
>>>Claudio Grondi wrote:
>>>
>>>
>>>>[EMAIL PROTECTED] wrote:
>>>>
>>>>
>>>>>Here is my script:
>>>>>
>>>>
>>>>>from mechanize import *
>>>>>from BeautifulSoup import *
>>>>
>>>>>import StringIO
>>>>>b = Browser()
>>>>>f = b.open("http://www.translate.ru/text.asp?lang=ru";)
>>>>>b.select_form(nr=0)
>>>>>b["source"] = "hello python"
>>>>>html = b.submit().get_data()
>>>>>soup = BeautifulSoup(html)
>>>>>print soup.find("span", id = "r_text").string
>>>>>
>>>>>OUTPUT:
>>>>>привет
>>>>>питон
>>>>>--
>>>>>In russian it looks like:
>>>>>"привет питон"
>>>>>
>>>>>How can I translate this using standard Python libraries??
>>>>>
>>>>>--
>>>>>Pak Andrei, http://paxoblog.blogspot.com, icq://97449800
>>>>>
>>>>
>>>>Translate to what and with what purpose?
>>>>
>>>>Assuming your intention is to get a Python Unicode string, what about:
>>>>
>>>>strHTML = 'привет
>>>>питон'
>>>>strUnicodeHexCode = strHTML.replace('&#','\u').replace(';','')
>>>>strUnicode = eval("u'%s'"%strUnicodeHexCode)
>>>>
>>>>?
>>>>
>>>>I am sure, there is a more elegant and direct solution, but just wanted
>>>>to provide here some quick response.
>>>>
>>>>Claudio Grondi
>>>
>>>
>>>Thank you, Claudio.
>>>Really interest solution, but it doesn't work...
>>>
>>>In [19]: strHTML = 'привет
>>>питон'
>>>
>>>In [20]: strUnicodeHexCode = strHTML.replace('&#','\u').replace(';','')
>>>
>>>In [21]: strUnicode = eval("u'%s'"%strUnicodeHexCode)
>>>
>>>In [22]: print strUnicode
>>>---
>>>exceptions.UnicodeEncodeErrorTraceback (most
>>>recent call last)
>>>
>>>C:\Documents and Settings\dron\
>>>
>>>C:\usr\lib\encodings\cp866.py in encode(self, input, errors)
>>> 16 def encode(self,input,errors='strict'):
>>> 17
>>>---> 18 return codecs.charmap_encode(input,errors,encoding_map)
>>> 19
>>> 20 def decode(self,input,errors='strict'):
>>>
>>>UnicodeEncodeError: 'charmap' codec can't encode characters in position
>>>0-5: character maps to
>>>
>>>In [23]: print strUnicode.encode("utf-8")
>>>сВЗсВИсВАсБ┤сБ╖сВР сВЗсВАсВРсВЖсВЕ
>>><-- it's not my string "привет питон"
>>>
>>>In [24]: strUnicode.encode("utf-8")
>>>Out[24]:
>>>'\xe1\x82\x87\xe1\x82\x88\xe1\x82\x80\xe1\x81\xb4\xe1\x81\xb7\xe1\x82\x90
>>>\xe1\x82\x87\xe1\x82\x80\xe1\x82\x90\xe1\x82\x86\xe1\x82\
>>>x85' <-- and too many chars
>>>
>>
>>Have you considered, that the HTML page specifies charset=windows-1251
>>in its
>> tag ?
>>You are apparently on Linux or so, so I can't track this problem down
>>having only a Windows box here, but inbetween I know that there is
>>another problem with it:
>>I have erronously assumed, that the numbers in п are hexadecimal,
>>but they are decimal, so it is necessary to do hex(int('1087')) on them
>>to get at the right code to put into eval().
>>As you know now the idea I hope you will succeed as I did with:
>>
>> >>> lstIntUnicodeDecimalCode = strHTML.replace('&#','').split(';')
>> >>> lstIntUnicodeDecimalCode
>>['1087', '1088', '1080', '1074', '1077', '1090', ' 1087', '1080',
>>'1090', '1086', '1085', '']
>> >>> lstIntUnicodeDecimalCode = lstIntUnicodeDecimalCode[:-1]
>> >>> lstHexUnicode = [ hex(int(item)) for item in lstIntUnicodeDecimalCode]
>> >>
Re: Html character entity conversion
[EMAIL PROTECTED] wrote:
> Claudio Grondi wrote:
>
>>[EMAIL PROTECTED] wrote:
>>
>>>Here is my script:
>>>
>>>from mechanize import *
>>>from BeautifulSoup import *
>>>import StringIO
>>>b = Browser()
>>>f = b.open("http://www.translate.ru/text.asp?lang=ru";)
>>>b.select_form(nr=0)
>>>b["source"] = "hello python"
>>>html = b.submit().get_data()
>>>soup = BeautifulSoup(html)
>>>print soup.find("span", id = "r_text").string
>>>
>>>OUTPUT:
>>>привет
>>>питон
>>>--
>>>In russian it looks like:
>>>"привет питон"
>>>
>>>How can I translate this using standard Python libraries??
>>>
>>>--
>>>Pak Andrei, http://paxoblog.blogspot.com, icq://97449800
>>>
>>
>>Translate to what and with what purpose?
>>
>>Assuming your intention is to get a Python Unicode string, what about:
>>
>>strHTML = 'привет
>>питон'
>>strUnicodeHexCode = strHTML.replace('&#','\u').replace(';','')
>>strUnicode = eval("u'%s'"%strUnicodeHexCode)
>>
>>?
>>
>>I am sure, there is a more elegant and direct solution, but just wanted
>>to provide here some quick response.
>>
>>Claudio Grondi
>
>
> Thank you, Claudio.
> Really interest solution, but it doesn't work...
>
> In [19]: strHTML = 'привет
> питон'
>
> In [20]: strUnicodeHexCode = strHTML.replace('&#','\u').replace(';','')
>
> In [21]: strUnicode = eval("u'%s'"%strUnicodeHexCode)
>
> In [22]: print strUnicode
> ---
> exceptions.UnicodeEncodeErrorTraceback (most
> recent call last)
>
> C:\Documents and Settings\dron\
>
> C:\usr\lib\encodings\cp866.py in encode(self, input, errors)
> 16 def encode(self,input,errors='strict'):
> 17
> ---> 18 return codecs.charmap_encode(input,errors,encoding_map)
> 19
> 20 def decode(self,input,errors='strict'):
>
> UnicodeEncodeError: 'charmap' codec can't encode characters in position
> 0-5: character maps to
>
> In [23]: print strUnicode.encode("utf-8")
> сВЗсВИсВАсБ┤сБ╖сВР сВЗсВАсВРсВЖсВЕ
> <-- it's not my string "привет питон"
>
> In [24]: strUnicode.encode("utf-8")
> Out[24]:
> '\xe1\x82\x87\xe1\x82\x88\xe1\x82\x80\xe1\x81\xb4\xe1\x81\xb7\xe1\x82\x90
> \xe1\x82\x87\xe1\x82\x80\xe1\x82\x90\xe1\x82\x86\xe1\x82\
> x85' <-- and too many chars
>
Have you considered, that the HTML page specifies charset=windows-1251
in its
tag ?
You are apparently on Linux or so, so I can't track this problem down
having only a Windows box here, but inbetween I know that there is
another problem with it:
I have erronously assumed, that the numbers in п are hexadecimal,
but they are decimal, so it is necessary to do hex(int('1087')) on them
to get at the right code to put into eval().
As you know now the idea I hope you will succeed as I did with:
>>> lstIntUnicodeDecimalCode = strHTML.replace('&#','').split(';')
>>> lstIntUnicodeDecimalCode
['1087', '1088', '1080', '1074', '1077', '1090', ' 1087', '1080',
'1090', '1086', '1085', '']
>>> lstIntUnicodeDecimalCode = lstIntUnicodeDecimalCode[:-1]
>>> lstHexUnicode = [ hex(int(item)) for item in lstIntUnicodeDecimalCode]
>>> lstHexUnicode
['0x43f', '0x440', '0x438', '0x432', '0x435', '0x442', '0x43f', '0x438',
'0x442', '0x43e', '0x43d']
>>> eval( 'u"%s"'%''.join(lstHexUnicode).replace('0x','\u0' ) )
u'\u043f\u0440\u0438\u0432\u0435\u0442\u043f\u0438\u0442\u043e\u043d'
>>> strUnicode = eval(
'u"%s"'%''.join(lstHexUnicode).replace('0x','\u0' ) )
>>> print strUnicode
приветпитон
Sorry for that mess not taking the space into consideration, but I think
you can get the idea anyway.
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: Html character entity conversion
[EMAIL PROTECTED] wrote:
> Here is my script:
>
> from mechanize import *
> from BeautifulSoup import *
> import StringIO
> b = Browser()
> f = b.open("http://www.translate.ru/text.asp?lang=ru";)
> b.select_form(nr=0)
> b["source"] = "hello python"
> html = b.submit().get_data()
> soup = BeautifulSoup(html)
> print soup.find("span", id = "r_text").string
>
> OUTPUT:
> привет
> питон
> --
> In russian it looks like:
> "привет питон"
>
> How can I translate this using standard Python libraries??
>
> --
> Pak Andrei, http://paxoblog.blogspot.com, icq://97449800
>
Translate to what and with what purpose?
Assuming your intention is to get a Python Unicode string, what about:
strHTML = 'привет
питон'
strUnicodeHexCode = strHTML.replace('&#','\u').replace(';','')
strUnicode = eval("u'%s'"%strUnicodeHexCode)
?
I am sure, there is a more elegant and direct solution, but just wanted
to provide here some quick response.
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: Snapshot+Clipboard
Yves Lange wrote:
> Hello,
> i'm searching a method to take a snapshot and save it in a jpg, bmp or
> gif file. I tried with win32api and win32con but it save the snapshot to
> the clipboard, so i tried to redirect this in a file but i have some
> problems while getting the IMAGE stocked in the clipboard and save it to
> a file. Can somebody help me ?
>
> Questions:
> -How can i read the snapshot in the clipboard ?
> -How can i take a snapshot in a different way (less difficult) ?
>
> Thks.
Use PIL which on Windows supports taking snapshots of the screen.
import ImageGrab
GrabbedImage = ImageGrab.grab() # store screenshot as "RGB" Image
GrabbedImage.save("TheScreenshot.jpg") # PIL evaluates extension
For more details see:
http://effbot.org/imagingbook/imagegrab.htm
(works on Windows only)
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: Tkinter binding and keyboard hardware settings (WinXP)
[EMAIL PROTECTED] wrote:
> Hello,
>
> I have created a simple canvas in Tkinter to display a number of
> PhotoImages, I then bind a key (in my case, ) to start a loop that
> plays through a list of PhotoImages to make it an animation of sorts.
> What I noticed is, after holding down the key for a certain time, it
> would not loop through all 6 animations, instead it would just go
> through the first 2 and reset it self. I changed the key bind to
> Button-1 and did not encounter this problem. I then went into my
> keyboard hardware settings and changed the repeat rate to the longest
> delay and the repeat delay to the longest delay. This improved it some
> what, but I don't think it is practical to do this everytime I start up
> the program. My question is, is there a way to bind a key so as to make
> it ignore the keyboard's delay and repeat rate settings? Here's the
> code just so you can get an idea:
>
> # GIF Animation Test
>
> from Tkinter import *
> from time import sleep
>
> class GIF:
> def __init__(self, root):
> self.root = root
> self.canvas = Canvas(root, bg="white", width=225, height=200)
> self.canvas.pack()
> self.cronoUpImageList =[
> "cronoup1.gif", "cronoup2.gif", "cronoup3.gif",
> "cronoup4.gif", "cronoup5.gif", "cronoup6.gif"]
> self.gifImage = PhotoImage(file=self.cronoUpImageList[0])
> self.cronoDefUp = PhotoImage(file="cronodefup.gif")
> self.drawImage = self.canvas.create_image(112.5, 100, image=
> self.cronoDefUp)
> self.canvas.bind("", self.keyboardPrsUp)
> self.canvas.bind("", self.keyboardRlsUp)
>
> def keyboardPrsUp(self, event):
currentTime = ...
if( currentTime - self.timeOfLastKeyPush < self.minDelay):
pass
else:
self.timeOfLastKeyPush = currentTime
...
> self.canvas.delete(self.drawImage)
> self.counter = 0
> self.killEvent = False
> while self.killEvent != True:
> print self.cronoUpImageList[self.counter]
> self.gifImage =
> PhotoImage(file=self.cronoUpImageList[self.counter])
> self.drawImage = self.canvas.create_image(112.5, 100,
> image=
> self.gifImage)
> self.counter = self.counter + 1
> if self.counter > 5:
> self.counter = 0
> self.canvas.update()
> sleep(.05)
>
>
> def keyboardRlsUp(self, event):
> self.counter = 0
> self.killEvent = True
> self.canvas.delete(self.drawImage)
> self.drawImage = self.canvas.create_image(112.5, 100, image=
> self.cronoDefUp)
>
>
> root = Tk()
> root.title("GIF Animation Test")
> app = GIF(root)
> root.mainloop()
>
What about simple tracking of time (see above NOT tested rough draft) in
the keyboardPrsUp and keyboardRlsUp functions, so, that events raised
earlier than after a specified delay period will just become ignored?
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: WANTED: logging of all file operations on Windows
Tim Golden wrote:
> Claudio Grondi wrote:
>
>> I am aware, that it is maybe the wrong group to ask this question, but
>> as I would like to know the history of past file operations from
>> within a Python script I see a chance, that someone in this group was
>> into it already and is so kind to share here his experience.
>>
>> I can't believe, that using NTFS file system in Microsoft Windows 2000
>> or XP it is not possible to track file events as:
>>
>> - updating/modifying of an existing file/directory
>> - deleting an existing file/directory
>> - creating a new file/directory
>> - _moving_ an existing file/directory (should _NOT_ be covered by the
>> event duo of deleting an existing and creating a new file/directory)
>>
>> Any hints towards enlightenment?
>>
>> Claudio Grondi
>
>
> On the offchance that you haven't seen it, you might
> look at this:
>
> http://timgolden.me.uk/python/win32_how_do_i/watch_directory_for_changes.html#use_readdirectorychanges
>
>
>
> but since it doesn't fulfil your criterion of *not*
> representing renames by a delete and an add, it may
> well not be suitable. Apart from that, I think it does
> what you want.
>
> TJG
Here a small update to the code at
http://timgolden.me.uk/python/win32_how_do_i/watch_directory_for_changes.html#use_readdirectorychanges
:
ACTIONS = {
1 : "Created",
2 : "Deleted",
3 : "Updated",
4 : "Renamed from something"
5 : "Renamed to something",
}
The correction above is according to entries:
#define FILE_ACTION_ADDED 0x0001
#define FILE_ACTION_REMOVED 0x0002
#define FILE_ACTION_MODIFIED 0x0003
#define FILE_ACTION_RENAMED_OLD_NAME 0x0004
#define FILE_ACTION_RENAMED_NEW_NAME 0x0005
in ..\PlatformSDK\Include\WinNT.h
Claudio Grondi
--
http://mail.python.org/mailman/listinfo/python-list
Re: WANTED: logging of all file operations on Windows
Tim Golden wrote: > Claudio Grondi wrote: > >> I am aware, that it is maybe the wrong group to ask this question, but >> as I would like to know the history of past file operations from >> within a Python script I see a chance, that someone in this group was >> into it already and is so kind to share here his experience. >> >> I can't believe, that using NTFS file system in Microsoft Windows 2000 >> or XP it is not possible to track file events as: >> >> - updating/modifying of an existing file/directory >> - deleting an existing file/directory >> - creating a new file/directory >> - _moving_ an existing file/directory (should _NOT_ be covered by the >> event duo of deleting an existing and creating a new file/directory) >> >> Any hints towards enlightenment? >> >> Claudio Grondi > > > On the offchance that you haven't seen it, you might > look at this: > > http://timgolden.me.uk/python/win32_how_do_i/watch_directory_for_changes.html#use_readdirectorychanges > > > > but since it doesn't fulfil your criterion of *not* > representing renames by a delete and an add, it may > well not be suitable. Apart from that, I think it does > what you want. > > TJG It seems, that it will be necessary to use some logic based on the sequence of events to exactly detect rename and move changes done to files/directories, but in principle it is the best approach I know about yet. Thank you! By the way: Is there something similar/same available for Linux? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: WANTED: logging of all file operations on Windows
faulkner wrote: > you want a directory watching daemon. it isn't hard at all to build > from scratch. > first, determine which directories should be watched. > then, os.walk each directory, building a mapping from filename to mtime > [modified time; os.path.getmtime]. > next is your main event loop. this while loop consists of os.walk-ing > each directory again, comparing the current mtime to the corresponding > entry in the mapping. if they differ, or if a filename isn't in the > mapping, something happened, at which point you can logick out whether > a file was moved, deleted, changed, or created. > > so many folks have looked for this that i'll just write a generic one > and put it in the cheeseshop. look for "dirmon" in about a week. Yes, I _know_ about it and exactly this knowledge is the reason I am looking for tracking single file system related _events_ as I expect a professional operating system like Windows to provide such service. If there is none, this will be sure a severe reason to go for Linux if it has such one instead of going for a SVN server or special file systems if there are any. Has someone experience with SVN handling million(s) of files? The problem is, that brute force applied to large amount of files/directories is not a convenient way to backup/synchronize the few new/changed/deleted/moved files/directories multiple times a day as the brute force approach just makes the hard drive(s) unnecessary wasting much energy and getting hot. Claudio Grondi > > > Claudio Grondi wrote: > >>I am aware, that it is maybe the wrong group to ask this question, but >>as I would like to know the history of past file operations from within >>a Python script I see a chance, that someone in this group was into it >>already and is so kind to share here his experience. >> >>I have put already much efforts into this subject googling around, but >>up to now in vain. Best option I encountered yet is usage of >>the Greyware 'System Change Log' service which monitors disks for >>changes (http://www.greyware.com/software/systemchangelog/index.asp), >>but in own tests it turned out, that the created log file does not cover >>all file events as e.g. it is not possible to detect when a file is >>moved to a new directory (creation of a new file is logged, but deletion >>is not, not mentioning I would expect a file 'move' event). >>The own Windows logging service rejected to start on my XP SP2 system >>for unknown to me reasons - I don't know how to get it to work (yes, I >>have used the administrator account). >> >>I can't believe, that using NTFS file system in Microsoft Windows 2000 >>or XP it is not possible to track file events as: >> >>- updating/modifying of an existing file/directory >>- deleting an existing file/directory >>- creating a new file/directory >>- _moving_ an existing file/directory (should _NOT_ be covered by the >>event duo of deleting an existing and creating a new file/directory) >> >>Any hints towards enlightenment? >> >>Claudio Grondi > > -- http://mail.python.org/mailman/listinfo/python-list
WANTED: logging of all file operations on Windows
I am aware, that it is maybe the wrong group to ask this question, but as I would like to know the history of past file operations from within a Python script I see a chance, that someone in this group was into it already and is so kind to share here his experience. I have put already much efforts into this subject googling around, but up to now in vain. Best option I encountered yet is usage of the Greyware 'System Change Log' service which monitors disks for changes (http://www.greyware.com/software/systemchangelog/index.asp), but in own tests it turned out, that the created log file does not cover all file events as e.g. it is not possible to detect when a file is moved to a new directory (creation of a new file is logged, but deletion is not, not mentioning I would expect a file 'move' event). The own Windows logging service rejected to start on my XP SP2 system for unknown to me reasons - I don't know how to get it to work (yes, I have used the administrator account). I can't believe, that using NTFS file system in Microsoft Windows 2000 or XP it is not possible to track file events as: - updating/modifying of an existing file/directory - deleting an existing file/directory - creating a new file/directory - _moving_ an existing file/directory (should _NOT_ be covered by the event duo of deleting an existing and creating a new file/directory) Any hints towards enlightenment? Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: numarray
bruce wrote: > robert > > i did an > python>>> import numpy > a = array([['q','a'],['w','e']]) not tested, but you usually need to mention where to find array: a = numpy.array([['q','a'],['w','e']]) > > and it didn't work... > > i used > >>from import numpy * > > and it seems to accept the 'array' word.. .looks like it will work... from numpy import * makes numpy.array available as array in your code. I would be kind of you to post what you have exactly done as the phrase "from import ..." can't be it. Claudio Grondi > > what's the difference between 'import numpy', and "from import numpy *" > > comments... > > thanks > > -bruce > > > -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] Behalf > Of Robert Kern > Sent: Tuesday, July 04, 2006 9:42 PM > To: python-list@python.org > Subject: Re: numarray > > > bruce wrote: > >>hi... >> >>i'm trying to find numarray.. i found the numpy on sourceforge and >>downloaded/installed.. >> >>i did a >>python>> import numarray >> >>and got an error... > > > Never just say "I got an error." It tells us nothing. Copy-and-paste the > exact > error message. I presume, however, that you installed numpy, not numarray. > They > are not the same thing. > > >>the docs that i've seen point to the sourceforge area.. but i only see >>numpy.. which appears to incorporate numarray.. > > > No, it replaces numarray. > > http://www.scipy.org/NumPy > > >>my goal is to somehow define multi-dimensional arrays of strengs... > > > >>> from numpy import * > >>> a = array([['some', 'strings'],['in an', 'array']], dtype=object) > >>> a > array([[some, strings], > [in an, array]], dtype=object) > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." >-- Umberto Eco > > -- > http://mail.python.org/mailman/listinfo/python-list > -- http://mail.python.org/mailman/listinfo/python-list
Re: wanted: framework for creating nice step by step graphical visualisations of running Python code
[EMAIL PROTECTED] wrote: > I remember Gato: > http://gato.sourceforge.net/ > It animates only algorithms on graphs, but it seems a starting point, > and it works. > > I vaguely remember another system, but probably not very good. > > Bye, > bearophile > Yes, I have noticed Gato already before, but was not able to find my way into it - I am missing a kind of tutorial explaining what it is all about - the description of available classes or demos don't tell me much about it, so I have no idea how to start - is there any tutorial out there explaining it from the very beginning what is it for, how it does it and why? By the way: it seems to be very slow on my 3 GHz Pentium 4 system ... Any other hints? Claudio -- http://mail.python.org/mailman/listinfo/python-list
wanted: framework for creating nice step by step graphical visualisations of running Python code
Today I bumped by chance into explaining what algorithms do by using animation (Java applets): http://www-sr.informatik.uni-tuebingen.de/~buehler/BM/BM1.html Is there any tool in Python (except pyGame, Tkinter or other general purpose visualization tools) I am not aware of which would make it easy to create a similar, animated run through Python script code? A free debugging tool capable of stepping line by line through Python code showing values of selected objects will do as a first approach, but it would be nice to be able to output also some graphics and/or text like it is done in the mentioned above example at pre-defined points in code called there /* visualisation step */ Any hints towards getting or constructing such a framework are welcome. Claudio Grondi -- http://mail.python.org/mailman/listinfo/python-list
Re: How do you use this list ?
Bo Yang wrote: > Hi everyone , > I have join this list for about 4 months , and everyday I receive > hundreds of > mails . There is no means to read all of them , so I just read something > interesting > for me . But if so , there are too much mails pile up in my inbox , I > want to ask > how do you use this list , reading every mail come in or just read what > you think > interesting ? > > Thank you ! > > > Best Regard ! I am using a newsreader capable of showing posting in their threads in a hierarchical view. I read entire messages only if I mean the topic is of interest to me, if I have spotted something interesting in the first lines of a posting or if I mean, that I can immediately help writing a reply without much efforts on my side what is the case when the answer pops up in my mind already as I see the question. It was the topic "How do you use this list?" what attracted my attention so I looked into the content of your posting and decided to reply. My advice: download e.g. Thunderbird and use it as a newsreader instead of getting through not by thread and date structured piles of emails. I actually go through each single posting spending at least a second on each (sometimes a bit faster, sometimes much slower). This gives an effort of about 10 minutes of very concentrated work towards selection of interesting postings for detailed reading on a day with a huge traffic of around 600 new postings. Over years of reading I improved much the speed of perception at first glance if a posting or a thread is in my eyes worth to read it in detail. As I also in between know who writes here the best quality replies I usually read carefully all the details in postings of such authors skipping from reading another postings in same thread. Hope this helps. Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: Beginner Programmer Question
[EMAIL PROTECTED] wrote:
> [EMAIL PROTECTED] wrote:
>
>>Claudio Grondi wrote:
>>
>>>[EMAIL PROTECTED] wrote:
>>>
>>>>I am doing alot of reading and trying to teach myself how to program.
>>>>I can not figure out how to make "Write a program that continually
>>>>reads in numbers from the user and adds them together until the sum
>>>>reaches 100." this work. If someone could show me the correct code so i
>>>>can learn from that it would be much appreciated. Thanks
>>>>
>>>
>>>Isn't it your homework?
>>>Why can't you figure it out?
>>>What have you tried?
>>>
>>>Claudio
>>
>>I am doing alot of reading, and the problem didnt come with an answer.
>>I dont understand how to get it to continually input numbers and add
>>all those together
>
>
> #Add up to 100 program
>
> #What number are you adding up to?
> bigone = 100
>
> number = input("Whats the first number?")
> number2 = input ("Whats the second number?")
> nu3 = number+number2
> while nu3 < bigone:
# you are missing the input and building the sum here:
number3 = input ("Whats the next number?")
nu3 = nu3 + number3
> print ("Not there yet, next number please")
# so nu3 gets never changed and your loop runs forever.
>
>
> print "Finally there!"
>
> thats what i thought maybe it was...but after the first two numbers it
> just continually scrolls on the screen with finally there
Yes, because you have no input command in the loop and no sum changing
the value of nu3.
But you should get scrolling with "Not there yet, next number please"
not with "Finally there!" according to your code.
I suppose your next question will be what tutorial is the best for a
newbie, so I suggest you start with:
http://wiki.python.org/moin/BeginnersGuide
Claudio
--
http://mail.python.org/mailman/listinfo/python-list
Re: style question
Scott David Daniels wrote:
> Claudio Grondi wrote:
> <<>>
>
>> When necessary to skip first line _and_ indentation:
>> message = """
>> This is line 1
>> This is line 2
>> This is line 3
>> """.replace('\n ', '\n')[1:] # adjust here '\n ' to indentation
>
>
> Riffing on this idea:
> message = """
> This is line 1
> This is line 2
> This is line 3
> """.replace("""
> """, '\n')[1:]
This was intended as an excercise for the OP in case he likes that kind
of solution ...
Claudio
>
> --Scott David Daniels
> [EMAIL PROTECTED]
--
http://mail.python.org/mailman/listinfo/python-list
Re: Beginner Programmer Question
[EMAIL PROTECTED] wrote: > I am doing alot of reading and trying to teach myself how to program. > I can not figure out how to make "Write a program that continually > reads in numbers from the user and adds them together until the sum > reaches 100." this work. If someone could show me the correct code so i > can learn from that it would be much appreciated. Thanks > Isn't it your homework? Why can't you figure it out? What have you tried? Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: style question
Hari Sekhon wrote:
> Is it better to do:
>
> message = """This is line1.
> This is line2
> This is line3\n"""
>
> or
>
> message = "This is line1.\n
> message = message + "This is line2\n"
> message = message + "This is line3\n"
>
>
> Since the first method does not follow python's clean and easy looking
> indentation structure but the second just looks crude and ugly anyway.
>
> If I indent the first version so the text is lined up to match code
> indentation then this comes out in the input and isn't aligned there.
>
What about
message = """
This is line 1
This is line 2
This is line 3
"""
When it is necessary to skip first empty line):
message = """
This is line 1
This is line 2
This is line 3
"""[1:]
When necessary to skip first line _and_ indentation:
message = """
This is line 1
This is line 2
This is line 3
""".replace('\n ', '\n')[1:] # adjust here '\n ' to indentation
# ^-- gives 'This is line 1\nThis is line 2\nThis is line 3\n'
?
Claudio
--
http://mail.python.org/mailman/listinfo/python-list
Re: What is Expressiveness in a Computer Language
Xah Lee wrote: > in March, i posted a essay “What is Expressiveness in a Computer > Language”, archived at: > http://xahlee.org/perl-python/what_is_expresiveness.html > > I was informed then that there is a academic paper written on this > subject. > > On the Expressive Power of Programming Languages, by Matthias > Felleisen, 1990. > http://www.ccs.neu.edu/home/cobbe/pl-seminar-jr/notes/2003-sep-26/expressive-slides.pdf > > Has anyone read this paper? And, would anyone be interested in giving a > summary? > > thanks. > >Xah >[EMAIL PROTECTED] > ∑ http://xahlee.org/ > Looking this thread growing it appears to me, that at least one thing becomes evident here: Xah unwillingness to learn from the bad past experience contaminates others (who are still posting to his trolling threads). Here another try to rescue these ones who are just virgin enough not to know what I am speaking about: Citation from http://www.xahlee.org/Netiquette_dir/_/art_trolling.html : """ What I want this document to focus on is how to create entertaining trolls. I have drawn on the expertise of the writer's of some of Usenet's finest and best remembered trolls. Trolls are for fun. The object of recreational trolling is to sit back and laugh at all those gullible idiots that will believe *anything*. [...] Section 5Know Your Audience Remember that you have two audiences. The people who are going to get the maximum enjoyment out of your post are other trollers. You need to keep in contact with them through both your troll itself and the way you direct its effect. It is trollers that you are trying to entertain so be creative - trollers don't just want a laugh from you they want to see good trolls so that they can also learn how to improve their own in the never ending search for the perfect troll. [...] Section 6Following-Up Try not to follow-up to your own troll. The troll itself quickly becomes forgotten in the chaos and if you just sit back you can avoid being blamed for causing it. Remember, if you do follow up you are talking to an idiot. Treat them with the ill-respect they deserve. """ Claudio Grondi (a past 'gullible idiot' who learned to enjoy the fun of being the audience) -- http://mail.python.org/mailman/listinfo/python-list
Re: Standard Yes / No Windows Dialog box creation
[EMAIL PROTECTED] wrote: > I found a way to create "Open File" or "Open Folder" windows dialog > boxes, but not to create an easier Yes / No dialog box... > Maybe someone has a solution for this? > Do it just the same way as you did it with the "Open File" or "Open Folder" windows dialog. What is your problem with it? Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: how to get the length of a number
Stan Cook wrote:
> Can anyone tell me how to get the length of a number. I know
> len(string) will get the length of a string, but it doesn't like
> len(int). I seem to remember something like %s string. I tried to set
> a variable = to %s int, but that doesn't work. Is there a function I've
> forgotten about to convert an integer to a string?
>
> Regards
>
> Stan
len('%s'%(1234567,)) gives 7
len('%s'%(1234**45,)) and len(str(1234**45))
give 140
Claudio
--
http://mail.python.org/mailman/listinfo/python-list
Re: what are you using python language for?
hacker1017 wrote:
> im just asking out of curiosity.
It appears to me, that the natural language is not enough to record
thoughts/observations/enlightenments for being reviewed and used with
ease after a longer time, as for this purpose it is necessary to include
in such records some sort of activity and/or interactivity and this
requires utilization of a computer and a programming language.
Usage of Python (on top of the English language I am not native speaker
of and HTML way of formatting texts) saves me the work of documenting
the very basics of the programming language add-on used on top of
natural language for above purpose as it comes with documentation of own
elements
[i.e. keywords and concepts in form of definitions like: global_stmt ::=
"global" identifier ("," identifier)*]
from the very basic parser point of view.
Python makes an intuitive way of expressing algorithms and processes
easier by having many of for this purpose useful concepts already built-in.
By the way:
Which other programming languages provide documentation also via
giving definitions of keywords and concepts? Are there e.g. similar
definitions
[i.e e.g.: global_stmt ::= "global" identifier ("," identifier)*]
available for C/C++, Java, JavaScript?
Short expressed:
I use Python (and its huge amount of available modules) mainly as an
extension on top of natural English language and HTML formatted texts
leveraging this way the (re)use of textual recordings of ideas by
turning plain ASCII texts into interactive and searchable multimedia
content.
Claudio
--
http://mail.python.org/mailman/listinfo/python-list
Re: integer to binary...
[EMAIL PROTECTED] wrote: > [EMAIL PROTECTED] wrote: > >>does anyone know a module or something to convert numbers like integer >>to binary format ? >> >>for example I want to convert number 7 to 0111 so I can make some >>bitwise operations... >> >>Thanks > > > Use the gmpy module. > > import gmpy a = 14 b = 7 c = 8 > > help(gmpy.digits) > > Help on built-in function digits: > > digits(...) > digits(x[,base]): returns Python string representing x in the > given base (2 to 36, default 10 if omitted or 0); leading '-' > present if x<0, but no leading '+' if x>=0. x must be an mpz, > or else gets coerced into one. > > print gmpy.digits(a,2) > > 1110 > print gmpy.digits(b,2) > > 111 > print gmpy.digits(c,2) > > 1000 > > > help(gmpy.setbit) > > Help on built-in function setbit: > > setbit(...) > setbit(x,n,v=1): returns a copy of the value of x, with bit n set > to value v; n must be an ordinary Python int, >=0; v, 0 or !=0; > x must be an mpz, or else gets coerced to one. > > d = gmpy.setbit(c,1,1) print gmpy.digits(d,2) > > 1010 > > > > help(gmpy.scan1) > > Help on built-in function scan1: > > scan1(...) > scan1(x, n=0): returns the bit-index of the first 1-bit of x (that > is at least n); n must be an ordinary Python int, >=0. If no more > 1-bits are in x at or above bit-index n (which can only happen for > x>=0, notionally extended with infinite 0-bits), None is returned. > x must be an mpz, or else gets coerced to one. > > help(gmpy.scan0) > > Help on built-in function scan0: > > scan0(...) > scan0(x, n=0): returns the bit-index of the first 0-bit of x (that > is at least n); n must be an ordinary Python int, >=0. If no more > 0-bits are in x at or above bit-index n (which can only happen for > x<0, notionally extended with infinite 1-bits), None is returned. > x must be an mpz, or else gets coerced to one. > > print gmpy.scan1(a) > > 1 > print gmpy.scan1(b) > > 0 > print gmpy.scan1(c) > > 3 > print gmpy.scan1(d) > > 1 > print gmpy.scan0(a) > > 0 > print gmpy.scan0(b) > > 3 > print gmpy.scan0(c) > > 0 > print gmpy.scan0(d) > > 0 > > help(gmpy.popcount) > > Help on built-in function popcount: > > popcount(...) > popcount(x): returns the number of 1-bits set in x; note that > this is 'infinite' if x<0, and in that case, -1 is returned. > x must be an mpz, or else gets coerced to one. > > print gmpy.popcount(a) > > 3 > print gmpy.popcount(b) > > 3 > print gmpy.popcount(c) > > 1 > print gmpy.popcount(d) > > 2 > > > help(gmpy.hamdist) > > Help on built-in function hamdist: > > hamdist(...) > hamdist(x,y): returns the Hamming distance (number of bit-positions > where the bits differ) between x and y. x and y must be mpz, or > else > get coerced to mpz. > > print gmpy.hamdist(a,b) > > 2 > print gmpy.hamdist(a,c) > > 2 > print gmpy.hamdist(a,d) > > 1 > print gmpy.hamdist(b,c) > > 4 > print gmpy.hamdist(b,d) > > 3 > print gmpy.hamdist(c,d) > > 1 > For those digging deeper into this subject who are looking for speed, reading the past discussion on this newsgroup I was part of myself looking for fastest way of such integer to binary conversion can maybe be of interest: http://mail.python.org/pipermail/python-list/2006-January/319295.html (includes full source code of all compared approaches) Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: John Bokma harassment
Tim N. van der Leeuw wrote: > [EMAIL PROTECTED] wrote: > >>I agree there are limits to you right to free speech, but I believe Xah >>Lee is not crossing >>any boundaries. If he starts taking over newspapers and TV stations be >>sure to notify me, >>I might revise my position. >>Immanuel > > > Perhaps he's not crossing boundaries of free speech, but he's > repeatedly crossing boundaries on usenet nettiquette, even though > repeatedly he's being asked not to do so. (Extensive crossposting to > various usenetgroups / mailing lists, for instance). > > If he would just post his stuff on a blog and find a way to get people > to visit his blog, without crossposting to 10 usenest groups for each > post he makes to his blog, then nobody would mind him expressing his > opinions, and those interested could discuss them wildly on the blog. > > But I've repeatedly seen people telling him not to crosspost his essays > to so many newsgroups, yet he continues doing it. > If that's enough to quit his subscription with his ISP I don't know, > but since I've stopped following threads originated by him I don't know > what other grounds there would be. > > Cheers, > > --Tim > Well said, Tim. Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: Pyrex installation on windows XP: step-by-step guide
sturlamolden wrote: > Julien Fiore wrote: > >>Thanks for your remark, Sturlamolden. >> >>Is there a free version of the "Visual C++ 2003" compiler available on >>the web? I have found "Visual C++ 2005 Express edition" >>(http://msdn.microsoft.com/vstudio/express/visualc/). According to >>Micrsoft, it replaces VC++2003 >>(http://msdn.microsoft.com/visualc/vctoolkit2003/). Is VC++2005ee the >>good compiler to compile a Pyrex module (or any Python extension) ? >>Does it link with msvcr71.dll ? > > > The bad news is that "Visual C++ 2005 Express" links with msvcr80.dll, > which incompatible with both msvcrt.dll and msvcr71.dll. What you need > is the "Microsoft .NET Framework SDK Version 1.1". It contains version > 7.1 of Microsoft's C/C++ compiler and links with the correct CRT. > > http://tinyurl.com/5flob > > I am not sure if this is an optimizing compiler. The free available Visual C++ 2003 compiler is the optimizing one. You have to get it separately downloading the Visual C++ Toolkit 2003 which comes with the Professional version of Microsoft Visual C++ .NET 2003 compiler and linker. Claudio > Microsoft did not give > away their optimizing compiler prior to "Visual C++ 2005 Express". Even > the standard version of Visual Studio did not have an optimizing > compiler, it only shipped with the professional and enterprise > versions. If this compiler does not optimize, you may try to make > "Visual C++ 2005 Express" use the import library for msvcr71.dll which > ships with the .NET SDK. > > Now you know the meaning of the word "DLL HELL". > -- http://mail.python.org/mailman/listinfo/python-list
Re: open file with whitespaces
mardif wrote: > OK OK GUYS > I've found the solution: ( effectly, a friend of mine has found the > solution ) > > import os > > os.spawnl(os.P_WAIT, "c:\programmi\internet > explorer\iexplore.exe",'"C:\Documents and > Settings\michele\Desktop\ciccio.html"','"C:\Documents and > Settings\michele\Desktop\ciccio.html"') > > The secret are the ' simbols around arguments: > > ' "C:\Documents and Settings\michele\Desktop\ciccio.html" ' > > Without these, don't work! > > Very STRONG! > > bye and thx > Wasn't that what I have suggested? By the way: it is much better to use double backslashes here, as in case of other file/directory names (e.g. folder\name.txt folder\remote.htm i.e. for all the cases a backslash is followed by a letter making out of this twin one special character) this above won't work. But best is to use in full path file names forward slashes as Python handles them properly not only on *nix and Linux, but also on Windows: '"C:/Documents and Settings/michele/Desktop/ciccio.html"' The "secret" symbols around arguments are single quotation marks making the double quotation marks part of the string passed to the .spawnl() function. Check out in IDLE, that: >>> "x" 'x' >>> '"x"' '"x"' Command shell command arguments need to be enclosed in quotation marks in case spaces can occur in them (as it can happen in file names) as otherwise the spaces will be interpreted as separator characters between the arguments and as consequence of this the command shell command fails due to bad or wrong number of parameter. You see, no "secrets" here ... only some simple rules and a bit more understanding of what is going on behind the scenes. Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: open file with whitespaces
Christophe wrote: > mardif a écrit : > >> Hi guys. >> I've a very big big big problem: >> >> I've in my windows computer a file named cicciobello.html, located in >> c:\documents and settings\username\desktop\cicciobello.html. >> >> Now, I MUST open this file with os.spawn(os.P_WAIT , because I must >> wait the user cancel the explorer window, ok? >> And so, I write: >> >> import os >> os.spawnl(os.P_WAIT, "c:\programmi\internet explorer\iexplorer.exe", >> "c:\documents and settings\username\desktop\cicciobello.html") >> >> the python process don't fail, but explorer don't visualize correctly >> the file opened: i receive an "not found" error message. >> >> I've found that the whitespaces in file path are the problem. >> If you see to explorer address bar, you will find the address >> completely wrong... >> >> I've found no solution! >> Can you help me, please?? >> >> thx very very very much!!! >> > > And I thought the problem where the incorrectly used \ :) Try that first : > > os.spawnl(os.P_WAIT, r"c:\programmi\internet explorer\iexplorer.exe", > r"c:\documents and settings\username\desktop\cicciobello.html") Another option to try in case this is not the problem is to put the file name in quotation marks as it is necessary when using command line shell (not tested, just an idea): os.spawnl(os.P_WAIT,'"c:\\programmi\\internet explorer\\iexplorer.exe"', '"c:\\documents and settings\\username\\desktop\\cicciobello.html"') -- http://mail.python.org/mailman/listinfo/python-list
Re: Large Dictionaries
Chris Foote wrote: > Richie Hindle wrote: >> [Chris] >>> Has anyone written a fast hash module which is more optimal for >>> large datasets ? >> >> PyJudy might be what you're looking for, though I've never used it: >> >> http://www.dalkescientific.com/Python/PyJudy.html >> >> "Judy's key benefits are scalability, high performance, and memory >> efficiency. A Judy array is extensible and can scale up to a very large >> number of elements, bounded only by machine memory." ... "PyJudy arrays >> are similar to Python dictionaries and sets." > > Thanks for the suggestion Richie. PyJudy works brilliantly :-) > > Cheers, > Chris It seems, that there is no Microsoft Windows version of Judy available, so for this reason PyJudy won't work on Windows - am I right? Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: Large Dictionaries
Chris Foote wrote: > Claudio Grondi wrote: >> Chris Foote wrote: >>> Klaas wrote: >>> >>>>> 22.2s 20m25s[3] >>>> >>>> 20m to insert 1m keys? You are doing something wrong. >>> >>> I've put together some simplified test code, but the bsddb >>> module gives 11m for 1M keys: >>> >> I have run your code for the bsddb on my P4 2.8 GHz and have got: >> Number generator test for 100 number ranges >> with a maximum of 3 wildcard digits. >> Wed May 17 16:34:06 2006 dictionary population started >> Wed May 17 16:34:14 2006 dictionary population stopped, duration 8.4s >> Wed May 17 16:34:14 2006 StorageBerkeleyDB population started >> Wed May 17 16:35:59 2006 StorageBerkeleyDB population stopped, >> duration 104.3s > > >> Surprising here, that the dictionary population gives the same time, >> but the BerkeleyDB inserts the records 6 times faster on my computer >> than on yours. I am running Python 2.4.2 on Windows XP SP2, and you? > > Fedora core 5 with ext3 filesystem. The difference will be due to > the way that Windows buffers writes for the filesystem you're using > (it sounds like you're using a FAT-based file system). Ok, according to the Windows task manager the Python process reads/writes to the file system during the run of BerkeleyDB test around 7 GByte(!) of data and the hard drive is continuously busy, where the size of file I found in the Temp directory is always below 20 MByte. The hard drive access is probably the main reason for loosing time - here a question to BerkeleyDB experts: Can the BerkeleyDB via Python bsddb3 interface be tuned to use only RAM or as BerkeleyDB can scale to larger data amount it makes not much sense to tweak it into RAM? Chris, is maybe a RAM-disk the right way to go here to save time lost for accessing the file stored in the file system on the hard drive? The RAM requirements, according to Windows XP task manager, are below 100 MByte. I am using the NTFS file system (yes, I know, that FAT is in some configurations faster than NTFS) and XP Professional SP2 without any tuning of file system caching. The CPU is 100% busy. What CPU and RAM (SIMM, DDR, DDR2) do you have? I have 2GByte fast DDR PC400/3200 dual line RAM. It seems, that you are still not getting results within the range others experience running your code, so I suppose, it has something to do with the hardware you are using. > >>> Number generator test for 100 number ranges >>> with a maximum of 3 wildcard digits. >>> Wed May 17 22:18:17 2006 dictionary population started >>> Wed May 17 22:18:26 2006 dictionary population stopped, duration 8.6s >>> Wed May 17 22:18:27 2006 StorageBerkeleyDB population started >>> Wed May 17 22:29:32 2006 StorageBerkeleyDB population stopped, >>> duration 665.6s >>> Wed May 17 22:29:33 2006 StorageSQLite population started >>> Wed May 17 22:30:38 2006 StorageSQLite population stopped, duration >>> 65.5s >> As I don't have SQLite installed, it is interesting to see if the >> factor 10 in the speed difference between BerkeleyDB and SQLite can be >> confirmed by someone else. >> Why is SQLite faster here? I suppose, that SQLite first adds all the >> records and builds the index afterwards with all the records there >> (with db.commit()). > > SQLite is way faster because BerkeleyDB always uses a disk file, > and SQLite is in RAM only. One of the reasons I put an eye on BerkeleyDB is that it pretends to scale to a huge amount (Terrabyte) of data and don't need as much RAM as Python dictionary and it is not necessary to save/load pickled version of the data (i.e. here the dictionary) from/to RAM in order to work with it. I guess, that in your case BerkeleyDB is for the named reasons probably the right way to go, except your data will stay small and the Python dictionary with them will always fit into RAM. Now I am curious to know which path you have decided to go and why? Claudio > > Cheers, > Chris -- http://mail.python.org/mailman/listinfo/python-list
Re: Large Dictionaries
Chris Foote wrote:
> Klaas wrote:
>
>>> 22.2s 20m25s[3]
>>
>>
>> 20m to insert 1m keys? You are doing something wrong.
>
>
> Hi Mike.
>
> I've put together some simplified test code, but the bsddb
> module gives 11m for 1M keys:
>
I have run your code for the bsddb on my P4 2.8 GHz and have got:
Number generator test for 100 number ranges
with a maximum of 3 wildcard digits.
Wed May 17 16:34:06 2006 dictionary population started
Wed May 17 16:34:14 2006 dictionary population stopped, duration 8.4s
Wed May 17 16:34:14 2006 StorageBerkeleyDB population started
Wed May 17 16:35:59 2006 StorageBerkeleyDB population stopped, duration
104.3s
Surprising here, that the dictionary population gives the same time, but
the BerkeleyDB inserts the records 6 times faster on my computer than on
yours. I am running Python 2.4.2 on Windows XP SP2, and you?
> Number generator test for 100 number ranges
> with a maximum of 3 wildcard digits.
> Wed May 17 22:18:17 2006 dictionary population started
> Wed May 17 22:18:26 2006 dictionary population stopped, duration 8.6s
> Wed May 17 22:18:27 2006 StorageBerkeleyDB population started
> Wed May 17 22:29:32 2006 StorageBerkeleyDB population stopped, duration
> 665.6s
> Wed May 17 22:29:33 2006 StorageSQLite population started
> Wed May 17 22:30:38 2006 StorageSQLite population stopped, duration 65.5s
As I don't have SQLite installed, it is interesting to see if the factor
10 in the speed difference between BerkeleyDB and SQLite can be
confirmed by someone else.
Why is SQLite faster here? I suppose, that SQLite first adds all the
records and builds the index afterwards with all the records there (with
db.commit()).
Can the same be done in BerkeleyDB, or does BerkeleyDB not support
inserting of records without building an index each single insert
command? If yes, how?
Claudio
>
> test code is attached.
>
>> With bdb's it is crucial to insert keys in bytestring-sorted order.
>
>
> For the bsddb test, I'm using a plain string. (The module docs list a
> string being the only datatype supported for both keys & values).
>
>> Also, be sure to give it a decent amount of cache.
>
>
> The bsddb.hashopen() factory seems to have a bug in this regard; if you
> supply a cachesize argument, then it barfs:
>
>
> File "bsddb-test.py", line 67, in runtest
> db = bsddb.hashopen(None, flag='c', cachesize=8192)
> File "/usr/lib/python2.4/bsddb/__init__.py", line 288, in hashopen
> if cachesize is not None: d.set_cachesize(0, cachesize)
> bsddb._db.DBInvalidArgError: (22, 'Invalid argument --
> DB->set_cachesize: method not permitted when environment specified')
>
>
> I'll file a bug report on this if it isn't already fixed.
>
> Cheers,
> Chris
>
>
>
>
> import bsddb
> import random
> import time
> import sqlite
>
> import psyco
> psyco.full()
>
> class WallClockTimer(object):
> '''Simple timer for measuring tests.'''
> def __init__(self, msg):
> self.start = time.time()
> self.msg = msg
> print time.ctime(self.start), self.msg, 'started'
>
> def stop(self):
> end = time.time()
> duration = end - self.start
> print time.ctime(end), self.msg, 'stopped, duration %.1fs' % duration
>
> class NumberGen(object):
> '''Phone number generator for testing different methods of storage.'''
> def __init__(self, range_start, range_end, n, max_wildcards):
>
> print '''Number generator test for %d number ranges
> with a maximum of %d wildcard digits.''' % (n, max_wildcards)
>
> wildcards = range(0, max_wildcards + 1)
> # generate n unique numbers and store as keys in number_hash
> timer = WallClockTimer('dictionary population')
> self.number_hash = {}
>
> for i in xrange(0, n):
> unique = False
> while not unique:
> wildcard_digits = self.gen_wildcard_digits(wildcards)
> num = self.gen_number(range_start, range_end)
> if wildcard_digits > 0:
> num /= (10 ** wildcard_digits)
> key = (num, wildcard_digits)
> if self.number_hash.has_key(key):
> unique = False
> else:
> unique = True
> self.number_hash[key] = None
> timer.stop()
>
> def gen_wildcard_digits(self, wildcards):
> return random.choice(wildcards)
>
> def gen_number(self, start, end):
> return int(random.uniform(start, end))
>
> def storage_test(self, StorageTestClass):
> test = StorageTestClass(self.number_hash)
>
> class StorageTest(object):
> '''base class for testing storage. Derive a test
> class and provide your own runtest() method.'''
> def __init__(self, number_hash):
> timer = W
Re: Large Dictionaries
Chris Foote wrote:
> Claudio Grondi wrote:
>
>> Chris Foote wrote:
>>
>>> However, please note that the Python bsddb module doesn't support
>>> in-memory based databases - note the library documentation's[1] wording:
>>>
>>> "Files never intended to be preserved on disk may be created
>>> by passing None as the filename."
>>>
>>> which closely mirrors the Sleepycat documentation[2]:
>>>
>>> "In-memory databases never intended to be preserved on disk
>>> may be created by setting the file parameter to NULL."
>>>
>>> It does actually use a temporary file (in /var/tmp), for which
>>> performance for my purposes is unsatisfactory:
>>>
>>> # keys dictionary metakit bsddb (all using psyco)
>>> -- -- --- -
>>> 1M8.8s 22.2s 20m25s[3]
>>> 2M 24.0s 43.7s N/A
>>> 5M 115.3s105.4s N/A
>>>
>>> Cheers,
>>> Chris
>>>
>>> [1] bsddb docs:
>>> http://www.python.org/doc/current/lib/module-bsddb.html
>>>
>>> [2] Sleepycat BerkeleyDB C API:
>>> http://www.sleepycat.com/docs/api_c/db_open.html
>>>
>>> [3] Wall clock time. Storing the (long_integer, integer) key in
>>> string form "long_integer:integer" since bsddb doesn't support keys
>>> that aren't integers or strings.
>
> >
>
>> I have to admit, that I haven't wrote any own code to actually test
>> this, but if 20m25s for storing of a single MByte of strings in a
>> database table index column is really what you are getting, I can't
>> get rid of the feeling, that there is something elementary wrong with
>> your way doing it.
>
>
> Hi Claudio.
>
> 1M is one million, referring to the number of insertions of keys;
> not a Megabyte. I'm sorry that you took it that way :-(
>
> Berkeley DB is great for accessing data by key for things already
> stored on disk (i.e. read access), but write performance for each
> key-value pair is slow due to it being careful about flushing
> writes to disk by default.
>
>> Posting the code for your test cases appears to me to be the only
>> option to see what is the reason for the mystery you are getting here
>> (this will clarify also the other mysterious things considered by the
>> posters to this thread up to now).
>
>
> I agree that posting some test code would have proved useful, but
> the code is big and has too many interdependencies on external
> things (e.g. databases, threads & pyro RPC calls) to allow me
> to separate out examples easily. But if you go back to my
> original posting, I think my question was quite clear.
>
> Best regards,
> Chris
>
I have some code demonstrating the mystery from my point of view
(including timings which differ from what you experience in the order of
a magnitude on my 2.8 GHz P4):
dctA = {}
import time
strt = time.clock()
lst = range(12345678901000L, 12345678901000L + 500)
import random
endt = time.clock()
print "lst = range(12345678901L, 12345678901L + 500) [s] : ", endt -
strt
strt = time.clock()
random.shuffle(lst)
endt = time.clock()
print "random.shuffle(lst) [s] : ", endt -
strt
random.shuffle(lst)
counter = 0
for item in lst:
if (counter == 0):
print "Listing of some of lst items:"
if (counter > 4):
print ":END of listing of lst items"
break
else:
print "item no. %i "%(counter,), repr(item)
counter += 1
#:for
# raw_input('continue with ENTER) #>')
strt = time.clock()
dctA.fromkeys(lst, None)
endt = time.clock()
print "dctA.fromkeys(lst, None) [s] : ", endt -
strt
# raw_input('continue with ENTER) #>')
strt = time.clock()
strLst = []
for item in lst: strLst.append(str(item))
dctA = {}
endt = time.clock()
print "for item in lst: strLst.append(str(item)) [s] : ", endt -
strt
# raw_input('continue with ENTER) #>')
strt = time.clock()
dctA = dctA.fromkeys(strLst, None)
endt = time.clock()
print "dctA.fromkeys(strLst, None) [s] : ", endt -
strt
# raw_input('continue with ENTER) #>')
print "len(dctA) : %i " % (len(dctA),)
counter = 0
for key in dctA.keys():
if (counter == 0):
print "Listing of some of dctA items:"
if (counter > 4):
print ":END of listing of dctA items"
break
else:
print "key no. %i "%(counter,), repr(key)
counte
Re: Large Dictionaries
Chris Foote wrote: > Claudio Grondi wrote: > >> Chris Foote wrote: >> >>> p.s. Disk-based DBs are out of the question because most >>> key lookups will result in a miss, and lookup time is >>> critical for this application. >>> >> Python Bindings (\Python24\Lib\bsddb vers. 4.3.0) and the DLL for >> BerkeleyDB (\Python24\DLLs\_bsddb.pyd vers. 4.2.52) are included in >> the standard Python 2.4 distribution. > > > However, please note that the Python bsddb module doesn't support > in-memory based databases - note the library documentation's[1] wording: > > "Files never intended to be preserved on disk may be created by > passing None as the filename." > > which closely mirrors the Sleepycat documentation[2]: > > "In-memory databases never intended to be preserved on disk > may be created by setting the file parameter to NULL." > > It does actually use a temporary file (in /var/tmp), for which > performance for my purposes is unsatisfactory: > > # keys dictionary metakit bsddb (all using psyco) > -- -- --- - > 1M8.8s 22.2s 20m25s[3] > 2M 24.0s 43.7s N/A > 5M 115.3s105.4s N/A > > Cheers, > Chris > > [1] bsddb docs: > http://www.python.org/doc/current/lib/module-bsddb.html > > [2] Sleepycat BerkeleyDB C API: > http://www.sleepycat.com/docs/api_c/db_open.html > > [3] Wall clock time. Storing the (long_integer, integer) key in string > form "long_integer:integer" since bsddb doesn't support keys that aren't > integers or strings. I have to admit, that I haven't wrote any own code to actually test this, but if 20m25s for storing of a single MByte of strings in a database table index column is really what you are getting, I can't get rid of the feeling, that there is something elementary wrong with your way doing it. Posting the code for your test cases appears to me to be the only option to see what is the reason for the mystery you are getting here (this will clarify also the other mysterious things considered by the posters to this thread up to now). Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: IDLE confusion
MrBlueSky wrote:
> Hi, I'm trying to use IDLE to develop My First Python App and my head
> hurts...
>
> I've a file called spalvi.py with this in it:
> from Test import *
> firstTest("Mike")
>
> And a file called Test.py with this in it:
> def firstTest(name):
> print "Yo",name
>
> I open spalvi.py with IDLE and Run it. It says "Yo Mike".
> I use the File menu to open Test.py and change the message from "Yo" to
> "Hi".
> I Run it again it still says "Yo Mike" :-(
> I close everything down, open spalvi.py with IDLE and Run it again. It
> says "Hi Mike".
>
> So I'm obviously not using IDLE in the "right" way. But what *is* the
> "right" way, when you're trying to develop using several source files?
>
> John
>
You need some deeper understanding of what import does and what happens
when you import again (after the library files have changed).
Try in the IDLE menu [Shell] "Restart Shell" (Ctrl+F6) each time you
have changed something in your files - this "resets" anything previously
imported, which stays the same way otherwise.
Claudio
--
http://mail.python.org/mailman/listinfo/python-list
Re: Large Dictionaries
Chris Foote wrote: > Hi all. > > I have the need to store a large (10M) number of keys in a hash table, > based on a tuple of (long_integer, integer). The standard python > dictionary works well for small numbers of keys, but starts to > perform badly for me inserting roughly 5M keys: > > # keys dictionary metakit (both using psyco) > -- -- --- > 1M8.8s 22.2s > 2M 24.0s 43.7s > 5M 115.3s105.4s > > Has anyone written a fast hash module which is more optimal for > large datasets ? > > p.s. Disk-based DBs are out of the question because most > key lookups will result in a miss, and lookup time is > critical for this application. > > Cheers, > Chris Python Bindings (\Python24\Lib\bsddb vers. 4.3.0) and the DLL for BerkeleyDB (\Python24\DLLs\_bsddb.pyd vers. 4.2.52) are included in the standard Python 2.4 distribution. "Berkeley DB was 20 times faster than other databases. It has the operational speed of a main memory database, the startup and shut down speed of a disk-resident database, and does not have the overhead of a client-server inter-process communication." Ray Van Tassle, Senior Staff Engineer, Motorola Please let me/us know if it is what you are looking for. Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: How to install pyTrix?
DeepBlue wrote: > Hi all, > > can any1 please tell me how to install pyTrix? > > thx! The first step should be to get it, but how can it be got? Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: I have problems with creating the classic game Wumpus. the file: http://esnips.c
Ben Sizer wrote: > connyledin wrote: > >>Im trying to create a version of the game Wumpus. Mine is called >>Belzebub. But im STUCK! And its due tuesday 2 maj. Im panicing! Can >>some one help me?? >>here is the file: >>http://esnips.com/webfolder/b71bfe95-d363-4dd3-bfad-3a9e36d0 >> >>What i have the biggest problems with now is between line 8 and 23. > > > Perhaps you could post those lines so we get an idea of what's going > wrong. You also really need to explain what exactly you are hoping to > achieve, and in what way it isn't working. I don't think anybody is > likely to give you a full solution because that would defeat the > object. > I have trouble to get rid of the impression, that the (hidden from direct detection) goal of the original posting is some kind of promotion. Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: ANN: 3 new ShowMeDo.com videos on Python Objects
Ian Ozsvald wrote: > Summary: > At http://ShowMeDo.com we have 3 new videos by Jerol Harrington > introducing Python Objects: > http://showmedo.com/videoListPage?listKey=IntroductionToPythonObjectsUsingIPython_JerolH > > > and 5 videos for beginners to wxPython: > http://showmedo.com/videoListPage?listKey=PythonWxPythonBeginnersSeries > making a total of 20 ShowMeDo videos, mostly about Python, all free. > > Detail: > Jerol Harrington's ShowMeDos use the IPython shell to give an > introduction to using 'class', '__init__', inheritance and attributes. > I have been using Python for 3 years and I got something out of each of > the videos. These videos represent our first user-contribution - thanks > Jerol! > > In the 5 wxPython videos Kyran Dale walks you through the first steps > of using wxPython - from downloading, installing (MS Windows), a first > application and on to menubars and event handling. > > Also, in our latest update we have enabled voting on other user's requests: > http://showmedo.com/requests > There are 14 requests right now, feel free to share your view on which > videos you would like to see. Some of the requests include List > Comprehensions, py2exe, wxGlade, PyDev and other IDEs. > Please get in touch if we're missing a topic that ought to be included, > we will do our best to have new ShowMeDos made. > > About ShowMeDo.com: > Free videos (we call them ShowMeDos) showing you how to do things. The > videos are made by us and our users, for everyone. > > Ian Ozsvald, Kyran Dale > > [EMAIL PROTECTED] > http://ShowMeDo.com (http://blog.showmedo.com) > A small hint for everyone interested in gaining deep understanding about Python objects: To gain deep understanding of what objects in Python are is one of the biggest challenges when learning Python. There are so many different ways of understanding what Python objects are and how Python works with them (depending on how deep you want your understanding to be, how deep you are ready to dig into this subject and how experienced in programming and smart you are), that whenever in the past someone asked a question related to such deep understanding in this or other Python related newsgroups, the outcome was a never ending discussion thread with many controversial postings. Please be aware, that the video about Python objects (I have checked out myself part I) is not consistent in using terms and explaining things, so take care assuming that you get enlightened about Python objects watching it. Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: Difference in Python and Ruby interactive shells
dmh2000 wrote:
> I am experimenting with the interactive interpreter environments of
> Python and Ruby and I ran into what seems to be a fundamental
> difference. However I may be doing something wrong in Python. Please
> comment and correct me if I am wrong
>
> In both languages, you can start up the interactive interpreter
> ('python' and 'irb'), load source files and do stuff, like create
> objects and call their methods. When you want to change something, you
> can edit those same source files outside the environment and reload
> them from within the interactive environment. But, here is the
> difference: with Python, when you reload the source file (module in
> Python terms), it seems that your existing variables stay bound to the
> implementations from the old version of the module. In Ruby, after a
> reload, your existing variables point to the new implementation. At
> least, that is what happens with classes and their methods. This means
> that in Python, if you have an application built up interactively, with
> references to objects, data structures pointing to objects, etc., you
> would have to reconstruct that application to get the new
> implementation. With Ruby, when you load the new implementation, you
> get it immediately.
I am not fully sure it is what you are speaking about, but I mean, that
you have overlooked, that in the IDLE menu item [Shell] you have the
option to restart the shell what takes you back to the initial state.
It would be interesting to know for me here, if there is in Ruby the
option to keep the old values when starting a new run?
Claudio
>
> here is a simple example:
>
> ---
> Python interactive shell (python.exe)
>
> C:\home\dh0072\rq>python
> Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)]
> on win32
> Type "help", "copyright", "credits" or "license" for more information.
>
> # load a local file b.py for test
>
>
import b
>
>
> # instantiate an object and call a method
>
>
x = b.B()
x.p()
>
> B
>
> # edit b.py offline to change method 'p' to do something different then
> reload
>
>
reload(b)
>
>
x.p()
>
> B
>
> # binding of variable 'x' IS NOT changed. points to old version
>
>
y = b.B()
y.p()
>
> B
>
> # new instance of 'b' points to new version
>
>
> ---
> Ruby interactive shell (irb.exe)
>
> C:\home\dh0072\rq>irb
>
> # load a local file b.py for test
>
> irb(main):001:0> load "b.rb"
> => true
>
> # instantiate an object and call a method
>
> irb(main):002:0> x = B.new
> => #
> irb(main):003:0> x.p
> B
> => nil
>
> # edit b.py offline to change method 'p' to do something different then
> reload
>
> irb(main):004:0> load "b.rb"
> => true
> irb(main):005:0> x.p
>
> => nil
>
> # binding of variable 'x' IS changed. points to new version
>
> irb(main):006:0> y = B.new
> => #
> irb(main):007:0> y.p
>
> => nil
>
> # new instance of 'b' points to new version
>
--
http://mail.python.org/mailman/listinfo/python-list
Re: Identifying filled circles in a scanned image
Douglas Douglas wrote: > First of all. Thanks Claudio and nikie for your fast answers. > > What I want to do is process the forms automatically. Each circle match a > letter (like in a quiz). So if the user select option A in a row, he fills the > first circle. If he select option C in the next row, he fills the third circle > of that row. Get it? > > What I want is to write a script that takes the images and say: > Form 1: > Options Selected: A, B, C, A, B, C, A, D, E, A > Form 2: > Options Selected: B, B, B, A, A, A, C, D, E, A > > > I've read the PIL tutorial. I've processed images before using PIL (basic > stuff > though). I thought about using the difference method of the ImageChops module. > It almost worked. The problem is that when the scanner takes the pages it > doesn't put it in the same exact position in its bed. There's not a lot of > difference, but there's a little. > > I also thought about drawing references in the forms (like a line or an X) I > could look for. > > I like to hear your comments now that I explained a little more. The in my eyes simplest solution is already proposed by Paul. There is a nice object detection (look for .find_objects in numarray docu) built-in into the numarray module you can use to search for the circles and other elements in your form. Check it out if you want to speed up detection of objects in the scan or areas of the scan of the form (use PILs .tostring() and numarray .fromstring() to pass the picture pixel data from PIL to numarray). Claudio > > > > __ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Identifying filled circles in a scanned image
Douglas Douglas wrote: > Hi everybody. > > I have a paper form that I scan into an image. My user fills some circles in > this paper form using black ink. Every form has ten rows with five circles > each > and the user fills only one circle for each row. > > I was wondering if I could use the Python Imaging Library to process these > forms. I know the Image class is really powerful, but I can't think of a way > of > how to locate wich circle was filled. > > Could anybody please give me an advice on this? If you know the position of the circles in the picture (if you scanned properly you know it), you can test the mean darkness of the pixels within the circle (dark pixels: filled circle, bright pixels: not filled one). PIL allows access to any single pixel value (e.g. triple of values in case of RGB) in the picture, so, that you can test it against self-chosen threshold values. It could be helpful to try to be a bit more specific about your problem (you have read the PIL tutorial, right?) if this above is not what you are looking for. Claudio > > Thanks. > > __ > Do You Yahoo!? > Tired of spam? Yahoo! Mail has the best spam protection around > http://mail.yahoo.com -- http://mail.python.org/mailman/listinfo/python-list
Re: a problem to solve
John Salerno wrote: > Ok, here's a problem I've sort of assigned to myself for fun, but it's > turning out to be quite a pain to wrap my mind around. It's from a > puzzle game. It will help if you look at this image: > > http://www.johnjsal.devisland.net/switches.jpg > > Here's the situation: Each of the four rows in the diagram is considered > a single 'panel'. Each panel has eight 'switches', which are composed of > two columns each, and these columns have a total of 20 lights (10 in one > column, 10 in the other). The picture hopefully makes this description > clear. > > The shaded boxes denote which lights turn on when you select that > particular switch. So, the very first switch in the first row, if turned > on, would turn on the first four lights, not the fifth, turn on the > sixth, not the seventh, and turn on 8-14, etc. up to the 20th light. > > You can only turn on one switch per panel, so eventually you will have > four switches lit. > > What you are shooting for is to find a combination of four switches so > that exactly three lights in the same location are lit, no more or less. > So turning on the first switch in each row would not work, because that > would mean that the second light in each switch would be lit, and you > can't have all four lit, just three. > > So anyway, the reason I describe all this isn't so you can solve it for > me, because I really want to figure it out. I just was hoping you can > give me some tips as to how to implement the algorithm. Maybe Python has > some functions that I can use to make it easier than it seems. > > My plan right now is to do a comparison of panel 1, switch 1, light 1 > with panel 2, switch 1, light 1, then panel 3, switch 1, light 1, etc., > which sounds scary. I didn't know if Python had a way to compare the > whole switch in one step, perhaps. > > Also, I was wondering how I might implement the switches as data > structures. Right now I have them as a list of 32 strings, but I thought > maybe some type of bitwise comparisons could help. > > Anyway, any advice for how to proceed would be great! I hope I described > it well enough. 1. implement the switches as a list of integers composed out of bits for each light (1 for light turned on, 0 for off). 2. you can find if four lights are turned on by doing bit comparison of four integers (the & operator). If you get as result a non-zero value you know, that there are four lights in same location switched on 3. run a loop over the first eight switches and in this loop a loop over the second row, etc., so that you get all possible arrangement tested (four nested loops) 4. if you find a case the four switches gives a zero value, test if you have exactly three ligths on You need to know: what is a list of integers how to do bitwise and with integers (the & operator) how to loop over list elements using their index ( e.g with "for listIndex in range(0,8):") how to set single bits of an integer ( e.g. intVal | 0x01, intVal | 0x02, intVal | 0x04, etc. ) Hope this helps Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: why? [win32com/WMI]
Sergey wrote:
> import win32com.client
>
> loc = win32com.client.Dispatch("WbemScripting.SWbemLocator")
> svc = loc.ConnectServer("srv", "root/cimv2", "[EMAIL PROTECTED]", "**")
> sys = svc.get("Win32_Process")
> sys.create("notepad.exe")
>
> =>
>
> Traceback (most recent call last):
> File "remote.py", line 6, in ?
> sys.create("notepad.exe")
> TypeError: 'int' object is not callable
>
>
I have no idea what are you doing in your code or if the following has
something to do with your problem, but sometimes it good to know, that
Windows requires Unicode strings as parameter and giving full path file
names is always better than to rely on some default mechanisms.
Claudio
--
http://mail.python.org/mailman/listinfo/python-list
Re: Searching for uniqness in a list of data
rh0dium wrote:
> Hi all,
>
> I am having a bit of difficulty in figuring out an efficient way to
> split up my data and identify the unique pieces of it.
>
> list=['1p2m_3.3-1.8v_sal_ms','1p2m_3.3-1.8_sal_log']
>
> Now I want to split each item up on the "_" and compare it with all
> others on the list, if there is a difference I want to create a list of
> the possible choices, and ask the user which choice of the list they
> want. I have the questioning part under control. I can't seem to get
> my hands around the logic - the list could be 2 items or 100 long. The
> point of this is that I am trying to narrow a decision down for an end
> user. In other words the end user needs to select one of the list
> items, and by breaking it down for them I hope to simplify this.
>
> list=['1p2m_3.3-1.8v_sal_ms','1p6m_3.3-1.8_sal_log']
> would only question the first data set ['1p2m', '1p6m' ]
>
> list=['1p2m_3.3-1.8v_sal_ms','1p2m_3.3-1.8v_pol_ms','1p3m_3.3-18.v_sal_ms']
> If on the list ['1p2m','1p2m','1p3m'] the user selected 1p2m then the
> next list would only be ['sal','pol']
> but if the user initially only selected 1p3m they would be done..
>
> I hope this clarifies what I am trying to do. I just can't seem to get
> my hands around this - so an explaination of logic would really be
> helpfull. I picture a 2d list but I can't seem to get it..
>
list=['1p2m_3.3-1.8v_sal_ms','1p2m_3.3-1.8v_pol_ms','1p3m_3.3-18.v_sal_ms']
dictQlevel_1={}
dictQlevel_2={}
dictQlevel_3={}
for item in list:
splitted = item.split('_')
dictQlevel_1[splitted[0]] = True
dictQlevel_2[splitted[1]] = True
dictQlevel_3[splitted[2]] = True
print 'choose one of: '
for key_1 in dictQlevel_1.keys():
print key_1
print
usrInput = raw_input()
if usrInput == '':
print 'choose one of: '
for key_1 in dictQlevel_1.keys():
for key_2 in dictQlevel_2.keys():
print key_1, key_2
print
usrInput = raw_input()
else:
pass
# or do something
# etc.
Hope it is what you are looking for.
Claudio
--
http://mail.python.org/mailman/listinfo/python-list
Re: Firebird and Python
haxier wrote: > All the info you need is in the kinterbasdb module. I've worked with it > under windows and Linux and... "it just works". Really well indeed. I'd > recommend it a lot. > > http://kinterbasdb.sourceforge.net/dist_docs/usage.html#faq_fep_embedded_using_with > > -- > Asier. > Thanks - you have provided exactly the information I asked for. Thanks also to Magnus (yes I asked for the in-process feature). Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: Firebird and Python
Ray Cote wrote: > At 5:07 PM +0100 2/27/06, Magnus Lycka wrote: > >> >> I'm still interested >> in experiences from Pythonistas using Firebird-- >> especially embedded. > > > Works great. > Python and Firebird embedded (at least on Windows) is very simple to use. > Not currently using it on other platforms. > --Ray > May I ask what do I need to use embedded Firebird from Python on Windows? Which Firebird download (http://www.firebirdsql.org/download/prerelease/win32/Firebird-2.0.0.12169-0_embed_win32.zip ?) and which Python module (http://kinterbasdb.sourceforge.net/ ?) Do I understand it right, that I can use Firebird database in the same way as I can use the Berkeley one provided in Python standard distribution? Claudio -- http://mail.python.org/mailman/listinfo/python-list
Re: bsddb3 database file, are there any unexpected file size limits occuring in practice?
Klaas wrote: > Claudio writes: > >>I am on a Windows using the NTFS file system, so I don't expect problems >>with too large file size. > > > how large can files grow on NTFS? I know little about it. No practical limit on current harddrives. i.e.: Maximum file size Theory: 16 exabytes minus 1 KB (2**64 bytes minus 1 KB) Implementation: 16 terabytes minus 64 KB (2**44 bytes minus 64 KB) Maximum volume size Theory:2**64 clusters minus 1 Implementation: 256 terabytes minus 64 KB (2**32 clusters minus 1) Files per volume 4,294,967,295 (2**32 minus 1 file) > > >>(I suppose it in having only 256 MB RAM available that time) as it is >>known that MySQL databases larger than 2 GByte exist and are in daily >>use :-( . > > > Do you have more ram now? I have now 3 GByte RAM on my best machine, but Windows allows a process not to exceed 2 GByte, so in practice a little bit less than 2 GByte are the actual upper limit. I've used berkeley dbs up to around 5 gigs > in size and they performed fine. However, it is quite important that > the working set of the database (it's internal index pages) can fit > into available ram. If they are swapping in and out, there will be > problems. Thank you very much for your reply. In my current project I expect the data to have much less volume than the indexes. In my failed MySQL project the size of the indexes was appr. same as the size of the indexed data (1 GByte). In my current project I expect the total size of the indexes to exceed by far the size of the data indexed, but because Berkeley does not support multiple indexed columns (i.e. only one key value column as index) if I access the database files one after another (not simultaneously) it should work without problems with RAM, right? Do the data volume required to store the key values have impact on the size of the index pages or does the size of the index pages depend only on the number of records and kind of the index (btree, hash)? In last case, I were free to use for the key values also larger sized data columns without running into the problems with RAM size for the index itself, else I were forced to use key columns storing a kind of hash to get their size down (and two dictionaries instead of one). What is the upper limit of number of records in practice? Theoretical, as given in the tutorial, Berkeley is capable of holding up to billions of records with sizes of up to 4 GB each single record with tables up to total storage size of 256 TB of data. By the way: are billions in the given context multiple of 1.000.000.000 or of 1.000.000.000.000 i.e. in US or British sense? I expect the number of records in my project in the order of tens of millions (multiple of 10.000.000). I would be glad to hear if someone has already successful run Berkeley with this or larger amount of records and how much RAM and which OS had the therefore used machine (I am on Windows XP with 3 GByte RAM). Claudio > > -Mike > -- http://mail.python.org/mailman/listinfo/python-list
Re: How to do an 'inner join' with dictionaries
[EMAIL PROTECTED] wrote:
> Let's say I have two dictionaries:
> dict1 is 1:23, 2:76, 4:56
> dict2 is 23:A, 76:B, 56:C
>
> How do I get a dictionary that is
> 1:A, 2:B, 4:C
>
Just copy/paste the following source code to a file and run it:
sourceCodeToExecute = """
dict1 = { 1:23,2:76, 4:56 }
dict2 = { 23:'A', 76:'B', 56:'C' }
# How do I get a dictionary that is
dict3 = { 1:'A', 2:'B', 4:'C' }
dict4 = {}
for key in dict1.keys():
dict4[key] = dict2[dict1[key]]
#:for
print 'dict3 == dict4 is', dict3 == dict4
"""
print
print ' execution of following source code: '
print
print '"""',
print sourceCodeToExecute
print '"""'
print
print ' results in output of: '
print
exec(sourceCodeToExecute)
The core idea of an inner join:
dict2[dict1[key]]
Hope this helps inspite of the fact there is some more complicated code
involved, which I provided to show the beauty of Python from the point
of view of my own style of writing 'tutorial-like' code which explains
itself by its output.
Claudio
--
http://mail.python.org/mailman/listinfo/python-list
