Re: [Tutor] sockets, files, threads
On Tue, 18 Jan 2005, Marilyn Davis wrote: > while 1: > if log.level & log.calls: > log.it("fd%d:py_daemon.py: Waiting ...", self.descriptor) > try: > client_socket, client_addr = self.server_socket.accept() > except (EOFError, KeyboardInterrupt): > self.close_up() > Spawn(client_socket).start() > > > The problem is that, as part of program flow, it appears to run after > > the try block. But in one particular case of program flow, > > 'client_socket' will not be set to a valid value. It is better to put > > that statement a > > I don't understand. Which particular case of program flow will > 'client_socket' be invalid and yet make it through the socket.accept > call? Hi Marilyn, If there is an EOFError or an KeyboardInterrupt, client_socket will maintain the same value as the previous iteration through the whole loop. What this potentially means is that, under strange situations, Spawn() could be called on the same client_socket twice. The issue is that the whole thing's in a 'while' loop, so we have to be careful that the state from the previous loop iteration doesn't leak into the current iteration. [About using the Standard Library] > And since then, please don't shoot me, but I don't immediately trust the > modules. I read them and see how many times they loop through the data, > and how many copies of the data they put into memory -- and usually > decide to write the simple things I need myself, looping zero times and > keeping only one block in memory. Hmm.. . Do you remember which Standard Library modules you were looking at earlier? Perhaps there was some funky stuff happening, in which case we should try to fix it, so that no one else runs into the same problems. > > ### > > class FileReader(TokenReader): > > def __init__(self, file_socket): > > self.local_name = file_socket.local_name > > self.fread~er_name = '/tmp/fsf%s' % self.local_name > > file_lock.acquire() > > self.freader = open(self.freader_name, "w+") > > file_lock.release() > > ### > > > > The locks around the open() calls are unnecessary: you do not need to > > synchronize file opening here, as there's no way for another thread to > > get into the same initializer of the same instance. > > But I think that I'm only wrapping the calls that create file > descriptors, because those get trampled on. I'm almost positive that the builtin open() function is thread-safe. Let me check that really fast... /*** Within Objects/fileobject.c: open_the_file() ***/ if (NULL == f->f_fp && NULL != name) { Py_BEGIN_ALLOW_THREADS f->f_fp = fopen(name, mode); Py_END_ALLOW_THREADS } /**/ Hmmm! If really depends if the underlying C's fopen() Standard C library is thread safe. This is true in modern versions of LIBC, so I don't think there's anything to worry about here, unless you're using a very ancient version of Unix. > But, I did take out threading and the big error went away. I'm done > with threading, unless I see a big need one day. I don't know what I'll > tell students from here on. I'd point them to John Ousterhout's article on "Why Threads are a Bad Idea (For Most Purposes)": http://home.pacbell.net/ouster/threads.pdf > > I thought about this a little bit more: there is one place where you > > do need locks. [text cut] > But, notice that the call to: > > threading.Thread.__init__(self, name = self.local_name) > > came after, so the Spawn.no manipulation always happens in the main > thread. You're right! I must have been completely delirious at that point. *grin* > One problem remains after removing the threading stuff. I still get > those pesky: > > close failed: [Errno 9] Bad file descriptor > > even though my logged openings and closings match up one-to-one. Ok, then that's a good thing to know: since threading is off, we now know that that error message has nothing to do with threads. > Now then, I haven't added that line of code to each except clause yet > because 1) it doesn't seem to cause any problem 2) it's a bunch of > busy-work and 3) I'm hoping that, when your illness clears, you'll think > of something less brute-force for me to do. I'd recommend using traceback.format_exc(). It's infuriating to get an error message like that, and not to know where in the world it's coming from. Python's default behavior for exceptions is to print out a good stack trace: one of the best things about Python's default exception handler it is that it gives us a local picture of the error. For the most part, we know around what line number we should be looking at. When we write our own except handlers, the responsibility falls on us to record good error messages. If anything, our own debugging systems should be even better than Python's. So ma
Re: [Tutor] Shadowrun programs
Jack Cruzan wrote: I am on the other coast but I too am looking for a group to play Shadowrun with. With at least three people interested think we could either -- 1) Port or create a python based SRCG (useless python still has a challenge for a collaberative effort.) 2) Make a SR rpg (I was think of in the style of Rogue or net hack but could be something more in depth) 3) Fun and Profit! -err uhm something like it? My time is unfortunately limited. However, I am available for reviews, ideas, playtesting, etc. For the record, I am a Linux user so it would be nice if any GUIs used a cross-platform toolkit. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] A somewhat easier way to parse XML
Max Noel wrote: Hi everyone, I've just spent the last few hours learning how to use the DOM XML API (to be more precise, the one that's in PyXML), instead of revising for my exams :p. My conclusion so far: it sucks (and so does SAX because I can't see a way to use it for OOP or "recursive" XML trees). I'm certain it can be used to do extremely powerful stuff, but as far as usability is concerned, it's ridiculously verbose and naming is inconsistent. I've had a look at Java DOM as well, and it's apparently the same. I share your opinion that DOM is a pita. It's the same in Java because it is a 'language-neutral' spec - i.e. it sucks equally in every language :-) For Python, take a look at ElementTree, it is way easier to use. Amara looks interesting too. http://effbot.org/zone/element-index.htm http://uche.ogbuji.net/uche.ogbuji.net/tech/4Suite/amara/ For Java, try dom4j. http://www.dom4j.org Many people have tried to make more Pythonic XML libraries, you might want to look around before you write your own. Kent This afternoon, I read a bit about YAML and its basic philosophy that everything can be represented as a mix of lists, dictionaries and scalars. Now, setting aside the fact that one look at YAML made me want to ditch XML for data storage purposes completely (which I can't do since there's no Java YAML parser that I know of so far), it came to my mind once again that this is the one thing I want to be able to do in XML. Chances are that's all what 9 out of 10 programmers want to do with XML. In fact, I find it appalling that none of the "standard" XML parsers (DOM, SAX) provides an easy way to do that (yeah, I know that's what more or less what the shelve module does, but I want a language-independent way). So, to wrap my head around DOM, I set out to write a little script that does just that. Introducing xmldict.py and the DataNode class. For example, given the following XML file: Body 6 Quickness 9 ...the DataNode class (yeah, I think I may have implemented that in a slightly bizarre fashion) will produce the following dictionary: {u'attribute': [{u'@key': u'BOD', u'name': u'Body', u'rating': u'6'}, {u'@key': u'QCK', u'name': u'Quickness', u'rating': u'9'}]} As you can see, everything is represented in a mix of dictionaries, lists and unicode strings, and can now be used by a normal human being to write a program that uses this data. Comments, criticism, improvements, suggestions, [whatever]... Would be appreciated. Feel free to use it if you wish. Thanks for your attention. -- Max maxnoel_fr at yahoo dot fr -- ICQ #85274019 "Look at you hacker... A pathetic creature of meat and bone, panting and sweating as you run through my corridors... How can you challenge a perfect, immortal machine?" ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Selecting text
Dear group: I have two lists: 1. Lseq: >>> len(Lseq) 30673 >>> Lseq[20:25] ['NM_025164', 'NM_025164', 'NM_012384', 'NM_006380', 'NM_007032','NM_014332'] 2. refseq: >>> len(refseq) 1080945 >>> refseq[0:25] ['>gi|10047089|ref|NM_014332.1| Homo sapiens small muscle protein, X-linked (SMPX), mRNA', 'GTTCTCAATACCGGGAGAGGCACAGAGCTATTTCAGCCACATGGCATCGGAATTGAGATCGCAGCT', 'CAGAGGACACCGGGCGTTCCACCTTCCAAGGAGCTTTGTATTCTTGCATCTGGCTGCCTGGGACTT', 'CCCTTAGGCAGTAAACAAATACATAAAGCAGGGATAAGACTGCATGAATATGTCGAAACAGCCAGTTTCC', 'AATGTTAGAGCCATCCAGGCAAATATCAATATTCCAATGGGAGCCTTTCGGCCAGGAGCAGGTCAA', 'CCAGAAGGAATGTACTCCTGAAGTGGAGGAGGGTGTTCCTCCCACCTCGGATGAGGAGAAGAAGCC', 'AATTCCAGGAGCGAAGAAACTTCCAGGACCTGCAGTCAATCTATCGGAAATCCAGAATATTGTGAA', 'CTTATGTAAAGCTGAACAGTAGTAGGAAGAAGGATTGATGTGAAGAAATAAAGAGGCA', 'GAAGATGGATTCAATAGCTCACTATATATTTGTATGATGATTGTGAACCTCCTGAATGCCTG', 'AGACTCTAGCAGAAATGGCCTGTTTGTACATTTATATCTCTTCCTTCTAGTTGGCTGTATTTCTTACTTT', 'ATCTTCATGGCACCTCACAGAACAAATTAGCCCATAAATTCAACACCTGGAGGGTGTGGGAG', 'GAGGGATATGAATGGAGAATGATATGGCAATGTGCCTAACGAGATGGTTTCCCAAGCT', 'ACTTCCTACAGTAGGTCAATATTTGGAATGCGAGTTCTTCACCAAATTATGTCACTAA', 'ACTTTGTATGAGTTCAAATAAATATTTGACTAAATGTTGTGA', '>gi|10047091|ref|NM_013259.1| Homo sapiens neuronal protein (NP25), mRNA', 'TGTGCTGCTATTGTGTGGATGCCGCGCGTGTCTTCTCTTCTTTCCAGAGATGGCTAACACCCGAGC', 'TATGGCTTAAGCCGAGAGGTGCAGGAGAAGATCGAGCAGAAGTATGATGCGGACCTGGAGAACAAGCTGG', 'TGGACTGGATCATCCTGCAGTGCGCCGAGGACATAGAGCACCCGCCGGCAGGGCCCACAGAA', 'ATGGTTAATGGACGGGACGGTCCTGTGCAAGCTGATAAATAGTTTATACCCACCAGGACAAGAGCCCATA', 'CCCAAGATCTCAGAGTCAAAGATGGCAAGCAGATGGAGCAAATCTCCCAGTTCCTGCTGCGG', 'AGACCTATGGTGTCAGAACCACCGACATCTTTCAGACGGTGGATCTATGGGAAGGGAAGGACATGGCAGC', 'TGTGCAGAGGACCCTGATGGCTTTAGGCAGCGTTGCAGTCACCAAGGATGATGGCTGCTATCAGAG', 'CCATCCTGGTTTCACAGGAAAGCCCAGCAGAATCGGAGAGGCCCGAGGAGCAGCTTCGCCAGGGAC', 'AGAACGTAATAGGCCTGCAGATGGGCAGCAACAAGGGAGCCTCCCAGGCGGGCATGACAGGGTACGGGAT', 'GCCCAGGCAGATCATGTTAGGACGCGGCATCCTGTGGTAGAGAGGACGAATGTTCCACACCATGGT'] If Lseq[i] is present in refseq[k], then I am interested in printing starting from refseq[k] until the element that starts with '>' sign. my Lseq has NM_014332 element and this is also present in second list refseq. I want to print starting from element where NM_014332 is present until next element that starts with '>' sign. In this case, it would be: '>gi|10047089|ref|NM_014332.1| Homo sapiens small muscle protein, X-linked (SMPX), mRNA', 'GTTCTCAATACCGGGAGAGGCACAGAGCTATTTCAGCCACATGGCATCGGAATTGAGATCGCAGCT', 'CAGAGGACACCGGGCGTTCCACCTTCCAAGGAGCTTTGTATTCTTGCATCTGGCTGCCTGGGACTT', 'CCCTTAGGCAGTAAACAAATACATAAAGCAGGGATAAGACTGCATGAATATGTCGAAACAGCCAGTTTCC', 'AATGTTAGAGCCATCCAGGCAAATATCAATATTCCAATGGGAGCCTTTCGGCCAGGAGCAGGTCAA', 'CCAGAAGGAATGTACTCCTGAAGTGGAGGAGGGTGTTCCTCCCACCTCGGATGAGGAGAAGAAGCC', 'AATTCCAGGAGCGAAGAAACTTCCAGGACCTGCAGTCAATCTATCGGAAATCCAGAATATTGTGAA', 'CTTATGTAAAGCTGAACAGTAGTAGGAAGAAGGATTGATGTGAAGAAATAAAGAGGCA', 'GAAGATGGATTCAATAGCTCACTATATATTTGTATGATGATTGTGAACCTCCTGAATGCCTG', 'AGACTCTAGCAGAAATGGCCTGTTTGTACATTTATATCTCTTCCTTCTAGTTGGCTGTATTTCTTACTTT', 'ATCTTCATGGCACCTCACAGAACAAATTAGCCCATAAATTCAACACCTGGAGGGTGTGGGAG', 'GAGGGATATGAATGGAGAATGATATGGCAATGTGCCTAACGAGATGGTTTCCCAAGCT', 'ACTTCCTACAGTAGGTCAATATTTGGAATGCGAGTTCTTCACCAAATTATGTCACTAA', 'ACTTTGTATGAGTTCAAATAAATATTTGACTAAATGTTGTGA' I could not think of any smart way to do this, although I have tried like this: >>> for ele1 in Lseq: for ele2 in refseq: if ele1 in ele2: k = ele2 s = refseq[ele2].startswith('>') print k,s Traceback (most recent call last): File "", line 5, in -toplevel- s = refseq[ele2].startswith('>') TypeError: list indices must be integers I do not know how to dictate to python to select lines between two > symbols. Could any one help me thanks. K __ Do you Yahoo!? Yahoo! Mail - 250MB free storage. Do more. Manage less. http://info.mail.yahoo.com/mail_250 ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] A somewhat easier way to parse XML
* Max Noel <[EMAIL PROTECTED]> [2005-01-19 00:17]: > Hi everyone, > > I've just spent the last few hours learning how to use the DOM XML > API (to be more precise, the one that's in PyXML), instead of revising > for > my exams :p. My conclusion so far: it sucks (and so does SAX because I > can't see a way to use it for OOP or "recursive" XML trees). > I'm certain it can be used to do extremely powerful stuff, but as > far as usability is concerned, it's ridiculously verbose and naming is > inconsistent. I've had a look at Java DOM as well, and it's apparently > the same. I'm kind of in the same boat as you are and I have come to the conclusion that XML is intended to answer specific questions with discreet answers, not walk the DOM to create a dictionary. I _think_ the idea behind this is that it would be redundant. You already have a "dictionary" of sorts in the XML itself, why create a new one? For me, it seems that the way you are supposed to interact with an XML DOM is to already know what you are looking for, and in theory, you _should_ know ;-) Still, I can't help wishing I had a simple way to create a dict from a DOM. From a Python perspective, that seems more "Pythonic" to me as well. I guess it's just a different way of looking at it. -- David Rock [EMAIL PROTECTED] pgpq4Ua00eRSO.pgp Description: PGP signature ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Shadowrun programs
I am on the other coast but I too am looking for a group to play Shadowrun with. With at least three people interested think we could either -- 1) Port or create a python based SRCG (useless python still has a challenge for a collaberative effort.) 2) Make a SR rpg (I was think of in the style of Rogue or net hack but could be something more in depth) 3) Fun and Profit! -err uhm something like it? > man, I bought the first edition within weeks of it coming out. Spent > part of junior high, high school and a little of college playing all 3 > editions. Miss those days. > > *ANY OF YOU IN THE BAY AREA STILL RUNNING??* > > Enclosed is a cleaned up version using functions. This way, if you want > you could support differing rule versions. Or play around with unit testing. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] py2exe
Hi, > Does ExFileSelectDialog do anything that the standard Tk file dialogs don't? > (if you don't know about it: have a look at the tkFileDialog module) (*Ashamed*) Indeed not. Well... tkFileDialog is a bit more "bulky" that ExFileSelectDialot, but the functionalities are alike. Actually, my Tk reference is "An Introduction to Tkinter" http://www.pythonware.com/library/tkinter/introduction/index.htm I missed tkFileDialog even if I went through the doc 2 or 3 times looking for a substitute. Thanks a lot, Guille ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sockets, files, threads
Thank you so much Danny. I know how hard it is to look at and comment on other people's code. You know I teach C and Python and I have to say, though, that reading students' Python is 100 times easier than reading their C. And, I hope you're feeling better. I hate to think of you struggling through my code when you should be resting. > > Hi Marilyn, > > > [Long program comments ahead. Please forgive me if some of the comments > are overbearing; I'm trying to get over a cold, and I'm very grumpy. > *grin*] > > > Some comments on the code follow. I'll be focusing on: > > > http://www.maildance.com/python/doorman/py_daemon.py > > One of the import statements: > > ### > from signal import * > ### > > > may not be safe. According to: > > http://docs.python.org/tut/node8.html#SECTION00841 > > """Note that in general the practice of importing * from a module or > package is frowned upon, since it often causes poorly readable code. Note that it says it is frowned upon, and I almost never do it, because it 'often causes poorly readable code' -- but not in this case. There is only signal.signal and signal.alarm and the signal numbers, none of which are easy to mistake for something else. I have never seen 'unsafe' --unless you cause yourself a bug by hiding something from yourself, which would be hard to do in this case. But, ok, to encourage you to make suggestions, I used the regular import on signals. No change to the bugs. > However, it is okay to use it to save typing in interactive sessions, and > certain modules are designed to export only names that follow certain > patterns.""" > > > There's a block of code in Server.run() that looks problematic: Darn it! I managed to put up a hybrid version. Here's what I thought I should have, to wrap a lock around the creation of the file descriptor for the client socket: self.server_socket.setblocking(0) while 1: if log.level & log.calls: log.it("fd%d:py_daemon.py: Waiting ...", self.descriptor) try: file_lock.acquire() try: client_socket, client_addr = self.server_socket.accept() finally: file_lock.release() except socket.error, msg: if str(msg).find('Resource temporarily unavailable') > -1: time.sleep(.2) continue except (EOFError, KeyboardInterrupt): self.close_up() Spawn(client_socket).start() Not that it matters to your suggestions. But without locks, this reduces to: while 1: if log.level & log.calls: log.it("fd%d:py_daemon.py: Waiting ...", self.descriptor) try: client_socket, client_addr = self.server_socket.accept() except (EOFError, KeyboardInterrupt): self.close_up() Spawn(client_socket).start() But, I take it, you want me to put Spawn(client_socket).start() in that try: clause. > > The problem is that, as part of program flow, it appears to run after the > try block. But in one particular case of program flow, 'client_socket' > will not be set to a valid value. It is better to put that statement a I don't understand. Which particular case of program flow will 'client_socket' be invalid and yet make it through the socket.accept call? > few lines up, right where 'client_socket' is initialized. Like this: > > ### > try: > client_socket, client_addr = self.server_socket.accept() > Spawn(client_socket).start() > except socket.error, msg: > time.sleep(.5) > except (EOFError, KeyboardInterrupt): > self.close_up() > ### > > Not only does this make it more clear where 'client_socket' is being used, > but it ends up making the code shorter. I've dropped the 'continue' > statement, as it becomes superfluous when the Spawn() moves into the try's > body. But but but, that wraps the whole thread in one try/except clause. That Spawn() call starts the thread. If a client_socket generates an error, we don't want the main thread to sleep. All the socket operations in Spawn.start() are also wrapped in their own (hopefully) intelligent try/excepts. I thought it was good practice to -not- put extra stuff in a try/except. > > In fact, the placement of that statement may account partially for the > error message that you were running into earlier. The earlier error > message: > > ### > close failed: [Errno 9] Bad file descriptor > ### > > can easliy occur if there's an EOFError or KeyboardInterrupt under the > original code. And although KeyboardInterrupt might be unusual, I'm not > so sure if EOFError is. I thought I listed exactly the two errors that come from pounding on the keyboard. I try to be careful not to mask any tracebacks. And, the original w
Re: [Tutor] need advice on streamlining code...
yeah, I wasn't sure about that readline/lines thing, cos I'm not sure how popen works. Well, I'm not positive about it either! But that doesn't mean that you can't comment out what you had, try this, and uncomment the previous if it doesn't work. I would imagine that it works because it seems to me if it can split the whole input, it can certainly split just one split deep!!! Anyway... Jacob On Mon, 17 Jan 2005 21:38:37 -0500, Jacob S. <[EMAIL PROTECTED]> wrote: I seem to always be the one to suggest this, but -- "String methods are better than using the string module because the string module has been ?deprecated? or will be soon. I think that is the word here. So, do this instead." insideipgrepfd = os.popen("grep ifconfig_fxp0 /etc/rc.conf") insideipgrep = insideipgrepfd.readline() ## Says same thing below -- readline() just reads first line insideipfield, insideip = insideipgrep[0].strip().split("=") insideipsplit = insideip.split() insideipquads = insideipsplit[1].split(".") insidemaskquads = insideipsplit[4].split(".") And, heck, I know it wouldn't be that simple, but if the line stays the same but just the numbers change, you can do, insideipgrepfd = os.popen("grep ifconfig_fxp0 /etc/rc.conf") insideipgrep = insideipgrepfd.readlines() ##Wait, if you're just using the first line use insideipgrepfd.readline() insideipgrep = insideipgrep.lstrip("ifconfig_fxp0=\"inet ") temp = insideipgrep.split(" netmask ") insideipquads = temp[0].split(".") insideipmaskquads = temp[1].split(".") Warning, code just above is not very stable -- if the text of the line changes in anyway it won't work. HTH, Jacob Schmidt > The following block of code works, and provides the necessary output > I'm > looking for...but I have a feeling that it's working through sheer > brute > force and could be better: > >insideipgrepfd = os.popen("grep ifconfig_fxp0 /etc/rc.conf") >insideipgrep = insideipgrepfd.readlines() >insideipfield, insideip = > string.split(string.strip(insideipgrep[0]), > "=") >insideipsplit = string.split(insideip, " ") >insideipquads = string.split(insideipsplit[1], ".") >insidemaskquads = string.split(insideipsplit[4], ".") > > the line in /etc/rc.conf looks like: > > ifconfig_fxp0="inet 172.31.2.100 netmask 255.255.255.0" > > Any and all thoughts/pointers are appreciated. > >~elh > > -- > Eric L. Howard e l h @ o u t r e a c h n e t w o r k s . c o > m > > www.OutreachNetworks.com > 313.297.9900 > > JabberID: [EMAIL PROTECTED] Advocate of the Theocratic > Rule > ___ > Tutor maillist - Tutor@python.org > http://mail.python.org/mailman/listinfo/tutor > > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor -- 'There is only one basic human right, and that is to do as you damn well please. And with it comes the only basic human duty, to take the consequences. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] A somewhat easier way to parse XML
Hi everyone, I've just spent the last few hours learning how to use the DOM XML API (to be more precise, the one that's in PyXML), instead of revising for my exams :p. My conclusion so far: it sucks (and so does SAX because I can't see a way to use it for OOP or "recursive" XML trees). I'm certain it can be used to do extremely powerful stuff, but as far as usability is concerned, it's ridiculously verbose and naming is inconsistent. I've had a look at Java DOM as well, and it's apparently the same. This afternoon, I read a bit about YAML and its basic philosophy that everything can be represented as a mix of lists, dictionaries and scalars. Now, setting aside the fact that one look at YAML made me want to ditch XML for data storage purposes completely (which I can't do since there's no Java YAML parser that I know of so far), it came to my mind once again that this is the one thing I want to be able to do in XML. Chances are that's all what 9 out of 10 programmers want to do with XML. In fact, I find it appalling that none of the "standard" XML parsers (DOM, SAX) provides an easy way to do that (yeah, I know that's what more or less what the shelve module does, but I want a language-independent way). So, to wrap my head around DOM, I set out to write a little script that does just that. Introducing xmldict.py and the DataNode class. For example, given the following XML file: Body 6 Quickness 9 ...the DataNode class (yeah, I think I may have implemented that in a slightly bizarre fashion) will produce the following dictionary: {u'attribute': [{u'@key': u'BOD', u'name': u'Body', u'rating': u'6'}, {u'@key': u'QCK', u'name': u'Quickness', u'rating': u'9'}]} As you can see, everything is represented in a mix of dictionaries, lists and unicode strings, and can now be used by a normal human being to write a program that uses this data. Comments, criticism, improvements, suggestions, [whatever]... Would be appreciated. Feel free to use it if you wish. Thanks for your attention. xmldict.py Description: Binary data -- Max maxnoel_fr at yahoo dot fr -- ICQ #85274019 "Look at you hacker... A pathetic creature of meat and bone, panting and sweating as you run through my corridors... How can you challenge a perfect, immortal machine?"___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] style question: when to "hide" variable, modules
On Jan 18, 2005, at 22:50, Kent Johnson wrote: Python, instead, lets you change what attribute access means. The way to do this is with 'properties'. This is kind of an advanced topic, here are two references: http://www.python.org/2.2.1/descrintro.html#property http://www.python.org/doc/2.2.3/whatsnew/sect- rellinks.html#SECTION00034 I have to admit, this is brilliant. Thanks for the pointer, Kent. -- Max maxnoel_fr at yahoo dot fr -- ICQ #85274019 "Look at you hacker... A pathetic creature of meat and bone, panting and sweating as you run through my corridors... How can you challenge a perfect, immortal machine?" ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] style question: when to "hide" variable, modules
Paul Tremblay wrote: On Fri, Jan 14, 2005 at 08:11:32PM -0500, Kent Johnson wrote: - typical Python style is *not* to define setter and getter functions. If you need to mediate attribute access you can do it later using properties. I treid to figure out how this works with no luck. It seems that when you use property, you don't necessarily update the attribute. It seems I want the setattr() and getattr() functions? I always thought it was bad programming to alter an attribute directly in OO programming, but it seems I am mistaken? Well, in Java and C++ programming that is what I was taught. Python culture is different and it seems to work OK. class Backup: __init__(self): self.some_var = 5 burn_obj = Burn(self) class Burn: __init__(self, main): setattr(main, 'some_var', 6) main.some_var = 6 will work just fine. The reason that is given for using accessors is that it gives you a layer of flexibility; if you want to change the representation of the data, or make it a computed attribute, you can do that without impacting clients. Python, instead, lets you change what attribute access means. The way to do this is with 'properties'. This is kind of an advanced topic, here are two references: http://www.python.org/2.2.1/descrintro.html#property http://www.python.org/doc/2.2.3/whatsnew/sect-rellinks.html#SECTION00034 ??? - you might want to consolidate your classes into a small number of modules. This is the style of much of the Python standard library. Look at optparse.py for an example (picked more-or-less at random). I couldn't find the optparse.py module. It's in the standard library - /lib/optparse.py Kent But looking at other packages and modules, it seems that the one module shouldn't run more than 500 lines. I * could* consolidate many of my classes in one file, but that would make very long files. Thanks Paul ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] style question: when to "hide" variable, modules
Thanks for your input (and others, too!). On Fri, Jan 14, 2005 at 08:11:32PM -0500, Kent Johnson wrote: > Date: Fri, 14 Jan 2005 20:11:32 -0500 > From: Kent Johnson <[EMAIL PROTECTED]> > User-Agent: Mozilla Thunderbird 1.0 (Windows/20041206) > Cc: tutor@python.org > Subject: Re: [Tutor] style question: when to "hide" variable, modules > > A few thoughts: > - you might want to make a configuration object that you can pass around, > this is probably better than passing around an instance of the main Burn > class. I actually pass around many instances. Maybe this is bad programming, but I can't think of a better way to share attributes across modules. For example, one class splits lists. I want both my message class and my burn class to have access to the current state of the split list. > - typical Python style is *not* to define setter and getter functions. If > you need to mediate attribute access you can do it later using properties. I treid to figure out how this works with no luck. It seems that when you use property, you don't necessarily update the attribute. It seems I want the setattr() and getattr() functions? I always thought it was bad programming to alter an attribute directly in OO programming, but it seems I am mistaken? class Backup: __init__(self): self.some_var = 5 burn_obj = Burn(self) class Burn: __init__(self, main): setattr(main, 'some_var', 6) ??? > - you might want to consolidate your classes into a small number of > modules. This is the style of much of the Python standard library. Look at > optparse.py for an example (picked more-or-less at random). I couldn't find the optparse.py module. But looking at other packages and modules, it seems that the one module shouldn't run more than 500 lines. I * could* consolidate many of my classes in one file, but that would make very long files. Thanks Paul > - if one class has a helper class or function that isn't used anywhere > else, put the helper into the same module as the client and name the helper > starting with _. > - I tend to worry more about naming with _ in a module that is likely to be > reused. For a module that will probably be only be used once, I don't use _ > at all. In a module that has a public API I try to use _ to distinguish the > implementation details from the public stuff. > > HTH > Kent > > Paul Tremblay wrote: > >During the recent discussion on jython, a poster > >brought up the good point that one should hide > >variables and modules to indicate that they are > >not for public use: > > > >self.__for_internal_purposes = 5 > > > >__internal_stuff.py > >""" > >This module only makes sense for use with > >the parent module. > >""" > > > >So one could write several modules using these > >guidelines. One could then issue the command > >from a shell pydoc internal_stuff, and sure enough, > >one gets a nice listing of all the methods with the > >__methods listed first, signalling to someone who > >hasn't written the program (or to the author who > >returns to it a few years later), that one shouldn't > >look at these methods when looking what is useful > >from the script. > > > >My question is, how far should one take these guidlines? > > > >I am working on an OO script with several modules that > >backs up a hardrive. There will be about 10 modules, but > >really only *one* of them, called backup, would be used > >as a pubilc interface. Should I name the rest of them things > >like __handle_archive.py, __file_system.py, and so on? That > >seems odd, since I have never seen this done before. However, > >it does have an elegance of clearly telling someone how to > >hook into the modules. > > > >Also, maybe more importantly, I want to know how to handle > >attributes that need to be shared. Imagine that there are around > >20 attributes, such as the level the program runs at, and the number > >of the disk being copied that need to be shared with different > >modules. Right now, my setup looks like this: > > > ># this module is called backup.py > > > >class Backup: > > > > __init__(self, verbose, level ): > > self.__make_objects() > > self.verbose = verbose > > self.level = level > > > > > > def __make_objects(self): > >self.burn_obj = paxbac.burn.Burn(self) > >self.archive_obj = paxbac.archive.Archive(self) > > > > def get_disk(self): > >return self.__disk > > > > def set_disk(self, the_num): > > self.__disk = the_num > > > > def backup(self): > > archive_obj.archive() > > burn_obj.burn() > > > >* > > > >#this is aother module called burn.py > > > >class Burn: > > def __init__(self, main): > > self.__main = main > > > > def burn(self): > > cd_num = self.main.get_disk() > > if self__main.level > 3: > >sys.stdout.write('Burning disk %s\n' %cd_num) > > > > > > > >The main module backukp provides a number of public > >methods that are really not public. For examle, no > >one is going to want to use thi
Re: [Tutor] sockets, files, threads
On Tue, 18 Jan 2005, Danny Yoo wrote: > In fact, as far as I can tell, none of the Spawn() threads are > communicating with each other. As long as your threads are working > independently of each other --- and as long as they are not writing to > global variables --- you do not need locks. > > In summary, a lot of that locking code isn't doing anything, and may > actually be a source of problems if we're not careful. Hi Marilyn, I thought about this a little bit more: there is one place where you do need locks. Locking needs to be done around Spawn's setting of its 'no' class attribute: ### class Spawn(threading.Thread): no = 1 def __init__(self, client_socket): Spawn.no = Spawn.no + 2 % 3 ## some code later... self.local_name = str(Spawn.no) The problem is that it's possible that two Spawn threads will be created with the same local_name, because they are both using a shared resource: the class attribute 'no'. Two thread executions can interleave. Let's draw it out. Imagine we bring two threads t1 and t2 up, and that Spawn.no is set to 1. Now, as both are initializing, imagine that t1 runs it's initializer ### t1 ### def __init__(self, client_socket): Spawn.no = Spawn.no + 2 % 3 ## and then passes up control. At this point, Spawn.no = 3. Imagine that t2 now starts up: ### t2 ### def __init__(self, client_socket): Spawn.no = Spawn.no + 2 % 3 ### some code later self.local_name = str(Spawn.no) ## Now Spawn.no = 5, and that's what t2 uses to initialize itself. When t1 starts off where it left off, ### t1 ### ### some code later self.local_name = str(Spawn.no) ## it too uses Spawn.no, which is still set to 5. That's the scenario we need to prevent. Spawn.no is a shared resource: it can have only one value, and unfortunately, both threads can use the same value for their own local_names. This is a major bug, because much of your code assumes that the local_name attribute is unique. This is a situation where synchronization locks are appropriate and necessary. We'll want to use locks around the whole access to Spawn.no. Something like: ### class Spawn(threading.Thread): no = 1 no_lock = threading.Lock() def __init__(self, client_socket): no_lock.acquire() try: Spawn.no = Spawn.no + 2 % 3 ## ... rest of initializer body is here finally: no_lock.release() ### should do the trick. I'm being a little paranoid by using the try/finally block, but a little paranoia might be justified here. *grin* We need to be careful of how locks work. The following code is broken: ### class Spawn(threading.Thread): no = 1 no_lock = threading.Lock() def __init__(self, client_socket):## buggy no_lock.acquire() Spawn.no = Spawn.no + 2 % 3 no_lock.release() self.descriptor = client_socket.fileno() self.client_socket = client_socket no_lock.acquire() self.local_name = str(Spawn.no) no_lock.release() ### ... ### This code suffers from the same bug as the original code: two threads can come in, interleave, and see the same Spawn.no. It's not enough to put lock acquires() and releases() everywhere: we do have to use them carefully or else lose the benefit that they might provide. Anyway, I hope this helps! ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] py2exe
Guillermo Fernandez Castellanos wrote: The thing is, pmw does not seem to have ExFileSelectDialog. Any hint about how I could substitute this lack? Does ExFileSelectDialog do anything that the standard Tk file dialogs don't? (if you don't know about it: have a look at the tkFileDialog module) -- John. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Shadowrun programs
On Jan 18, 2005, at 17:52, Jack Cruzan wrote: Greetings! Ok if any of you read my earlier email you would have seen that: A) I am a newbie programmer. B) A Shadowrun gamer. C) In over my head making a SRCG (Shadowrun Character Generator) so with that in mind gave up making my python based SRCG. Instead I am making python Shadowrun utilities! The first of which was a program size and cost calculator for decker's utilities and skillsofts. (I always hated flipping back and forth in the book to figure it out). Thought I would let you see it and maybe critique? Be kind its first original program I have written. (Well thats not php or shell script). Doesn't look bad. However, you should aim to repeat yourself as little as possible, and the "The program's[...]" string is repeated 4 times. You should delay its display until the end of the program, like that: if rating < 4: availability = "2/7" cost = rating * 100 # same for other options... # [skipping to the end of the program] print "The program's Availability is ", availability, " days and costs ", cost Of course, an even better way is to use string formatting in the print line. But that's not really important. -- Max maxnoel_fr at yahoo dot fr -- ICQ #85274019 "Look at you hacker... A pathetic creature of meat and bone, panting and sweating as you run through my corridors... How can you challenge a perfect, immortal machine?" ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Shadowrun programs
Greetings! Ok if any of you read my earlier email you would have seen that: A) I am a newbie programmer. B) A Shadowrun gamer. C) In over my head making a SRCG (Shadowrun Character Generator) so with that in mind gave up making my python based SRCG. Instead I am making python Shadowrun utilities! The first of which was a program size and cost calculator for decker's utilities and skillsofts. (I always hated flipping back and forth in the book to figure it out). Thought I would let you see it and maybe critique? Be kind its first original program I have written. (Well thats not php or shell script). code follows: ## #Module:SRprsize.py # #Description: Shadowrun program size and cost calculator # #Author: Jack Cruzan jcruzan at gmail dot com# #Date:01/18/05 # ### #get program's size rating = int(raw_input("What is the rating of the program you desire?")) ratingValue = rating**2 multiplier = int(raw_input("What is this programs multiplier?")) #determine the programs cost if rating < 4: print "The programs Availability is 2/7 days and costs:", rating*100 elif rating < 7: print "The programs Availability is 4/7 days and costs:", rating*200 elif rating < 10: print "The programs Availibility is 8/14 days and costs:", rating*500 else: print "The programs Availibility is 16/30 days and costs:", rating*1000 print "The program size is:", ratingValue*multiplier ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Testing
Hi, I have not been getting mail, so I thought I'd see what's up. Thanks. Jim James D Homme, Information Design + Development Highmark Inc. [EMAIL PROTECTED] 412-544-0527 "A gentle answer turns away wrath, but a harsh word stirs up anger." ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: Sending a command to a program using os.system (was [Tutor]: Using os.popen*() and os.spawn*() to interact with a DOS box)
You might be interested in this: http://www.tizmoi.net/watsup/intro.html Kent Orri Ganel wrote: Actually, what I want to do is set Audacity up so that it automatically begins recording when a song plays in Windows Media Player (to begin with) and stops when the song is finished playing. I've already figured out how to force it to record and stop when I want to (though in a kind of ugly hackish way)(*), and now I'm working on learning the controls of WMP so I can test things like time left in the song, etc. I dl'ed the WMP 10 SDK but haven't really had a chance to take a thorough look at it yet, so hopefully it'll have what I need. Cheers, Orri ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] CGI help
Your CGI will be run fresh each time the user submits the form. If you want data to persist between submits you have to either - save the data to a persistent store - a file or database - use a different way to handle the form - I'm not sure of all the options but I think mod_python and FastCGI both keep the CGI process running - or use a server like CherryPy or TwistedMatrix. Kent [EMAIL PROTECTED] wrote: Im trying to make a game where in a dictionary is a list of questions and answers. If i try to output 6 at a time to a cgi script and submit will my script be reset so my delete function of not getting the same question again not work? if this doesnt make sense ill repost ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] Notetab and Python
Hi, Does anyone use Notetab to compile Python programs? If so, are you able to share the clip you use? Thanks. Jim James D Homme, Information Design + Development Highmark Inc. [EMAIL PROTECTED] 412-544-0527 "A gentle answer turns away wrath, but a harsh word stirs up anger." ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
[Tutor] CGI help
Im trying to make a game where in a dictionary is a list of questions and answers. If i try to output 6 at a time to a cgi script and submit will my script be reset so my delete function of not getting the same question again not work? if this doesnt make sense ill repost ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Objects & Classes...
On Jan 18, 2005, at 03:46, Liam Clarke wrote: Curious - what's mod_python? A Python module for the Apache web server, that among other things addresses the main shortcoming of CGI: mod_python (and mod_perl, mod_php and mod_ruby, for that matter) keeps the interpreter into memory (and as part of the server, so to speak). The server doesn't need to load and initialize a new interpreter each time a page is requested. This yields a tremendous speed increase (speaking in requests/second there, not in script execution time), and ensures that mod_python scales up *much* better than CGI. http://www.modpython.org/ -- Max maxnoel_fr at yahoo dot fr -- ICQ #85274019 "Look at you hacker... A pathetic creature of meat and bone, panting and sweating as you run through my corridors... How can you challenge a perfect, immortal machine?" ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] need advice on streamlining code...
Jacob S. wrote: insideipgrep = insideipgrep.lstrip("ifconfig_fxp0=\"inet ") No! The argument to lstrip() is a list of characters, any of which will be stripped! It is *not* a prefix to remove! >>> insideipgrep='ifconfig if 00=_xxx Wow' >>> insideipgrep.lstrip("ifconfig_fxp0=\"inet ") 'Wow' You could use insideipgrep = insideipgrep[len("ifconfig_fxp0=\"inet "):] One minor suggestion - use embedded underscores or capital letters to make your variable names easier to read - instead of insideipgrep use insideIpGrep or inside_ip_grep. Kent ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] sockets, files, threads
Hi Marilyn, [Long program comments ahead. Please forgive me if some of the comments are overbearing; I'm trying to get over a cold, and I'm very grumpy. *grin*] Some comments on the code follow. I'll be focusing on: > http://www.maildance.com/python/doorman/py_daemon.py One of the import statements: ### from signal import * ### may not be safe. According to: http://docs.python.org/tut/node8.html#SECTION00841 """Note that in general the practice of importing * from a module or package is frowned upon, since it often causes poorly readable code. However, it is okay to use it to save typing in interactive sessions, and certain modules are designed to export only names that follow certain patterns.""" There's a block of code in Server.run() that looks problematic: ### while 1: if log.level & log.calls: log.it("fd%d:py_daemon.py: Waiting ...", self.descriptor) try: client_socket, client_addr = self.server_socket.accept() except socket.error, msg: time.sleep(.5) continue except (EOFError, KeyboardInterrupt): self.close_up() Spawn(client_socket).start() ### The potentially buggy line is the last one: Spawn(client_socket).start() The problem is that, as part of program flow, it appears to run after the try block. But in one particular case of program flow, 'client_socket' will not be set to a valid value. It is better to put that statement a few lines up, right where 'client_socket' is initialized. Like this: ### try: client_socket, client_addr = self.server_socket.accept() Spawn(client_socket).start() except socket.error, msg: time.sleep(.5) except (EOFError, KeyboardInterrupt): self.close_up() ### Not only does this make it more clear where 'client_socket' is being used, but it ends up making the code shorter. I've dropped the 'continue' statement, as it becomes superfluous when the Spawn() moves into the try's body. In fact, the placement of that statement may account partially for the error message that you were running into earlier. The earlier error message: ### close failed: [Errno 9] Bad file descriptor ### can easliy occur if there's an EOFError or KeyboardInterrupt under the original code. And although KeyboardInterrupt might be unusual, I'm not so sure if EOFError is. Furthermore, the 'except' blocks can emit more debugging information. You mentioned earlier that you're not getting good error tracebacks when exceptions occur. Let's fix that. Use the 'traceback' module here for all except blocks in your program. This will ensure that you get a good stack trace that we can inspect. Here's what the code block looks like, with those adjustments: ### try: client_socket, client_addr = self.server_socket.accept() Spawn(client_socket).start() except socket.error, msg: time.sleep(.5) log.it(traceback.format_exc()) except (EOFError, KeyboardInterrupt): self.close_up() log.it(traceback.format_exc()) ### I'm using Python 2.4's format_exc() function to get the stack trace as a string. If this version is not available to you, you can use this workaround format_exc(): ### import StringIO import traceback def format_exc(): """Returns the exception as a string.""" f = StringIO.StringIO() traceback.print_exc(file = f) return f.getvalue() ### Once you've made the correction and augmented all the except blocks with traceback logging, try running your program again. We should then be better able to trace down the root cause of those errors. Let me just look through a little bit more, just to see what else we can pick out. >From what I read of the code so far, I believe that a lot of the locking that the code is doing is also superfluous. You do not need to lock local variables, nor do you have to usually lock things during object initialization. For example, FileReader.__init__() has the following code: ### class FileReader(TokenReader): def __init__(self, file_socket): self.local_name = file_socket.local_name self.freader_name = '/tmp/fsf%s' % self.local_name file_lock.acquire() self.freader = open(self.freader_name, "w+") file_lock.release() ### The locks around the open() calls are unnecessary: you do not need to synchronize file opening here, as there's no way for another thread to get into the same initializer of the same instance. In fact, as far as I can tell, none of the Spawn() threads are communicating with each other. As long as your threads are working independently of each other --- and as long as they are not writing to global variables --- you do not need locks. In summary, a lot of that locking
Re: [Tutor] py2exe
Hi, > When I tried, I couldn't figure out how to freeze Tix. I ended up giving up > and > recoding all my Tix stuff to use Pmw instead... > (although I was a lot more new to python in those days) Indeed, I think I'm going to do the same. I find the situation really surprising though. When I go to the dist/tcl subdirectory I see tcl8.4, tk8.4 but no tix directory. Whatever... I think I'll do the same anc switch to pmw. The thing is, pmw does not seem to have ExFileSelectDialog. Any hint about how I could substitute this lack? Thanks, Guille Thanks, Guille > (and the fact that Pmw has actual documentation is an added bonus over Tix > :-) ) > > -- > John. > ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] RE:
Title: RE: Gopinath V, ASDC Chennai wrote: On Wed, 12 Jan 2005, Orri Ganel wrote: > >>> stuff = [[0,'sdfsd','wrtew'], [1, 'rht','erterg']] > >>> stuff > [[0, 'sdfsd', 'wrtew'], [1, 'rht', 'erterg']] > >>> print [stuff[i][0] for i in range(len(stuff))] > [0, 1] Hi Orri, An alternative way to write this is: ### print [row[0] for row in stuff] ### which extracts the first element out of every "row" sublist in 'stuff'. Best of wishes! This is fine.i just want to know if row is a reserve word ? or is it a built in function in IDLe environment .. the word row is not highlighted ,what data type is (row) ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor That'll teach me to read the whole question. In the above list comprehension, stuff is the defined list. row is a variable created by the list comprehension. Basically, this is another way of saying: for row in stuff: print row[0] or for i in range(len(stuff)): print stuff[i][0] In this case, row's type is whatever type that element in stuff is (which is a list). Ie: >>> stuff = [[0,'sdfsd','wrtew'], [1, 'rht','erterg']] >>> print [(row[0], type(row), type(row[0])) for row in stuff] [(0, , ), (1, , )] -- Email: singingxduck AT gmail DOT com AIM: singingxduck Programming Python for the fun of it. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] RE:
Title: RE: Gopinath V, ASDC Chennai wrote: On Wed, 12 Jan 2005, Orri Ganel wrote: > >>> stuff = [[0,'sdfsd','wrtew'], [1, 'rht','erterg']] > >>> stuff > [[0, 'sdfsd', 'wrtew'], [1, 'rht', 'erterg']] > >>> print [stuff[i][0] for i in range(len(stuff))] > [0, 1] Hi Orri, An alternative way to write this is: ### print [row[0] for row in stuff] ### which extracts the first element out of every "row" sublist in 'stuff'. Best of wishes! This is fine.i just want to know if row is a reserve word ? or is it a built in function in IDLe environment .. the word row is not highlighted ,what data type is (row) ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor >>> row Traceback (most recent call last): File "", line 1, in -toplevel- row NameError: name 'row' is not defined >>> import row Traceback (most recent call last): File "", line 1, in -toplevel- import row ImportError: No module named row Looks like its not a reserved word. -- Email: singingxduck AT gmail DOT com AIM: singingxduck Programming Python for the fun of it. ___ Tutor maillist - Tutor@python.org http://mail.python.org/mailman/listinfo/tutor