Re: [Tutor] Parsing data from a set of files iteratively
FINAL SOLUTION: ### LOOP OVER DIRECTORY location = '/Users/spyros/Desktop/3NY8MODELSHUMAN/HomologyModels' zdata = [] for filename in os.listdir(location): filename = os.path.join(location, filename) try: zdata.extend(extract_zcoord(filename)) except NameError: print No such file! except SyntaxError: print Check Your Syntax! except IOError: print PDB file NOT FOUND! else: continue print 'Z-VALUES FOR ALL CHARGED RESIDUES' print zdata #diagnostic ### WRITE Z-COORDINATE LIST TO A BINARY FILE import pickle f1 = open(z_coords1.dat, wb) pickle.dump(zdata, f1) f1.close() f2 = open(z_coords1.dat, rb) zdata1 = pickle.load(f2) f2.close() assert zdata == zdata1, error in pickle/unpickle round trip! On Wed, May 30, 2012 at 1:09 AM, Steven D'Aprano st...@pearwood.infowrote: Steven D'Aprano wrote: location = '/Users/spyros/Desktop/**3NY8MODELSHUMAN/**HomologyModels/' zdata = [] for filename in os.listdir(location): zdata.extend(get_zcoords(**filename)) I only had the filename and not its path, that's why the system was not able to locate the file, so filename = os.path.join(location, filename) was used to solve that. Many thanks to everyone for their time and efforts! Spyros Hah, that can't work. listdir returns the name of the file, but not the file's path, which means that Python will only look in the current directory. You need something like this: location = '/Users/spyros/Desktop/**3NY8MODELSHUMAN/**HomologyModels/' zdata = [] for filename in os.listdir(location): zdata.extend(get_zcoords(os.**path.join(location, filename))) Sorry about that. -- Steven __**_ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/**mailman/listinfo/tutorhttp://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Parsing data from a set of files iteratively
On Wed, May 30, 2012 at 07:00:30AM +0100, Spyros Charonis wrote: FINAL SOLUTION: Not quite. You are making the mistake of many newbies to treat Python exceptions as a problem to be covered up and hidden, instead of as a useful source of information. To quote Chris Smith: I find it amusing when novice programmers believe their main job is preventing programs from crashing. ... More experienced programmers realize that correct code is great, code that crashes could use improvement, but incorrect code that doesn't crash is a horrible nightmare. -- http://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ There is little as painful as a program which prints An error occurred and then *keeps working*. What does this mean? Can I trust that the program's final result is correct? How can it be correct if an error occurred? What error occurred? How do I fix it? Exceptions are your friend, not your enemy. An exception tells you that there is a problem with your program that needs to be fixed. Don't cover-up exceptions unless you absolutely have to. Sadly, your indentation is still being broken when you post. Please ensure you include indentation, and disable HTML or Rich Text posting. I have tried to guess the correct indentation below, and fix it in place, but apologies if I get it wrong. ### LOOP OVER DIRECTORY location = '/Users/spyros/Desktop/3NY8MODELSHUMAN/HomologyModels' zdata = [] for filename in os.listdir(location): filename = os.path.join(location, filename) try: zdata.extend(extract_zcoord(filename)) except NameError: print No such file! Incorrect. When a file is missing, you do not get NameError. This except-clause merely disguises programming errors in favour of a misleading and incorrect error message. If you get a NameError, your program has a bug. Don't just hide the bug, fix it. except SyntaxError: print Check Your Syntax! This except-clause is even more useless. SyntaxErrors happen when the code is compiled, not run, so by the time the for-loop is entered, the code has already been compiled and cannot possibly raise SyntaxError. Even if it could, what is the point of this? Instead of a useful exception traceback, which tells you not only which line contains the error, but even highlights the point of the error with a ^ caret, you hide all the useful information and tease the user with a useless message Check Your Syntax!. Again, if your program raises a SyntaxError, it has a bug. Don't hide the bug, fix it. except IOError: print PDB file NOT FOUND! This, at least, is somewhat less useless than the others. At least it is a valid exception, and if your intention is to skip missing files, catching IOError is a reasonable way to do it. But you don't just get IOError for *missing* files, but also for *unreadable* files, perhaps because you don't have permission to read them, or perhaps because the file is corrupt and can't be read. In any case, as usual, imagine yourself as the recipient of this message: PDB file NOT FOUND! -- what do you expect to do about it? Which file is missing or unreadable? How can you tell? Is this a problem? Are your results still valid without that PDB file's data? If this can be be ignored, IGNORE IT! Don't bother the user with scary messages that a problem occurred, if it isn't a problem! At *most*, print a notice that you have skipped a file: print Skipping file, filename (perhaps giving the reason for skipping it). Or even just ignore it completely: pass else: continue This is pointless. All it does is what would have been done anyway: if no exception occurs, it continues to the next loop. Get rid of it: your code will be shorter and neater without this unnecessary two lines. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Parsing data from a set of files iteratively
On Wed, May 30, 2012 at 8:16 AM, Steven D'Aprano st...@pearwood.infowrote: On Wed, May 30, 2012 at 07:00:30AM +0100, Spyros Charonis wrote: FINAL SOLUTION: Not quite. You are making the mistake of many newbies to treat Python exceptions as a problem to be covered up and hidden, instead of as a useful source of information. To quote Chris Smith: I find it amusing when novice programmers believe their main job is preventing programs from crashing. ... More experienced programmers realize that correct code is great, code that crashes could use improvement, but incorrect code that doesn't crash is a horrible nightmare. -- http://cdsmith.wordpress.com/2011/01/09/an-old-article-i-wrote/ Ok, so basically wrong code beats useless code. There is little as painful as a program which prints An error occurred and then *keeps working*. What does this mean? Can I trust that the program's final result is correct? How can it be correct if an error occurred? What error occurred? How do I fix it? My understanding is that an except clause will catch a relevant error and raise an exception if there is one, discontinuing program execution. Exceptions are your friend, not your enemy. An exception tells you that there is a problem with your program that needs to be fixed. Don't cover-up exceptions unless you absolutely have to. Sadly, your indentation is still being broken when you post. Please ensure you include indentation, and disable HTML or Rich Text posting. I have tried to guess the correct indentation below, and fix it in place, but apologies if I get it wrong. Yes, that is the way my code looks in a python interpreter ### LOOP OVER DIRECTORY location = '/Users/spyros/Desktop/3NY8MODELSHUMAN/HomologyModels' zdata = [] for filename in os.listdir(location): filename = os.path.join(location, filename) try: zdata.extend(extract_zcoord(filename)) except NameError: print No such file! Incorrect. When a file is missing, you do not get NameError. This except-clause merely disguises programming errors in favour of a misleading and incorrect error message. If you get a NameError, your program has a bug. Don't just hide the bug, fix it. except SyntaxError: print Check Your Syntax! This except-clause is even more useless. SyntaxErrors happen when the code is compiled, not run, so by the time the for-loop is entered, the code has already been compiled and cannot possibly raise SyntaxError. What I meant was, check the syntax of my pathname specification, i.e. check that I did not make a type when writing the path of the directory I want to scan over. I realize syntax has a much more specific meaning in the context of programming - code syntax! Even if it could, what is the point of this? Instead of a useful exception traceback, which tells you not only which line contains the error, but even highlights the point of the error with a ^ caret, you hide all the useful information and tease the user with a useless message Check Your Syntax!. Ok, I didn't realize I was being so reckless - thanks for pointing that out. Again, if your program raises a SyntaxError, it has a bug. Don't hide the bug, fix it. except IOError: print PDB file NOT FOUND! This, at least, is somewhat less useless than the others. At least it is a valid exception, and if your intention is to skip missing files, catching IOError is a reasonable way to do it. But you don't just get IOError for *missing* files, but also for *unreadable* files, perhaps because you don't have permission to read them, or perhaps because the file is corrupt and can't be read. Understood, but given that I am reading and processing are standard ASCII text files, there is no good reason (which I can think of) that the files would be *unreadable* I verified that I had read/write permissions for all my files, which are the default access privileges anyway (for the owner). In any case, as usual, imagine yourself as the recipient of this message: PDB file NOT FOUND! -- what do you expect to do about it? Which file is missing or unreadable? How can you tell? Is this a problem? Are your results still valid without that PDB file's data? Perhaps because I was writing the program I didn't think that this message would be confusing to others, but it did help in making clear that there was a different error (in this case, the absence of **filename = os.path.join(location, filename)** to join a filename to its pathway). Without the PDB file's data, there would be no results - because the program operates on each file of a directory successively (all files are .pdb files) and uses data in the file to build a list. So, since I was working on a directory with only PDB files this error says it hasn't found them - which points to a more basic error (the one mentioned above). If this can be be ignored, IGNORE IT! Don't bother the
[Tutor] Joining all strings in stringList into one string
Hello all, I am working on learning Python(on my own) and ran into an exercise that I figured out but I wanted to know if there was a different way to write the code? I know he wanted a different answer for the body because we haven't gotten to the ' '.join() command yet. This is what I have: def joinStrings(stringList): string = [] for string in stringList: print ''.join(stringList) def main(): print joinStrings(['very', 'hot', 'day']) print joinStrings(['this', 'is', 'it']) print joinStrings(['1', '2', '3', '4', '5']) main() thanks all ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Joining all strings in stringList into one string
Seems like a lot of extra work for joining the strings. You should only need: ''.join(['very', 'hot', 'day']) (no spaces) ' '.join(['very', 'hot', 'day']) (if you want spaces) glen On Wed, May 30, 2012 at 11:21 AM, Akeria Timothy akeriatimo...@gmail.comwrote: Hello all, I am working on learning Python(on my own) and ran into an exercise that I figured out but I wanted to know if there was a different way to write the code? I know he wanted a different answer for the body because we haven't gotten to the ' '.join() command yet. This is what I have: def joinStrings(stringList): string = [] for string in stringList: print ''.join(stringList) def main(): print joinStrings(['very', 'hot', 'day']) print joinStrings(['this', 'is', 'it']) print joinStrings(['1', '2', '3', '4', '5']) main() thanks all ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Joining all strings in stringList into one string
On 05/30/2012 12:21 PM, Akeria Timothy wrote: Hello all, I am working on learning Python(on my own) and ran into an exercise that I figured out but I wanted to know if there was a different way to write the code? I know he wanted a different answer for the body because we haven't gotten to the ' '.join() command yet. This is what I have: def joinStrings(stringList): string = [] for string in stringList: print ''.join(stringList) def main(): print joinStrings(['very', 'hot', 'day']) print joinStrings(['this', 'is', 'it']) print joinStrings(['1', '2', '3', '4', '5']) main() thanks all I'm not sure what you want us to do here. You don't quote the exercise description, so about all I can do is point out the bugs and inefficiencies in the code. Your indentation won't work. You have string = indented by one column, and the for loop indented by 3 more. Those two lines have to start in the same column. Of course, if you had run it that way, the compiler would have told you that. So presumably you've retyped it into this message, which is very bad practice. Use copy/paste. File timothy.py, line 3 for string in stringList: ^ IndentationError: unexpected indent Fixing that. Next problem is that main() uses the results of joinStrings(), but there is no return statement in joinStrings(). So it prints None. Presumably you meant to have joinStrings() return its result, INSTEAD of printing it. Next, in joinStrings(), you start by initializing string=. But then you immediately overwrite that value with the for loop, so it was wasted. Presumably you wanted to use two different names there, and do something to the first variable each time through the loop. Next, in the loop, you do the same work multiple times, never using the individual value of string. So, please tell us the actual assignment, and we can help you get there. There is certainly a way to have joinStrings() do the equivalent of .join(), but it wouldn't look much like what you've written. Usually, the practice of making a docstring for each function can help you get the purpose straight in your mind. What does the function expect as argument(s), and what will it do, and what will it return? And what side effects does it have, like printing ? Usually, a function either does some work, or it displays some results. Doing both is a sign of a program that's still being debugged. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Joining all strings in stringList into one string
On 5/30/2012 12:21 PM, Akeria Timothy wrote: In addition to the other comments I point out that join is not a /command/, it is a /string method/. Python does not have commands. -- Bob Gailer 919-636-4239 Chapel Hill NC ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Joining all strings in stringList into one string
On Wed, 2012-05-30 at 12:21 -0400, Akeria Timothy wrote: [...] def joinStrings(stringList): string = [] indentation error in that the above line and the below line should have the same indent level. Also the above line and the following line are both definitions of the variable string so the above is actually redundant. for string in stringList: print ''.join(stringList) Given the variable doesn't appear in the block I wonder if the code reflects the intended algorithm? def main(): print joinStrings(['very', 'hot', 'day']) print joinStrings(['this', 'is', 'it']) print joinStrings(['1', '2', '3', '4', '5']) main() The above code, with the trivial indent fix, outputs: veryhotday veryhotday veryhotday None thisisit thisisit thisisit None 12345 12345 12345 12345 12345 None Is this what was desired? I am tempted to think that actually what was desired was: veryhotday thisisit 12345 in which case I would suggest the code should perhaps read: def main(): print ''.join(['very', 'hot', 'day']) print ''.join(['this', 'is', 'it']) print ''.join(['1', '2', '3', '4', '5']) if __name__ == '__main__': main() but, mayhap, I am missing the intention. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Joining all strings in stringList into one string
A procedural point here: You forgot to include the list, and just replied to me privately. Normally, what you should do is a Reply-All. Or else make sure tutor@python.org is one of the To: or CC: list On 05/30/2012 01:40 PM, Akeria Timothy wrote: I did copy and paste and I'm learning Python on my own using Hands-on Pyton Tutorial and this was one of the exercises. He gave us the everything but the body. So this is what I started out with: def joinStrings(stringList): '''Join all the strings in stringList into one string, and return the result. For example: print joinStrings(['very', 'hot', 'day']) 'veryhotday' ''' # finish the code for this function Very good. Why did you remove that docstring from your own code before posting it? def main(): print joinStrings(['very', 'hot', 'day']) print joinStrings(['this', 'is', 'it']) print joinStrings(['1', '2', '3', '4', '5']) main() SNIP I know it's wrong but it worked so that's why I'm asking for the proper way to do it. The output I got from your code (after fixing the indent) was davea@think:~/temppython$ python timothy.py veryhotday veryhotday veryhotday None thisisit thisisit thisisit None 12345 12345 12345 12345 12345 None and what I think the instructor expected was: veryhotday thisisit 12345 Similar, but not the same. First, the function is NOT supposed to print anything. According to the docstring, the function returns the result, not prints it. So you could come up with: def joinStrings(stringList): '''Join all the strings in stringList into one string, and return the result. For example: print joinStrings(['very', 'hot', 'day']) 'veryhotday' ''' # finish the code for this function return .join(stringList) But as you said in your original message, he presumably hasn't introduced that join method yet, so you would like to do it by hand. Very good way to learn. You had the right concept, of having an 'accumulator to gather the parts of the result together. However, you are not trying to return a list, you're trying to return a string. So the accumulator must be an empty string. I'll give it a different name, so it doesn't conflict with the loop variable. res = for string in stringList: something here that modifies res return res Your challenge is now to supply the line in the middle. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Parsing data from a set of files iteratively
Spyros Charonis wrote:
On Wed, May 30, 2012 at 8:16 AM, Steven D'Aprano st...@pearwood.infowrote:
[...]
There is little as painful as a program which prints An error occurred
and then *keeps working*. What does this mean? Can I trust that the
program's final result is correct? How can it be correct if an error
occurred? What error occurred? How do I fix it?
My understanding is that an except clause will catch a relevant error and
raise an exception if there is one, discontinuing program execution.
No, the opposite. An except clause will catch the exception and *continue*
execution past the end of the try...except block.
Python automatically raises exceptions and halts execution if you do nothing.
For example:
py for x in (1, 0, 2):
... print(1/x)
...
1.0
Traceback (most recent call last):
File stdin, line 2, in module
ZeroDivisionError: division by zero
Notice that the first time around the loop, 1.0 is printed. The second time,
an error occurs (1/0 is not defined), so Python raises an exception. Because I
don't catch that exception, it halts execution of the loop and prints the
traceback.
But a try...except clause will catch the exception and keep going:
py for x in (1, 0, 2):
... try:
... print(1/x)
... except ZeroDivisionError:
... print('something bad happened')
...
1.0
something bad happened
0.5
There are good reasons for catching exceptions. Sometimes you can recover from
the error, or skip the bad data. Sometimes one calculation fails but you can
try another one. Or you might want to catch an exception of one type, and
replace it with a different exception with a more appropriate error message.
But all too often, I see beginners catching exceptions and just covering them
up, or replacing useful tracebacks which help with debugging with bland and
useless generic error messages like An error occurred.
except SyntaxError:
print Check Your Syntax!
This except-clause is even more useless. SyntaxErrors happen when the
code is compiled, not run, so by the time the for-loop is entered, the
code has already been compiled and cannot possibly raise SyntaxError.
What I meant was, check the syntax of my pathname specification, i.e. check
that I
did not make a type when writing the path of the directory I want to scan
over. I realize
syntax has a much more specific meaning in the context of programming -
code syntax!
That's not what SyntaxError does in Python. Python only understands one form
of syntax: *Python* syntax, not the syntax of pathnames to files. If you type
the wrong pathname:
pathname = My Documents!letters!personal!letter to my mother+doc
Python will not raise SyntaxError. It will try to open the file called
My Documents!letters!personal!letter to my mother+doc
*exactly* as you typed it, and either succeed (if by some unimaginable fluke
there happens to be a file of that name!) or fail. If it fails, you will get
an OSError or IOError, depending on the nature of the failure reported by the
operating system.
[...]
But you don't just get IOError for *missing* files, but also for
*unreadable* files, perhaps because you don't have permission to read
them, or perhaps because the file is corrupt and can't be read.
Understood, but given that I am reading and processing are standard ASCII
text files,
there is no good reason (which I can think of) that the files would be
*unreadable*
*Any* file can be unreadable. The disk may develop a fault, and no longer be
able to read the file's data blocks. Or the file system may be corrupted and
the operating system can see that the file is there, but not where it is. If
the file is on a network share, the network may have gone down halfway through
reading the file. If it's on a USB stick or external hard drive, somebody
might have unplugged it, or the connector might be wobbly.
Normally, for a script like this, failure to read a file should be considered
a fatal error. If a file which *should* be there is no longer there, you
should report the problem and halt. I recommend that you don't catch the
exception at all, just let the traceback occur as normal.
I verified that I had read/write permissions for all my files, which are
the default access privileges anyway (for the owner).
Fine, but I'm talking in general rather than specific for you. In general,
file not found is not the only error you can get. There is a remarkably
large number of things that can go wrong when reading files, fortunately most
of them are very rare.
Consider what your code does. First, you ask the operating system for a list
of the files in a directory, using os.listdir. Then you expect that some of
those files might be missing, and try to catch the exception. Is this
reasonable? Do you actually expect the operating system will lie to you and
say that files are there that actually don't exist?
For a robust application that runs in an environment where it is
