Re: [Tutor] Reading/dealing/matching with truly huge (ascii) files
Elaina Ann Hyde wrote: Thanks for all the helpful hints, I really like the idea of using distances instead of a limit. Walter was right that the 'i !=j' condition was causing problems. I think that Alan and Steven's use of the index separately was great as it makes this much easier to test (and yes 'astropysics' is a valid package, it's in there for later when I convert astrophysical coordinates and whatnot, pretty great but a little buggy FYI). So I thought, hey, why not try to do a little of all these ideas, and, if you'll forgive the change in syntax, I think the problem is that the file might really just be too big to handle, and I'm not sure I have the right idea with the best_match: The errors are as follows: dat2=asciitable.read(y,Reader=asciitable.NoHeader,data_start=4,fill_values=['nan','-9.999']) File /Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site- packages/asciitable-0.8.0-py2.7.egg/asciitable/ui.py, line 131, in read dat = _guess(table, new_kwargs) File /Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site- packages/asciitable-0.8.0-py2.7.egg/asciitable/ui.py, line 175, in _guess dat = reader.read(table) File /Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site- packages/asciitable-0.8.0-py2.7.egg/asciitable/core.py, line 841, in read self.lines = self.inputter.get_lines(table) File /Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/site- packages/asciitable-0.8.0-py2.7.egg/asciitable/core.py, line 158, in get_lines lines = table.splitlines() MemoryError -- So this means I don't have enough memory to run through the large file? Even if I just read in with asciitable I get this problem, I looked again and the large file is 1.5GB of text lines, so very large. I was thinking of trying to tell the read function to skip lines that are too far away, the file is much, much bigger than the area I need. Thanks for the comments so far. ~Elaina Hmm, 1.5GB would be about 30,000 bytes per line if the 50,000 lines you mentioned before are correct. What does $ wc bigfile say? Can you give the first few lines of bigfile here or on pastebin.com? I don't have asciitables installed but a quick look into the code suggests it consumes a lot more memory than necessary to solve your problem. If the file format is simple a viable alternative may be to extract the interesting columns manually together with the line index. Once you have the best matches you can build the result from bigfile and the indices of the best matches. Alternatively you can split bigfile into a few parts, calculate the best matches for each part, and finally calculate the best matches of the partial best matches combined. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] help writing functions
On 23/02/12 00:59, Saad Javed wrote: [CODE]feed = urllib.urlopen(rssPage) #rssPage: address of xml feed tree = etree.parse(feed) x = tree.xpath(/rss/channel/item/title/text()) x = str(x[0]) for tag in tags: #tags is a list of items like hdtv, xvid, 720p etc x = re.sub(r'\b' + tag + r'\b', '', x) z = re.sub(r'[^\w\s]', '', x) y = tree1.xpath(/rss/channel/item/pubDate/text()) print %s - %s %(z.rstrip(), y[0][:16])[/CODE] Please don;t insert wiki style markers, its just confusing. Also please use plain text for email otherwise the formatting tends to get lost. [CODE]def get_value(feed): try: url = urllib2.urlopen(feed) tree = etree.parse(url) x = tree.xpath(/rss/channel/item/title/text()) y = tree.xpath(/rss/channel/item/pubDate/text()) x = str(x[0]) y = str(y[0][:16]) return x return y This will always return x and never y because a return statement terminates the function. If you want to return both values you need to put them in the same return statement: return x,y If you want to return only one you need to select which with an if/else. return x if some condition else y or just if some condition return x else return y If you prefer. except SyntaxError: You should probably not try catching Syntax errors since thats usually a fault in your code! Instead fix the syntax error. def del_tag(x): tags = ['HDTV', 'LOL', 'VTV', 'x264', 'DIMENSION', 'XviD', '720P', 'IMMERSE', '720p', 'X264'] for tag in tags: x = re.sub(r'\b' + tag + r'\b', '', x) y = re.sub(r'[^\w\s]', '', x) You don't return x or y so they get thrown away at the end of the function. Which makes the whole thing a waste of space... def main(): a = get_value(rssPage) b = del_tag(a) print b if __name__ == '__main__': main()[/CODE] Running this code returns [B]None[/B]. None is the default return value if you do not provide one. main has no return statement. Neither does del_tag() Both main and del_tag will therefore return None. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which computer operating system is best for Python developers?
On 23/02/12 01:00, Tamar Osher wrote: Hi. I am still having trouble installing and using Python on my (new) Windows 7 computer, but I plan to contact Microsoft forums and see if they can help me, since this is a Windows problem and not a Python problem. I doubt if you have any big issues. You probably only need to set two environment variables that the latest Pythion installers do not set for you. (I'm not sure why!) My question: For the future, what type of computer is best for Python developers? Do all Python developers use some type of Unix computer? By no means, one of Pythons strengths is that the same code can run on many OS. But as Steven has mentioned many developers use Linux because GNU/Linux is designed as a developer's OS and comes with oodles of tools. Most of those are available for Windows too but you have to go find them, download them and install them. I used Python on Windows for 11 years. I only switched to Linux full time a year ago when Windows 7 became too expensive and I decided I wasn't paying that much for an OS when a free alternative existed! But I still have Python on my Win7 work's laptop. And I also have it on my 10 year old MacOS iBook. And the same python code runs on all of them... So you don't need to switch OS, just tweak a couple of settings. Go through the procedure I outlined in my previous mail and tell us how you get on. It may be a reinstall of Python is needed, or it may just be the two settings I mentioned above (PATH and PYTHONPATH) One thing: If you do a reinstall, download the ActiveState version rather than the Python.org version. Active state tweak their Windows version of Python to include a bunch of extra goodies for Windows programmers. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Reading/dealing/matching with truly huge (ascii) files
On 23/02/12 01:55, Elaina Ann Hyde wrote: ns/7.2/lib/python2.7/site-packages/asciitable-0.8.0-py2.7.egg/asciitable/core.py, line 158, in get_lines lines = table.splitlines() MemoryError -- So this means I don't have enough memory to run through the large file? Probably, or the code you are using is doing something extremely inefficient. Even if I just read in with asciitable I get this problem, I looked again and the large file is 1.5GB of text lines, so very large. How much RAM do you have? Probably only 1-2G? so I'd suggest trying another approach. Peter has suggested a couple of ideas. The other way is to simply load both files into database tables and use a SQL SELECT to pull out the combined lines. This will probably be faster than trying to do line by line stitch ups in Python. You can also use the SQL interactive prompt to experiment with the query till you are sure its right! Do you know any SQL? If not it is very easy to learn. (See the database topic in my tutorial(v2 only) ) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] Solve wave equation
Hi, I am trying to write some code that will solve the 2D wave equation by the finite difference method, but it just returns an array full of zeros and NaN's. Not sure where I am going wrong, the code is attached so if could someone point me in the right direction I'd appreciate this. Thanks D # 2D Finite Distance Wave Equation. from pylab import * from numpy import math # Set up variables. nx = 100 nz = 100 nsteps = 300 c = 4000 dt = 1**-4 h = 1 t = arange(0,nsteps,dt) # Define source as a spike. s = zeros(nsteps) s[1] = 1 s[2] = 2 s[3] = 1 # Position source. xs = 50 zs = 50 ##plot(t,s) ##show() # Set up pressure field. p=empty([nx,nz,nsteps]) for t in range(0,nsteps-1): for z in range(0,nz-1): for x in range(0,nx-1): p[x,z,t] = 0 # Solve wave equation. for t in range(2,nsteps-1): for z in range(1,nz-1): for x in range(2,nx-1): p[xs,zs,t] = s[t] k = (c*dt/h)**2 p[x,z,t] = 2*p[x,z,t-1] - p[x,z,t-2] + k*(p[x+1,z,t-1]-4*p[x,z,t-1]+p[x-1,z,t-1]+p[x,z+1,t-1]+p[x,z-1,t-1]) ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Solve wave equation
David Craig wrote: Hi, I am trying to write some code that will solve the 2D wave equation by the finite difference method, but it just returns an array full of zeros and NaN's. Not sure where I am going wrong, the code is attached so if could someone point me in the right direction I'd appreciate this. I'm not enough of a mathematician to tell what the solution of the 2D wave equation by the finite difference method should be. Are you sure that an array full of zeroes and NANs is not the right answer? Perhaps this is telling you that there is no solution? A couple of other random stumbles in the dark: dt = 1**-4 Do you realise that 1**-4 == 1? # Set up pressure field. p=empty([nx,nz,nsteps]) This sets p to an array with random values (whatever rubbish just happens to be in memory at the time). You are expected to initialize the values yourself. Since you go on to try to initialize to all zeroes, you should consider using numpy's zeroes() function instead of empty(). for t in range(0,nsteps-1): for z in range(0,nz-1): for x in range(0,nx-1): p[x,z,t] = 0 I'm pretty sure you end the loops one iteration too early here. Your array is indexed by x = 0...nx-1 inclusive z = 0...nz-1 inclusive t = 0...nsteps-1 inclusive (that is, the half-open intervals [0, nx) etc.) but you initialize only the values x = 0...nx-2 inclusive, etc. This is because Python's range() is already half-open on the right: py range(0, 10) [0, 1, 2, 3, 4, 5, 6, 7, 8, 9] By subtracting one, you miss an item: py range(0, 10-1) [0, 1, 2, 3, 4, 5, 6, 7, 8] -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which computer operating system is best for Python developers?
On 23/02/2012 09:00, Alan Gauld wrote: By no means, one of Pythons strengths is that the same code can run on many OS. But as Steven has mentioned many developers use Linux because GNU/Linux is designed as a developer's OS and comes with oodles of tools. Most of those are available for Windows too but you have to go find them, download them and install them. One thing: If you do a reinstall, download the ActiveState version rather than the Python.org version. Active state tweak their Windows version of Python to include a bunch of extra goodies for Windows programmers. Just seconding both of Alan's points here. I have been fruitfully using Python on Windows for more than 12 years now and I am one of the very few core developers who works in Windows (although sadly lacking the time at the moment to contribute much). I develop Python-based websites which run unaltered on my Win7 laptop, my WinXP desktop, and whatever flavour of Linux my hosting provider is using. (It could be RedHat or CentOS but I don't care because it just works). You need to do a very small bit of initial assumption-bashing to ensure that things will work across platforms, but once that's done you never have to change anything again. I also recommend the ActiveState distro. It sets Python up on the PATH and adds pip in the right places. Both of those are easy enough to do for yourself, but it's nice to have it done for you. TJG ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which computer operating system is best for Python developers?
On 23/02/2012 02:00, Tamar Osher wrote: Hi. I am still having trouble installing and using Python on my (new) Windows 7 computer, but I plan to contact Microsoft forums and see if they can help me, since this is a Windows problem and not a Python problem. My question: For the future, what type of computer is best for Python developers? Do all Python developers use some type of Unix computer? I appreciate your feedback and look forward to hearing from you. Thanks for your time. That's actually a good question. I am in Instrumentation Engineering (Electronics), and a lot of the software is designed to run only in Windows, or, the equivalent software in Linux is more for hobbyists than for professionnals (Think software for printed circuit boards), or, in the best case there are versions that run on Linux, but they aren't as featured or updated as the one that run on Windows. So I work on Windows. I have both Python26 and Python32 installed (Python26 for things like Numpy/SciPy). I also installed VirtualBox and run Ubuntu 11.10. It's nice because most of the software I use is on Windows, but I fire up the virtual machine sometimes for programming stuff. So it depends what you do and the software you use. One last point: Having two versions of Python, here's what I did in order to chose which version is used depending what I'm doing (If I'm tinkering with Numpy, I must use Python26) Python 2.6 is installed in C:\Python26 Python 3.2 is installed in C:\Python32 I added both paths to the Windows Environment Variables. I created two .bat files named Python26.bat and Python32.bat, each one in the respective directory. The Python26.bat file contains the follwoing lines: @echo off C:\Python26\python.exe %1 And the Python32.bat file contains the follwoing lines: @echo off C:\Python32\python.exe %1 So, if I write python26 test.py in the command line, the script gets executed by Python 2.6. In the same way, if I write python32 test.py, it gets executed by Python 3.2.. Just a little thing to make my life easier. PS: If you have any question regarding the setting of the Virtual Box to run Ubuntu as a guest on Windows, feel free to ask for details. I'll be glad to provide links and things like that. -- ~Jugurtha Hadjar, ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Solve wave equation
Hooray it works, thanks On 02/23/2012 01:39 PM, Ken Oliver wrote: Do you really want dt = 1**-4 or would 10**-4 be better -Original Message- From: David Craigdcdavem...@gmail.com Sent: Feb 23, 2012 7:57 AM To: tutor@python.org Subject: [Tutor] Solve wave equation Hi, I am trying to write some code that will solve the 2D wave equation by the finite difference method, but it just returns an array full of zeros and NaN's. Not sure where I am going wrong, the code is attached so if could someone point me in the right direction I'd appreciate this. Thanks D . ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] help writing functions
Sorry for the formatting. Added return statements to both functions. Adding return [x, y] to get_value func. That solved the problem. Thank you! :) Saad On Thursday, February 23, 2012, Alan Gauld wrote: On 23/02/12 00:59, Saad Javed wrote: [CODE]feed = urllib.urlopen(rssPage) #rssPage: address of xml feed tree = etree.parse(feed) x = tree.xpath(/rss/channel/item/**title/text()) x = str(x[0]) for tag in tags: #tags is a list of items like hdtv, xvid, 720p etc x = re.sub(r'\b' + tag + r'\b', '', x) z = re.sub(r'[^\w\s]', '', x) y = tree1.xpath(/rss/channel/**item/pubDate/text()) print %s - %s %(z.rstrip(), y[0][:16])[/CODE] Please don;t insert wiki style markers, its just confusing. Also please use plain text for email otherwise the formatting tends to get lost. [CODE]def get_value(feed): try: url = urllib2.urlopen(feed) tree = etree.parse(url) x = tree.xpath(/rss/channel/item/**title/text()) y = tree.xpath(/rss/channel/item/**pubDate/text()) x = str(x[0]) y = str(y[0][:16]) return x return y This will always return x and never y because a return statement terminates the function. If you want to return both values you need to put them in the same return statement: return x,y If you want to return only one you need to select which with an if/else. return x if some condition else y or just if some condition return x else return y If you prefer. except SyntaxError: You should probably not try catching Syntax errors since thats usually a fault in your code! Instead fix the syntax error. def del_tag(x): tags = ['HDTV', 'LOL', 'VTV', 'x264', 'DIMENSION', 'XviD', '720P', 'IMMERSE', '720p', 'X264'] for tag in tags: x = re.sub(r'\b' + tag + r'\b', '', x) y = re.sub(r'[^\w\s]', '', x) You don't return x or y so they get thrown away at the end of the function. Which makes the whole thing a waste of space... def main(): a = get_value(rssPage) b = del_tag(a) print b if __name__ == '__main__': main()[/CODE] Running this code returns [B]None[/B]. None is the default return value if you do not provide one. main has no return statement. Neither does del_tag() Both main and del_tag will therefore return None. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ __**_ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/**mailman/listinfo/tutorhttp://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which computer operating system is best for Python developers?
On 02/23/2012 10:23 AM, Jugurtha Hadjar wrote: SNIP One last point: Having two versions of Python, here's what I did in order to chose which version is used depending what I'm doing (If I'm tinkering with Numpy, I must use Python26) Python 2.6 is installed in C:\Python26 Python 3.2 is installed in C:\Python32 I added both paths to the Windows Environment Variables. I created two .bat files named Python26.bat and Python32.bat, each one in the respective directory. The Python26.bat file contains the follwoing lines: @echo off C:\Python26\python.exe %1 And the Python32.bat file contains the follwoing lines: @echo off C:\Python32\python.exe %1 SNIP I'm not running Windows any more (except in a VirtualBox), but your batch files could be improved: Since you only have the one useful line in the batch file, just put the @ on that line; no need to turn off echo for the whole file, when the file is one line. And i forget whether it's %* or %$but there is a substitution string you can use to mean all the arguments, rather than just taking one. Remember that sometimes people want to run python with multiple arguments. I also made a bat directory, and added it to the PATH. Then you put both those batch files, plus some others, into the bat directory. That assumes you'll keep finding more utilities you'd like to add to the path. No biggie, just trying to help. There are more complicated changes that I used to do, which I won't mention here. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which computer operating system is best for Python developers?
On 23/02/2012 16:46, Dave Angel wrote: I'm not running Windows any more (except in a VirtualBox), but your batch files could be improved: Since you only have the one useful line in the batch file, just put the @ on that line; no need to turn off echo for the whole file, when the file is one line. And i forget whether it's %* or %$but there is a substitution string you can use to mean all the arguments, rather than just taking one. Remember that sometimes people want to run python with multiple arguments. I also made a bat directory, and added it to the PATH. Then you put both those batch files, plus some others, into the bat directory. That assumes you'll keep finding more utilities you'd like to add to the path. No biggie, just trying to help. There are more complicated changes that I used to do, which I won't mention here. -- DaveA Hey, these are very neat ideas (especially the bat folder and the arguments point) .. Thank you ! -- ~Jugurtha Hadjar, ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Which computer operating system is best for Python developers?
On 23/02/2012 15:23, Jugurtha Hadjar wrote: On 23/02/2012 02:00, Tamar Osher wrote: Hi. I am still having trouble installing and using Python on my (new) Windows 7 computer, but I plan to contact Microsoft forums and see if they can help me, since this is a Windows problem and not a Python problem. My question: For the future, what type of computer is best for Python developers? Do all Python developers use some type of Unix computer? I appreciate your feedback and look forward to hearing from you. Thanks for your time. That's actually a good question. I am in Instrumentation Engineering (Electronics), and a lot of the software is designed to run only in Windows, or, the equivalent software in Linux is more for hobbyists than for professionnals (Think software for printed circuit boards), or, in the best case there are versions that run on Linux, but they aren't as featured or updated as the one that run on Windows. So I work on Windows. I have both Python26 and Python32 installed (Python26 for things like Numpy/SciPy). I also installed VirtualBox and run Ubuntu 11.10. It's nice because most of the software I use is on Windows, but I fire up the virtual machine sometimes for programming stuff. So it depends what you do and the software you use. One last point: Having two versions of Python, here's what I did in order to chose which version is used depending what I'm doing (If I'm tinkering with Numpy, I must use Python26) Python 2.6 is installed in C:\Python26 Python 3.2 is installed in C:\Python32 I added both paths to the Windows Environment Variables. I created two .bat files named Python26.bat and Python32.bat, each one in the respective directory. The Python26.bat file contains the follwoing lines: @echo off C:\Python26\python.exe %1 And the Python32.bat file contains the follwoing lines: @echo off C:\Python32\python.exe %1 So, if I write python26 test.py in the command line, the script gets executed by Python 2.6. In the same way, if I write python32 test.py, it gets executed by Python 3.2.. Just a little thing to make my life easier. PS: If you have any question regarding the setting of the Virtual Box to run Ubuntu as a guest on Windows, feel free to ask for details. I'll be glad to provide links and things like that. Have you seen pylauncher? https://bitbucket.org/vinay.sajip/pylauncher Here's the background http://www.python.org/dev/peps/pep-0397/ -- Cheers. Mark Lawrence. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Writing to a file/changing the file name
On 2/23/2012 12:04 AM, Michael Lewis wrote: Hi everyone, I have a program where I open a file (recipe.txt), I read that file and write it to another file. I am doing some multiplying of numbers in between; however, my question is, when I name the file I am writing to, the file extension is changed, but the file name is not. What am I doing wrong? Christian gave you the answer (he fished for you) How to fish for yourself: Before posting a question do a walkthru of the program. This means pretending you are the Python interpreter and your job is to execute the program step-by-step, writing down the effect of each statement You will find many answers yourself this way, saving all of us time and energy. in the above case take the statement new_file_name = file_name + '2' and do this: file_name is recipe.txt file_name + '2' is here you pretend to be the interpreter and answer the question what is recipe.txt + 2 If you come up with anything other than recipe.txt2 then you need to review how Python works. Another idea is: when you get an unexpected result there must be a very good reason for it. Do a little hunting. Each time you solve a problem for yourself you get stronger. HTH -- Bob Gailer 919-636-4239 Chapel Hill NC ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Solve wave equation
OK so I can solve the equation but now I am having trouble plotting the solution! I would like to produce a surface plot with colors defined by p and animate it. That is plot the value of p at all x and z, over time (t). My code to get p is below but I really have no idea how to plot this. Anyone know the best way to go about this? thanks, D Original Message Subject:Re: [Tutor] Solve wave equation Date: Thu, 23 Feb 2012 15:24:45 + From: David Craig dcdavem...@gmail.com To: tutor@python.org Hooray it works, thanks On 02/23/2012 01:39 PM, Ken Oliver wrote: Do you really want dt = 1**-4 or would 10**-4 be better -Original Message- From: David Craigdcdavem...@gmail.com Sent: Feb 23, 2012 7:57 AM To: tutor@python.org Subject: [Tutor] Solve wave equation Hi, I am trying to write some code that will solve the 2D wave equation by the finite difference method, but it just returns an array full of zeros and NaN's. Not sure where I am going wrong, the code is attached so if could someone point me in the right direction I'd appreciate this. Thanks D . # 2D Finite Distance Wave Equation. from pylab import * from numpy import math ion() # Set up variables. nx = 100 nz = 100 nsteps = 300 c = 3500 dt = 10**-4 h = 1 t = arange(0,nsteps,dt) # Define source as a spike. s = zeros(nsteps) s[1] = 1 s[2] = 2 s[3] = 1 # Position source. xs = 50 zs = 50 ##plot(t,s) ##show() # Set up pressure field. p=empty([nx,nz,nsteps]) for t in range(0,nsteps-1): for z in range(0,nz-1): for x in range(0,nx-1): p[x,z,t] = 0 # Solve wave equation for t in range(2,nsteps-1): for z in range(1,nz-1): for x in range(2,nx-1): p[xs,zs,t] = s[t] k = (c*dt/h)**2 p[x,z,t] = 2*p[x,z,t-1] - p[x,z,t-2] + k*(p[x+1,z,t-1]-4*p[x,z,t-1]+p[x-1,z,t-1]+p[x,z+1,t-1]+p[x,z-1,t-1]) #Plot somehow draw() #close() ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Solve wave equation
OK so I can solve the equation but now I am having trouble plotting the solution! I would like to produce a surface plot with colors defined by p and animate it. That is plot the value of p at all x and z, over time (t). My code to get p is below but I really have no idea how to plot this. Anyone know the best way to go about this? A quick search on matplotlib surface plot (since you're already using matplotlib) led me to http://matplotlib.sourceforge.net/mpl_toolkits/mplot3d/tutorial.html#surface-plots Perhaps that's something you can use? Your statement 'colors defined by p' leads more to using contour plots instead, and variants thereof: http://matplotlib.sourceforge.net/api/pyplot_api.html#matplotlib.pyplot.contourf I assume with animate it, you just want to show every plot, continuously updating it. With matplotlib, that may be a bit slow, but you could save each figure to disk and then use some tool to put them into a movie (mpeg, wmv, animated gif, anything) if this is something you want to present to other people. Evert thanks, D Original Message Subject: Re: [Tutor] Solve wave equation Date: Thu, 23 Feb 2012 15:24:45 + From: David Craig dcdavem...@gmail.com To: tutor@python.org Hooray it works, thanks On 02/23/2012 01:39 PM, Ken Oliver wrote: Do you really want dt = 1**-4 or would 10**-4 be better -Original Message- From: David Craig dcdavem...@gmail.com Sent: Feb 23, 2012 7:57 AM To: tutor@python.org Subject: [Tutor] Solve wave equation Hi, I am trying to write some code that will solve the 2D wave equation by the finite difference method, but it just returns an array full of zeros and NaN's. Not sure where I am going wrong, the code is attached so if could someone point me in the right direction I'd appreciate this. Thanks D . # 2D Finite Distance Wave Equation. from pylab import * from numpy import math ion() # Set up variables. nx = 100 nz = 100 nsteps = 300 c = 3500 dt = 10**-4 h = 1 t = arange(0,nsteps,dt) # Define source as a spike. s = zeros(nsteps) s[1] = 1 s[2] = 2 s[3] = 1 # Position source. xs = 50 zs = 50 ##plot(t,s) ##show() # Set up pressure field. p=empty([nx,nz,nsteps]) for t in range(0,nsteps-1): for z in range(0,nz-1): for x in range(0,nx-1): p[x,z,t] = 0 # Solve wave equation for t in range(2,nsteps-1): for z in range(1,nz-1): for x in range(2,nx-1): p[xs,zs,t] = s[t] k = (c*dt/h)**2 p[x,z,t] = 2*p[x,z,t-1] - p[x,z,t-2] + k*(p[x+1,z,t-1]-4*p[x,z,t-1]+p[x-1,z,t-1]+p[x,z+1,t-1]+p[x,z-1,t-1]) #Plot somehow draw() #close() ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] how to uninstall distutils?
Hi, I have distutils 0.9 install under Python 2.7. I want to uninstall it. I am on a Macbook pro running Lion. The site-packages directory is in /Lib/Python/2.7/site-packages. There distutils exists along with a lot of other stuff I have installed. I do not have pip installed and in fact installed distutils 0.9 from the distutils directory by running as usual sudo python2.7 setup.py install. I am unsure whether I can just remove the distutils stuff in the site-packages directory and do so safely. Can you please advise me on the safe way to completely remove distutils? Thanks for your help. Comer ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] how to uninstall distutils?
Wow, I screwed up. I meant docutils rather than distutils !! Sorry for the crazy fingers. On Thu, Feb 23, 2012 at 3:14 PM, Comer Duncan comer.dun...@gmail.com wrote: Hi, I have distutils 0.9 install under Python 2.7. I want to uninstall it. I am on a Macbook pro running Lion. The site-packages directory is in /Lib/Python/2.7/site-packages. There distutils exists along with a lot of other stuff I have installed. I do not have pip installed and in fact installed distutils 0.9 from the distutils directory by running as usual sudo python2.7 setup.py install. I am unsure whether I can just remove the distutils stuff in the site-packages directory and do so safely. Can you please advise me on the safe way to completely remove distutils? Thanks for your help. Comer ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Reading/dealing/matching with truly huge (ascii) files
On Thu, Feb 23, 2012 at 9:07 PM, Alan Gauld alan.ga...@btinternet.comwrote:
On 23/02/12 01:55, Elaina Ann Hyde wrote:
ns/7.2/lib/python2.7/site-**packages/asciitable-0.8.0-py2.**7.egg/asciitable/core.py,
line 158, in get_lines
lines = table.splitlines()
MemoryError
--
So this means I don't have enough memory to run through the large file?
Probably, or the code you are using is doing something extremely
inefficient.
Even if I just read in with asciitable I get this problem, I looked
again and the large file is 1.5GB of text lines, so very large.
How much RAM do you have? Probably only 1-2G? so I'd suggest trying
another approach.
Peter has suggested a couple of ideas.
The other way is to simply load both files into database tables and use a
SQL SELECT to pull out the combined lines. This will probably be faster
than trying to do line by line stitch ups in Python.
You can also use the SQL interactive prompt to experiment with the query
till you are sure its right!
Do you know any SQL? If not it is very easy to learn.
(See the database topic in my tutorial(v2 only) )
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
__**_
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/**mailman/listinfo/tutorhttp://mail.python.org/mailman/listinfo/tutor
Ok, if I use awk I seperate the file into an edible 240MB chunk, I do my
initial sorting there. Now, having learned my lesson from last time, using
numpy is/can be faster than looping for an array, so if I want to find the
minimum distance and get matches. I cobbled these together and now the
matching is reasonably fast and seems to be doing quite well.
#!/usr/bin/python
# import modules used here -- sys is a very standard one
import sys
import asciitable
import matplotlib
import matplotlib.path as mpath
import matplotlib.pyplot as plt
from matplotlib.pyplot import figure, show
from matplotlib.patches import Ellipse
import scipy
import numpy as np
from numpy import *
import math
import pylab
import random
from pylab import *
import astropysics
import astropysics.obstools
import astropysics.coords
from astropysics.coords import ICRSCoordinates,GalacticCoordinates
x=open('Core_rod_name.list')
y=open('2MASS_subsetJKmikesigs18_19_36_27')
dat=asciitable.read(x,Reader=asciitable.CommentedHeader,
fill_values=['','-999.99'])
#first convert from decimal radians to degrees
Radeg=dat['ra-drad']*180./math.pi
Decdeg=dat['dec-drad']*180./math.pi
dat2 = asciitable.read(y,Reader=asciitable.NoHeader,fill_values=['
nan','-99.9'])
fopen = open('allfiles_rod2Mass.list','w')
#here are the 2 values for the large file
#converts hexadecimal in multiple columns to regular degrees
Radeg2 = 15*(dat2['col1']+(dat2['col2']/60.)+(dat2['col3']/(60.*60.)))
Decdeg2 = dat2['col4']-(dat2['col5']/60.)-(dat2['col6']/(60.*60.))
#try defining distances instead of a limit
#built in numpy function faster than a loop, combine numpy and loop
def distance(Ra1,Dec1,Ra2,Dec2):
x = Ra1 - Ra2
y = Dec1 - Dec2
return np.sqrt(x*x+y*y)
fopen=open('matches.list','w')
best_match=[]
for i in xrange(len(Radeg)):
dist2 = np.array(distance(Radeg[i],Decdeg[i],Radeg2,Decdeg2))
best_match = where(dist2==min(dist2))[0][0]
Rab = Radeg2[best_match]
Decb = Decdeg2[best_match]
fopen.write(str(dist2[best_match])++ .join([str(k) for k in
list(dat[i])])+ + .join([str(k) for k in
list(dat2[best_match])])+\n)
fopen.close()
--
Thanks everyone! All your comments were really helpful, I think I might
even be getting the hang of this!
~Elaina
--
PhD Candidate
Department of Physics and Astronomy
Faculty of Science
Macquarie University
North Ryde, NSW 2109, Australia
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Reading/dealing/matching with truly huge (ascii) files
Did you try loadtxt() from numpy? http://mail.scipy.org/pipermail/scipy-user/2010-August/026431.html the poster above notes that 2.5 million lines and 10 columns takes 3 minutes to load. Asokan Pichai ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] Reading/dealing/matching with truly huge (ascii) files
On 24/02/12 05:11, Elaina Ann Hyde wrote: Ok, if I use awk I seperate the file into an edible 240MB chunk, Why awk? Python is nearly always faster than awk... Even nawk or gawk. awk is a great language but I rarely use it nowadays other than for one liners because perl/python/ruby are all generally faster for non trivial tasks. split OTOH should be faster still for chunking a file. But by keeping the chunking as part of your program you don't have the switching time between apps. -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
