finding masking boundary indices
Hi, I have a masked array like in the attached link, I wanted to find indices of the bounds where the mask is false ie in this case of depth file where there is depth less than shore. Is there a pythonic way of finding the boundary indices? please advice? https://drive.google.com/file/d/0B3heUQNme7G5d2dYZzgxTG1NdG8/edit?usp=sharing -- https://mail.python.org/mailman/listinfo/python-list
Re: writing fortran equivalent binary file using python
Thank you,
But it wont allow to write it in unformatted way so
that the fortran code can read
with
open(11,file=input.bin)
read(11) IWI,JWI,XFIN,YFIN,DXIN,DYIN,NREC,WDAY
with best regards,
sudheer
On Thu, Nov 14, 2013 at 7:48 PM, Oscar Benjamin
oscar.j.benja...@gmail.comwrote:
On 14 November 2013 00:53, Sudheer Joseph sjo.in...@gmail.com wrote:
My trial code with Python (data is read from file here)
from netCDF4 import Dataset as nc
import numpy as np
XFIN=0.0,YFIN=-90.0,NREC=1461,DXIN=0.5;DYIN=0.5
TITLE=NCMRWF 6HOURLY FORCING MKS
nf=nc('ncmrwf_uv.nc')
ncv=nf.variables.keys()
IWI=len(nf.variables[ncv[0]])
JWI=len(nf.variables[ncv[1]])
WDAY=nf.varlables[ncv[2]][0:NREC]
U=nf.variables[ncv[3]][0:NREC,:,:]
V=nf.variables[ncv[4]][0:NREC,:,:]
bf=open('ncmrwf_uv.bin',wb)
f.write(TITLE)
f.write(IWI,JWI,XFIN,YFIN,DXIN,DYIN,NREC,WDAY)
for i in np.arange(0,NREC):
f.write(U[i,:,:],V[i,:,:])
f.close()
But the issue is that f.write do not allow multiple values( it allows
one by one so throws an error with above code ) on same write statement
like in the fortran code. experts may please advice if there a solution for
this?
Can you just call write twice? e.g.:
f.write(U[i,:,:])
f.write(V[i,:,:])
Oscar
--
with best regards
Sudheer
**
Dr. Sudheer Joseph
Scientist
INDIAN NATIONAL CENTRE FOR OCEAN INFORMATION SERVICES (INCOIS)
MINISTRY OF EARTH SCIENCES, GOVERNMENT OF INDIA
OCEAN VALLEY PRAGATHI NAGAR (BO)
OPP.JNTU, NIZAMPET SO
Andhra Pradesh, India. PIN- 500 090.
TEl:+91-40-23044600(R),Tel:+91-9440832534(Mobile)
Tel:+91-40-23886047(O),Fax:+91-40-23892910(O)
E-mail: sudheer.jos...@yahoo.com; s...@incois.gov.in.
Web- http://oppamthadathil.tripod.com
--* ---
The ultimate measure of a man is
not where he stands in moments of
comfort and convenience, but where
he stands at times of challenge and
controversy.
Martin Luther King, Jr.
What we have done for ourselves alone dies with us.
What we have done for others and the world remains and is immortal.
- Albert Pines
--
https://mail.python.org/mailman/listinfo/python-list
writing fortran equivalent binary file using python
Hi,
I need to write a binary file exactly as written by fortran code below
to be read by another code which is part of a model which is not advisable to
edit.I would like to use python for this purpose as python has mode flexibility
and easy coding methods.
character(40) :: TITLE=122322242
integer :: IWI,JWI
real :: XFIN,YFIN,DXIN=0.5,DYIN=0.5,WDAY(6000)
XFIN=0.0,YFIN=-90.0,NREC=1461,DXIN=0.5;DYIN=0.5;IWI=720;JWI=361
real,allocatable,dimension(:,:,:) :: VAR1_VAL
real,allocatable,dimension(:,:,:) :: VAR2_VAL
open(11,file=outf,form='UNFORMATTED')
WRITE(11) TITLE
WRITE(11) NX,NY,XFIN,YFIN,DXIN,DYIN,NREC,WDAY
write(*,'(A10,2f10.3)') START=,VAR1_VAL(1,1,1),VAR2_VAL(1,1,1)
write(*,'(A10,2f10.3)') END=,VAR1_VAL(nx,ny,nrec),VAR2_VAL(nx,ny,nrec)
do i=1,NREC
WRITE(11) VAR1_VAL(:,:,i),VAR2_VAL(:,:,i)
WRITE(*,'(2I10,f10.3)') NX,NY,WDAY(i)
enddo
My trial code with Python (data is read from file here)
from netCDF4 import Dataset as nc
import numpy as np
XFIN=0.0,YFIN=-90.0,NREC=1461,DXIN=0.5;DYIN=0.5
TITLE=NCMRWF 6HOURLY FORCING MKS
nf=nc('ncmrwf_uv.nc')
ncv=nf.variables.keys()
IWI=len(nf.variables[ncv[0]])
JWI=len(nf.variables[ncv[1]])
WDAY=nf.varlables[ncv[2]][0:NREC]
U=nf.variables[ncv[3]][0:NREC,:,:]
V=nf.variables[ncv[4]][0:NREC,:,:]
bf=open('ncmrwf_uv.bin',wb)
f.write(TITLE)
f.write(IWI,JWI,XFIN,YFIN,DXIN,DYIN,NREC,WDAY)
for i in np.arange(0,NREC):
f.write(U[i,:,:],V[i,:,:])
f.close()
But the issue is that f.write do not allow multiple values( it allows one by
one so throws an error with above code ) on same write statement like in the
fortran code. experts may please advice if there a solution for this?
with best regards,
Sudheer
--
https://mail.python.org/mailman/listinfo/python-list
Re: refresing the edited python function
Thank you, But I wish if there was a foolproof reload with best regards, Sudheer - Original Message - From: Jean-Michel Pichavant jeanmic...@sequans.com To: Sudheer Joseph sudheer.jos...@yahoo.com Cc: python-list@python.org Sent: Tuesday, 20 August 2013 10:07 PM Subject: Re: refresing the edited python function - Original Message - Hi, I have been using ipython and ipython with qtconsole and working on a code with functions. Each time I make a modification in function I have to quit IPTHON console (in both with and with out qt console ) and reload the function freshly. If I need to see the changed I made in the function. I tried below options del function name import the module again by issuing from xxx.py import yy import xxx.py make changes reload(xxx.py) this works only if the the function in the code has same name as the code. But even this do not reflect the changes made by editing the code. So what is the standard way to update the function for further tests after an edit? with best regards, Sudheer Hi, My standard way ;) : 1/ create a file 2/ edit the code 3/ run ipython (with %pdb on) 4/ within ipython run myfile.py 5/ check / introspect /debug 6/ change the code 7/ exit ipython 8/ reenter ipython 9/ using the ipython shell history, reexecute the file (2 key press) and go back to 5/ I used to reload my objects, it's been useful until one time when I lost a lot of time because of some nasty side effect. In the end it's not worth it. Always quit the shell, always. JM -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you. -- http://mail.python.org/mailman/listinfo/python-list
Re: refresing the edited python function
Thank you Dieter, I never thought it will be so difficult task, All I was thinking was that, I just do not know how it is done. I wonder how the code developers work in this case every time a function is modified one has to restart the console is a nightmare... Hope one day some solution will be evolved. with best regards, Sudheer From: dieter die...@handshake.de To: python-list@python.org Sent: Monday, 19 August 2013 11:48 AM Subject: Re: refresing the edited python function Sudheer Joseph sudheer.jos...@yahoo.com writes: I have been using ipython and ipython with qtconsole and working on a code with functions. Each time I make a modification in function I have to quit IPTHON console (in both with and with out qt console ) and reload the function freshly. If I need to see the changed I made in the function. I tried below options del function name import the module again by issuing from xxx.py import yy import xxx.py make changes reload(xxx.py) this works only if the the function in the code has same name as the code. But even this do not reflect the changes made by editing the code. So what is the standard way to update the function for further tests after an edit? Getting changes into a running application is difficult. Python has not been designed to make this easy. The reload above is one partial way to achieve something like this. The reload causes the module to be reloaded. If you have changed the modules code, these changes will be reflected *inside* the reloaded module. However, other modules may have imported objects from this module (as in your from xxx.py import yy). To see changes in those objects, they, too, must repeat the import (otherwise, they continue to use the old, unchanged object). There is an additional approach, used e.g. by plone.reload. In this approach, the objects are modified in place. All usage points of the modified object will see changes. However, there are (quite severe) limitations to what changes can be made in place. Thus, this, too, does not give a complete solution. In simple cases, one of those approaches can avoid a restart after modifications. However, in general, a restart is required. -- http://mail.python.org/mailman/listinfo/python-list -- http://mail.python.org/mailman/listinfo/python-list
Re: refresing the edited python function
- Original Message - From: Dave Angel da...@davea.name To: python-list@python.org Cc: Sent: Monday, 19 August 2013 4:45 PM Subject: Re: refresing the edited python function Sudheer Joseph wrote: Thank you Dieter, I never thought it will be so difficult task, All I was thinking was that, I just do not know how it is done. I wonder how the code developers work in this case every time a function is modified one has to restart the console is a nightmare... Hope one day some solution will be evolved. with best regards, Sudheer Please don't top-post, and please use text messages, rather than html mail, when posting on this list. Seems to me your problem is with ipython's IDE, not with Python. Python requires you to rerun your application when making most changes to code. But it doesn't say anything about restarting a console, whatever that is in this context. I use Komodo IDE when i want an IDE functionality, and never restart Komodo, over hours of work. Thank you Dave, I will make sure that when I post next time. with best regards, Sudheer -- http://mail.python.org/mailman/listinfo/python-list
refresing the edited python function
Hi, I have been using ipython and ipython with qtconsole and working on a code with functions. Each time I make a modification in function I have to quit IPTHON console (in both with and with out qt console ) and reload the function freshly. If I need to see the changed I made in the function. I tried below options del function name import the module again by issuing from xxx.py import yy import xxx.py make changes reload(xxx.py) this works only if the the function in the code has same name as the code. But even this do not reflect the changes made by editing the code. So what is the standard way to update the function for further tests after an edit? with best regards, Sudheer *** Sudheer Joseph Indian National Centre for Ocean Information Services Ministry of Earth Sciences, Govt. of India POST BOX NO: 21, IDA Jeedeemetla P.O. Via Pragathi Nagar,Kukatpally, Hyderabad; Pin:5000 55 Tel:+91-40-23886047(O),Fax:+91-40-23895011(O), Tel:+91-40-23044600(R),Tel:+91-40-9440832534(Mobile) E-mail:sjo.in...@gmail.com;sudheer.jos...@yahoo.com Web- http://oppamthadathil.tripod.com ***-- http://mail.python.org/mailman/listinfo/python-list
python netcdf
Dear Members, Is there a way to get the time:origin attribute from a netcdf file as string using the Python netcdf? with best regards, Sudheer -- http://mail.python.org/mailman/listinfo/python-list
Re: python netcdf
Thank you very much Jason
With best regards
Sudheer
On Thursday, June 6, 2013, Jason Swails wrote:
On Wed, Jun 5, 2013 at 9:07 PM, Sudheer Joseph
sjo.in...@gmail.comjavascript:_e({}, 'cvml', 'sjo.in...@gmail.com');
wrote:
Dear Members,
Is there a way to get the time:origin attribute from a
netcdf file as string using the Python netcdf?
Attributes of the NetCDF file and attributes of each of the variables can
be accessed via the dot-operator, as per standard Python.
For instance, suppose that your NetCDF file has a Conventions attribute,
you can access it via:
ncfile.Conventions
Suppose that your variable, time, has an attribute origin, you can get
it via:
ncfile.variables['time'].origin
Of course there's the question of what NetCDF bindings you're going to
use. The options that I'm familiar with are the ScientificPython's
NetCDFFile class (Scientific.IO.NetCDF.NetCDFFile), pynetcdf (which is just
the ScientificPython's class in a standalone format), and the netCDF4
package. Each option has a similar API with attributes accessed the same
way.
An example with netCDF4 (which is newer, has NetCDF 4 capabilities, and
appears to be more supported):
from netCDF4 import Dataset
ncfile = Dataset('my_netcdf_file.nc', 'r')
origin = ncfile.variables['time'].origin
etc. etc.
The variables and dimensions of a NetCDF file are stored in dictionaries,
and the data from variables are accessible via slicing:
time_data = ncfile.variables['time'][:]
The slice returns a numpy ndarray.
HTH,
Jason
--
Sent from my iPad Mini
--
http://mail.python.org/mailman/listinfo/python-list
Re: netcdF4 variables
Thank you very much it works for me. with best regards, Sudheer On Saturday, June 1, 2013 12:51:01 PM UTC+5:30, Andreas Perstinger wrote: On 01.06.2013 05:30, Sudheer Joseph wrote: some hing like a list xx=nc,variables[:] should get me all variable names with out other surrounding stuff?? In [4]: ncf.variables Out[4]: OrderedDict([(u'LON', netCDF4.Variable object at 0x254aad0), [SNIP] It looks like variables is an OrderedDict. Thus ncf.variables.keys() should return a view (or list, depending on your python version) of all keys, i.e. all variable names. Bye, Andreas -- http://mail.python.org/mailman/listinfo/python-list
netcdF4 variables
Dear members, I have been using python NetcdF for some time. I understand that we can get variables from a netcdf one by one by using temp=ncf.variable['temp'][:] but is there a way to get a list of variables with out the rest of the stuff as seen below? some hing like a list xx=nc,variables[:] should get me all variable names with out other surrounding stuff?? with best regards. Sudheer In [4]: ncf.variables Out[4]: OrderedDict([(u'LON', netCDF4.Variable object at 0x254aad0), (u'LAT', netCDF4.Variable object at 0x254ab50), (u'DEPTH1_1', netCDF4.Variable object at 0x254abd0), (u'TAX', netCDF4.Variable object at 0x254ac50), (u'DIF_FD1', netCDF4.Variable object at 0x254acd0), (u'DIF_FD2', netCDF4.Variable object at 0x254ad50), (u'DIF_FD3', netCDF4.Variable object at 0x254add0), (u'DIF_FD4', netCDF4.Variable object at 0x254ae50), (u'DIF_FD5', netCDF4.Variable object at 0x254aed0), (u'DEPTH', netCDF4.Variable object at 0x254af50), (u'DEPTH_bnds', netCDF4.Variable object at 0x261b050), (u'TIME', netCDF4.Variable object at 0x261b0d0), (u'TEMP_BIAS', netCDF4.Variable object at 0x261b150)]) -- http://mail.python.org/mailman/listinfo/python-list
formatted output
Dear members, I need to print few arrays in a tabular form for example below array IL has 25 elements, is there an easy way to print this as 5x5 comma separated table? in python IL=[] for i in np.arange(1,bno+1): IL.append(i) print(IL) % in fortran I could do it as below % integer matrix(5,5) in=0 do, k=1,5 do, l=1,5 in=in+1 matrix(k,l)=in enddo enddo m=5 n=5 do, i=1,m write(*,(5i5)) ( matrix(i,j), j=1,n ) enddo end -- http://mail.python.org/mailman/listinfo/python-list
memory management
HI,
I have been trying to compute cross correlation between a time series
at a location f(1) and the timeseries of spatial data f(XYT) and saving the
resulting correlation coefficients and lags in a 3 dimensional array which is
of fairly big size. Though the code I made for this purpose works up to few
iterations then it hangs due to apparent memory crunch. Can anybody suggest a
better way to handle this situation so that the computation and data storing
can be done with out hangups. Finally I intend to save the data as netcdf file
which is not implemented as of now. Below is the piece of code I wrote for this
purpose.
from mpl_toolkits.basemap import Basemap as bm, shiftgrid, cm
import numpy as np
import matplotlib.pyplot as plt
from netCDF4 import Dataset
from math import pow, sqrt
import sys
from scipy.stats import t
indep=120
nlags=365
ncin = Dataset('qu_ru.nc', 'r')
lons = ncin.variables['LON421_600'][:]
lats = ncin.variables['LAT81_220'][:]
dep = ncin.variables['DEPTH1_29'][:]
adep=(dep==indep).nonzero()
didx=int(adep[0])
qu = ncin.variables['qu'][:,:,:]
#qv = ncin.variables['QV'][0,:,:]
ru = ncin.variables['ru'][:,didx,0,0]
ncin.close()
fig = plt.figure()
ax = fig.add_axes([0.1,0.1,0.8,0.8])
# use major and minor sphere radii from WGS84 ellipsoid.
m = bm(projection='cyl', llcrnrlon=30, llcrnrlat=-40,urcrnrlon=120,
urcrnrlat=30)
# transform to nx x ny regularly spaced 5km native projection grid
nx = int((m.xmax-m.xmin))+1; ny = int((m.ymax-m.ymin)+1)
q=ru[1:2190]
qmean=np.mean(q)
qstd=np.std(q)
qnorm=(q-qmean)/qstd
lags3d=np.arange(731*140*180).reshape(731,140,180)
r3d=np.arange(731*140*180).reshape(731,140,180)
for i in np.arange(len(lons)):
for j in np.arange(len(lats)):
print i,j
p=qu[1:2190,j,i].squeeze()
p.shape
pmean=np.mean(p)
pstd=np.std(p)
pnorm=(p-pmean)/pstd
n=len(p)
# fg=plt.figure()
c=plt.xcorr(p,q,usevlines=True,maxlags=nlags,normed=True,lw=2)
acp=plt.acorr(p,usevlines=True,maxlags=nlags,normed=True,lw=2)
acq=plt.acorr(q,usevlines=True,maxlags=nlags,normed=True,lw=2)
acp[1][nlags]=0
acq[1][nlags]=0
lags=c[0]
r=c[1]
lags3d[:,j,i]=lags
r3d[:,j,i]=r
--
http://mail.python.org/mailman/listinfo/python-list
Re: memory management
Python version and OS please. And is the Python 32bit or 64bit? How much RAM does the computer have, and how big are the swapfiles ? Python 2.7.3 ubuntu 12.04 64 bit 4GB RAM Fairly big is fairly vague. To some people, a list with 100k members is huge, but not to a modern computer. I have a data loaded to memory from netcdf file which is 2091*140*180 grid points (2091 time, 140 latitude 180 longitude) apart from this I define a 2 3d arrays r3d and lags3d to store the output for writing out to netcdf file after completion. How have you checked whether it's running out of memory? Have you run 'top' on it? Or is that just a guess? I have not done this but the speed (assessed from the listing of grid i and j) get stopped after j=6 ie after running 6 longitude grids) Will check the top as you suggested Here is the result of top it used about 3gB memory PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 3069 sjo 20 0 3636m 3.0g 2504 D3 78.7 3:07.44 python I haven't used numpy, scipy, nor matplotlib, and it's been a long time since I did correlations. But are you sure you're not just implementing an O(n**3) algorithm or something, and it's just extremely slow? Correlation do not involve such computation normally, I am not sure if internally python does some thing like that. with best regards, Sudheer from mpl_toolkits.basemap import Basemap as bm, shiftgrid, cm import numpy as np import matplotlib.pyplot as plt from netCDF4 import Dataset from math import pow, sqrt import sys from scipy.stats import t snip -- DaveA -- http://mail.python.org/mailman/listinfo/python-list
