Thanks Stephan, It doesn't look like CDAT has 'daily' option - it has yearly, seasonal and monthly! I would need to look into IRIS more as it is new to me and I can't quiet figure out all the steps required for xray, although it looks great.
Another way around was after converting to localtime_day I could append the corresponding hourly arrays to a list, concatenate, calculate max and make the max equal to that localtime_day. Then I could delete everything in that list and repeat by looping though the hours of the next day and append to the empty list. Although I really don't know how to get this to work. On Thu, May 22, 2014 at 10:56 AM, Stephan Hoyer <sho...@gmail.com> wrote: > Hello anonymous, > > I recently wrote a package "xray" (http://xray.readthedocs.org/) > specifically to make it easier to work with high-dimensional labeled data, > as often found in NetCDF files. Xray has a groupby method for grouping over > subsets of your data, which would seem well suited to what you're trying to > do. Something like the following might work: > > ds = xray.open_dataset(ncfile) > tmax = ds['temperature'].groupby('time.hour').max() > > It also might be worth looking at other more data analysis packages, > either more generic (e.g., pandas, http://pandas.pydata.org/) or > weather/climate data specific (e.g., Iris, http://scitools.org.uk/iris/and > CDAT, > http://www2-pcmdi.llnl.gov/cdat/manuals/cdutil/cdat_utilities.html). > > Cheers, > Stephan > > > On Wed, May 21, 2014 at 5:27 PM, questions anon > <questions.a...@gmail.com>wrote: > >> >> I have hourly 2D temperature data in a monthly netcdf and I would like to >> find the daily maximum temperature. The shape of the netcdf is (744, 106, >> 193) >> >> I would like to use the year-month-day as a new list name (i.e. >> 2009-03-01, 2009-03-02....2009-03-31) and then add each of the hours worth >> of temperature data to each corresponding list. Therefore each new list >> should contain 24 hours worth of data and the shape should be (24,106,193) >> . This is the part I cannot seem to get to work. I am using datetime and >> then groupby to group by date but I am not sure how to use the output to >> make a new list name and then add the data for that day into that list. see >> below and attached for my latest attempt. Any feedback will be greatly >> appreciated. >> >> >> >> from netCDF4 import Dataset >> >> import numpy as np >> >> import matplotlib.pyplot as plt >> >> from mpl_toolkits.basemap import Basemap >> >> from netcdftime import utime >> >> from datetime import datetime as dt >> >> import os >> >> import gc >> >> from numpy import * >> >> import pytz >> >> from itertools import groupby >> >> >> MainFolder=r"/DATA/2009/03" >> >> dailydate=[] >> >> alltime=[] >> >> lists={} >> >> >> >> ncvariablename='T_SFC' >> >> >> for (path, dirs, files) in os.walk(MainFolder): >> >> for ncfile in files: >> >> print ncfile >> >> fileext='.nc' >> >> if ncfile.endswith(ncvariablename+'.nc'): >> >> print "dealing with ncfiles:", path+ncfile >> >> ncfile=os.path.join(path,ncfile) >> >> ncfile=Dataset(ncfile, 'r+', 'NETCDF4') >> >> variable=ncfile.variables[ncvariablename][:,:,:] >> >> TIME=ncfile.variables['time'][:] >> >> ncfile.close() >> >> for temp, time in zip((variable[:]),(TIME[:])): >> >> cdftime=utime('seconds since 1970-01-01 00:00:00') >> >> ncfiletime=cdftime.num2date(time) >> >> timestr=str(ncfiletime) >> >> utc_dt = dt.strptime(timestr, '%Y-%m-%d %H:%M:%S') >> >> au_tz = pytz.timezone('Australia/Sydney') >> >> local_dt = utc_dt.replace(tzinfo=pytz.utc).astimezone(au_tz) >> >> alltime.append(local_dt) >> >> for k, g in groupby(alltime, key=lambda d: d.date()): >> >> kstrp_local=k.strftime('%Y-%m-%d_%H') >> >> klocal_date=k.strftime('%Y-%m-%d') >> >> dailydate.append(klocal_date) >> >> for n in dailydate: >> >> lists[n]=[] >> >> lists[n].append(temp) >> >> >> big_array=np.ma.concatenate(lists[n]) >> >> DailyTemp=big_array.max(axis=0) >> >> >> _______________________________________________ >> NumPy-Discussion mailing list >> NumPy-Discussion@scipy.org >> http://mail.scipy.org/mailman/listinfo/numpy-discussion >> >> > > _______________________________________________ > NumPy-Discussion mailing list > NumPy-Discussion@scipy.org > http://mail.scipy.org/mailman/listinfo/numpy-discussion > >
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org http://mail.scipy.org/mailman/listinfo/numpy-discussion
