Re: [Tutor] Help with iterators
Dear list,
Sorry for the delay -- it has taken some time for me to get these emails.
It appears i made some dumb error when typing out the description.
Mitya Sirenef was correct to ignore my words and to focus on my code.
Thanks for your help. I may ask again / for more help when i feel i
have tried sufficiently hard to absorb the answers below.
Thanks again
mj
On 22/03/2013, at 6:24 PM, tutor-requ...@python.org
tutor-requ...@python.org wrote:
Send Tutor mailing list submissions to
tutor@python.org
To subscribe or unsubscribe via the World Wide Web, visit
http://mail.python.org/mailman/listinfo/tutor
or, via email, send a message with subject or body 'help' to
tutor-requ...@python.org
You can reach the person managing the list at
tutor-ow...@python.org
When replying, please edit your Subject line so it is more specific
than Re: Contents of Tutor digest...
Today's Topics:
1. Re: Help with iterators (Mitya Sirenef)
2. Re: Help with iterators (Steven D'Aprano)
3. Re: Help with iterators (Steven D'Aprano)
4. Re: Help with iterators (Mitya Sirenef)
5. Please Help (Arijit Ukil)
--
Message: 1
Date: Thu, 21 Mar 2013 21:39:12 -0400
From: Mitya Sirenef msire...@lightbird.net
To: tutor@python.org
Subject: Re: [Tutor] Help with iterators
Message-ID: 514bb640.5050...@lightbird.net
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
On 03/21/2013 08:39 PM, Matthew Johnson wrote:
Dear list,
I have been trying to understand out how to use iterators and in
particular groupby statements. I am, however, quite lost.
I wish to subset the below list, selecting the observations that have
an ID ('realtime_start') value that is greater than some date (i've
used the variable name maxDate), and in the case that there is more
than one such record, returning only the one that has the largest ID
('realtime_start').
The code below does the job, however i have the impression that it
might be done in a more python way using iterators and groupby
statements.
could someone please help me understand how to go from this code to
the pythonic idiom?
thanks in advance,
Matt Johnson
_
## Code example
import pprint
obs = [{'date': '2012-09-01',
'realtime_end': '2013-02-18',
'realtime_start': '2012-10-15',
'value': '231.951'},
{'date': '2012-09-01',
'realtime_end': '2013-02-18',
'realtime_start': '2012-11-15',
'value': '231.881'},
{'date': '2012-10-01',
'realtime_end': '2013-02-18',
'realtime_start': '2012-11-15',
'value': '231.751'},
{'date': '2012-10-01',
'realtime_end': '-12-31',
'realtime_start': '2012-12-19',
'value': '231.623'},
{'date': '2013-02-01',
'realtime_end': '-12-31',
'realtime_start': '2013-03-21',
'value': '231.157'},
{'date': '2012-11-01',
'realtime_end': '2013-02-18',
'realtime_start': '2012-12-14',
'value': '231.025'},
{'date': '2012-11-01',
'realtime_end': '-12-31',
'realtime_start': '2013-01-19',
'value': '231.071'},
{'date': '2012-12-01',
'realtime_end': '2013-02-18',
'realtime_start': '2013-01-16',
'value': '230.979'},
{'date': '2012-12-01',
'realtime_end': '-12-31',
'realtime_start': '2013-02-19',
'value': '231.137'},
{'date': '2012-12-01',
'realtime_end': '-12-31',
'realtime_start': '2013-03-19',
'value': '231.197'},
{'date': '2013-01-01',
'realtime_end': '-12-31',
'realtime_start': '2013-02-21',
'value': '231.198'},
{'date': '2013-01-01',
'realtime_end': '-12-31',
'realtime_start': '2013-03-21',
'value': '231.222'}]
maxDate = 2013-03-21
dobs = dict([(d, []) for d in set([e['date'] for e in obs])])
for o in obs:
dobs[o['date']].append(o)
dobs_subMax = dict([(k, [d for d in v if d['realtime_start'] = maxDate])
for k, v in dobs.items()])
rts = lambda x: x['realtime_start']
mmax = [sorted(e, key=rts)[-1] for e in dobs_subMax.values() if e]
mmax.sort(key = lambda x: x['date'])
pprint.pprint(mmax)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
You can do it with groupby like so:
from itertools import groupby
from operator import itemgetter
maxDate = 2013-03-21
mmax= list()
obs.sort(key=itemgetter('date'))
for k, group in groupby(obs, key=itemgetter('date')):
group = [dob for dob in group if dob['realtime_start'] = maxDate]
if group:
group.sort(key=itemgetter('realtime_start'))
mmax.append(group[-1])
pprint.pprint(mmax)
Note that writing multiply-nested comprehensions like you did results in
very unreadable code. Do you find this code more readable?
-m
--
Lark's Tongue Guide to Python: http://lightbird.net/larks/
Many a man fails as an original thinker simply because his memory it too
good. Friedrich Nietzsche
--
Message: 2
[Tutor] Help with iterators
Dear list,
I have been trying to understand out how to use iterators and in
particular groupby statements. I am, however, quite lost.
I wish to subset the below list, selecting the observations that have
an ID ('realtime_start') value that is greater than some date (i've
used the variable name maxDate), and in the case that there is more
than one such record, returning only the one that has the largest ID
('realtime_start').
The code below does the job, however i have the impression that it
might be done in a more python way using iterators and groupby
statements.
could someone please help me understand how to go from this code to
the pythonic idiom?
thanks in advance,
Matt Johnson
_
## Code example
import pprint
obs = [{'date': '2012-09-01',
'realtime_end': '2013-02-18',
'realtime_start': '2012-10-15',
'value': '231.951'},
{'date': '2012-09-01',
'realtime_end': '2013-02-18',
'realtime_start': '2012-11-15',
'value': '231.881'},
{'date': '2012-10-01',
'realtime_end': '2013-02-18',
'realtime_start': '2012-11-15',
'value': '231.751'},
{'date': '2012-10-01',
'realtime_end': '-12-31',
'realtime_start': '2012-12-19',
'value': '231.623'},
{'date': '2013-02-01',
'realtime_end': '-12-31',
'realtime_start': '2013-03-21',
'value': '231.157'},
{'date': '2012-11-01',
'realtime_end': '2013-02-18',
'realtime_start': '2012-12-14',
'value': '231.025'},
{'date': '2012-11-01',
'realtime_end': '-12-31',
'realtime_start': '2013-01-19',
'value': '231.071'},
{'date': '2012-12-01',
'realtime_end': '2013-02-18',
'realtime_start': '2013-01-16',
'value': '230.979'},
{'date': '2012-12-01',
'realtime_end': '-12-31',
'realtime_start': '2013-02-19',
'value': '231.137'},
{'date': '2012-12-01',
'realtime_end': '-12-31',
'realtime_start': '2013-03-19',
'value': '231.197'},
{'date': '2013-01-01',
'realtime_end': '-12-31',
'realtime_start': '2013-02-21',
'value': '231.198'},
{'date': '2013-01-01',
'realtime_end': '-12-31',
'realtime_start': '2013-03-21',
'value': '231.222'}]
maxDate = 2013-03-21
dobs = dict([(d, []) for d in set([e['date'] for e in obs])])
for o in obs:
dobs[o['date']].append(o)
dobs_subMax = dict([(k, [d for d in v if d['realtime_start'] = maxDate])
for k, v in dobs.items()])
rts = lambda x: x['realtime_start']
mmax = [sorted(e, key=rts)[-1] for e in dobs_subMax.values() if e]
mmax.sort(key = lambda x: x['date'])
pprint.pprint(mmax)
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
[Tutor] getting and using information dict objects
Hi,
I am trying to make a move from excel to python. My main need is
economic data -- to do economic analysis.
I have found the FRED package, and appear to have connected to the St
Louis Fed's FRED and downloaded some data, but i'm not sure what to do
with the objects that are returned.
_
import fred
fred.key(fredKey)
(i guessed that i should not provide my own FRED API key)
gnp = fred.series('GNPCA')
i can see that the object returned is a dict
type(gnp)
type 'dict'
but i cannot see how to access the observations, or use them in any way.
After a bit of fiddling about, i managed to find another method, which
i think also returns a dictionary of values and meta-data:
gnpObvs = fred.observations('GNPCA')
however I cannot figure out how to get the observation values out and
use them.
I may be thinking in the wrong framework -- i guess i'm expecting
something that is an intuitive as an excel table.
The end game for me is making plots of variables, combining them, and
doing regressions -- i cannot see how i might take what i'm getting
from these FRED / Python dicts to actual data analysis.
Could someone please help me take these first steps?
thanks + best regards
matt
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] getting and using information dict objects
I can see i was being unclear: i wish to replace my analysis in excel with analysis in python. I am fairly sure i am querying the FRED API, but i am unsure how to _access and use_ the dict objects that it is returning. For example, how would i just print out values? Thanks for your help On 24/02/2013, at 9:12 AM, Albert-Jan Roskam fo...@yahoo.com wrote: I am trying to make a move from excel to python. My main need is economic data -- to do economic analysis. If you say Python and Excel the first thing that comes to mind is the xlrd package snip The end game for me is making plots of variables, combining them, and doing regressions -- i cannot see how i might take what i'm getting from these FRED / Python dicts to actual data analysis. Could someone please help me take these first steps? I'm not familiar with the Fred package. Pandas (http://pandas.pydata.org/) seems to be perfect for what you're doing. It's an R-like layer on top of numpy. It can be used to read Excel and many other formats and it works well with matplotlib. It's a cool tool to manipulate/massage/munge data, then analyze it (though R has still more, often ueber-exotic, stats). Its origin lies in the analysis of... economic data. Best used with IPython. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] getting and using information dict objects
Thanks very much; hopefully that's the boost i need to get rolling. Best regards matt On 24/02/2013, at 10:40 AM, Robert Sjoblom robert.sjob...@gmail.com wrote: I am fairly sure i am querying the FRED API, but i am unsure how to _access and use_ the dict objects that it is returning. For example, how would i just print out values? If it's a dict object, the standard dictionary behavior and methods should work. I've not looked closely at the FRED API, but something like (untested): for key in dictionary: print(key, dictionary[key]) could possibly get you started. -- best regards, Robert S. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] getting and using information dict objects
For the sake of those who finds this thread -- the date / value pairs
can be printed by the following:
import fred
fred.key(fredKey)
gnpObvs = fred.observations('GNPCA')
for i in range(1, len(gnpObvs['observations']['observation'])):
print gnpObvs['observations']['observation'][i]['date'],
gnpObvs['observations']['observation'][i]['value']
mj
On 24 February 2013 10:51, Matthew Johnson mcoog...@gmail.com wrote:
Thanks very much; hopefully that's the boost i need to get rolling.
Best regards
matt
On 24/02/2013, at 10:40 AM, Robert Sjoblom robert.sjob...@gmail.com wrote:
I am fairly sure i am querying the FRED API, but i am unsure how to
_access and use_ the dict objects that it is returning. For example,
how would i just print out values?
If it's a dict object, the standard dictionary behavior and methods
should work. I've not looked closely at the FRED API, but something
like (untested):
for key in dictionary:
print(key, dictionary[key])
could possibly get you started.
--
best regards,
Robert S.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
