6
2 1 32 3.5 1112484819 2005-04-02 23:33:39
3 1 47 3.5 1112484727 2005-04-02 23:32:07
4 1 50 3.5 1112484580 2005-04-02 23:29:40
I'm trying two methods:
Method 1 (makes sense)
> ratings[['movieId', 'rating']].groupb
It worked very well.
>>
>> Now I want to use it to process a database table. I can select the rows in
>> the desired sequence with no problem. However, I am using asyncio, so I am
>> reading the rows asynchronously.
>>
>> Before I spend hours trying to figure
table. I can select the rows in
> the desired sequence with no problem. However, I am using asyncio, so I am
> reading the rows asynchronously.
>
> Before I spend hours trying to figure it out, can anyone confirm if this is
> doable at all, or is groupby not designed for this.
Most
I am using asyncio, so I am
reading the rows asynchronously.
My 'reader' class has __aiter__() and __anext__() defined.
If I pass the reader to groupby, I get the error message 'object is not
iterable'.
Before I spend hours trying to figure it out, can anyone confirm if
On Sun, May 15, 2016 at 7:07 AM David Shi wrote:
> Hello, Michael,
>
> Pandas GroupBy does not behave consistently.
>
> Last time, when we had conversation, I used grouby. It works well.
>
> Now, I thought to re-write the program, so that I can end up with a clean
> scri
Hello, Michael,
Pandas GroupBy does not behave consistently.
Last time, when we had conversation, I used grouby. It works well.
Now, I thought to re-write the program, so that I can end up with a clean
script.
But, the problem is that a lot of columns are missing after groupby application.
Any
t that situation it's simplest to just
convert each group to a list for processing it:
for k,g in groupby(...):
gs = list(g)
# do stuff with gs
calling list(g) reads all the elements in the group, advancing groupby's
internal iterator to the next group. Since you'
2013/2/26 Ian Kelly :
> On Tue, Feb 26, 2013 at 9:27 AM, andrea crotti
> wrote:
>> So I was trying to use groupby (which I used in the past), but I
>> noticed a very strange thing if using list on
>> the result:
>
> As stated in the docs:
>
> ""&qu
On Tue, Feb 26, 2013 at 9:27 AM, andrea crotti
wrote:
> So I was trying to use groupby (which I used in the past), but I
> noticed a very strange thing if using list on
> the result:
As stated in the docs:
"""
The returned group is itself an iterator that shares the
What I would like to do is allow a tuple/list of index values, rather
> than a single index value to be summed up, so you could say
> group_results(mylist,[0],[1,2]) would return [(1, 5,7), (2, 7,9)] but
> I'm struggling to do so, any thoughts? Cheers
>
> from itertools impor
than a single index value to be summed up, so you could say
group_results(mylist,[0],[1,2]) would return [(1, 5,7), (2, 7,9)] but
I'm struggling to do so, any thoughts? Cheers
from itertools import groupby as gb
from operator import itemgetter as ig
def group_results(table,keys,value
e following:
> I have a list of dictionaries, something like
>
> [ { "a": 1, "b": 1, "c": 3 },
> { "a": 1, "b": 1, "c": 4 },
> ...
> ]
>
> and I'd like to iterate through all items with, e.g., "a"
> Are you basically after this, then?
>
> for a, a_iter in groupby(my_list, itemgetter('a')):
>print 'New A', a
>for b, b_iter in groupby(a_iter, itemgetter('b')):
>b_list = list(b_iter)
>for p in [&#
Nico Schlömer wrote:
> So when I go like
>
> for item in list:
> item[1].sort()
>
> I actually modify *list*? I didn't realize that; I thought it'd just
> be a copy of it.
No, I misunderstood your code there. Modifying the objects inside the list
is fine, but I don't thing you do that, provi
On 4 May, 12:36, Nico Schlömer wrote:
> > Does this example help at all?
>
> Thanks, that clarified things a lot!
>
> To make it easier, let's just look at 'a' and 'b':
>
> > my_list.sort( key=itemgetter('a','b','c&#x
st of dictionaries, something like
>>
>> [ { "a": 1, "b": 1, "c": 3 },
>> { "a": 1, "b": 1, "c": 4 },
>> ...
>> ]
>>
>> and I'd like to iterate through all items with, e.g., "a":
> Does this example help at all?
Thanks, that clarified things a lot!
To make it easier, let's just look at 'a' and 'b':
> my_list.sort( key=itemgetter('a','b','c') )
> for a, a_iter in groupby(my_list, itemgetter('a')):
em is the following:
> I have a list of dictionaries, something like
>
> [ { "a": 1, "b": 1, "c": 3 },
> { "a": 1, "b": 1, "c": 4 },
> ...
> ]
>
> and I'd like to iterate through all items wit
have a list of dictionaries, something like
>
> [ { "a": 1, "b": 1, "c": 3 },
> { "a": 1, "b": 1, "c": 4 },
> ...
> ]
>
> and I'd like to iterate through all items with, e.g., "a":1. What I do
"a": 1, "b": 1, "c": 3 },
{ "a": 1, "b": 1, "c": 4 },
...
]
and I'd like to iterate through all items with, e.g., "a":1. What I do
is sort and then groupby,
my_list.sort( key=operator.itemgetter('a') )
my_l
On Jan 4, 4:55 pm, [EMAIL PROTECTED] wrote:
> Petr thanks so much for your input. I'll try to learnSQL, especially
> if I'll do a lot of database work.
>
> I tried to do it John's way as en exercise and I'm happy to say I
> understand a lot more. Basically I didn't realize I could nest
> dictiona
Petr thanks so much for your input. I'll try to learn SQL, especially
if I'll do a lot of database work.
I tried to do it John's way as en exercise and I'm happy to say I
understand a lot more. Basically I didn't realize I could nest
dictionaries like db = {country:{genre:{sub_genre:3}}} and cal
On Jan 3, 3:41 pm, [EMAIL PROTECTED] wrote:
> Yes in the sense that the top part will have merged cells so that
> Horror and Classics don't need to be repeated every time, but the
> headers aren't the important part. At this point I'm more interested
> in organizing the data itself and i can worry
Yes in the sense that the top part will have merged cells so that
Horror and Classics don't need to be repeated every time, but the
headers aren't the important part. At this point I'm more interested
in organizing the data itself and i can worry about putting it into a
new excel file later.
--
h
> So the data comes in as a long list. I'm dealing with some
> information on various countries with 6 pieces of information to
> pivot. Just to make it simple it's like a video store database. The
> data is like [Country, Category, Sub Category, Film Title, Director,
> Number of Copies]. data
Sorry for the delay in my response. New Year's Eve and moving
apartment
> - Where the data come from (I mean: are your data in Excel already
> when you get them)?
> - If your primary source of data is the Excel file, how do you read
> data from the Excel file to Python (I mean did you solve this
> Yes, I realize Excel has excellent support for pivot tables. However,
> I hate how Excel does it and, for my particular excel files, I need
> them to be formated in an automated way because I will have a number
> of them over time and I'd prefer just to have python do it in a flash
> than to do
On Dec 29, 3:00 pm, [EMAIL PROTECTED] wrote:
> Patrick,
>
> in your first posting you are writing "... I'm trying to learn how to
> make pivot tables from some excel sheets...". Can you be more specific
> please? AFIK Excel offers very good support for pivot tables. So why
> to read tabular data fr
Patrick,
in your first posting you are writing "... I'm trying to learn how to
make pivot tables from some excel sheets...". Can you be more specific
please? AFIK Excel offers very good support for pivot tables. So why
to read tabular data from the Excel sheet and than transform it to
pivot tabel
On Dec 29, 11:51 am, [EMAIL PROTECTED] wrote:
> John would you mind walking me through your class in normal speak?
Yes.
> I
> only have a vague idea of why it works and this would help me a lot to
> get a grip on classes and this sort of particular problem.
It's about time you got a *concrete*
Petr, thanks for the SQL suggestion, but I'm having enough trouble in
Python.
John would you mind walking me through your class in normal speak? I
only have a vague idea of why it works and this would help me a lot to
get a grip on classes and this sort of particular problem. The next
step is to
On Dec 29, 9:58 am, [EMAIL PROTECTED] wrote:
> What about to let SQL to work for you.
The OP is "trying to learn how to make pivot tables from some excel
sheets". You had better give him a clue on how to use ODBC on an
"excel sheet" :-)
[snip]
> SELECT
> NAME,
> sum (AMOUNT) as TOTAL,
> sum (case
What about to let SQL to work for you.
HTH
Petr Jakes
Tested on Python 2.5.1
8<--
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import sqlite3
con = sqlite3.connect(":memory:")
cur = con.cursor()
inputData=(
('Bob', 'Morn', 240),
('Bob', 'Aft', 300),
('Joe', 'Morn', 70),
('
Wow, I did not realize it would be this complicated! I'm fairly new
to Python and somehow I thought I could find a simpler solution. I'll
have to mull over this to fully understand how it works for a bit.
Thanks a lot!
On Dec 28, 4:03 am, John Machin <[EMAIL PROTECTED]> wrote:
> On Dec 28, 11:4
On Dec 28, 11:48 am, John Machin <[EMAIL PROTECTED]> wrote:
> On Dec 28, 10:05 am, [EMAIL PROTECTED] wrote:
>
>
> > If you have any ideas about how to solve this pivot table issue, which
> > seems to be scant on Google, I'd much appreciate it. I know I can do
> > this in Excel easily with the auto
On Dec 28, 10:05 am, [EMAIL PROTECTED] wrote:
> On Dec 27, 10:59 pm, John Machin <[EMAIL PROTECTED]> wrote:
>
> > On Dec 28, 4:56 am, [EMAIL PROTECTED] wrote:
>
> > > from itertools import groupby
>
> > You seem to have overlooked this important sentence in
On Dec 27, 10:59 pm, John Machin <[EMAIL PROTECTED]> wrote:
> On Dec 28, 4:56 am, [EMAIL PROTECTED] wrote:
>
> > from itertools import groupby
>
> You seem to have overlooked this important sentence in the
> documentation: "Generally, the iterable needs to alr
On Dec 28, 4:56 am, [EMAIL PROTECTED] wrote:
> from itertools import groupby
You seem to have overlooked this important sentence in the
documentation: "Generally, the iterable needs to already be sorted on
the same key function"
--
http://mail.python.org/mailman/listinfo/python-list
ow to sum these values because of the
iteration. I always get an error like: TypeError: iteration over non-
sequence
Here's the code:
from itertools import groupby
data = [['Bob', 'Morn', 240],['Bob', 'Aft', 300],['Joe', 'Morn', 70],
On Oct 15, 8:02 pm, 7stud <[EMAIL PROTECTED]> wrote:
> t = timeit.Timer("test3()", "from __main__ import test3, key, data")
> print t.timeit()
> t = timeit.Timer("test1()", "from __main__ import test1, data")
> print t.timeit()
>
> -
On Oct 15, 11:02 pm, 7stud <[EMAIL PROTECTED]> wrote:
> I'm applying groupby() in a very simplistic way to split up some data,
> but when I timeit against another method, it takes twice as long. The
> following groupby() code groups the data between the "" strings
I'm applying groupby() in a very simplistic way to split up some data,
but when I timeit against another method, it takes twice as long. The
following groupby() code groups the data between the "" strings:
data = [
"1.5","","2.5","3.5"
rit, it would be fun to compare several different
> > approaches to the same problem using re.finditer, itertools.groupby,
> > or the tokenize module. To get the ball rolling, here is one variant:
>
> > from itertools import groupby
>
> > def blocks(s, start, end):
> >
or the tokenize module. To get the ball rolling, here is one variant:
>
> from itertools import groupby
>
> def blocks(s, start, end):
> def classify(c, ingroup=[0], delim={start:2, end:3}):
> result = delim.get(c, ingroup[0])
> ingroup[0] = result in (1, 2)
Roman Bertle wrote:
> Hello,
>
> there is an example how to use groupby in the itertools documentation
> (http://docs.python.org/lib/itertools-example.html):
>
> # Show a dictionary sorted and grouped by value
>>>> from operator import itemgetter
>>>>
Hello,
there is an example how to use groupby in the itertools documentation
(http://docs.python.org/lib/itertools-example.html):
# Show a dictionary sorted and grouped by value
>>> from operator import itemgetter
>>> d = dict(a=1, b=2, c=1, d=2, e=1, f=2, g=3)
>>> di
James Stroud <[EMAIL PROTECTED]> wrote:
> Alex Martelli wrote:
> > James Stroud <[EMAIL PROTECTED]> wrote:
> >...
> >
> >>def doit(rows, doers, i=0):
> >> for r, alist in groupby(rows, itemgetter(i)):
> >> if len(doers) >
Alex Martelli wrote:
> James Stroud <[EMAIL PROTECTED]> wrote:
>...
>
>>def doit(rows, doers, i=0):
>> for r, alist in groupby(rows, itemgetter(i)):
>> if len(doers) > 1:
>> doit(alist, doers[1:], i+1)
>> doers[0](r)
>
>
Alex Martelli wrote:
> James Stroud <[EMAIL PROTECTED]> wrote:
>...
>
>>def doit(rows, doers, i=0):
>> for r, alist in groupby(rows, itemgetter(i)):
>> if len(doers) > 1:
>> doit(alist, doers[1:], i+1)
>> doers[0](r)
>
>
James Stroud <[EMAIL PROTECTED]> wrote:
...
> def doit(rows, doers, i=0):
>for r, alist in groupby(rows, itemgetter(i)):
> if len(doers) > 1:
>doit(alist, doers[1:], i+1)
> doers[0](r)
Isn't this making N useless slices (thus copies, for m
Frank Millman <[EMAIL PROTECTED]> wrote:
> Benji York wrote:
> > Frank Millman wrote:
> > > reader = csv.reader(open('trans.csv', 'rb'))
> > > rows = []
> > > for row in reader:
> > > rows.append(row)
> >
> > Why do you create a list of rows instead of just iterating over the
> > reader direct
Benji York wrote:
> Frank Millman wrote:
> > reader = csv.reader(open('trans.csv', 'rb'))
> > rows = []
> > for row in reader:
> > rows.append(row)
>
> Why do you create a list of rows instead of just iterating over the
> reader directly?
> --
> Benji York
A - didn't think of it - good idea
John Machin" <[EMAIL PROTECTED]> wrote in message
news:[EMAIL PROTECTED]
> On 14/06/2006 8:38 AM, Robert Kern wrote:
> > Gary Herron wrote:
> >> John Machin wrote:
> >>
> >>> On 13/06/2006 6:28 PM, Paul McGuire wrote:
> >>>
> &
On 14/06/2006 8:38 AM, Robert Kern wrote:
> Gary Herron wrote:
>> John Machin wrote:
>>
>>> On 13/06/2006 6:28 PM, Paul McGuire wrote:
>>>
>>>> (Oh, and I like groupby too! Combine it with sort to quickly create
>>>> histograms.)
&g
Gary Herron wrote:
> John Machin wrote:
>
>>On 13/06/2006 6:28 PM, Paul McGuire wrote:
>>
>>>(Oh, and I like groupby too! Combine it with sort to quickly create
>>>histograms.)
>>>
>>># tally a histogram of a list of values from 1-10
&g
On 14/06/2006 8:06 AM, Gary Herron wrote:
> John Machin wrote:
>> On 13/06/2006 6:28 PM, Paul McGuire wrote:
>>
>>
>>> (Oh, and I like groupby too! Combine it with sort to quickly create
>>> histograms.)
>>>
>>> # tally a histogram of a
John Machin wrote:
> On 13/06/2006 6:28 PM, Paul McGuire wrote:
>
>
>> (Oh, and I like groupby too! Combine it with sort to quickly create
>> histograms.)
>>
>> # tally a histogram of a list of values from 1-10
>> dataValueRange = range(1,11)
>> d
On 13/06/2006 6:28 PM, Paul McGuire wrote:
> (Oh, and I like groupby too! Combine it with sort to quickly create
> histograms.)
>
> # tally a histogram of a list of values from 1-10
> dataValueRange = range(1,11)
> data = [random.choice(dataValueRange) for i in xrange(1)
Not related to itertools.groupby, but the csv.reader object...
If for some reason you have malformed CSV files, with embedded newlines
or something of that effect, it will raise an exception. To skip those,
you will need a construct of something like this:
raw_csv_in = file('filenamehere.csv')
fo
w with the saved
>> values, and taking action if the values differ. The more break fields
>> there are, the fiddlier it gets.
>>
>> I was going to do the same in python, but then I vaguely remembered
>> reading about 'groupby'. It took a little while to figure
; there are, the fiddlier it gets.
>
> I was going to do the same in python, but then I vaguely remembered
> reading about 'groupby'. It took a little while to figure it out, but
> once I had cracked it, it transformed the task into one of utter
> simplicity.
>
> H
Frank Millman wrote:
> reader = csv.reader(open('trans.csv', 'rb'))
> rows = []
> for row in reader:
> rows.append(row)
Why do you create a list of rows instead of just iterating over the
reader directly?
--
Benji York
--
http://mail.python.org/mailman/listinfo/python-list
Frank;
I would just like to thank-you for this timely post.
I am working on a reporting project that needed "groupby" functionality
and I was going to sit down this morning to rework some "very ugly
code" into some "not quite so ugly code".
Your post got me pointed t
Paul McGuire wrote:
> >
> > reader = csv.reader(open('trans.csv', 'rb'))
> > rows = []
> > for row in reader:
> > rows.append(row)
> >
>
> This is untested, but you might think about converting your explicit "for...
> append" loop into either a list comp,
>
> rows = [row for row in reader]
row in reader]
or just a plain list constructor:
rows = list(reader)
Neh?
-- Paul
(Oh, and I like groupby too! Combine it with sort to quickly create
histograms.)
# tally a histogram of a list of values from 1-10
dataValueRange = range(1,11)
data = [random.choice(dataValueRange) for i in xran
, the fiddlier it gets.
>
> I was going to do the same in python, but then I vaguely remembered
> reading about 'groupby'. It took a little while to figure it out, but
> once I had cracked it, it transformed the task into one of utter
> simplicity.
>
> Here is an exam
about 'groupby'. It took a little while to figure it out, but
once I had cracked it, it transformed the task into one of utter
simplicity.
Here is an example. Imagine a transaction file sorted by branch,
account number, and date, and you want to break on
"Paul McGuire" <[EMAIL PROTECTED]> wrote in message
news:[EMAIL PROTECTED]
> So here's how to save the values from the iterators while iterating over
the
> groupby:
>
> >>> m = [(x,list(y)) for x,y in groupby([1, 1, 1, 2, 2, 3])]
> >>> m
"Bryan" <[EMAIL PROTECTED]> wrote in message
news:[EMAIL PROTECTED]
> George Sakkis wrote:
> > "The returned group is itself an iterator that shares the underlying
> > iterable with groupby(). Because the source is shared, when the groupby
> > object is
George Sakkis wrote:
> Bryan wrote:
>
>> can some explain why in the 2nd example, m doesn't print the list [1, 1, 1]
>> which i had expected?
>>
>>
>> >>> for k, g in groupby([1, 1, 1, 2, 2, 3]):
>> ... print k, list(g)
>>
Bryan wrote:
> can some explain why in the 2nd example, m doesn't print the list [1, 1, 1]
> which i had expected?
>
>
> >>> for k, g in groupby([1, 1, 1, 2, 2, 3]):
> ... print k, list(g)
> ...
> 1 [1, 1, 1]
> 2 [2, 2]
> 3 [3]
>
>
> >
can some explain why in the 2nd example, m doesn't print the list [1, 1, 1]
which i had expected?
>>> for k, g in groupby([1, 1, 1, 2, 2, 3]):
... print k, list(g)
...
1 [1, 1, 1]
2 [2, 2]
3 [3]
>>> m = list(groupby([1, 1, 1, 2, 2, 3]))
>>> m
[(1, )
72 matches
Mail list logo
