t;Local mode (-x local) when I ran it on my laptop, and mapreduce
> >> >>>mode when I ran it on ec2 cluster.
> >> >>>
> >> >>>2. If mapreduce mode, did you look into the hadoop log to see
> >> >>> how much slow do
>>> 3. What kind of query is it?
>> >>>
>> >>> The input is gzipped json files which has one event per line.
>> >>> Then I do some hourly aggregation on the raw events, then do
>> >>> bunch of groupping, joining and s
t; >>>
> >>> Daniel
> >>>
> >>> Someone mentioned it's EC2's I/O performance. But I'm sure there
> >>>are plenty of people using EC2/EMR running big MR jobs so more
> >>>likely I have some configuration issues? My
t;>> median, variance) on some fields.
>>>
>>> Daniel
>>>
>>> Someone mentioned it's EC2's I/O performance. But I'm sure there
>>> are plenty of people using EC2/EMR running big MR jobs so more
>>> likely
that running on my laptop is faster tells me
this is a separate issue.
Thanks!
On 06/13/2011 11:54 AM, Dexin Wang wrote:
Hi,
This is probably not directly a Pig question.
Anyone running Pig on amazon EC2 instances? Something's
ance. But I'm sure there are
> plenty of people using EC2/EMR running big MR jobs so more likely I have
> some configuration issues? My jobs can be optimized a bit but the fact that
> running on my laptop is faster tells me this is a separate issue.
>
> Thanks!
>
>
>
>>
ues? My jobs can be optimized a bit but the
fact that running on my laptop is faster tells me this is a separate
issue.
Thanks!
On 06/13/2011 11:54 AM, Dexin Wang wrote:
Hi,
This is probably not directly a Pig question.
Anyone running Pig on amazon E
ion issues? My jobs can be optimized a bit but the fact that
running on my laptop is faster tells me this is a separate issue.
Thanks!
> On 06/13/2011 11:54 AM, Dexin Wang wrote:
>
>> Hi,
>>
>> This is probably not directly a Pig question.
>>
>> Anyone runn
directly a Pig question.
Anyone running Pig on amazon EC2 instances? Something's not making sense to
me. I ran a Pig script that has about 10 mapred jobs in it on a 16 node
cluster using m1.small. It took *13 minutes*. The job reads input from S3
and writes output to S3. But from the log
Hi,
This is probably not directly a Pig question.
Anyone running Pig on amazon EC2 instances? Something's not making sense to
me. I ran a Pig script that has about 10 mapred jobs in it on a 16 node
cluster using m1.small. It took *13 minutes*. The job reads input from S3
and writes output
10 matches
Mail list logo
