ill use this size to split
>>>> the input LZO files and u will get multiple mappers (and make sure the
>>>> input LZO files are indexed I.e. .LZO.INDEX files are created)
>>>>
>>>> sanjay
>>>>
>>>>
>>>> From: Edward C
;
>>>
>>> If u set this value "low" then the MR job will use this size to split
>>> the input LZO files and u will get multiple mappers (and make sure the
>>> input LZO files are indexed I.e. .LZO.INDEX files are created)
>>>
>>> sanjay
O.INDEX files are created)
>>
>> sanjay
>>
>>
>> From: Edward Capriolo
>> Reply-To: "user@hive.apache.org"
>> Date: Wednesday, August 21, 2013 10:43 AM
>> To: "user@hive.apache.org"
>> Subject: Re: only one map
ly-To: "user@hive.apache.org"
> Date: Wednesday, August 21, 2013 10:43 AM
> To: "user@hive.apache.org"
> Subject: Re: only one mapper
>
> LZO files are only splittable if you index them. Sequence files
> compresses with LZO are splittable without being index
g>"
mailto:user@hive.apache.org>>
Subject: Re: only one mapper
LZO files are only splittable if you index them. Sequence files compresses with
LZO are splittable without being indexed.
Snappy + SequenceFile is a better option then LZO.
On Wed, Aug 21, 2013 at 1:39 PM, Igor Tatarin
> LZO files are combinable so check your max split setting.
>>
>> http://mail-archives.apache.org/mod_mbox/hive-user/201107.mbox/%3c4e328964.7000...@gmail.com%3E
>>
>> igor
>> decide.com
>>
>>
>>
>> On Wed, Aug 21, 2013 at 2:17 AM, 闫昆 wrote:
>>
ing.
>
> http://mail-archives.apache.org/mod_mbox/hive-user/201107.mbox/%3c4e328964.7000...@gmail.com%3E
>
> igor
> decide.com
>
>
>
> On Wed, Aug 21, 2013 at 2:17 AM, 闫昆 wrote:
>
>> hi all when i use hive
>> hive job make only one mapper actually my file spli
LZO files are combinable so check your max split setting.
http://mail-archives.apache.org/mod_mbox/hive-user/201107.mbox/%3c4e328964.7000...@gmail.com%3E
igor
decide.com
On Wed, Aug 21, 2013 at 2:17 AM, 闫昆 wrote:
> hi all when i use hive
> hive job make only one mapper actually my file
hi all when i use hive
hive job make only one mapper actually my file split 18 block my block size
is 128MB and data size 2GB
i use lzo compression and create file.lzo and make index file.lzo.index
i use hive 0.10.0
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set
using only one mapper
** **
** **
On Thu, Jul 28, 2011 at 9:23 AM, Wojciech Langiewicz<
wlangiew...@gmail.com> wrote:
Hello,
I'm having isssue running Hive jobs after updating from Hive 0.5 to Hive
0.7 (from CDHb4 to CDHu1).
No matter what query I'm running Hive is alway
**
>
> The maximum size chunk that map input should be split
>
> into.
>
>
>
>
>
> Thanks
>
> Vaibhav
>
> ** **
>
> *From:* Edward Capriolo [mailto:edlinuxg...@gmail.com]
> *Sent:* Thursday, July 28, 2011 7:10 AM
>
:
mapred.max.split.size
134217728
The maximum size chunk that map input should be split
into.
Thanks
Vaibhav
From: Edward Capriolo [mailto:edlinuxg...@gmail.com]
Sent: Thursday, July 28, 2011 7:10 AM
To: user@hive.apache.org
Subject: Re: Hive 0.7 using only one mapper
On Thu, Jul 28, 2011 at 9:23 AM
On Thu, Jul 28, 2011 at 9:23 AM, Wojciech Langiewicz
wrote:
> Hello,
> I'm having isssue running Hive jobs after updating from Hive 0.5 to Hive
> 0.7 (from CDHb4 to CDHu1).
>
> No matter what query I'm running Hive is always using one mapper.
> I have tried different queries with various sizes of
Hello,
I'm having isssue running Hive jobs after updating from Hive 0.5 to Hive
0.7 (from CDHb4 to CDHu1).
No matter what query I'm running Hive is always using one mapper.
I have tried different queries with various sizes of input and ones with
many reducers or no reducers.
For version 0.5
14 matches
Mail list logo
