, 2016 7:24 PM
To: Piyush Mukati mailto:piyush.muk...@gmail.com>>;
user@hadoop.apache.org <mailto:user@hadoop.apache.org>
Subject: Re: merging small files in HDFS
Hi,
if I correctly understand your request you need only to merge some data
resulting from an hdfs write
op fs -getmerge
>>
>> --Senthil
>> -Original Message-
>> From: Giovanni Mascari [mailto:giovanni.masc...@polito.it]
>> Sent: Thursday, November 03, 2016 7:24 PM
>> To: Piyush Mukati ; user@hadoop.apache.org
>> Subject: Re: merging small files in HDFS
&g
t;
> hadoop fs -getmerge
>
> --Senthil
> -Original Message-
> From: Giovanni Mascari [mailto:giovanni.masc...@polito.it]
> Sent: Thursday, November 03, 2016 7:24 PM
> To: Piyush Mukati ; user@hadoop.apache.org
> Subject: Re: merging small files in HDFS
>
> Hi,
>
Hi ,
You need to write a map method to just parse input file and pass it to
reducer.. use only reducer..so that all maps output will go to one reducer
and one file gets created,which is merge of input files..
On 03-Nov-2016 8:54 pm, "Piyush Mukati" wrote:
> Hi,
> I want to merge multiple files
> From: Giovanni Mascari [mailto:giovanni.masc...@polito.it]
>
> Sent: Thursday, November 03, 2016 7:24 PM
>
> To: Piyush Mukati ; user@hadoop.apache.org
>
> Subject: Re: merging small files in HDFS
>
>
>
> Hi,
>
> if I correctly understand your request you need
ukati ; user@hadoop.apache.org
Subject: Re: merging small files in HDFS
Hi,
if I correctly understand your request you need only to merge some data
resulting from an hdfs write operation.
In this case, I suppose that your best option is to use hadoop-stream with
'cat' command.
take a l
Hi,
if I correctly understand your request you need only to merge some data
resulting from an hdfs write operation.
In this case, I suppose that your best option is to use hadoop-stream
with 'cat' command.
take a look here:
https://hadoop.apache.org/docs/r1.2.1/streaming.html
Regards
Il 03/1