but it's map only job At 2012-09-27 05:39:39,"Chen Song" <chen.song...@gmail.com> wrote: As far as I know, the number of files emitted would be determined by the number of mappers for a map only job and the number of reducers for a map reduce job.
So it totally depends how your query translates into a MR job. You can enforce it by setting the property mapred.reduce.tasks=1 Chen On Wed, Sep 19, 2012 at 11:25 PM, 王锋 <wfeng1...@163.com> wrote: Hi I tried to convert and merge many small text files using RCFiles using hivesql,but hive produced some small rcfiles. set hive.exec.compress.output=true; set mapred.output.compress=true; set mapred.output.compression.codec=com.hadoop.compression.lzo.LzoCodec; set io.compression.codecs=com.hadoop.compression.lzo.LzoCodec; hive.merge.mapfiles=true hive.merge.mapredfiles=true hive.merge.size.per.task=640000000 hive.merge.size.smallfiles.avgsize=80000000 insert overwrite table rctable select ..... the settings: hive.merge.mapfiles=true hive.merge.mapredfiles=true hive.merge.size.per.task=640000000 hive.merge.size.smallfiles.avgsize=80000000 didn't work. who could tell me how to solve it? -- Chen Song
