Also check hive.merge.size.per.task and hive.merge.smallfiles.avgsize. On Mon, Apr 20, 2015 at 8:29 AM, patcharee <patcharee.thong...@uni.no> wrote:
> Hi, > > How to set the configuration hive-site.xml to automatically merge small > orc file (output from mapreduce job) in hive 0.14 ? > > This is my current configuration> > > <property> > <name>hive.merge.mapfiles</name> > <value>true</value> > </property> > > <property> > <name>hive.merge.mapredfiles</name> > <value>true</value> > </property> > > <property> > <name>hive.merge.orcfile.stripe.level</name> > <value>true</value> > </property> > > However the output from a mapreduce job, which is stored into an orc file, > was not merged. This is the output> > > -rwxr-xr-x 1 root hdfs 0 2015-04-20 15:23 > /apps/hive/warehouse/coordinate/zone=2/_SUCCESS > -rwxr-xr-x 1 root hdfs 29072 2015-04-20 15:23 > /apps/hive/warehouse/coordinate/zone=2/part-r-00000 > -rwxr-xr-x 1 root hdfs 29049 2015-04-20 15:23 > /apps/hive/warehouse/coordinate/zone=2/part-r-00001 > -rwxr-xr-x 1 root hdfs 29075 2015-04-20 15:23 > /apps/hive/warehouse/coordinate/zone=2/part-r-00002 > > Any ideas? > > BR, > Patcharee >
