How to enable compaction for table with external data?

2015-09-14 Thread Sachin Pasalkar
Hi, We are writing direct orc file from storm topology instead of using hive streaming (Due to performance issue with our data). However, we want to compact the data. So we have added the "NO_AUTO_COMPACTION"=“false” option in table which we created to read data(1.6 GB scattered in multiple sm

Re: How to enable compaction for table with external data?

2015-09-15 Thread Alan Gates
If you want it to compact automatically you should not put NO_AUTO_COMPACTION in the table properties. First question, did you turn on the compactor on your metastore thrift server? To do this you need to set a couple of values in the metastore's hive-site.xml: hive.compactor.initiator.on=t

Re: How to enable compaction for table with external data?

2015-09-15 Thread Sachin Pasalkar
ot;dev@hive.apache.org<mailto:dev@hive.apache.org>" mailto:dev@hive.apache.org>> Subject: Re: How to enable compaction for table with external data? If you want it to compact automatically you should not put NO_AUTO_COMPACTION in the table properties. First question, did you turn on the

Re: How to enable compaction for table with external data?

2015-09-25 Thread Alan Gates
achin From: Alan Gates mailto:alanfga...@gmail.com>> Reply-To: "dev@hive.apache.org <mailto:dev@hive.apache.org>" mailto:dev@hive.apache.org>> Date: Tuesday, 15 September 2015 10:30 pm To: "dev@hive.apache.org <mailto:dev@hive.apache.org>" mailto:dev@hive.