Chunky, You should have run: msck repair table <Table name>; Sorry, I should have made it clear in my last reply. I have added an entry to Hive wiki for benefit of others: https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-Recoverpartitions
Mark On Tue, Nov 6, 2012 at 9:55 PM, Chunky Gupta <chunky.gu...@vizury.com>wrote: > Hi Mark, > I didn't get any error. > I ran this on hive console:- > "msck table Table_Name;" > It says Ok and showed the execution time as 1.050 sec. > But when I checked partitions for table using > "show partitions Table_Name;" > It didn't show me any partitions. > > Thanks, > Chunky. > > > On Tue, Nov 6, 2012 at 10:38 PM, Mark Grover > <grover.markgro...@gmail.com>wrote: > >> Glad to hear, Chunky. >> >> Out of curiosity, what errors did you get when using msck? >> >> >> On Tue, Nov 6, 2012 at 5:14 AM, Chunky Gupta <chunky.gu...@vizury.com>wrote: >> >>> Hi Mark, >>> I tried msck, but it is not working for me. I have written a python >>> script to partition the data individually. >>> >>> Thank you Edward, Mark and Dean. >>> Chunky. >>> >>> >>> On Mon, Nov 5, 2012 at 11:08 PM, Mark Grover < >>> grover.markgro...@gmail.com> wrote: >>> >>>> Chunky, >>>> I have used "recover partitions" command on EMR, and that worked fine. >>>> >>>> However, take a look at https://issues.apache.org/jira/browse/HIVE-874. >>>> Seems >>>> like msck command in Apache Hive does the same thing. Try it out and let us >>>> know it goes. >>>> >>>> Mark >>>> >>>> On Mon, Nov 5, 2012 at 7:56 AM, Edward Capriolo >>>> <edlinuxg...@gmail.com>wrote: >>>> >>>>> Recover partitions should work the same way for different file systems. >>>>> >>>>> Edward >>>>> >>>>> On Mon, Nov 5, 2012 at 9:33 AM, Dean Wampler >>>>> <dean.wamp...@thinkbiganalytics.com> wrote: >>>>> > Writing a script to add the external partitions individually is the >>>>> only way >>>>> > I know of. >>>>> > >>>>> > Sent from my rotary phone. >>>>> > >>>>> > >>>>> > On Nov 5, 2012, at 8:19 AM, Chunky Gupta <chunky.gu...@vizury.com> >>>>> wrote: >>>>> > >>>>> > Hi Dean, >>>>> > >>>>> > Actually I was having Hadoop and Hive cluster on EMR and I have S3 >>>>> storage >>>>> > containing logs which updates daily and having partition with >>>>> date(dt). And >>>>> > I was using this recover partition. >>>>> > Now I wanted to shift to EC2 and have my own Hadoop and Hive >>>>> cluster. So, >>>>> > what is the alternate of using recover partition in this case, if >>>>> you have >>>>> > any idea ? >>>>> > I found one way of individually partitioning all dates, so I have to >>>>> write >>>>> > script for that to do so for all dates. Is there any easiest way >>>>> other than >>>>> > this ? >>>>> > >>>>> > Thanks, >>>>> > Chunky >>>>> > >>>>> > >>>>> > >>>>> > On Mon, Nov 5, 2012 at 6:28 PM, Dean Wampler >>>>> > <dean.wamp...@thinkbiganalytics.com> wrote: >>>>> >> >>>>> >> The RECOVER PARTITIONS is an enhancement added by Amazon to their >>>>> version >>>>> >> of Hive. >>>>> >> >>>>> >> >>>>> >> >>>>> http://docs.amazonwebservices.com/ElasticMapReduce/latest/DeveloperGuide/emr-hive-additional-features.html >>>>> >> >>>>> >> <shameless-plus> >>>>> >> Chapter 21 of Programming Hive discusses this feature and other >>>>> aspects >>>>> >> of using Hive in EMR. >>>>> >> </shameless-plug> >>>>> >> >>>>> >> dean >>>>> >> >>>>> >> >>>>> >> On Mon, Nov 5, 2012 at 5:34 AM, Chunky Gupta < >>>>> chunky.gu...@vizury.com> >>>>> >> wrote: >>>>> >>> >>>>> >>> Hi, >>>>> >>> >>>>> >>> I am having a cluster setup on EC2 with Hadoop version 0.20.2 and >>>>> Hive >>>>> >>> version 0.8.1 (I configured everything) . I have created a table >>>>> using :- >>>>> >>> >>>>> >>> CREATE EXTERNAL TABLE XXX ( YYY )PARTITIONED BY ( ZZZ )ROW FORMAT >>>>> >>> DELIMITED FIELDS TERMINATED BY 'WWW' LOCATION >>>>> 's3://my-location/data/'; >>>>> >>> >>>>> >>> Now I am trying to recover partition using :- >>>>> >>> >>>>> >>> ALTER TABLE XXX RECOVER PARTITIONS; >>>>> >>> >>>>> >>> but I am getting this error :- "FAILED: Parse Error: line 1:12 >>>>> cannot >>>>> >>> recognize input near 'XXX' 'RECOVER' 'PARTITIONS' in alter table >>>>> statement" >>>>> >>> >>>>> >>> Doing same steps on a cluster setup on EMR with Hadoop version >>>>> 1.0.3 and >>>>> >>> Hive version 0.8.1 (Configured by EMR), works fine. >>>>> >>> >>>>> >>> So is this a version issue or am I missing some configuration >>>>> changes in >>>>> >>> EC2 setup ? >>>>> >>> I am not able to find exact solution for this problem on internet. >>>>> Please >>>>> >>> help me. >>>>> >>> >>>>> >>> Thanks, >>>>> >>> Chunky. >>>>> >>> >>>>> >>> >>>>> >>> >>>>> >> >>>>> >> >>>>> >> >>>>> >> -- >>>>> >> Dean Wampler, Ph.D. >>>>> >> thinkbiganalytics.com >>>>> >> +1-312-339-1330 >>>>> >> >>>>> >> >>>>> > >>>>> >>>> >>>> >>> >> >
