Nope Daniel, That extra '/' isn't causing any trouble. -Harsha On Thu, Apr 30, 2015 at 7:03 AM, Daniel Haviv < daniel.ha...@veracity-group.com> wrote:
> I think you have an extra '/' in the HDFS URI > > Daniel > > On 30 באפר׳ 2015, at 16:46, Harsha N <harsha.hadoo...@gmail.com> wrote: > > Thanks for your reply, > > analyze table table1 partition (dt=201501) compute statistics;--returns > the same error > *Total jobs = 1* > *Launching Job 1 out of 1* > *Number of reduce tasks is set to 0 since there's no reduce operator* > *java.io.IOException: cannot find dir = > hdfs:///data/dt =201501/1430201400/* > *in pathToPartitionInfo: [hdfs:/data/dt=201501/1430201400/]* > * at > org.apache.hadoop.hive.ql.io.HiveFileFormatUtils.getPartitionDescFromPathRecursively(HiveFileFormatUtils.java:347)* > > To Add On > I am working on External tables in Hive 0.13.1-cdh5.3.2 > > -Harsha > > On Thu, Apr 30, 2015 at 12:37 AM, Mich Talebzadeh <m...@peridale.co.uk> > wrote: > >> Hi Harsha, >> >> >> >> Have you updated stats on table1 after partition adding? In other words >> it is possible that the optimiser is not aware of that partition yet? >> >> >> >> analyze table table1 partition (dt=201501) compute statistics; >> >> >> >> HTH >> >> >> >> Mich Talebzadeh >> >> >> >> http://talebzadehmich.wordpress.com >> >> >> >> Author of the books* "A Practitioner’s Guide to Upgrading to Sybase** >> ASE 15", **ISBN 978-0-9563693-0-7*. >> >> co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN >> 978-0-9759693-0-4* >> >> *Publications due shortly:* >> >> *Creating in-memory Data Grid for Trading Systems with Oracle TimesTen >> and Coherence Cache* >> >> *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume >> one out shortly >> >> >> >> NOTE: The information in this email is proprietary and confidential. This >> message is for the designated recipient only, if you are not the intended >> recipient, you should destroy it immediately. Any information in this >> message shall not be understood as given or endorsed by Peridale Ltd, its >> subsidiaries or their employees, unless expressly so stated. It is the >> responsibility of the recipient to ensure that this email is virus free, >> therefore neither Peridale Ltd, its subsidiaries nor their employees accept >> any responsibility. >> >> >> >> *From:* Harsha N [mailto:harsha.hadoo...@gmail.com] >> *Sent:* 30 April 2015 07:24 >> *To:* user@hive.apache.org >> *Subject:* Hive Alter Partition Location Issue >> >> >> >> Hi All, >> >> Can experts share your view on Hive behaviour in below scenario. >> >> >> >> I am facing below issue on using alter partition locations in hive. >> >> >> >> *select count(*) from table1 where dt = 201501;* >> >> >> >> *Total jobs = 1* >> >> *Launching Job 1 out of 1* >> >> *Number of reduce tasks determined at compile time: 1* >> >> *In order to change the average load for a reducer (in bytes):* >> >> * set hive.exec.reducers.bytes.per.reducer=<number>* >> >> *In order to limit the maximum number of reducers:* >> >> * set hive.exec.reducers.max=<number>* >> >> *In order to set a constant number of reducers:* >> >> * set mapreduce.job.reduces=<number>* >> >> *java.io.IOException: cannot find dir = >> hdfs:///data/dt =201501/1430201400/* >> >> *in pathToPartitionInfo: [hdfs:/data/dt=201501/1430201400/]* >> >> >> >> Below are the steps I have followed. >> >> I have altered a partition location in hive using below command. >> >> ALTER TABLE table1 PARTITION (dt=201501) SET LOCATION >> 'hdfs:///data/dt=201501/1430201400/'; >> >> >> >> I have inserted new data into this new location. >> >> >> >> INSERT INTO TABLE table1 >> >> SELECT * FROM table2 where dt=201501 >> >> >> >> *select count(*) from table1 where dt = 201501; doesn't work but * >> >> *select * from table1 where dt = 201501 works good.* >> >> >> >> Please let me know if you need more information. >> >> >> >> Thanks >> >> Harsha >> > >
