Is the job starting and getting stuck in the reducer like you mentioned in the initial mail? or the job itself is not starting?
~Rajesh.B On Wed, Jan 6, 2016 at 1:47 PM, Mich Talebzadeh <m...@peridale.co.uk> wrote: > Hi, > > > > Thanks for your help. I downloaded and installed snappy libraries as they > were missing. > > > > Setting Hive execution engine to tez and doing a simple query. Hive is > stuck > > > > > set hive.execution.engine=tez; > > 16/01/06 08:20:22 [main]: DEBUG parse.VariableSubstitution: Substitution > is on: tez > > > > > > > > In debug mode I get > > > > 16/01/06 08:22:07 [IPC Client (2116259755) connection to localhost/ > 127.0.0.1:8032 from hduser]: DEBUG ipc.Client: IPC Client (2116259755) > connection to localhost/127.0.0.1:8032 from hduser got value #188 > > 16/01/06 08:22:07 [main]: DEBUG ipc.ProtobufRpcEngine: Call: > getApplicationReport took 1ms > > 16/01/06 08:22:07 [LeaseRenewer:hduser@rhes564:9000]: DEBUG > hdfs.LeaseRenewer: Lease renewer daemon for [] with renew id 1 executed > > > > > > Which does not progress. Sounds like the same issue as I saw with Spark. > > > > Can I do something like? > > > > set tez.master=yarn-client; > > > > > > Thanks > > > > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > *Sybase ASE 15 Gold Medal Award 2008* > > A Winning Strategy: Running the most Critical Financial Data on ASE 15 > > > http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf > > Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE > 15", ISBN 978-0-9563693-0-7*. > > co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN > 978-0-9759693-0-4* > > *Publications due shortly:* > > *Complex Event Processing in Heterogeneous Environments*, ISBN: > 978-0-9563693-3-8 > > *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume > one out shortly > > > > http://talebzadehmich.wordpress.com > > > > NOTE: The information in this email is proprietary and confidential. This > message is for the designated recipient only, if you are not the intended > recipient, you should destroy it immediately. Any information in this > message shall not be understood as given or endorsed by Peridale Technology > Ltd, its subsidiaries or their employees, unless expressly so stated. It is > the responsibility of the recipient to ensure that this email is virus > free, therefore neither Peridale Ltd, its subsidiaries nor their employees > accept any responsibility. > > > > *From:* Rajesh Balamohan [mailto:rajesh.balamo...@gmail.com] > *Sent:* 05 January 2016 11:46 > > *To:* user@hive.apache.org > *Subject:* Re: Hive on TEZ fails starting > > > > Try ' beeline --hiveconf tez.task.launch.env=" > LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" > --hiveconf > tez.am.launch.env="LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native" > '. Please check if you have the lib*.so available in the native folder (or > point it to the folder which contains the so files) > > > > ~Rajesh.B > > > > > > On Tue, Jan 5, 2016 at 4:00 PM, Mich Talebzadeh <m...@peridale.co.uk> > wrote: > > Hi, > > > > I have added the following to the LD_LIBRARY_PATH and JAVA_LIBRARY_PATH > > > > export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native > > export JAVA_LIBRARY_PATH=$JAVA_LIBRARY_PATH:$HADOOP_COMMON_HOME/lib/native > > > > Trying to use TEZ, I still get the same error > > > > 0: jdbc:hive2://rhes564:10010/default> set hive.execution.engine=tez; > > No rows affected (0.002 seconds) > > 0: jdbc:hive2://rhes564:10010/default> use oraclehadoop; > > No rows affected (0.019 seconds) > > 0: jdbc:hive2://rhes564:10010/default> select count(1) from sales; > > INFO : Tez session hasn't been created yet. Opening session > > INFO : > > > > INFO : Status: Running (Executing on YARN cluster with App id > application_1451986680090_0002) > > > > INFO : Map 1: -/- Reducer 2: 0/1 > > INFO : Map 1: 0/1 Reducer 2: 0/1 > > INFO : Map 1: 0(+1)/1 Reducer 2: 0/1 > > INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1 > > INFO : Map 1: 0(+1,-1)/1 Reducer 2: 0/1 > > INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1 > > INFO : Map 1: 0(+1,-2)/1 Reducer 2: 0/1 > > INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1 > > INFO : Map 1: 0(+1,-3)/1 Reducer 2: 0/1 > > > > ERROR : Vertex failed, vertexName=Map 1, > vertexId=vertex_1451986680090_0002_1_00, diagnostics=[Task failed, > taskId=task_1451986680090_0002_1_00_000000, diagnostics=[TaskAttempt 0 > failed, info=[Error: Failure while running task:java.lang.RuntimeException: > java.lang.UnsatisfiedLinkError: > org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z > > > > > > > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEAAAAWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > *Sybase ASE 15 Gold Medal Award 2008* > > A Winning Strategy: Running the most Critical Financial Data on ASE 15 > > > http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908.pdf > > Author of the books* "A Practitioner’s Guide to Upgrading to Sybase ASE > 15", ISBN 978-0-9563693-0-7*. > > co-author *"Sybase Transact SQL Guidelines Best Practices", ISBN > 978-0-9759693-0-4* > > *Publications due shortly:* > > *Complex Event Processing in Heterogeneous Environments*, ISBN: > 978-0-9563693-3-8 > > *Oracle and Sybase, Concepts and Contrasts*, ISBN: 978-0-9563693-1-4, volume > one out shortly > > > > http://talebzadehmich.wordpress.com > > > > NOTE: The information in this email is proprietary and confidential. This > message is for the designated recipient only, if you are not the intended > recipient, you should destroy it immediately. Any information in this > message shall not be understood as given or endorsed by Peridale Technology > Ltd, its subsidiaries or their employees, unless expressly so stated. It is > the responsibility of the recipient to ensure that this email is virus > free, therefore neither Peridale Ltd, its subsidiaries nor their employees > accept any responsibility. > > > > *From:* Rajesh Balamohan [mailto:rajesh.balamo...@gmail.com] > *Sent:* 05 January 2016 00:35 > *To:* user@hive.apache.org > *Subject:* Re: Hive on TEZ fails starting > > > > By default it should add "LD_LIBRARY_PATH" in the container (ref: > https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L358) > . In case your snappy native libs are present elsewhere in the cluster > deployment, you can override using "tez.task.launch.env" and > "tez.am.launch.env" (refer: > https://github.com/apache/tez/blob/abfc8bfb0a8620d31697a31ad516674a8d3f9f7c/tez-api/src/main/java/org/apache/tez/dag/api/TezConfiguration.java#L412 > ) > > > > ~Rajesh.B > > > > > > > > > > -- > > ~Rajesh.B > -- ~Rajesh.B
