Thanks I am trying to use TEZ as execution engine for hive instead of MR.
I have Hive 1.2.1 on Spark 1.3.1 working fine. That is the only version of Spark that works as execution engine for Hive (not the latest 1.5.1 I am afraid). The problem is that the performance is better than Hive on MR when I use Spark but otherwise the performance on joining 1 billion rows with another table takes 17 minutes compared to the same join in Sybase IQ DW that takes 30 seconds! The reason is attributed to Spark 1.3.1 not having full optimisation capabilities as it is older version So I would like to try Hive on TEZ which basically is MR on DAG. Hopefully that may give a better performance. Mich Talebzadeh Sybase ASE 15 Gold Medal Award 2008 A Winning Strategy: Running the most Critical Financial Data on ASE 15 <http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908 .pdf> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908. pdf Author of the books "A Practitioner's Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7. co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4 Publications due shortly: Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8 Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly <http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility. From: [email protected] [mailto:[email protected]] On Behalf Of Bikas Saha Sent: 31 December 2015 01:21 To: [email protected] Subject: RE: Getting error compiling TEZ 0.7 Also, if 0.7.1 turns out to be a binary release (as currently planned) then you probably wont have to build Tez from scratch after that. From: Jianfeng (Jeff) Zhang [mailto:[email protected]] Sent: Wednesday, December 30, 2015 4:53 PM To: [email protected] <mailto:[email protected]> Subject: Re: Getting error compiling TEZ 0.7 This is a known issue which is fixed in 0.7.1, please try to use maven 3.3+ https://issues.apache.org/jira/browse/TEZ-2560 Best Regard, Jeff Zhang From: Mich Talebzadeh <[email protected] <mailto:[email protected]> > Reply-To: "[email protected] <mailto:[email protected]> " <[email protected] <mailto:[email protected]> > Date: Thursday, December 31, 2015 at 4:54 AM To: "[email protected] <mailto:[email protected]> " <[email protected] <mailto:[email protected]> > Subject: Getting error compiling TEZ 0.7 Hi, Trying to build TEZ 0.7 in /usr/lib/apache-tez-0.7.0-src using mvn -X clean package -DskipTests=true -Dmaven.javadoc.skip=true with mvn version 3.2.5 (as opposed to 3.3) as I read that I can build it OK with 3.2.5 following the same error ass below mvn --version Apache Maven 3.2.5 (12a6b3acb947671f09b81f49094c53f426d8cea1; 2014-12-14T17:29:23+00:00) Maven home: /usr/local/apache-maven/apache-maven-3.2.5 Java version: 1.7.0_25, vendor: Oracle Corporation Java home: /usr/java/jdk1.7.0_25/jre I get this error [INFO] tez-ui ............................................. FAILURE [ 0.411 s] [ DEBUG] -- end configuration -- [INFO] Running 'npm install --color=false' in /usr/lib/apache-tez-0.7.0-src/tez-ui/src/main/webapp [INFO] /usr/lib/apache-tez-0.7.0-src/tez-ui/src/main/webapp/node/with_new_path.sh: line 3: 23781 Aborted "$@" [ERROR] Failed to execute goal com.github.eirslett:frontend-maven-plugin:0.0.16:npm (npm install) on project tez-ui: Failed to run task: 'npm install --color=false' failed. (error code 134) -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal com.github.eirslett:frontend-maven-plugin:0.0.16:npm (npm install) on project tez-ui: Failed to run task any ideas as there is little info available in net. Thanks Mich Talebzadeh Sybase ASE 15 Gold Medal Award 2008 A Winning Strategy: Running the most Critical Financial Data on ASE 15 <http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908 .pdf> http://login.sybase.com/files/Product_Overviews/ASE-Winning-Strategy-091908. pdf Author of the books "A Practitioner's Guide to Upgrading to Sybase ASE 15", ISBN 978-0-9563693-0-7. co-author "Sybase Transact SQL Guidelines Best Practices", ISBN 978-0-9759693-0-4 Publications due shortly: Complex Event Processing in Heterogeneous Environments, ISBN: 978-0-9563693-3-8 Oracle and Sybase, Concepts and Contrasts, ISBN: 978-0-9563693-1-4, volume one out shortly <http://talebzadehmich.wordpress.com/> http://talebzadehmich.wordpress.com NOTE: The information in this email is proprietary and confidential. This message is for the designated recipient only, if you are not the intended recipient, you should destroy it immediately. Any information in this message shall not be understood as given or endorsed by Peridale Technology Ltd, its subsidiaries or their employees, unless expressly so stated. It is the responsibility of the recipient to ensure that this email is virus free, therefore neither Peridale Ltd, its subsidiaries nor their employees accept any responsibility.
