Hi Tsuyoshi, I have a set of scripts to enable you to build hive-on-tez very quickly on an HDP2 Sandbox.
https://github.com/t3rmin4t0r/tez-autobuild The commands it uses to pull hive/tez & build it should be easy to emulate. Once you run your commands and finish a session (i.e close hive-shell). You can do yarn logs -applicationId <application_...> | grep HISTORY > history.log that can be processed into an image using this script https://github.com/t3rmin4t0r/tez-swimlanes This is a somewhat useful substitute for the web joh history view. You will end up with a view like this http://random.notmysock.org/query27.svg which tracks each container + vertex run within the task (& each block links to its log file in the history). I will reply in a couple of hours with the details on loading raw data into TPC-H/TPC-DS tables. I do most of the data loads into our partitioned table and I'm yet to completely replace myself by a simple shell script :) Cheers, Gopal On Fri, Dec 6, 2013 at 11:21 AM, Hitesh Shah <[email protected]> wrote: > Fwding to dev@ list as there are probably some folks who do not monitor > the user@ lists at the moment. > > -- Hitesh > > Begin forwarded message: > > > From: Tsuyoshi OZAWA <[email protected]> > > Date: December 5, 2013 11:33:43 AM PST > > To: [email protected] > > Subject: How to setup tez cluster for TPC-DS benchmark > > Reply-To: [email protected] > > > > Hi, > > > > I read the article about tez and stinger project. > > > > > http://hortonworks.com/blog/stinger-phase-2-the-journey-to-100x-faster-hive/ > > > > I'd like to run this benchmark with tez, but I don't know how to setup > > tez for it. Do you have any docs to do this? > > > > - Tsuyoshi > > -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
