logs.gz <https://drive.google.com/file/d/0B-RFcYxUIHzzUVJpRzVDZXB5TUk/view?usp=drive_web> Hi Rajesh
PFA the gziped logs. FYI It's a single file, when you'll gunzip it, it'll be around 1.5gb in size. One more thing which you might find useful: In the dmOutputTez file i could see following line, which suggests that TEZ created a total of 7660 tasks. This is surprising as my data is only few mbs(10-15 mb max). How is this number of tasks decided? is there any property to tune it? 2015-07-07 05:37:02,647 [Timer-0] INFO org.apache.pig.backend.hadoop.executionengine.tez.TezJob - DAG Status: status=RUNNING, progress=TotalTasks: 7660 Succeeded: 0 Running: 0 Failed: 0 Killed: 0, diagnostics= Thanks On Mon, Jul 6, 2015 at 8:34 PM, Rajesh Balamohan <[email protected] > wrote: > yarn logs -applicationId application_1436152736518_0210 > > You can possibly send the output to a log file, gzip it and post it. > > ~Rajesh.B > > On Mon, Jul 6, 2015 at 8:12 PM, Sachin Sabbarwal < > [email protected]> wrote: > >> Hi >> Thanks for reply. My tez-site.xml contains only following: >> >> <configuration> >> <property> >> <name>tez.lib.uris</name> >> <value>${fs.defaultFS}/apps/tez-0.5/tez-0.5.3.tar.gz, >> ${fs.defaultFS}/apps/tez-0.5/*,${fs.defaultFS}/apps/tez-0.5/lib/*</value> >> </property> >> </configuration> >> >> PFA the application logs. Here is the version information: >> 1. Hadoop version: Hadoop 2.5.0-cdh5.3.1 >> 2. Pig: Apache Pig version 0.14.0 (r1640057) >> 3. Tez: 0.5.3 >> >> Lemme know if anything else is needed. >> >> Thanks in advance >> >> On Mon, Jul 6, 2015 at 7:07 PM, Rajesh Balamohan < >> [email protected]> wrote: >> >>> Can you post the application logs, tez-site.xml and also the version >>> details? >>> >>> ~Rajesh.B >>> >>> On Mon, Jul 6, 2015 at 6:38 PM, Sachin Sabbarwal < >>> [email protected]> wrote: >>> >>>> >>>> ---------- Forwarded message ---------- >>>> From: Sachin Sabbarwal <[email protected]> >>>> Date: Mon, Jul 6, 2015 at 5:34 PM >>>> Subject: Same pig script running slower with Tez as compared with run >>>> in Mapred mode >>>> To: [email protected] >>>> >>>> >>>> Hello Guys >>>> Trying Apache Tez. >>>> I've setup to use pig in TEZ mode. >>>> I'm running a pig script against i) no data and ii) with some data. >>>> In case i) when i run with pig using TEZ mode my pig scripts completes >>>> run in ~40secs. Whereas when i run case i) with mapred it takes around 7-8 >>>> mins. >>>> in case ii) when run with pig using TEZ, same pig script takes around >>>> 14-15 mins but with mapred it takes around 10 mins. >>>> When i'm running same pig script with production data(which is much >>>> more than the data i used here to run case i) and (ii) ) the job takes >>>> hours to complete. >>>> Hence I'm trying tez to run my pig job in a faster mode. I'm not really >>>> sure what i might be missing here. Please help, ask for any further info if >>>> required. >>>> >>>> >>>> Thanks >>>> -- >>>> Sachin Sabbarwal >>>> Linkedin: https://www.linkedin.com/profile?viewProfile=&key=95777265 >>>> Facebook: facebook.com/sachinsabbarwal >>>> Quora: http://www.quora.com/Sachin-Sabbarwal >>>> Blog: http://sachinsabbarwal.tumblr.com/ >>>> >>>> >>>> >>>> -- >>>> Sachin Sabbarwal >>>> Linkedin: https://www.linkedin.com/profile?viewProfile=&key=95777265 >>>> Facebook: facebook.com/sachinsabbarwal >>>> Quora: http://www.quora.com/Sachin-Sabbarwal >>>> Blog: http://sachinsabbarwal.tumblr.com/ >>>> >>> >>> >>> >>> -- >>> ~Rajesh.B >>> >> >> >> >> -- >> Sachin Sabbarwal >> Linkedin: https://www.linkedin.com/profile?viewProfile=&key=95777265 >> Facebook: facebook.com/sachinsabbarwal >> Quora: http://www.quora.com/Sachin-Sabbarwal >> Blog: http://sachinsabbarwal.tumblr.com/ >> > > > > -- > ~Rajesh.B > -- Sachin Sabbarwal Linkedin: https://www.linkedin.com/profile?viewProfile=&key=95777265 Facebook: facebook.com/sachinsabbarwal Quora: http://www.quora.com/Sachin-Sabbarwal Blog: http://sachinsabbarwal.tumblr.com/
