When you run pig in local mode "pig -x local" the data will be in local fs path ej. "/home/user/data" not in hdfs fs!. You can use hadoop fs -put hdfs://local/data/path /local/data/path to copy your hdfs to your local data path
El sáb., 8 de agosto de 2015 7:33 PM, vishal donderia < [email protected]> escribió: > Hi, > > Pig is a power tool to run map reduces job but i want to run pig in local > mode. > The same pig works fine in mapreduce mode and same pig runs well in local > mode also when data is present. > > But it fails when data is not present in local mode but run well in > mapreduce mode. > > When data is not present i have configured path of empty dir like shown > below > > > A= load ' > > *hdfs://localhost:9000/user/vishaldonderia/data/DataModelsLocalMode/ProcessDataModelsAction/tmp/1388538000/empty'* > USING PigStorage(',') > > *Failed Jobs:* > *JobId Alias Feature Message Outputs* > *N/A > > data_records,message_records,msc_voice_data_without_fwding_records,out_roamer_trip_model_domestic_usage_data,trip_model_msc_message_data,trip_model_msc_voice_data,trip_model_sgsn_data,voice_records > MAP_ONLY Message: org.apache.pig.backend.executionengine.ExecException: > ERROR 2118: Input path does not exist: > file:/tmp/temp616730105/tmp1274352221* > * at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:275)* > * at > org.apache.hadoop.mapred.JobClient.writeNewSplits(JobClient.java:1054)* > * at org.apache.hadoop.mapred.JobClient.writeSplits(JobClient.java:1071)* > * at org.apache.hadoop.mapred.JobClient.access$700(JobClient.java:179)* > * at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:983)* > * at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)* > * at java.security.AccessController.doPrivileged(Native Method)* > * at javax.security.auth.Subject.doAs(Subject.java:415)* > * at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)* > * at > org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:936)* > * at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:910)* > * at org.apache.hadoop.mapred.jobcontrol.Job.submit(Job.java:378)* > * at > > org.apache.hadoop.mapred.jobcontrol.JobControl.startReadyJobs(JobControl.java:247)* > * at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)* > * at > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)* > * at > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)* > * at java.lang.reflect.Method.invoke(Method.java:606)* > * at > > org.apache.pig.backend.hadoop20.PigJobControl.mainLoopAction(PigJobControl.java:157)* > * at > org.apache.pig.backend.hadoop20.PigJobControl.run(PigJobControl.java:134)* > * at java.lang.Thread.run(Thread.java:745)* > * at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:279)* > *Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: > Input path does not exist: file:/tmp/temp616730105/tmp1274352221* > * at > > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:235)* > * at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigFileInputFormat.listStatus(PigFileInputFormat.java:37)* > * at > > org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:252)* > * at > > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:261)* > * ... 20 more* > * file:/tmp/temp616730105/tmp1115492154,* > > > i*nput(s):* > *Successfully read 0 records from: > > "hdfs://localhost:9000/user/vishaldonderia/data/DataModelsLocalMode/ProcessDataModelsAction/tmp/1388538000/empty"* > *Successfully read 0 records from: > > "hdfs://localhost:9000/user/vishaldonderia/data/DataModelsLocalMode/ProcessDataModelsAction/tmp/1388538000/empty"* > > *Output(s):* > *Failed to produce result in "file:/tmp/temp616730105/tmp1115492154"* > > *Counters:* > *Total records written : 0* > *Total bytes written : 0* > *Spillable Memory Manager spill count : 0* > *Total bags proactively spilled: 0* > *Total records proactively spilled: 0* > > *Job DAG:* > *job_local410195113_0001 -> null,* > *job_local1526910904_0002 -> null,* > *null* > > > There is Error in getSplits() function. > > Then i run pig in local mode in grunt line by line and i was surpised to > find out that there is one line > > D = Union A,B,C; > > I am able to dump all 3 alias A , B , C , its works fine but when i am try > to dump D it is giving the same error as above. > > All three alias have same schema, no casting problem , not permisson > problem in /tmp dir > > Note: same thing works fine in mapreduce mode. > > > Please Help > > -- > > Thanking You > With Regards > > Vishal Donderia > [email protected] > +91-9711556310 >
