HCatSchema is in hcatalog-core.jar and it is missing in the backend. You
will need to register several jars in pig script:
hcatalog-core.jar
hive-metastore.jar
libthrift.jar
hive-core.jar
libfb303.jar
jdo*-api.jar
slf4j-api.jar
When you run in command line, bin/pig script does that for you. But in
eclipse, you will need to do it by yourself.
Daniel
On 3/3/15, 4:40 AM, "李运田" <cumt...@163.com> wrote:
>2015-03-03 19:54:43,072 INFO [main]
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Created MRAppMaster for
>application appattempt_1421806758143_0252_000001
>2015-03-03 19:54:43,623 WARN [main] org.apache.hadoop.conf.Configuration:
>job.xml:an attempt to override final parameter:
>mapreduce.job.end-notification.max.retry.interval; Ignoring.
>2015-03-03 19:54:43,650 WARN [main] org.apache.hadoop.conf.Configuration:
>job.xml:an attempt to override final parameter:
>mapreduce.job.end-notification.max.attempts; Ignoring.
>2015-03-03 19:54:43,778 WARN [main]
>org.apache.hadoop.util.NativeCodeLoader: Unable to load native-hadoop
>library for your platform... using builtin-java classes where applicable
>2015-03-03 19:54:43,798 INFO [main]
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Executing with tokens:
>2015-03-03 19:54:43,798 INFO [main]
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Kind: YARN_AM_RM_TOKEN,
>Service: , Ident:
>(org.apache.hadoop.yarn.security.AMRMTokenIdentifier@307c9a57)
>2015-03-03 19:54:43,833 INFO [main]
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster: The specific max
>attempts: 2 for application: 252. Attempt num: 1 is last retry: false
>2015-03-03 19:54:43,841 INFO [main]
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster: Using mapred
>newApiCommitter.
>2015-03-03 19:54:43,985 WARN [main] org.apache.hadoop.conf.Configuration:
>job.xml:an attempt to override final parameter:
>mapreduce.job.end-notification.max.retry.interval; Ignoring.
>2015-03-03 19:54:43,999 WARN [main] org.apache.hadoop.conf.Configuration:
>job.xml:an attempt to override final parameter:
>mapreduce.job.end-notification.max.attempts; Ignoring.
>2015-03-03 19:54:44,535 WARN [main]
>org.apache.hadoop.hdfs.BlockReaderLocal: The short-circuit local reads
>feature cannot be used because libhadoop cannot be loaded.
>2015-03-03 19:54:44,719 INFO [main]
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster: OutputCommitter set in
>config null
>2015-03-03 19:54:44,844 INFO [main]
>org.apache.hadoop.service.AbstractService: Service
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster failed in state INITED;
>cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
>java.io.IOException: Deserialization error:
>org.apache.hcatalog.data.schema.HCatSchema
>org.apache.hadoop.yarn.exceptions.YarnRuntimeException:
>java.io.IOException: Deserialization error:
>org.apache.hcatalog.data.schema.HCatSchema
> at
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRApp
>Master.java:473)
> at
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceInit(MRAppMaster.jav
>a:374)
> at
>org.apache.hadoop.service.AbstractService.init(AbstractService.java:163)
> at
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1456
>)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
>org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.
>java:1548)
> at
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRApp
>Master.java:1453)
> at
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1386)
>Caused by: java.io.IOException: Deserialization error:
>org.apache.hcatalog.data.schema.HCatSchema
> at
>org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.jav
>a:59)
> at org.apache.pig.impl.util.UDFContext.deserialize(UDFContext.java:192)
> at
>org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil.setupUDFCont
>ext(MapRedUtil.java:173)
> at
>org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputForm
>at.setupUdfEnvAndStores(PigOutputFormat.java:229)
> at
>org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputForm
>at.getOutputCommitter(PigOutputFormat.java:275)
> at
>org.apache.hadoop.mapreduce.v2.app.MRAppMaster.createOutputCommitter(MRApp
>Master.java:471)
> ... 8 more
>Caused by: java.lang.ClassNotFoundException:
>org.apache.hcatalog.data.schema.HCatSchema
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> at java.lang.Class.forName0(Native Method)
>
>
>
>
>
>
>
>在 2015-02-28 09:37:29，"李运田" <cumt...@163.com> 写道：
>
>pigServer.registerQuery("tmp = load 'pig' using
>org.apache.hcatalog.pig.HCatLoader();");
>pigServer.registerQuery("tmp = foreach tmp generate id;");
>when I execute this，I cant get any error
>but,When I execute
>pigServer.registerQuery("store tmp into 'hive' using
>org.apache.hcatalog.pig.HCatStorer();");
>I get error like this:::
>org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1066: Unable to
>open iterator for alias tmp
> at org.apache.pig.PigServer.openIterator(PigServer.java:880)
> at org.gradle.PigHiveHCat.run(PigHiveHCat.java:68)
> at org.gradle.PigHiveHCat.main(PigHiveHCat.java:28)
>Caused by: java.io.IOException: Job terminated with anomalous status
>FAILED
> at org.apache.pig.PigServer.openIterator(PigServer.java:872)
> ... 2 more
>..........................................................................
>.................................
>pigServer.registerQuery("a = LOAD '/user/hadoop/pig.txt' ;");
>pigServer.store("a", "/user/hadoop/pig1.txt");
>pigServer.registerQuery("store a into '/user/hadoop/pig2.txt';");
>........................................this is OK,so I think something
>about Hcatalog is wrong...........................
>
>
>At 2015-02-28 01:21:12, "Alan Gates" <alanfga...@gmail.com> wrote:
>What error message are you getting?
>
>Alan.
>
>
>李运田
>February 26, 2015 at 18:58
>I want to use hcatalog in eclipse to deal with tables in hive.
>but I cant store table into hive::
>pigServer.registerQuery("tmp = load 'pig' using
>org.apache.hcatalog.pig.HCatLoader();");
>pigServer.registerQuery("tmp = foreach tmp generate id;");
>pigServer.registerQuery("store tmp into 'hive' using
>org.apache.hcatalog.pig.HCatStorer();");
>I can store into file::
>pigServer.registerQuery("a = LOAD '/user/hadoop/pig.txt' ;");
>pigServer.store("a", "/user/hadoop/pig1.txt");
>pigServer.registerQuery("store a into '/user/hadoop/pig2.txt';");
>perhaps ,the hcatalog jars are wrong?
>
>
