Hi,
I am running pig embeeded in java which uses HCatLoader to query from hive
table.I am doing this by creating an PigServer object in mapreduce mode.I
have added the necessary jars but when i run the program HCatLoader is
unable to find the table though it is present.I think it is not able to
access desired hive meta store, which is accessed though hiveserver2.
Kindly help.
Following is the code snippet :
PigServerpig = HLPigUtil.GetPigInstance();
pig.setJobName("load RehabilitateSessionByPig");
String loaddataSrc1 = "data1 = LOAD '" + srcPath1 + "' USING
PigStorage('\\u0001') AS (sid:chararray,uid:chararray);";
pig.registerQuery(loaddataSrc1);
String loaddataSrc2 = "data2 = LOAD '" + db+"."+tbl + "' USING
org.apache.hive.hcatalog.pig.HCatLoader();";
pig.registerQuery(loaddataSrc2);
String joineddata = "joineddata = COGROUP datasrc1 BY (sid,uid), datasrc2
BY (sid,pid);";
pig.registerQuery(joineddata);
Here is the exception :
2015-10-21 14:14:28,788 INFO [main]
org.apache.hadoop.hive.metastore.HiveMetaStore: 0: Opening raw store
with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore
2015-10-21 14:14:28,819 INFO [main]
org.apache.hadoop.hive.metastore.ObjectStore: ObjectStore, initialize
called
2015-10-21 14:14:29,008 INFO [main] DataNucleus.Persistence:
Property datanucleus.cache.level2 unknown - will be ignored
2015-10-21 14:14:29,008 INFO [main] DataNucleus.Persistence:
Property hive.metastore.integral.jdo.pushdown unknown - will be
ignored
2015-10-21 14:14:29,181 WARN [main] DataNucleus.Connection: BoneCP
specified but not present in CLASSPATH (or one of dependencies)
2015-10-21 14:14:29,669 WARN [main] DataNucleus.Connection: BoneCP
specified but not present in CLASSPATH (or one of dependencies)
2015-10-21 14:14:36,768 INFO [main]
org.apache.hadoop.hive.metastore.ObjectStore: Setting MetaStore object
pin classes with
hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order"
2015-10-21 14:14:36,813 INFO [main]
org.apache.hadoop.hive.metastore.MetaStoreDirectSql: MySQL check
failed, assuming we are not on mysql: Lexical error at line 1, column
5. Encountered: "@" (64), after : "".
2015-10-21 14:14:38,181 INFO [main] DataNucleus.Datastore: The class
"org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
"embedded-only" so does not have its own datastore table.
2015-10-21 14:14:38,182 INFO [main] DataNucleus.Datastore: The class
"org.apache.hadoop.hive.metastore.model.MOrder" is tagged as
"embedded-only" so does not have its own datastore table.
2015-10-21 14:14:43,727 INFO [main] DataNucleus.Datastore: The class
"org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as
"embedded-only" so does not have its own datastore table.
2015-10-21 14:14:43,728 INFO [main] DataNucleus.Datastore: The class
"org.apache.hadoop.hive.metastore.model.MOrder" is tagged as
"embedded-only" so does not have its own datastore table.
2015-10-21 14:14:45,163 INFO [main]
org.apache.hadoop.hive.metastore.ObjectStore: Initialized ObjectStore
2015-10-21 14:14:45,467 WARN [main]
org.apache.hadoop.hive.metastore.ObjectStore: Version information not
found in metastore. hive.metastore.schema.verification is not enabled
so recording the schema version 0.13.0
2015-10-21 14:14:46,054 INFO [main]
org.apache.hadoop.hive.metastore.HiveMetaStore: Added admin role in
metastore
2015-10-21 14:14:46,063 INFO [main]
org.apache.hadoop.hive.metastore.HiveMetaStore: Added public role in
metastore
2015-10-21 14:14:46,380 INFO [main]
org.apache.hadoop.hive.metastore.HiveMetaStore: No user is added in
admin role, since config is empty
2015-10-21 14:14:46,464 INFO [main]
org.apache.hadoop.hive.metastore.HiveMetaStore: 0: get_databases:
NonExistentDatabaseUsedForHealthCheck
2015-10-21 14:14:46,464 INFO [main]
org.apache.hadoop.hive.metastore.HiveMetaStore.audit:
ugi=hadoop ip=unknown-ip-addr cmd=get_databases:
NonExistentDatabaseUsedForHealthCheck
2015-10-21 14:14:46,484 INFO [main]
org.apache.hadoop.hive.metastore.HiveMetaStore: 0: get_table : db=star
tbl=session
2015-10-21 14:14:46,485 INFO [main]
org.apache.hadoop.hive.metastore.HiveMetaStore.audit:
ugi=adoop ip=unknown-ip-addr cmd=get_table : db=star tbl=session
2015-10-21 14:14:46,505 ERROR [main] org.apache.pig.PigServer:
exception during parsing: Error during parsing. Table not found :
star.session table not found
Failed to parse: Can not retrieve schema from loader
org.apache.hive.hcatalog.pig.HCatLoader@6db13624
at org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:198)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1676)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1623)
at org.apache.pig.PigServer.registerQuery(PigServer.java:575)
at org.apache.pig.PigServer.registerQuery(PigServer.java:588)
