Hi, I try to use CarbonThriftServer and Beeline to query, but failed. 1. CarbonThriftServer run and exit with FINISHED state soon.
Submit script: spark-submit --queue spark --conf spark.sql.hive.thriftServer.singleSession=true --class org.apache.carbondata.spark.thriftserver.CarbonThriftServer --jars /usr/hdp/current/spark-client/lib/datanucleus-api-jdo-3.2.6.jar,/usr/hdp/current/spark-client/lib/datanucleus-rdbms-3.2.9.jar,/usr/hdp/current/spark-client/lib/datanucleus-core-3.2.10.jar /usr/hdp/2.5.0.0-1245/spark/carbonlib/carbondata_2.10-0.2.1-incubating-SNAPSHOT-shade-hadoop2.7.3.jar hdfs://julong/carbondata/carbonstore Here is the log: 17/01/16 09:49:36 INFO ContainerManagementProtocolProxy: Opening proxy : dpnode08:45454 17/01/16 09:49:36 INFO ContainerManagementProtocolProxy: Opening proxy : dpnode05:45454 17/01/16 09:49:40 INFO YarnClusterSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (dpnode08:48441) with ID 1 17/01/16 09:49:40 INFO BlockManagerMasterEndpoint: Registering block manager dpnode08:39271 with 511.5 MB RAM, BlockManagerId(1, dpnode08, 39271) 17/01/16 09:49:46 INFO YarnClusterSchedulerBackend: Registered executor NettyRpcEndpointRef(null) (dpnode05:35569) with ID 2 17/01/16 09:49:46 INFO YarnClusterSchedulerBackend: SchedulerBackend is ready for scheduling beginning after reached minRegisteredResourcesRatio: 0.8 17/01/16 09:49:46 INFO YarnClusterScheduler: YarnClusterScheduler.postStartHook done 17/01/16 09:49:46 INFO BlockManagerMasterEndpoint: Registering block manager dpnode05:41205 with 511.5 MB RAM, BlockManagerId(2, dpnode05, 41205) 17/01/16 09:49:46 INFO CarbonProperties: Driver Property file path: /usr/hdp/2.5.0.0-1245/spark/conf/carbon.properties 17/01/16 09:49:46 INFO CarbonProperties: Driver ------Using Carbon.properties -------- 17/01/16 09:49:46 INFO CarbonProperties: Driver {carbon.number.of.cores.while.loading=6, carbon.number.of.cores.while.compacting=4, carbon.sort.file.buffer.size=20, carbon.inmemory.record.size=120000, carbon.sort.size=500000, carbon.graph.rowset.size=100000, carbon.ddl.base.hdfs.url=/user/spark, carbon.compaction.level.threshold=8,6, carbon.number.of.cores=4, carbon.kettle.home=/usr/hdp/2.5.0.0-1245/spark/carbonlib/carbonplugins, carbon.storelocation=hdfs://julong/carbondata/carbonstore, carbon.enable.auto.load.merge=true, carbon.enableXXHash=true, carbon.sort.intermediate.files.limit=100, carbon.major.compaction.size=1024, carbon.badRecords.location=/opt/Carbon/Spark/badrecords, carbon.use.local.dir=true, carbon.enable.quick.filter=false} 17/01/16 09:49:52 INFO CarbonContext: Initializing execution hive, version 1.2.1 17/01/16 09:49:52 INFO ClientWrapper: Inspected Hadoop version: 2.7.3.2.5.0.0-1245 17/01/16 09:49:52 INFO ClientWrapper: Loaded org.apache.hadoop.hive.shims.Hadoop23Shims for Hadoop version 2.7.3.2.5.0.0-1245 17/01/16 09:49:53 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 17/01/16 09:49:53 INFO ObjectStore: ObjectStore, initialize called 17/01/16 09:49:53 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored 17/01/16 09:49:53 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored 17/01/16 09:50:01 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 17/01/16 09:50:02 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 17/01/16 09:50:02 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 17/01/16 09:50:09 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 17/01/16 09:50:09 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 17/01/16 09:50:11 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 17/01/16 09:50:11 INFO ObjectStore: Initialized ObjectStore 17/01/16 09:50:11 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0 17/01/16 09:50:12 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException 17/01/16 09:50:12 INFO HiveMetaStore: Added admin role in metastore 17/01/16 09:50:12 INFO HiveMetaStore: Added public role in metastore 17/01/16 09:50:12 INFO HiveMetaStore: No user is added in admin role, since config is empty 17/01/16 09:50:13 INFO HiveMetaStore: 0: get_all_databases 17/01/16 09:50:13 INFO audit: ugi=spark ip=unknown-ip-addr cmd=get_all_databases 17/01/16 09:50:13 INFO HiveMetaStore: 0: get_functions: db=default pat=* 17/01/16 09:50:13 INFO audit: ugi=spark ip=unknown-ip-addr cmd=get_functions: db=default pat=* 17/01/16 09:50:13 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table. 17/01/16 09:50:14 INFO SessionState: Created local directory: /data06/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/container_e26_1484211706075_0020_01_000001/tmp/yarn 17/01/16 09:50:14 INFO SessionState: Created local directory: /data06/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/container_e26_1484211706075_0020_01_000001/tmp/32b000cb-dbec-4399-9edd-3a0872b5942b_resources 17/01/16 09:50:14 INFO SessionState: Created HDFS directory: /tmp/hive/spark/32b000cb-dbec-4399-9edd-3a0872b5942b 17/01/16 09:50:14 INFO SessionState: Created local directory: /data06/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/container_e26_1484211706075_0020_01_000001/tmp/yarn/32b000cb-dbec-4399-9edd-3a0872b5942b 17/01/16 09:50:14 INFO SessionState: Created HDFS directory: /tmp/hive/spark/32b000cb-dbec-4399-9edd-3a0872b5942b/_tmp_space.db 17/01/16 09:50:14 INFO CarbonContext: default warehouse location is /user/hive/warehouse 17/01/16 09:50:14 INFO CarbonContext: Initializing HiveMetastoreConnection version 1.2.1 using Spark classes. 17/01/16 09:50:14 INFO ClientWrapper: Inspected Hadoop version: 2.7.3.2.5.0.0-1245 17/01/16 09:50:14 INFO ClientWrapper: Loaded org.apache.hadoop.hive.shims.Hadoop23Shims for Hadoop version 2.7.3.2.5.0.0-1245 17/01/16 09:50:15 INFO HiveMetaStore: 0: Opening raw store with implemenation class:org.apache.hadoop.hive.metastore.ObjectStore 17/01/16 09:50:15 INFO ObjectStore: ObjectStore, initialize called 17/01/16 09:50:15 INFO Persistence: Property datanucleus.cache.level2 unknown - will be ignored 17/01/16 09:50:15 INFO Persistence: Property hive.metastore.integral.jdo.pushdown unknown - will be ignored 17/01/16 09:50:23 INFO ObjectStore: Setting MetaStore object pin classes with hive.metastore.cache.pinobjtypes="Table,StorageDescriptor,SerDeInfo,Partition,Database,Type,FieldSchema,Order" 17/01/16 09:50:25 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 17/01/16 09:50:25 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 17/01/16 09:50:32 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MFieldSchema" is tagged as "embedded-only" so does not have its own datastore table. 17/01/16 09:50:32 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MOrder" is tagged as "embedded-only" so does not have its own datastore table. 17/01/16 09:50:34 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 17/01/16 09:50:34 INFO ObjectStore: Initialized ObjectStore 17/01/16 09:50:34 WARN ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version 1.2.0 17/01/16 09:50:35 WARN ObjectStore: Failed to get database default, returning NoSuchObjectException 17/01/16 09:50:35 INFO HiveMetaStore: Added admin role in metastore 17/01/16 09:50:35 INFO HiveMetaStore: Added public role in metastore 17/01/16 09:50:35 INFO HiveMetaStore: No user is added in admin role, since config is empty 17/01/16 09:50:36 INFO HiveMetaStore: 0: get_all_databases 17/01/16 09:50:36 INFO audit: ugi=spark ip=unknown-ip-addr cmd=get_all_databases 17/01/16 09:50:36 INFO HiveMetaStore: 0: get_functions: db=default pat=* 17/01/16 09:50:36 INFO audit: ugi=spark ip=unknown-ip-addr cmd=get_functions: db=default pat=* 17/01/16 09:50:36 INFO Datastore: The class "org.apache.hadoop.hive.metastore.model.MResourceUri" is tagged as "embedded-only" so does not have its own datastore table. 17/01/16 09:50:37 INFO SessionState: Created local directory: /data06/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/container_e26_1484211706075_0020_01_000001/tmp/cf5abd18-5b31-483f-89be-8e4ae65a0cb7_resources 17/01/16 09:50:37 INFO SessionState: Created HDFS directory: /tmp/hive/spark/cf5abd18-5b31-483f-89be-8e4ae65a0cb7 17/01/16 09:50:37 INFO SessionState: Created local directory: /data06/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/container_e26_1484211706075_0020_01_000001/tmp/yarn/cf5abd18-5b31-483f-89be-8e4ae65a0cb7 17/01/16 09:50:37 INFO SessionState: Created HDFS directory: /tmp/hive/spark/cf5abd18-5b31-483f-89be-8e4ae65a0cb7/_tmp_space.db 17/01/16 09:50:41 INFO CompositeService: Operation log root directory is created: /data06/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/container_e26_1484211706075_0020_01_000001/tmp/yarn/operation_logs 17/01/16 09:50:42 INFO AbstractService: HiveServer2: Async execution pool size 100 17/01/16 09:50:42 INFO AbstractService: Service:OperationManager is inited. 17/01/16 09:50:42 INFO AbstractService: Service: SessionManager is inited. 17/01/16 09:50:42 INFO AbstractService: Service: CLIService is inited. 17/01/16 09:50:42 INFO AbstractService: Service:ThriftBinaryCLIService is inited. 17/01/16 09:50:42 INFO AbstractService: Service: HiveServer2 is inited. 17/01/16 09:50:42 INFO AbstractService: Service:OperationManager is started. 17/01/16 09:50:42 INFO AbstractService: Service:SessionManager is started. 17/01/16 09:50:42 INFO AbstractService: Service:CLIService is started. 17/01/16 09:50:42 INFO ObjectStore: ObjectStore, initialize called 17/01/16 09:50:42 INFO Query: Reading in results for query "org.datanucleus.store.rdbms.query.SQLQuery@0" since the connection used is closing 17/01/16 09:50:42 INFO MetaStoreDirectSql: Using direct SQL, underlying DB is DERBY 17/01/16 09:50:42 INFO ObjectStore: Initialized ObjectStore 17/01/16 09:50:42 INFO HiveMetaStore: 0: get_databases: default 17/01/16 09:50:42 INFO audit: ugi=spark ip=unknown-ip-addr cmd=get_databases: default 17/01/16 09:50:42 INFO HiveMetaStore: 0: Shutting down the object store... 17/01/16 09:50:42 INFO audit: ugi=spark ip=unknown-ip-addr cmd=Shutting down the object store... 17/01/16 09:50:42 INFO HiveMetaStore: 0: Metastore shutdown complete. 17/01/16 09:50:42 INFO audit: ugi=spark ip=unknown-ip-addr cmd=Metastore shutdown complete. 17/01/16 09:50:42 INFO AbstractService: Service:ThriftBinaryCLIService is started. 17/01/16 09:50:42 INFO AbstractService: Service:HiveServer2 is started. 17/01/16 09:50:42 INFO ApplicationMaster: Final app status: SUCCEEDED, exitCode: 0 17/01/16 09:50:42 INFO ThriftCLIService: Starting ThriftBinaryCLIService on port 10000 with 5...500 worker threads 17/01/16 09:50:42 INFO SparkContext: Invoking stop() from shutdown hook 17/01/16 09:50:42 INFO HiveServer2: Shutting down HiveServer2 17/01/16 09:50:42 INFO ThriftCLIService: Thrift server has stopped 17/01/16 09:50:42 INFO AbstractService: Service:ThriftBinaryCLIService is stopped. 17/01/16 09:50:42 INFO AbstractService: Service:OperationManager is stopped. 17/01/16 09:50:42 INFO AbstractService: Service:SessionManager is stopped. 17/01/16 09:50:42 INFO AbstractService: Service:CLIService is stopped. 17/01/16 09:50:42 INFO AbstractService: Service:HiveServer2 is stopped. 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/sqlserver/session/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/sqlserver/session,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/sqlserver/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/sqlserver,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/static/sql,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/execution/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/execution,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/SQL,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/metrics/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage/kill,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/api,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/static,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/threadDump/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/threadDump,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/executors,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/environment/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/environment,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/rdd/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/rdd,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/storage,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/pool/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/pool,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/stage,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/stages,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/job/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/job,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs/json,null} 17/01/16 09:50:42 INFO ContextHandler: stopped o.s.j.s.ServletContextHandler{/jobs,null} 17/01/16 09:50:42 INFO SparkUI: Stopped Spark web UI at http://192.168.50.8:36579 17/01/16 09:50:42 INFO YarnAllocator: Driver requested a total number of 0 executor(s). 17/01/16 09:50:42 INFO YarnClusterSchedulerBackend: Shutting down all executors 17/01/16 09:50:42 INFO YarnClusterSchedulerBackend: Asking each executor to shut down 17/01/16 09:50:42 INFO SchedulerExtensionServices: Stopping SchedulerExtensionServices (serviceOption=None, services=List(), started=false) 17/01/16 09:50:42 INFO MapOutputTrackerMasterEndpoint: MapOutputTrackerMasterEndpoint stopped! 17/01/16 09:50:42 INFO MemoryStore: MemoryStore cleared 17/01/16 09:50:42 INFO BlockManager: BlockManager stopped 17/01/16 09:50:42 INFO BlockManagerMaster: BlockManagerMaster stopped 17/01/16 09:50:42 INFO OutputCommitCoordinator$OutputCommitCoordinatorEndpoint: OutputCommitCoordinator stopped! 17/01/16 09:50:42 INFO SparkContext: Successfully stopped SparkContext 17/01/16 09:50:42 INFO ApplicationMaster: Unregistering ApplicationMaster with SUCCEEDED 17/01/16 09:50:42 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down remote daemon. 17/01/16 09:50:42 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon shut down; proceeding with flushing remote transports. 17/01/16 09:50:42 INFO AMRMClientImpl: Waiting for application to be successfully unregistered. 17/01/16 09:50:42 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut down. 17/01/16 09:50:42 INFO ApplicationMaster: Deleting staging directory .sparkStaging/application_1484211706075_0020 17/01/16 09:50:42 INFO ShutdownHookManager: Shutdown hook called 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data11/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-45b2767d-5201-4af8-8ba8-93a1130e4a6a 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data06/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/container_e26_1484211706075_0020_01_000001/tmp/spark-c1a03ca3-8dd5-466a-8d1e-2f3bc7280bdc 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data08/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-208e5903-abed-494c-879b-e2d79764c1b9 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data09/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-19e56c57-f2b8-4c40-94a3-d230f8ee0ff9 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data03/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-6fbf013a-cd00-49ae-a43a-6237331a615b 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data04/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-44650c84-ac7f-4908-b038-e16f6cdc59cd 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data02/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-2c200b2c-0f2b-45e2-a305-977b99ed94a3 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data12/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-582b6acc-a159-49ff-832c-472aacae8894 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data05/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-86a35020-0827-4eb4-b40c-d7e708ab62f5 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data06/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-209e2803-4190-4366-9f35-3d2e21e2ff48 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data07/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-1ae5a1f4-18c9-4de9-94bf-df6aaa693df5 17/01/16 09:50:42 INFO ShutdownHookManager: Deleting directory /data10/hadoop/yarn/local/usercache/spark/appcache/application_1484211706075_0020/spark-b78c9877-6007-400d-b067-3e55d969542c 2. Beeline client can not query carbon data, and create carbon table. 0: jdbc:hive2://dpnode03:10000> select * from sale limit 10; +-----------+--+ | sale.col | +-----------+--+ +-----------+--+ 0: jdbc:hive2://dpnode03:10000> create table info (`name` string, `age` int) stored by 'carbondata'; Error: Error while compiling statement: FAILED: SemanticException Cannot find class 'carbondata' (state=42000,code=40000) Thanks. -- View this message in context: http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/unable-to-use-CarbonThriftServer-and-Beeline-client-tp6252.html Sent from the Apache CarbonData Mailing List archive mailing list archive at Nabble.com.