Andrew Guo created TAJO-655:
-------------------------------
Summary: QueryMaster sent "Select query" command to
TajoWorker,but the TajoWorker don't working
Key: TAJO-655
URL: https://issues.apache.org/jira/browse/TAJO-655
Project: Tajo
Issue Type: Bug
Components: query master, worker
Affects Versions: 0.8-incubating
Environment: CentOS 6.3 32bit,jdk 1.7,hadoop 2.2.0
Reporter: Andrew Guo
Fix For: 0.8-incubating
Hi
Yesterday ,You provide tajo branch-0.8.0 already installed. In the
http://xxx:26080 web ui,the TajaWorker is live .In the TSQL interface input
"select * from test" command,but Prompt "Internal error!" error.
TajoWorker error log content:
=======================================================
Block Id: eb_1393922377833_0005_000001 [ROOT]
=======================================================
SCAN(0) on test
=> target list: test.id (INT4), test.interface (TEXT), test.start_time
(TEXT), test.byte (TEXT)
=> out schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time
(TEXT),test.byte (TEXT)}
=> in schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time
(TEXT),test.byte (TEXT)}
=======================================================
Block Id: eb_1393922377833_0005_000002 [TERMINAL]
=======================================================
2014-03-05 10:48:18,219 INFO master.TajoAsyncDispatcher
(TajoAsyncDispatcher.java:start(101)) - AsyncDispatcher
started:q_1393922377833_0005
2014-03-05 10:48:18,222 INFO querymaster.Query (Query.java:handle(641)) -
Processing q_1393922377833_0005 of type START
2014-03-05 10:48:18,223 INFO querymaster.SubQuery
(SubQuery.java:initTaskScheduler(661)) -
org.apache.tajo.master.DefaultTaskScheduler is chosen for the task scheduling
2014-03-05 10:48:18,230 INFO storage.AbstractStorageManager
(AbstractStorageManager.java:listStatus(382)) - Total input paths to process : 1
2014-03-05 10:48:18,236 ERROR querymaster.SubQuery
(SubQuery.java:transition(645)) - SubQuery (eb_1393922377833_0005_000001)
ERROR:
java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
at
org.apache.tajo.storage.AbstractStorageManager.newScannerInstance(AbstractStorageManager.java:661)
at
org.apache.tajo.storage.StorageManager.getScanner(StorageManager.java:60)
at
org.apache.tajo.storage.AbstractStorageManager.getScanner(AbstractStorageManager.java:106)
at
org.apache.tajo.storage.AbstractStorageManager.getFileScanner(AbstractStorageManager.java:94)
at
org.apache.tajo.storage.AbstractStorageManager.isSplittable(AbstractStorageManager.java:399)
at
org.apache.tajo.storage.AbstractStorageManager.getSplits(AbstractStorageManager.java:554)
at
org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.scheduleFragmentsForLeafQuery(SubQuery.java:872)
at
org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.schedule(SubQuery.java:778)
at
org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.transition(SubQuery.java:629)
at
org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.transition(SubQuery.java:610)
at
org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at org.apache.tajo.master.querymaster.SubQuery.handle(SubQuery.java:587)
at
org.apache.tajo.master.querymaster.Query$StartTransition.transition(Query.java:332)
at
org.apache.tajo.master.querymaster.Query$StartTransition.transition(Query.java:321)
at
org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
at
org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
at
org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
at
org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
at org.apache.tajo.master.querymaster.Query.handle(Query.java:646)
at org.apache.tajo.master.querymaster.Query.handle(Query.java:58)
at
org.apache.tajo.master.TajoAsyncDispatcher.dispatch(TajoAsyncDispatcher.java:137)
at
org.apache.tajo.master.TajoAsyncDispatcher$1.run(TajoAsyncDispatcher.java:79)
at java.lang.Thread.run(Thread.java:744)
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
at
org.apache.tajo.storage.AbstractStorageManager.newScannerInstance(AbstractStorageManager.java:659)
... 25 more
Caused by: java.lang.IllegalArgumentException: Compression codec
com.hadoop.compression.lzo.LzoCodec not found.
at
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:135)
at
org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175)
at org.apache.tajo.storage.CSVFile$CSVScanner.<init>(CSVFile.java:245)
... 30 more
Caused by: java.lang.ClassNotFoundException: Class
com.hadoop.compression.lzo.LzoCodec not found
at
org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626)
at
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:128)
... 32 more
2014-03-05 10:48:18,237 INFO querymaster.Query (Query.java:handle(654)) -
q_1393922377833_0005 Query Transitioned from QUERY_NEW to QUERY_RUNNING
2014-03-05 10:48:18,238 INFO querymaster.Query (Query.java:handle(641)) -
Processing q_1393922377833_0005 of type SUBQUERY_COMPLETED
2014-03-05 10:48:18,238 INFO querymaster.Query (Query.java:handle(641)) -
Processing q_1393922377833_0005 of type QUERY_COMPLETED
2014-03-05 10:48:18,238 INFO querymaster.Query (Query.java:handle(654)) -
q_1393922377833_0005 Query Transitioned from QUERY_RUNNING to QUERY_ERROR
2014-03-05 10:48:18,238 INFO querymaster.QueryMasterTask
(QueryMasterTask.java:handle(266)) - Query completion notified from
q_1393922377833_0005
2014-03-05 10:48:18,238 INFO querymaster.QueryMasterTask
(QueryMasterTask.java:handle(277)) - Query final state: QUERY_ERROR
2014-03-05 10:48:18,239 INFO querymaster.QueryMasterTask
(QueryMasterTask.java:stop(172)) - Stopping QueryMasterTask:q_1393922377833_0005
2014-03-05 10:48:18,246 WARN master.TajoAsyncDispatcher
(TajoAsyncDispatcher.java:stop(115)) - Interrupted Exception while stopping
2014-03-05 10:48:18,247 INFO master.TajoAsyncDispatcher
(TajoAsyncDispatcher.java:stop(122)) - AsyncDispatcher
stopped:q_1393922377833_0005
2014-03-05 10:48:18,248 INFO querymaster.QueryMasterTask
(QueryMasterTask.java:stop(200)) - Stopped QueryMasterTask:q_1393922377833_0005
2014-03-05 10:48:18,248 INFO querymaster.QueryMaster
(QueryMaster.java:cleanup(167)) - cleanup query resources : q_1393922377833_0005
2014-03-05 10:48:18,419 INFO worker.TajoWorkerClientService
(TajoWorkerClientService.java:closeQuery(211)) - Stop Query:q_1393922377833_0005
2014-03-05 10:48:18,844 INFO querymaster.QueryMasterManagerService
(QueryMasterManagerService.java:executeQuery(224)) - Receive executeQuery
request:q_1393922377833_0006
2014-03-05 10:48:18,845 INFO querymaster.QueryMaster
(QueryMaster.java:handle(374)) - Start
QueryStartEventHandler:q_1393922377833_0006
2014-03-05 10:48:18,853 INFO querymaster.QueryMasterTask
(QueryMasterTask.java:initStagingDir(403)) - The staging dir 'null' is created.
2014-03-05 10:48:18,943 INFO rewrite.BasicQueryRewriteEngine
(BasicQueryRewriteEngine.java:rewrite(64)) - The rule "ProjectionPushDown "
rewrites the query.
2014-03-05 10:48:18,944 INFO global.GlobalPlanner
(GlobalPlanner.java:build(103)) -
-----------------------------
Query Block Graph
The QueryMaster log content:
-----------------------------
Query Block Graph
-----------------------------
|-#ROOT
-----------------------------
Optimization Log:
-----------------------------
SCAN(0) on test
=> target list: test.id (INT4), test.interface (TEXT), test.start_time
(TEXT), test.byte (TEXT)
=> out schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time
(TEXT),test.byte (TEXT)}
=> in schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time
(TEXT),test.byte (TEXT)}
2014-03-05 10:48:11,760 INFO master.GlobalEngine
(GlobalEngine.java:createLogicalPlan(252)) -
=============================================
2014-03-05 10:48:11,761 INFO master.TajoAsyncDispatcher
(TajoAsyncDispatcher.java:start(101)) - AsyncDispatcher
started:QueryInProgress:q_1393922377833_0003
2014-03-05 10:48:11,761 INFO querymaster.QueryInProgress
(QueryInProgress.java:startQueryMaster(151)) - Initializing QueryInProgress for
QueryID=q_1393922377833_0003
2014-03-05 10:48:11,761 INFO rm.TajoWorkerResourceManager
(TajoWorkerResourceManager.java:allocateQueryMaster(201)) -
q_1393922377833_0003's QueryMaster is host:slave2,
port=28093,28091,28092,53265, slots=m:1024,d:1.0,c:1, used=m:512,d:0.0,c:0
2014-03-05 10:48:11,762 INFO querymaster.QueryInProgress
(QueryInProgress.java:connectQueryMaster(194)) - Connect to
QueryMaster:slave2/192.168.1.79:28093
2014-03-05 10:48:11,767 INFO querymaster.QueryInProgress
(QueryInProgress.java:submmitQueryToMaster(215)) - Call executeQuery to
:slave2:28093,q_1393922377833_0003
2014-03-05 10:48:12,013 INFO querymaster.QueryInProgress
(QueryInProgress.java:heartbeat(246)) - Received QueryMaster
heartbeat:q_1393922377833_0003state=QUERY_RUNNING,progress=NaN,
queryMaster=host:slave2, port=28093,28091,28092,0, slots=m:0,d:0.0,c:0,
used=m:0,d:0.0,c:0
2014-03-05 10:48:12,015 INFO querymaster.QueryInProgress
(QueryInProgress.java:heartbeat(246)) - Received QueryMaster
heartbeat:q_1393922377833_0003state=QUERY_ERROR,progress=NaN,
queryMaster=host:slave2, port=28093,28091,28092,0, slots=m:0,d:0.0,c:0,
used=m:0,d:0.0,c:0
2014-03-05 10:48:12,015 INFO querymaster.QueryJobManager
(QueryJobManager.java:stopQuery(139)) - Stop
QueryInProgress:q_1393922377833_0003
2014-03-05 10:48:12,016 INFO querymaster.QueryInProgress
(QueryInProgress.java:stop(104)) -
=========================================================
2014-03-05 10:48:12,016 INFO querymaster.QueryInProgress
(QueryInProgress.java:stop(105)) - Stop query:q_1393922377833_0003
2014-03-05 10:48:12,016 INFO rm.TajoWorkerResourceManager
(TajoWorkerResourceManager.java:stopQueryMaster(634)) - release QueryMaster
resource:q_1393922377833_0003,host:slave2, port=28093,28091,28092,53265,
slots=m:1024,d:1.0,c:1, used=m:0,d:0.0,c:0
2014-03-05 10:48:12,016 INFO querymaster.QueryInProgress
(QueryInProgress.java:stop(113)) - q_1393922377833_0003 QueryMaster stopped
2014-03-05 10:48:12,019 INFO master.TajoAsyncDispatcher
(TajoAsyncDispatcher.java:stop(122)) - AsyncDispatcher
stopped:QueryInProgress:q_1393922377833_0003
2014-03-05 10:48:16,253 INFO master.GlobalEngine
(GlobalEngine.java:executeQuery(107)) - SQL: select * from test
2014-03-05 10:48:16,253 INFO master.GlobalEngine
(GlobalEngine.java:executeQuery(126)) - hive.query.mode:false
2014-03-05 10:48:16,342 INFO master.GlobalEngine
(GlobalEngine.java:createLogicalPlan(246)) -
=============================================
2014-03-05 10:48:16,342 INFO master.GlobalEngine
(GlobalEngine.java:createLogicalPlan(247)) - Non Optimized Query:
-----------------------------
Query Block Graph
-----------------------------
--
This message was sent by Atlassian JIRA
(v6.2#6252)
