[
https://issues.apache.org/jira/browse/TAJO-655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13920460#comment-13920460
]
Hyunsik Choi commented on TAJO-655:
-----------------------------------
Hi Guo,
According to the error message, the worker cannot load LzoCodec library as
follows:
{noformat}
at
org.apache.tajo.storage.AbstractStorageManager.newScannerInstance(AbstractStorageManager.java:659)
... 25 more
Caused by: java.lang.IllegalArgumentException: Compression codec
com.hadoop.compression.lzo.LzoCodec not found.
at
org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:135)
at
org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175)
at org.apache.tajo.storage.CSVFile$CSVScanner.<init>(CSVFile.java:245)
{noformat}
Since Tajo uses the configs of an existing hadoop cluster, the tajo cluster
will find LzoCodec if your hadoop has the configs for LzoCodec. Could you check
the Lzo config in your hadoop cluster?
> QueryMaster sent "Select query" command to TajoWorker,but the TajoWorker
> don't working
> ---------------------------------------------------------------------------------------
>
> Key: TAJO-655
> URL: https://issues.apache.org/jira/browse/TAJO-655
> Project: Tajo
> Issue Type: Bug
> Components: query master, worker
> Affects Versions: 0.8-incubating
> Environment: CentOS 6.3 32bit,jdk 1.7,hadoop 2.2.0
> Reporter: Andrew Guo
> Fix For: 0.8-incubating
>
>
> Hi
> Yesterday ,You provide tajo branch-0.8.0 already installed. In the
> http://xxx:26080 web ui,the TajaWorker is live .In the TSQL interface input
> "select * from test" command,but Prompt "Internal error!" error.
> TajoWorker error log content:
> =======================================================
> Block Id: eb_1393922377833_0005_000001 [ROOT]
> =======================================================
> SCAN(0) on test
> => target list: test.id (INT4), test.interface (TEXT), test.start_time
> (TEXT), test.byte (TEXT)
> => out schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time
> (TEXT),test.byte (TEXT)}
> => in schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time
> (TEXT),test.byte (TEXT)}
> =======================================================
> Block Id: eb_1393922377833_0005_000002 [TERMINAL]
> =======================================================
> 2014-03-05 10:48:18,219 INFO master.TajoAsyncDispatcher
> (TajoAsyncDispatcher.java:start(101)) - AsyncDispatcher
> started:q_1393922377833_0005
> 2014-03-05 10:48:18,222 INFO querymaster.Query (Query.java:handle(641)) -
> Processing q_1393922377833_0005 of type START
> 2014-03-05 10:48:18,223 INFO querymaster.SubQuery
> (SubQuery.java:initTaskScheduler(661)) -
> org.apache.tajo.master.DefaultTaskScheduler is chosen for the task scheduling
> 2014-03-05 10:48:18,230 INFO storage.AbstractStorageManager
> (AbstractStorageManager.java:listStatus(382)) - Total input paths to process
> : 1
> 2014-03-05 10:48:18,236 ERROR querymaster.SubQuery
> (SubQuery.java:transition(645)) - SubQuery (eb_1393922377833_0005_000001)
> ERROR:
> java.lang.RuntimeException: java.lang.reflect.InvocationTargetException
> at
> org.apache.tajo.storage.AbstractStorageManager.newScannerInstance(AbstractStorageManager.java:661)
> at
> org.apache.tajo.storage.StorageManager.getScanner(StorageManager.java:60)
> at
> org.apache.tajo.storage.AbstractStorageManager.getScanner(AbstractStorageManager.java:106)
> at
> org.apache.tajo.storage.AbstractStorageManager.getFileScanner(AbstractStorageManager.java:94)
> at
> org.apache.tajo.storage.AbstractStorageManager.isSplittable(AbstractStorageManager.java:399)
> at
> org.apache.tajo.storage.AbstractStorageManager.getSplits(AbstractStorageManager.java:554)
> at
> org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.scheduleFragmentsForLeafQuery(SubQuery.java:872)
> at
> org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.schedule(SubQuery.java:778)
> at
> org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.transition(SubQuery.java:629)
> at
> org.apache.tajo.master.querymaster.SubQuery$InitAndRequestContainer.transition(SubQuery.java:610)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$MultipleInternalArc.doTransition(StateMachineFactory.java:385)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at org.apache.tajo.master.querymaster.SubQuery.handle(SubQuery.java:587)
> at
> org.apache.tajo.master.querymaster.Query$StartTransition.transition(Query.java:332)
> at
> org.apache.tajo.master.querymaster.Query$StartTransition.transition(Query.java:321)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$SingleInternalArc.doTransition(StateMachineFactory.java:362)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:302)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
> at
> org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
> at org.apache.tajo.master.querymaster.Query.handle(Query.java:646)
> at org.apache.tajo.master.querymaster.Query.handle(Query.java:58)
> at
> org.apache.tajo.master.TajoAsyncDispatcher.dispatch(TajoAsyncDispatcher.java:137)
> at
> org.apache.tajo.master.TajoAsyncDispatcher$1.run(TajoAsyncDispatcher.java:79)
> at java.lang.Thread.run(Thread.java:744)
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> at
> org.apache.tajo.storage.AbstractStorageManager.newScannerInstance(AbstractStorageManager.java:659)
> ... 25 more
> Caused by: java.lang.IllegalArgumentException: Compression codec
> com.hadoop.compression.lzo.LzoCodec not found.
> at
> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:135)
> at
> org.apache.hadoop.io.compress.CompressionCodecFactory.<init>(CompressionCodecFactory.java:175)
> at org.apache.tajo.storage.CSVFile$CSVScanner.<init>(CSVFile.java:245)
> ... 30 more
> Caused by: java.lang.ClassNotFoundException: Class
> com.hadoop.compression.lzo.LzoCodec not found
> at
> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:1626)
> at
> org.apache.hadoop.io.compress.CompressionCodecFactory.getCodecClasses(CompressionCodecFactory.java:128)
> ... 32 more
> 2014-03-05 10:48:18,237 INFO querymaster.Query (Query.java:handle(654)) -
> q_1393922377833_0005 Query Transitioned from QUERY_NEW to QUERY_RUNNING
> 2014-03-05 10:48:18,238 INFO querymaster.Query (Query.java:handle(641)) -
> Processing q_1393922377833_0005 of type SUBQUERY_COMPLETED
> 2014-03-05 10:48:18,238 INFO querymaster.Query (Query.java:handle(641)) -
> Processing q_1393922377833_0005 of type QUERY_COMPLETED
> 2014-03-05 10:48:18,238 INFO querymaster.Query (Query.java:handle(654)) -
> q_1393922377833_0005 Query Transitioned from QUERY_RUNNING to QUERY_ERROR
> 2014-03-05 10:48:18,238 INFO querymaster.QueryMasterTask
> (QueryMasterTask.java:handle(266)) - Query completion notified from
> q_1393922377833_0005
> 2014-03-05 10:48:18,238 INFO querymaster.QueryMasterTask
> (QueryMasterTask.java:handle(277)) - Query final state: QUERY_ERROR
> 2014-03-05 10:48:18,239 INFO querymaster.QueryMasterTask
> (QueryMasterTask.java:stop(172)) - Stopping
> QueryMasterTask:q_1393922377833_0005
> 2014-03-05 10:48:18,246 WARN master.TajoAsyncDispatcher
> (TajoAsyncDispatcher.java:stop(115)) - Interrupted Exception while stopping
> 2014-03-05 10:48:18,247 INFO master.TajoAsyncDispatcher
> (TajoAsyncDispatcher.java:stop(122)) - AsyncDispatcher
> stopped:q_1393922377833_0005
> 2014-03-05 10:48:18,248 INFO querymaster.QueryMasterTask
> (QueryMasterTask.java:stop(200)) - Stopped
> QueryMasterTask:q_1393922377833_0005
> 2014-03-05 10:48:18,248 INFO querymaster.QueryMaster
> (QueryMaster.java:cleanup(167)) - cleanup query resources :
> q_1393922377833_0005
> 2014-03-05 10:48:18,419 INFO worker.TajoWorkerClientService
> (TajoWorkerClientService.java:closeQuery(211)) - Stop
> Query:q_1393922377833_0005
> 2014-03-05 10:48:18,844 INFO querymaster.QueryMasterManagerService
> (QueryMasterManagerService.java:executeQuery(224)) - Receive executeQuery
> request:q_1393922377833_0006
> 2014-03-05 10:48:18,845 INFO querymaster.QueryMaster
> (QueryMaster.java:handle(374)) - Start
> QueryStartEventHandler:q_1393922377833_0006
> 2014-03-05 10:48:18,853 INFO querymaster.QueryMasterTask
> (QueryMasterTask.java:initStagingDir(403)) - The staging dir 'null' is
> created.
> 2014-03-05 10:48:18,943 INFO rewrite.BasicQueryRewriteEngine
> (BasicQueryRewriteEngine.java:rewrite(64)) - The rule "ProjectionPushDown "
> rewrites the query.
> 2014-03-05 10:48:18,944 INFO global.GlobalPlanner
> (GlobalPlanner.java:build(103)) -
> -----------------------------
> Query Block Graph
> The QueryMaster log content:
> -----------------------------
> Query Block Graph
> -----------------------------
> |-#ROOT
> -----------------------------
> Optimization Log:
> -----------------------------
> SCAN(0) on test
> => target list: test.id (INT4), test.interface (TEXT), test.start_time
> (TEXT), test.byte (TEXT)
> => out schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time
> (TEXT),test.byte (TEXT)}
> => in schema: {(4) test.id (INT4),test.interface (TEXT),test.start_time
> (TEXT),test.byte (TEXT)}
> 2014-03-05 10:48:11,760 INFO master.GlobalEngine
> (GlobalEngine.java:createLogicalPlan(252)) -
> =============================================
> 2014-03-05 10:48:11,761 INFO master.TajoAsyncDispatcher
> (TajoAsyncDispatcher.java:start(101)) - AsyncDispatcher
> started:QueryInProgress:q_1393922377833_0003
> 2014-03-05 10:48:11,761 INFO querymaster.QueryInProgress
> (QueryInProgress.java:startQueryMaster(151)) - Initializing QueryInProgress
> for QueryID=q_1393922377833_0003
> 2014-03-05 10:48:11,761 INFO rm.TajoWorkerResourceManager
> (TajoWorkerResourceManager.java:allocateQueryMaster(201)) -
> q_1393922377833_0003's QueryMaster is host:slave2,
> port=28093,28091,28092,53265, slots=m:1024,d:1.0,c:1, used=m:512,d:0.0,c:0
> 2014-03-05 10:48:11,762 INFO querymaster.QueryInProgress
> (QueryInProgress.java:connectQueryMaster(194)) - Connect to
> QueryMaster:slave2/192.168.1.79:28093
> 2014-03-05 10:48:11,767 INFO querymaster.QueryInProgress
> (QueryInProgress.java:submmitQueryToMaster(215)) - Call executeQuery to
> :slave2:28093,q_1393922377833_0003
> 2014-03-05 10:48:12,013 INFO querymaster.QueryInProgress
> (QueryInProgress.java:heartbeat(246)) - Received QueryMaster
> heartbeat:q_1393922377833_0003state=QUERY_RUNNING,progress=NaN,
> queryMaster=host:slave2, port=28093,28091,28092,0, slots=m:0,d:0.0,c:0,
> used=m:0,d:0.0,c:0
> 2014-03-05 10:48:12,015 INFO querymaster.QueryInProgress
> (QueryInProgress.java:heartbeat(246)) - Received QueryMaster
> heartbeat:q_1393922377833_0003state=QUERY_ERROR,progress=NaN,
> queryMaster=host:slave2, port=28093,28091,28092,0, slots=m:0,d:0.0,c:0,
> used=m:0,d:0.0,c:0
> 2014-03-05 10:48:12,015 INFO querymaster.QueryJobManager
> (QueryJobManager.java:stopQuery(139)) - Stop
> QueryInProgress:q_1393922377833_0003
> 2014-03-05 10:48:12,016 INFO querymaster.QueryInProgress
> (QueryInProgress.java:stop(104)) -
> =========================================================
> 2014-03-05 10:48:12,016 INFO querymaster.QueryInProgress
> (QueryInProgress.java:stop(105)) - Stop query:q_1393922377833_0003
> 2014-03-05 10:48:12,016 INFO rm.TajoWorkerResourceManager
> (TajoWorkerResourceManager.java:stopQueryMaster(634)) - release QueryMaster
> resource:q_1393922377833_0003,host:slave2, port=28093,28091,28092,53265,
> slots=m:1024,d:1.0,c:1, used=m:0,d:0.0,c:0
> 2014-03-05 10:48:12,016 INFO querymaster.QueryInProgress
> (QueryInProgress.java:stop(113)) - q_1393922377833_0003 QueryMaster stopped
> 2014-03-05 10:48:12,019 INFO master.TajoAsyncDispatcher
> (TajoAsyncDispatcher.java:stop(122)) - AsyncDispatcher
> stopped:QueryInProgress:q_1393922377833_0003
> 2014-03-05 10:48:16,253 INFO master.GlobalEngine
> (GlobalEngine.java:executeQuery(107)) - SQL: select * from test
> 2014-03-05 10:48:16,253 INFO master.GlobalEngine
> (GlobalEngine.java:executeQuery(126)) - hive.query.mode:false
> 2014-03-05 10:48:16,342 INFO master.GlobalEngine
> (GlobalEngine.java:createLogicalPlan(246)) -
> =============================================
> 2014-03-05 10:48:16,342 INFO master.GlobalEngine
> (GlobalEngine.java:createLogicalPlan(247)) - Non Optimized Query:
> -----------------------------
> Query Block Graph
> -----------------------------
--
This message was sent by Atlassian JIRA
(v6.2#6252)
