-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/34457/
-----------------------------------------------------------
(Updated June 17, 2015, 5:26 a.m.)
Review request for lens and Amareshwari Sriramadasu.
Changes
-------
Rebase it to master
Remove only consoleAppender
Refctor the code(Move all log related methods to LoggerUtil class)
Changes in log statements
Bugs: LENS-25
https://issues.apache.org/jira/browse/LENS-25
Repository: lens
Description
-------
Added a cli logger for showing the query execution status
Added a verbose option
Diffs (updated)
-----
lens-cli/src/main/java/org/apache/lens/cli/commands/LensConnectionCommands.java
675e6c8
lens-client/src/main/java/org/apache/lens/client/LensClient.java ca017b8
lens-client/src/main/java/org/apache/lens/client/LensClientConfig.java
cf3a97c
lens-client/src/main/java/org/apache/lens/client/LensStatement.java e2693c9
src/site/apt/user/cli.apt 20785f1
tools/conf-pseudo-distr/client/log4j.properties 5520304
tools/conf/client/log4j.properties 5520304
Diff: https://reviews.apache.org/r/34457/diff/
Testing
-------
Tested on cli
**Enable logging by settign verbose value to true (by default to true)**
**lens-shell>query execute cube select product_details.category, store_sales
from sales where time_range_in(order_time, '2015-04-11-00', '2015-04-13-01')
--verbose true**
2015-05-20 08:42:05,563 [ LensStatement.java:Spring Shell:227] - Query
org.apache.lens.api.query.LensQuery@58dd5a16 submitted to HiveDriver
2015-05-20 08:42:15,510 [ LensStatement.java:Spring Shell:232] -
RUNNING:Query is running in HiveServer! -
Progress:0.3333333432674408:[{"taskId":"Stage-5","type":"MAPREDLOCAL","externalHandle":"Stage-5","taskState":"FINISHED_STATE"},{"taskId":"Stage-2","type":"MAPRED","externalHandle":null,"taskState":"RUNNING_STATE"},{"taskId":"Stage-0","type":"MOVE","externalHandle":"Stage-0","taskState":"UNKNOWN_STATE"}]
2015-05-20 08:42:25,561 [ LensStatement.java:Spring Shell:232] -
SUCCESSFUL:Query is successful! - Result Available
product_details.category _c1
Results of query stored at :
file:/tmp/lensreports/hdfsout/256d4c84-496c-49e8-993c-3d8b8f0451a5 -1 rows
process in (27) seconds.
**Disable logging by setting verbose value to false (by default to true)**
lens-shell>query execute cube select sample_dim.name, sample_dim2.name from
sample_dim where sample_dim.name != 'first' --verbose false
sample_dim.name sample_dim2.name
Results of query stored at :
file:/tmp/lensreports/hdfsout/a6abe7e3-e398-4c47-916c-80ab80c49c0b -1 rows
process in (8) seconds.
Tested on cli:
lens-shell>debug
lens-shell>query execute cube select sample_dim.name, measure4 from sample_cube
where time_range_in(dt, '2014-06-25-00', '2014-06-26-00') order by
sample_dim.name limit 10
16 Jun 2015 12:09:51,439 [Spring Shell] INFO cliLogger - Query handle
80563d54-65b6-4e5e-8fae-5f55e9653fcf
16 Jun 2015 12:09:58,516 [Spring Shell] INFO cliLogger - User query: cube
select sample_dim.name, measure4 from sample_cube where time_range_in(dt,
'2014-06-25-00', '2014-06-26-00') order by sample_dim.name limit 10 was
submitted to org.apache.lens.driver.hive.HiveDriver
16 Jun 2015 12:09:58,516 [Spring Shell] INFO cliLogger - Driver query:
INSERT OVERWRITE DIRECTORY
"file:/tmp/lensreports/hdfsout/80563d54-65b6-4e5e-8fae-5f55e9653fcf" SELECT (
sample_dim . name ), min(( sample_cube . measure4 )) FROM local_fact2
sample_cube join local_dim_table4 sample_dim on sample_cube.dim3 =
sample_dim.id and (sample_dim.dt = 'latest') WHERE (((( sample_cube . dt ) =
'2014-06-25' ))) GROUP BY ( sample_dim . name ) ORDER BY sample_dim . name
asc LIMIT 10 and Driver handle: OperationHandle [opType=EXECUTE_STATEMENT,
getHandleIdentifier()=aa101354-730f-4bfa-ba07-457a75ff27a2]
16 Jun 2015 12:10:08,380 [Spring Shell] INFO cliLogger - RUNNING:Query is
running in HiveServer! -
Progress:0.5:[{"taskId":"Stage-6","type":"MAPREDLOCAL","externalHandle":"Stage-6","taskState":"FINISHED_STATE"},{"taskId":"Stage-2","type":"MAPRED","externalHandle":null,"taskState":"FINISHED_STATE"},{"taskId":"Stage-3","type":"MAPRED","externalHandle":null,"taskState":"RUNNING_STATE"},{"taskId":"Stage-0","type":"MOVE","externalHandle":"Stage-0","taskState":"UNKNOWN_STATE"}]
16 Jun 2015 12:10:12,748 [Spring Shell] INFO cliLogger - SUCCESSFUL:Query is
successful! - Result Available
sample_dim.name _c1
Results of query stored at :
file:/tmp/lensreports/hdfsout/80563d54-65b6-4e5e-8fae-5f55e9653fcf -1 rows
process in (21) seconds.
lens-shell>debug false
lens-shell>verbose
lens-shell>query execute cube select sample_dim.name, measure4 from sample_cube
where time_range_in(dt, '2014-06-25-00', '2014-06-26-00') order by
sample_dim.name limit 10
log4j:WARN No appenders could be found for logger
(org.apache.lens.api.response.LensJAXBContextResolver).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more
info.
16 Jun 2015 12:13:14,533 [Spring Shell] INFO cliLogger - Query handle
f0ddc70c-4228-43da-9f34-bd6e3946220b
16 Jun 2015 12:13:19,740 [Spring Shell] INFO cliLogger - User query: cube
select sample_dim.name, measure4 from sample_cube where time_range_in(dt,
'2014-06-25-00', '2014-06-26-00') order by sample_dim.name limit 10 was
submitted to org.apache.lens.driver.hive.HiveDriver
16 Jun 2015 12:13:19,740 [Spring Shell] INFO cliLogger - Driver query:
INSERT OVERWRITE DIRECTORY
"file:/tmp/lensreports/hdfsout/f0ddc70c-4228-43da-9f34-bd6e3946220b" SELECT (
sample_dim . name ), min(( sample_cube . measure4 )) FROM local_fact2
sample_cube join local_dim_table sample_dim on sample_cube.dim3 = sample_dim.id
and (sample_dim.dt = 'latest') WHERE (((( sample_cube . dt ) = '2014-06-25'
))) GROUP BY ( sample_dim . name ) ORDER BY sample_dim . name asc LIMIT 10
and Driver handle: OperationHandle [opType=EXECUTE_STATEMENT,
getHandleIdentifier()=082bb77e-fa62-47f6-9ffc-3c3d51c5ef52]
16 Jun 2015 12:13:29,800 [Spring Shell] INFO cliLogger - RUNNING:Query is
running in HiveServer! -
Progress:0.5:[{"taskId":"Stage-6","type":"MAPREDLOCAL","externalHandle":"Stage-6","taskState":"FINISHED_STATE"},{"taskId":"Stage-2","type":"MAPRED","externalHandle":null,"taskState":"FINISHED_STATE"},{"taskId":"Stage-3","type":"MAPRED","externalHandle":null,"taskState":"RUNNING_STATE"},{"taskId":"Stage-0","type":"MOVE","externalHandle":"Stage-0","taskState":"UNKNOWN_STATE"}]
16 Jun 2015 12:13:31,488 [Spring Shell] INFO cliLogger - SUCCESSFUL:Query is
successful! - Result Available
sample_dim.name _c1
Results of query stored at :
file:/tmp/lensreports/hdfsout/f0ddc70c-4228-43da-9f34-bd6e3946220b -1 rows
process in (16) seconds.
lens-shell>
Thanks,
Raju Bairishetti
