[jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs

2014-11-08 Thread Lefty Leverenz (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Lefty Leverenz updated HIVE-8785:
-
Labels: TODOC14  (was: )

> HiveServer2 LogDivertAppender should be more selective for beeline getLogs
> --
>
> Key: HIVE-8785
> URL: https://issues.apache.org/jira/browse/HIVE-8785
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Thejas M Nair
>  Labels: TODOC14
> Fix For: 0.14.0
>
> Attachments: HIVE-8785.1.patch, HIVE-8785.2.patch, HIVE-8785.3.patch, 
> HIVE-8785.4.patch, HIVE-8785.4.patch, HIVE-8785.5.patch
>
>
> A simple query run via beeline JDBC like {{explain select count(1) from 
> testing.foo;}} produces 50 lines of output which looks like 
> {code}
> 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo;
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select 
> count(1) from testing.foo
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959379 end=1415262959380 duration=1 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/11/06 00:35:59 INFO ql.Context: New scratch dir is 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10)
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> oldColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> newColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation
> 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959381 end=1415262959401 duration=20 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], 
> properties:null)
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959378 end=1415262959402 duration=24 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> ++--+
> |  Explain   |
> ++--+
> | STAGE DEPENDENCIES:|
> |   Stage-0 is a root stage  |
> ||
> | STAGE PLANS:   |
> |   Stage: Stage-0   |
> | Fetch Operator |
> |   limit: 1 |
> |   Processor Tree:  |
> | ListSink   |
> ||
> ++--+
> 10 rows selected (0.1 seconds)
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating 
> a lock manager
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) 
> from testing.foo
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959403 end=1415262959405 duration=2 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apach

[jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs

2014-11-08 Thread Thejas M Nair (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-8785:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

Patch committed to trunk and 0.14 branch.
Thanks for the reviews [~gopalv] [~hagleitn]

> HiveServer2 LogDivertAppender should be more selective for beeline getLogs
> --
>
> Key: HIVE-8785
> URL: https://issues.apache.org/jira/browse/HIVE-8785
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Thejas M Nair
> Fix For: 0.14.0
>
> Attachments: HIVE-8785.1.patch, HIVE-8785.2.patch, HIVE-8785.3.patch, 
> HIVE-8785.4.patch, HIVE-8785.4.patch, HIVE-8785.5.patch
>
>
> A simple query run via beeline JDBC like {{explain select count(1) from 
> testing.foo;}} produces 50 lines of output which looks like 
> {code}
> 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo;
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select 
> count(1) from testing.foo
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959379 end=1415262959380 duration=1 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/11/06 00:35:59 INFO ql.Context: New scratch dir is 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10)
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> oldColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> newColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation
> 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959381 end=1415262959401 duration=20 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], 
> properties:null)
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959378 end=1415262959402 duration=24 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> ++--+
> |  Explain   |
> ++--+
> | STAGE DEPENDENCIES:|
> |   Stage-0 is a root stage  |
> ||
> | STAGE PLANS:   |
> |   Stage: Stage-0   |
> | Fetch Operator |
> |   limit: 1 |
> |   Processor Tree:  |
> | ListSink   |
> ||
> ++--+
> 10 rows selected (0.1 seconds)
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating 
> a lock manager
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) 
> from testing.foo
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959403 end=1415262959405 duration=2 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:

[jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs

2014-11-08 Thread Thejas M Nair (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-8785:

Attachment: HIVE-8785.5.patch

Yup, race condition! Serializing now -
HIVE-8785.5.patch - includes Gunther's 1 line change to TezTask logging


> HiveServer2 LogDivertAppender should be more selective for beeline getLogs
> --
>
> Key: HIVE-8785
> URL: https://issues.apache.org/jira/browse/HIVE-8785
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Thejas M Nair
> Fix For: 0.14.0
>
> Attachments: HIVE-8785.1.patch, HIVE-8785.2.patch, HIVE-8785.3.patch, 
> HIVE-8785.4.patch, HIVE-8785.4.patch, HIVE-8785.5.patch
>
>
> A simple query run via beeline JDBC like {{explain select count(1) from 
> testing.foo;}} produces 50 lines of output which looks like 
> {code}
> 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo;
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select 
> count(1) from testing.foo
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959379 end=1415262959380 duration=1 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/11/06 00:35:59 INFO ql.Context: New scratch dir is 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10)
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> oldColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> newColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation
> 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959381 end=1415262959401 duration=20 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], 
> properties:null)
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959378 end=1415262959402 duration=24 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> ++--+
> |  Explain   |
> ++--+
> | STAGE DEPENDENCIES:|
> |   Stage-0 is a root stage  |
> ||
> | STAGE PLANS:   |
> |   Stage: Stage-0   |
> | Fetch Operator |
> |   limit: 1 |
> |   Processor Tree:  |
> | ListSink   |
> ||
> ++--+
> 10 rows selected (0.1 seconds)
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating 
> a lock manager
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) 
> from testing.foo
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959403 end=1415262959405 duration=2 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.a

[jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs

2014-11-07 Thread Gunther Hagleitner (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-8785:
-
Attachment: HIVE-8785.4.patch

> HiveServer2 LogDivertAppender should be more selective for beeline getLogs
> --
>
> Key: HIVE-8785
> URL: https://issues.apache.org/jira/browse/HIVE-8785
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Thejas M Nair
> Fix For: 0.14.0
>
> Attachments: HIVE-8785.1.patch, HIVE-8785.2.patch, HIVE-8785.3.patch, 
> HIVE-8785.4.patch, HIVE-8785.4.patch
>
>
> A simple query run via beeline JDBC like {{explain select count(1) from 
> testing.foo;}} produces 50 lines of output which looks like 
> {code}
> 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo;
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select 
> count(1) from testing.foo
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959379 end=1415262959380 duration=1 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/11/06 00:35:59 INFO ql.Context: New scratch dir is 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10)
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> oldColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> newColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation
> 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959381 end=1415262959401 duration=20 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], 
> properties:null)
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959378 end=1415262959402 duration=24 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> ++--+
> |  Explain   |
> ++--+
> | STAGE DEPENDENCIES:|
> |   Stage-0 is a root stage  |
> ||
> | STAGE PLANS:   |
> |   Stage: Stage-0   |
> | Fetch Operator |
> |   limit: 1 |
> |   Processor Tree:  |
> | ListSink   |
> ||
> ++--+
> 10 rows selected (0.1 seconds)
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating 
> a lock manager
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) 
> from testing.foo
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959403 end=1415262959405 duration=2 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 0

[jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs

2014-11-07 Thread Thejas M Nair (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-8785:

Attachment: HIVE-8785.4.patch

HIVE-8785.4.patch - addressing review comments from Gopal. Added comments in 
review board about logging format suggestions.



> HiveServer2 LogDivertAppender should be more selective for beeline getLogs
> --
>
> Key: HIVE-8785
> URL: https://issues.apache.org/jira/browse/HIVE-8785
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Thejas M Nair
> Fix For: 0.14.0
>
> Attachments: HIVE-8785.1.patch, HIVE-8785.2.patch, HIVE-8785.3.patch, 
> HIVE-8785.4.patch
>
>
> A simple query run via beeline JDBC like {{explain select count(1) from 
> testing.foo;}} produces 50 lines of output which looks like 
> {code}
> 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo;
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select 
> count(1) from testing.foo
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959379 end=1415262959380 duration=1 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/11/06 00:35:59 INFO ql.Context: New scratch dir is 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10)
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> oldColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> newColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation
> 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959381 end=1415262959401 duration=20 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], 
> properties:null)
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959378 end=1415262959402 duration=24 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> ++--+
> |  Explain   |
> ++--+
> | STAGE DEPENDENCIES:|
> |   Stage-0 is a root stage  |
> ||
> | STAGE PLANS:   |
> |   Stage: Stage-0   |
> | Fetch Operator |
> |   limit: 1 |
> |   Processor Tree:  |
> | ListSink   |
> ||
> ++--+
> 10 rows selected (0.1 seconds)
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating 
> a lock manager
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) 
> from testing.foo
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959403 end=1415262959405 duration=2 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Dr

[jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs

2014-11-07 Thread Thejas M Nair (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-8785:

Attachment: HIVE-8785.3.patch

HIVE-8785.3.patch - some more test fixes.

> HiveServer2 LogDivertAppender should be more selective for beeline getLogs
> --
>
> Key: HIVE-8785
> URL: https://issues.apache.org/jira/browse/HIVE-8785
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Thejas M Nair
> Fix For: 0.14.0
>
> Attachments: HIVE-8785.1.patch, HIVE-8785.2.patch, HIVE-8785.3.patch
>
>
> A simple query run via beeline JDBC like {{explain select count(1) from 
> testing.foo;}} produces 50 lines of output which looks like 
> {code}
> 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo;
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select 
> count(1) from testing.foo
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959379 end=1415262959380 duration=1 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/11/06 00:35:59 INFO ql.Context: New scratch dir is 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10)
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> oldColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> newColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation
> 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959381 end=1415262959401 duration=20 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], 
> properties:null)
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959378 end=1415262959402 duration=24 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> ++--+
> |  Explain   |
> ++--+
> | STAGE DEPENDENCIES:|
> |   Stage-0 is a root stage  |
> ||
> | STAGE PLANS:   |
> |   Stage: Stage-0   |
> | Fetch Operator |
> |   limit: 1 |
> |   Processor Tree:  |
> | ListSink   |
> ||
> ++--+
> 10 rows selected (0.1 seconds)
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating 
> a lock manager
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) 
> from testing.foo
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959403 end=1415262959405 duration=2 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59

[jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs

2014-11-07 Thread Thejas M Nair (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-8785:

Attachment: HIVE-8785.2.patch

> HiveServer2 LogDivertAppender should be more selective for beeline getLogs
> --
>
> Key: HIVE-8785
> URL: https://issues.apache.org/jira/browse/HIVE-8785
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Thejas M Nair
> Fix For: 0.14.0
>
> Attachments: HIVE-8785.1.patch, HIVE-8785.2.patch
>
>
> A simple query run via beeline JDBC like {{explain select count(1) from 
> testing.foo;}} produces 50 lines of output which looks like 
> {code}
> 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo;
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select 
> count(1) from testing.foo
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959379 end=1415262959380 duration=1 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/11/06 00:35:59 INFO ql.Context: New scratch dir is 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10)
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> oldColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> newColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation
> 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959381 end=1415262959401 duration=20 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], 
> properties:null)
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959378 end=1415262959402 duration=24 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> ++--+
> |  Explain   |
> ++--+
> | STAGE DEPENDENCIES:|
> |   Stage-0 is a root stage  |
> ||
> | STAGE PLANS:   |
> |   Stage: Stage-0   |
> | Fetch Operator |
> |   limit: 1 |
> |   Processor Tree:  |
> | ListSink   |
> ||
> ++--+
> 10 rows selected (0.1 seconds)
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating 
> a lock manager
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) 
> from testing.foo
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959403 end=1415262959405 duration=2 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting task [Stage-1:EXPLAIN] in serial 
> m

[jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs

2014-11-07 Thread Thejas M Nair (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-8785:

Fix Version/s: 0.14.0

> HiveServer2 LogDivertAppender should be more selective for beeline getLogs
> --
>
> Key: HIVE-8785
> URL: https://issues.apache.org/jira/browse/HIVE-8785
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Thejas M Nair
> Fix For: 0.14.0
>
> Attachments: HIVE-8785.1.patch
>
>
> A simple query run via beeline JDBC like {{explain select count(1) from 
> testing.foo;}} produces 50 lines of output which looks like 
> {code}
> 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo;
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select 
> count(1) from testing.foo
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959379 end=1415262959380 duration=1 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/11/06 00:35:59 INFO ql.Context: New scratch dir is 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10)
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> oldColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> newColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation
> 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959381 end=1415262959401 duration=20 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], 
> properties:null)
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959378 end=1415262959402 duration=24 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> ++--+
> |  Explain   |
> ++--+
> | STAGE DEPENDENCIES:|
> |   Stage-0 is a root stage  |
> ||
> | STAGE PLANS:   |
> |   Stage: Stage-0   |
> | Fetch Operator |
> |   limit: 1 |
> |   Processor Tree:  |
> | ListSink   |
> ||
> ++--+
> 10 rows selected (0.1 seconds)
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating 
> a lock manager
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) 
> from testing.foo
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959403 end=1415262959405 duration=2 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting task [Stage-1:EXPLAIN] in serial 
> mode
> 14/11/06 00:35:59 INF

[jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs

2014-11-07 Thread Thejas M Nair (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-8785:

Attachment: HIVE-8785.1.patch

> HiveServer2 LogDivertAppender should be more selective for beeline getLogs
> --
>
> Key: HIVE-8785
> URL: https://issues.apache.org/jira/browse/HIVE-8785
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Thejas M Nair
> Attachments: HIVE-8785.1.patch
>
>
> A simple query run via beeline JDBC like {{explain select count(1) from 
> testing.foo;}} produces 50 lines of output which looks like 
> {code}
> 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo;
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select 
> count(1) from testing.foo
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959379 end=1415262959380 duration=1 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/11/06 00:35:59 INFO ql.Context: New scratch dir is 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10)
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> oldColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> newColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation
> 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959381 end=1415262959401 duration=20 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], 
> properties:null)
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959378 end=1415262959402 duration=24 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> ++--+
> |  Explain   |
> ++--+
> | STAGE DEPENDENCIES:|
> |   Stage-0 is a root stage  |
> ||
> | STAGE PLANS:   |
> |   Stage: Stage-0   |
> | Fetch Operator |
> |   limit: 1 |
> |   Processor Tree:  |
> | ListSink   |
> ||
> ++--+
> 10 rows selected (0.1 seconds)
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating 
> a lock manager
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) 
> from testing.foo
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959403 end=1415262959405 duration=2 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting task [Stage-1:EXPLAIN] in serial 
> mode
> 14/11/06 00:35:59 INFO log.PerfLogger:  start

[jira] [Updated] (HIVE-8785) HiveServer2 LogDivertAppender should be more selective for beeline getLogs

2014-11-07 Thread Thejas M Nair (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-8785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-8785:

Status: Patch Available  (was: Open)

> HiveServer2 LogDivertAppender should be more selective for beeline getLogs
> --
>
> Key: HIVE-8785
> URL: https://issues.apache.org/jira/browse/HIVE-8785
> Project: Hive
>  Issue Type: Bug
>Reporter: Gopal V
>Assignee: Thejas M Nair
> Attachments: HIVE-8785.1.patch
>
>
> A simple query run via beeline JDBC like {{explain select count(1) from 
> testing.foo;}} produces 50 lines of output which looks like 
> {code}
> 0: jdbc:hive2://localhost:10002> explain select count(1) from testing.foo;
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parsing command: explain select 
> count(1) from testing.foo
> 14/11/06 00:35:59 INFO parse.ParseDriver: Parse Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959379 end=1415262959380 duration=1 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed phase 1 of Semantic 
> Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for source tables
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for subqueries
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Get metadata for destination 
> tables
> 14/11/06 00:35:59 INFO ql.Context: New scratch dir is 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed getting MetaData in 
> Semantic Analysis
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Set stats collection dir : 
> hdfs://cn041-10.l42scl.hortonworks.com:8020/tmp/hive/gopal/6b3980f6-3238-4e91-ae53-cb3f54092dab/hive_2014-11-06_00-35-59_379_317426424610374080-1/-ext-10002
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for FS(16)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(15)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(14)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for RS(13)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for GBY(12)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for SEL(11)
> 14/11/06 00:35:59 INFO ppd.OpProcFactory: Processing for TS(10)
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> oldColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO optimizer.ColumnPrunerProcFactory: RS 13 
> newColExprMap: {VALUE._col0=Column[_col0]}
> 14/11/06 00:35:59 INFO parse.SemanticAnalyzer: Completed plan generation
> 14/11/06 00:35:59 INFO ql.Driver: Semantic Analysis Completed
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959381 end=1415262959401 duration=20 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Returning Hive schema: 
> Schema(fieldSchemas:[FieldSchema(name:Explain, type:string, comment:null)], 
> properties:null)
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959378 end=1415262959402 duration=24 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> ++--+
> |  Explain   |
> ++--+
> | STAGE DEPENDENCIES:|
> |   Stage-0 is a root stage  |
> ||
> | STAGE PLANS:   |
> |   Stage: Stage-0   |
> | Fetch Operator |
> |   limit: 1 |
> |   Processor Tree:  |
> | ListSink   |
> ||
> ++--+
> 10 rows selected (0.1 seconds)
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Concurrency mode is disabled, not creating 
> a lock manager
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting command: explain select count(1) 
> from testing.foo
> 14/11/06 00:35:59 INFO log.PerfLogger:  start=1415262959403 end=1415262959405 duration=2 
> from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO log.PerfLogger:  from=org.apache.hadoop.hive.ql.Driver>
> 14/11/06 00:35:59 INFO ql.Driver: Starting task [Stage-1:EXPLAIN] in serial 
> mode
> 14/11/06 00:35:59 INFO log.PerfLogger: