[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-11 Thread Fan Yunbo (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15481681#comment-15481681
 ] 

Fan Yunbo commented on HIVE-14686:
--

thanks for you review.

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.2.1, 2.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch, HIVE-14686.4.patch, HIVE-14686.5.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-09 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476757#comment-15476757
 ] 

Hive QA commented on HIVE-14686:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12827726/HIVE-14686.5.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10544 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_optimization_acid]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats0]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1139/testReport
Console output: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1139/console
Test logs: 
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-1139/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12827726 - PreCommit-HIVE-MASTER-Build

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch, HIVE-14686.4.patch, HIVE-14686.5.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-09 Thread Rui Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476462#comment-15476462
 ] 

Rui Li commented on HIVE-14686:
---

+1 pending test

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch, HIVE-14686.4.patch, HIVE-14686.5.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-09 Thread Fan Yunbo (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476439#comment-15476439
 ] 

Fan Yunbo commented on HIVE-14686:
--

[~ruili] Thanks for your advice again.
The braces of if statement have been added.

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch, HIVE-14686.4.patch, HIVE-14686.5.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-09 Thread Rui Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476386#comment-15476386
 ] 

Rui Li commented on HIVE-14686:
---

Thanks Yunbo for the update. Just a nit:
{code}
+if (command_type == CREATE_TABLE || command_type == CTLT)
+queryState.setCommandType(HiveOperation.CREATETABLE);
+else if (command_type == CTAS)
+queryState.setCommandType(HiveOperation.CREATETABLE_AS_SELECT);
+else
+throw new SemanticException("Unrecognized command.");
{code}
Please don't omit the braces of if statement.

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch, HIVE-14686.4.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-09 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476379#comment-15476379
 ] 

Hive QA commented on HIVE-14686:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12827716/HIVE-14686.4.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10544 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dynpart_sort_optimization_acid]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats0]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1138/testReport
Console output: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1138/console
Test logs: 
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-1138/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12827716 - PreCommit-HIVE-MASTER-Build

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch, HIVE-14686.4.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-09 Thread Fan Yunbo (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476358#comment-15476358
 ] 

Fan Yunbo commented on HIVE-14686:
--

[~ruili]Thanks for apply so quickly.
Sorry for misunderstanding.
Update again.

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch, HIVE-14686.4.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-09 Thread Rui Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476295#comment-15476295
 ] 

Rui Li commented on HIVE-14686:
---

Sorry if I wasn't being clear enough. I think we should set the cmd type after 
the for loop. For example, in the for loop we may firstly decide 
{{command_type}} is CTAS. But later we may also find CTLT so this is an error. 
In this case, we shouldn't modify the cmd type of the query state.

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch, HIVE-14686.4.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-09 Thread Fan Yunbo (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476187#comment-15476187
 ] 

Fan Yunbo commented on HIVE-14686:
--

[~ruili] 
I also thank so and I update the patch.
It's really nice of you to review it again. 

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch, HIVE-14686.4.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-09 Thread Rui Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15476051#comment-15476051
 ] 

Rui Li commented on HIVE-14686:
---

Hi Yunbo, how about setting the command type after the for loop
{code}
for (int num = 1; num < numCh; num++)
{code}
Because in the loop we can hit errors like CTAS_CTLT_COEXISTENCE. So let's not 
change the original behaviour in that case. I guess it also means you don't 
have to set a default value.

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-07 Thread Fan Yunbo (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15472336#comment-15472336
 ] 

Fan Yunbo commented on HIVE-14686:
--

[~ruili] Agree with you and thanks for your advice.
I made a change:
set command type once the value of command_type decided;
change related tests output files.
And these failure tests seems not related.


> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-07 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15470166#comment-15470166
 ] 

Hive QA commented on HIVE-14686:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12827337/HIVE-14686.3.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10453 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineWithArgs - did not produce a TEST-*.xml file
TestHiveCli - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1119/testReport
Console output: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1119/console
Test logs: 
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-1119/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12827337 - PreCommit-HIVE-MASTER-Build

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch, 
> HIVE-14686.3.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-05 Thread Rui Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15466173#comment-15466173
 ] 

Rui Li commented on HIVE-14686:
---

Thanks [~fanyunbojerry] for the update. Seems some test failures are related. 
Can you take a look at that?
Besides, I think a more general solution is to set command type once the value 
of {{command_type}} is decided.

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-04 Thread Fan Yunbo (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15463963#comment-15463963
 ] 

Fan Yunbo commented on HIVE-14686:
--

thank you for your reply [~ruili]
I hava changed the code to the latest and the issue still exists.
Here is the debug info when execute the query. 
{code}
42a5ec91-6a86-4480-84fc-c892029b498f main[1] print queryState.getCommandType()
 queryState.getCommandType() = "CREATETABLE"
{code}
I hava update the patch.
Could you please review it again, thank you.

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-02 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15460455#comment-15460455
 ] 

Hive QA commented on HIVE-14686:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12826784/HIVE-14686.2.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10443 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineWithArgs - did not produce a TEST-*.xml file
TestHiveCli - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.org.apache.hadoop.hive.cli.TestCliDriver
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ctas]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[ctas]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ctas]
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1088/testReport
Console output: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1088/console
Test logs: 
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-1088/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12826784 - PreCommit-HIVE-MASTER-Build

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch, HIVE-14686.2.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-01 Thread Rui Li (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15457213#comment-15457213
 ] 

Rui Li commented on HIVE-14686:
---

I think the patch doesn't apply to latest code.
[~fanyunbojerry], could you verify if the issue still exists with latest master 
or branch-1 code?

> Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS 
> ... AS"
> --
>
> Key: HIVE-14686
> URL: https://issues.apache.org/jira/browse/HIVE-14686
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 1.1.0
>Reporter: Fan Yunbo
>Assignee: Fan Yunbo
> Fix For: 2.2.0
>
> Attachments: HIVE-14686.1.patch
>
>
> See the query: 
> {code}
> create table if not exists DST as select * from SRC;
> {code}
> if the table DST doesn't exist, SessionState.get().getHiveOperation() will 
> return HiveOperation.CREATETABLE_AS_SELECT;
> But if the table DST already exists, it will return HiveOperation.CREATETABLE;
> It really makes some trouble for those who judge operation type by 
> SessionState.get().getHiveOperation().
> The reason I find out is that the function analyzeCreateTable in 
> SemanticAnalyzer.java will return null and won't set the correct command type 
> if the table already exists.
> Here is the related code:
> {code}
> // check for existence of table
> if (ifNotExists) {
>   try {
> Table table = getTable(qualifiedTabName, false);
> if (table != null) { // table exists
>   return null;
> }
>   } catch (HiveException e) {
> // should not occur since second parameter to getTableWithQN is false
> throw new IllegalStateException("Unxpected Exception thrown: " + 
> e.getMessage(), e);
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-14686) Get unexpected command type when execute query "CREATE TABLE IF NOT EXISTS ... AS"

2016-09-01 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-14686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15455842#comment-15455842
 ] 

Hive QA commented on HIVE-14686:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12826591/HIVE-14686.1.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1072/testReport
Console output: 
https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1072/console
Test logs: 
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-1072/

Messages:
{noformat}
 This message was trimmed, see log for full details 
[INFO] --- maven-jar-plugin:2.4:jar (default-jar) @ hive-llap-tez ---
[INFO] Building jar: 
/data/hive-ptest/working/apache-github-source-source/llap-tez/target/hive-llap-tez-2.2.0-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-site-plugin:3.3:attach-descriptor (attach-descriptor) @ 
hive-llap-tez ---
[INFO] 
[INFO] --- maven-install-plugin:2.4:install (default-install) @ hive-llap-tez 
---
[INFO] Installing 
/data/hive-ptest/working/apache-github-source-source/llap-tez/target/hive-llap-tez-2.2.0-SNAPSHOT.jar
 to 
/data/hive-ptest/working/maven/org/apache/hive/hive-llap-tez/2.2.0-SNAPSHOT/hive-llap-tez-2.2.0-SNAPSHOT.jar
[INFO] Installing 
/data/hive-ptest/working/apache-github-source-source/llap-tez/pom.xml to 
/data/hive-ptest/working/maven/org/apache/hive/hive-llap-tez/2.2.0-SNAPSHOT/hive-llap-tez-2.2.0-SNAPSHOT.pom
[INFO] 
[INFO] 
[INFO] Building Spark Remote Client 2.2.0-SNAPSHOT
[INFO] 
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ spark-client ---
[INFO] Deleting 
/data/hive-ptest/working/apache-github-source-source/spark-client/target
[INFO] Deleting 
/data/hive-ptest/working/apache-github-source-source/spark-client (includes = 
[datanucleus.log, derby.log], excludes = [])
[INFO] 
[INFO] --- maven-enforcer-plugin:1.3.1:enforce (enforce-no-snapshots) @ 
spark-client ---
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ spark-client 
---
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ 
spark-client ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 
/data/hive-ptest/working/apache-github-source-source/spark-client/src/main/resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ spark-client ---
[INFO] Executing tasks

main:
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ spark-client 
---
[INFO] Compiling 28 source files to 
/data/hive-ptest/working/apache-github-source-source/spark-client/target/classes
[WARNING] 
/data/hive-ptest/working/apache-github-source-source/spark-client/src/main/java/org/apache/hive/spark/client/SparkClientUtilities.java:
 
/data/hive-ptest/working/apache-github-source-source/spark-client/src/main/java/org/apache/hive/spark/client/SparkClientUtilities.java
 uses or overrides a deprecated API.
[WARNING] 
/data/hive-ptest/working/apache-github-source-source/spark-client/src/main/java/org/apache/hive/spark/client/SparkClientUtilities.java:
 Recompile with -Xlint:deprecation for details.
[WARNING] 
/data/hive-ptest/working/apache-github-source-source/spark-client/src/main/java/org/apache/hive/spark/client/rpc/RpcDispatcher.java:
 Some input files use unchecked or unsafe operations.
[WARNING] 
/data/hive-ptest/working/apache-github-source-source/spark-client/src/main/java/org/apache/hive/spark/client/rpc/RpcDispatcher.java:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ 
spark-client ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ spark-client ---
[INFO] Executing tasks

main:
[mkdir] Created dir: 
/data/hive-ptest/working/apache-github-source-source/spark-client/target/tmp
[mkdir] Created dir: 
/data/hive-ptest/working/apache-github-source-source/spark-client/target/warehouse
[mkdir] Created dir: 
/data/hive-ptest/working/apache-github-source-source/spark-client/target/tmp/conf
 [copy] Copying 15 files to 
/data/hive-ptest/working/apache-github-source-source/spark-client/target/tmp/conf
[INFO] Executed tasks
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @ 
spark-client ---
[INFO] Compiling 5 source files to 
/data/hive-ptest/working/apache-github-source-source/spark-client/target/test-classes
[INFO] 
[INFO] ---