[jira] [Created] (HIVE-24188) CTLT from MM to External fails because table txn properties are not skipped

[Naresh P R (Jira)]

Naresh P R created HIVE-24188:
-

 Summary: CTLT from MM to External fails because table txn 
properties are not skipped
 Key: HIVE-24188
 URL: https://issues.apache.org/jira/browse/HIVE-24188
 Project: Hive
  Issue Type: Bug
Reporter: Naresh P R
Assignee: Naresh P R


Repro steps

 
{code:java}
set hive.support.concurrency=true;
set hive.txn.manager=org.apache.hadoop.hive.ql.lockmgr.DbTxnManager;
create table test_mm(age int, name string) partitioned by(dept string) stored 
as orc tblproperties('transactional'='true', 
'transactional_properties'='default');
create external table test_external like test_mm LOCATION 
'${system:test.tmp.dir}/create_like_mm_to_external';
{code}
Fails with below exception
{code:java}
Error: Error while processing statement: FAILED: Execution Error, return code 1 
from org.apache.hadoop.hive.ql.exec.DDLTask. 
MetaException(message:default.test_external cannot be declared transactional 
because it's an external table) (state=08S01,code=1){code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-24187) Handle _files creation for HA config with same nameservice on source and destination

[Pravin Sinha (Jira)]

Pravin Sinha created HIVE-24187:
---

 Summary: Handle _files creation for HA config with same 
nameservice on source and destination
 Key: HIVE-24187
 URL: https://issues.apache.org/jira/browse/HIVE-24187
 Project: Hive
  Issue Type: Improvement
Reporter: Pravin Sinha
Assignee: Pravin Sinha


Current HA is supported only for different nameservices on Source and 
Destination. We need to add support of same nameservice on Source and 
Destination.
Local nameservice will be passed correctly to the repl command.
Remote nameservice will be a random name and corresponding configs for the same.

Example:
Clusters originally configured with ns for hdfs:
src: ns1
target : ns1

We can denote remote name with some random name, say for example: nsRemote. 
This is how the command will see the ns w.r.t source and target:

Repl Dump : src: ns1, target: nsRemote
Repl Load: src: nsRemote, target: ns1

Entries in the _files(for managed table data loc) will be made with nsRemote in 
stead of ns1(for src).
Example: 
hdfs://nsRemote/whLoc/dbName.db/table1:checksum:subDir:hdfs://nsRemote/cmroot

Same way list of external table data locations will also be modified using 
nsRemote in stead of ns1(for src).

New configs can control the behavior:
*hive.repl.ha.datapath.replace.remote.nameservice = *
*hive.repl.ha.datapath.replace.remote.nameservice.name = *

Based on the above configs replacement of nameservice can be done.

This will also require that 'hive.repl.rootdir' is passed accordingly during 
dump and load:
Repl dump:
||Repl Operation||Repl Command||
|*Staging on source cluster*|
|Repl Dump|repl dump dbName with('hive.repl.rootdir'='hdfs://ns1/stagingLoc')|
|Repl Load|repl load dbName into dbName 
with('hive.repl.rootdir'='hdfs://nsRemote/stagingLoc')|
|*Staging on target cluster*|
|Repl Dump|repl dump dbName 
with('hive.repl.rootdir'='hdfs://nsRemote/stagingLoc')|
|Repl Load|repl load dbName into dbName 
with('hive.repl.rootdir'='hdfs://ns1/stagingLoc')|



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (HIVE-24186) The aggregate class operation fails when the CBO is false

[GuangMing Lu (Jira)]

GuangMing Lu created HIVE-24186:
---

 Summary: The aggregate class operation fails when the CBO is false
 Key: HIVE-24186
 URL: https://issues.apache.org/jira/browse/HIVE-24186
 Project: Hive
  Issue Type: Bug
  Components: CBO, SQL
Affects Versions: 3.1.0
Reporter: GuangMing Lu
 Fix For: 3.1.2, 3.1.0


{code:java}
create table table_1
(
idx string, 
namex string
) stored as orc;
create table table_2
(
sid string,
sname string
)stored as orc;

set hive.cbo.enable=false;

explain
insert into table table_1(idx , namex)
select t.sid idx, '123' namex 
from table_2 t
group by t.sid
order by 1,2;
{code}
Executing the above SQL will report an error, errors as follows:
{code:java}
org.apache.hive.service.cli.HiveSQLException: Error while compiling statement: 
FAILED: SemanticException [Error 10004]: Line 4:7 Invalid table alias or column 
reference 't': (possible column names are: _col0, _col1)
    at 
org.apache.hive.service.cli.operation.Operation.toSQLException(Operation.java:341)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.cli.operation.SQLOperation.prepare(SQLOperation.java:215)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.cli.operation.SQLOperation.runInternal(SQLOperation.java:316)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.cli.operation.Operation.run(Operation.java:253) 
~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementInternal(HiveSessionImpl.java:684)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.cli.session.HiveSessionImpl.executeStatementAsync(HiveSessionImpl.java:670)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at sun.reflect.GeneratedMethodAccessor151.invoke(Unknown Source) ~[?:?]
    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 ~[?:1.8.0_242]
    at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_242]
    at 
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:78)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.cli.session.HiveSessionProxy.access$000(HiveSessionProxy.java:36)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.cli.session.HiveSessionProxy$1.run(HiveSessionProxy.java:63)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at java.security.AccessController.doPrivileged(Native Method) 
~[?:1.8.0_242]
    at javax.security.auth.Subject.doAs(Subject.java:422) ~[?:1.8.0_242]
    at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1737)
 ~[hadoop-common-3.1.1-hw-ei-302001-SNAPSHOT.jar:?]
    at 
org.apache.hive.service.cli.session.HiveSessionProxy.invoke(HiveSessionProxy.java:59)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at com.sun.proxy.$Proxy66.executeStatementAsync(Unknown Source) ~[?:?]
    at 
org.apache.hive.service.cli.CLIService.executeStatementAsync(CLIService.java:342)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.cli.thrift.ThriftCLIService.executeNewStatement(ThriftCLIService.java:1144)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.cli.thrift.ThriftCLIService.ExecuteStatement(ThriftCLIService.java:1280)
 ~[hive-service-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1557)
 ~[hive-service-rpc-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hive.service.rpc.thrift.TCLIService$Processor$ExecuteStatement.getResult(TCLIService.java:1542)
 ~[hive-service-rpc-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39) 
~[hive-exec-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39) 
~[hive-exec-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 
org.apache.hadoop.hive.metastore.security.HadoopThriftAuthBridge$Server$TUGIAssumingProcessor.process(HadoopThriftAuthBridge.java:648)
 ~[hive-exec-3.1.0-hw-ei-302001-SNAPSHOT.jar:3.1.0-hw-ei-302001-SNAPSHOT]
    at 

[jira] [Created] (HIVE-24185) Upgrade snappy-java to 1.1.7.5

[Panagiotis Garefalakis (Jira)]

Panagiotis Garefalakis created HIVE-24185:
-

 Summary: Upgrade snappy-java to 1.1.7.5
 Key: HIVE-24185
 URL: https://issues.apache.org/jira/browse/HIVE-24185
 Project: Hive
  Issue Type: Bug
Affects Versions: 4.0.0
Reporter: Panagiotis Garefalakis
Assignee: Panagiotis Garefalakis


Bump version to take advantage of perf improvements, glibc compatibility etc.

https://github.com/xerial/snappy-java/blob/master/Milestone.md#snappy-java-117-2017-11-30



--
This message was sent by Atlassian Jira
(v8.3.4#803005)