[jira] [Updated] (HIVE-3703) Hive Query Explain Plan JSON not being created properly

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3703:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed. Thanks Mayank

> Hive Query Explain Plan JSON not being created properly
> ---
>
> Key: HIVE-3703
> URL: https://issues.apache.org/jira/browse/HIVE-3703
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Mayank Garg
>Assignee: Mayank Garg
>Priority: Minor
> Attachments: HIVE-3703.2.patch.txt
>
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> There is an option to generate a JSON query plan for the hive query, however, 
> the JSON being created is invalid and json_decoders are unable to decode it

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3718) Add check to determine whether partition can be dropped at Semantic Analysis time

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3718:
-

Status: Open  (was: Patch Available)

Can you refresh ? This conflicts with patch which got checked in.

> Add check to determine whether partition can be dropped at Semantic Analysis 
> time
> -
>
> Key: HIVE-3718
> URL: https://issues.apache.org/jira/browse/HIVE-3718
> Project: Hive
>  Issue Type: Task
>  Components: CLI
>Reporter: Pamela Vagata
>Assignee: Pamela Vagata
>Priority: Minor
> Attachments: HIVE-3718.1.patch.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3740) Insert into table with partition : overwriting the partition instead of appending records while running the hql script

[Sujesh Chirackkal (JIRA)]

Sujesh Chirackkal created HIVE-3740:
---

 Summary: Insert into table with partition : overwriting the 
partition instead of appending records while running the hql script
 Key: HIVE-3740
 URL: https://issues.apache.org/jira/browse/HIVE-3740
 Project: Hive
  Issue Type: Bug
  Components: CLI
Affects Versions: 0.8.1
Reporter: Sujesh Chirackkal


insert into table with partition works well with CLI. But overwriting the 
partition when executed through hql script.

Table Schema
(Table Name: jobs_status, partitioned on column job_name)
completed_time string
job_name string


hive script (status.hql) contains the below commands
use sample_db;
insert into table job_status 
partition(job_name='testJob')
select t.completionTime as completed_time from (select count(1) as cnt, 
from_unixtime(unix_timestamp()) as completionTime)t;

The above commands when executed through CLI works well and append new records 
into the partition. But when executing the script as hive -f status.hql, it 
overwrites the existing records from the partition and insert only the new 
record.



--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3471) Implement grouping sets in hive

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502700#comment-13502700
 ] 

Namit Jain commented on HIVE-3471:
--

Updated the documentation at 
https://cwiki.apache.org/confluence/display/Hive/Enhanced+Aggregation,+Cube,+Grouping+and+Rollup

> Implement grouping sets in hive
> ---
>
> Key: HIVE-3471
> URL: https://issues.apache.org/jira/browse/HIVE-3471
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Ivan Gorbachev
> Fix For: 0.10.0
>
> Attachments: jira-3471.0.patch, jira-3471.1.patch, jira-3471.2.patch, 
> jira-3471.3.pach
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3735) PTest doesn't work due to hive snapshot version upgrade to 11

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502772#comment-13502772
 ] 

Hudson commented on HIVE-3735:
--

Integrated in Hive-trunk-h0.21 #1811 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1811/])
HIVE-3735 PTest doesn't work due to hive snapshot version upgrade to 11
(Gang Tim Liu via namit) (Revision 1412416)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412416
Files : 
* /hive/trunk/testutils/ptest/hivetest.py


> PTest doesn't work due to hive snapshot version upgrade to 11
> -
>
> Key: HIVE-3735
> URL: https://issues.apache.org/jira/browse/HIVE-3735
> Project: Hive
>  Issue Type: Bug
>  Components: Testing Infrastructure
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>Priority: Critical
> Fix For: 0.10.0
>
> Attachments: HIVE-3735.patch.1
>
>
> PTest fails. Error
> ::
> [ivy:resolve] :: UNRESOLVED DEPENDENCIES ::
> [ivy:resolve] ::
> [ivy:resolve] :: org.apache.hive#hive-builtins;0.11.0-SNAPSHOT: not found
> [ivy:resolve] ::

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3722) Create index fails on CLI using remote metastore

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502773#comment-13502773
 ] 

Hudson commented on HIVE-3722:
--

Integrated in Hive-trunk-h0.21 #1811 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1811/])
HIVE-3722 Create index fails on CLI using remote metastore
(Kevin Wilfong via namit) (Revision 1412415)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412415
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java
* /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java
* /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHiveRemote.java


> Create index fails on CLI using remote metastore
> 
>
> Key: HIVE-3722
> URL: https://issues.apache.org/jira/browse/HIVE-3722
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Fix For: 0.10.0
>
> Attachments: HIVE-3722.1.patch.txt
>
>
> If the CLI uses a remote metastore and the user attempts to create an index 
> without a comment, it will fail with a NPE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1811 - Still Failing

[Apache Jenkins Server]

Changes for Build #1769

Changes for Build #1770
[namit] HIVE-3570 Add/fix facility to collect operator specific statisticsin 
hive + add hash-in/hash-out
counter for GroupBy Optr (Satadru Pan via namit)

[namit] HIVE-3554 Hive List Bucketing - Query logic
(Gang Tim Liu via namit)

[cws] HIVE-3563. Drop database cascade fails when there are indexes on any 
tables (Prasad Mujumdar via cws)


Changes for Build #1771
[kevinwilfong] HIVE-3640. Reducer allocation is incorrect if enforce bucketing 
and mapred.reduce.tasks are both set. (Vighnesh Avadhani via kevinwilfong)


Changes for Build #1772

Changes for Build #1773

Changes for Build #1774

Changes for Build #1775
[namit] HIVE-3673 Sort merge join not used when join columns have different 
names
(Kevin Wilfong via namit)


Changes for Build #1776
[kevinwilfong] HIVE-3627. eclipse misses library: 
javolution-@javolution-version@.jar. (Gang Tim Liu via kevinwilfong)


Changes for Build #1777
[kevinwilfong] HIVE-3524. Storing certain Exception objects thrown in 
HiveMetaStore.java in MetaStoreEndFunctionContext. (Maheshwaran Srinivasan via 
kevinwilfong)

[cws] HIVE-1977. DESCRIBE TABLE syntax doesn't support specifying a database 
qualified table name (Zhenxiao Luo via cws)

[cws] HIVE-3674. Test case TestParse broken after recent checkin (Sambavi 
Muthukrishnan via cws)


Changes for Build #1778
[cws] HIVE-1362. Column level scalar valued statistics on Tables and Partitions 
(Shreepadma Venugopalan via cws)


Changes for Build #1779

Changes for Build #1780
[kevinwilfong] HIVE-3686. Fix compile errors introduced by the interaction of 
HIVE-1362 and HIVE-3524. (Shreepadma Venugopalan via kevinwilfong)


Changes for Build #1781
[namit] HIVE-3687 smb_mapjoin_13.q is nondeterministic
(Kevin Wilfong via namit)


Changes for Build #1782
[hashutosh] HIVE-2715: Upgrade Thrift dependency to 0.9.0 (Ashutosh Chauhan)


Changes for Build #1783
[kevinwilfong] HIVE-3654. block relative path access in hive. (njain via 
kevinwilfong)

[hashutosh] HIVE-3658 : Unable to generate the Hbase related unit tests using 
velocity templates on Windows (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3661 : Remove the Windows specific = related swizzle path 
changes from Proxy FileSystems (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3480 : : Fix the file handle leaks in Symbolic 
& Symlink related input formats. (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1784
[kevinwilfong] HIVE-3675. NaN does not work correctly for round(n). (njain via 
kevinwilfong)

[cws] HIVE-3651. bucketmapjoin?.q tests fail with hadoop 0.23 (Prasad Mujumdar 
via cws)


Changes for Build #1785
[namit] HIVE-3613 Implement grouping_id function
(Ian Gorbachev via namit)

[namit] HIVE-3692 Update parallel test documentation
(Ivan Gorbachev via namit)

[namit] HIVE-3649 Hive List Bucketing - enhance DDL to specify list bucketing 
table
(Gang Tim Liu via namit)


Changes for Build #1786
[namit] HIVE-3696 Revert HIVE-3483 which causes performance regression
(Gang Tim Liu via namit)


Changes for Build #1787
[kevinwilfong] HIVE-3621. Make prompt in Hive CLI configurable. (Jingwei Lu via 
kevinwilfong)

[kevinwilfong] HIVE-3695. TestParse breaks due to HIVE-3675. (njain via 
kevinwilfong)


Changes for Build #1788
[kevinwilfong] HIVE-3557. Access to external URLs in hivetest.py. (Ivan 
Gorbachev via kevinwilfong)


Changes for Build #1789
[hashutosh] HIVE-3662 : TestHiveServer: testScratchDirShouldClearWhileStartup 
is failing on Windows (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3659 : TestHiveHistory::testQueryloglocParentDirNotExist Test 
fails on Windows because of some resource leaks in ZK (Kanna Karanam via 
Ashutosh Chauhan)

[hashutosh] HIVE-3663 Unable to display the MR Job file path on Windows in case 
of MR job failures.  (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1790

Changes for Build #1791

Changes for Build #1792

Changes for Build #1793
[hashutosh] HIVE-3704 : name of some metastore scripts are not per convention 
(Ashutosh Chauhan)


Changes for Build #1794
[hashutosh] HIVE-3243 : ignore white space between entries of hive/hbase table 
mapping (Shengsheng Huang via Ashutosh Chauhan)

[hashutosh] HIVE-3215 : JobDebugger should use RunningJob.getTrackingURL 
(Bhushan Mandhani via Ashutosh Chauhan)


Changes for Build #1795
[cws] HIVE-3437. 0.23 compatibility: fix unit tests when building against 0.23 
(Chris Drome via cws)

[hashutosh] HIVE-3626 : RetryingHMSHandler should wrap JDOException inside 
MetaException (Bhushan Mandhani via Ashutosh Chauhan)

[hashutosh] HIVE-3560 : Hive always prints a warning message when using remote 
metastore (Travis Crawford via Ashutosh Chauhan)


Changes for Build #1796

Changes for Build #1797
[hashutosh] HIVE-3664 : Avoid to create a symlink for hive-contrib.jar file in 
dist\lib folder. (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1798
[namit] HIVE-3706 getBoolVar in FileSinkOperator can be optimi

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #206

[Apache Jenkins Server]

See 


--
[...truncated 10309 lines...]

compile-test:
 [echo] Project: serde
[javac] Compiling 26 source files to 

[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

create-dirs:
 [echo] Project: service
 [copy] Warning: 

 does not exist.

init:
 [echo] Project: service

ivy-init-settings:
 [echo] Project: service

ivy-resolve:
 [echo] Project: service
[ivy:resolve] :: loading settings :: file = 

[ivy:report] Processing 

 to 


ivy-retrieve:
 [echo] Project: service

compile:
 [echo] Project: service

ivy-resolve-test:
 [echo] Project: service

ivy-retrieve-test:
 [echo] Project: service

compile-test:
 [echo] Project: service
[javac] Compiling 2 source files to 


test:
 [echo] Project: hive

test-shims:
 [echo] Project: hive

test-conditions:
 [echo] Project: shims

gen-test:
 [echo] Project: shims

create-dirs:
 [echo] Project: shims
 [copy] Warning: 

 does not exist.

init:
 [echo] Project: shims

ivy-init-settings:
 [echo] Project: shims

ivy-resolve:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 

[ivy:report] Processing 

 to 


ivy-retrieve:
 [echo] Project: shims

compile:
 [echo] Project: shims
 [echo] Building shims 0.20

build_shims:
 [echo] Project: shims
 [echo] Compiling 

 against hadoop 0.20.2 
(

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 


ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.20S

build_shims:
 [echo] Project: shims
 [echo] Compiling 

 against hadoop 1.0.0 
(

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 


ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.23

build_shims:
 [echo] Project: shims
 [echo] Compiling 

 against hadoop 0.23.3 
(

[jira] [Updated] (HIVE-3073) Hive List Bucketing - DML support

[Gang Tim Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Tim Liu updated HIVE-3073:
---

Attachment: HIVE-3073.patch.19

> Hive List Bucketing - DML support 
> --
>
> Key: HIVE-3073
> URL: https://issues.apache.org/jira/browse/HIVE-3073
> Project: Hive
>  Issue Type: New Feature
>  Components: SQL
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-3073.patch.12, HIVE-3073.patch.13, 
> HIVE-3073.patch.15, HIVE-3073.patch.18, HIVE-3073.patch.19
>
>
> If a hive table column has skewed keys, query performance on non-skewed key 
> is always impacted. Hive List Bucketing feature will address it:
> https://cwiki.apache.org/Hive/listbucketing.html
> This jira issue will track DML change for the feature:
> 1. single skewed column
> 2. manual load data

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #206

[Apache Jenkins Server]

See 

--
[...truncated 36947 lines...]
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/tmp/jenkins/hive_2012-11-22_12-56-35_772_8338557949700474160/-mr-1
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: load data local inpath 
'
 into table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] Copying data from 

[junit] Loading data to table default.testhivedrivertable
[junit] Copying file: 

[junit] POSTHOOK: query: load data local inpath 
'
 into table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: select * from testhivedrivertable limit 10
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: 
file:/tmp/jenkins/hive_2012-11-22_12-56-40_122_966010130315422314/-mr-1
[junit] POSTHOOK: query: select * from testhivedrivertable limit 10
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/tmp/jenkins/hive_2012-11-22_12-56-40_122_966010130315422314/-mr-1
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (key i

[jira] [Commented] (HIVE-3721) ALTER TABLE ADD PARTS should check for valid partition spec and throw a SemanticException if part spec is not valid

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3721?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502935#comment-13502935
 ] 

Hudson commented on HIVE-3721:
--

Integrated in Hive-trunk-h0.21 #1812 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1812/])
HIVE-3721 ALTER TABLE ADD PARTS should check for valid partition spec and 
throw a SemanticException
if part spec is not valid (Pamela Vagata via namit) (Revision 1412432)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412432
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
* /hive/trunk/ql/src/test/queries/clientnegative/alter_table_add_partition.q
* /hive/trunk/ql/src/test/results/clientnegative/alter_table_add_partition.q.out


> ALTER TABLE ADD PARTS should check for valid partition spec and throw a 
> SemanticException if part spec is not valid
> ---
>
> Key: HIVE-3721
> URL: https://issues.apache.org/jira/browse/HIVE-3721
> Project: Hive
>  Issue Type: Task
>Reporter: Pamela Vagata
>Assignee: Pamela Vagata
>Priority: Minor
> Fix For: 0.10.0
>
> Attachments: HIVE-3721.1.patch.txt, HIVE-3721.2.patch.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3703) Hive Query Explain Plan JSON not being created properly

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13502936#comment-13502936
 ] 

Hudson commented on HIVE-3703:
--

Integrated in Hive-trunk-h0.21 #1812 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1812/])
HIVE-3703 Hive Query Explain Plan JSON not being created properly
(Mayank Garg via namit) (Revision 1412465)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412465
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java
* /hive/trunk/ql/src/test/queries/clientpositive/plan_json.q
* /hive/trunk/ql/src/test/results/clientnegative/alter_view_failure5.q.out
* /hive/trunk/ql/src/test/results/clientnegative/alter_view_failure7.q.out
* /hive/trunk/ql/src/test/results/clientpositive/plan_json.q.out


> Hive Query Explain Plan JSON not being created properly
> ---
>
> Key: HIVE-3703
> URL: https://issues.apache.org/jira/browse/HIVE-3703
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Mayank Garg
>Assignee: Mayank Garg
>Priority: Minor
> Attachments: HIVE-3703.2.patch.txt
>
>   Original Estimate: 12h
>  Remaining Estimate: 12h
>
> There is an option to generate a JSON query plan for the hive query, however, 
> the JSON being created is invalid and json_decoders are unable to decode it

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1812 - Still Failing

[Apache Jenkins Server]

Changes for Build #1769

Changes for Build #1770
[namit] HIVE-3570 Add/fix facility to collect operator specific statisticsin 
hive + add hash-in/hash-out
counter for GroupBy Optr (Satadru Pan via namit)

[namit] HIVE-3554 Hive List Bucketing - Query logic
(Gang Tim Liu via namit)

[cws] HIVE-3563. Drop database cascade fails when there are indexes on any 
tables (Prasad Mujumdar via cws)


Changes for Build #1771
[kevinwilfong] HIVE-3640. Reducer allocation is incorrect if enforce bucketing 
and mapred.reduce.tasks are both set. (Vighnesh Avadhani via kevinwilfong)


Changes for Build #1772

Changes for Build #1773

Changes for Build #1774

Changes for Build #1775
[namit] HIVE-3673 Sort merge join not used when join columns have different 
names
(Kevin Wilfong via namit)


Changes for Build #1776
[kevinwilfong] HIVE-3627. eclipse misses library: 
javolution-@javolution-version@.jar. (Gang Tim Liu via kevinwilfong)


Changes for Build #1777
[kevinwilfong] HIVE-3524. Storing certain Exception objects thrown in 
HiveMetaStore.java in MetaStoreEndFunctionContext. (Maheshwaran Srinivasan via 
kevinwilfong)

[cws] HIVE-1977. DESCRIBE TABLE syntax doesn't support specifying a database 
qualified table name (Zhenxiao Luo via cws)

[cws] HIVE-3674. Test case TestParse broken after recent checkin (Sambavi 
Muthukrishnan via cws)


Changes for Build #1778
[cws] HIVE-1362. Column level scalar valued statistics on Tables and Partitions 
(Shreepadma Venugopalan via cws)


Changes for Build #1779

Changes for Build #1780
[kevinwilfong] HIVE-3686. Fix compile errors introduced by the interaction of 
HIVE-1362 and HIVE-3524. (Shreepadma Venugopalan via kevinwilfong)


Changes for Build #1781
[namit] HIVE-3687 smb_mapjoin_13.q is nondeterministic
(Kevin Wilfong via namit)


Changes for Build #1782
[hashutosh] HIVE-2715: Upgrade Thrift dependency to 0.9.0 (Ashutosh Chauhan)


Changes for Build #1783
[kevinwilfong] HIVE-3654. block relative path access in hive. (njain via 
kevinwilfong)

[hashutosh] HIVE-3658 : Unable to generate the Hbase related unit tests using 
velocity templates on Windows (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3661 : Remove the Windows specific = related swizzle path 
changes from Proxy FileSystems (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3480 : : Fix the file handle leaks in Symbolic 
& Symlink related input formats. (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1784
[kevinwilfong] HIVE-3675. NaN does not work correctly for round(n). (njain via 
kevinwilfong)

[cws] HIVE-3651. bucketmapjoin?.q tests fail with hadoop 0.23 (Prasad Mujumdar 
via cws)


Changes for Build #1785
[namit] HIVE-3613 Implement grouping_id function
(Ian Gorbachev via namit)

[namit] HIVE-3692 Update parallel test documentation
(Ivan Gorbachev via namit)

[namit] HIVE-3649 Hive List Bucketing - enhance DDL to specify list bucketing 
table
(Gang Tim Liu via namit)


Changes for Build #1786
[namit] HIVE-3696 Revert HIVE-3483 which causes performance regression
(Gang Tim Liu via namit)


Changes for Build #1787
[kevinwilfong] HIVE-3621. Make prompt in Hive CLI configurable. (Jingwei Lu via 
kevinwilfong)

[kevinwilfong] HIVE-3695. TestParse breaks due to HIVE-3675. (njain via 
kevinwilfong)


Changes for Build #1788
[kevinwilfong] HIVE-3557. Access to external URLs in hivetest.py. (Ivan 
Gorbachev via kevinwilfong)


Changes for Build #1789
[hashutosh] HIVE-3662 : TestHiveServer: testScratchDirShouldClearWhileStartup 
is failing on Windows (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3659 : TestHiveHistory::testQueryloglocParentDirNotExist Test 
fails on Windows because of some resource leaks in ZK (Kanna Karanam via 
Ashutosh Chauhan)

[hashutosh] HIVE-3663 Unable to display the MR Job file path on Windows in case 
of MR job failures.  (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1790

Changes for Build #1791

Changes for Build #1792

Changes for Build #1793
[hashutosh] HIVE-3704 : name of some metastore scripts are not per convention 
(Ashutosh Chauhan)


Changes for Build #1794
[hashutosh] HIVE-3243 : ignore white space between entries of hive/hbase table 
mapping (Shengsheng Huang via Ashutosh Chauhan)

[hashutosh] HIVE-3215 : JobDebugger should use RunningJob.getTrackingURL 
(Bhushan Mandhani via Ashutosh Chauhan)


Changes for Build #1795
[cws] HIVE-3437. 0.23 compatibility: fix unit tests when building against 0.23 
(Chris Drome via cws)

[hashutosh] HIVE-3626 : RetryingHMSHandler should wrap JDOException inside 
MetaException (Bhushan Mandhani via Ashutosh Chauhan)

[hashutosh] HIVE-3560 : Hive always prints a warning message when using remote 
metastore (Travis Crawford via Ashutosh Chauhan)


Changes for Build #1796

Changes for Build #1797
[hashutosh] HIVE-3664 : Avoid to create a symlink for hive-contrib.jar file in 
dist\lib folder. (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1798
[namit] HIVE-3706 getBoolVar in FileSinkOperator can be optimi

[jira] [Resolved] (HIVE-3737) show partitions command do not accept dbname

[Sujesh Chirackkal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sujesh Chirackkal resolved HIVE-3737.
-

  Resolution: Duplicate
Release Note: This is addressed in HIVE-3589 now along with describe 
command.

HIVE-3589 was raised for describe. Now this is addressed along with the same 
issue.

> show partitions command do not accept dbname
> 
>
> Key: HIVE-3737
> URL: https://issues.apache.org/jira/browse/HIVE-3737
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI
>Affects Versions: 0.8.1
>Reporter: Sujesh Chirackkal
>Priority: Minor
>
> show partitions dbname.tablename throws error "mimatched input . expecting 
> EOF near dbname'
> Summary; command accpets only table name as input. Similar issue was reported 
> for describe command also. (Hive-3589)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3734) Static partition DML create duplicate files and records

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503005#comment-13503005
 ] 

Namit Jain commented on HIVE-3734:
--


I dont see any list bucketing here.
testtable is a normal table - not stored as sub-directories.

Are you saying this problem exists in hive trunk ?

> Static partition DML create duplicate files and records
> ---
>
> Key: HIVE-3734
> URL: https://issues.apache.org/jira/browse/HIVE-3734
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>
> Static DML create duplicate files and record.
> Given the following test case, hive will return 2 records:
> 484   val_484
> 484   val_484
> but srcpart returns one record:
> 484   val_484
> If you look at file system, DML generates duplicate file with the same 
> content:
> -rw-r--r-- 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 00_0
> -rwxr-xr-x 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 01_0
> Test Case
> ===
> set hive.mapred.supports.subdirectories=true;
> set hive.exec.dynamic.partition=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
> set hive.merge.mapfiles=false;  
> set hive.merge.mapredfiles=false;
> set mapred.input.dir.recursive=true;
> create table testtable (key String, value String) partitioned by (ds String, 
> hr String) ;
> -- list bucketing DML
> explain extended
> insert overwrite table testtable partition (ds='2008-04-08', hr='11') select 
> key, value from srcpart where ds='2008-04-08';
> insert overwrite table testtable partition (ds='2008-04-08', hr='11') select 
> key, value from srcpart where ds='2008-04-08';
> -- check DML result
> desc formatted testtable partition (ds='2008-04-08', hr='11');
> select count(1) from srcpart where ds='2008-04-08';
> select count(1) from testtable where ds='2008-04-08';
> select key, value from srcpart where ds='2008-04-08' and hr='11' and key = 
> "484";
> set hive.optimize.listbucketing=true;
> explain extended
> select key, value from testtable where ds='2008-04-08' and hr='11' and key = 
> "484";
> select key, value from testtable where ds='2008-04-08' and hr='11' and key = 
> "484";
> ===

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3650) Hive List Bucketing - validation

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503006#comment-13503006
 ] 

Namit Jain commented on HIVE-3650:
--

When you have time, can you document which validations are remaining from the 
list above ?

> Hive List Bucketing - validation
> 
>
> Key: HIVE-3650
> URL: https://issues.apache.org/jira/browse/HIVE-3650
> Project: Hive
>  Issue Type: New Feature
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>Priority: Minor
>
> Many validations are done in each patch. This issue tracks left-over from 
> complete list
> https://cwiki.apache.org/confluence/display/Hive/ListBucketing

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3741) Driver.validateConfVariables() should perform more validations

[Namit Jain (JIRA)]

Namit Jain created HIVE-3741:


 Summary: Driver.validateConfVariables() should perform more 
validations
 Key: HIVE-3741
 URL: https://issues.apache.org/jira/browse/HIVE-3741
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Namit Jain


Like List Bucketing, it should also check for HIVE_OPTIMIZE_UNION_REMOVE.


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-3698) enhanceModel.notRequired is incorrectly determined

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-3698.


   Resolution: Fixed
Fix Version/s: 0.9.1

Committed to 0.9 branch and trunk.

> enhanceModel.notRequired is incorrectly determined
> --
>
> Key: HIVE-3698
> URL: https://issues.apache.org/jira/browse/HIVE-3698
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure, Metastore
>Affects Versions: 0.10.0, 0.9.1
>Reporter: Chris Drome
>Assignee: Chris Drome
> Fix For: 0.9.1
>
> Attachments: HIVE-3698-0.9.patch, HIVE-3698-trunk.patch
>
>
> enhanceModel.notRequired in metastore/build.xml is not correctly determined 
> which can result in datanucleus running twice. This in turn can cause other 
> issues when building with JDK7.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3680) Include Table information in Hive's AddPartitionEvent.

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-3680:
---

Fix Version/s: 0.9.1

> Include Table information in Hive's AddPartitionEvent.
> --
>
> Key: HIVE-3680
> URL: https://issues.apache.org/jira/browse/HIVE-3680
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.9.1
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-3680.branch9.patch, HIVE-3680.trunk.patch
>
>
> This has to do with a minor overhaul of the HCatalog notifications that we're 
> attempting in HCATALOG-546.
> It is proposed that HCatalog's notifications (on Add/Drop of Partitions) 
> provide details to identify the affected partitions. 
> Using the Partition object in AddPartitionEvent, one is able to retrieve the 
> values of the partition-keys and the name of the Table. However, the 
> partition-keys themselves aren't available (since the Table instance isn't 
> part of the AddPartitionEvent).
> Adding the table-reference to the AddPartitionEvent and DropPartitionEvent 
> classes will expose all the info we need. (The alternative is to query the 
> metastore for the table's schema and use the partition-keys from there. :/)
> I'll post a patch for this shortly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3680) Include Table information in Hive's AddPartitionEvent.

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503016#comment-13503016
 ] 

Ashutosh Chauhan commented on HIVE-3680:


Committed to 0.9 branch as well.

> Include Table information in Hive's AddPartitionEvent.
> --
>
> Key: HIVE-3680
> URL: https://issues.apache.org/jira/browse/HIVE-3680
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.9.1
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-3680.branch9.patch, HIVE-3680.trunk.patch
>
>
> This has to do with a minor overhaul of the HCatalog notifications that we're 
> attempting in HCATALOG-546.
> It is proposed that HCatalog's notifications (on Add/Drop of Partitions) 
> provide details to identify the affected partitions. 
> Using the Partition object in AddPartitionEvent, one is able to retrieve the 
> values of the partition-keys and the name of the Table. However, the 
> partition-keys themselves aren't available (since the Table instance isn't 
> part of the AddPartitionEvent).
> Adding the table-reference to the AddPartitionEvent and DropPartitionEvent 
> classes will expose all the info we need. (The alternative is to query the 
> metastore for the table's schema and use the partition-keys from there. :/)
> I'll post a patch for this shortly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3722) Create index fails on CLI using remote metastore

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503019#comment-13503019
 ] 

Ashutosh Chauhan commented on HIVE-3722:


Committed to 0.10 branch as well.

> Create index fails on CLI using remote metastore
> 
>
> Key: HIVE-3722
> URL: https://issues.apache.org/jira/browse/HIVE-3722
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Fix For: 0.10.0
>
> Attachments: HIVE-3722.1.patch.txt
>
>
> If the CLI uses a remote metastore and the user attempts to create an index 
> without a comment, it will fail with a NPE.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-2800) NPE in "create index" without comment clause in external metastore

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-2800.


   Resolution: Fixed
Fix Version/s: 0.10.0

Fixed as a part of HIVE-3722

> NPE in "create index" without comment clause in external metastore
> --
>
> Key: HIVE-2800
> URL: https://issues.apache.org/jira/browse/HIVE-2800
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.7.1, 0.8.1, 0.9.0
>Reporter: Daniel Dai
>Assignee: Sushanth Sowmyan
>Priority: Minor
> Fix For: 0.10.0
>
>
> This happens only when using external metastore (with --hiveconf 
> hive.metastore.uris=thrift://localhost:8088 --hiveconf 
> hive.metastore.local=false). Also if I gave a comment in the statement, this 
> exception go away.
> Here is the statement:
> create index test111 on table hcat_test(name) as 'compact' with deferred 
> rebuild;
> Here is the stack:
> 2012-02-10 17:07:42,612 ERROR exec.Task (SessionState.java:printError(380)) - 
> FAILED: Error in metadata: java.lang.NullPointerException
> org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:725)
> at 
> org.apache.hadoop.hive.ql.exec.DDLTask.createIndex(DDLTask.java:822)
> at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:231)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:134)
> at 
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1291)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1082)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:933)
> at 
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:255)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:212)
> at 
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:671)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:554)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> Caused by: java.lang.NullPointerException
> at 
> org.apache.thrift.protocol.TBinaryProtocol.writeString(TBinaryProtocol.java:185)
> at org.apache.hadoop.hive.metastore.api.Index.write(Index.java:1032)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$add_index_args.write(ThriftHiveMetastore.java:47518)
> at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:63)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.send_add_index(ThriftHiveMetastore.java:1675)
> at 
> org.apache.hadoop.hive.metastore.api.ThriftHiveMetastore$Client.add_index(ThriftHiveMetastore.java:1666)
> at 
> org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createIndex(HiveMetaStoreClient.java:853)
> at org.apache.hadoop.hive.ql.metadata.Hive.createIndex(Hive.java:722)
> ... 17 more

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3723) Hive Driver leaks ZooKeeper connections

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503024#comment-13503024
 ] 

Ashutosh Chauhan commented on HIVE-3723:


Around line 910 if compile returns non-zero value, then also we 
releaseLocks(List) and then return. There also, we should call destroy() 
instead of releaseLocks.

> Hive Driver leaks ZooKeeper connections
> ---
>
> Key: HIVE-3723
> URL: https://issues.apache.org/jira/browse/HIVE-3723
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-3723.1-r1411423.patch
>
>
> In certain error cases (i.e.: statement fails to compile, semantic errors) 
> the hive driver leaks zookeeper connections.
> This can be seen in the TestNegativeCliDriver test which accumulates a large 
> number of open file handles and fails if the max allowed number of file 
> handles isn't at least 2048.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3723) Hive Driver leaks ZooKeeper connections

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-3723:
---

Status: Open  (was: Patch Available)

> Hive Driver leaks ZooKeeper connections
> ---
>
> Key: HIVE-3723
> URL: https://issues.apache.org/jira/browse/HIVE-3723
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-3723.1-r1411423.patch
>
>
> In certain error cases (i.e.: statement fails to compile, semantic errors) 
> the hive driver leaks zookeeper connections.
> This can be seen in the TestNegativeCliDriver test which accumulates a large 
> number of open file handles and fails if the max allowed number of file 
> handles isn't at least 2048.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3698) enhanceModel.notRequired is incorrectly determined

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503025#comment-13503025
 ] 

Hudson commented on HIVE-3698:
--

Integrated in Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #207 (See 
[https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/207/])
HIVE-3698 : enhanceModel.notRequired is incorrectly determined (Chris Drome 
via Ashutosh Chauhan) (Revision 1412749)

 Result = FAILURE
hashutosh : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412749
Files : 
* /hive/branches/branch-0.9/metastore/build.xml


> enhanceModel.notRequired is incorrectly determined
> --
>
> Key: HIVE-3698
> URL: https://issues.apache.org/jira/browse/HIVE-3698
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure, Metastore
>Affects Versions: 0.10.0, 0.9.1
>Reporter: Chris Drome
>Assignee: Chris Drome
> Fix For: 0.9.1
>
> Attachments: HIVE-3698-0.9.patch, HIVE-3698-trunk.patch
>
>
> enhanceModel.notRequired in metastore/build.xml is not correctly determined 
> which can result in datanucleus running twice. This in turn can cause other 
> issues when building with JDK7.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3680) Include Table information in Hive's AddPartitionEvent.

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503026#comment-13503026
 ] 

Hudson commented on HIVE-3680:
--

Integrated in Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #207 (See 
[https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/207/])
HIVE-3680 : Include Table information in Hive's AddPartitionEvent. (Mithun 
RadhaKrishnan via Ashutosh Chauhan) (Revision 1412750)

 Result = FAILURE
hashutosh : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1412750
Files : 
* 
/hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java
* 
/hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/events/AddPartitionEvent.java
* 
/hive/branches/branch-0.9/metastore/src/java/org/apache/hadoop/hive/metastore/events/DropPartitionEvent.java
* 
/hive/branches/branch-0.9/metastore/src/test/org/apache/hadoop/hive/metastore/TestMetaStoreEventListener.java


> Include Table information in Hive's AddPartitionEvent.
> --
>
> Key: HIVE-3680
> URL: https://issues.apache.org/jira/browse/HIVE-3680
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.9.1
>Reporter: Mithun Radhakrishnan
>Assignee: Mithun Radhakrishnan
> Fix For: 0.10.0, 0.9.1
>
> Attachments: HIVE-3680.branch9.patch, HIVE-3680.trunk.patch
>
>
> This has to do with a minor overhaul of the HCatalog notifications that we're 
> attempting in HCATALOG-546.
> It is proposed that HCatalog's notifications (on Add/Drop of Partitions) 
> provide details to identify the affected partitions. 
> Using the Partition object in AddPartitionEvent, one is able to retrieve the 
> values of the partition-keys and the name of the Table. However, the 
> partition-keys themselves aren't available (since the Table instance isn't 
> part of the AddPartitionEvent).
> Adding the table-reference to the AddPartitionEvent and DropPartitionEvent 
> classes will expose all the info we need. (The alternative is to query the 
> metastore for the table's schema and use the partition-keys from there. :/)
> I'll post a patch for this shortly.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #207

[Apache Jenkins Server]

See 


Changes:

[hashutosh] HIVE-3680 : Include Table information in Hive's AddPartitionEvent. 
(Mithun RadhaKrishnan via Ashutosh Chauhan)

[hashutosh] HIVE-3698 : enhanceModel.notRequired is incorrectly determined 
(Chris Drome via Ashutosh Chauhan)

--
[...truncated 9914 lines...]
 [echo] Project: serde
[javac] Compiling 26 source files to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/serde/test/classes
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

create-dirs:
 [echo] Project: service
 [copy] Warning: 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/service/src/test/resources
 does not exist.

init:
 [echo] Project: service

ivy-init-settings:
 [echo] Project: service

ivy-resolve:
 [echo] Project: service
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml
[ivy:report] Processing 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-service-default.xml
 to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-service-default.html

ivy-retrieve:
 [echo] Project: service

compile:
 [echo] Project: service

ivy-resolve-test:
 [echo] Project: service

ivy-retrieve-test:
 [echo] Project: service

compile-test:
 [echo] Project: service
[javac] Compiling 2 source files to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/service/test/classes

test:
 [echo] Project: hive

test-shims:
 [echo] Project: hive

test-conditions:
 [echo] Project: shims

gen-test:
 [echo] Project: shims

create-dirs:
 [echo] Project: shims
 [copy] Warning: 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/test/resources
 does not exist.

init:
 [echo] Project: shims

ivy-init-settings:
 [echo] Project: shims

ivy-resolve:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml
[ivy:report] Processing 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml
 to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-shims-default.html

ivy-retrieve:
 [echo] Project: shims

compile:
 [echo] Project: shims
 [echo] Building shims 0.20

build_shims:
 [echo] Project: shims
 [echo] Compiling 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java
 against hadoop 0.20.2 
(/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hadoopcore/hadoop-0.20.2)

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml

ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.20S

build_shims:
 [echo] Project: shims
 [echo] Compiling 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common-secure/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20S/java
 against hadoop 1.0.0 
(/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hadoopcore/hadoop-1.0.0)

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml

ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.23

build_shims:
 [echo] Project: shims
 [echo] Compiling 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common-secure/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SN

Hive-trunk-h0.21 - Build # 1813 - Still Failing

[Apache Jenkins Server]

Changes for Build #1770
[namit] HIVE-3570 Add/fix facility to collect operator specific statisticsin 
hive + add hash-in/hash-out
counter for GroupBy Optr (Satadru Pan via namit)

[namit] HIVE-3554 Hive List Bucketing - Query logic
(Gang Tim Liu via namit)

[cws] HIVE-3563. Drop database cascade fails when there are indexes on any 
tables (Prasad Mujumdar via cws)


Changes for Build #1771
[kevinwilfong] HIVE-3640. Reducer allocation is incorrect if enforce bucketing 
and mapred.reduce.tasks are both set. (Vighnesh Avadhani via kevinwilfong)


Changes for Build #1772

Changes for Build #1773

Changes for Build #1774

Changes for Build #1775
[namit] HIVE-3673 Sort merge join not used when join columns have different 
names
(Kevin Wilfong via namit)


Changes for Build #1776
[kevinwilfong] HIVE-3627. eclipse misses library: 
javolution-@javolution-version@.jar. (Gang Tim Liu via kevinwilfong)


Changes for Build #1777
[kevinwilfong] HIVE-3524. Storing certain Exception objects thrown in 
HiveMetaStore.java in MetaStoreEndFunctionContext. (Maheshwaran Srinivasan via 
kevinwilfong)

[cws] HIVE-1977. DESCRIBE TABLE syntax doesn't support specifying a database 
qualified table name (Zhenxiao Luo via cws)

[cws] HIVE-3674. Test case TestParse broken after recent checkin (Sambavi 
Muthukrishnan via cws)


Changes for Build #1778
[cws] HIVE-1362. Column level scalar valued statistics on Tables and Partitions 
(Shreepadma Venugopalan via cws)


Changes for Build #1779

Changes for Build #1780
[kevinwilfong] HIVE-3686. Fix compile errors introduced by the interaction of 
HIVE-1362 and HIVE-3524. (Shreepadma Venugopalan via kevinwilfong)


Changes for Build #1781
[namit] HIVE-3687 smb_mapjoin_13.q is nondeterministic
(Kevin Wilfong via namit)


Changes for Build #1782
[hashutosh] HIVE-2715: Upgrade Thrift dependency to 0.9.0 (Ashutosh Chauhan)


Changes for Build #1783
[kevinwilfong] HIVE-3654. block relative path access in hive. (njain via 
kevinwilfong)

[hashutosh] HIVE-3658 : Unable to generate the Hbase related unit tests using 
velocity templates on Windows (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3661 : Remove the Windows specific = related swizzle path 
changes from Proxy FileSystems (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3480 : : Fix the file handle leaks in Symbolic 
& Symlink related input formats. (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1784
[kevinwilfong] HIVE-3675. NaN does not work correctly for round(n). (njain via 
kevinwilfong)

[cws] HIVE-3651. bucketmapjoin?.q tests fail with hadoop 0.23 (Prasad Mujumdar 
via cws)


Changes for Build #1785
[namit] HIVE-3613 Implement grouping_id function
(Ian Gorbachev via namit)

[namit] HIVE-3692 Update parallel test documentation
(Ivan Gorbachev via namit)

[namit] HIVE-3649 Hive List Bucketing - enhance DDL to specify list bucketing 
table
(Gang Tim Liu via namit)


Changes for Build #1786
[namit] HIVE-3696 Revert HIVE-3483 which causes performance regression
(Gang Tim Liu via namit)


Changes for Build #1787
[kevinwilfong] HIVE-3621. Make prompt in Hive CLI configurable. (Jingwei Lu via 
kevinwilfong)

[kevinwilfong] HIVE-3695. TestParse breaks due to HIVE-3675. (njain via 
kevinwilfong)


Changes for Build #1788
[kevinwilfong] HIVE-3557. Access to external URLs in hivetest.py. (Ivan 
Gorbachev via kevinwilfong)


Changes for Build #1789
[hashutosh] HIVE-3662 : TestHiveServer: testScratchDirShouldClearWhileStartup 
is failing on Windows (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3659 : TestHiveHistory::testQueryloglocParentDirNotExist Test 
fails on Windows because of some resource leaks in ZK (Kanna Karanam via 
Ashutosh Chauhan)

[hashutosh] HIVE-3663 Unable to display the MR Job file path on Windows in case 
of MR job failures.  (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1790

Changes for Build #1791

Changes for Build #1792

Changes for Build #1793
[hashutosh] HIVE-3704 : name of some metastore scripts are not per convention 
(Ashutosh Chauhan)


Changes for Build #1794
[hashutosh] HIVE-3243 : ignore white space between entries of hive/hbase table 
mapping (Shengsheng Huang via Ashutosh Chauhan)

[hashutosh] HIVE-3215 : JobDebugger should use RunningJob.getTrackingURL 
(Bhushan Mandhani via Ashutosh Chauhan)


Changes for Build #1795
[cws] HIVE-3437. 0.23 compatibility: fix unit tests when building against 0.23 
(Chris Drome via cws)

[hashutosh] HIVE-3626 : RetryingHMSHandler should wrap JDOException inside 
MetaException (Bhushan Mandhani via Ashutosh Chauhan)

[hashutosh] HIVE-3560 : Hive always prints a warning message when using remote 
metastore (Travis Crawford via Ashutosh Chauhan)


Changes for Build #1796

Changes for Build #1797
[hashutosh] HIVE-3664 : Avoid to create a symlink for hive-contrib.jar file in 
dist\lib folder. (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1798
[namit] HIVE-3706 getBoolVar in FileSinkOperator can be optimized
(Kevin Wilfong via na

[jira] [Updated] (HIVE-3073) Hive List Bucketing - DML support

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3073:
-

Status: Open  (was: Patch Available)

comments on phabricator

> Hive List Bucketing - DML support 
> --
>
> Key: HIVE-3073
> URL: https://issues.apache.org/jira/browse/HIVE-3073
> Project: Hive
>  Issue Type: New Feature
>  Components: SQL
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-3073.patch.12, HIVE-3073.patch.13, 
> HIVE-3073.patch.15, HIVE-3073.patch.18, HIVE-3073.patch.19
>
>
> If a hive table column has skewed keys, query performance on non-skewed key 
> is always impacted. Hive List Bucketing feature will address it:
> https://cwiki.apache.org/Hive/listbucketing.html
> This jira issue will track DML change for the feature:
> 1. single skewed column
> 2. manual load data

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3734) Static partition DML create duplicate files and records

[Gang Tim Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503030#comment-13503030
 ] 

Gang Tim Liu commented on HIVE-3734:


Yes, per my test. 

We can double confirm.

Sent from my iPhone




> Static partition DML create duplicate files and records
> ---
>
> Key: HIVE-3734
> URL: https://issues.apache.org/jira/browse/HIVE-3734
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>
> Static DML create duplicate files and record.
> Given the following test case, hive will return 2 records:
> 484   val_484
> 484   val_484
> but srcpart returns one record:
> 484   val_484
> If you look at file system, DML generates duplicate file with the same 
> content:
> -rw-r--r-- 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 00_0
> -rwxr-xr-x 1 gang THEFACEBOOK\Domain Users 5812 Nov 21 17:55 01_0
> Test Case
> ===
> set hive.mapred.supports.subdirectories=true;
> set hive.exec.dynamic.partition=true;
> set hive.exec.dynamic.partition.mode=nonstrict;
> set hive.input.format=org.apache.hadoop.hive.ql.io.HiveInputFormat;
> set hive.merge.mapfiles=false;  
> set hive.merge.mapredfiles=false;
> set mapred.input.dir.recursive=true;
> create table testtable (key String, value String) partitioned by (ds String, 
> hr String) ;
> -- list bucketing DML
> explain extended
> insert overwrite table testtable partition (ds='2008-04-08', hr='11') select 
> key, value from srcpart where ds='2008-04-08';
> insert overwrite table testtable partition (ds='2008-04-08', hr='11') select 
> key, value from srcpart where ds='2008-04-08';
> -- check DML result
> desc formatted testtable partition (ds='2008-04-08', hr='11');
> select count(1) from srcpart where ds='2008-04-08';
> select count(1) from testtable where ds='2008-04-08';
> select key, value from srcpart where ds='2008-04-08' and hr='11' and key = 
> "484";
> set hive.optimize.listbucketing=true;
> explain extended
> select key, value from testtable where ds='2008-04-08' and hr='11' and key = 
> "484";
> select key, value from testtable where ds='2008-04-08' and hr='11' and key = 
> "484";
> ===

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3650) Hive List Bucketing - validation

[Gang Tim Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13503031#comment-13503031
 ] 

Gang Tim Liu commented on HIVE-3650:


Yes, I will update wiki.

Sent from my iPhone




> Hive List Bucketing - validation
> 
>
> Key: HIVE-3650
> URL: https://issues.apache.org/jira/browse/HIVE-3650
> Project: Hive
>  Issue Type: New Feature
>Affects Versions: 0.10.0
>Reporter: Gang Tim Liu
>Priority: Minor
>
> Many validations are done in each patch. This issue tracks left-over from 
> complete list
> https://cwiki.apache.org/confluence/display/Hive/ListBucketing

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira