date:20121109


[ 
https://issues.apache.org/jira/browse/HIVE-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493812#comment-13493812
 ] 

Namit Jain commented on HIVE-3696:
--

+1

 Revert HIVE-3483 which causes performance regression
 

 Key: HIVE-3696
 URL: https://issues.apache.org/jira/browse/HIVE-3696
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Gang Tim Liu
Priority: Critical
 Attachments: HIVE-3696.patch.1


 HIVE-3483 causes performance regression.
 We'd like to revert it first and find another solution for it later.
 This issue is to track revert and HIVE-3693 tracks subsequent solution.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3471) Implement grouping sets in hive


 [ 
https://issues.apache.org/jira/browse/HIVE-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3471:
-

Status: Open  (was: Patch Available)

Can you refresh, and recreate the patch ?

 Implement grouping sets in hive
 ---

 Key: HIVE-3471
 URL: https://issues.apache.org/jira/browse/HIVE-3471
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor
Reporter: Namit Jain
Assignee: Ivan Gorbachev
 Attachments: jira-3471.0.patch, jira-3471.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3675) NaN does not work correctly for round(n)


[ 
https://issues.apache.org/jira/browse/HIVE-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493827#comment-13493827
 ] 

Hudson commented on HIVE-3675:
--

Integrated in Hive-trunk-h0.21 #1784 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1784/])
HIVE-3675. NaN does not work correctly for round(n). (njain via 
kevinwilfong) (Revision 1407320)

 Result = FAILURE
kevinwilfong : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1407320
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFRound.java
* /hive/trunk/ql/src/test/queries/clientpositive/udf_round_2.q
* /hive/trunk/ql/src/test/results/clientpositive/udf4.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_round.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_round_2.q.out


 NaN does not work correctly for round(n)
 

 Key: HIVE-3675
 URL: https://issues.apache.org/jira/browse/HIVE-3675
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
 Fix For: 0.10.0

 Attachments: hive.3675.1.patch, hive.3675.2.patch, hive.3675.3.patch


 It works correctly for round(n, d)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3651) bucketmapjoin?.q tests fail with hadoop 0.23


[ 
https://issues.apache.org/jira/browse/HIVE-3651?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493826#comment-13493826
 ] 

Hudson commented on HIVE-3651:
--

Integrated in Hive-trunk-h0.21 #1784 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1784/])
HIVE-3651. bucketmapjoin?.q tests fail with hadoop 0.23 (Prasad Mujumdar 
via cws) (Revision 1407312)

 Result = FAILURE
cws : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1407312
Files : 
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/BucketMapJoinContext.java


 bucketmapjoin?.q  tests fail with hadoop 0.23
 -

 Key: HIVE-3651
 URL: https://issues.apache.org/jira/browse/HIVE-3651
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.10.0
Reporter: Prasad Mujumdar
Assignee: Prasad Mujumdar
 Fix For: 0.10.0

 Attachments: HIVE-3651-1.patch


 The hive.log show error in MR job -
 Task failed!
 Task ID:
   Stage-1
 The job log has following error -
 2012-11-01 15:51:20,253 WARN  mapred.LocalJobRunner 
 (LocalJobRunner.java:run(479)) - job_local_0001
 java.lang.Exception: java.lang.RuntimeException: 
 org.apache.hadoop.hive.ql.metadata.HiveException: 
 /home/prasadm/repos/apache/hive-patches/build/ql/scratchdir/local/hive_2012-11-01_15-51-06_176_6704298995984162430/-local-10003/HashTable-Stage-1/MapJoin-b-11-srcbucket21.txt.hashtable
  (No such file or directory)
 at 
 org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:400)
 Caused by: java.lang.RuntimeException: 
 org.apache.hadoop.hive.ql.metadata.HiveException: 
 /home/prasadm/repos/apache/hive-patches/build/ql/scratchdir/local/hive_2012-11-01_15-51-06_176_6704298995984162430/-local-10003/HashTable-Stage-1/MapJoin-b-11-srcbucket21.txt.hashtable
  (No such file or directory)
 at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
 at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:54)
 at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:399)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:334)
 at 
 org.apache.hadoop.mapred.LocalJobRunner$Job$MapTaskRunnable.run(LocalJobRunner.java:232)
 at 
 java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
 at java.util.concurrent.FutureTask.run(FutureTask.java:166)
 at 
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
 at 
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
 at java.lang.Thread.run(Thread.java:679)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1784 - Still Failing

Changes for Build #1764
[kevinwilfong] HIVE-3610. Add a command Explain dependency ... (Sambavi 
Muthukrishnan via kevinwilfong)


Changes for Build #1765

Changes for Build #1766
[hashutosh] HIVE-3441 : testcases escape1,escape2 fail on windows (Thejas Nair 
via Ashutosh Chauhan)

[kevinwilfong] HIVE-3499. add tests to use bucketing metadata for partitions. 
(njain via kevinwilfong)


Changes for Build #1767
[kevinwilfong] HIVE-3276. optimize union sub-queries. (njain via kevinwilfong)


Changes for Build #1768

Changes for Build #1769

Changes for Build #1770
[namit] HIVE-3570 Add/fix facility to collect operator specific statisticsin 
hive + add hash-in/hash-out
counter for GroupBy Optr (Satadru Pan via namit)

[namit] HIVE-3554 Hive List Bucketing - Query logic
(Gang Tim Liu via namit)

[cws] HIVE-3563. Drop database cascade fails when there are indexes on any 
tables (Prasad Mujumdar via cws)


Changes for Build #1771
[kevinwilfong] HIVE-3640. Reducer allocation is incorrect if enforce bucketing 
and mapred.reduce.tasks are both set. (Vighnesh Avadhani via kevinwilfong)


Changes for Build #1772

Changes for Build #1773

Changes for Build #1774

Changes for Build #1775
[namit] HIVE-3673 Sort merge join not used when join columns have different 
names
(Kevin Wilfong via namit)


Changes for Build #1776
[kevinwilfong] HIVE-3627. eclipse misses library: 
javolution-@javolution-version@.jar. (Gang Tim Liu via kevinwilfong)


Changes for Build #1777
[kevinwilfong] HIVE-3524. Storing certain Exception objects thrown in 
HiveMetaStore.java in MetaStoreEndFunctionContext. (Maheshwaran Srinivasan via 
kevinwilfong)

[cws] HIVE-1977. DESCRIBE TABLE syntax doesn't support specifying a database 
qualified table name (Zhenxiao Luo via cws)

[cws] HIVE-3674. Test case TestParse broken after recent checkin (Sambavi 
Muthukrishnan via cws)


Changes for Build #1778
[cws] HIVE-1362. Column level scalar valued statistics on Tables and Partitions 
(Shreepadma Venugopalan via cws)


Changes for Build #1779

Changes for Build #1780
[kevinwilfong] HIVE-3686. Fix compile errors introduced by the interaction of 
HIVE-1362 and HIVE-3524. (Shreepadma Venugopalan via kevinwilfong)


Changes for Build #1781
[namit] HIVE-3687 smb_mapjoin_13.q is nondeterministic
(Kevin Wilfong via namit)


Changes for Build #1782
[hashutosh] HIVE-2715: Upgrade Thrift dependency to 0.9.0 (Ashutosh Chauhan)


Changes for Build #1783
[kevinwilfong] HIVE-3654. block relative path access in hive. (njain via 
kevinwilfong)

[hashutosh] HIVE-3658 : Unable to generate the Hbase related unit tests using 
velocity templates on Windows (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3661 : Remove the Windows specific = related swizzle path 
changes from Proxy FileSystems (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3480 : Resource leak: Fix the file handle leaks in Symbolic 
 Symlink related input formats. (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1784
[kevinwilfong] HIVE-3675. NaN does not work correctly for round(n). (njain via 
kevinwilfong)

[cws] HIVE-3651. bucketmapjoin?.q tests fail with hadoop 0.23 (Prasad Mujumdar 
via cws)




4 tests failed.
REGRESSION:  
org.apache.hadoop.hive.metastore.TestMetaStoreEndFunctionListener.testEndFunctionListener

Error Message:
-1

Stack Trace:
java.lang.ArrayIndexOutOfBoundsException: -1
at java.util.ArrayList.get(ArrayList.java:324)
at 
org.apache.hadoop.hive.metastore.TestMetaStoreEndFunctionListener.testEndFunctionListener(TestMetaStoreEndFunctionListener.java:99)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.run(JUnitTestRunner.java:518)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.launch(JUnitTestRunner.java:1052)
at 
org.apache.tools.ant.taskdefs.optional.junit.JUnitTestRunner.main(JUnitTestRunner.java:906)


FAILED:  
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_stats_aggregator_error_1

Error Message:
Forked Java VM exited abnormally. Please note the time in the report does not 
reflect the time until the VM exit.

Stack Trace:

[jira] [Updated] (HIVE-3073) Hive List Bucketing - DML support


 [ 
https://issues.apache.org/jira/browse/HIVE-3073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3073:
-

Status: Open  (was: Patch Available)

comments

 Hive List Bucketing - DML support 
 --

 Key: HIVE-3073
 URL: https://issues.apache.org/jira/browse/HIVE-3073
 Project: Hive
  Issue Type: New Feature
  Components: SQL
Affects Versions: 0.10.0
Reporter: Gang Tim Liu
Assignee: Gang Tim Liu
 Attachments: HIVE-3073.patch.12, HIVE-3073.patch.13


 If a hive table column has skewed keys, query performance on non-skewed key 
 is always impacted. Hive List Bucketing feature will address it:
 https://cwiki.apache.org/Hive/listbucketing.html
 This jira issue will track DML change for the feature:
 1. single skewed column
 2. manual load data

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3647) map-side groupby wrongly due to HIVE-3432


 [ 
https://issues.apache.org/jira/browse/HIVE-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3647:
-

Attachment: hive.3647.6.patch

 map-side groupby wrongly due to HIVE-3432
 -

 Key: HIVE-3647
 URL: https://issues.apache.org/jira/browse/HIVE-3647
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
 Attachments: hive.3647.1.patch, hive.3647.2.patch, hive.3647.3.patch, 
 hive.3647.4.patch, hive.3647.5.patch, hive.3647.6.patch


 There seems to be a bug due to HIVE-3432.
 We are converting the group by to a map side group by after only looking at
 sorting columns. This can give wrong results if the data is sorted and
 bucketed by different columns.
 Add some tests for that scenario, verify and fix any issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3647) map-side groupby wrongly due to HIVE-3432


 [ 
https://issues.apache.org/jira/browse/HIVE-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3647:
-

Attachment: hive.3647.7.patch

 map-side groupby wrongly due to HIVE-3432
 -

 Key: HIVE-3647
 URL: https://issues.apache.org/jira/browse/HIVE-3647
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
 Attachments: hive.3647.1.patch, hive.3647.2.patch, hive.3647.3.patch, 
 hive.3647.4.patch, hive.3647.5.patch, hive.3647.6.patch, hive.3647.7.patch


 There seems to be a bug due to HIVE-3432.
 We are converting the group by to a map side group by after only looking at
 sorting columns. This can give wrong results if the data is sorted and
 bucketed by different columns.
 Add some tests for that scenario, verify and fix any issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3647) map-side groupby wrongly due to HIVE-3432


[ 
https://issues.apache.org/jira/browse/HIVE-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493842#comment-13493842
 ] 

Namit Jain commented on HIVE-3647:
--

addressed comments from Mark.

 map-side groupby wrongly due to HIVE-3432
 -

 Key: HIVE-3647
 URL: https://issues.apache.org/jira/browse/HIVE-3647
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
 Attachments: hive.3647.1.patch, hive.3647.2.patch, hive.3647.3.patch, 
 hive.3647.4.patch, hive.3647.5.patch, hive.3647.6.patch, hive.3647.7.patch


 There seems to be a bug due to HIVE-3432.
 We are converting the group by to a map side group by after only looking at
 sorting columns. This can give wrong results if the data is sorted and
 bucketed by different columns.
 Add some tests for that scenario, verify and fix any issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3598) physical optimizer changes for auto sort-merge join


[ 
https://issues.apache.org/jira/browse/HIVE-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493844#comment-13493844
 ] 

Namit Jain commented on HIVE-3598:
--

Done as part of HIVE-3403

 physical optimizer changes for auto sort-merge join
 ---

 Key: HIVE-3598
 URL: https://issues.apache.org/jira/browse/HIVE-3598
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
 Fix For: 0.10.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-3598) physical optimizer changes for auto sort-merge join


 [ 
https://issues.apache.org/jira/browse/HIVE-3598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain resolved HIVE-3598.
--

   Resolution: Fixed
Fix Version/s: 0.10.0
 Assignee: Namit Jain
 Hadoop Flags: Reviewed

 physical optimizer changes for auto sort-merge join
 ---

 Key: HIVE-3598
 URL: https://issues.apache.org/jira/browse/HIVE-3598
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
Assignee: Namit Jain
 Fix For: 0.10.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-3681) Compile errors after HIVE-1362


 [ 
https://issues.apache.org/jira/browse/HIVE-3681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain resolved HIVE-3681.
--

   Resolution: Fixed
Fix Version/s: 0.10.0
 Hadoop Flags: Reviewed

 Compile errors after HIVE-1362
 --

 Key: HIVE-3681
 URL: https://issues.apache.org/jira/browse/HIVE-3681
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
 Fix For: 0.10.0


 Compile errors after HIVE-1362

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-3633) sort-merge join does not work with sub-queries


 [ 
https://issues.apache.org/jira/browse/HIVE-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain reassigned HIVE-3633:


Assignee: Namit Jain

 sort-merge join does not work with sub-queries
 --

 Key: HIVE-3633
 URL: https://issues.apache.org/jira/browse/HIVE-3633
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain

 Consider the following query:
 create table smb_bucket_1(key int, value string) CLUSTERED BY (key) SORTED BY 
 (key) INTO 6 BUCKETS STORED AS TEXTFILE;
 create table smb_bucket_2(key int, value string) CLUSTERED BY (key) SORTED BY 
 (key) INTO 6 BUCKETS STORED AS TEXTFILE;
 -- load the above tables
 set hive.optimize.bucketmapjoin = true;
 set hive.optimize.bucketmapjoin.sortedmerge = true;
 set hive.input.format = 
 org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
 explain
 select count(*) from
 (
 select /*+mapjoin(a)*/ a.key as key1, b.key as key2, a.value as value1, 
 b.value as value2
 from smb_bucket_1 a join smb_bucket_2 b on a.key = b.key)
 subq;
 The above query does not use sort-merge join. This would be very useful as we 
 automatically convert the queries to use sorting and bucketing properties for 
 join.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3633) sort-merge join does not work with sub-queries


 [ 
https://issues.apache.org/jira/browse/HIVE-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3633:
-

Attachment: hive.3633.1.patch

 sort-merge join does not work with sub-queries
 --

 Key: HIVE-3633
 URL: https://issues.apache.org/jira/browse/HIVE-3633
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
 Attachments: hive.3633.1.patch


 Consider the following query:
 create table smb_bucket_1(key int, value string) CLUSTERED BY (key) SORTED BY 
 (key) INTO 6 BUCKETS STORED AS TEXTFILE;
 create table smb_bucket_2(key int, value string) CLUSTERED BY (key) SORTED BY 
 (key) INTO 6 BUCKETS STORED AS TEXTFILE;
 -- load the above tables
 set hive.optimize.bucketmapjoin = true;
 set hive.optimize.bucketmapjoin.sortedmerge = true;
 set hive.input.format = 
 org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
 explain
 select count(*) from
 (
 select /*+mapjoin(a)*/ a.key as key1, b.key as key2, a.value as value1, 
 b.value as value2
 from smb_bucket_1 a join smb_bucket_2 b on a.key = b.key)
 subq;
 The above query does not use sort-merge join. This would be very useful as we 
 automatically convert the queries to use sorting and bucketing properties for 
 join.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3633) sort-merge join does not work with sub-queries


[ 
https://issues.apache.org/jira/browse/HIVE-3633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13493945#comment-13493945
 ] 

Namit Jain commented on HIVE-3633:
--

https://reviews.facebook.net/D6615

 sort-merge join does not work with sub-queries
 --

 Key: HIVE-3633
 URL: https://issues.apache.org/jira/browse/HIVE-3633
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
 Attachments: hive.3633.1.patch


 Consider the following query:
 create table smb_bucket_1(key int, value string) CLUSTERED BY (key) SORTED BY 
 (key) INTO 6 BUCKETS STORED AS TEXTFILE;
 create table smb_bucket_2(key int, value string) CLUSTERED BY (key) SORTED BY 
 (key) INTO 6 BUCKETS STORED AS TEXTFILE;
 -- load the above tables
 set hive.optimize.bucketmapjoin = true;
 set hive.optimize.bucketmapjoin.sortedmerge = true;
 set hive.input.format = 
 org.apache.hadoop.hive.ql.io.BucketizedHiveInputFormat;
 explain
 select count(*) from
 (
 select /*+mapjoin(a)*/ a.key as key1, b.key as key2, a.value as value1, 
 b.value as value2
 from smb_bucket_1 a join smb_bucket_2 b on a.key = b.key)
 subq;
 The above query does not use sort-merge join. This would be very useful as we 
 automatically convert the queries to use sorting and bucketing properties for 
 join.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3696) Revert HIVE-3483 which causes performance regression


 [ 
https://issues.apache.org/jira/browse/HIVE-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3696:
-

   Resolution: Fixed
Fix Version/s: 0.10.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Tim

 Revert HIVE-3483 which causes performance regression
 

 Key: HIVE-3696
 URL: https://issues.apache.org/jira/browse/HIVE-3696
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Gang Tim Liu
Priority: Critical
 Fix For: 0.10.0

 Attachments: HIVE-3696.patch.1


 HIVE-3483 causes performance regression.
 We'd like to revert it first and find another solution for it later.
 This issue is to track revert and HIVE-3693 tracks subsequent solution.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3613) Implement grouping_id function


[ 
https://issues.apache.org/jira/browse/HIVE-3613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494039#comment-13494039
 ] 

Hudson commented on HIVE-3613:
--

Integrated in Hive-trunk-h0.21 #1785 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1785/])
HIVE-3613 Implement grouping_id function
(Ian Gorbachev via namit) (Revision 1407371)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1407371
Files : 
* /hive/trunk/data/files/groupby_groupingid.txt
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/VirtualColumn.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
* /hive/trunk/ql/src/test/queries/clientnegative/groupby_grouping_id1.q
* /hive/trunk/ql/src/test/queries/clientpositive/groupby_grouping_id1.q
* /hive/trunk/ql/src/test/queries/clientpositive/groupby_grouping_id2.q
* /hive/trunk/ql/src/test/results/clientnegative/groupby_grouping_id1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_grouping_id1.q.out
* /hive/trunk/ql/src/test/results/clientpositive/groupby_grouping_id2.q.out


 Implement grouping_id function
 --

 Key: HIVE-3613
 URL: https://issues.apache.org/jira/browse/HIVE-3613
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Ivan Gorbachev
Assignee: Ivan Gorbachev
 Fix For: 0.10.0

 Attachments: hive.3613.2.patch, jira-3613.0.patch, jira-3613.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3649) Hive List Bucketing - enhance DDL to specify list bucketing table


[ 
https://issues.apache.org/jira/browse/HIVE-3649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494040#comment-13494040
 ] 

Hudson commented on HIVE-3649:
--

Integrated in Hive-trunk-h0.21 #1785 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1785/])
HIVE-3649 Hive List Bucketing - enhance DDL to specify list bucketing table
(Gang Tim Liu via namit) (Revision 1407353)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1407353
Files : 
* /hive/trunk/metastore/if/hive_metastore.thrift
* /hive/trunk/metastore/scripts/upgrade/derby/010-HIVE-3649.derby.sql
* /hive/trunk/metastore/scripts/upgrade/derby/hive-schema-0.10.0.derby.sql
* /hive/trunk/metastore/scripts/upgrade/derby/upgrade-0.9.0-to-0.10.0.derby.sql
* /hive/trunk/metastore/scripts/upgrade/mysql/010-HIVE-3649.mysql.sql
* /hive/trunk/metastore/scripts/upgrade/mysql/hive-schema-0.10.0.mysql.sql
* /hive/trunk/metastore/scripts/upgrade/mysql/upgrade-0.9.0-to-0.10.0.mysql.sql
* /hive/trunk/metastore/scripts/upgrade/oracle/010-HIVE-3649.oracle.sql
* /hive/trunk/metastore/scripts/upgrade/oracle/hive-schema-0.10.0.oracle.sql
* /hive/trunk/metastore/scripts/upgrade/postgres/010-HIVE-3649.postgres.sql
* /hive/trunk/metastore/scripts/upgrade/postgres/hive-schema-0.10.0.postgres.sql
* 
/hive/trunk/metastore/scripts/upgrade/postgres/upgrade-0.9.0-to-0.10.0.postgres.sql
* /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp
* /hive/trunk/metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Database.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/EnvironmentContext.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Index.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Partition.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/PrincipalPrivilegeSet.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Schema.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SerDeInfo.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/SkewedInfo.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StorageDescriptor.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/Table.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java
* 
/hive/trunk/metastore/src/gen/thrift/gen-php/hive_metastore/hive_metastore_types.php
* /hive/trunk/metastore/src/gen/thrift/gen-php/metastore/Types.php
* /hive/trunk/metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py
* /hive/trunk/metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java
* 
/hive/trunk/metastore/src/model/org/apache/hadoop/hive/metastore/model/MStorageDescriptor.java
* /hive/trunk/metastore/src/model/package.jdo
* 
/hive/trunk/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java
* 
/hive/trunk/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Operator.java
* 
/hive/trunk/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Query.java
* 
/hive/trunk/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Stage.java
* 
/hive/trunk/ql/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/ql/plan/api/Task.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/ImportSemanticAnalyzer.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/LoadSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/CreateTableDesc.java
* /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/QTestUtil.java
* /hive/trunk/ql/src/test/org/apache/hadoop/hive/ql/metadata/TestHive.java
* /hive/trunk/ql/src/test/queries/clientnegative/load_stored_as_dirs.q
*

[jira] [Commented] (HIVE-3692) Update parallel test documentation


[ 
https://issues.apache.org/jira/browse/HIVE-3692?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494041#comment-13494041
 ] 

Hudson commented on HIVE-3692:
--

Integrated in Hive-trunk-h0.21 #1785 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1785/])
HIVE-3692 Update parallel test documentation
(Ivan Gorbachev via namit) (Revision 1407360)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1407360
Files : 
* /hive/trunk/testutils/ptest/README


 Update parallel test documentation
 --

 Key: HIVE-3692
 URL: https://issues.apache.org/jira/browse/HIVE-3692
 Project: Hive
  Issue Type: Bug
Reporter: Ivan Gorbachev
Assignee: Ivan Gorbachev
  Labels: hive, testing
 Fix For: 0.10.0

 Attachments: jira-3692.0.patch


 https://github.com/apache/hive/blob/trunk/testutils/ptest/README has 
 incorrect json object:
 {code}
 {
 qfile_hosts: [
 [hostname1, 2],
 [hostname2, 4],
 [hostname3, 4],
 ],
 other_hosts: [
 [hostname1, 2],
 [hostname4, 5]
 ],
 master_base_path: ${{HOME}}/hivetests,
 host_base_path: /mnt/drive/hivetests
 java_home: /opt/jdk
 }
 {code}
 Should update it with right syntax

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1785 - Still Failing

Changes for Build #1764
[kevinwilfong] HIVE-3610. Add a command Explain dependency ... (Sambavi 
Muthukrishnan via kevinwilfong)


Changes for Build #1765

Changes for Build #1766
[hashutosh] HIVE-3441 : testcases escape1,escape2 fail on windows (Thejas Nair 
via Ashutosh Chauhan)

[kevinwilfong] HIVE-3499. add tests to use bucketing metadata for partitions. 
(njain via kevinwilfong)


Changes for Build #1767
[kevinwilfong] HIVE-3276. optimize union sub-queries. (njain via kevinwilfong)


Changes for Build #1768

Changes for Build #1769

Changes for Build #1770
[namit] HIVE-3570 Add/fix facility to collect operator specific statisticsin 
hive + add hash-in/hash-out
counter for GroupBy Optr (Satadru Pan via namit)

[namit] HIVE-3554 Hive List Bucketing - Query logic
(Gang Tim Liu via namit)

[cws] HIVE-3563. Drop database cascade fails when there are indexes on any 
tables (Prasad Mujumdar via cws)


Changes for Build #1771
[kevinwilfong] HIVE-3640. Reducer allocation is incorrect if enforce bucketing 
and mapred.reduce.tasks are both set. (Vighnesh Avadhani via kevinwilfong)


Changes for Build #1772

Changes for Build #1773

Changes for Build #1774

Changes for Build #1775
[namit] HIVE-3673 Sort merge join not used when join columns have different 
names
(Kevin Wilfong via namit)


Changes for Build #1776
[kevinwilfong] HIVE-3627. eclipse misses library: 
javolution-@javolution-version@.jar. (Gang Tim Liu via kevinwilfong)


Changes for Build #1777
[kevinwilfong] HIVE-3524. Storing certain Exception objects thrown in 
HiveMetaStore.java in MetaStoreEndFunctionContext. (Maheshwaran Srinivasan via 
kevinwilfong)

[cws] HIVE-1977. DESCRIBE TABLE syntax doesn't support specifying a database 
qualified table name (Zhenxiao Luo via cws)

[cws] HIVE-3674. Test case TestParse broken after recent checkin (Sambavi 
Muthukrishnan via cws)


Changes for Build #1778
[cws] HIVE-1362. Column level scalar valued statistics on Tables and Partitions 
(Shreepadma Venugopalan via cws)


Changes for Build #1779

Changes for Build #1780
[kevinwilfong] HIVE-3686. Fix compile errors introduced by the interaction of 
HIVE-1362 and HIVE-3524. (Shreepadma Venugopalan via kevinwilfong)


Changes for Build #1781
[namit] HIVE-3687 smb_mapjoin_13.q is nondeterministic
(Kevin Wilfong via namit)


Changes for Build #1782
[hashutosh] HIVE-2715: Upgrade Thrift dependency to 0.9.0 (Ashutosh Chauhan)


Changes for Build #1783
[kevinwilfong] HIVE-3654. block relative path access in hive. (njain via 
kevinwilfong)

[hashutosh] HIVE-3658 : Unable to generate the Hbase related unit tests using 
velocity templates on Windows (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3661 : Remove the Windows specific = related swizzle path 
changes from Proxy FileSystems (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3480 : Resource leak: Fix the file handle leaks in Symbolic 
 Symlink related input formats. (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1784
[kevinwilfong] HIVE-3675. NaN does not work correctly for round(n). (njain via 
kevinwilfong)

[cws] HIVE-3651. bucketmapjoin?.q tests fail with hadoop 0.23 (Prasad Mujumdar 
via cws)


Changes for Build #1785
[namit] HIVE-3613 Implement grouping_id function
(Ian Gorbachev via namit)

[namit] HIVE-3692 Update parallel test documentation
(Ivan Gorbachev via namit)

[namit] HIVE-3649 Hive List Bucketing - enhance DDL to specify list bucketing 
table
(Gang Tim Liu via namit)




7 tests failed.
REGRESSION:  
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testSynchronized

Error Message:
expected:1 but was:2

Stack Trace:
junit.framework.AssertionFailedError: expected:1 but was:2
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:277)
at junit.framework.Assert.assertEquals(Assert.java:64)
at junit.framework.Assert.assertEquals(Assert.java:195)
at junit.framework.Assert.assertEquals(Assert.java:201)
at 
org.apache.hadoop.hive.metastore.TestHiveMetaStore.testSynchronized(TestHiveMetaStore.java:2090)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)
at junit.framework.TestSuite.run(TestSuite.java:227)
at

[jira] [Commented] (HIVE-3696) Revert HIVE-3483 which causes performance regression

2012-11-09 Thread Gang Tim Liu (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494045#comment-13494045
 ] 

Gang Tim Liu commented on HIVE-3696:


Namit thanks a lot

Sent from my iPhone




 Revert HIVE-3483 which causes performance regression
 

 Key: HIVE-3696
 URL: https://issues.apache.org/jira/browse/HIVE-3696
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Gang Tim Liu
Priority: Critical
 Fix For: 0.10.0

 Attachments: HIVE-3696.patch.1


 HIVE-3483 causes performance regression.
 We'd like to revert it first and find another solution for it later.
 This issue is to track revert and HIVE-3693 tracks subsequent solution.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3697) External JAR files on HDFS can lead to race condition with hive.downloaded.resources.dir

2012-11-09 Thread Chris McConnell (JIRA)

Chris McConnell created HIVE-3697:
-

 Summary: External JAR files on HDFS can lead to race condition 
with hive.downloaded.resources.dir
 Key: HIVE-3697
 URL: https://issues.apache.org/jira/browse/HIVE-3697
 Project: Hive
  Issue Type: Bug
Reporter: Chris McConnell


I've seen situations where utilizing JAR files on HDFS can cause job failures 
via CNFE or JVM crashes. 

This is difficult to replicate, seems to be related to JAR size, latency 
between client and HDFS cluster, but I've got some example stack traces below. 
Seems that the calls made to FileSystem (copyToLocal) which are static and will 
be executed to delete the current local copy can cause the file(s) to be 
removed during job processing.

We should consider changing the default for hive.downloaded.resources.dir to 
include some level of uniqueness per job. We should not consider 
hive.session.id however, as execution of multiple statements via the same 
user/session which might access the same JAR files will utilize the same 
session.

A proposal might be to utilize System.nanoTime() -- which might be enough to 
avoid the issue, although it's not perfect (depends on JVM and system for level 
of precision) as part of the default 
(/tmp/${user.name}/resources/System.nanoTime()/). 

If anyone else has hit this, would like to capture environment information as 
well. Perhaps there is something else at play here. 

Here are some examples of the errors:

for i in {0..2}; do hive -S -f query.q done
[2] 48405
[3] 48406
[4] 48407
% #
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGBUS (0x7) at pc=0x7fb10bd931f0, pid=48407, tid=140398456698624
#
# JRE version: 6.0_31-b04
# Java VM: Java HotSpot(TM) 64-Bit Server VM (20.6-b01 mixed mode linux-amd64 
compressed oops)
# Problematic frame:
# C  [libzip.so+0xb1f0]  __int128+0x60
#
# An error report file with more information is saved as:
# /home/.../hs_err_pid48407.log
#
# If you would like to submit a bug report, please visit:
#   http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
java.lang.NoClassDefFoundError: com/example/udf/Lower
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:247)
at 
org.apache.hadoop.hive.ql.exec.FunctionTask.getUdfClass(FunctionTask.java:105)
at 
org.apache.hadoop.hive.ql.exec.FunctionTask.createFunction(FunctionTask.java:75)
at 
org.apache.hadoop.hive.ql.exec.FunctionTask.execute(FunctionTask.java:63)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
at 
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1331)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1117)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:950)
at 
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:341)
at 
org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:439)
at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:449)
at 
org.apache.hadoop.hive.cli.CliDriver.processInitFiles(CliDriver.java:485)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:692)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:607)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
Caused by: java.lang.ClassNotFoundException: com.example.udf.Lower
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
... 24 more
FAILED: Execution Error, return code -101 from 
org.apache.hadoop.hive.ql.exec.FunctionTask

Another:
for i in {0..2}; do hive -S -f query.q done
[1] 16294 
[2] 16295 
[3] 16296 
[]$ Couldn't create directory /tmp/ctm/resources/
Couldn't create directory /tmp/ctm/resources/

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA

[jira] [Updated] (HIVE-3697) External JAR files on HDFS can lead to race condition with hive.downloaded.resources.dir

2012-11-09 Thread Chris McConnell (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-3697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris McConnell updated HIVE-3697:
--

Affects Version/s: 0.9.0

 External JAR files on HDFS can lead to race condition with 
 hive.downloaded.resources.dir
 

 Key: HIVE-3697
 URL: https://issues.apache.org/jira/browse/HIVE-3697
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Chris McConnell

 I've seen situations where utilizing JAR files on HDFS can cause job failures 
 via CNFE or JVM crashes. 
 This is difficult to replicate, seems to be related to JAR size, latency 
 between client and HDFS cluster, but I've got some example stack traces 
 below. Seems that the calls made to FileSystem (copyToLocal) which are static 
 and will be executed to delete the current local copy can cause the file(s) 
 to be removed during job processing.
 We should consider changing the default for hive.downloaded.resources.dir to 
 include some level of uniqueness per job. We should not consider 
 hive.session.id however, as execution of multiple statements via the same 
 user/session which might access the same JAR files will utilize the same 
 session.
 A proposal might be to utilize System.nanoTime() -- which might be enough to 
 avoid the issue, although it's not perfect (depends on JVM and system for 
 level of precision) as part of the default 
 (/tmp/${user.name}/resources/System.nanoTime()/). 
 If anyone else has hit this, would like to capture environment information as 
 well. Perhaps there is something else at play here. 
 Here are some examples of the errors:
 for i in {0..2}; do hive -S -f query.q done
 [2] 48405
 [3] 48406
 [4] 48407
 % #
 # A fatal error has been detected by the Java Runtime Environment:
 #
 #  SIGBUS (0x7) at pc=0x7fb10bd931f0, pid=48407, tid=140398456698624
 #
 # JRE version: 6.0_31-b04
 # Java VM: Java HotSpot(TM) 64-Bit Server VM (20.6-b01 mixed mode linux-amd64 
 compressed oops)
 # Problematic frame:
 # C  [libzip.so+0xb1f0]  __int128+0x60
 #
 # An error report file with more information is saved as:
 # /home/.../hs_err_pid48407.log
 #
 # If you would like to submit a bug report, please visit:
 #   http://java.sun.com/webapps/bugreport/crash.jsp
 # The crash happened outside the Java Virtual Machine in native code.
 # See problematic frame for where to report the bug.
 #
 java.lang.NoClassDefFoundError: com/example/udf/Lower
 at java.lang.Class.forName0(Native Method)
 at java.lang.Class.forName(Class.java:247)
 at 
 org.apache.hadoop.hive.ql.exec.FunctionTask.getUdfClass(FunctionTask.java:105)
 at 
 org.apache.hadoop.hive.ql.exec.FunctionTask.createFunction(FunctionTask.java:75)
 at 
 org.apache.hadoop.hive.ql.exec.FunctionTask.execute(FunctionTask.java:63)
 at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:153)
 at 
 org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57)
 at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1331)
 at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1117)
 at org.apache.hadoop.hive.ql.Driver.run(Driver.java:950)
 at 
 org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:258)
 at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:215)
 at 
 org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:406)
 at 
 org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:341)
 at 
 org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:439)
 at 
 org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:449)
 at 
 org.apache.hadoop.hive.cli.CliDriver.processInitFiles(CliDriver.java:485)
 at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:692)
 at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:607)
 at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
 at 
 sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
 at 
 sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
 at java.lang.reflect.Method.invoke(Method.java:597)
 at org.apache.hadoop.util.RunJar.main(RunJar.java:208)
 Caused by: java.lang.ClassNotFoundException: com.example.udf.Lower
 at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
 at java.security.AccessController.doPrivileged(Native Method)
 at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
 at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
 ... 24 more
 FAILED: Execution Error, return code -101 from

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #193

See 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/193/

--
[...truncated 10125 lines...]
 [echo] Project: odbc
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/odbc/src/conf
 does not exist.

ivy-resolve-test:
 [echo] Project: odbc

ivy-retrieve-test:
 [echo] Project: odbc

compile-test:
 [echo] Project: odbc

create-dirs:
 [echo] Project: serde
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/serde/src/test/resources
 does not exist.

init:
 [echo] Project: serde

ivy-init-settings:
 [echo] Project: serde

ivy-resolve:
 [echo] Project: serde
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml
[ivy:report] Processing 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/193/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-serde-default.xml
 to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/193/artifact/hive/build/ivy/report/org.apache.hive-hive-serde-default.html

ivy-retrieve:
 [echo] Project: serde

dynamic-serde:

compile:
 [echo] Project: serde

ivy-resolve-test:
 [echo] Project: serde

ivy-retrieve-test:
 [echo] Project: serde

compile-test:
 [echo] Project: serde
[javac] Compiling 26 source files to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/193/artifact/hive/build/serde/test/classes
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

create-dirs:
 [echo] Project: service
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/service/src/test/resources
 does not exist.

init:
 [echo] Project: service

ivy-init-settings:
 [echo] Project: service

ivy-resolve:
 [echo] Project: service
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml
[ivy:report] Processing 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/193/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-service-default.xml
 to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/193/artifact/hive/build/ivy/report/org.apache.hive-hive-service-default.html

ivy-retrieve:
 [echo] Project: service

compile:
 [echo] Project: service

ivy-resolve-test:
 [echo] Project: service

ivy-retrieve-test:
 [echo] Project: service

compile-test:
 [echo] Project: service
[javac] Compiling 2 source files to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/193/artifact/hive/build/service/test/classes

test:
 [echo] Project: hive

test-shims:
 [echo] Project: hive

test-conditions:
 [echo] Project: shims

gen-test:
 [echo] Project: shims

create-dirs:
 [echo] Project: shims
 [copy] Warning: 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/test/resources
 does not exist.

init:
 [echo] Project: shims

ivy-init-settings:
 [echo] Project: shims

ivy-resolve:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml
[ivy:report] Processing 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/193/artifact/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml
 to 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/193/artifact/hive/build/ivy/report/org.apache.hive-hive-shims-default.html

ivy-retrieve:
 [echo] Project: shims

compile:
 [echo] Project: shims
 [echo] Building shims 0.20

build_shims:
 [echo] Project: shims
 [echo] Compiling 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/shims/src/common/java;/home/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java
 against hadoop 0.20.2 
(https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/193/artifact/hive/build/hadoopcore/hadoop-0.20.2)

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/ws/hive/ivy/ivysettings.xml

ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.20S

build_shims:
 [echo] Project: shims
 [echo] Compiling

[jira] [Commented] (HIVE-3647) map-side groupby wrongly due to HIVE-3432

2012-11-09 Thread Mark Grover (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-3647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494100#comment-13494100
 ] 

Mark Grover commented on HIVE-3647:
---

More comments!

 map-side groupby wrongly due to HIVE-3432
 -

 Key: HIVE-3647
 URL: https://issues.apache.org/jira/browse/HIVE-3647
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
 Attachments: hive.3647.1.patch, hive.3647.2.patch, hive.3647.3.patch, 
 hive.3647.4.patch, hive.3647.5.patch, hive.3647.6.patch, hive.3647.7.patch


 There seems to be a bug due to HIVE-3432.
 We are converting the group by to a map side group by after only looking at
 sorting columns. This can give wrong results if the data is sorted and
 bucketed by different columns.
 Add some tests for that scenario, verify and fix any issues.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3621) Make prompt in Hive CLI configurable


 [ 
https://issues.apache.org/jira/browse/HIVE-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-3621:


Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed, thanks Jingwei.

 Make prompt in Hive CLI configurable
 

 Key: HIVE-3621
 URL: https://issues.apache.org/jira/browse/HIVE-3621
 Project: Hive
  Issue Type: Improvement
  Components: CLI
Affects Versions: 0.9.0
Reporter: Jingwei Lu
Assignee: Jingwei Lu
Priority: Minor
  Labels: newbie, patch
 Fix For: 0.10.0

 Attachments: HIVE-3621.patch.1.txt, HIVE-3621.patch.2.txt

   Original Estimate: 48h
  Remaining Estimate: 48h

 Right now the Hive CLI prompt just says hive, for users (primarily power 
 users) who run in different clusters it can be easy to forget which cluster 
 your Hive CLI is pointing to.  If we change the Hive CLI prompt to be 
 something like hive(silver) it would be much clearer.  We could 
 potentially extend this to namespaces as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3695) TestParse breaks due to HIVE-3675


 [ 
https://issues.apache.org/jira/browse/HIVE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-3695:


Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed, thanks Namit.

 TestParse breaks due to HIVE-3675
 -

 Key: HIVE-3695
 URL: https://issues.apache.org/jira/browse/HIVE-3695
 Project: Hive
  Issue Type: Bug
  Components: Tests
Reporter: Namit Jain
Assignee: Namit Jain
 Attachments: hive.3695.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Jenkins build is back to normal : Hive-0.9.1-SNAPSHOT-h0.21 #193

See https://builds.apache.org/job/Hive-0.9.1-SNAPSHOT-h0.21/193/

[jira] [Updated] (HIVE-3471) Implement grouping sets in hive

2012-11-09 Thread Ivan Gorbachev (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Gorbachev updated HIVE-3471:
-

Attachment: jira-3471.2.patch

 Implement grouping sets in hive
 ---

 Key: HIVE-3471
 URL: https://issues.apache.org/jira/browse/HIVE-3471
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor
Reporter: Namit Jain
Assignee: Ivan Gorbachev
 Attachments: jira-3471.0.patch, jira-3471.1.patch, jira-3471.2.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3471) Implement grouping sets in hive

2012-11-09 Thread Ivan Gorbachev (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-3471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Gorbachev updated HIVE-3471:
-

Status: Patch Available  (was: Open)

 Implement grouping sets in hive
 ---

 Key: HIVE-3471
 URL: https://issues.apache.org/jira/browse/HIVE-3471
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor
Reporter: Namit Jain
Assignee: Ivan Gorbachev
 Attachments: jira-3471.0.patch, jira-3471.1.patch, jira-3471.2.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1786 - Still Failing

Changes for Build #1764
[kevinwilfong] HIVE-3610. Add a command Explain dependency ... (Sambavi 
Muthukrishnan via kevinwilfong)


Changes for Build #1765

Changes for Build #1766
[hashutosh] HIVE-3441 : testcases escape1,escape2 fail on windows (Thejas Nair 
via Ashutosh Chauhan)

[kevinwilfong] HIVE-3499. add tests to use bucketing metadata for partitions. 
(njain via kevinwilfong)


Changes for Build #1767
[kevinwilfong] HIVE-3276. optimize union sub-queries. (njain via kevinwilfong)


Changes for Build #1768

Changes for Build #1769

Changes for Build #1770
[namit] HIVE-3570 Add/fix facility to collect operator specific statisticsin 
hive + add hash-in/hash-out
counter for GroupBy Optr (Satadru Pan via namit)

[namit] HIVE-3554 Hive List Bucketing - Query logic
(Gang Tim Liu via namit)

[cws] HIVE-3563. Drop database cascade fails when there are indexes on any 
tables (Prasad Mujumdar via cws)


Changes for Build #1771
[kevinwilfong] HIVE-3640. Reducer allocation is incorrect if enforce bucketing 
and mapred.reduce.tasks are both set. (Vighnesh Avadhani via kevinwilfong)


Changes for Build #1772

Changes for Build #1773

Changes for Build #1774

Changes for Build #1775
[namit] HIVE-3673 Sort merge join not used when join columns have different 
names
(Kevin Wilfong via namit)


Changes for Build #1776
[kevinwilfong] HIVE-3627. eclipse misses library: 
javolution-@javolution-version@.jar. (Gang Tim Liu via kevinwilfong)


Changes for Build #1777
[kevinwilfong] HIVE-3524. Storing certain Exception objects thrown in 
HiveMetaStore.java in MetaStoreEndFunctionContext. (Maheshwaran Srinivasan via 
kevinwilfong)

[cws] HIVE-1977. DESCRIBE TABLE syntax doesn't support specifying a database 
qualified table name (Zhenxiao Luo via cws)

[cws] HIVE-3674. Test case TestParse broken after recent checkin (Sambavi 
Muthukrishnan via cws)


Changes for Build #1778
[cws] HIVE-1362. Column level scalar valued statistics on Tables and Partitions 
(Shreepadma Venugopalan via cws)


Changes for Build #1779

Changes for Build #1780
[kevinwilfong] HIVE-3686. Fix compile errors introduced by the interaction of 
HIVE-1362 and HIVE-3524. (Shreepadma Venugopalan via kevinwilfong)


Changes for Build #1781
[namit] HIVE-3687 smb_mapjoin_13.q is nondeterministic
(Kevin Wilfong via namit)


Changes for Build #1782
[hashutosh] HIVE-2715: Upgrade Thrift dependency to 0.9.0 (Ashutosh Chauhan)


Changes for Build #1783
[kevinwilfong] HIVE-3654. block relative path access in hive. (njain via 
kevinwilfong)

[hashutosh] HIVE-3658 : Unable to generate the Hbase related unit tests using 
velocity templates on Windows (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3661 : Remove the Windows specific = related swizzle path 
changes from Proxy FileSystems (Kanna Karanam via Ashutosh Chauhan)

[hashutosh] HIVE-3480 : Resource leak: Fix the file handle leaks in Symbolic 
 Symlink related input formats. (Kanna Karanam via Ashutosh Chauhan)


Changes for Build #1784
[kevinwilfong] HIVE-3675. NaN does not work correctly for round(n). (njain via 
kevinwilfong)

[cws] HIVE-3651. bucketmapjoin?.q tests fail with hadoop 0.23 (Prasad Mujumdar 
via cws)


Changes for Build #1785
[namit] HIVE-3613 Implement grouping_id function
(Ian Gorbachev via namit)

[namit] HIVE-3692 Update parallel test documentation
(Ivan Gorbachev via namit)

[namit] HIVE-3649 Hive List Bucketing - enhance DDL to specify list bucketing 
table
(Gang Tim Liu via namit)


Changes for Build #1786
[namit] HIVE-3696 Revert HIVE-3483 which causes performance regression
(Gang Tim Liu via namit)




6 tests failed.
FAILED:  
org.apache.hadoop.hive.metastore.TestRemoteHiveMetaStore.testSynchronized

Error Message:
expected:1 but was:2

Stack Trace:
junit.framework.AssertionFailedError: expected:1 but was:2
at junit.framework.Assert.fail(Assert.java:47)
at junit.framework.Assert.failNotEquals(Assert.java:277)
at junit.framework.Assert.assertEquals(Assert.java:64)
at junit.framework.Assert.assertEquals(Assert.java:195)
at junit.framework.Assert.assertEquals(Assert.java:201)
at 
org.apache.hadoop.hive.metastore.TestHiveMetaStore.testSynchronized(TestHiveMetaStore.java:2090)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at junit.framework.TestCase.runTest(TestCase.java:168)
at junit.framework.TestCase.runBare(TestCase.java:134)
at junit.framework.TestResult$1.protect(TestResult.java:110)
at junit.framework.TestResult.runProtected(TestResult.java:128)
at junit.framework.TestResult.run(TestResult.java:113)
at junit.framework.TestCase.run(TestCase.java:124)
at junit.framework.TestSuite.runTest(TestSuite.java:232)

[jira] [Commented] (HIVE-3483) joins using partitioned table give incorrect results on windows


[ 
https://issues.apache.org/jira/browse/HIVE-3483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494399#comment-13494399
 ] 

Hudson commented on HIVE-3483:
--

Integrated in Hive-trunk-h0.21 #1786 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1786/])
HIVE-3696 Revert HIVE-3483 which causes performance regression
(Gang Tim Liu via namit) (Revision 1407429)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1407429
Files : 
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java


 joins using partitioned table give incorrect results on windows 
 

 Key: HIVE-3483
 URL: https://issues.apache.org/jira/browse/HIVE-3483
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Thejas M Nair
Assignee: Thejas M Nair
 Fix For: 0.10.0

 Attachments: HIVE-3483.1.patch, HIVE-3483.2.patch


 Hive CombineFilter.getSplits returns single split instead of multiple splits 
 on windows. The comparison in CombineFilter.accept() was using paths 
 converted to strings instead of using Paths themselves, and one of the 
 strings had /C: and other had C: .
 The tests that are affected by this issue include - 
 auto_join18.q,auto_join18_multi_distinct.q,
 join18.q,join18_multi_distinct.q,ppd_union_view.q,create_view.q and union20.q

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3696) Revert HIVE-3483 which causes performance regression


[ 
https://issues.apache.org/jira/browse/HIVE-3696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494400#comment-13494400
 ] 

Hudson commented on HIVE-3696:
--

Integrated in Hive-trunk-h0.21 #1786 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1786/])
HIVE-3696 Revert HIVE-3483 which causes performance regression
(Gang Tim Liu via namit) (Revision 1407429)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1407429
Files : 
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java


 Revert HIVE-3483 which causes performance regression
 

 Key: HIVE-3696
 URL: https://issues.apache.org/jira/browse/HIVE-3696
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.10.0
Reporter: Gang Tim Liu
Priority: Critical
 Fix For: 0.10.0

 Attachments: HIVE-3696.patch.1


 HIVE-3483 causes performance regression.
 We'd like to revert it first and find another solution for it later.
 This issue is to track revert and HIVE-3693 tracks subsequent solution.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: implement a udf to keep hive session alive for certain amount of time

2012-11-09 Thread Johnny Zhang



 On Nov. 4, 2012, 4:59 a.m., Mark Grover wrote:
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSleep.java,
   line 36
  https://reviews.apache.org/r/7848/diff/1/?file=185096#file185096line36
 
  I don't have a strong opinion about this but is sleep the right name 
  for this UDF? Sleep is how this UDF keeps the Hive session alive but it 
  might not convey to a user what this UDF does. How about something like 
  session_keep_alive? I am open to other suggestions as well. 
  
  Again, not a deal-breaker:-) However, if you do decide to change the 
  name, don't forget to change all references of sleep in the code (log 
  statements, exception messages, etc.).

the reason give a name 'sleep' is because Hadoop used to have a similar example 
job 
http://grepcode.com/file/repository.cloudera.com/content/repositories/releases/com.cloudera.hadoop/hadoop-examples/0.20.2-737/org/apache/hadoop/examples/SleepJob.java
which does nothing but keeping running a MR job, which does nothing. 
Let's see what's other people's opinion


 On Nov. 4, 2012, 4:59 a.m., Mark Grover wrote:
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSleep.java,
   line 37
  https://reviews.apache.org/r/7848/diff/1/?file=185096#file185096line37
 
  Specify in the explain statement what the units of the duration being 
  specified are (seconds?)

agree, will fix it


 On Nov. 4, 2012, 4:59 a.m., Mark Grover wrote:
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSleep.java,
   line 41
  https://reviews.apache.org/r/7848/diff/1/?file=185096#file185096line41
 
  Better to use GenericUDFSleep.class as argument

agree, will fix it


 On Nov. 4, 2012, 4:59 a.m., Mark Grover wrote:
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSleep.java,
   line 52
  https://reviews.apache.org/r/7848/diff/1/?file=185096#file185096line52
 
  1. A better exception to throw here is UDFArgumentLengthException
  2. It's always nice to see as a user what was the expected and the 
  actual value when something goes wrong. Consider printing out the type of 
  the argument received in the exception message. This type can be retrieved 
  by arguments[0].getTypeName()

agree that need to use 'arguments[0].getTypeName()' to print out what's the 
argument type of user input, will fix it.


 On Nov. 4, 2012, 4:59 a.m., Mark Grover wrote:
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSleep.java,
   line 52
  https://reviews.apache.org/r/7848/diff/1/?file=185096#file185096line52
 
  I am being nitpicky here but a better exception to throw here would be: 
  UDFArgumentTypeException. Also, when seeing an error message as a user, 
  it's always nice to contrast the actual vs. expected. Here is the expected 
  type is int but it will nice to print out the type of the argument that the 
  UDF received. You can retrieve by arguments[i].getTypeName()

agree, will fix it


 On Nov. 4, 2012, 4:59 a.m., Mark Grover wrote:
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSleep.java,
   line 55
  https://reviews.apache.org/r/7848/diff/1/?file=185096#file185096line55
 
  The UDF is returning a MapInt, Int even though you don't really want 
  to return anything. I think you should use a void object inspector. For 
  details, look at 
  http://svn.apache.org/viewvc/hive/trunk/serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorFactory.java?view=markup

agree, will change it to return 
PrimitiveObjectInspectorFactory.writableStringObjectInspector since will print 
message in the end


 On Nov. 4, 2012, 4:59 a.m., Mark Grover wrote:
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSleep.java,
   line 62
  https://reviews.apache.org/r/7848/diff/1/?file=185096#file185096line62
 
  Better to use ObjectInspectorConverter to avoid the string parsing 
  penalty.
  
  For reference, take a look at how this UDF reads an integer argument:
  
  http://svn.apache.org/viewvc/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFElt.java?view=markup

agree, will fix it.


 On Nov. 4, 2012, 4:59 a.m., Mark Grover wrote:
  http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSleep.java,
   line 71
  https://reviews.apache.org/r/7848/diff/1/?file=185096#file185096line71
 
  Any particular reason why we don't just Thread.sleep(numLoop * 1000) 
  without any loops? Is that because we want to log every 4 seconds?

yes, that's the reason. Just want to print something so that people know what's 
going on, especially when it sleeps for a while.


 On Nov. 4,

Re: Review Request: implement a udf to keep hive session alive for certain amount of time

2012-11-09 Thread Johnny Zhang


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/7848/
---

(Updated Nov. 9, 2012, 11:10 p.m.)


Review request for hive.


Changes
---

I update the diff to address Mark's comments. Thanks, Mark.


Description
---

To make testing issues like HIVE-3590 convenient, we can implement a UDF to 
keep hive session alive for a given time. The patch introduce a new UDF sleep() 
which does this without introducing any data/load to cluster.


This addresses bug HIVE-3666.
https://issues.apache.org/jira/browse/HIVE-3666


Diffs (updated)
-

  
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
 1405251 
  
http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFSleep.java
 PRE-CREATION 

Diff: https://reviews.apache.org/r/7848/diff/


Testing
---

have tested it with Hive CLI and Hive Server session, and it can keep them 
alive by the given seconds


Thanks,

Johnny Zhang

[jira] [Updated] (HIVE-3666) implement a udf to keep hive session alive for certain amount of time

2012-11-09 Thread Johnny Zhang (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-3666?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Johnny Zhang updated HIVE-3666:
---

Attachment: HIVE-3666.patch

 implement a udf to keep hive session alive for certain amount of time
 -

 Key: HIVE-3666
 URL: https://issues.apache.org/jira/browse/HIVE-3666
 Project: Hive
  Issue Type: New Feature
  Components: UDF
Affects Versions: 0.10.0
Reporter: Johnny Zhang
Assignee: Johnny Zhang
 Fix For: 0.10.0

 Attachments: HIVE-3666.patch, HIVE-3666.patch


 To make testing issues like HIVE-3590 convenient, we can implement a UDF to 
 keep hive session alive for a given time. The patch introduce a new UDF 
 sleep() which does this without introducing any data/load to cluster.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-933) Infer bucketing/sorting properties


 [ 
https://issues.apache.org/jira/browse/HIVE-933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-933:
---

Attachment: HIVE-933.3.patch.txt

 Infer bucketing/sorting properties
 --

 Key: HIVE-933
 URL: https://issues.apache.org/jira/browse/HIVE-933
 Project: Hive
  Issue Type: New Feature
  Components: Query Processor
Reporter: Namit Jain
Assignee: Kevin Wilfong
 Attachments: HIVE-933.1.patch.txt, HIVE-933.2.patch.txt, 
 HIVE-933.3.patch.txt


 This is a long-term plan, and may require major changes.
 From the query, we can figure out the sorting/bucketing properties, and 
 change the metadata of the destination at that time.
 However, this means that different partitions may have different metadata. 
 Currently, the query plan is same for all the 
 partitions of the table - we can do the following:
 1. In the first cut, have a simple approach where you take the union all 
 metadata, and create the most defensive plan.
 2. Enhance mapredWork() to include partition specific operator trees.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2640) Add alterPartition to AlterHandler interface


[ 
https://issues.apache.org/jira/browse/HIVE-2640?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494424#comment-13494424
 ] 

Phabricator commented on HIVE-2640:
---

kevinwilfong has closed the revision HIVE-2640 [jira] Add alterPartition to 
AlterHandler interface.

REVISION DETAIL
  https://reviews.facebook.net/D699

To: JIRA, njain, kevinwilfong
Cc: heyongqiang, njain


 Add alterPartition to AlterHandler interface
 

 Key: HIVE-2640
 URL: https://issues.apache.org/jira/browse/HIVE-2640
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Fix For: 0.9.0

 Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2640.D699.1.patch, 
 ASF.LICENSE.NOT.GRANTED--HIVE-2640.D699.2.patch


 Adding alterPartition to the AlterHandler interface would allow for 
 customized functionality to be executed as part of altering a partition, much 
 like it is already allowed for alterTable.
 Based on the name of the interface, and a comment in the AlterHandler code, 
 it looks like alterPartition was meant to be included along with alterTable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2823) Add logging of total run time of Driver


[ 
https://issues.apache.org/jira/browse/HIVE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494427#comment-13494427
 ] 

Phabricator commented on HIVE-2823:
---

kevinwilfong has closed the revision HIVE-2823 [jira] Add logging of total run 
time of Driver.

REVISION DETAIL
  https://reviews.facebook.net/D1977

To: JIRA, njain, kevinwilfong


 Add logging of total run time of Driver
 ---

 Key: HIVE-2823
 URL: https://issues.apache.org/jira/browse/HIVE-2823
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Fix For: 0.9.0

 Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2823.D1977.1.patch, 
 ASF.LICENSE.NOT.GRANTED--HIVE-2823.D1977.2.patch


 We are currently logging the run time of several components of the Driver, 
 e.g. compilation and map reduce execution time.  It would be helpful to log 
 the total run time of the Driver's run method as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2795) View partitions do not have a storage descriptor


[ 
https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494428#comment-13494428
 ] 

Phabricator commented on HIVE-2795:
---

kevinwilfong has closed the revision HIVE-2795 [jira] View partitions do not 
have a storage descriptor.

REVISION DETAIL
  https://reviews.facebook.net/D1683

To: JIRA, njain, kevinwilfong
Cc: njain


 View partitions do not have a storage descriptor
 

 Key: HIVE-2795
 URL: https://issues.apache.org/jira/browse/HIVE-2795
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Namit Jain
Assignee: Kevin Wilfong
 Fix For: 0.9.0

 Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2795.D1683.1.patch, 
 ASF.LICENSE.NOT.GRANTED--HIVE-2795.D1683.2.patch, 
 ASF.LICENSE.NOT.GRANTED--HIVE-2795.D1683.3.patch


 Besides being an inconsistency, it causes errors.
 Calling describe formatted on a view partition throws an exception
 java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) 
  
   at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570)
 because it does not have a column descriptor, which is part of the storage 
 descriptor.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2779) Improve hooks run in Driver


[ 
https://issues.apache.org/jira/browse/HIVE-2779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494430#comment-13494430
 ] 

Phabricator commented on HIVE-2779:
---

kevinwilfong has closed the revision HIVE-2779 [jira] Improve hooks run in 
Driver.

REVISION DETAIL
  https://reviews.facebook.net/D1599

To: JIRA, njain, kevinwilfong
Cc: njain


 Improve hooks run in Driver
 ---

 Key: HIVE-2779
 URL: https://issues.apache.org/jira/browse/HIVE-2779
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Fix For: 0.9.0

 Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2779.D1599.1.patch, 
 ASF.LICENSE.NOT.GRANTED--HIVE-2779.D1599.2.patch, 
 ASF.LICENSE.NOT.GRANTED--HIVE-2779.D1599.3.patch


 There are some small improvements that can be made to the hooks which are run 
 in the Driver:
 1) The code to get hooks has been clearly just been copy+pasted for each of 
 Pre/Post/OnFailure/SemanticAnalyzer hooks.  This code should be consolidated 
 into a single method.
 2) There is a lot more information available to SemanticAnalyzer hooks which 
 ran after semantic analysis than to those that run before, such as inputs and 
 outputs.  We should make some of this information available to those hooks, 
 preferably through HiveSemanticAnalyzerHookContext, so that existing hooks 
 aren't broken.
 3) Currently, possibly unintentionally, hooks are initialized and run in the 
 order they appear in the comma separated list that is the value of the 
 configuration variable.  This is a useful property, we should add comments 
 indicating this is desired and add a unit test to enforce it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3557) Access to external URLs in hivetest.py


 [ 
https://issues.apache.org/jira/browse/HIVE-3557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-3557:


Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed, thanks Ivan.

 Access to external URLs in hivetest.py 
 ---

 Key: HIVE-3557
 URL: https://issues.apache.org/jira/browse/HIVE-3557
 Project: Hive
  Issue Type: Improvement
Reporter: Ivan Gorbachev
Assignee: Ivan Gorbachev
 Attachments: jira-3557.0.patch, jira-3557.1.patch


 1. Migrate all non-HTTP urls to HTTP.
 2. Add HTTP_PROXY support

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3698) enhanceModel.notRequired is incorrectly determined

Chris Drome created HIVE-3698:
-

 Summary: enhanceModel.notRequired is incorrectly determined
 Key: HIVE-3698
 URL: https://issues.apache.org/jira/browse/HIVE-3698
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure, Metastore
Affects Versions: 0.10.0, 0.9.1
Reporter: Chris Drome


enhanceModel.notRequired in metastore/build.xml is not correctly determined 
which can result in datanucleus running twice. This in turn can cause other 
issues when building with JDK7.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3698) enhanceModel.notRequired is incorrectly determined


 [ 
https://issues.apache.org/jira/browse/HIVE-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Drome updated HIVE-3698:
--

Assignee: Chris Drome

 enhanceModel.notRequired is incorrectly determined
 --

 Key: HIVE-3698
 URL: https://issues.apache.org/jira/browse/HIVE-3698
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure, Metastore
Affects Versions: 0.10.0, 0.9.1
Reporter: Chris Drome
Assignee: Chris Drome

 enhanceModel.notRequired in metastore/build.xml is not correctly determined 
 which can result in datanucleus running twice. This in turn can cause other 
 issues when building with JDK7.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3698) enhanceModel.notRequired is incorrectly determined


 [ 
https://issues.apache.org/jira/browse/HIVE-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Drome updated HIVE-3698:
--

Attachment: HIVE-3698-trunk.patch
HIVE-3698-0.9.patch

 enhanceModel.notRequired is incorrectly determined
 --

 Key: HIVE-3698
 URL: https://issues.apache.org/jira/browse/HIVE-3698
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure, Metastore
Affects Versions: 0.10.0, 0.9.1
Reporter: Chris Drome
Assignee: Chris Drome
 Attachments: HIVE-3698-0.9.patch, HIVE-3698-trunk.patch


 enhanceModel.notRequired in metastore/build.xml is not correctly determined 
 which can result in datanucleus running twice. This in turn can cause other 
 issues when building with JDK7.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3693) Performance regression introduced by HIVE-3483

2012-11-09 Thread Thejas M Nair (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-3693?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-3693:


Attachment: HIVE-3693.1.patch

HIVE-3693.1.patch - Changes the Path to string conversions in CombineFilter to 
be consistent. Instead of using Path.toString() in one place and using 
Path.toUri().getPath() in other, using Path.toUri().getPath() in both places. 


 Performance regression introduced by HIVE-3483
 --

 Key: HIVE-3693
 URL: https://issues.apache.org/jira/browse/HIVE-3693
 Project: Hive
  Issue Type: Bug
Reporter: Gang Tim Liu
Priority: Minor
 Attachments: HIVE-3693.1.patch


 https://issues.apache.org/jira/browse/HIVE-3483 introduced a performance 
 regression in the client side during split computation.
 The client side spends a lot more time in the split computation phase. The 
 problem is checkFilterPathContains method.
 While investigating, can you create a config to disable it by default?
 thanks

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3621) Make prompt in Hive CLI configurable


[ 
https://issues.apache.org/jira/browse/HIVE-3621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494503#comment-13494503
 ] 

Hudson commented on HIVE-3621:
--

Integrated in Hive-trunk-h0.21 #1787 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1787/])
HIVE-3621. Make prompt in Hive CLI configurable. (Jingwei Lu via 
kevinwilfong) (Revision 1407581)

 Result = FAILURE
kevinwilfong : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1407581
Files : 
* /hive/trunk/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/conf/hive-default.xml.template


 Make prompt in Hive CLI configurable
 

 Key: HIVE-3621
 URL: https://issues.apache.org/jira/browse/HIVE-3621
 Project: Hive
  Issue Type: Improvement
  Components: CLI
Affects Versions: 0.9.0
Reporter: Jingwei Lu
Assignee: Jingwei Lu
Priority: Minor
  Labels: newbie, patch
 Fix For: 0.10.0

 Attachments: HIVE-3621.patch.1.txt, HIVE-3621.patch.2.txt

   Original Estimate: 48h
  Remaining Estimate: 48h

 Right now the Hive CLI prompt just says hive, for users (primarily power 
 users) who run in different clusters it can be easy to forget which cluster 
 your Hive CLI is pointing to.  If we change the Hive CLI prompt to be 
 something like hive(silver) it would be much clearer.  We could 
 potentially extend this to namespaces as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3695) TestParse breaks due to HIVE-3675


[ 
https://issues.apache.org/jira/browse/HIVE-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494504#comment-13494504
 ] 

Hudson commented on HIVE-3695:
--

Integrated in Hive-trunk-h0.21 #1787 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1787/])
HIVE-3695. TestParse breaks due to HIVE-3675. (njain via kevinwilfong) 
(Revision 1407558)

 Result = FAILURE
kevinwilfong : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1407558
Files : 
* /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml


 TestParse breaks due to HIVE-3675
 -

 Key: HIVE-3695
 URL: https://issues.apache.org/jira/browse/HIVE-3695
 Project: Hive
  Issue Type: Bug
  Components: Tests
Reporter: Namit Jain
Assignee: Namit Jain
 Attachments: hive.3695.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3675) NaN does not work correctly for round(n)


[ 
https://issues.apache.org/jira/browse/HIVE-3675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494505#comment-13494505
 ] 

Hudson commented on HIVE-3675:
--

Integrated in Hive-trunk-h0.21 #1787 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1787/])
HIVE-3695. TestParse breaks due to HIVE-3675. (njain via kevinwilfong) 
(Revision 1407558)

 Result = FAILURE
kevinwilfong : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1407558
Files : 
* /hive/trunk/ql/src/test/results/compiler/plan/udf4.q.xml


 NaN does not work correctly for round(n)
 

 Key: HIVE-3675
 URL: https://issues.apache.org/jira/browse/HIVE-3675
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Namit Jain
Assignee: Namit Jain
 Fix For: 0.10.0

 Attachments: hive.3675.1.patch, hive.3675.2.patch, hive.3675.3.patch


 It works correctly for round(n, d)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3698) enhanceModel.notRequired is incorrectly determined


[ 
https://issues.apache.org/jira/browse/HIVE-3698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13494512#comment-13494512
 ] 

Chris Drome commented on HIVE-3698:
---

branch-0.9:
https://reviews.facebook.net/D6633

trunk:
https://reviews.facebook.net/D6639

 enhanceModel.notRequired is incorrectly determined
 --

 Key: HIVE-3698
 URL: https://issues.apache.org/jira/browse/HIVE-3698
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure, Metastore
Affects Versions: 0.10.0, 0.9.1
Reporter: Chris Drome
Assignee: Chris Drome
 Attachments: HIVE-3698-0.9.patch, HIVE-3698-trunk.patch


 enhanceModel.notRequired in metastore/build.xml is not correctly determined 
 which can result in datanucleus running twice. This in turn can cause other 
 issues when building with JDK7.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3662) TestHiveServer: testScratchDirShouldClearWhileStartup is failing on Windows

2012-11-09 Thread Ashutosh Chauhan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-3662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-3662:
---

   Resolution: Fixed
Fix Version/s: (was: 0.9.1)
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Kanna!

 TestHiveServer: testScratchDirShouldClearWhileStartup is failing on Windows
 ---

 Key: HIVE-3662
 URL: https://issues.apache.org/jira/browse/HIVE-3662
 Project: Hive
  Issue Type: Bug
  Components: Tests
Affects Versions: 0.10.0, 0.9.1
Reporter: Kanna Karanam
Assignee: Kanna Karanam
  Labels: Windows
 Fix For: 0.10.0

 Attachments: HIVE-3662.1.patch.txt


 Test case is attempting to delete the ScratchDir but it is failing on Windows 
 because one of the subfolders (local scratchdir) in use. So change the 
 location of the local scratch directory.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification

[
https://issues.apache.org/jira/browse/HIVE-3384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Drome updated HIVE-3384:
--

Attachment: HIVE-JDK7-JDBC.patch

A couple of comments, rather than returning null or doing nothing for a void
method, you should probably throw an exception. That way it is clear that the
method has not been implemented yet.

Other parts of the code are throwing SQLException(Method not supported), so
you might want to do the same.

You missed some methods in HiveBaseResultSet.

I've included the patch that I'm using which contains these changes. You might
want to merge them together.

Also, I think you should submit your review request to phabricator instead of
apache review board.
https://cwiki.apache.org/confluence/display/Hive/PhabricatorCodeReview

HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC
specification
--

Key: HIVE-3384
URL: https://issues.apache.org/jira/browse/HIVE-3384
Project: Hive
Issue Type: Bug
Components: JDBC
Reporter: Weidong Bian
Priority: Minor
Attachments: HIVE-3384.2.patch, HIVE-3384.patch, HIVE-JDK7-JDBC.patch

jdbc module couldn't be compiled with jdk7 as it adds some abstract method in
the JDBC specification
some error info:
error: HiveCallableStatement is not abstract and does not override abstract
method TgetObject(String,ClassT) in CallableStatement
.
.
.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3659) TestHiveHistory::testQueryloglocParentDirNotExist Test fails on Windows because of some resource leaks in ZK

2012-11-09 Thread Ashutosh Chauhan (JIRA)

[
https://issues.apache.org/jira/browse/HIVE-3659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ashutosh Chauhan updated HIVE-3659:
---

Resolution: Fixed
Fix Version/s: (was: 0.9.1)
Status: Resolved (was: Patch Available)

Committed to trunk. Thanks, Kanna!

TestHiveHistory::testQueryloglocParentDirNotExist Test fails on Windows
because of some resource leaks in ZK

Key: HIVE-3659
URL: https://issues.apache.org/jira/browse/HIVE-3659
Project: Hive
Issue Type: Bug
Components: Tests
Affects Versions: 0.10.0, 0.9.1
Reporter: Kanna Karanam
Assignee: Kanna Karanam
Labels: Windows
Fix For: 0.10.0

Attachments: HIVE-3659.1.patch.txt

Hive uses ZK for locking. In some test cases, ZK is not behaving well. In
thread dumps, I saw it is waiting for locks to be released but they were not
getting released. Hive tries to release locks but keeps failing, it
eventually times out for its release attempts, which in default settings
takes 10 mins. This is also the cause of why some queries take
extra-ordinarily long to run. I suggest to disable ZK locking till ZK is
certified for windows.
In this test case, I don’t see a requirement to use ZK so I am disabling the
HiveConf.ConfVars.HIVE_SUPPORT_CONCURRENCY to work around the issue.

[jira] [Updated] (HIVE-3663) Unable to display the MR Job file path on Windows in case of MR job failures.

2012-11-09 Thread Ashutosh Chauhan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-3663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-3663:
---

   Resolution: Fixed
Fix Version/s: (was: 0.9.1)
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Kanna!

 Unable to display the MR Job file path on Windows in case of MR job failures.
 -

 Key: HIVE-3663
 URL: https://issues.apache.org/jira/browse/HIVE-3663
 Project: Hive
  Issue Type: Bug
  Components: Tests
Affects Versions: 0.10.0, 0.9.1
Reporter: Kanna Karanam
Assignee: Kanna Karanam
  Labels: Windows
 Fix For: 0.10.0

 Attachments: HIVE-3663.1.patch.txt


 Because of this bunch of CLI negative tests are failing on windows.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3384) HIVE JDBC module won't compile under JDK1.7 as new methods added in JDBC specification