[jira] [Updated] (HIVE-3314) Extract global limit configuration to optimizer

2012-08-30 Thread Carl Steinbach (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-3314:
-

Fix Version/s: 0.10.0

> Extract global limit configuration to optimizer
> ---
>
> Key: HIVE-3314
> URL: https://issues.apache.org/jira/browse/HIVE-3314
> Project: Hive
>  Issue Type: Task
>  Components: Query Processor
>Affects Versions: 0.10.0
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Fix For: 0.10.0
>
> Attachments: HIVE-3314.1.patch.txt
>
>
> SemanticAnalyzer is growing bigger and bigger. If some codes can be separated 
> cleanly, it would be better to do that for simplicity.
> Was in part of HIVE-2925. Suggested to separate issue as 
> https://issues.apache.org/jira/browse/HIVE-2925?focusedCommentId=13423754&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13423754

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3413) Fix pdk.PluginTest on hadoop23

2012-08-30 Thread Carl Steinbach (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-3413:
-

Component/s: Tests

> Fix pdk.PluginTest on hadoop23
> --
>
> Key: HIVE-3413
> URL: https://issues.apache.org/jira/browse/HIVE-3413
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Affects Versions: 0.9.0
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3413.1.patch.txt, HIVE-3413.2.patch.txt, 
> HIVE-3413.3.patch.txt
>
>
> When running Hive test on Hadoop0.23, pdk.PluginTest is failing:
> test:
> [junit] Running org.apache.hive.pdk.PluginTest
> [junit] Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> [junit] Total MapReduce jobs = 1
> [junit] Launching Job 1 out of 1
> [junit] Number of reduce tasks determined at compile time: 1
> [junit] In order to change the average load for a reducer (in bytes):
> [junit]   set hive.exec.reducers.bytes.per.reducer=
> [junit] In order to limit the maximum number of reducers:
> [junit]   set hive.exec.reducers.max=
> [junit] In order to set a constant number of reducers:
> [junit]   set mapred.reduce.tasks=
> [junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is 
> deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the 
> log4j.properties files.
> [junit] Execution log at: 
> /tmp/cloudera/cloudera_20120828184545_6deeb166-7dd4-40d3-9ff7-c5d5277aee39.log
> [junit] java.io.IOException: Cannot initialize Cluster. Please check your 
> configuration for mapreduce.framework.name and the correspond server 
> addresses.
> [junit] at 
> org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:123)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:85)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:78)
> [junit] at org.apache.hadoop.mapred.JobClient.init(JobClient.java:487)
> [junit] at 
> org.apache.hadoop.mapred.JobClient.(JobClient.java:466)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:424)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:688)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
> [junit] Job Submission failed with exception 'java.io.IOException(Cannot 
> initialize Cluster. Please check your configuration for 
> mapreduce.framework.name and the correspond server addresses.)'
> [junit] Execution failed with exit status: 1
> [junit] Obtaining error information
> [junit]
> [junit] Task failed!
> [junit] Task ID:
> [junit]   Stage-1
> [junit]
> [junit] Logs:
> [junit]
> [junit] /tmp/cloudera/hive.log
> [junit] FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask]>)
> [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> With details in ./build/builtins/TEST-org.apache.hive.pdk.PluginTest.txt:
> Testsuite: org.apache.hive.pdk.PluginTest
> Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> - Standard Error -
> GLOBAL SETUP:  Copying file: 
> file:/home/cloudera/Code/hive2/builtins/test/onerow.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/onerow
> Copying file: file:/home/cloudera/Code/hive2/builtins/test/iris.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/iris
> org.apache.hive.builtins.UDAFUnionMap TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_840355011.txt
> GLOBAL TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_25225.txt
> OK
> Time taken: 6.874 seconds
> OK
> Time taken: 0.512 seconds
> -  ---
> Testcase: SELECT size(UNION_MAP(MAP(sepal_width, sepal_length))) FROM iris 
> took 4.428 sec
> FAILED
> expected:<[23]> but was:<[
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=
> In order to set a constant number of reducers:
>   set mapred.re

[jira] [Commented] (HIVE-3413) Fix pdk.PluginTest on hadoop23

2012-08-30 Thread Carl Steinbach (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445728#comment-13445728
 ] 

Carl Steinbach commented on HIVE-3413:
--

+1. Will commit if tests pass.

> Fix pdk.PluginTest on hadoop23
> --
>
> Key: HIVE-3413
> URL: https://issues.apache.org/jira/browse/HIVE-3413
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Affects Versions: 0.9.0
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3413.1.patch.txt, HIVE-3413.2.patch.txt, 
> HIVE-3413.3.patch.txt
>
>
> When running Hive test on Hadoop0.23, pdk.PluginTest is failing:
> test:
> [junit] Running org.apache.hive.pdk.PluginTest
> [junit] Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> [junit] Total MapReduce jobs = 1
> [junit] Launching Job 1 out of 1
> [junit] Number of reduce tasks determined at compile time: 1
> [junit] In order to change the average load for a reducer (in bytes):
> [junit]   set hive.exec.reducers.bytes.per.reducer=
> [junit] In order to limit the maximum number of reducers:
> [junit]   set hive.exec.reducers.max=
> [junit] In order to set a constant number of reducers:
> [junit]   set mapred.reduce.tasks=
> [junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is 
> deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the 
> log4j.properties files.
> [junit] Execution log at: 
> /tmp/cloudera/cloudera_20120828184545_6deeb166-7dd4-40d3-9ff7-c5d5277aee39.log
> [junit] java.io.IOException: Cannot initialize Cluster. Please check your 
> configuration for mapreduce.framework.name and the correspond server 
> addresses.
> [junit] at 
> org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:123)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:85)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:78)
> [junit] at org.apache.hadoop.mapred.JobClient.init(JobClient.java:487)
> [junit] at 
> org.apache.hadoop.mapred.JobClient.(JobClient.java:466)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:424)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:688)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
> [junit] Job Submission failed with exception 'java.io.IOException(Cannot 
> initialize Cluster. Please check your configuration for 
> mapreduce.framework.name and the correspond server addresses.)'
> [junit] Execution failed with exit status: 1
> [junit] Obtaining error information
> [junit]
> [junit] Task failed!
> [junit] Task ID:
> [junit]   Stage-1
> [junit]
> [junit] Logs:
> [junit]
> [junit] /tmp/cloudera/hive.log
> [junit] FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask]>)
> [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> With details in ./build/builtins/TEST-org.apache.hive.pdk.PluginTest.txt:
> Testsuite: org.apache.hive.pdk.PluginTest
> Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> - Standard Error -
> GLOBAL SETUP:  Copying file: 
> file:/home/cloudera/Code/hive2/builtins/test/onerow.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/onerow
> Copying file: file:/home/cloudera/Code/hive2/builtins/test/iris.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/iris
> org.apache.hive.builtins.UDAFUnionMap TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_840355011.txt
> GLOBAL TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_25225.txt
> OK
> Time taken: 6.874 seconds
> OK
> Time taken: 0.512 seconds
> -  ---
> Testcase: SELECT size(UNION_MAP(MAP(sepal_width, sepal_length))) FROM iris 
> took 4.428 sec
> FAILED
> expected:<[23]> but was:<[
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=
> In order to limit the maximum number of reducers:
>   set hive.exec.red

[jira] [Commented] (HIVE-3419) drop partition does not work for non-equality oprator.

2012-08-30 Thread rohithsharma (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445726#comment-13445726
 ] 

rohithsharma commented on HIVE-3419:


JDO throws NullPointerException for above partitions.

{noformat}
2012-08-31 12:13:26,466 DEBUG metastore.ObjectStore 
(ObjectStore.java:listMPartitionsByFilter(1680)) - Executing 
listMPartitionsByFilter
2012-08-31 12:13:26,474 DEBUG metastore.ObjectStore 
(ObjectStore.java:makeQueryFilterString(1636)) - jdoFilter =  
partitionName.substring(partitionName.indexOf("c=")+2) != hive_filter_param_0
2012-08-31 12:13:26,475 DEBUG metastore.ObjectStore 
(ObjectStore.java:listMPartitionsByFilter(1701)) - Filter specified is c <> 
'2', JDOQL filter is table.tableName == t1 && table.database.name == t2 && ( 
partitionName.substring(partitionName.indexOf("c=")+2) != hive_filter_param_0 )
2012-08-31 12:13:26,478 ERROR ql.Driver (SessionState.java:printError(400)) - 
FAILED: Error in semantic analysis: Partition not found c <> '2'
org.apache.hadoop.hive.ql.parse.SemanticException: Partition not found c <> '2'
at 
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.addTableDropPartsOutputs(DDLSemanticAnalyzer.java:2249)
at 
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeAlterTableDropParts(DDLSemanticAnalyzer.java:1818)
at 
org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:300)
at 
org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:244)
...
...
...
Caused by: java.lang.NullPointerException
at 
org.datanucleus.store.mapped.mapping.MappingHelper.getMappingIndices(MappingHelper.java:35)
at 
org.datanucleus.store.mapped.expression.StatementText.applyParametersToStatement(StatementText.java:194)
at 
org.datanucleus.store.rdbms.query.RDBMSQueryUtils.getPreparedStatementForQuery(RDBMSQueryUtils.java:233)
at 
org.datanucleus.store.rdbms.query.legacy.SQLEvaluator.evaluate(SQLEvaluator.java:115)
at 
org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.performExecute(JDOQLQuery.java:288)
at org.datanucleus.store.query.Query.executeQuery(Query.java:1657)
at 
org.datanucleus.store.rdbms.query.legacy.JDOQLQuery.executeQuery(JDOQLQuery.java:245)
at org.datanucleus.store.query.Query.executeWithMap(Query.java:1526)
at org.datanucleus.jdo.JDOQuery.executeWithMap(JDOQuery.java:334)
at 
org.apache.hadoop.hive.metastore.ObjectStore.listMPartitionsByFilter(ObjectStore.java:1711)
at 
org.apache.hadoop.hive.metastore.ObjectStore.getPartitionsByFilter(ObjectStore.java:1586)
{noformat}

> drop partition does not work for non-equality oprator.
> --
>
> Key: HIVE-3419
> URL: https://issues.apache.org/jira/browse/HIVE-3419
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore, Query Processor
>Affects Versions: 0.10.0, 0.9.1
> Environment: Hive -0.9.1
> postgres-9.0.7 as metastore.
>Reporter: rohithsharma
>
> Drop partition query failes when below query is executed with postgres as 
> metastore.Whereas when derby is used, drop partition query is pased.
> create table ptestfilter (a string, b int) partitioned by (c string);
> alter table ptestfilter1 add partition (c='1');
> alter table ptestfilter1 add partition (c='2');
> "alter table ptestfilter1 drop partition (c!='2');"--> failed

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3393) get_json_object and json_tuple should use Jackson library

2012-08-30 Thread Namit Jain (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445724#comment-13445724
 ] 

Namit Jain commented on HIVE-3393:
--

+1

> get_json_object and json_tuple should use Jackson library
> -
>
> Key: HIVE-3393
> URL: https://issues.apache.org/jira/browse/HIVE-3393
> Project: Hive
>  Issue Type: Improvement
>  Components: UDF
>Affects Versions: 0.10.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
>Priority: Minor
> Attachments: HIVE-3393.1.patch.txt
>
>
> The Jackson library's JSON parsers have been shown to be significantly faster 
> that json.org's.  The library is already included, so I can't think of a 
> reason not to use it.
> There's also the potential for further improvements in replacing many of the 
> try catch blocks with if statements.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3419) drop partition does not work for non-equality oprator.

2012-08-30 Thread rohithsharma (JIRA) 
rohithsharma created HIVE-3419:
--

 Summary: drop partition does not work for non-equality oprator.
 Key: HIVE-3419
 URL: https://issues.apache.org/jira/browse/HIVE-3419
 Project: Hive
  Issue Type: Bug
  Components: Metastore, Query Processor
Affects Versions: 0.10.0, 0.9.1
 Environment: Hive -0.9.1
postgres-9.0.7 as metastore.
Reporter: rohithsharma


Drop partition query failes when below query is executed with postgres as 
metastore.Whereas when derby is used, drop partition query is pased.

create table ptestfilter (a string, b int) partitioned by (c string);
alter table ptestfilter1 add partition (c='1');
alter table ptestfilter1 add partition (c='2');
"alter table ptestfilter1 drop partition (c!='2');"--> failed


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3393) get_json_object and json_tuple should use Jackson library

2012-08-30 Thread Namit Jain (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3393:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed. Thanks Kevin

> get_json_object and json_tuple should use Jackson library
> -
>
> Key: HIVE-3393
> URL: https://issues.apache.org/jira/browse/HIVE-3393
> Project: Hive
>  Issue Type: Improvement
>  Components: UDF
>Affects Versions: 0.10.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
>Priority: Minor
> Attachments: HIVE-3393.1.patch.txt
>
>
> The Jackson library's JSON parsers have been shown to be significantly faster 
> that json.org's.  The library is already included, so I can't think of a 
> reason not to use it.
> There's also the potential for further improvements in replacing many of the 
> try catch blocks with if statements.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3391) Keep the original query in HiveDriverRunHookContextImpl

2012-08-30 Thread Carl Steinbach (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Carl Steinbach updated HIVE-3391:
-

Status: Open  (was: Patch Available)

More comments on phabricator. Thanks.

> Keep the original query in HiveDriverRunHookContextImpl
> ---
>
> Key: HIVE-3391
> URL: https://issues.apache.org/jira/browse/HIVE-3391
> Project: Hive
>  Issue Type: Improvement
>Reporter: Dawid Dabrowski
>Assignee: Dawid Dabrowski
>Priority: Minor
> Attachments: HIVE-3391.1.patch.txt
>
>   Original Estimate: 72h
>  Time Spent: 96h
>  Remaining Estimate: 0h
>
> It'd be useful to have access to the original query in hooks. The hook that's 
> executed first is HiveDriverRunHook, let's add it there.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2084) Upgrade datanucleus from 2.0.3 to 3.0.1

2012-08-30 Thread Carl Steinbach (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-2084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445528#comment-13445528
 ] 

Carl Steinbach commented on HIVE-2084:
--

@Sushanth: Looks like this patch fell out of sync with trunk. Would you be 
willing to rebase and post a new copy? Thanks.

> Upgrade datanucleus from 2.0.3 to 3.0.1
> ---
>
> Key: HIVE-2084
> URL: https://issues.apache.org/jira/browse/HIVE-2084
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Ning Zhang
>Assignee: Sushanth Sowmyan
>  Labels: datanucleus
> Attachments: ASF.LICENSE.NOT.GRANTED--HIVE-2084.D2397.1.patch, 
> HIVE-2084.1.patch.txt, HIVE-2084.2.patch.txt, HIVE-2084.patch
>
>
> It seems the datanucleus 2.2.3 does a better join in caching. The time it 
> takes to get the same set of partition objects takes about 1/4 of the time it 
> took for the first time. While with 2.0.3, it took almost the same amount of 
> time in the second execution. We should retest the test case mentioned in 
> HIVE-1853, HIVE-1862.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3413) Fix pdk.PluginTest on hadoop23

2012-08-30 Thread Zhenxiao Luo (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-3413:
---

Status: Patch Available  (was: Open)

> Fix pdk.PluginTest on hadoop23
> --
>
> Key: HIVE-3413
> URL: https://issues.apache.org/jira/browse/HIVE-3413
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3413.1.patch.txt, HIVE-3413.2.patch.txt, 
> HIVE-3413.3.patch.txt
>
>
> When running Hive test on Hadoop0.23, pdk.PluginTest is failing:
> test:
> [junit] Running org.apache.hive.pdk.PluginTest
> [junit] Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> [junit] Total MapReduce jobs = 1
> [junit] Launching Job 1 out of 1
> [junit] Number of reduce tasks determined at compile time: 1
> [junit] In order to change the average load for a reducer (in bytes):
> [junit]   set hive.exec.reducers.bytes.per.reducer=
> [junit] In order to limit the maximum number of reducers:
> [junit]   set hive.exec.reducers.max=
> [junit] In order to set a constant number of reducers:
> [junit]   set mapred.reduce.tasks=
> [junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is 
> deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the 
> log4j.properties files.
> [junit] Execution log at: 
> /tmp/cloudera/cloudera_20120828184545_6deeb166-7dd4-40d3-9ff7-c5d5277aee39.log
> [junit] java.io.IOException: Cannot initialize Cluster. Please check your 
> configuration for mapreduce.framework.name and the correspond server 
> addresses.
> [junit] at 
> org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:123)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:85)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:78)
> [junit] at org.apache.hadoop.mapred.JobClient.init(JobClient.java:487)
> [junit] at 
> org.apache.hadoop.mapred.JobClient.(JobClient.java:466)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:424)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:688)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
> [junit] Job Submission failed with exception 'java.io.IOException(Cannot 
> initialize Cluster. Please check your configuration for 
> mapreduce.framework.name and the correspond server addresses.)'
> [junit] Execution failed with exit status: 1
> [junit] Obtaining error information
> [junit]
> [junit] Task failed!
> [junit] Task ID:
> [junit]   Stage-1
> [junit]
> [junit] Logs:
> [junit]
> [junit] /tmp/cloudera/hive.log
> [junit] FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask]>)
> [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> With details in ./build/builtins/TEST-org.apache.hive.pdk.PluginTest.txt:
> Testsuite: org.apache.hive.pdk.PluginTest
> Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> - Standard Error -
> GLOBAL SETUP:  Copying file: 
> file:/home/cloudera/Code/hive2/builtins/test/onerow.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/onerow
> Copying file: file:/home/cloudera/Code/hive2/builtins/test/iris.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/iris
> org.apache.hive.builtins.UDAFUnionMap TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_840355011.txt
> GLOBAL TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_25225.txt
> OK
> Time taken: 6.874 seconds
> OK
> Time taken: 0.512 seconds
> -  ---
> Testcase: SELECT size(UNION_MAP(MAP(sepal_width, sepal_length))) FROM iris 
> took 4.428 sec
> FAILED
> expected:<[23]> but was:<[
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=
> In order to set a constant number of reducers:
>   set mapred.reduce.tasks=
> W

[jira] [Updated] (HIVE-3413) Fix pdk.PluginTest on hadoop23

2012-08-30 Thread Zhenxiao Luo (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-3413:
---

Attachment: HIVE-3413.3.patch.txt

> Fix pdk.PluginTest on hadoop23
> --
>
> Key: HIVE-3413
> URL: https://issues.apache.org/jira/browse/HIVE-3413
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3413.1.patch.txt, HIVE-3413.2.patch.txt, 
> HIVE-3413.3.patch.txt
>
>
> When running Hive test on Hadoop0.23, pdk.PluginTest is failing:
> test:
> [junit] Running org.apache.hive.pdk.PluginTest
> [junit] Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> [junit] Total MapReduce jobs = 1
> [junit] Launching Job 1 out of 1
> [junit] Number of reduce tasks determined at compile time: 1
> [junit] In order to change the average load for a reducer (in bytes):
> [junit]   set hive.exec.reducers.bytes.per.reducer=
> [junit] In order to limit the maximum number of reducers:
> [junit]   set hive.exec.reducers.max=
> [junit] In order to set a constant number of reducers:
> [junit]   set mapred.reduce.tasks=
> [junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is 
> deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the 
> log4j.properties files.
> [junit] Execution log at: 
> /tmp/cloudera/cloudera_20120828184545_6deeb166-7dd4-40d3-9ff7-c5d5277aee39.log
> [junit] java.io.IOException: Cannot initialize Cluster. Please check your 
> configuration for mapreduce.framework.name and the correspond server 
> addresses.
> [junit] at 
> org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:123)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:85)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:78)
> [junit] at org.apache.hadoop.mapred.JobClient.init(JobClient.java:487)
> [junit] at 
> org.apache.hadoop.mapred.JobClient.(JobClient.java:466)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:424)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:688)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
> [junit] Job Submission failed with exception 'java.io.IOException(Cannot 
> initialize Cluster. Please check your configuration for 
> mapreduce.framework.name and the correspond server addresses.)'
> [junit] Execution failed with exit status: 1
> [junit] Obtaining error information
> [junit]
> [junit] Task failed!
> [junit] Task ID:
> [junit]   Stage-1
> [junit]
> [junit] Logs:
> [junit]
> [junit] /tmp/cloudera/hive.log
> [junit] FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask]>)
> [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> With details in ./build/builtins/TEST-org.apache.hive.pdk.PluginTest.txt:
> Testsuite: org.apache.hive.pdk.PluginTest
> Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> - Standard Error -
> GLOBAL SETUP:  Copying file: 
> file:/home/cloudera/Code/hive2/builtins/test/onerow.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/onerow
> Copying file: file:/home/cloudera/Code/hive2/builtins/test/iris.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/iris
> org.apache.hive.builtins.UDAFUnionMap TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_840355011.txt
> GLOBAL TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_25225.txt
> OK
> Time taken: 6.874 seconds
> OK
> Time taken: 0.512 seconds
> -  ---
> Testcase: SELECT size(UNION_MAP(MAP(sepal_width, sepal_length))) FROM iris 
> took 4.428 sec
> FAILED
> expected:<[23]> but was:<[
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=
> In order to set a constant number of reducers:
>   set mapred.reduce.tasks=
> WARN

[jira] [Commented] (HIVE-3413) Fix pdk.PluginTest on hadoop23

2012-08-30 Thread Zhenxiao Luo (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445520#comment-13445520
 ] 

Zhenxiao Luo commented on HIVE-3413:


the missing hadoop-minicluster dependency should not go into ql/ivy.xml.
It should be in hadoop23.test, putting into build/ivy/lib/test
pdk plugin test is triggered via builtin/build.xml
add the dependency in builtin/ivy.xml, and also update builtin/build.xml,
so that ivy-retrieve-test dependency is added to target test

Updated patch submitted for review at:
https://reviews.facebook.net/D5001

> Fix pdk.PluginTest on hadoop23
> --
>
> Key: HIVE-3413
> URL: https://issues.apache.org/jira/browse/HIVE-3413
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3413.1.patch.txt, HIVE-3413.2.patch.txt
>
>
> When running Hive test on Hadoop0.23, pdk.PluginTest is failing:
> test:
> [junit] Running org.apache.hive.pdk.PluginTest
> [junit] Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> [junit] Total MapReduce jobs = 1
> [junit] Launching Job 1 out of 1
> [junit] Number of reduce tasks determined at compile time: 1
> [junit] In order to change the average load for a reducer (in bytes):
> [junit]   set hive.exec.reducers.bytes.per.reducer=
> [junit] In order to limit the maximum number of reducers:
> [junit]   set hive.exec.reducers.max=
> [junit] In order to set a constant number of reducers:
> [junit]   set mapred.reduce.tasks=
> [junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is 
> deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the 
> log4j.properties files.
> [junit] Execution log at: 
> /tmp/cloudera/cloudera_20120828184545_6deeb166-7dd4-40d3-9ff7-c5d5277aee39.log
> [junit] java.io.IOException: Cannot initialize Cluster. Please check your 
> configuration for mapreduce.framework.name and the correspond server 
> addresses.
> [junit] at 
> org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:123)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:85)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:78)
> [junit] at org.apache.hadoop.mapred.JobClient.init(JobClient.java:487)
> [junit] at 
> org.apache.hadoop.mapred.JobClient.(JobClient.java:466)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:424)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:688)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
> [junit] Job Submission failed with exception 'java.io.IOException(Cannot 
> initialize Cluster. Please check your configuration for 
> mapreduce.framework.name and the correspond server addresses.)'
> [junit] Execution failed with exit status: 1
> [junit] Obtaining error information
> [junit]
> [junit] Task failed!
> [junit] Task ID:
> [junit]   Stage-1
> [junit]
> [junit] Logs:
> [junit]
> [junit] /tmp/cloudera/hive.log
> [junit] FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask]>)
> [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> With details in ./build/builtins/TEST-org.apache.hive.pdk.PluginTest.txt:
> Testsuite: org.apache.hive.pdk.PluginTest
> Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> - Standard Error -
> GLOBAL SETUP:  Copying file: 
> file:/home/cloudera/Code/hive2/builtins/test/onerow.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/onerow
> Copying file: file:/home/cloudera/Code/hive2/builtins/test/iris.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/iris
> org.apache.hive.builtins.UDAFUnionMap TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_840355011.txt
> GLOBAL TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_25225.txt
> OK
> Time taken: 6.874 seconds
> OK
> Time taken: 0.512 seconds
> -  ---
> Testcase: SELECT size(UNION_MAP(MAP(sepal_width, sepal_length))) FROM iris 
> took 4.428 sec
> FAILED
> expected:<[23]> but was:<[
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845

[jira] [Updated] (HIVE-3413) Fix pdk.PluginTest on hadoop23

2012-08-30 Thread Zhenxiao Luo (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-3413:
---

Status: Open  (was: Patch Available)

> Fix pdk.PluginTest on hadoop23
> --
>
> Key: HIVE-3413
> URL: https://issues.apache.org/jira/browse/HIVE-3413
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.9.0
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3413.1.patch.txt, HIVE-3413.2.patch.txt
>
>
> When running Hive test on Hadoop0.23, pdk.PluginTest is failing:
> test:
> [junit] Running org.apache.hive.pdk.PluginTest
> [junit] Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> [junit] Total MapReduce jobs = 1
> [junit] Launching Job 1 out of 1
> [junit] Number of reduce tasks determined at compile time: 1
> [junit] In order to change the average load for a reducer (in bytes):
> [junit]   set hive.exec.reducers.bytes.per.reducer=
> [junit] In order to limit the maximum number of reducers:
> [junit]   set hive.exec.reducers.max=
> [junit] In order to set a constant number of reducers:
> [junit]   set mapred.reduce.tasks=
> [junit] WARNING: org.apache.hadoop.metrics.jvm.EventCounter is 
> deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the 
> log4j.properties files.
> [junit] Execution log at: 
> /tmp/cloudera/cloudera_20120828184545_6deeb166-7dd4-40d3-9ff7-c5d5277aee39.log
> [junit] java.io.IOException: Cannot initialize Cluster. Please check your 
> configuration for mapreduce.framework.name and the correspond server 
> addresses.
> [junit] at 
> org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:123)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:85)
> [junit] at org.apache.hadoop.mapreduce.Cluster.(Cluster.java:78)
> [junit] at org.apache.hadoop.mapred.JobClient.init(JobClient.java:487)
> [junit] at 
> org.apache.hadoop.mapred.JobClient.(JobClient.java:466)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.execute(ExecDriver.java:424)
> [junit] at 
> org.apache.hadoop.hive.ql.exec.ExecDriver.main(ExecDriver.java:688)
> [junit] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> [junit] at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> [junit] at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> [junit] at java.lang.reflect.Method.invoke(Method.java:616)
> [junit] at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
> [junit] Job Submission failed with exception 'java.io.IOException(Cannot 
> initialize Cluster. Please check your configuration for 
> mapreduce.framework.name and the correspond server addresses.)'
> [junit] Execution failed with exit status: 1
> [junit] Obtaining error information
> [junit]
> [junit] Task failed!
> [junit] Task ID:
> [junit]   Stage-1
> [junit]
> [junit] Logs:
> [junit]
> [junit] /tmp/cloudera/hive.log
> [junit] FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask]>)
> [junit] Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> With details in ./build/builtins/TEST-org.apache.hive.pdk.PluginTest.txt:
> Testsuite: org.apache.hive.pdk.PluginTest
> Tests run: 1, Failures: 1, Errors: 0, Time elapsed: 33.9 sec
> - Standard Error -
> GLOBAL SETUP:  Copying file: 
> file:/home/cloudera/Code/hive2/builtins/test/onerow.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/onerow
> Copying file: file:/home/cloudera/Code/hive2/builtins/test/iris.txt
> Deleted /home/cloudera/Code/hive2/build/builtins/warehouse/iris
> org.apache.hive.builtins.UDAFUnionMap TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_840355011.txt
> GLOBAL TEARDOWN:
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_25225.txt
> OK
> Time taken: 6.874 seconds
> OK
> Time taken: 0.512 seconds
> -  ---
> Testcase: SELECT size(UNION_MAP(MAP(sepal_width, sepal_length))) FROM iris 
> took 4.428 sec
> FAILED
> expected:<[23]> but was:<[
> Hive history 
> file=/tmp/cloudera/hive_job_log_cloudera_201208281845_172375530.txt
> Total MapReduce jobs = 1
> Launching Job 1 out of 1
> Number of reduce tasks determined at compile time: 1
> In order to change the average load for a reducer (in bytes):
>   set hive.exec.reducers.bytes.per.reducer=
> In order to limit the maximum number of reducers:
>   set hive.exec.reducers.max=
> In order to set a constant number of reducers:
>   set mapred.reduce.tasks=
> WARNING: org.apache.hadoop.

[jira] [Commented] (HIVE-3323) ThriftSerde: Enable enum to string conversions

2012-08-30 Thread Ashutosh Chauhan (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445500#comment-13445500
 ] 

Ashutosh Chauhan commented on HIVE-3323:


Travis,
Can you create a review request on reviews.apache.org ?

> ThriftSerde: Enable enum to string conversions
> --
>
> Key: HIVE-3323
> URL: https://issues.apache.org/jira/browse/HIVE-3323
> Project: Hive
>  Issue Type: Improvement
>Affects Versions: 0.10.0
>Reporter: Travis Crawford
>Assignee: Travis Crawford
> Attachments: HIVE-3323_enum_to_string.1.patch, 
> HIVE-3323_enum_to_string.2.patch, HIVE-3323_enum_to_string.3.patch, 
> HIVE-3323_enum_to_string.4.patch, HIVE-3323_enum_to_string.5.patch
>
>
> When using serde-reported schemas with the ThriftDeserializer, Enum fields 
> are presented as {{struct}}
> Many users expect to work with the string values, which is both easier and 
> more meaningful as the string value communicates what is represented.
> Hive should provide a mechanism to optionally convert enum values to strings.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3121) JDBC driver's getCatalogs() method returns schema/db information

2012-08-30 Thread Carl Steinbach (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445486#comment-13445486
 ] 

Carl Steinbach commented on HIVE-3121:
--

@Richard: Sorry for the slow response. Unfortunately the code review link does 
not get automatically posted to JIRA. Can you please paste the link here? 
Thanks.

> JDBC driver's getCatalogs() method returns schema/db information
> 
>
> Key: HIVE-3121
> URL: https://issues.apache.org/jira/browse/HIVE-3121
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 0.9.0
>Reporter: Carl Steinbach
>Assignee: Richard Ding
> Attachments: hive-3121_1.patch, hive-3121.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3121) JDBC driver's getCatalogs() method returns schema/db information

2012-08-30 Thread Richard Ding (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Richard Ding updated HIVE-3121:
---

Assignee: Richard Ding

> JDBC driver's getCatalogs() method returns schema/db information
> 
>
> Key: HIVE-3121
> URL: https://issues.apache.org/jira/browse/HIVE-3121
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 0.9.0
>Reporter: Carl Steinbach
>Assignee: Richard Ding
> Attachments: hive-3121_1.patch, hive-3121.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3410) All operators's conf should inherit from a common class

2012-08-30 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445344#comment-13445344
 ] 

Hudson commented on HIVE-3410:
--

Integrated in Hive-trunk-h0.21 #1637 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1637/])
HIVE-3410 All operators's conf should inherit from a common class
(Namit via Carl) (Revision 1378659)

 Result = SUCCESS
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1378659
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/Driver.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/QueryPlan.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExecMapper.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/ExplainTask.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/GroupByOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Operator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/OperatorFactory.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SMBMapJoinOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/SkewJoinHandler.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/TerminalOperator.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/index/compact/CompactIndexHandler.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveFileFormatUtils.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/HiveInputFormat.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/lib/DefaultGraphWalker.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Table.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/BucketMapJoinOptimizer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPruner.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcCtx.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRFileSink1.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMROperator.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRProcContext.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRRedSink1.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRRedSink2.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRRedSink3.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRRedSink4.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRTableScan1.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMRUnion1.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GlobalLimitOptimizer.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/GroupByOptimizer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/JoinReorder.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/MapJoinProcessor.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkDeDuplication.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SimpleFetchOptimizer.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/SortedMergeBucketMapJoinOptimizer.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/index/RewriteCanApplyCtx.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/index/RewriteCanApplyProcFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/index/RewriteGBUsingIndex.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/index/RewriteQueryUsingAggregateIndex.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/index/RewriteQueryUsingAggregateIndexCtx.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/lineage/ExprProcCtx.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/lineage/ExprProcFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/lineage/LineageCtx.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/optimizer/lineage/OpProcFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hi

[jira] [Commented] (HIVE-3245) UTF encoded data not displayed correctly by Hive driver

2012-08-30 Thread Mark Grover (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13445297#comment-13445297
 ] 

Mark Grover commented on HIVE-3245:
---

I got into some trouble with the JDBC driver on Hive 0.7.1 as well. I did some 
poking around too but couldn't spend much time. While doing so, I got to 
org.apache.hadoop.hive.jdbc.HiveQueryResultSet class. Inside next(), the code 
has:

{code:title=HiveQueryResultSet.java|borderStyle=solid}
Object data = serde.deserialize(new BytesWritable(rowStr.getBytes()));
{code}

Now, getBytes() comes in two variants, one that takes no parameters and uses 
the default encoding (like in the above row) or one that explicitly takes the 
encoding as parameter. I have a hunch that this could be a problem and that the 
encoding should be sent as a parameter. However, I haven't gotten the chance to 
verify/refute my hunch.

> UTF encoded data not displayed correctly by Hive driver
> ---
>
> Key: HIVE-3245
> URL: https://issues.apache.org/jira/browse/HIVE-3245
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 0.8.0
>Reporter: N Campbell
> Attachments: ASF.LICENSE.NOT.GRANTED--screenshot-1.jpg, CERT.TLJA.txt
>
>
> various foreign language data (i.e. japanese, thai etc) is loaded into string 
> columns via tab delimited text files. A simple projection of the columns in 
> the table is not displaying the correct data. Exporting the data from Hive 
> and looking at the files implies the data is loaded properly. it appears to 
> be an encoding issue at the driver but unaware of any required URL connection 
> properties re encoding that Hive JDBC requires.
> create table if not exists CERT.TLJA_JP_E ( RNUM int , C1 string, ORD int)
> row format delimited
> fields terminated by '\t'
> stored as textfile;
> create table if not exists CERT.TLJA_JP ( RNUM int , C1 string, ORD int)
> stored as sequencefile;
> load data local inpath '/home/hadoopadmin/jdbc-cert/CERT/CERT.TLJA_JP.txt'
> overwrite into table CERT.TLJA_JP_E;
> insert overwrite table CERT.TLJA_JP  select * from CERT.TLJA_JP_E;

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3391) Keep the original query in HiveDriverRunHookContextImpl

2012-08-30 Thread Dawid Dabrowski (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-3391?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Dawid Dabrowski updated HIVE-3391:
--

Status: Patch Available  (was: Open)

https://reviews.facebook.net/D4923

> Keep the original query in HiveDriverRunHookContextImpl
> ---
>
> Key: HIVE-3391
> URL: https://issues.apache.org/jira/browse/HIVE-3391
> Project: Hive
>  Issue Type: Improvement
>Reporter: Dawid Dabrowski
>Assignee: Dawid Dabrowski
>Priority: Minor
> Attachments: HIVE-3391.1.patch.txt
>
>   Original Estimate: 72h
>  Time Spent: 96h
>  Remaining Estimate: 0h
>
> It'd be useful to have access to the original query in hooks. The hook that's 
> executed first is HiveDriverRunHook, let's add it there.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-3056: Ability to bulk update location field in Db/Table/Partition records

2012-08-30 Thread Shreepadma Venugopalan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6650/#review10848
---



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


Will do.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


I'll remove HiveMetaTool from the logging information.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


Will do.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


Will remove.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


This logic checks if the location URI on disk matches the location URI that 
the user has specified for the old-nn-location. ObjectStore retrieves the 
locationURI from the metastore and needs to verify that the locationURI in 
metastore matches the URI that the user has passed in before updating the 
records in the metastore. I'm not sure why this logic should be in 
HiveMetaTool. 



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


This logic checks if the URI in the metastore record matches the NN 
location URI that the user has passed in. ObjectStore retrieves the metastore 
records and decides to update a given record based on whether the record's 
current location URI matches the URI that the user has passed in. This logic is 
an integral and central part of the update logic which belongs in the 
ObjectStore. I don't see how this logic can belong elsewhere. 



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


Will remove.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


The output of the dryRun i.e., the current location and the new location 
are returned as a map of string values that are printed to stdout by 
HiveMetaTool. I think most of the logic in this function belongs in the 
ObjectStore because it uses the JDO and DN  to access and write to the 
metastore records. Based on the previous revisions, 
I understand that we want to encapsulate all JDO/DN logic within 
ObjectStore.

I'll rename dryRun to isDryRun.

I think we should do the update in a single transaction ie., the HA upgrade 
should be atomic. If for some reason the upgrade fails, I think its better to 
have all of the records in the same state. The admin may later choose to retry 
the update. If we allowed non-atomic or partial updates then we would end up in 
a state where some records point to the HA NN and some others don't. In such a 
scenario some tables and the underlying data would be inaccessible. On the 
other hand if we allowed only an atomic upgrade and if the upgrade of the 
metastore failed the admin may choose not to use a HA NN till the HA upgrade of 
the metastore can be successfully completed.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


I'll split this method into updates for DB, SerdeParam and 
StorageDescriptor.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


Will do.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


Will remove.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


This tab was added by eclipse when I split the line up. I'll try to remove 
this.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


I think we should either update all the metastore records or none. Updating 
some of the records and failing in between leaves the metastore in an 
inconsistent state with some records updated and others not. I think the 
metastore update should be atomic. Please see my earlier comment on this.



metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java


While some of the logic such as some of the printing can be moved to the 
metatool, most of the logic in this method i.e., the logic that accesses the 
metastore through JDO should probably reside in objectstore. While executing 
some of that logic it becomes useful to output information e.g., the stage of 
processing we are in etc. I think its OK to use LOG.info to do some of that 
since the purpose of doing so is for debugging if there

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #121

2012-08-30 Thread Apache Jenkins Server 
See 


--
[...truncated 10219 lines...]
 [echo] Project: odbc
 [copy] Warning: 

 does not exist.

ivy-resolve-test:
 [echo] Project: odbc

ivy-retrieve-test:
 [echo] Project: odbc

compile-test:
 [echo] Project: odbc

create-dirs:
 [echo] Project: serde
 [copy] Warning: 

 does not exist.

init:
 [echo] Project: serde

ivy-init-settings:
 [echo] Project: serde

ivy-resolve:
 [echo] Project: serde
[ivy:resolve] :: loading settings :: file = 

[ivy:report] Processing 

 to 


ivy-retrieve:
 [echo] Project: serde

dynamic-serde:

compile:
 [echo] Project: serde

ivy-resolve-test:
 [echo] Project: serde

ivy-retrieve-test:
 [echo] Project: serde

compile-test:
 [echo] Project: serde
[javac] Compiling 26 source files to 

[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

create-dirs:
 [echo] Project: service
 [copy] Warning: 

 does not exist.

init:
 [echo] Project: service

ivy-init-settings:
 [echo] Project: service

ivy-resolve:
 [echo] Project: service
[ivy:resolve] :: loading settings :: file = 

[ivy:report] Processing 

 to 


ivy-retrieve:
 [echo] Project: service

compile:
 [echo] Project: service

ivy-resolve-test:
 [echo] Project: service

ivy-retrieve-test:
 [echo] Project: service

compile-test:
 [echo] Project: service
[javac] Compiling 2 source files to 


test:
 [echo] Project: hive

test-shims:
 [echo] Project: hive

test-conditions:
 [echo] Project: shims

gen-test:
 [echo] Project: shims

create-dirs:
 [echo] Project: shims
 [copy] Warning: 

 does not exist.

init:
 [echo] Project: shims

ivy-init-settings:
 [echo] Project: shims

ivy-resolve:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 

[ivy:report] Processing 

 to 


ivy-retrieve:
 [echo] Project: shims

compile:
 [echo] Project: shims
 [echo] Building shims 0.20

build_shims:
 [echo] Project: shims
 [echo] Compiling 

 against hadoop 0.20.2 
(

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 


ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.20S

build_shims:
 [echo] Project: shims
 [echo] Compiling

[jira] [Commented] (HIVE-3417) mulit inserts when the from statement is a subquery,this is a bug

2012-08-30 Thread Zhuoluo (Clark) Yang (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-3417?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444756#comment-13444756
 ] 

Zhuoluo (Clark) Yang commented on HIVE-3417:


I think this bug was involved by HIVE-1538 for the optimizer prunes the wrong 
filters. And I think we can modify the optimizer to make it work in good 
manners.

> mulit inserts when the from statement is a subquery,this is a bug
> -
>
> Key: HIVE-3417
> URL: https://issues.apache.org/jira/browse/HIVE-3417
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor, SQL
>Affects Versions: 0.8.1
> Environment: Linux 3.0.0-14-generic #23-Ubuntu SMP Mon Nov 21 
> 20:34:47 UTC 2011 i686 i686 i386 GNU/Linux
> java version "1.6.0_25"
> hadoop-0.20.2-cdh3u0
> hive-0.8.1
>Reporter: caofangkun
>
> vi mulit-insert.sql
> create table src (key string, value string);
> load data local inpath './in1.txt' overwrite into table src;
> drop table if exists test1;
> drop table if exists test2;
> create table test1 (key string, value string) partitioned by (dt string);
> create table test2 (key string, value string) partitioned by (dt string);
> select * from src;
> from (select * from src
>   where key is not null
>   ) --there is a bug here 
> insert overwrite table test1 PARTITION (dt='1') select key ,value where 
> key='48'
> insert overwrite table test2 PARTITION (dt='2') select key, value where 
> key='100';
> select * from test1;
> select * from test2;
> test1 and test2 shoud both have a single line of context.But it's not .
> Has a Solution:
> when set hive.ppd.remove.duplicatefilters=false;
> this's not such bug.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira