[jira] [Assigned] (HIVE-2005) Implement BETWEEN operator

[John Sichi (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi reassigned HIVE-2005:


Assignee: Navis  (was: Carl Steinbach)

> Implement BETWEEN operator
> --
>
> Key: HIVE-2005
> URL: https://issues.apache.org/jira/browse/HIVE-2005
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.9.0
>Reporter: Carl Steinbach
>Assignee: Navis
> Fix For: 0.9.0
>
> Attachments: HIVE-2005.1.patch.txt
>
>
> Reference: 
> http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2005) Implement BETWEEN operator

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164215#comment-13164215
 ] 

jirapos...@reviews.apache.org commented on HIVE-2005:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3008/
---

(Updated 2011-12-07 07:43:39.996429)


Review request for hive, John Sichi and Carl Steinbach.


Changes
---

1. naming 'between' made exception.
2. added ASF License header.
3. result of show_functions.q has additional function named 'union_map'. it's 
not mine.


Summary
---

Revised version using 'between' UDF


This addresses bug HIVE-2005.
https://issues.apache.org/jira/browse/HIVE-2005


Diffs (updated)
-

  ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION 
  ql/src/test/results/clientpositive/show_functions.q.out e63516d 
  ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 
  ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/3008/diff


Testing
---

added new test : udf_between.q


Thanks,

Navis



> Implement BETWEEN operator
> --
>
> Key: HIVE-2005
> URL: https://issues.apache.org/jira/browse/HIVE-2005
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.9.0
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.9.0
>
> Attachments: HIVE-2005.1.patch.txt
>
>
> Reference: 
> http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-2005 Implement BETWEEN operator

[Navis Ryu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3008/
---

(Updated 2011-12-07 07:43:39.996429)


Review request for hive, John Sichi and Carl Steinbach.


Changes
---

1. naming 'between' made exception.
2. added ASF License header.
3. result of show_functions.q has additional function named 'union_map'. it's 
not mine.


Summary
---

Revised version using 'between' UDF


This addresses bug HIVE-2005.
https://issues.apache.org/jira/browse/HIVE-2005


Diffs (updated)
-

  ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION 
  ql/src/test/results/clientpositive/show_functions.q.out e63516d 
  ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java e1c47b4 
  ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java 
PRE-CREATION 

Diff: https://reviews.apache.org/r/3008/diff


Testing
---

added new test : udf_between.q


Thanks,

Navis

Problems while installing SQOOP

[Bhavesh Shah]

Hello,
I am installing SQOOP on my PC. But i m facing some problems like
while executing command as:
hadoop@ubuntu:~/sqoop-1.3.0-cdh3u1$ bin/sqoop

then I get warning related to HBase.
But when I edited bin/configure-sqoop file with commenting all lines
of hbase condition then I am getting some exceptions

I have attached some images related to it. Pls find my attachments and
suggest me some solution on it.


--
Thanks and Regards,
Bhavesh Shah

[jira] [Updated] (HIVE-2332) If all of the parameters of distinct functions are exists in group by columns, query fails in runtime

[Navis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-2332:


Attachment: HIVE-2332.2.patch.txt

https://reviews.apache.org/r/1314/

> If all of the parameters of distinct functions are exists in group by 
> columns, query fails in runtime
> -
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt, HIVE-2332.2.patch.txt
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
>   at java.util.ArrayList.RangeCheck(ArrayList.java:547)
>   at java.util.ArrayList.get(ArrayList.java:322)
>   at 
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
>   at 
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
>   at 
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
>   at 
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
>   at 
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
>   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
>   at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
>   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key 
> column, is the problem, which should be equal or more. 
> Would it be solved if add some key expression? I'll try.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2332) If all of the parameters of distinct functions are exists in group by columns, query fails in runtime

[Navis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-2332:


Fix Version/s: 0.9.0
   Status: Patch Available  (was: Open)

> If all of the parameters of distinct functions are exists in group by 
> columns, query fails in runtime
> -
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
>   at java.util.ArrayList.RangeCheck(ArrayList.java:547)
>   at java.util.ArrayList.get(ArrayList.java:322)
>   at 
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
>   at 
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
>   at 
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
>   at 
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
>   at 
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
>   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
>   at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
>   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key 
> column, is the problem, which should be equal or more. 
> Would it be solved if add some key expression? I'll try.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2332) If all of the parameters of distinct functions are exists in group by columns, query fails in runtime

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164193#comment-13164193
 ] 

jirapos...@reviews.apache.org commented on HIVE-2332:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1314/
---

(Updated 2011-12-07 07:01:30.547976)


Review request for hive, John Sichi and Carl Steinbach.


Changes
---

Adding null keys induced confusions especially for optimizers. This patch just 
modifies key-order minimizing side effects.


Summary
---

If all of the distinct params are in group by keys, union column reserved for 
distinct params would not be added, which making problems initializing RS 
operator.

This patch is just a simple bypass adding dummy expression for the union 
column. Someone would know better way to resolve the problem.


This addresses bug HIVE-2332.
https://issues.apache.org/jira/browse/HIVE-2332


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 732a5aa 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java c6ae55d 
  ql/src/test/queries/clientpositive/groupby_distinct_samekey.q PRE-CREATION 
  ql/src/test/results/clientpositive/groupby_distinct_samekey.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/1314/diff


Testing
---

added clientpositive/groupby_distinct_samekey.q


Thanks,

Navis



> If all of the parameters of distinct functions are exists in group by 
> columns, query fails in runtime
> -
>
> Key: HIVE-2332
> URL: https://issues.apache.org/jira/browse/HIVE-2332
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2332.1.patch.txt
>
>
> select sum(key_int1), sum(distinct key_int1) from t1 group by key_int1;
> fails with message..
> {code}
> FAILED: Execution Error, return code 2 from 
> org.apache.hadoop.hive.ql.exec.MapRedTask
> {code}
> hadoop says..
> {code}
> Caused by: java.lang.IndexOutOfBoundsException: Index: 1, Size: 1
>   at java.util.ArrayList.RangeCheck(ArrayList.java:547)
>   at java.util.ArrayList.get(ArrayList.java:322)
>   at 
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.init(StandardStructObjectInspector.java:95)
>   at 
> org.apache.hadoop.hive.serde2.objectinspector.StandardStructObjectInspector.(StandardStructObjectInspector.java:86)
>   at 
> org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorFactory.getStandardStructObjectInspector(ObjectInspectorFactory.java:252)
>   at 
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initEvaluatorsAndReturnStruct(ReduceSinkOperator.java:188)
>   at 
> org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.processOp(ReduceSinkOperator.java:197)
>   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
>   at 
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:85)
>   at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
>   at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:744)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:532)
> {code}
> I think the deficient number of key expression, compared to number of key 
> column, is the problem, which should be equal or more. 
> Would it be solved if add some key expression? I'll try.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-2332 If all of the parameters of distinct functions are exists in group by columns, query fails in runtime

[Navis Ryu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1314/
---

(Updated 2011-12-07 07:01:30.547976)


Review request for hive, John Sichi and Carl Steinbach.


Changes
---

Adding null keys induced confusions especially for optimizers. This patch just 
modifies key-order minimizing side effects.


Summary
---

If all of the distinct params are in group by keys, union column reserved for 
distinct params would not be added, which making problems initializing RS 
operator.

This patch is just a simple bypass adding dummy expression for the union 
column. Someone would know better way to resolve the problem.


This addresses bug HIVE-2332.
https://issues.apache.org/jira/browse/HIVE-2332


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/ReduceSinkOperator.java 732a5aa 
  ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java c6ae55d 
  ql/src/test/queries/clientpositive/groupby_distinct_samekey.q PRE-CREATION 
  ql/src/test/results/clientpositive/groupby_distinct_samekey.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/1314/diff


Testing
---

added clientpositive/groupby_distinct_samekey.q


Thanks,

Navis

[jira] [Updated] (HIVE-2630) TestHiveServer doesn't produce a JUnit report file

[Marek Sapota (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marek Sapota updated HIVE-2630:
---

Attachment: HIVE-2630.D621.1.patch

Forgot to do the license grant thing again.

> TestHiveServer doesn't produce a JUnit report file
> --
>
> Key: HIVE-2630
> URL: https://issues.apache.org/jira/browse/HIVE-2630
> Project: Hive
>  Issue Type: Bug
>Reporter: Marek Sapota
>Assignee: Marek Sapota
> Fix For: 0.9.0
>
> Attachments: HIVE-2630.D621.1.patch, HIVE-2630.D621.1.patch
>
>
> Run `ant test --Dtestcase=TestHiveServer && ant testreport`, the test report 
> will be empty because not 'TEST--*.xml' file was produced by JUnit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2362) HiveConf properties not appearing in the output of 'set' or 'set -v'

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164171#comment-13164171
 ] 

Hudson commented on HIVE-2362:
--

Integrated in Hive-trunk-h0.21 #1128 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1128/])
HIVE-2362. HiveConf properties not appearing in the output of 'set' or 'set 
-v'
(Carl Steinbach via jvs)

jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211261
Files : 
* /hive/trunk/build-common.xml
* /hive/trunk/common/build.xml
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/common/src/test
* /hive/trunk/common/src/test/org
* /hive/trunk/common/src/test/org/apache
* /hive/trunk/common/src/test/org/apache/hadoop
* /hive/trunk/common/src/test/org/apache/hadoop/hive
* /hive/trunk/common/src/test/org/apache/hadoop/hive/conf
* /hive/trunk/common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
* /hive/trunk/common/src/test/resources
* /hive/trunk/common/src/test/resources/hive-site.xml
* /hive/trunk/conf/hive-default.xml
* /hive/trunk/eclipse-templates/.classpath


> HiveConf properties not appearing in the output of 'set' or 'set -v'
> 
>
> Key: HIVE-2362
> URL: https://issues.apache.org/jira/browse/HIVE-2362
> Project: Hive
>  Issue Type: Bug
>  Components: CLI, Configuration
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
>Priority: Blocker
> Fix For: 0.9.0
>
> Attachments: HIVE-2362.1.patch.txt, HIVE-2362.2.patch.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2630) TestHiveServer doesn't produce a JUnit report file

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164172#comment-13164172
 ] 

Hudson commented on HIVE-2630:
--

Integrated in Hive-trunk-h0.21 #1128 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1128/])
HIVE-2630 [jira] TestHiveServer doesn't produce a JUnit report file
(Marek Sapota via John Sichi)

Summary:
Remove property overrides

Run `ant test Dtestcase=TestHiveServer && ant testreport`, the test report
will be empty because not 'TEST*.xml' file was produced by JUnit.

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: mareksapotafb, jsichi

Differential Revision: 621

jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211265
Files : 
* /hive/trunk/service/build.xml


> TestHiveServer doesn't produce a JUnit report file
> --
>
> Key: HIVE-2630
> URL: https://issues.apache.org/jira/browse/HIVE-2630
> Project: Hive
>  Issue Type: Bug
>Reporter: Marek Sapota
>Assignee: Marek Sapota
> Fix For: 0.9.0
>
> Attachments: HIVE-2630.D621.1.patch
>
>
> Run `ant test --Dtestcase=TestHiveServer && ant testreport`, the test report 
> will be empty because not 'TEST--*.xml' file was produced by JUnit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2622) Hive POMs reference the wrong Hadoop artifacts

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164114#comment-13164114
 ] 

Hudson commented on HIVE-2622:
--

Integrated in Hive-trunk-h0.23.0 #5 (See 
[https://builds.apache.org/job/Hive-trunk-h0.23.0/5/])
HIVE-2622 : Hive POMs reference the wrong Hadoop artifacts (Carl Steinbach 
via Ashutosh Chauhan)

hashutosh : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211271
Files : 
* /hive/trunk/cli/ivy.xml
* /hive/trunk/common/ivy.xml
* /hive/trunk/contrib/ivy.xml
* /hive/trunk/hbase-handler/ivy.xml
* /hive/trunk/hwi/ivy.xml
* /hive/trunk/metastore/ivy.xml
* /hive/trunk/ql/ivy.xml
* /hive/trunk/serde/ivy.xml
* /hive/trunk/shims/ivy.xml


> Hive POMs reference the wrong Hadoop artifacts
> --
>
> Key: HIVE-2622
> URL: https://issues.apache.org/jira/browse/HIVE-2622
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.9.0
>
> Attachments: HIVE-2622.1.patch.txt, HIVE-2622.D633.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2467) HA Support for Metastore Server

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164113#comment-13164113
 ] 

Hudson commented on HIVE-2467:
--

Integrated in Hive-trunk-h0.23.0 #5 (See 
[https://builds.apache.org/job/Hive-trunk-h0.23.0/5/])
HIVE-2467 : HA Support for Metastore Server (Thomas Weise via Ashutosh 
Chauhan)

hashutosh : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211275
Files : 
* /hive/trunk/shims/ivy.xml
* 
/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/HadoopThriftAuthBridge20S.java
* 
/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/MemoryTokenStore.java
* 
/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/TokenStoreDelegationTokenSecretManager.java
* 
/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/hive/thrift/ZooKeeperTokenStore.java
* /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/security
* /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/security/token
* /hive/trunk/shims/src/0.20S/java/org/apache/hadoop/security/token/delegation
* 
/hive/trunk/shims/src/0.20S/java/org/apache/hadoop/security/token/delegation/HiveDelegationTokenSupport.java
* 
/hive/trunk/shims/src/test/org/apache/hadoop/hive/thrift/TestHadoop20SAuthBridge.java


> HA Support for Metastore Server 
> 
>
> Key: HIVE-2467
> URL: https://issues.apache.org/jira/browse/HIVE-2467
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore, Security, Server Infrastructure
>Affects Versions: 0.8.0, 0.9.0
>Reporter: Thomas Weise
>Assignee: Thomas Weise
> Fix For: 0.9.0
>
> Attachments: HIVE-2467.2.patch, HIVE-2467.patch
>
>
> We require HA deployment for metastore server for HCatalog:
> * Multiple server instances run behind VIP
> * Database provides HA
> Metastore server instances will need to be able to share any state required 
> for VIP outside RDBMS. As of Hive 0.8 affected conversational state that 
> needs to support VIP/HA setup is limited to current delegation tokens. Is 
> this correct?
> We are planning to use ZooKeeper to share current delegation tokens and 
> master keys between nodes of the VIP. ZK is already (optionally) used by Hive 
> for concurrency control. Access to ZK would be limited on the network level 
> or in the future, when ZooKeeper supports security, through Kerberos, similar 
> to NN access.
> Currently Hive taps into Hadoop core security delegation token support 
> through extension of
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager
> A solution could amend the Hive specific extension to support:
> * Pluggable delegation token and master key store (ZooKeeper as alternative 
> for in-memory AbstractDelegationTokenSecretManager)
> * Delegation token retrieval from token store when not found in memory 
> (wrap/extend retrievePassword(...))
> * Cancellation of token in token store
> * Purging of expired tokens from token store
> http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2362) HiveConf properties not appearing in the output of 'set' or 'set -v'

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164081#comment-13164081
 ] 

Hudson commented on HIVE-2362:
--

Integrated in Hive-trunk-h0.23.0 #4 (See 
[https://builds.apache.org/job/Hive-trunk-h0.23.0/4/])
HIVE-2362. HiveConf properties not appearing in the output of 'set' or 'set 
-v'
(Carl Steinbach via jvs)

jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211261
Files : 
* /hive/trunk/build-common.xml
* /hive/trunk/common/build.xml
* /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
* /hive/trunk/common/src/test
* /hive/trunk/common/src/test/org
* /hive/trunk/common/src/test/org/apache
* /hive/trunk/common/src/test/org/apache/hadoop
* /hive/trunk/common/src/test/org/apache/hadoop/hive
* /hive/trunk/common/src/test/org/apache/hadoop/hive/conf
* /hive/trunk/common/src/test/org/apache/hadoop/hive/conf/TestHiveConf.java
* /hive/trunk/common/src/test/resources
* /hive/trunk/common/src/test/resources/hive-site.xml
* /hive/trunk/conf/hive-default.xml
* /hive/trunk/eclipse-templates/.classpath


> HiveConf properties not appearing in the output of 'set' or 'set -v'
> 
>
> Key: HIVE-2362
> URL: https://issues.apache.org/jira/browse/HIVE-2362
> Project: Hive
>  Issue Type: Bug
>  Components: CLI, Configuration
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
>Priority: Blocker
> Fix For: 0.9.0
>
> Attachments: HIVE-2362.1.patch.txt, HIVE-2362.2.patch.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2630) TestHiveServer doesn't produce a JUnit report file

[Hudson (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164082#comment-13164082
 ] 

Hudson commented on HIVE-2630:
--

Integrated in Hive-trunk-h0.23.0 #4 (See 
[https://builds.apache.org/job/Hive-trunk-h0.23.0/4/])
HIVE-2630 [jira] TestHiveServer doesn't produce a JUnit report file
(Marek Sapota via John Sichi)

Summary:
Remove property overrides

Run `ant test Dtestcase=TestHiveServer && ant testreport`, the test report
will be empty because not 'TEST*.xml' file was produced by JUnit.

Test Plan: EMPTY

Reviewers: JIRA, jsichi

Reviewed By: jsichi

CC: mareksapotafb, jsichi

Differential Revision: 621

jvs : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1211265
Files : 
* /hive/trunk/service/build.xml


> TestHiveServer doesn't produce a JUnit report file
> --
>
> Key: HIVE-2630
> URL: https://issues.apache.org/jira/browse/HIVE-2630
> Project: Hive
>  Issue Type: Bug
>Reporter: Marek Sapota
>Assignee: Marek Sapota
> Fix For: 0.9.0
>
> Attachments: HIVE-2630.D621.1.patch
>
>
> Run `ant test --Dtestcase=TestHiveServer && ant testreport`, the test report 
> will be empty because not 'TEST--*.xml' file was produced by JUnit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2467) HA Support for Metastore Server

[Ashutosh Chauhan (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2467?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-2467:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

+1 Committed to trunk. Thanks, Thomas for this useful contribution.

> HA Support for Metastore Server 
> 
>
> Key: HIVE-2467
> URL: https://issues.apache.org/jira/browse/HIVE-2467
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore, Security, Server Infrastructure
>Affects Versions: 0.8.0, 0.9.0
>Reporter: Thomas Weise
>Assignee: Thomas Weise
> Fix For: 0.9.0
>
> Attachments: HIVE-2467.2.patch, HIVE-2467.patch
>
>
> We require HA deployment for metastore server for HCatalog:
> * Multiple server instances run behind VIP
> * Database provides HA
> Metastore server instances will need to be able to share any state required 
> for VIP outside RDBMS. As of Hive 0.8 affected conversational state that 
> needs to support VIP/HA setup is limited to current delegation tokens. Is 
> this correct?
> We are planning to use ZooKeeper to share current delegation tokens and 
> master keys between nodes of the VIP. ZK is already (optionally) used by Hive 
> for concurrency control. Access to ZK would be limited on the network level 
> or in the future, when ZooKeeper supports security, through Kerberos, similar 
> to NN access.
> Currently Hive taps into Hadoop core security delegation token support 
> through extension of
> org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager
> A solution could amend the Hive specific extension to support:
> * Pluggable delegation token and master key store (ZooKeeper as alternative 
> for in-memory AbstractDelegationTokenSecretManager)
> * Delegation token retrieval from token store when not found in memory 
> (wrap/extend retrievePassword(...))
> * Cancellation of token in token store
> * Purging of expired tokens from token store
> http://www.mail-archive.com/hcatalog-user@incubator.apache.org/msg00053.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2622) Hive POMs reference the wrong Hadoop artifacts

[Ashutosh Chauhan (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-2622:
---

   Resolution: Fixed
Fix Version/s: (was: 0.8.0)
   0.9.0
   Status: Resolved  (was: Patch Available)

+1 Committed to trunk.

> Hive POMs reference the wrong Hadoop artifacts
> --
>
> Key: HIVE-2622
> URL: https://issues.apache.org/jira/browse/HIVE-2622
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.9.0
>
> Attachments: HIVE-2622.1.patch.txt, HIVE-2622.D633.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2630) TestHiveServer doesn't produce a JUnit report file

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164045#comment-13164045
 ] 

Phabricator commented on HIVE-2630:
---

mareksapotafb has committed the revision "HIVE-2630 [jira] TestHiveServer 
doesn't produce a JUnit report file".

REVISION DETAIL
  https://reviews.facebook.net/D621

COMMIT
  https://reviews.facebook.net/rHIVE1211265


> TestHiveServer doesn't produce a JUnit report file
> --
>
> Key: HIVE-2630
> URL: https://issues.apache.org/jira/browse/HIVE-2630
> Project: Hive
>  Issue Type: Bug
>Reporter: Marek Sapota
>Assignee: Marek Sapota
> Fix For: 0.9.0
>
> Attachments: HIVE-2630.D621.1.patch
>
>
> Run `ant test --Dtestcase=TestHiveServer && ant testreport`, the test report 
> will be empty because not 'TEST--*.xml' file was produced by JUnit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-2630) TestHiveServer doesn't produce a JUnit report file

[John Sichi (Resolved) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi resolved HIVE-2630.
--

   Resolution: Fixed
Fix Version/s: 0.9.0
 Hadoop Flags: Reviewed

Committed to trunk.  Thanks Marek!


> TestHiveServer doesn't produce a JUnit report file
> --
>
> Key: HIVE-2630
> URL: https://issues.apache.org/jira/browse/HIVE-2630
> Project: Hive
>  Issue Type: Bug
>Reporter: Marek Sapota
>Assignee: Marek Sapota
> Fix For: 0.9.0
>
> Attachments: HIVE-2630.D621.1.patch
>
>
> Run `ant test --Dtestcase=TestHiveServer && ant testreport`, the test report 
> will be empty because not 'TEST--*.xml' file was produced by JUnit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2362) HiveConf properties not appearing in the output of 'set' or 'set -v'

[John Sichi (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi updated HIVE-2362:
-

   Resolution: Fixed
Fix Version/s: (was: 0.8.0)
   0.9.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed to trunk, thanks Carl!  I've marked this as 0.9 only for now...please 
re-add 0.8 when you cut the new branch.

> HiveConf properties not appearing in the output of 'set' or 'set -v'
> 
>
> Key: HIVE-2362
> URL: https://issues.apache.org/jira/browse/HIVE-2362
> Project: Hive
>  Issue Type: Bug
>  Components: CLI, Configuration
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
>Priority: Blocker
> Fix For: 0.9.0
>
> Attachments: HIVE-2362.1.patch.txt, HIVE-2362.2.patch.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2631) Make Hive work with Hadoop 1.0.0

[Ashutosh Chauhan (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-2631:
---

Attachment: hive-2631.patch

Since its merely a name change for hadoop and not a code change, This is a 
superficial change required only because Hive explicitly checks for Hadoop 
version number.  

> Make Hive work with Hadoop 1.0.0
> 
>
> Key: HIVE-2631
> URL: https://issues.apache.org/jira/browse/HIVE-2631
> Project: Hive
>  Issue Type: Bug
>  Components: Shims
>Affects Versions: 0.9.0
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: hive-2631.patch
>
>
> With Hadoop 1.0.0 around the corner ( 
> http://mail-archives.apache.org/mod_mbox/hadoop-general/20.mbox/%3C9D6B6144-F4E0-4A31-883F-2AC504727A1F%40hortonworks.com%3E
>  ), it will be useful to make Hive work with it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-1487) parallelize test query runs

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-1487:
--

Attachment: HIVE-1487.D603.3.patch

mareksapotafb updated the revision "HIVE-1487 [jira] Add ptest script".
Reviewers: JIRA, jsichi

  It should, I made a mistake manually writing a commit message.  It should get 
fixed after this diff.

REVISION DETAIL
  https://reviews.facebook.net/D603

AFFECTED FILES
  testutils/ptest/Buffer.py
  testutils/ptest/Process.py
  testutils/ptest/README
  testutils/ptest/Report.py
  testutils/ptest/Ssh.py
  testutils/ptest/config.py
  testutils/ptest/hivetest.py
  testutils/ptest/templates/Properties.html
  testutils/ptest/templates/TestCase.html
  testutils/ptest/templates/TestRun.html
  testutils/ptest/templates/TestSuite.html
  testutils/ptest/templates/common.css


> parallelize test query runs
> ---
>
> Key: HIVE-1487
> URL: https://issues.apache.org/jira/browse/HIVE-1487
> Project: Hive
>  Issue Type: Improvement
>  Components: Testing Infrastructure
>Reporter: Joydeep Sen Sarma
>Assignee: Marek Sapota
> Attachments: HIVE-1487.D603.1.patch, HIVE-1487.D603.3.patch
>
>
> HIVE-1464 speeded up serial runs somewhat - but looks like it's still too 
> slow. we should use parallel junit or some similar setup to run test queries 
> in parallel. this should be really easy as well need to just use a separate 
> warehouse/metadb and potentiall mapred system dir location.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-2631) Make Hive work with Hadoop 1.0.0

[Ashutosh Chauhan (Created) (JIRA)]

Make Hive work with Hadoop 1.0.0


 Key: HIVE-2631
 URL: https://issues.apache.org/jira/browse/HIVE-2631
 Project: Hive
  Issue Type: Bug
  Components: Shims
Affects Versions: 0.9.0
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan


With Hadoop 1.0.0 around the corner ( 
http://mail-archives.apache.org/mod_mbox/hadoop-general/20.mbox/%3C9D6B6144-F4E0-4A31-883F-2AC504727A1F%40hortonworks.com%3E
 ), it will be useful to make Hive work with it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1040) use sed rather than diff for masking out noise in diff-based tests

[John Sichi (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13164002#comment-13164002
 ] 

John Sichi commented on HIVE-1040:
--

Regarding the masking implementation, the while loop is worst-case O(n^2), and 
requires the entire file to be loaded into memory. It would be reasonable to 
instead stream the file line-by-line, since we never do multi-line replacements.

> use sed rather than diff for masking out noise in diff-based tests
> --
>
> Key: HIVE-1040
> URL: https://issues.apache.org/jira/browse/HIVE-1040
> Project: Hive
>  Issue Type: Improvement
>  Components: Testing Infrastructure
>Affects Versions: 0.4.1
>Reporter: John Sichi
>Assignee: Marek Sapota
>Priority: Minor
> Attachments: HIVE-1040-code-patch.patch, HIVE-1040.1.patch, 
> HIVE-1040.2.patch, HIVE-1040.D597.1.patch, HIVE-1040.D597.2.patch
>
>
> The current diff -I approach has two problems:  (1) it does not allow 
> resolution finer than line-level, so it's impossible to mask out pattern 
> occurrences within a line, and (2) it produces unmasked files, so if you run 
> diff on the command line to compare the result .q.out with the checked-in 
> file, you see the noise.
> My suggestion is to first run sed to replace noise patterns with an 
> unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files 
> without using any -I.
> This would require a one-time hit to update all existing .q.out files so that 
> they would contain the pre-masked results.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [jira] [Assigned] (HIVE-1772) optimize join followed by a groupby

[Jie Li]

I happened to notice this as well.

>From the query plan Hive already considers the group-by in the first job,
so the second job is very fast. But it's still better to eliminate the
second job.

Jie

On Tue, Dec 6, 2011 at 7:04 PM, John Sichi (Assigned) (JIRA) <
j...@apache.org> wrote:

>
> [
> https://issues.apache.org/jira/browse/HIVE-1772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel]
>
> John Sichi reassigned HIVE-1772:
> 
>
>Assignee: Navis
>
> > optimize join followed by a groupby
> > ---
> >
> > Key: HIVE-1772
> > URL: https://issues.apache.org/jira/browse/HIVE-1772
> > Project: Hive
> >  Issue Type: Improvement
> >  Components: Query Processor
> >Reporter: Namit Jain
> >Assignee: Navis
> > Attachments: HIVE-1772.1.patch
> >
> >
> > explain SELECT x.key, count(1) FROM src1 x JOIN src y ON (x.key = y.key)
> group by x.key;
> > STAGE DEPENDENCIES:
> >   Stage-1 is a root stage
> >   Stage-2 depends on stages: Stage-1
> >   Stage-0 is a root stage
> > The above query issues 2 map-reduce jobs.
> > The first MR job performs the join, whereas the second MR performs the
> group by.
> > Since the data is already sorted, the group by can be performed in the
> reducer of the join itself.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators:
> https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>
>
>
>

[jira] [Assigned] (HIVE-2573) Create per-session function registry

[John Sichi (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi reassigned HIVE-2573:


Assignee: Navis

> Create per-session function registry 
> -
>
> Key: HIVE-2573
> URL: https://issues.apache.org/jira/browse/HIVE-2573
> Project: Hive
>  Issue Type: Improvement
>  Components: Server Infrastructure
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-2573.1.patch.txt, HIVE-2573.2.patch.txt, 
> HIVE-2573.3.patch.txt
>
>
> Currently the function registry is shared resource and could be overrided by 
> other users when using HiveServer. If per-session function registry is 
> provided, this situation could be prevented.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2261) Add cleanup stages for UDFs

[John Sichi (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi reassigned HIVE-2261:


Assignee: Navis

> Add cleanup stages for UDFs
> ---
>
> Key: HIVE-2261
> URL: https://issues.apache.org/jira/browse/HIVE-2261
> Project: Hive
>  Issue Type: Wish
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
>
> In some cases, we bind values at last stage of big SQL from other sources, 
> especially from memcached. I made that kind of UDFs for internal-use.
> I found 'initialize' method of GenericUDF class is good place for making 
> connections to memcached cluster, but failed to find  somewhere to 
> close/cleanup the connections. If there is cleaup method in GenericUDF class, 
> things can be more neat. If initializing entity like map/reduce/fetch could 
> be also providable to life-cycles(init/close), that makes perfect.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2361) Add some UDFs which help to migrate Oracle to Hive

[John Sichi (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi reassigned HIVE-2361:


Assignee: JunHo Cho

>  Add some UDFs which help to migrate Oracle to Hive
> ---
>
> Key: HIVE-2361
> URL: https://issues.apache.org/jira/browse/HIVE-2361
> Project: Hive
>  Issue Type: New Feature
>  Components: UDF
>Affects Versions: 0.8.0
>Reporter: JunHo Cho
>Assignee: JunHo Cho
>Priority: Minor
>  Labels: features
> Attachments: nexr-udf.tar
>
>
> Here some UDFs which can be matched to oracle functions:
> There are two kinds of oracle functions. one is scalar function and another 
> is analytic function.
> Most scalar functions in Oracle can be converted to hive's udf directly.  
> Oracle Scalar Function
> GenericUDFDecode : Compares first argument to each other value one by one. 
> e.g., DECODE(x,0,'zero',1,'one') will return 'zero' if x is 0
> GenericUDFGreatest : Return the greatest of the list of one or more 
> expressions. e.g., GREATEST(2,5,12,3) will return 12
> GenericUDFInstr : Return the location of a substring in a string. e.g., 
> INSTR('next', 'e') will return 2
> GenericUDFLnnvl : Evaluate a condition when one or both operands of the 
> condition may be null. e.g., LNNVL(2 > 4) will return true
> GenericUDFNVL : Replace null with a string in the results of a query. e.g., 
> NVL(null,'hive') will return hive
> GenericUDFNVL2 : Determine the value returned by a query based on whether a 
> specified expression is null or not null. e.g., NVL2(null,'not null','null 
> value') will return 'null value'
> GenericUDFToNumber : Convert a string to a number. e.g., 
> TO_NUMBER('112','999') will return 112
> GenericUDFTrunc : Returns a date truncated to a specific unit of measure. 
> e.g., TRUNC('2002-11-02 01:01:01','') will return '2002-01-01 00:00:00'
> Oracle Analytic Function
> Most analytic functions in Oracle can't be converted to hive's query and udf 
> directly.
> Following udfs should be used with DISTRIBUTED, SORT BY and HASH of hive to 
> support analytic functions 
> e.q., SELECT _FUNC_(hash(col1), col2, ...) FROM SELECT ~ FROM table 
> DISTRIBUTED BY hash(col1) SORT BY col1, col2 ...
> GenericUDFSum : Calculate a cumulative sum.
> GenericUDFRank : Assign a sequential order, or rank within some group based 
> on key.
> GenericUDFDenseRank : Act like RANK function except that it assigns 
> consecutive ranks.
> GenericUDFRowNumber : Return sequence integer value within some group based 
> on key.
> GenericUDFMax : Determine the highest value within some group based on key.
> GenericUDFMin : Determine the lowest value within some group based on key.
> GenericUDFLag : Access data from a previous row.
> This udfs was developed with hive-pdk

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2283) Backtracking real column names for EXPLAIN output

[John Sichi (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi reassigned HIVE-2283:


Assignee: Navis

> Backtracking real column names for EXPLAIN output
> -
>
> Key: HIVE-2283
> URL: https://issues.apache.org/jira/browse/HIVE-2283
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Affects Versions: 0.8.0
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-2283.1.patch, HIVE-2283.2.patch, HIVE-2283.3.patch, 
> HIVE-2283.test.patch
>
>
> GUI people suggested that showing real column names for result of EXPLAIN 
> statement would make customers feel more comfortable with HIVE. I agreed and 
> working on it. 
> {code}
> a. current EXPLAIN
>  Select Operator
>expressions:
>  expr: _col10
>  type: int
>  expr: _col17
>  type: string
>Group By Operator
>  keys:
>expr: _col0
>type: int
>expr: _col17
>type: int
> b. suggested EXPLAIN
>  Select Operator
>expressions: _col10=t2.key_int1, _col17=upper(t1.key_int1), 
> _col22=t3.key_string2
>Group By Operator
>  keys: _col10=t2.key_int1, _col17=upper(t1.key_int1)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-1772) optimize join followed by a groupby

[John Sichi (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi reassigned HIVE-1772:


Assignee: Navis

> optimize join followed by a groupby
> ---
>
> Key: HIVE-1772
> URL: https://issues.apache.org/jira/browse/HIVE-1772
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Navis
> Attachments: HIVE-1772.1.patch
>
>
> explain SELECT x.key, count(1) FROM src1 x JOIN src y ON (x.key = y.key) 
> group by x.key;
> STAGE DEPENDENCIES:
>   Stage-1 is a root stage
>   Stage-2 depends on stages: Stage-1
>   Stage-0 is a root stage
> The above query issues 2 map-reduce jobs. 
> The first MR job performs the join, whereas the second MR performs the group 
> by.
> Since the data is already sorted, the group by can be performed in the 
> reducer of the join itself.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2339) Preserve RS key columns in columnExprMap after CP optimization, which might be useful to other optimizers

[John Sichi (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi reassigned HIVE-2339:


Assignee: Navis

> Preserve RS key columns in columnExprMap after CP optimization, which might 
> be useful to other optimizers
> -
>
> Key: HIVE-2339
> URL: https://issues.apache.org/jira/browse/HIVE-2339
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Fix For: 0.9.0
>
> Attachments: HIVE-2339.1.patch
>
>
> In ColumnPrunerProcFactory#pruneReduceSinkOperator, only VALUE parts are 
> retained from columnExprMap. Doesn't anyone want KEY parts to retained, 
> either? In my case, it was very useful for backtracking column names and 
> removing RS in *-RS-*-RS-GBY case.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2340) optimize orderby followed by a groupby

[John Sichi (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

John Sichi reassigned HIVE-2340:


Assignee: Navis

> optimize orderby followed by a groupby
> --
>
> Key: HIVE-2340
> URL: https://issues.apache.org/jira/browse/HIVE-2340
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-2340.1.patch.txt
>
>
> Before implementing optimizer for JOIN-GBY, try to implement RS-GBY 
> optimizer(cluster-by following group-by).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2622) Hive POMs reference the wrong Hadoop artifacts

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2622:
--

Attachment: HIVE-2622.D633.1.patch

cwsteinbach requested code review of "HIVE-2622 [jira] Hive POMs reference the 
wrong Hadoop artifacts".
Reviewers: JIRA

  Fix Hadoop dependency in ivy.xml files

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D633

AFFECTED FILES
  cli/ivy.xml
  common/ivy.xml
  contrib/ivy.xml
  hbase-handler/ivy.xml
  hwi/ivy.xml
  metastore/ivy.xml
  ql/ivy.xml
  serde/ivy.xml
  shims/ivy.xml

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/1395/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> Hive POMs reference the wrong Hadoop artifacts
> --
>
> Key: HIVE-2622
> URL: https://issues.apache.org/jira/browse/HIVE-2622
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.8.0
>
> Attachments: HIVE-2622.1.patch.txt, HIVE-2622.D633.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1487) parallelize test query runs

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163972#comment-13163972
 ] 

Phabricator commented on HIVE-1487:
---

jsichi has requested changes to the revision "HIVE-1487 [jira] parallelize test 
query runs".

  We can commit this once we get to the bottom of minimr failures.

INLINE COMMENTS
  testutils/ptest/README:48 Typo:  loose -> lose
  testutils/ptest/Ssh.py:21 Let's expunge this comment.
  testutils/ptest/config.py:83 Typo:  faled -> failed

REVISION DETAIL
  https://reviews.facebook.net/D603


> parallelize test query runs
> ---
>
> Key: HIVE-1487
> URL: https://issues.apache.org/jira/browse/HIVE-1487
> Project: Hive
>  Issue Type: Improvement
>  Components: Testing Infrastructure
>Reporter: Joydeep Sen Sarma
>Assignee: Marek Sapota
> Attachments: HIVE-1487.D603.1.patch
>
>
> HIVE-1464 speeded up serial runs somewhat - but looks like it's still too 
> slow. we should use parallel junit or some similar setup to run test queries 
> in parallel. this should be really easy as well need to just use a separate 
> warehouse/metadb and potentiall mapred system dir location.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2622) Hive POMs reference the wrong Hadoop artifacts

[John Sichi (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163934#comment-13163934
 ] 

John Sichi commented on HIVE-2622:
--

Can you create a review request in Phabricator?

https://cwiki.apache.org/confluence/display/Hive/PhabricatorCodeReview

> Hive POMs reference the wrong Hadoop artifacts
> --
>
> Key: HIVE-2622
> URL: https://issues.apache.org/jira/browse/HIVE-2622
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.8.0
>
> Attachments: HIVE-2622.1.patch.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2630) TestHiveServer doesn't produce a JUnit report file

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163921#comment-13163921
 ] 

Phabricator commented on HIVE-2630:
---

jsichi has accepted the revision "HIVE-2630 [jira] TestHiveServer doesn't 
produce a JUnit report file".

  +1, will commit when tests pass.

REVISION DETAIL
  https://reviews.facebook.net/D621


> TestHiveServer doesn't produce a JUnit report file
> --
>
> Key: HIVE-2630
> URL: https://issues.apache.org/jira/browse/HIVE-2630
> Project: Hive
>  Issue Type: Bug
>Reporter: Marek Sapota
>Assignee: Marek Sapota
> Attachments: HIVE-2630.D621.1.patch
>
>
> Run `ant test --Dtestcase=TestHiveServer && ant testreport`, the test report 
> will be empty because not 'TEST--*.xml' file was produced by JUnit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-1040) use sed rather than diff for masking out noise in diff-based tests

[John Sichi (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163919#comment-13163919
 ] 

John Sichi commented on HIVE-1040:
--

It's kind of bad that we have binary data in .q.out files...HIVE-2482 would be 
the correct way to fix that (using a UDF to display as hex).


> use sed rather than diff for masking out noise in diff-based tests
> --
>
> Key: HIVE-1040
> URL: https://issues.apache.org/jira/browse/HIVE-1040
> Project: Hive
>  Issue Type: Improvement
>  Components: Testing Infrastructure
>Affects Versions: 0.4.1
>Reporter: John Sichi
>Assignee: Marek Sapota
>Priority: Minor
> Attachments: HIVE-1040-code-patch.patch, HIVE-1040.1.patch, 
> HIVE-1040.2.patch, HIVE-1040.D597.1.patch, HIVE-1040.D597.2.patch
>
>
> The current diff -I approach has two problems:  (1) it does not allow 
> resolution finer than line-level, so it's impossible to mask out pattern 
> occurrences within a line, and (2) it produces unmasked files, so if you run 
> diff on the command line to compare the result .q.out with the checked-in 
> file, you see the noise.
> My suggestion is to first run sed to replace noise patterns with an 
> unlikely-to-occur string like ZYZZYZVA, and then diff the pre-masked files 
> without using any -I.
> This would require a one-time hit to update all existing .q.out files so that 
> they would contain the pre-masked results.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2630) TestHiveServer doesn't produce a JUnit report file

[Phabricator (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163907#comment-13163907
 ] 

Phabricator commented on HIVE-2630:
---

mareksapotafb has added reviewers to the revision "HIVE-2630 [jira] 
TestHiveServer doesn't produce a JUnit report file".
Added Reviewers: jsichi

REVISION DETAIL
  https://reviews.facebook.net/D621


> TestHiveServer doesn't produce a JUnit report file
> --
>
> Key: HIVE-2630
> URL: https://issues.apache.org/jira/browse/HIVE-2630
> Project: Hive
>  Issue Type: Bug
>Reporter: Marek Sapota
>Assignee: Marek Sapota
> Attachments: HIVE-2630.D621.1.patch
>
>
> Run `ant test --Dtestcase=TestHiveServer && ant testreport`, the test report 
> will be empty because not 'TEST--*.xml' file was produced by JUnit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2630) TestHiveServer doesn't produce a JUnit report file

[Marek Sapota (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Marek Sapota updated HIVE-2630:
---

Description: Run `ant test --Dtestcase=TestHiveServer && ant testreport`, 
the test report will be empty because not 'TEST--*.xml' file was produced by 
JUnit.  (was: Run `ant test -Dtestcase=TestHiveServer && ant testreport`, the 
test report will be empty because not 'TEST-*.xml' file was produced by JUnit.)

> TestHiveServer doesn't produce a JUnit report file
> --
>
> Key: HIVE-2630
> URL: https://issues.apache.org/jira/browse/HIVE-2630
> Project: Hive
>  Issue Type: Bug
>Reporter: Marek Sapota
>Assignee: Marek Sapota
> Attachments: HIVE-2630.D621.1.patch
>
>
> Run `ant test --Dtestcase=TestHiveServer && ant testreport`, the test report 
> will be empty because not 'TEST--*.xml' file was produced by JUnit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2630) TestHiveServer doesn't produce a JUnit report file

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2630:
--

Attachment: HIVE-2630.D621.1.patch

mareksapotafb requested code review of "HIVE-2630 [jira] TestHiveServer doesn't 
produce a JUnit report file".
Reviewers: JIRA

  Remove property overrides

  Run `ant test Dtestcase=TestHiveServer && ant testreport`, the test 
report will be empty because not 'TEST*.xml' file was produced by JUnit.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D621

AFFECTED FILES
  service/build.xml

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/1377/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> TestHiveServer doesn't produce a JUnit report file
> --
>
> Key: HIVE-2630
> URL: https://issues.apache.org/jira/browse/HIVE-2630
> Project: Hive
>  Issue Type: Bug
>Reporter: Marek Sapota
>Assignee: Marek Sapota
> Attachments: HIVE-2630.D621.1.patch
>
>
> Run `ant test -Dtestcase=TestHiveServer && ant testreport`, the test report 
> will be empty because not 'TEST-*.xml' file was produced by JUnit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2629) Make a single Hive binary work with both 0.20.x and 0.23.0

[Carl Steinbach (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2629?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163900#comment-13163900
 ] 

Carl Steinbach commented on HIVE-2629:
--

Relevant comment from HIVE-2468:
https://issues.apache.org/jira/browse/HIVE-2468?focusedCommentId=13163313&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13163313


> Make a single Hive binary work with both 0.20.x and 0.23.0
> --
>
> Key: HIVE-2629
> URL: https://issues.apache.org/jira/browse/HIVE-2629
> Project: Hive
>  Issue Type: Bug
>  Components: Shims
>Reporter: Carl Steinbach
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-2630) TestHiveServer doesn't produce a JUnit report file

[Marek Sapota (Created) (JIRA)]

TestHiveServer doesn't produce a JUnit report file
--

 Key: HIVE-2630
 URL: https://issues.apache.org/jira/browse/HIVE-2630
 Project: Hive
  Issue Type: Bug
Reporter: Marek Sapota
Assignee: Marek Sapota
 Attachments: HIVE-2630.D621.1.patch

Run `ant test -Dtestcase=TestHiveServer && ant testreport`, the test report 
will be empty because not 'TEST-*.xml' file was produced by JUnit.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-2629) Make a single Hive binary work with both 0.20.x and 0.23.0

[Carl Steinbach (Created) (JIRA)]

Make a single Hive binary work with both 0.20.x and 0.23.0
--

 Key: HIVE-2629
 URL: https://issues.apache.org/jira/browse/HIVE-2629
 Project: Hive
  Issue Type: Bug
  Components: Shims
Reporter: Carl Steinbach




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2468) Make Hive compile against Hadoop 0.23

[Ahmed Radwan (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163892#comment-13163892
 ] 

Ahmed Radwan commented on HIVE-2468:


Used these flags in recompiling. Rerunning against 0.23, the previous runtime 
exception seem to disappear. Thanks!

> Make Hive compile against Hadoop 0.23
> -
>
> Key: HIVE-2468
> URL: https://issues.apache.org/jira/browse/HIVE-2468
> Project: Hive
>  Issue Type: Task
>Reporter: Konstantin Shvachko
>Assignee: Carl Steinbach
> Fix For: 0.8.0, 0.9.0
>
> Attachments: HIVE-2468.patch, HIVE-2468.patch, HIVE-2468.patch, 
> HIVE-2468.patch, HIVE-2468.patch
>
>
> Due to restructure of Hadoop 0.22 branch compared to Hadoop 0.20 Hive does 
> not compile against 0.22

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2362) HiveConf properties not appearing in the output of 'set' or 'set -v'

[John Sichi (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163803#comment-13163803
 ] 

John Sichi commented on HIVE-2362:
--

+1, will commit when tests pass.

> HiveConf properties not appearing in the output of 'set' or 'set -v'
> 
>
> Key: HIVE-2362
> URL: https://issues.apache.org/jira/browse/HIVE-2362
> Project: Hive
>  Issue Type: Bug
>  Components: CLI, Configuration
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
>Priority: Blocker
> Fix For: 0.8.0
>
> Attachments: HIVE-2362.1.patch.txt, HIVE-2362.2.patch.txt
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-0.8.0-SNAPSHOT-h0.21 - Build # 122 - Fixed

[Apache Jenkins Server]

Changes for Build #121

Changes for Build #122



All tests passed

The Apache Jenkins build system has built Hive-0.8.0-SNAPSHOT-h0.21 (build #122)

Status: Fixed

Check console output at 
https://builds.apache.org/job/Hive-0.8.0-SNAPSHOT-h0.21/122/ to view the 
results.

Hive-trunk-h0.21 - Build # 1127 - Fixed

[Apache Jenkins Server]

Changes for Build #1114

Changes for Build #1115
[cws] HIVE-2468. Make Hive compile against Hadoop 0.23 (Tom White via cws)


Changes for Build #1116

Changes for Build #1117
[jvs] HIVE-2253. Merge failing of join tree in exceptional case
(Navis Ryu via jvs)


Changes for Build #1118
[namit] HIVE-2619 Add hook to run in metastore's endFunction which can collect 
more fb303 counters
(Kevin Wilfong via namit)


Changes for Build #1119
[cws] HIVE-2523. Add a new builtins subproject (John Sichi via cws)


Changes for Build #1120

Changes for Build #1121
[heyongqiang] Hive-2607: Add caching to json_tuple (Kevin Wilfong via He 
Yongqiang)


Changes for Build #1122

Changes for Build #1123

Changes for Build #1124

Changes for Build #1125
[jvs] Fix eclipse classpath template broken in HIVE-2523.
(Carl Steinbach via jvs)


Changes for Build #1126
[jvs] HIVE-2625. Fix maven-build Ant target
(Carl Steinbach via jvs)


Changes for Build #1127



All tests passed

The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1127)

Status: Fixed

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1127/ to 
view the results.

[jira] [Updated] (HIVE-2628) move one line log from MapOperator to HiveContextAwareRecordReader

[He Yongqiang (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

He Yongqiang updated HIVE-2628:
---

Status: Patch Available  (was: Open)

> move one line log from MapOperator to HiveContextAwareRecordReader
> --
>
> Key: HIVE-2628
> URL: https://issues.apache.org/jira/browse/HIVE-2628
> Project: Hive
>  Issue Type: Improvement
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-2628.D615.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2628) move one line log from MapOperator to HiveContextAwareRecordReader

[Phabricator (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-2628:
--

Attachment: HIVE-2628.D615.1.patch

heyongqiang requested code review of "HIVE-2628 [jira] move one line log from 
MapOperator to HiveContextAwareRecordReader".
Reviewers: JIRA

  move one line log from MapOperator to HiveContextAwareRecordReader



TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D615

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapOperator.java
  ql/src/java/org/apache/hadoop/hive/ql/io/HiveContextAwareRecordReader.java

MANAGE HERALD DIFFERENTIAL RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/1365/

Tip: use the X-Herald-Rules header to filter Herald messages in your client.


> move one line log from MapOperator to HiveContextAwareRecordReader
> --
>
> Key: HIVE-2628
> URL: https://issues.apache.org/jira/browse/HIVE-2628
> Project: Hive
>  Issue Type: Improvement
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-2628.D615.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2628) move one line log from MapOperator to HiveContextAwareRecordReader

[He Yongqiang (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163757#comment-13163757
 ] 

He Yongqiang commented on HIVE-2628:


https://reviews.facebook.net/D615

> move one line log from MapOperator to HiveContextAwareRecordReader
> --
>
> Key: HIVE-2628
> URL: https://issues.apache.org/jira/browse/HIVE-2628
> Project: Hive
>  Issue Type: Improvement
>Reporter: He Yongqiang
>Assignee: He Yongqiang
> Attachments: HIVE-2628.D615.1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-2628) move one line log from MapOperator to HiveContextAwareRecordReader

[He Yongqiang (Created) (JIRA)]

move one line log from MapOperator to HiveContextAwareRecordReader
--

 Key: HIVE-2628
 URL: https://issues.apache.org/jira/browse/HIVE-2628
 Project: Hive
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: He Yongqiang




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2617) Insert overwrite table db.tname fails if partition already exists

[Chinna Rao Lalam (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163692#comment-13163692
 ] 

Chinna Rao Lalam commented on HIVE-2617:


Changed the code like if table contains the database name called the alter 
partition using the dbname and tablename.

> Insert overwrite table db.tname fails if partition already exists 
> --
>
> Key: HIVE-2617
> URL: https://issues.apache.org/jira/browse/HIVE-2617
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Reporter: Aniket Mokashi
>Assignee: Chinna Rao Lalam
> Attachments: HIVE-2617.patch
>
>
> Insert Overwrite table db.tname fails if partition already exists.
> For example-
> insert overwrite table db.tname PARTITION(part='p') select .. from t2 where 
> part='p'; fails if partition 'p' already exists. Workaround is - use db; and 
> the fire the command.
> From the source code-
> alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String 
> tablename as argument and loses db information. Table table = 
> newTable(tablename) is called to retrieve table from name. But, it relies on 
> currentDatabase value (hence the workaround).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2617) Insert overwrite table db.tname fails if partition already exists

[Chinna Rao Lalam (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chinna Rao Lalam updated HIVE-2617:
---

Status: Patch Available  (was: Open)

> Insert overwrite table db.tname fails if partition already exists 
> --
>
> Key: HIVE-2617
> URL: https://issues.apache.org/jira/browse/HIVE-2617
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Reporter: Aniket Mokashi
>Assignee: Chinna Rao Lalam
> Attachments: HIVE-2617.patch
>
>
> Insert Overwrite table db.tname fails if partition already exists.
> For example-
> insert overwrite table db.tname PARTITION(part='p') select .. from t2 where 
> part='p'; fails if partition 'p' already exists. Workaround is - use db; and 
> the fire the command.
> From the source code-
> alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String 
> tablename as argument and loses db information. Table table = 
> newTable(tablename) is called to retrieve table from name. But, it relies on 
> currentDatabase value (hence the workaround).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2617) Insert overwrite table db.tname fails if partition already exists

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163691#comment-13163691
 ] 

jirapos...@reviews.apache.org commented on HIVE-2617:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3026/
---

Review request for hive.


Summary
---

Insert Overwrite table db.tname fails if partition already exists.
 For example-
 insert overwrite table db.tname PARTITION(part='p') select .. from t2 where 
part='p'; fails if partition 'p' already exists. 

Internal of the code not consider the database name of the table and it is 
continued with the default database.

Changed the code like if table contains the database name called the alter 
partition using the dbname and tablename.


This addresses bug HIVE-2617.
https://issues.apache.org/jira/browse/HIVE-2617


Diffs
-

  trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1210917 
  trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q 
PRE-CREATION 
  trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/3026/diff


Testing
---

Added new testcase and all unit tests are passed


Thanks,

chinna



> Insert overwrite table db.tname fails if partition already exists 
> --
>
> Key: HIVE-2617
> URL: https://issues.apache.org/jira/browse/HIVE-2617
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Reporter: Aniket Mokashi
>Assignee: Chinna Rao Lalam
> Attachments: HIVE-2617.patch
>
>
> Insert Overwrite table db.tname fails if partition already exists.
> For example-
> insert overwrite table db.tname PARTITION(part='p') select .. from t2 where 
> part='p'; fails if partition 'p' already exists. Workaround is - use db; and 
> the fire the command.
> From the source code-
> alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String 
> tablename as argument and loses db information. Table table = 
> newTable(tablename) is called to retrieve table from name. But, it relies on 
> currentDatabase value (hence the workaround).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-2617) Insert overwrite table db.tname fails if partition already exists

[Chinna Rao Lalam (Assigned) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chinna Rao Lalam reassigned HIVE-2617:
--

Assignee: Chinna Rao Lalam

> Insert overwrite table db.tname fails if partition already exists 
> --
>
> Key: HIVE-2617
> URL: https://issues.apache.org/jira/browse/HIVE-2617
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Reporter: Aniket Mokashi
>Assignee: Chinna Rao Lalam
> Attachments: HIVE-2617.patch
>
>
> Insert Overwrite table db.tname fails if partition already exists.
> For example-
> insert overwrite table db.tname PARTITION(part='p') select .. from t2 where 
> part='p'; fails if partition 'p' already exists. Workaround is - use db; and 
> the fire the command.
> From the source code-
> alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String 
> tablename as argument and loses db information. Table table = 
> newTable(tablename) is called to retrieve table from name. But, it relies on 
> currentDatabase value (hence the workaround).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Review Request: Insert overwrite table db.tname fails if partition already exists

[chinnarao]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3026/
---

Review request for hive.


Summary
---

Insert Overwrite table db.tname fails if partition already exists.
 For example-
 insert overwrite table db.tname PARTITION(part='p') select .. from t2 where 
part='p'; fails if partition 'p' already exists. 

Internal of the code not consider the database name of the table and it is 
continued with the default database.

Changed the code like if table contains the database name called the alter 
partition using the dbname and tablename.


This addresses bug HIVE-2617.
https://issues.apache.org/jira/browse/HIVE-2617


Diffs
-

  trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java 1210917 
  trunk/ql/src/test/queries/clientpositive/insert2_overwrite_partitions.q 
PRE-CREATION 
  trunk/ql/src/test/results/clientpositive/insert2_overwrite_partitions.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/3026/diff


Testing
---

Added new testcase and all unit tests are passed


Thanks,

chinna

[jira] [Updated] (HIVE-2617) Insert overwrite table db.tname fails if partition already exists

[Chinna Rao Lalam (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chinna Rao Lalam updated HIVE-2617:
---

Attachment: HIVE-2617.patch

> Insert overwrite table db.tname fails if partition already exists 
> --
>
> Key: HIVE-2617
> URL: https://issues.apache.org/jira/browse/HIVE-2617
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Reporter: Aniket Mokashi
> Attachments: HIVE-2617.patch
>
>
> Insert Overwrite table db.tname fails if partition already exists.
> For example-
> insert overwrite table db.tname PARTITION(part='p') select .. from t2 where 
> part='p'; fails if partition 'p' already exists. Workaround is - use db; and 
> the fire the command.
> From the source code-
> alterPartition(tbl.getTableName(), new Partition(tbl, tpart)); takes String 
> tablename as argument and loses db information. Table table = 
> newTable(tablename) is called to retrieve table from name. But, it relies on 
> currentDatabase value (hence the workaround).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2340) optimize orderby followed by a groupby

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163523#comment-13163523
 ] 

jirapos...@reviews.apache.org commented on HIVE-2340:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2991/
---

(Updated 2011-12-06 11:02:00.777597)


Review request for hive and Carl Steinbach.


Changes
---

I've overrode existing optimizer by mistake. Fixed it.


Summary
---

Mostly copied from existing code. Not tested intensively yet, but it is seemed 
to be used frequently for us.


This addresses bug HIVE-2340.
https://issues.apache.org/jira/browse/HIVE-2340


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 
82a141d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkDeDuplication.java 
e91b4d5 
  ql/src/test/queries/clientpositive/reduce_deduplicate_extended.q PRE-CREATION 
  ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/2991/diff


Testing
---

new test cases added : reduce_deduplicate_extended.q


Thanks,

Navis



> optimize orderby followed by a groupby
> --
>
> Key: HIVE-2340
> URL: https://issues.apache.org/jira/browse/HIVE-2340
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor
>Reporter: Navis
>Priority: Minor
> Attachments: HIVE-2340.1.patch.txt
>
>
> Before implementing optimizer for JOIN-GBY, try to implement RS-GBY 
> optimizer(cluster-by following group-by).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-2340: optimizer for RS-RS-GBY case

[Navis Ryu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/2991/
---

(Updated 2011-12-06 11:02:00.777597)


Review request for hive and Carl Steinbach.


Changes
---

I've overrode existing optimizer by mistake. Fixed it.


Summary
---

Mostly copied from existing code. Not tested intensively yet, but it is seemed 
to be used frequently for us.


This addresses bug HIVE-2340.
https://issues.apache.org/jira/browse/HIVE-2340


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ColumnPrunerProcFactory.java 
82a141d 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/ReduceSinkDeDuplication.java 
e91b4d5 
  ql/src/test/queries/clientpositive/reduce_deduplicate_extended.q PRE-CREATION 
  ql/src/test/results/clientpositive/reduce_deduplicate_extended.q.out 
PRE-CREATION 

Diff: https://reviews.apache.org/r/2991/diff


Testing
---

new test cases added : reduce_deduplicate_extended.q


Thanks,

Navis

[jira] [Commented] (HIVE-2005) Implement BETWEEN operator

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163464#comment-13163464
 ] 

jirapos...@reviews.apache.org commented on HIVE-2005:
-



bq.  On 2011-12-06 05:29:34, Carl Steinbach wrote:
bq.  > * show_functions.q.out needs to be updated. There might be other test 
diffs too.

added already (r2)


bq.  On 2011-12-06 05:29:34, Carl Steinbach wrote:
bq.  > 
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java, line 1
bq.  > 
bq.  >
bq.  > Missing ASF License header.

ah, ok.


bq.  On 2011-12-06 05:29:34, Carl Steinbach wrote:
bq.  > ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java, line 
356
bq.  > 
bq.  >
bq.  > Is it possible to register this as "between", or does that result in 
syntax errors?

I remember naming 'between' made a parsing exception. I'll try one more time.


bq.  On 2011-12-06 05:29:34, Carl Steinbach wrote:
bq.  > 
ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java, line 
15
bq.  > 
bq.  >
bq.  > Since we still end up expressing this as two UDFs, would it make 
more sense to implement this as a rewrite rule in the parser instead of as its 
own UDF? Implementing it as a special purpose UDF results in it appearing in 
the FunctionRegistry, and in the output of "show functions", which isn't super 
desirable for things that are fundamentally syntactic sugar.

The first version was implemented by using rewrite rule, something like 'A 
BETWEEN B AND C' --> 'B <= A AND A <= C'. But this made target expression(A) 
evaluated twice which made some queries slow. But I agree on the 'syntactic 
sugar' part.

Making system functions not visible was requested before (to me). Would it be 
useful for the problem?


- Navis


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3008/#review3656
---


On 2011-12-06 00:17:31, Navis Ryu wrote:
bq.  
bq.  ---
bq.  This is an automatically generated e-mail. To reply, visit:
bq.  https://reviews.apache.org/r/3008/
bq.  ---
bq.  
bq.  (Updated 2011-12-06 00:17:31)
bq.  
bq.  
bq.  Review request for hive, John Sichi and Carl Steinbach.
bq.  
bq.  
bq.  Summary
bq.  ---
bq.  
bq.  Revised version using 'between' UDF
bq.  
bq.  
bq.  This addresses bug HIVE-2005.
bq.  https://issues.apache.org/jira/browse/HIVE-2005
bq.  
bq.  
bq.  Diffs
bq.  -
bq.  
bq.ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java c7342c8 
bq.ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 
bq.ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java 
PRE-CREATION 
bq.ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION 
bq.ql/src/test/results/clientpositive/show_functions.q.out 05b784b 
bq.ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION 
bq.  
bq.  Diff: https://reviews.apache.org/r/3008/diff
bq.  
bq.  
bq.  Testing
bq.  ---
bq.  
bq.  added new test : udf_between.q
bq.  
bq.  
bq.  Thanks,
bq.  
bq.  Navis
bq.  
bq.



> Implement BETWEEN operator
> --
>
> Key: HIVE-2005
> URL: https://issues.apache.org/jira/browse/HIVE-2005
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Affects Versions: 0.9.0
>Reporter: Carl Steinbach
>Assignee: Carl Steinbach
> Fix For: 0.9.0
>
> Attachments: HIVE-2005.1.patch.txt
>
>
> Reference: 
> http://dev.mysql.com/doc/refman/5.0/en/comparison-operators.html#operator_between

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Review Request: HIVE-2005 Implement BETWEEN operator

[Navis Ryu]

> On 2011-12-06 05:29:34, Carl Steinbach wrote:
> > * show_functions.q.out needs to be updated. There might be other test diffs 
> > too.

added already (r2)


> On 2011-12-06 05:29:34, Carl Steinbach wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java, 
> > line 1
> > 
> >
> > Missing ASF License header.

ah, ok.


> On 2011-12-06 05:29:34, Carl Steinbach wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java, line 356
> > 
> >
> > Is it possible to register this as "between", or does that result in 
> > syntax errors?

I remember naming 'between' made a parsing exception. I'll try one more time.


> On 2011-12-06 05:29:34, Carl Steinbach wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java, 
> > line 15
> > 
> >
> > Since we still end up expressing this as two UDFs, would it make more 
> > sense to implement this as a rewrite rule in the parser instead of as its 
> > own UDF? Implementing it as a special purpose UDF results in it appearing 
> > in the FunctionRegistry, and in the output of "show functions", which isn't 
> > super desirable for things that are fundamentally syntactic sugar.

The first version was implemented by using rewrite rule, something like 'A 
BETWEEN B AND C' --> 'B <= A AND A <= C'. But this made target expression(A) 
evaluated twice which made some queries slow. But I agree on the 'syntactic 
sugar' part.

Making system functions not visible was requested before (to me). Would it be 
useful for the problem?


- Navis


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3008/#review3656
---


On 2011-12-06 00:17:31, Navis Ryu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/3008/
> ---
> 
> (Updated 2011-12-06 00:17:31)
> 
> 
> Review request for hive, John Sichi and Carl Steinbach.
> 
> 
> Summary
> ---
> 
> Revised version using 'between' UDF
> 
> 
> This addresses bug HIVE-2005.
> https://issues.apache.org/jira/browse/HIVE-2005
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java c7342c8 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g eecd9e7 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFBetween.java 
> PRE-CREATION 
>   ql/src/test/queries/clientpositive/udf_between.q PRE-CREATION 
>   ql/src/test/results/clientpositive/show_functions.q.out 05b784b 
>   ql/src/test/results/clientpositive/udf_between.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/3008/diff
> 
> 
> Testing
> ---
> 
> added new test : udf_between.q
> 
> 
> Thanks,
> 
> Navis
> 
>

[jira] [Updated] (HIVE-2264) Hive server is SHUTTING DOWN when invalid queries beeing executed.

[Navis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-2264:


Attachment: HIVE-2264.1.patch.txt

> Hive server is SHUTTING DOWN when invalid queries beeing executed.
> --
>
> Key: HIVE-2264
> URL: https://issues.apache.org/jira/browse/HIVE-2264
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.9.0
> Environment: SuSE-Linux-11
>Reporter: rohithsharma
>Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2264.1.patch.txt
>
>
> When invalid query is beeing executed, Hive server is shutting down.
> {noformat}
> "CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds 
> string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'"
> "ALTER TABLE SAMPLETABLE add Partition(ds='sf') location 
> '/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'"
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2264) Hive server is SHUTTING DOWN when invalid queries beeing executed.

[jirapos...@reviews.apache.org (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163454#comment-13163454
 ] 

jirapos...@reviews.apache.org commented on HIVE-2264:
-


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3025/
---

Review request for hive, John Sichi and Carl Steinbach.


Summary
---

Currently, if Driver fails executing any of the task, it tries suicide by 
System.exit() which should not be happened in our use cases.

This is simple hot-fix I've provided to my coworkers. Someone could do more 
elaborated work.


This addresses bug HIVE-2264.
https://issues.apache.org/jira/browse/HIVE-2264


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/Driver.java da02fa5 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java f612119 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java a3e40f7 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java d2b12a7 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java a4e59ca 
  ql/src/test/queries/clientnegative/driver_suicide.q PRE-CREATION 
  ql/src/test/results/clientnegative/driver_suicide.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/3025/diff


Testing
---

test added : driver_suicide.q (negative)


Thanks,

Navis



> Hive server is SHUTTING DOWN when invalid queries beeing executed.
> --
>
> Key: HIVE-2264
> URL: https://issues.apache.org/jira/browse/HIVE-2264
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.9.0
> Environment: SuSE-Linux-11
>Reporter: rohithsharma
>Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2264.1.patch.txt
>
>
> When invalid query is beeing executed, Hive server is shutting down.
> {noformat}
> "CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds 
> string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'"
> "ALTER TABLE SAMPLETABLE add Partition(ds='sf') location 
> '/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'"
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2264) Hive server is SHUTTING DOWN when invalid queries beeing executed.

[Navis (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-2264:


Fix Version/s: 0.9.0
Affects Version/s: (was: 0.7.1)
   (was: 0.7.0)
   0.9.0
   Status: Patch Available  (was: Open)

https://reviews.apache.org/r/3025/

> Hive server is SHUTTING DOWN when invalid queries beeing executed.
> --
>
> Key: HIVE-2264
> URL: https://issues.apache.org/jira/browse/HIVE-2264
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.9.0
> Environment: SuSE-Linux-11
>Reporter: rohithsharma
>Priority: Critical
> Fix For: 0.9.0
>
> Attachments: HIVE-2264.1.patch.txt
>
>
> When invalid query is beeing executed, Hive server is shutting down.
> {noformat}
> "CREATE TABLE SAMPLETABLE(IP STRING , showtime BIGINT ) partitioned by (ds 
> string,ipz int) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\040'"
> "ALTER TABLE SAMPLETABLE add Partition(ds='sf') location 
> '/user/hive/warehouse' Partition(ipz=100) location '/user/hive/warehouse'"
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

Review Request: HIVE-2264 Hive server is SHUTTING DOWN when invalid queries beeing executed

[Navis Ryu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/3025/
---

Review request for hive, John Sichi and Carl Steinbach.


Summary
---

Currently, if Driver fails executing any of the task, it tries suicide by 
System.exit() which should not be happened in our use cases.

This is simple hot-fix I've provided to my coworkers. Someone could do more 
elaborated work.


This addresses bug HIVE-2264.
https://issues.apache.org/jira/browse/HIVE-2264


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/Driver.java da02fa5 
  ql/src/java/org/apache/hadoop/hive/ql/exec/ExecDriver.java f612119 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapRedTask.java a3e40f7 
  ql/src/java/org/apache/hadoop/hive/ql/exec/MapredLocalTask.java d2b12a7 
  ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java a4e59ca 
  ql/src/test/queries/clientnegative/driver_suicide.q PRE-CREATION 
  ql/src/test/results/clientnegative/driver_suicide.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/3025/diff


Testing
---

test added : driver_suicide.q (negative)


Thanks,

Navis

[jira] [Commented] (HIVE-2623) Add Integer type compressors

[Krishna Kumar (Commented) (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13163437#comment-13163437
 ] 

Krishna Kumar commented on HIVE-2623:
-

RCFile size stats:

   gzip/bzip2/uber+compressor

steppedpowerlaw1S 49455/45746/36168(IntegerEliasGamma1S)
steppedpowerlaw2S 61425/56546/35998(IntegerEliasGamma2S) 
steppedpowerlawTimestamp 120895/201378/35875(DeltaIntegerEliasGamma1S)
steppedpowerlawIQ 84818/54463/36125(DeltaIntegerEliasGamma2S)

> Add Integer type compressors
> 
>
> Key: HIVE-2623
> URL: https://issues.apache.org/jira/browse/HIVE-2623
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor, Serializers/Deserializers
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
>Priority: Minor
> Attachments: HIVE-2623.v0.patch, HIVE-2623.v1.patch, 
> steppedpowerlaw1S.gz, steppedpowerlaw2S.gz, steppedpowerlawIQ.gz, 
> steppedpowerlawTimestamp.gz
>
>
> Type-specific compressors for integers.
> Starting with elias gamma which prefers small values as per a power-law like 
> distribution. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2623) Add Integer type compressors

[Krishna Kumar (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krishna Kumar updated HIVE-2623:


Attachment: HIVE-2623.v1.patch

Added 3 three more elias gamma compressors - so now supporting one-sided, 
two-sided distributions about 0, and two compressors for delta coding one-sided 
and two-sided differences.

> Add Integer type compressors
> 
>
> Key: HIVE-2623
> URL: https://issues.apache.org/jira/browse/HIVE-2623
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor, Serializers/Deserializers
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
>Priority: Minor
> Attachments: HIVE-2623.v0.patch, HIVE-2623.v1.patch, 
> steppedpowerlaw1S.gz, steppedpowerlaw2S.gz, steppedpowerlawIQ.gz, 
> steppedpowerlawTimestamp.gz
>
>
> Type-specific compressors for integers.
> Starting with elias gamma which prefers small values as per a power-law like 
> distribution. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2623) Add Integer type compressors

[Krishna Kumar (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krishna Kumar updated HIVE-2623:


Attachment: steppedpowerlawTimestamp.gz
steppedpowerlawIQ.gz
steppedpowerlaw2S.gz
steppedpowerlaw1S.gz

synthetic datasets for the four compressors

> Add Integer type compressors
> 
>
> Key: HIVE-2623
> URL: https://issues.apache.org/jira/browse/HIVE-2623
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor, Serializers/Deserializers
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
>Priority: Minor
> Attachments: HIVE-2623.v0.patch, steppedpowerlaw1S.gz, 
> steppedpowerlaw2S.gz, steppedpowerlawIQ.gz, steppedpowerlawTimestamp.gz
>
>
> Type-specific compressors for integers.
> Starting with elias gamma which prefers small values as per a power-law like 
> distribution. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2623) Add Integer type compressors

[Krishna Kumar (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2623?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krishna Kumar updated HIVE-2623:


Attachment: (was: steppedpowerlaw.gz)

> Add Integer type compressors
> 
>
> Key: HIVE-2623
> URL: https://issues.apache.org/jira/browse/HIVE-2623
> Project: Hive
>  Issue Type: Sub-task
>  Components: Query Processor, Serializers/Deserializers
>Reporter: Krishna Kumar
>Assignee: Krishna Kumar
>Priority: Minor
> Attachments: HIVE-2623.v0.patch
>
>
> Type-specific compressors for integers.
> Starting with elias gamma which prefers small values as per a power-law like 
> distribution. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira