[jira] [Updated] (HIVE-4018) MapJoin failing with Distributed Cache error

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4018:
-

Status: Open  (was: Patch Available)

comments

> MapJoin failing with Distributed Cache error
> 
>
> Key: HIVE-4018
> URL: https://issues.apache.org/jira/browse/HIVE-4018
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 0.11.0
>Reporter: Amareshwari Sriramadasu
>Assignee: Amareshwari Sriramadasu
> Fix For: 0.11.0
>
> Attachments: HIVE-4018.patch, hive.4018.test.2.patch, 
> HIVE-4018-test.patch
>
>
> When I'm a running a star join query after HIVE-3784, it is failing with 
> following error:
> 2013-02-13 08:36:04,584 ERROR org.apache.hadoop.hive.ql.exec.MapJoinOperator: 
> Load Distributed Cache Error
> 2013-02-13 08:36:04,585 FATAL ExecMapper: 
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.EOFException
>   at 
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:189)
>   at 
> org.apache.hadoop.hive.ql.exec.MapJoinOperator.cleanUpInputFileChangedOp(MapJoinOperator.java:203)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1421)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425)
>   at 
> org.apache.hadoop.hive.ql.exec.Operator.cleanUpInputFileChanged(Operator.java:1425)
>   at 
> org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:614)
>   at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:144)
>   at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
>   at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:391)
>   at org.apache.hadoop.mapred.MapTask.run(MapTask.java:325)
>   at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:416)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
>   at org.apache.hadoop.mapred.Child.main(Child.java:260)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4068) Size of aggregation buffer which uses non-primitive type is not estimated correctly

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4068:
-

Status: Open  (was: Patch Available)

comments

> Size of aggregation buffer which uses non-primitive type is not estimated 
> correctly
> ---
>
> Key: HIVE-4068
> URL: https://issues.apache.org/jira/browse/HIVE-4068
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-4068.D8859.1.patch, HIVE-4068.D8859.2.patch, 
> HIVE-4068.D8859.3.patch
>
>
> Currently, hive assumes an aggregation buffer which holds a map is occupying 
> just 256 byte (fixed). If it's bigger than that in real, OutOfMemoryError can 
> be thrown (especially for >1k buffer). 
> workaround : set hive.map.aggr.hash.percentmemory= default(0.5)>

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4068) Size of aggregation buffer which uses non-primitive type is not estimated correctly

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4068?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589329#comment-13589329
 ] 

Phabricator commented on HIVE-4068:
---

njain has commented on the revision "HIVE-4068 [jira] Size of aggregation 
buffer which uses non-primitive type is not estimated correctly".

  yes, but can you also make other aggregation buffers extend 
AbstractAggregationBuffer ?

REVISION DETAIL
  https://reviews.facebook.net/D8859

To: JIRA, navis
Cc: njain


> Size of aggregation buffer which uses non-primitive type is not estimated 
> correctly
> ---
>
> Key: HIVE-4068
> URL: https://issues.apache.org/jira/browse/HIVE-4068
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-4068.D8859.1.patch, HIVE-4068.D8859.2.patch, 
> HIVE-4068.D8859.3.patch
>
>
> Currently, hive assumes an aggregation buffer which holds a map is occupying 
> just 256 byte (fixed). If it's bigger than that in real, OutOfMemoryError can 
> be thrown (especially for >1k buffer). 
> workaround : set hive.map.aggr.hash.percentmemory= default(0.5)>

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4071) Map-join outer join produces incorrect results.

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4071:
-

Status: Open  (was: Patch Available)

can you create a phabricator entry ?

> Map-join outer join produces incorrect results.
> ---
>
> Key: HIVE-4071
> URL: https://issues.apache.org/jira/browse/HIVE-4071
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.11.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-4071.patch
>
>
> For example, if one sets the size of noConditionalTask.size to 10 with 
> corresponding auto join configurations set to true in auto_join28.q instead 
> of the current smalltable.filesize configuration, we will observe different 
> results if a select query is run. (The test only has explain statements at 
> present).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4080) Add Lead & Lag UDAFs

[Phabricator (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-4080:
--

Attachment: HIVE-4080.D8961.3.patch

hbutani updated the revision "HIVE-4080 [jira] Add Lead & Lag UDAFs".

- add Lead and Lag UDAFs, fix issues specified in review

Reviewers: ashutoshc, JIRA

REVISION DETAIL
  https://reviews.facebook.net/D8961

CHANGE SINCE LAST DIFF
  https://reviews.facebook.net/D8961?vs=28803&id=28833#toc

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/PTFTranslator.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFLag.java
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFLead.java
  ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFLeadLag.java
  ql/src/test/queries/clientpositive/leadlag_queries.q
  ql/src/test/results/clientpositive/leadlag_queries.q.out

To: JIRA, ashutoshc, hbutani


> Add Lead & Lag UDAFs
> 
>
> Key: HIVE-4080
> URL: https://issues.apache.org/jira/browse/HIVE-4080
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-4080.1.patch.txt, HIVE-4080.D8961.1.patch, 
> HIVE-4080.D8961.2.patch, HIVE-4080.D8961.3.patch
>
>
> Currently we support Lead/Lag as navigation UDFs usable with Windowing.
> To be standard compliant we need to support Lead & Lag UDAFs.
> Will continue to support Lead/Lag UDFs as arguments to UDAFs when Windowing 
> is in play. 
> Currently allow Lead/Lag expressions to appear in SelectLists even when they 
> are not arguments to UDAFs. Support for this feature will probably be 
> removed. Causes ambiguities when Query contains different partition clauses. 
> Will provide more details with associated Jira to remove this feature.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3849) Aliased column in where clause for multi-groupby single reducer cannot be resolved

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589352#comment-13589352
 ] 

Phabricator commented on HIVE-3849:
---

njain has commented on the revision "HIVE-3849 [jira] Columns are not extracted 
for multi-groupby single reducer case somtimes".

  I will take a look

REVISION DETAIL
  https://reviews.facebook.net/D7713

To: JIRA, navis
Cc: njain


> Aliased column in where clause for multi-groupby single reducer cannot be 
> resolved
> --
>
> Key: HIVE-3849
> URL: https://issues.apache.org/jira/browse/HIVE-3849
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-3849.D7713.1.patch, HIVE-3849.D7713.2.patch, 
> HIVE-3849.D7713.3.patch, HIVE-3849.D7713.4.patch, HIVE-3849.D7713.5.patch, 
> HIVE-3849.D7713.6.patch, HIVE-3849.D7713.7.patch, HIVE-3849.D7713.8.patch
>
>
> Verifying HIVE-3847, I've found an exception is thrown before meeting the 
> error situation described in it. Something like, 
> FAILED: SemanticException [Error 10025]: Line 40:6 Expression not in GROUP BY 
> key 'crit5'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Vikram Dixit K (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vikram Dixit K updated HIVE-3891:
-

Attachment: auto_sortmerge_join_1.q.out
auto_sortmerge_join_1.q

Hi Namit,

I have added some more variations to the queries in auto_sortmerge_join_1.q. I 
originally did not catch that there was a SMB map join in stage-1. That as you 
say is correct. One thing I see is that if I try to join sorted, bucketed 
tables on different keys (last join in the attached file), I see a SMB Map join 
operator followed by a reduce side join. It seems like the reduce side join 
could actually become a map join operator. Is this possible?

Thanks
Vikram.

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4071) Map-join outer join produces incorrect results.

[Vikram Dixit K (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589359#comment-13589359
 ] 

Vikram Dixit K commented on HIVE-4071:
--

~Namit

I am facing some issues with setting up phabricator. I had opened a review 
board request earlier. Providing link here for reference.

https://reviews.apache.org/r/9597/

> Map-join outer join produces incorrect results.
> ---
>
> Key: HIVE-4071
> URL: https://issues.apache.org/jira/browse/HIVE-4071
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.11.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-4071.patch
>
>
> For example, if one sets the size of noConditionalTask.size to 10 with 
> corresponding auto join configurations set to true in auto_join28.q instead 
> of the current smalltable.filesize configuration, we will observe different 
> results if a select query is run. (The test only has explain statements at 
> present).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3849) Aliased column in where clause for multi-groupby single reducer cannot be resolved

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589362#comment-13589362
 ] 

Phabricator commented on HIVE-3849:
---

njain has accepted the revision "HIVE-3849 [jira] Columns are not extracted for 
multi-groupby single reducer case somtimes".

REVISION DETAIL
  https://reviews.facebook.net/D7713

BRANCH
  DPAL-1948

ARCANIST PROJECT
  hive

To: JIRA, njain, navis
Cc: njain


> Aliased column in where clause for multi-groupby single reducer cannot be 
> resolved
> --
>
> Key: HIVE-3849
> URL: https://issues.apache.org/jira/browse/HIVE-3849
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-3849.D7713.1.patch, HIVE-3849.D7713.2.patch, 
> HIVE-3849.D7713.3.patch, HIVE-3849.D7713.4.patch, HIVE-3849.D7713.5.patch, 
> HIVE-3849.D7713.6.patch, HIVE-3849.D7713.7.patch, HIVE-3849.D7713.8.patch
>
>
> Verifying HIVE-3847, I've found an exception is thrown before meeting the 
> error situation described in it. Something like, 
> FAILED: SemanticException [Error 10025]: Line 40:6 Expression not in GROUP BY 
> key 'crit5'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589373#comment-13589373
 ] 

Namit Jain commented on HIVE-3891:
--

Yes, that is true even before my patch


> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589374#comment-13589374
 ] 

Namit Jain commented on HIVE-3891:
--

I verified the last query in the attached file in trunk, and got the same 
results

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4071) Map-join outer join produces incorrect results.

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589394#comment-13589394
 ] 

Namit Jain commented on HIVE-4071:
--

minor comments

> Map-join outer join produces incorrect results.
> ---
>
> Key: HIVE-4071
> URL: https://issues.apache.org/jira/browse/HIVE-4071
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.11.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
> Attachments: HIVE-4071.patch
>
>
> For example, if one sets the size of noConditionalTask.size to 10 with 
> corresponding auto join configurations set to true in auto_join28.q instead 
> of the current smalltable.filesize configuration, we will observe different 
> results if a select query is run. (The test only has explain statements at 
> present).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4072) Hive eclipse build path update for string template jar

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589403#comment-13589403
 ] 

Namit Jain commented on HIVE-4072:
--

+1

> Hive eclipse build path update for string template jar
> --
>
> Key: HIVE-4072
> URL: https://issues.apache.org/jira/browse/HIVE-4072
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.11.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
>Priority: Minor
> Attachments: HIVE-4072.patch
>
>
> StringTemplate jar version needs to be updated for hive to work with eclipse 
> without user intervention.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4072) Hive eclipse build path update for string template jar

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4072:
-

   Resolution: Fixed
Fix Version/s: 0.11.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Vikram

> Hive eclipse build path update for string template jar
> --
>
> Key: HIVE-4072
> URL: https://issues.apache.org/jira/browse/HIVE-4072
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.11.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
>Priority: Minor
> Fix For: 0.11.0
>
> Attachments: HIVE-4072.patch
>
>
> StringTemplate jar version needs to be updated for hive to work with eclipse 
> without user intervention.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2597) Repeated key in GROUP BY is erroneously displayed when using DISTINCT

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-2597:
-

Status: Open  (was: Patch Available)

comments

> Repeated key in GROUP BY is erroneously displayed when using DISTINCT
> -
>
> Key: HIVE-2597
> URL: https://issues.apache.org/jira/browse/HIVE-2597
> Project: Hive
>  Issue Type: Bug
>Reporter: Alex Rovner
>Assignee: Navis
> Attachments: HIVE-2597.D8967.1.patch
>
>
> The following query was simplified for illustration purposes. 
> This works correctly:
> select client_tid, "" as myvalue1, "" as myvalue2 from clients cluster by 
> client_tid
> The intent here is to produce two empty columns in between data.
> The following query does not work:
> select distinct client_tid, "" as myvalue1, "" as myvalue2 from clients 
> cluster by client_tid
> FAILED: Error in semantic analysis: Line 1:44 Repeated key in GROUP BY ""
> The key is not repeated since the aliases were given. Seems like Hive is 
> ignoring the aliases when the "distinct" keyword is specified.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-2597) Repeated key in GROUP BY is erroneously displayed when using DISTINCT

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-2597?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589410#comment-13589410
 ] 

Phabricator commented on HIVE-2597:
---

njain has commented on the revision "HIVE-2597 [jira] Repeated key in GROUP BY 
is erroneously displayed when using DISTINCT".

INLINE COMMENTS
  ql/src/test/queries/clientpositive/groupby_constant.q:1 can you add the test 
you had in the description
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:2749 
spelling
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:2961 
spelling
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:3643 spell

REVISION DETAIL
  https://reviews.facebook.net/D8967

To: JIRA, navis
Cc: njain


> Repeated key in GROUP BY is erroneously displayed when using DISTINCT
> -
>
> Key: HIVE-2597
> URL: https://issues.apache.org/jira/browse/HIVE-2597
> Project: Hive
>  Issue Type: Bug
>Reporter: Alex Rovner
>Assignee: Navis
> Attachments: HIVE-2597.D8967.1.patch
>
>
> The following query was simplified for illustration purposes. 
> This works correctly:
> select client_tid, "" as myvalue1, "" as myvalue2 from clients cluster by 
> client_tid
> The intent here is to produce two empty columns in between data.
> The following query does not work:
> select distinct client_tid, "" as myvalue1, "" as myvalue2 from clients 
> cluster by client_tid
> FAILED: Error in semantic analysis: Line 1:44 Repeated key in GROUP BY ""
> The key is not repeated since the aliases were given. Seems like Hive is 
> ignoring the aliases when the "distinct" keyword is specified.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4042) ignore mapjoin hint

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589411#comment-13589411
 ] 

Namit Jain commented on HIVE-4042:
--

Refreshed

> ignore mapjoin hint
> ---
>
> Key: HIVE-4042
> URL: https://issues.apache.org/jira/browse/HIVE-4042
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.4042.1.patch, hive.4042.2.patch, hive.4042.3.patch, 
> hive.4042.4.patch, hive.4042.5.patch, hive.4042.6.patch, hive.4042.7.patch, 
> hive.4042.8.patch
>
>
> After HIVE-3784, in a production environment, it can become difficult to
> deploy since a lot of production queries can break.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4042) ignore mapjoin hint

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4042:
-

Attachment: hive.4042.8.patch

> ignore mapjoin hint
> ---
>
> Key: HIVE-4042
> URL: https://issues.apache.org/jira/browse/HIVE-4042
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.4042.1.patch, hive.4042.2.patch, hive.4042.3.patch, 
> hive.4042.4.patch, hive.4042.5.patch, hive.4042.6.patch, hive.4042.7.patch, 
> hive.4042.8.patch
>
>
> After HIVE-3784, in a production environment, it can become difficult to
> deploy since a lot of production queries can break.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3904) Replace hashmaps in JoinOperators to array

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589422#comment-13589422
 ] 

Phabricator commented on HIVE-3904:
---

njain has accepted the revision "HIVE-3904 [jira] Replace hashmaps in 
JoinOperators to array".

REVISION DETAIL
  https://reviews.facebook.net/D7959

BRANCH
  DPAL-1960

ARCANIST PROJECT
  hive

To: JIRA, njain, navis
Cc: njain


> Replace hashmaps in JoinOperators to array
> --
>
> Key: HIVE-3904
> URL: https://issues.apache.org/jira/browse/HIVE-3904
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3904.D7959.1.patch, HIVE-3904.D7959.2.patch
>
>
> Join operator has many HashMaps that maps tag to some internal 
> value(ExprEvals, OIs, etc.) and theses are accessed 5 or more times per an 
> object, which is seemed unnecessary overhead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4090) Use of hive.exec.script.allow.partial.consumption can produce partial results

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4090:
-

   Resolution: Fixed
Fix Version/s: 0.11.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Kevin

> Use of hive.exec.script.allow.partial.consumption can produce partial results
> -
>
> Key: HIVE-4090
> URL: https://issues.apache.org/jira/browse/HIVE-4090
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Fix For: 0.11.0
>
> Attachments: HIVE-4090.1.patch.txt
>
>
> When users execute use a transform script with the config 
> hive.exec.script.allow.partial.consumption set to true, it may produce 
> partial results.
> When this config is set the script may close it's input pipe before its 
> parent operator has finished passing it rows.  In the catch block for this 
> exception, the setDone method is called marking the operator as done.  
> However, there's a separate thread running to process rows passed from the 
> script back to Hive via stdout.  If this thread is not done processing rows, 
> any rows it forwards after the setDone method is called will not be passed to 
> its children.  This leads to partial results.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3904) Replace hashmaps in JoinOperators to array

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589422#comment-13589422
 ] 

Phabricator commented on HIVE-3904:
---

njain has accepted the revision "HIVE-3904 [jira] Replace hashmaps in 
JoinOperators to array".

REVISION DETAIL
  https://reviews.facebook.net/D7959

BRANCH
  DPAL-1960

ARCANIST PROJECT
  hive

To: JIRA, njain, navis
Cc: njain


> Replace hashmaps in JoinOperators to array
> --
>
> Key: HIVE-3904
> URL: https://issues.apache.org/jira/browse/HIVE-3904
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3904.D7959.1.patch, HIVE-3904.D7959.2.patch
>
>
> Join operator has many HashMaps that maps tag to some internal 
> value(ExprEvals, OIs, etc.) and theses are accessed 5 or more times per an 
> object, which is seemed unnecessary overhead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4079) Altering a view partition fails with NPE

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589440#comment-13589440
 ] 

Namit Jain commented on HIVE-4079:
--

+1

> Altering a view partition fails with NPE
> 
>
> Key: HIVE-4079
> URL: https://issues.apache.org/jira/browse/HIVE-4079
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-4079.1.patch.txt
>
>
> Altering a view partition e.g. to add partition parameters, fails with a null 
> pointer exception in the ObjectStore class.
> Currently, this is only possible using the metastore Thrift API and there are 
> no testcases for it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4079) Altering a view partition fails with NPE

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4079:
-

   Resolution: Fixed
Fix Version/s: 0.11.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Kevin

> Altering a view partition fails with NPE
> 
>
> Key: HIVE-4079
> URL: https://issues.apache.org/jira/browse/HIVE-4079
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Fix For: 0.11.0
>
> Attachments: HIVE-4079.1.patch.txt
>
>
> Altering a view partition e.g. to add partition parameters, fails with a null 
> pointer exception in the ObjectStore class.
> Currently, this is only possible using the metastore Thrift API and there are 
> no testcases for it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3490) Implement * or a.* for arguments to UDFs

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589447#comment-13589447
 ] 

Phabricator commented on HIVE-3490:
---

njain has commented on the revision "HIVE-3490 [jira] Implement * or a.* for 
arguments to UDFs".

  code changes look good

INLINE COMMENTS
  ql/src/test/queries/clientpositive/allcolref_in_udf.q:4 can you add limit 10 
to reduce the output
  ql/src/test/queries/clientpositive/allcolref_in_udf.q:8 what are the columns 
of concat(*) - is it a.* followed by b.*
  Can you add comments
  ql/src/test/queries/clientpositive/allcolref_in_udf.q:9 again, add a limit at 
the end

REVISION DETAIL
  https://reviews.facebook.net/D8889

To: JIRA, navis
Cc: njain


> Implement * or a.* for arguments to UDFs
> 
>
> Key: HIVE-3490
> URL: https://issues.apache.org/jira/browse/HIVE-3490
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor, UDF
>Reporter: Adam Kramer
>Assignee: Navis
> Attachments: HIVE-3490.D8889.1.patch
>
>
> For a random UDF, we should be able to use * or a.* to refer to "all of the 
> columns in their natural order." This is not currently implemented.
> I'm reporting this as a bug because it is a manner in which Hive is 
> inconsistent with the SQL spec, and because Hive claims to implement *.
> hive> select all_non_null(a.*) from table a where a.ds='2012-09-01';
> FAILED: ParseException line 1:25 mismatched input '*' expecting Identifier 
> near '.' in expression specification

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3628) Provide a way to use counters in Hive through UDF

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589481#comment-13589481
 ] 

Namit Jain commented on HIVE-3628:
--

+1

> Provide a way to use counters in Hive through UDF
> -
>
> Key: HIVE-3628
> URL: https://issues.apache.org/jira/browse/HIVE-3628
> Project: Hive
>  Issue Type: Improvement
>  Components: UDF
>Reporter: Viji
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-3628.D8007.1.patch, HIVE-3628.D8007.2.patch, 
> HIVE-3628.D8007.3.patch, HIVE-3628.D8007.4.patch, HIVE-3628.D8007.5.patch, 
> HIVE-3628.D8007.6.patch
>
>
> Currently it is not possible to generate counters through UDF. We should 
> support this. 
> Pig currently allows this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4007) Create abstract classes for serializer and deserializer

[Jarek Jarcec Cecho (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589604#comment-13589604
 ] 

Jarek Jarcec Cecho commented on HIVE-4007:
--

I'm still +1 (non-binding), thank you Namit for working on this!

Jarcec

> Create abstract classes for serializer and deserializer
> ---
>
> Key: HIVE-4007
> URL: https://issues.apache.org/jira/browse/HIVE-4007
> Project: Hive
>  Issue Type: Improvement
>  Components: Serializers/Deserializers
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.4007.1.patch, hive.4007.2.patch, hive.4007.3.patch, 
> hive.4007.4.patch
>
>
> Currently, it is very difficult to change the Serializer/Deserializer
> interface, since all the SerDes directly implement the interface.
> Instead, we should have abstract classes for implementing these interfaces.
> In case of a interface change, only the abstract class and the relevant 
> serde needs to change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4056) Extend rcfilecat to support (un)compressed size and no. of row

[Gang Tim Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589611#comment-13589611
 ] 

Gang Tim Liu commented on HIVE-4056:


Namit, thanks Tim



Sent from my iPhone 




> Extend rcfilecat to support (un)compressed size and no. of row
> --
>
> Key: HIVE-4056
> URL: https://issues.apache.org/jira/browse/HIVE-4056
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Fix For: 0.11.0
>
> Attachments: HIVE-4056.patch.1
>
>
> rcfilecat supports data and metadata:
> https://cwiki.apache.org/Hive/rcfilecat.html
> In metadata, it supports column statistics.
> It will be natural to extend metadata support to 
> 1. no. of rows 
> 2. uncompressed size for the file
> 3. compressed size for the file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4080) Add Lead & Lag UDAFs

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589612#comment-13589612
 ] 

Phabricator commented on HIVE-4080:
---

ashutoshc has accepted the revision "HIVE-4080 [jira] Add Lead & Lag UDAFs".

  +1

REVISION DETAIL
  https://reviews.facebook.net/D8961

BRANCH
  HIVE-4080

ARCANIST PROJECT
  hive

To: JIRA, ashutoshc, hbutani


> Add Lead & Lag UDAFs
> 
>
> Key: HIVE-4080
> URL: https://issues.apache.org/jira/browse/HIVE-4080
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-4080.1.patch.txt, HIVE-4080.D8961.1.patch, 
> HIVE-4080.D8961.2.patch, HIVE-4080.D8961.3.patch
>
>
> Currently we support Lead/Lag as navigation UDFs usable with Windowing.
> To be standard compliant we need to support Lead & Lag UDAFs.
> Will continue to support Lead/Lag UDFs as arguments to UDAFs when Windowing 
> is in play. 
> Currently allow Lead/Lag expressions to appear in SelectLists even when they 
> are not arguments to UDAFs. Support for this feature will probably be 
> removed. Causes ambiguities when Query contains different partition clauses. 
> Will provide more details with associated Jira to remove this feature.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3775) Unit test failures due to unspecified order of results in "show grant" command

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589619#comment-13589619
 ] 

Ashutosh Chauhan commented on HIVE-3775:


+1

> Unit test failures due to unspecified order of results in "show grant" command
> --
>
> Key: HIVE-3775
> URL: https://issues.apache.org/jira/browse/HIVE-3775
> Project: Hive
>  Issue Type: Bug
>  Components: Testing Infrastructure
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-3775.1-r1417768.patch, HIVE-3775.2.patch
>
>
> A number of unit tests (sometimes) using "show grant" fail, when run on 
> windows or previous failures put the database in an unexpected state.
> The reason is that the output of "show grant" is not specified to be in any 
> particular order, but the golden files expect it to be.
> The unit test framework should be extended to handled cases like that.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4074) Doc update for .8, .9 and .10

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589621#comment-13589621
 ] 

Ashutosh Chauhan commented on HIVE-4074:


+1

> Doc update for .8, .9 and .10
> -
>
> Key: HIVE-4074
> URL: https://issues.apache.org/jira/browse/HIVE-4074
> Project: Hive
>  Issue Type: Bug
>  Components: Documentation
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-4074.1.patch.tar.gz.part_a, 
> HIVE-4074.1.patch.tar.gz.part_b, HIVE-4074.1.patch.tar.gz.part_c
>
>
> Need to update the javadocs for releases 8, 9 and 10.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4072) Hive eclipse build path update for string template jar

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589672#comment-13589672
 ] 

Hudson commented on HIVE-4072:
--

Integrated in Hive-trunk-h0.21 #1991 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1991/])
HIVE-4072 Hive eclipse build path update for string template jar
(Vikram Dixit K via namit) (Revision 1451162)

 Result = SUCCESS
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451162
Files : 
* /hive/trunk/ivy/libraries.properties


> Hive eclipse build path update for string template jar
> --
>
> Key: HIVE-4072
> URL: https://issues.apache.org/jira/browse/HIVE-4072
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.11.0
>Reporter: Vikram Dixit K
>Assignee: Vikram Dixit K
>Priority: Minor
> Fix For: 0.11.0
>
> Attachments: HIVE-4072.patch
>
>
> StringTemplate jar version needs to be updated for hive to work with eclipse 
> without user intervention.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4056) Extend rcfilecat to support (un)compressed size and no. of row

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589673#comment-13589673
 ] 

Hudson commented on HIVE-4056:
--

Integrated in Hive-trunk-h0.21 #1991 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1991/])
HIVE-4056 Extend rcfilecat to support (un)compressed size and no. of row
Gang Tim Liu via namit) (Revision 1451130)

 Result = SUCCESS
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1451130
Files : 
* /hive/trunk/cli/src/java/org/apache/hadoop/hive/cli/RCFileCat.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/io/RCFile.java


> Extend rcfilecat to support (un)compressed size and no. of row
> --
>
> Key: HIVE-4056
> URL: https://issues.apache.org/jira/browse/HIVE-4056
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Fix For: 0.11.0
>
> Attachments: HIVE-4056.patch.1
>
>
> rcfilecat supports data and metadata:
> https://cwiki.apache.org/Hive/rcfilecat.html
> In metadata, it supports column statistics.
> It will be natural to extend metadata support to 
> 1. no. of rows 
> 2. uncompressed size for the file
> 3. compressed size for the file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1991 - Fixed

[Apache Jenkins Server]

Changes for Build #1990

Changes for Build #1991
[namit] HIVE-4072 Hive eclipse build path update for string template jar
(Vikram Dixit K via namit)

[namit] HIVE-4056 Extend rcfilecat to support (un)compressed size and no. of row
Gang Tim Liu via namit)




All tests passed

The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1991)

Status: Fixed

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1991/ to 
view the results.

[jira] [Updated] (HIVE-3849) Aliased column in where clause for multi-groupby single reducer cannot be resolved

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3849:
-

   Resolution: Fixed
Fix Version/s: 0.11.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Navis

> Aliased column in where clause for multi-groupby single reducer cannot be 
> resolved
> --
>
> Key: HIVE-3849
> URL: https://issues.apache.org/jira/browse/HIVE-3849
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Fix For: 0.11.0
>
> Attachments: HIVE-3849.D7713.1.patch, HIVE-3849.D7713.2.patch, 
> HIVE-3849.D7713.3.patch, HIVE-3849.D7713.4.patch, HIVE-3849.D7713.5.patch, 
> HIVE-3849.D7713.6.patch, HIVE-3849.D7713.7.patch, HIVE-3849.D7713.8.patch
>
>
> Verifying HIVE-3847, I've found an exception is thrown before meeting the 
> error situation described in it. Something like, 
> FAILED: SemanticException [Error 10025]: Line 40:6 Expression not in GROUP BY 
> key 'crit5'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3904) Replace hashmaps in JoinOperators to array

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589679#comment-13589679
 ] 

Namit Jain commented on HIVE-3904:
--

+1

> Replace hashmaps in JoinOperators to array
> --
>
> Key: HIVE-3904
> URL: https://issues.apache.org/jira/browse/HIVE-3904
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Attachments: HIVE-3904.D7959.1.patch, HIVE-3904.D7959.2.patch
>
>
> Join operator has many HashMaps that maps tag to some internal 
> value(ExprEvals, OIs, etc.) and theses are accessed 5 or more times per an 
> object, which is seemed unnecessary overhead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3904) Replace hashmaps in JoinOperators to array

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3904:
-

   Resolution: Fixed
Fix Version/s: 0.11.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Navis

> Replace hashmaps in JoinOperators to array
> --
>
> Key: HIVE-3904
> URL: https://issues.apache.org/jira/browse/HIVE-3904
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Fix For: 0.11.0
>
> Attachments: HIVE-3904.D7959.1.patch, HIVE-3904.D7959.2.patch
>
>
> Join operator has many HashMaps that maps tag to some internal 
> value(ExprEvals, OIs, etc.) and theses are accessed 5 or more times per an 
> object, which is seemed unnecessary overhead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3628) Provide a way to use counters in Hive through UDF

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3628:
-

   Resolution: Fixed
Fix Version/s: 0.11.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Navis

> Provide a way to use counters in Hive through UDF
> -
>
> Key: HIVE-3628
> URL: https://issues.apache.org/jira/browse/HIVE-3628
> Project: Hive
>  Issue Type: Improvement
>  Components: UDF
>Reporter: Viji
>Assignee: Navis
>Priority: Minor
> Fix For: 0.11.0
>
> Attachments: HIVE-3628.D8007.1.patch, HIVE-3628.D8007.2.patch, 
> HIVE-3628.D8007.3.patch, HIVE-3628.D8007.4.patch, HIVE-3628.D8007.5.patch, 
> HIVE-3628.D8007.6.patch
>
>
> Currently it is not possible to generate counters through UDF. We should 
> support this. 
> Pig currently allows this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3490) Implement * or a.* for arguments to UDFs

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3490:
-

Status: Open  (was: Patch Available)

minor comments

> Implement * or a.* for arguments to UDFs
> 
>
> Key: HIVE-3490
> URL: https://issues.apache.org/jira/browse/HIVE-3490
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor, UDF
>Reporter: Adam Kramer
>Assignee: Navis
> Attachments: HIVE-3490.D8889.1.patch
>
>
> For a random UDF, we should be able to use * or a.* to refer to "all of the 
> columns in their natural order." This is not currently implemented.
> I'm reporting this as a bug because it is a manner in which Hive is 
> inconsistent with the SQL spec, and because Hive claims to implement *.
> hive> select all_non_null(a.*) from table a where a.ds='2012-09-01';
> FAILED: ParseException line 1:25 mismatched input '*' expecting Identifier 
> near '.' in expression specification

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4077) alterPartition and alterPartitions methods in ObjectStore swallow exceptions

[Kevin Wilfong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-4077:


Attachment: HIVE-4077.3.patch.txt

> alterPartition and alterPartitions methods in ObjectStore swallow exceptions
> 
>
> Key: HIVE-4077
> URL: https://issues.apache.org/jira/browse/HIVE-4077
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-4077.1.patch.txt, HIVE-4077.2.patch.txt, 
> HIVE-4077.3.patch.txt
>
>
> The alterPartition and alterPartitions methods in the ObjectStore class throw 
> a MetaException in the case of a failure but do not include the cause, 
> meaning that information is lost.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4077) alterPartition and alterPartitions methods in ObjectStore swallow exceptions

[Kevin Wilfong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-4077:


Status: Patch Available  (was: Open)

> alterPartition and alterPartitions methods in ObjectStore swallow exceptions
> 
>
> Key: HIVE-4077
> URL: https://issues.apache.org/jira/browse/HIVE-4077
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-4077.1.patch.txt, HIVE-4077.2.patch.txt, 
> HIVE-4077.3.patch.txt
>
>
> The alterPartition and alterPartitions methods in the ObjectStore class throw 
> a MetaException in the case of a failure but do not include the cause, 
> meaning that information is lost.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-4080) Add Lead & Lag UDAFs

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-4080.


Resolution: Fixed

Committed to branch. Thanks, Harish!

> Add Lead & Lag UDAFs
> 
>
> Key: HIVE-4080
> URL: https://issues.apache.org/jira/browse/HIVE-4080
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-4080.1.patch.txt, HIVE-4080.D8961.1.patch, 
> HIVE-4080.D8961.2.patch, HIVE-4080.D8961.3.patch
>
>
> Currently we support Lead/Lag as navigation UDFs usable with Windowing.
> To be standard compliant we need to support Lead & Lag UDAFs.
> Will continue to support Lead/Lag UDFs as arguments to UDAFs when Windowing 
> is in play. 
> Currently allow Lead/Lag expressions to appear in SelectLists even when they 
> are not arguments to UDAFs. Support for this feature will probably be 
> removed. Causes ambiguities when Query contains different partition clauses. 
> Will provide more details with associated Jira to remove this feature.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-4075) TypeInfoFactory is not thread safe and is access by multiple threads

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-4075.


   Resolution: Fixed
Fix Version/s: 0.11.0
 Assignee: Brock Noland

Committed to trunk. Thanks, Brock!

> TypeInfoFactory is not thread safe and is access by multiple threads
> 
>
> Key: HIVE-4075
> URL: https://issues.apache.org/jira/browse/HIVE-4075
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 0.10.0
>Reporter: Brock Noland
>Assignee: Brock Noland
> Fix For: 0.11.0
>
> Attachments: HIVE-4075-0.patch
>
>
> TypeInfoFactory is not thread safe and when accessed by multiple threads 
> calls to any of the methods can modify hashmaps concurrently resulting in 
> infinite loops.
> {noformat}
> "pool-1-thread-240" prio=10 tid=0x2aabd8bf7000 nid=0x5f4a runnable 
> [0x44626000] 
> java.lang.Thread.State: RUNNABLE 
> at java.util.HashMap.get(HashMap.java:303) 
> at 
> org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getStructTypeInfo(TypeInfoFactory.java:94)
>  
> at 
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.initSerdeParams(LazySimpleSerDe.java:237)
>  
> at 
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.initialize(LazySimpleSerDe.java:182)
>  
> at 
> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:203)
>  
> at 
> org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260)
>  
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:167) 
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930) 
> at 
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropTable(DDLSemanticAnalyzer.java:706)
>  
> at 
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:212)
>  
> at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:246)
>  
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:432) 
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) 
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) 
> - locked <0x2aaac6e1c270> (a java.lang.Object) 
> at 
> org.apache.hive.service.cli.operation.SQLOperation.run(SQLOperation.java:94)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4075) TypeInfoFactory is not thread safe and is access by multiple threads

[Brock Noland (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589774#comment-13589774
 ] 

Brock Noland commented on HIVE-4075:


Thank you!

> TypeInfoFactory is not thread safe and is access by multiple threads
> 
>
> Key: HIVE-4075
> URL: https://issues.apache.org/jira/browse/HIVE-4075
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 0.10.0
>Reporter: Brock Noland
>Assignee: Brock Noland
> Fix For: 0.11.0
>
> Attachments: HIVE-4075-0.patch
>
>
> TypeInfoFactory is not thread safe and when accessed by multiple threads 
> calls to any of the methods can modify hashmaps concurrently resulting in 
> infinite loops.
> {noformat}
> "pool-1-thread-240" prio=10 tid=0x2aabd8bf7000 nid=0x5f4a runnable 
> [0x44626000] 
> java.lang.Thread.State: RUNNABLE 
> at java.util.HashMap.get(HashMap.java:303) 
> at 
> org.apache.hadoop.hive.serde2.typeinfo.TypeInfoFactory.getStructTypeInfo(TypeInfoFactory.java:94)
>  
> at 
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.initSerdeParams(LazySimpleSerDe.java:237)
>  
> at 
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe.initialize(LazySimpleSerDe.java:182)
>  
> at 
> org.apache.hadoop.hive.metastore.MetaStoreUtils.getDeserializer(MetaStoreUtils.java:203)
>  
> at 
> org.apache.hadoop.hive.ql.metadata.Table.getDeserializerFromMetaStore(Table.java:260)
>  
> at org.apache.hadoop.hive.ql.metadata.Table.checkValidity(Table.java:167) 
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:930) 
> at 
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeDropTable(DDLSemanticAnalyzer.java:706)
>  
> at 
> org.apache.hadoop.hive.ql.parse.DDLSemanticAnalyzer.analyzeInternal(DDLSemanticAnalyzer.java:212)
>  
> at 
> org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(BaseSemanticAnalyzer.java:246)
>  
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:432) 
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:337) 
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:906) 
> - locked <0x2aaac6e1c270> (a java.lang.Object) 
> at 
> org.apache.hive.service.cli.operation.SQLOperation.run(SQLOperation.java:94)
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4073) Make partition by optional in over clause

[Brock Noland (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589800#comment-13589800
 ] 

Brock Noland commented on HIVE-4073:


Just about got this done. Just fixing the fact that it allows invalid queries 
such as:

{noformat}
hive> select p_mfgr, 
> sum(p_retailprice) as s1  
> from part;
{noformat}

I think I will forgo forcing a single reducer at this point. In the future we 
can do that if desired.

> Make partition by optional in over clause
> -
>
> Key: HIVE-4073
> URL: https://issues.apache.org/jira/browse/HIVE-4073
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Ashutosh Chauhan
>Assignee: Brock Noland
> Attachments: HIVE-4073-0.patch
>
>
> select s, sum( i ) over() from tt; should work. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (HIVE-3847) ppd.remove.duplicatefilters removing filters too aggressively

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-3847.


   Resolution: Fixed
Fix Version/s: 0.11.0

Two sub-issues related to it are checked in.

> ppd.remove.duplicatefilters removing filters too aggressively
> -
>
> Key: HIVE-3847
> URL: https://issues.apache.org/jira/browse/HIVE-3847
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.9.0, 0.10.0, 0.11.0
>Reporter: Ashutosh Chauhan
> Fix For: 0.11.0
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1992 - Failure

[Apache Jenkins Server]

Changes for Build #1992



No tests ran.

The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1992)

Status: Failure

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1992/ to 
view the results.

[jira] [Created] (HIVE-4092) Store complete names of tables in column access analyzer

[Samuel Yuan (JIRA)]

Samuel Yuan created HIVE-4092:
-

 Summary: Store complete names of tables in column access analyzer
 Key: HIVE-4092
 URL: https://issues.apache.org/jira/browse/HIVE-4092
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Affects Versions: 0.11.0
Reporter: Samuel Yuan
Assignee: Samuel Yuan
Priority: Trivial
 Fix For: 0.11.0




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4092) Store complete names of tables in column access analyzer

[Samuel Yuan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Samuel Yuan updated HIVE-4092:
--

Description: Right now the db name is not being stored. We should store the 
complete name, which includes the db name, as the table access analyzer does.

> Store complete names of tables in column access analyzer
> 
>
> Key: HIVE-4092
> URL: https://issues.apache.org/jira/browse/HIVE-4092
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.11.0
>Reporter: Samuel Yuan
>Assignee: Samuel Yuan
>Priority: Trivial
> Fix For: 0.11.0
>
>
> Right now the db name is not being stored. We should store the complete name, 
> which includes the db name, as the table access analyzer does.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4092) Store complete names of tables in column access analyzer

[Phabricator (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-4092:
--

Attachment: HIVE-4092.HIVE-4092.HIVE-4092.D8985.1.patch

sxyuan requested code review of "HIVE-4092 [jira] Store complete names of 
tables in column access analyzer".

Reviewers: kevinwilfong

Right now the db name is not being stored by the column access analyzer. We 
should store the complete name, which includes the db name, as the table access 
analyzer does.

TEST PLAN
  Run test queries.

REVISION DETAIL
  https://reviews.facebook.net/D8985

AFFECTED FILES
  ql/src/test/results/clientpositive/column_access_stats.q.out
  ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnAccessAnalyzer.java

MANAGE HERALD RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/21783/

To: kevinwilfong, sxyuan
Cc: JIRA


> Store complete names of tables in column access analyzer
> 
>
> Key: HIVE-4092
> URL: https://issues.apache.org/jira/browse/HIVE-4092
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.11.0
>Reporter: Samuel Yuan
>Assignee: Samuel Yuan
>Priority: Trivial
> Fix For: 0.11.0
>
> Attachments: HIVE-4092.HIVE-4092.HIVE-4092.D8985.1.patch
>
>
> Right now the db name is not being stored. We should store the complete name, 
> which includes the db name, as the table access analyzer does.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4092) Store complete names of tables in column access analyzer

[Samuel Yuan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Samuel Yuan updated HIVE-4092:
--

Status: Patch Available  (was: Open)

> Store complete names of tables in column access analyzer
> 
>
> Key: HIVE-4092
> URL: https://issues.apache.org/jira/browse/HIVE-4092
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Affects Versions: 0.11.0
>Reporter: Samuel Yuan
>Assignee: Samuel Yuan
>Priority: Trivial
> Fix For: 0.11.0
>
> Attachments: HIVE-4092.HIVE-4092.HIVE-4092.D8985.1.patch
>
>
> Right now the db name is not being stored. We should store the complete name, 
> which includes the db name, as the table access analyzer does.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4073) Make partition by optional in over clause

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-4073:
---

Attachment: HIVE-4073-1.patch

[~ashutoshc]

Attached patch seems to work fine but ptf_negative_AggrFuncsWithNoGBYNoPartDef 
fails ,due to this change, which needs to be resolved somehow. Quick question 
about which direction I should take in implementation:

TL;DR: Should the query "select sum(p_retailprice) as s1 from part;" throw a 
SemanticException?

I assumed yes as postgres does not allow it and there is a test for it 
ptf_negative_AggrFuncsWithNoGBYNoPartDef. However, it looks like the behavior 
of moving aggregates with no group by op to window ops was specifically allowed 
based on the two comments below:
https://github.com/apache/hive/blob/ptf-windowing/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L858
https://github.com/apache/hive/blob/ptf-windowing/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java#L10561

> Make partition by optional in over clause
> -
>
> Key: HIVE-4073
> URL: https://issues.apache.org/jira/browse/HIVE-4073
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Ashutosh Chauhan
>Assignee: Brock Noland
> Attachments: HIVE-4073-0.patch, HIVE-4073-1.patch
>
>
> select s, sum( i ) over() from tt; should work. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4093) PTFTranslator.addPartitionExpressionsToOrderList has string format error

[Brock Noland (JIRA)]

Brock Noland created HIVE-4093:
--

 Summary: PTFTranslator.addPartitionExpressionsToOrderList has 
string format error
 Key: HIVE-4093
 URL: https://issues.apache.org/jira/browse/HIVE-4093
 Project: Hive
  Issue Type: Bug
  Components: PTF-Windowing
Reporter: Brock Noland
Assignee: Brock Noland
Priority: Minor




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4073) Make partition by optional in over clause

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589989#comment-13589989
 ] 

Ashutosh Chauhan commented on HIVE-4073:


No this query shouldn't be allowed. Should raise an exception. Comments says 
queries which has windowing in those cases we can do this. But query in 
question clearly has no windowing spec.

It will be good to force number of reducers to 1, else it will be wastage of 
resources. But is probably fine to do later since this empty over clause is 
kind of a corner case anyways.

> Make partition by optional in over clause
> -
>
> Key: HIVE-4073
> URL: https://issues.apache.org/jira/browse/HIVE-4073
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Ashutosh Chauhan
>Assignee: Brock Noland
> Attachments: HIVE-4073-0.patch, HIVE-4073-1.patch
>
>
> select s, sum( i ) over() from tt; should work. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4073) Make partition by optional in over clause

[Brock Noland (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13589997#comment-13589997
 ] 

Brock Noland commented on HIVE-4073:


Sounds good!

> Make partition by optional in over clause
> -
>
> Key: HIVE-4073
> URL: https://issues.apache.org/jira/browse/HIVE-4073
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Ashutosh Chauhan
>Assignee: Brock Noland
> Attachments: HIVE-4073-0.patch, HIVE-4073-1.patch
>
>
> select s, sum( i ) over() from tt; should work. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4073) Make partition by optional in over clause

[Harish Butani (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1358#comment-1358
 ] 

Harish Butani commented on HIVE-4073:
-

The ptf_negative_AggrFuncsWithNoGBYNoPartDef query is

select p_mfgr, 
sum(p_retailprice) as s1  
from part

Which should fail, right?

Changing ensureWindowingSourceHasPartitioning may not work. Its tied to how we 
infer windowing for aggregation invocation that don't have an explicit over 
clause.

The more direct fix maybe to do the partition on constant idea in 
SemanticAnalyzer::processWindowSpec.
The ASTTree for over() is ^(TOK_WINDOWSPEC) so this function should be invoked.
If hasPartSpec is false add the PartitionSpec on '0'
Does this make sense? 

Changing ensureWindowingSourceHasPartitioning will probably also affect how we 
handle multiple partitions. So i have another reason to try to avoid changing 
its logic :)

> Make partition by optional in over clause
> -
>
> Key: HIVE-4073
> URL: https://issues.apache.org/jira/browse/HIVE-4073
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Ashutosh Chauhan
>Assignee: Brock Noland
> Attachments: HIVE-4073-0.patch, HIVE-4073-1.patch
>
>
> select s, sum( i ) over() from tt; should work. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2843) UDAF to convert an aggregation to a map

[David Worms (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Worms updated HIVE-2843:
--

Attachment: hive-2843-dev.git.patch

> UDAF to convert an aggregation to a map
> ---
>
> Key: HIVE-2843
> URL: https://issues.apache.org/jira/browse/HIVE-2843
> Project: Hive
>  Issue Type: New Feature
>  Components: UDF
>Affects Versions: 0.9.0, 0.10.0
>Reporter: David Worms
>Priority: Minor
>  Labels: features, udf
> Attachments: HIVE-2843.1.patch.txt, HIVE-2843.D8745.1.patch, 
> hive-2843-dev.git.patch
>
>
> I propose the addition of two new Hive UDAF to help with maps in Apache Hive. 
> The source code is available on GitHub at https://github.com/wdavidw/hive-udf 
> in two Java classes: "UDAFToMap" and "UDAFToOrderedMap". The first function 
> convert an aggregation into a map and is internally using a Java `HashMap`. 
> The second function extends the first one. It convert an aggregation into an 
> ordered map and is internally using a Java `TreeMap`. They both extends the 
> `AbstractGenericUDAFResolver` class.
> Also, I have covered the motivations and usages of those UDAF in a blog post 
> at http://adaltas.com/blog/2012/03/06/hive-udaf-map-conversion/
> The full patch is available with tests as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4093) PTFTranslator.addPartitionExpressionsToOrderList has string format error

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-4093:
---

Attachment: HIVE-4093-0.patch

> PTFTranslator.addPartitionExpressionsToOrderList has string format error
> 
>
> Key: HIVE-4093
> URL: https://issues.apache.org/jira/browse/HIVE-4093
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Brock Noland
>Assignee: Brock Noland
>Priority: Minor
> Attachments: HIVE-4093-0.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4093) PTFTranslator.addPartitionExpressionsToOrderList has string format error

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-4093:
---

Status: Patch Available  (was: Open)

> PTFTranslator.addPartitionExpressionsToOrderList has string format error
> 
>
> Key: HIVE-4093
> URL: https://issues.apache.org/jira/browse/HIVE-4093
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Brock Noland
>Assignee: Brock Noland
>Priority: Minor
> Attachments: HIVE-4093-0.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4093) PTFTranslator.addPartitionExpressionsToOrderList string formatting results in MissingFormatArgumentException

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-4093:
---

Summary: PTFTranslator.addPartitionExpressionsToOrderList string formatting 
results in MissingFormatArgumentException  (was: 
PTFTranslator.addPartitionExpressionsToOrderList has string format error)

> PTFTranslator.addPartitionExpressionsToOrderList string formatting results in 
> MissingFormatArgumentException
> 
>
> Key: HIVE-4093
> URL: https://issues.apache.org/jira/browse/HIVE-4093
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Brock Noland
>Assignee: Brock Noland
>Priority: Minor
> Attachments: HIVE-4093-0.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4073) Make partition by optional in over clause

[Brock Noland (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590007#comment-13590007
 ] 

Brock Noland commented on HIVE-4073:


Got it, makes sense! Thx!

> Make partition by optional in over clause
> -
>
> Key: HIVE-4073
> URL: https://issues.apache.org/jira/browse/HIVE-4073
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Ashutosh Chauhan
>Assignee: Brock Noland
> Attachments: HIVE-4073-0.patch, HIVE-4073-1.patch
>
>
> select s, sum( i ) over() from tt; should work. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4093) PTFTranslator.addPartitionExpressionsToOrderList string formatting results in MissingFormatArgumentException

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590033#comment-13590033
 ] 

Ashutosh Chauhan commented on HIVE-4093:


I think we can get rid of sprintf() entirely and start using String.format() 
which essentially provides equivalent functionality. No point in having a 
function which is already provided in jdk.

> PTFTranslator.addPartitionExpressionsToOrderList string formatting results in 
> MissingFormatArgumentException
> 
>
> Key: HIVE-4093
> URL: https://issues.apache.org/jira/browse/HIVE-4093
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Brock Noland
>Assignee: Brock Noland
>Priority: Minor
> Attachments: HIVE-4093-0.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4093) PTFTranslator.addPartitionExpressionsToOrderList string formatting results in MissingFormatArgumentException

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-4093:
---

Status: Open  (was: Patch Available)

Ok sounds good!

> PTFTranslator.addPartitionExpressionsToOrderList string formatting results in 
> MissingFormatArgumentException
> 
>
> Key: HIVE-4093
> URL: https://issues.apache.org/jira/browse/HIVE-4093
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Brock Noland
>Assignee: Brock Noland
>Priority: Minor
> Attachments: HIVE-4093-0.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3775) Unit test failures due to unspecified order of results in "show grant" command

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-3775:
---

   Resolution: Fixed
Fix Version/s: 0.11.0
   Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks, Gunther!

> Unit test failures due to unspecified order of results in "show grant" command
> --
>
> Key: HIVE-3775
> URL: https://issues.apache.org/jira/browse/HIVE-3775
> Project: Hive
>  Issue Type: Bug
>  Components: Testing Infrastructure
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Fix For: 0.11.0
>
> Attachments: HIVE-3775.1-r1417768.patch, HIVE-3775.2.patch
>
>
> A number of unit tests (sometimes) using "show grant" fail, when run on 
> windows or previous failures put the database in an unexpected state.
> The reason is that the output of "show grant" is not specified to be in any 
> particular order, but the golden files expect it to be.
> The unit test framework should be extended to handled cases like that.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4093) Remove sprintf from PTFTranslator and use String.format()

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-4093:
---

Summary: Remove sprintf from PTFTranslator and use String.format()  (was: 
PTFTranslator.addPartitionExpressionsToOrderList string formatting results in 
MissingFormatArgumentException)

> Remove sprintf from PTFTranslator and use String.format()
> -
>
> Key: HIVE-4093
> URL: https://issues.apache.org/jira/browse/HIVE-4093
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Brock Noland
>Assignee: Brock Noland
>Priority: Minor
> Attachments: HIVE-4093-0.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3874) Create a new Optimized Row Columnar file format for Hive

[Pamela Vagata (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590092#comment-13590092
 ] 

Pamela Vagata commented on HIVE-3874:
-

Owen, would you mind filing the bug as a separate JIRA and committing the code 
as is? We are currently hunting down issues and putting together fixes :)

> Create a new Optimized Row Columnar file format for Hive
> 
>
> Key: HIVE-3874
> URL: https://issues.apache.org/jira/browse/HIVE-3874
> Project: Hive
>  Issue Type: Improvement
>  Components: Serializers/Deserializers
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
> Attachments: hive.3874.2.patch, HIVE-3874.D8529.1.patch, 
> HIVE-3874.D8529.2.patch, HIVE-3874.D8529.3.patch, HIVE-3874.D8529.4.patch, 
> HIVE-3874.D8871.1.patch, OrcFileIntro.pptx, orc.tgz
>
>
> There are several limitations of the current RC File format that I'd like to 
> address by creating a new format:
> * each column value is stored as a binary blob, which means:
> ** the entire column value must be read, decompressed, and deserialized
> ** the file format can't use smarter type-specific compression
> ** push down filters can't be evaluated
> * the start of each row group needs to be found by scanning
> * user metadata can only be added to the file when the file is created
> * the file doesn't store the number of rows per a file or row group
> * there is no mechanism for seeking to a particular row number, which is 
> required for external indexes.
> * there is no mechanism for storing light weight indexes within the file to 
> enable push-down filters to skip entire row groups.
> * the type of the rows aren't stored in the file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4034) Should be able to specify windowing spec without needing Between

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-4034:
---

Status: Patch Available  (was: Open)

Committed to branch. Thanks, Harish for review!

> Should be able to specify windowing spec without needing Between
> 
>
> Key: HIVE-4034
> URL: https://issues.apache.org/jira/browse/HIVE-4034
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-4034.patch
>
>
> Currently user need to do following:
> {noformat}
> select s, sum(b) over (distribute by i sort by si rows between unbounded 
> preceding and current row) from over100k;
> {noformat}
> but sql spec allows following as well:
> {noformat}
> select s, sum(b) over (distribute by i sort by si rows unbounded preceding) 
> from over100k;
> {noformat}
> In such cases {{current row}} should be assumed implicitly.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1993 - Still Failing

[Apache Jenkins Server]

Changes for Build #1992

Changes for Build #1993



No tests ran.

The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1993)

Status: Still Failing

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1993/ to 
view the results.

[jira] [Assigned] (HIVE-3954) flag indecating statistics is stale

[Gang Tim Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Tim Liu reassigned HIVE-3954:
--

Assignee: Dilip Joseph  (was: Gang Tim Liu)

> flag indecating statistics is stale
> ---
>
> Key: HIVE-3954
> URL: https://issues.apache.org/jira/browse/HIVE-3954
> Project: Hive
>  Issue Type: Improvement
>Reporter: Gang Tim Liu
>Assignee: Dilip Joseph
>
> We will introduce a flag to indicate whether statistics is stale.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4034) Should be able to specify windowing spec without needing Between

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-4034:
---

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Should be able to specify windowing spec without needing Between
> 
>
> Key: HIVE-4034
> URL: https://issues.apache.org/jira/browse/HIVE-4034
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-4034.patch
>
>
> Currently user need to do following:
> {noformat}
> select s, sum(b) over (distribute by i sort by si rows between unbounded 
> preceding and current row) from over100k;
> {noformat}
> but sql spec allows following as well:
> {noformat}
> select s, sum(b) over (distribute by i sort by si rows unbounded preceding) 
> from over100k;
> {noformat}
> In such cases {{current row}} should be assumed implicitly.  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hive-0.10.0-SNAPSHOT-h0.20.1 #79

[Apache Jenkins Server]

See 

--
[...truncated 42311 lines...]
[junit] Hadoop job information for null: number of mappers: 0; number of 
reducers: 0
[junit] 2013-02-28 16:48:00,486 null map = 100%,  reduce = 100%
[junit] Ended Job = job_local_0001
[junit] Execution completed successfully
[junit] Mapred Local Task Succeeded . Convert the Join into MapJoin
[junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/build/service/localscratchdir/hive_2013-02-28_16-47-57_321_2492181048399572743/-mr-1
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/build/service/tmp/hive_job_log_jenkins_201302281648_1558507329.txt
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] Copying file: 
file:/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/data/files/kv1.txt
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: load data local inpath 
'/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/data/files/kv1.txt'
 into table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] Copying data from 
file:/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/data/files/kv1.txt
[junit] Loading data to table default.testhivedrivertable
[junit] Table default.testhivedrivertable stats: [num_partitions: 0, 
num_files: 1, num_rows: 0, total_size: 5812, raw_data_size: 0]
[junit] POSTHOOK: query: load data local inpath 
'/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/data/files/kv1.txt'
 into table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: select * from testhivedrivertable limit 10
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: 
file:/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/build/service/localscratchdir/hive_2013-02-28_16-48-01_848_7718080740583222890/-mr-1
[junit] POSTHOOK: query: select * from testhivedrivertable limit 10
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/build/service/localscratchdir/hive_2013-02-28_16-48-01_848_7718080740583222890/-mr-1
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/build/service/tmp/hive_job_log_jenkins_201302281648_1098350051.txt
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[

[jira] [Commented] (HIVE-3952) merge map-job followed by map-reduce job

[Vinod Kumar Vavilapalli (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3952?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590141#comment-13590141
 ] 

Vinod Kumar Vavilapalli commented on HIVE-3952:
---

Okay, will do.

Tests passed except the two about input37.

> merge map-job followed by map-reduce job
> 
>
> Key: HIVE-3952
> URL: https://issues.apache.org/jira/browse/HIVE-3952
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Vinod Kumar Vavilapalli
> Attachments: HIVE-3952-20130226.txt, HIVE-3952-20130227.1.txt
>
>
> Consider the query like:
> select count(*) FROM
> ( select idOne, idTwo, value FROM
>   bigTable   
>   JOIN
> 
>   smallTableOne on (bigTable.idOne = smallTableOne.idOne) 
>   
>   ) firstjoin 
> 
> JOIN  
> 
> smallTableTwo on (firstjoin.idTwo = smallTableTwo.idTwo);
> where smallTableOne and smallTableTwo are smaller than 
> hive.auto.convert.join.noconditionaltask.size and
> hive.auto.convert.join.noconditionaltask is set to true.
> The joins are collapsed into mapjoins, and it leads to a map-only job
> (for the map-joins) followed by a map-reduce job (for the group by).
> Ideally, the map-only job should be merged with the following map-reduce job.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4093) Remove sprintf from PTFTranslator and use String.format()

[Brock Noland (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Brock Noland updated HIVE-4093:
---

Attachment: HIVE-4093-1.patch

Patch removes PTFUtils.sprintf, converts all calls to String.format(), and 
fixes several errors where the format was incorrect.

> Remove sprintf from PTFTranslator and use String.format()
> -
>
> Key: HIVE-4093
> URL: https://issues.apache.org/jira/browse/HIVE-4093
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Brock Noland
>Assignee: Brock Noland
>Priority: Minor
> Attachments: HIVE-4093-0.patch, HIVE-4093-1.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Vikram Dixit K (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590219#comment-13590219
 ] 

Vikram Dixit K commented on HIVE-3891:
--

Hi Namit,

Can you please create a review request. I see one issue in the 
AbstractSMBJoinProcessor. The 

{noformat}
String joinAlias =
  bigTablePosition == 0 ?
joinCtx.getLeftAlias() : joinCtx.getRightAliases()[bigTablePosition - 
1];
{noformat}

The joinCtx.getRightAliases() array size would be at most 2 in case of joining 
against different keys. However the code that produces the bigTablePosition 
traverses all the tables. If the big table position comes up as 2 in a 
multi-way join, this code will cause an ArrayOutOfBounds exception. This needs 
to be addressed.

Thanks
Vikram.

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Vikram Dixit K (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590221#comment-13590221
 ] 

Vikram Dixit K commented on HIVE-3891:
--

What I mean is right aliases can be only 1 (position 0; in case of join on 
different keys) so if the bigTablePosition comes out as 2, the code looks to 
find something in position 1 causing a failure.

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4094) decimal_3.q & decimal_serde.q fail on hadoop 2

[Gunther Hagleitner (JIRA)]

Gunther Hagleitner created HIVE-4094:


 Summary: decimal_3.q & decimal_serde.q fail on hadoop 2
 Key: HIVE-4094
 URL: https://issues.apache.org/jira/browse/HIVE-4094
 Project: Hive
  Issue Type: Bug
Reporter: Gunther Hagleitner
Assignee: Gunther Hagleitner


Some of the decimal unit tests fail on hadoop 2. The reason is unspecified 
order in some of the queries.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590224#comment-13590224
 ] 

Namit Jain commented on HIVE-3891:
--

The review request is there : https://reviews.facebook.net/D8439

I will get back to you on the comment

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3980) Cleanup after HIVE-3403

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3980:
-

Attachment: hive.3980.3.patch

> Cleanup after HIVE-3403
> ---
>
> Key: HIVE-3980
> URL: https://issues.apache.org/jira/browse/HIVE-3980
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3980.1.patch, hive.3980.2.patch, hive.3980.3.patch
>
>
> There have been a lot of comments on HIVE-3403, which involve changing 
> variable names/function names/adding more comments/general cleanup etc.
> Since HIVE-3403 involves a lot of refactoring, it was fairly difficult to
> address the comments there, since refreshing becomes impossible. This jira
> is to track those cleanups.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4094) decimal_3.q & decimal_serde.q fail on hadoop 2

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-4094:
-

Attachment: HIVE-4094.patch

Added order bys to the failing queries to lock down the output.

> decimal_3.q & decimal_serde.q fail on hadoop 2
> --
>
> Key: HIVE-4094
> URL: https://issues.apache.org/jira/browse/HIVE-4094
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-4094.patch
>
>
> Some of the decimal unit tests fail on hadoop 2. The reason is unspecified 
> order in some of the queries.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4094) decimal_3.q & decimal_serde.q fail on hadoop 2

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-4094:
-

Status: Patch Available  (was: Open)

> decimal_3.q & decimal_serde.q fail on hadoop 2
> --
>
> Key: HIVE-4094
> URL: https://issues.apache.org/jira/browse/HIVE-4094
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-4094.patch
>
>
> Some of the decimal unit tests fail on hadoop 2. The reason is unspecified 
> order in some of the queries.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590231#comment-13590231
 ] 

Namit Jain commented on HIVE-3891:
--

I dont see any change in AbstractSMBJoinProc in this patch

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Vikram Dixit K (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590232#comment-13590232
 ] 

Vikram Dixit K commented on HIVE-3891:
--

Hey Namit,

I thought that this was the patch that changed something in 
AbstractSMBJoinProcessor. I think this was done as part of HIVE-3403. Just 
caught this while doing some testing with this patch.

Thanks
Vikram.

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590233#comment-13590233
 ] 

Namit Jain commented on HIVE-3891:
--

I didn't understand your case.

A join B on A.key = B.key join C on A.key = C.key

or

A join B on A.key = B.key join C on A.value = C.value

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590234#comment-13590234
 ] 

Namit Jain commented on HIVE-3891:
--

Both of them should work.

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4077) alterPartition and alterPartitions methods in ObjectStore swallow exceptions

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590238#comment-13590238
 ] 

Namit Jain commented on HIVE-4077:
--

+1

> alterPartition and alterPartitions methods in ObjectStore swallow exceptions
> 
>
> Key: HIVE-4077
> URL: https://issues.apache.org/jira/browse/HIVE-4077
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-4077.1.patch.txt, HIVE-4077.2.patch.txt, 
> HIVE-4077.3.patch.txt
>
>
> The alterPartition and alterPartitions methods in the ObjectStore class throw 
> a MetaException in the case of a failure but do not include the cause, 
> meaning that information is lost.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Vikram Dixit K (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590250#comment-13590250
 ] 

Vikram Dixit K commented on HIVE-3891:
--

Hi Namit,

The test case I have is very similar to the star join case. I have a 4 way join 
with 3 dimension tables and 1 fact table. All the joins are on different keys. 
By default, the average partition size is used as the tie breaker for the 
choice of big table. However when I run this query, I get the array out of 
bounds exception as it chose the table from the 2nd join (T3 from below) as the 
big table (index = 2). However, the joinCtx right alias has only one table. So 
this resulted in a crash.

{noformat}
T1 \
  J1.out \
T2 /   J2.out \
  T3 /  J3.out (Final)
   T4 /
{noformat}

Thanks
Vikram.

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Vikram Dixit K (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590262#comment-13590262
 ] 

Vikram Dixit K commented on HIVE-3891:
--

I think all this needs is SizeBasedBigTableSelectorForAutoSMJ to have the 
following additional snippet:

{noformat}

if (parentOp instanceof TableScanOperator) {
topOps.add(parentOp);

else if (parentOp instanceof CommonJoinOperator) {
  return;
}

else {
...

{noformat}

We shouldn't be taking that operation all the way to the top of the operator 
tree which is what is happening. Let me know what you think? 

Thanks
Vikram.

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3980) Cleanup after HIVE-3403

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3980:
-

Status: Patch Available  (was: Open)

Tests passed after refresh

> Cleanup after HIVE-3403
> ---
>
> Key: HIVE-3980
> URL: https://issues.apache.org/jira/browse/HIVE-3980
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3980.1.patch, hive.3980.2.patch, hive.3980.3.patch
>
>
> There have been a lot of comments on HIVE-3403, which involve changing 
> variable names/function names/adding more comments/general cleanup etc.
> Since HIVE-3403 involves a lot of refactoring, it was fairly difficult to
> address the comments there, since refreshing becomes impossible. This jira
> is to track those cleanups.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4077) alterPartition and alterPartitions methods in ObjectStore swallow exceptions

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4077:
-

   Resolution: Fixed
Fix Version/s: 0.11.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Kevin

> alterPartition and alterPartitions methods in ObjectStore swallow exceptions
> 
>
> Key: HIVE-4077
> URL: https://issues.apache.org/jira/browse/HIVE-4077
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Fix For: 0.11.0
>
> Attachments: HIVE-4077.1.patch.txt, HIVE-4077.2.patch.txt, 
> HIVE-4077.3.patch.txt
>
>
> The alterPartition and alterPartitions methods in the ObjectStore class throw 
> a MetaException in the case of a failure but do not include the cause, 
> meaning that information is lost.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590294#comment-13590294
 ] 

Namit Jain commented on HIVE-3891:
--

canConvertJoinToBucketMapJoin:

   The join should be rejected by:

// This has already been inspected and rejected
if (context.getRejectedJoinOps().contains(joinOp)) {
  return false;
}


Can you open a separate jira for this ?
We should never run into this scenario.


> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3891) physical optimizer changes for auto sort-merge join

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13590295#comment-13590295
 ] 

Namit Jain commented on HIVE-3891:
--

This is a existing bug, and we need to debug this, but not as part of this 
patch.

> physical optimizer changes for auto sort-merge join
> ---
>
> Key: HIVE-3891
> URL: https://issues.apache.org/jira/browse/HIVE-3891
> Project: Hive
>  Issue Type: Bug
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: auto_sortmerge_join_1.q, auto_sortmerge_join_1.q.out, 
> hive.3891.1.patch, hive.3891.2.patch, hive.3891.3.patch, hive.3891.4.patch, 
> hive.3891.5.patch, hive.3891.6.patch, hive.3891.7.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira