[jira] [Updated] (HIVE-3820) Consider creating a literal like "D" or "BD" for representing Decimal type constants

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-3820:
-

Attachment: HIVE-3820.1.patch

> Consider creating a literal like "D" or "BD" for representing Decimal type 
> constants
> 
>
> Key: HIVE-3820
> URL: https://issues.apache.org/jira/browse/HIVE-3820
> Project: Hive
>  Issue Type: Bug
>Reporter: Mark Grover
> Attachments: HIVE-3820.1.patch, HIVE-3820.D8823.1.patch
>
>
> When the HIVE-2693 gets committed, users are going to see this behavior:
> {code}
> hive> select cast(3.14 as decimal) from decimal_3 limit 1;
> 3.140124344978758017532527446746826171875
> {code}
> That's intuitively incorrect but is the case because 3.14 (double) is being 
> converted to BigDecimal because of which there is a precision mismatch.
> We should consider creating a new literal for expressing constants of Decimal 
> type as Gunther suggested in HIVE-2693.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3820) Consider creating a literal like "D" or "BD" for representing Decimal type constants

[Phabricator (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3820?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Phabricator updated HIVE-3820:
--

Attachment: HIVE-3820.D8823.1.patch

hagleitn requested code review of "HIVE-3820 [jira] Consider creating a literal 
like "D" or "BD" for representing Decimal type constants".

Reviewers: JIRA

HIVE-3820: Decimal literals

When the HIVE-2693 gets committed, users are going to see this behavior:

hive> select cast(3.14 as decimal) from decimal_3 limit 1;
3.140124344978758017532527446746826171875

That's intuitively incorrect but is the case because 3.14 (double) is being 
converted to BigDecimal because of which there is a precision mismatch.

We should consider creating a new literal for expressing constants of Decimal 
type as Gunther suggested in HIVE-2693.

TEST PLAN
  EMPTY

REVISION DETAIL
  https://reviews.facebook.net/D8823

AFFECTED FILES
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
  ql/src/java/org/apache/hadoop/hive/ql/parse/HiveLexer.g
  ql/src/java/org/apache/hadoop/hive/ql/parse/IdentifiersParser.g
  ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java
  ql/src/test/queries/clientpositive/literal_decimal.q
  ql/src/test/results/clientpositive/literal_decimal.q.out
  
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/JavaBigDecimalObjectInspector.java

MANAGE HERALD RULES
  https://reviews.facebook.net/herald/view/differential/

WHY DID I GET THIS EMAIL?
  https://reviews.facebook.net/herald/transcript/21441/

To: JIRA, hagleitn


> Consider creating a literal like "D" or "BD" for representing Decimal type 
> constants
> 
>
> Key: HIVE-3820
> URL: https://issues.apache.org/jira/browse/HIVE-3820
> Project: Hive
>  Issue Type: Bug
>Reporter: Mark Grover
> Attachments: HIVE-3820.1.patch, HIVE-3820.D8823.1.patch
>
>
> When the HIVE-2693 gets committed, users are going to see this behavior:
> {code}
> hive> select cast(3.14 as decimal) from decimal_3 limit 1;
> 3.140124344978758017532527446746826171875
> {code}
> That's intuitively incorrect but is the case because 3.14 (double) is being 
> converted to BigDecimal because of which there is a precision mismatch.
> We should consider creating a new literal for expressing constants of Decimal 
> type as Gunther suggested in HIVE-2693.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4025) Add reflect UDF for member method invocation of column

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584127#comment-13584127
 ] 

Hudson commented on HIVE-4025:
--

Integrated in hive-trunk-hadoop1 #97 (See 
[https://builds.apache.org/job/hive-trunk-hadoop1/97/])
HIVE-4025 Add reflect UDF for member method invocation of column
(Navis via namit) (Revision 1448900)

 Result = ABORTED
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448900
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFReflect.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFReflect.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFReflect2.java
* /hive/trunk/ql/src/test/queries/clientpositive/udf_reflect2.q
* /hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_reflect2.q.out


> Add reflect UDF for member method invocation of column
> --
>
> Key: HIVE-4025
> URL: https://issues.apache.org/jira/browse/HIVE-4025
> Project: Hive
>  Issue Type: Improvement
>  Components: UDF
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Fix For: 0.11.0
>
> Attachments: HIVE-4025.D8601.1.patch, HIVE-4025.D8601.2.patch
>
>
> There are many useful non-static methods on type of primitive types. But 
> current reflect UDF cannot invoke those. For example,
> select reflect2(value, "replace", "val", "VALUE") from src;
> which replaces 'val' part of value column with 'VALUE'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-684) add UDF make_set

[PRETTY SITHARA (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

PRETTY SITHARA updated HIVE-684:


Status: Patch Available  (was: Open)

PATCH for HIVE-684

> add UDF make_set
> 
>
> Key: HIVE-684
> URL: https://issues.apache.org/jira/browse/HIVE-684
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
> Attachments: HIVE-684.1.patch.txt, input.txt.txt, make_set.q, 
> make_set.q.out
>
>
> add UDFmake_set
> look at
> http://dev.mysql.com/doc/refman/5.0/en/func-op-summary-ref.html
> for details

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4060) Make streams for types for ORC pluggable

[Namit Jain (JIRA)]

Namit Jain created HIVE-4060:


 Summary: Make streams for types for ORC pluggable
 Key: HIVE-4060
 URL: https://issues.apache.org/jira/browse/HIVE-4060
 Project: Hive
  Issue Type: Improvement
  Components: Serializers/Deserializers
Reporter: Namit Jain


Currently, all types are written in a hard-coded manner.

For eg. integers are always written in a RLE manner, which are written in a
binary manner. What if a method writing deltas is useful for some dataset ? 
It should be made pluggable, and the version should somehow indicate the policy.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4061) skip columns which are not accessed in the query for ORC

[Namit Jain (JIRA)]

Namit Jain created HIVE-4061:


 Summary: skip columns which are not accessed in the query for ORC
 Key: HIVE-4061
 URL: https://issues.apache.org/jira/browse/HIVE-4061
 Project: Hive
  Issue Type: Improvement
  Components: Serializers/Deserializers
Reporter: Namit Jain




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4062) use column statistics for ORC to evaluate predicates for ORC

[Namit Jain (JIRA)]

Namit Jain created HIVE-4062:


 Summary: use column statistics for ORC to evaluate predicates for 
ORC
 Key: HIVE-4062
 URL: https://issues.apache.org/jira/browse/HIVE-4062
 Project: Hive
  Issue Type: Improvement
  Components: Serializers/Deserializers
Reporter: Namit Jain




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4015) Add ORC file to the grammar as a file format

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584228#comment-13584228
 ] 

Namit Jain commented on HIVE-4015:
--

The following syntax should be supported:

create table foo() stored as ORCFile;

> Add ORC file to the grammar as a file format
> 
>
> Key: HIVE-4015
> URL: https://issues.apache.org/jira/browse/HIVE-4015
> Project: Hive
>  Issue Type: Improvement
>Reporter: Owen O'Malley
>Assignee: Owen O'Malley
>
> It would be much more convenient for users if we enable them to use ORC as a 
> file format in the HQL grammar. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-4015) Add ORC file to the grammar as a file format

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain reassigned HIVE-4015:


Assignee: (was: Owen O'Malley)

> Add ORC file to the grammar as a file format
> 
>
> Key: HIVE-4015
> URL: https://issues.apache.org/jira/browse/HIVE-4015
> Project: Hive
>  Issue Type: Improvement
>Reporter: Owen O'Malley
>
> It would be much more convenient for users if we enable them to use ORC as a 
> file format in the HQL grammar. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3775) Unit test failures due to unspecified order of results in "show grant" command

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3775:
-

Status: Open  (was: Patch Available)

comments

> Unit test failures due to unspecified order of results in "show grant" command
> --
>
> Key: HIVE-3775
> URL: https://issues.apache.org/jira/browse/HIVE-3775
> Project: Hive
>  Issue Type: Bug
>  Components: Testing Infrastructure
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-3775.1-r1417768.patch, HIVE-3775.2.patch
>
>
> A number of unit tests (sometimes) using "show grant" fail, when run on 
> windows or previous failures put the database in an unexpected state.
> The reason is that the output of "show grant" is not specified to be in any 
> particular order, but the golden files expect it to be.
> The unit test framework should be extended to handled cases like that.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-684) add UDF make_set

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-684:


Status: Open  (was: Patch Available)

can you create a phabricator entry ?

> add UDF make_set
> 
>
> Key: HIVE-684
> URL: https://issues.apache.org/jira/browse/HIVE-684
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
> Attachments: HIVE-684.1.patch.txt, input.txt.txt, make_set.q, 
> make_set.q.out
>
>
> add UDFmake_set
> look at
> http://dev.mysql.com/doc/refman/5.0/en/func-op-summary-ref.html
> for details

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Assigned] (HIVE-684) add UDF make_set

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain reassigned HIVE-684:
---

Assignee: PRETTY SITHARA

> add UDF make_set
> 
>
> Key: HIVE-684
> URL: https://issues.apache.org/jira/browse/HIVE-684
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: PRETTY SITHARA
> Attachments: HIVE-684.1.patch.txt, input.txt.txt, make_set.q, 
> make_set.q.out
>
>
> add UDFmake_set
> look at
> http://dev.mysql.com/doc/refman/5.0/en/func-op-summary-ref.html
> for details

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4063) Negative tests for types not supported by ORC

[Namit Jain (JIRA)]

Namit Jain created HIVE-4063:


 Summary: Negative tests for types not supported by ORC
 Key: HIVE-4063
 URL: https://issues.apache.org/jira/browse/HIVE-4063
 Project: Hive
  Issue Type: Improvement
  Components: Serializers/Deserializers
Reporter: Namit Jain


ORC supports a type specific encoding.
What happens if there is a type which is not supported by ORC - say UrlType,
but supported by Hive.
It should throw an error gracefully.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4025) Add reflect UDF for member method invocation of column

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584355#comment-13584355
 ] 

Hudson commented on HIVE-4025:
--

Integrated in Hive-trunk-h0.21 #1982 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1982/])
HIVE-4025 Add reflect UDF for member method invocation of column
(Navis via namit) (Revision 1448900)

 Result = SUCCESS
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448900
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFReflect.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFReflect.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFReflect2.java
* /hive/trunk/ql/src/test/queries/clientpositive/udf_reflect2.q
* /hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_reflect2.q.out


> Add reflect UDF for member method invocation of column
> --
>
> Key: HIVE-4025
> URL: https://issues.apache.org/jira/browse/HIVE-4025
> Project: Hive
>  Issue Type: Improvement
>  Components: UDF
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Fix For: 0.11.0
>
> Attachments: HIVE-4025.D8601.1.patch, HIVE-4025.D8601.2.patch
>
>
> There are many useful non-static methods on type of primitive types. But 
> current reflect UDF cannot invoke those. For example,
> select reflect2(value, "replace", "val", "VALUE") from src;
> which replaces 'val' part of value column with 'VALUE'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Hive-trunk-h0.21 - Build # 1982 - Fixed

[Apache Jenkins Server]

Changes for Build #1981
[kevinwilfong] HIVE-701. lots of reserved keywords in hive. (Samuel Yuan via 
kevinwilfong)

[namit] HIVE-3710 HiveConf.ConfVars.HIVE_STATS_COLLECT_RAWDATASIZE should not be
checked in FileSinkOperator (Gang Tim Liu via namit)

[hashutosh] HIVE-3788 : testCliDriver_repair fails on hadoop-1 (Gunther 
Hagleitner via Ashutosh Chauhan)


Changes for Build #1982
[namit] HIVE-4025 Add reflect UDF for member method invocation of column
(Navis via namit)




All tests passed

The Apache Jenkins build system has built Hive-trunk-h0.21 (build #1982)

Status: Fixed

Check console output at https://builds.apache.org/job/Hive-trunk-h0.21/1982/ to 
view the results.

[jira] [Commented] (HIVE-948) more query plan optimization rules

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584374#comment-13584374
 ] 

Ashutosh Chauhan commented on HIVE-948:
---

Following queries failed:
* testCliDriver_auto_smb_mapjoin_14
* testCliDriver_binarysortable_1
* testCliDriver_udf_reflect2

For last two queries, simply need to update .q.out files.

> more query plan optimization rules 
> ---
>
> Key: HIVE-948
> URL: https://issues.apache.org/jira/browse/HIVE-948
> Project: Hive
>  Issue Type: Improvement
>Reporter: Ning Zhang
>Assignee: Navis
> Attachments: HIVE-948.D8463.1.patch, HIVE-948.D8463.2.patch, 
> HIVE-948.D8463.3.patch, HIVE-948.D8463.3.patch, HIVE-948.D8463.4.patch, 
> HIVE-948.testresult_only.txt
>
>
> Many query plans are not optimal in that they contain redundant operators. 
> Some examples are unnecessary select operators (select followed by select, 
> select output being the same as input etc.). Even though these operators are 
> not very expensive, they could account for around 10% of CPU time in some 
> simple queries. It seems they are low-hanging fruits that we should pick 
> first. 
> BTW, it seems these optimization rules should be added at the last stage of 
> the physical optimization phase since some redundant operators are added to 
> facilitate physical plan generation. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3672) Support altering partition column type in Hive

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584405#comment-13584405
 ] 

Namit Jain commented on HIVE-3672:
--

+1

> Support altering partition column type in Hive
> --
>
> Key: HIVE-3672
> URL: https://issues.apache.org/jira/browse/HIVE-3672
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI, SQL
>Reporter: Jingwei Lu
>Assignee: Jingwei Lu
>  Labels: features
> Attachments: HIVE-3672.1.patch.txt, HIVE-3672.2.patch.txt, 
> HIVE-3672.3.patch.txt, HIVE-3672.4.patch.txt, HIVE-3672.5.patch.txt, 
> HIVE-3672.6.patch.txt, HIVE-3672.6.patch.txt, HIVE-3672.7.patch.txt, 
> HIVE-3672.8.patch.txt, HIVE-3672.9.patch.txt
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> Currently, Hive does not allow altering partition column types.  As we've 
> discouraged users from using non-string partition column types, this presents 
> a problem for users who want to change there partition columns to be strings, 
> they have to rename their table, create a new table, and copy all the data 
> over.
> To support this via the CLI, adding a command like ALTER TABLE  
> PARTITION COLUMN ( );

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3672) Support altering partition column type in Hive

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3672:
-

   Resolution: Fixed
Fix Version/s: 0.11.0
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Committed. Thanks Jingwei

> Support altering partition column type in Hive
> --
>
> Key: HIVE-3672
> URL: https://issues.apache.org/jira/browse/HIVE-3672
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI, SQL
>Reporter: Jingwei Lu
>Assignee: Jingwei Lu
>  Labels: features
> Fix For: 0.11.0
>
> Attachments: HIVE-3672.1.patch.txt, HIVE-3672.2.patch.txt, 
> HIVE-3672.3.patch.txt, HIVE-3672.4.patch.txt, HIVE-3672.5.patch.txt, 
> HIVE-3672.6.patch.txt, HIVE-3672.6.patch.txt, HIVE-3672.7.patch.txt, 
> HIVE-3672.8.patch.txt, HIVE-3672.9.patch.txt
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> Currently, Hive does not allow altering partition column types.  As we've 
> discouraged users from using non-string partition column types, this presents 
> a problem for users who want to change there partition columns to be strings, 
> they have to rename their table, create a new table, and copy all the data 
> over.
> To support this via the CLI, adding a command like ALTER TABLE  
> PARTITION COLUMN ( );

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3980) Cleanup after HIVE-3403

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-3980:
---

Status: Open  (was: Patch Available)

Current patch doesn't address all the concerns.

> Cleanup after HIVE-3403
> ---
>
> Key: HIVE-3980
> URL: https://issues.apache.org/jira/browse/HIVE-3980
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3980.1.patch, hive.3980.2.patch
>
>
> There have been a lot of comments on HIVE-3403, which involve changing 
> variable names/function names/adding more comments/general cleanup etc.
> Since HIVE-3403 involves a lot of refactoring, it was fairly difficult to
> address the comments there, since refreshing becomes impossible. This jira
> is to track those cleanups.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4035) Column Pruner for PTF Op

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584476#comment-13584476
 ] 

Ashutosh Chauhan commented on HIVE-4035:


https://reviews.facebook.net/D8829

> Column Pruner for PTF Op
> 
>
> Key: HIVE-4035
> URL: https://issues.apache.org/jira/browse/HIVE-4035
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Prajakta Kalmegh
> Attachments: HIVE-4035.1.patch.txt
>
>
> for a PTFOp for Windowing; should prune columns based on its children. 
> Virtual Columns should only be carried forward if needed. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-701) lots of reserved keywords in hive

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584490#comment-13584490
 ] 

Kevin Wilfong commented on HIVE-701:


AND, OR, and NOT also seem to be big contributors.

The warnings bring up a good point, e.g. if you run SELECT distinct col FROM 
... are you selecting a column named distinct and aliasing it as col or are 
your trying to select distinct values of the column called col

> lots of reserved keywords in hive
> -
>
> Key: HIVE-701
> URL: https://issues.apache.org/jira/browse/HIVE-701
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Samuel Yuan
> Fix For: 0.11.0
>
> Attachments: HIVE-701.1.patch.txt, HIVE-701.2.patch.txt, 
> HIVE-701.D8397.1.patch, HIVE-701.HIVE-701.D8397.2.patch, 
> HIVE-701.HIVE-701.D8397.3.patch
>
>
> There is a problem if we want to use some reserved keywords:
> for example, creating a function of name left/right ? left/right is already a 
> reserved keyword.
> The other way around should also be possible - if we want to add a 'show 
> tables status' and some applications already use status as a column name, 
> they should not break

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

jdo2-api dependency

[Nitay Joffe]

Hey guys,

The latest open source hive release (0.10.0) depends on javax.jdo artifact 
jdo2-api version 2.3-ec. This version is not actually in maven central, which 
means everyone who uses hive requires custom maven repository definitions which 
is discouraged by maven folks. I pinged the javax.jdo guys about it and they 
recommended we upgrade to 3.0. See 
http://mail-archives.apache.org/mod_mbox/db-jdo-dev/201302.mbox/%3CCAGZB7RguuEJnpVbtaqOgYEbsUNzP3aMSmM8SM8aOxcb-hLWwjg%40mail.gmail.com%3E
 for the conversation. Can you guys fix this?

Thanks,
- Nitay

[jira] [Commented] (HIVE-3980) Cleanup after HIVE-3403

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584520#comment-13584520
 ] 

Namit Jain commented on HIVE-3980:
--

[~ashutoshc], which ones ?

> Cleanup after HIVE-3403
> ---
>
> Key: HIVE-3980
> URL: https://issues.apache.org/jira/browse/HIVE-3980
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3980.1.patch, hive.3980.2.patch
>
>
> There have been a lot of comments on HIVE-3403, which involve changing 
> variable names/function names/adding more comments/general cleanup etc.
> Since HIVE-3403 involves a lot of refactoring, it was fairly difficult to
> address the comments there, since refreshing becomes impossible. This jira
> is to track those cleanups.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4064) Handle db qualified names consistently across all HiveQL statements

[Shreepadma Venugopalan (JIRA)]

Shreepadma Venugopalan created HIVE-4064:


 Summary: Handle db qualified names consistently across all HiveQL 
statements
 Key: HIVE-4064
 URL: https://issues.apache.org/jira/browse/HIVE-4064
 Project: Hive
  Issue Type: Bug
  Components: SQL
Affects Versions: 0.10.0
Reporter: Shreepadma Venugopalan


Hive doesn't consistently handle db qualified names across all HiveQL 
statements. While some HiveQL statements such as SELECT support DB qualified 
names, other such as CREATE INDEX doesn't. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4005) Column truncation

[Kevin Wilfong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-4005:


Attachment: HIVE-4005.5.patch.txt

> Column truncation
> -
>
> Key: HIVE-4005
> URL: https://issues.apache.org/jira/browse/HIVE-4005
> Project: Hive
>  Issue Type: New Feature
>  Components: CLI
>Affects Versions: 0.11.0
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-4005.1.patch.txt, HIVE-4005.2.patch.txt, 
> HIVE-4005.3.patch.txt, HIVE-4005.4.patch.txt, HIVE-4005.5.patch.txt
>
>
> Column truncation allows users to remove data for columns that are no longer 
> useful.
> This is done by removing the data for the column and setting the length of 
> the column data and related lengths to 0 in the RC file header.
> RC file was fixed to recognize columns with lengths of zero to be empty and 
> are treated as if the column doesn't exist in the data, a null is returned 
> for every value of that column in every row. This is the same thing that 
> happens when more columns are selected than exist in the file.
> A new command was added to the CLI
> TRUNCATE TABLE ... PARTITION ... COLUMNS ...
> This launches a map only job where each mapper rewrites a single file without 
> the unnecessary column data and the adjusted headers. It does not 
> uncompress/deserialize the data so it is much faster than rewriting the data 
> with NULLs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4036) remove use of FunctionRegistry during PTF Op initialization

[Harish Butani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-4036:


Attachment: HIVE-4036.1.patch.txt

> remove use of FunctionRegistry during PTF Op initialization
> ---
>
> Key: HIVE-4036
> URL: https://issues.apache.org/jira/browse/HIVE-4036
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-4036.1.patch.txt
>
>
> current way of initializing WindowFnDefs breaks down for dynamic UDAFs

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4036) remove use of FunctionRegistry during PTF Op initialization

[Harish Butani (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584572#comment-13584572
 ] 

Harish Butani commented on HIVE-4036:
-

patch attached. 
apply after 4035, because removes 'set' command added in .q file.

> remove use of FunctionRegistry during PTF Op initialization
> ---
>
> Key: HIVE-4036
> URL: https://issues.apache.org/jira/browse/HIVE-4036
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-4036.1.patch.txt
>
>
> current way of initializing WindowFnDefs breaks down for dynamic UDAFs

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4025) Add reflect UDF for member method invocation of column

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584574#comment-13584574
 ] 

Hudson commented on HIVE-4025:
--

Integrated in Hive-trunk-hadoop2 #134 (See 
[https://builds.apache.org/job/Hive-trunk-hadoop2/134/])
HIVE-4025 Add reflect UDF for member method invocation of column
(Navis via namit) (Revision 1448900)

 Result = FAILURE
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1448900
Files : 
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/FunctionRegistry.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDFReflect.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFReflect.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFReflect2.java
* /hive/trunk/ql/src/test/queries/clientpositive/udf_reflect2.q
* /hive/trunk/ql/src/test/results/clientpositive/show_functions.q.out
* /hive/trunk/ql/src/test/results/clientpositive/udf_reflect2.q.out


> Add reflect UDF for member method invocation of column
> --
>
> Key: HIVE-4025
> URL: https://issues.apache.org/jira/browse/HIVE-4025
> Project: Hive
>  Issue Type: Improvement
>  Components: UDF
>Reporter: Navis
>Assignee: Navis
>Priority: Trivial
> Fix For: 0.11.0
>
> Attachments: HIVE-4025.D8601.1.patch, HIVE-4025.D8601.2.patch
>
>
> There are many useful non-static methods on type of primitive types. But 
> current reflect UDF cannot invoke those. For example,
> select reflect2(value, "replace", "val", "VALUE") from src;
> which replaces 'val' part of value column with 'VALUE'

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4064) Handle db qualified names consistently across all HiveQL statements

[Jarek Jarcec Cecho (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584579#comment-13584579
 ] 

Jarek Jarcec Cecho commented on HIVE-4064:
--

Another command that is not working consistently is:

{code}
ALTER TABLE db.table ...
{code}

> Handle db qualified names consistently across all HiveQL statements
> ---
>
> Key: HIVE-4064
> URL: https://issues.apache.org/jira/browse/HIVE-4064
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 0.10.0
>Reporter: Shreepadma Venugopalan
>
> Hive doesn't consistently handle db qualified names across all HiveQL 
> statements. While some HiveQL statements such as SELECT support DB qualified 
> names, other such as CREATE INDEX doesn't. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4042) ignore mapjoin hint

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584584#comment-13584584
 ] 

Kevin Wilfong commented on HIVE-4042:
-

What if the table is not bucketed but the partitions are.  Granted this is an 
unusual case, but it's possible that this will prevent bucket map join or SMB 
join from being used where it normally would be.

> ignore mapjoin hint
> ---
>
> Key: HIVE-4042
> URL: https://issues.apache.org/jira/browse/HIVE-4042
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.4042.1.patch, hive.4042.2.patch
>
>
> After HIVE-3784, in a production environment, it can become difficult to
> deploy since a lot of production queries can break.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: jdo2-api dependency

[Jarek Jarcec Cecho]

Hi Nitay,
would you mind opening a JIRA for that?

Jarcec

On Fri, Feb 22, 2013 at 01:03:15PM -0500, Nitay Joffe wrote:
> Hey guys,
> 
> The latest open source hive release (0.10.0) depends on javax.jdo artifact 
> jdo2-api version 2.3-ec. This version is not actually in maven central, which 
> means everyone who uses hive requires custom maven repository definitions 
> which is discouraged by maven folks. I pinged the javax.jdo guys about it and 
> they recommended we upgrade to 3.0. See 
> http://mail-archives.apache.org/mod_mbox/db-jdo-dev/201302.mbox/%3CCAGZB7RguuEJnpVbtaqOgYEbsUNzP3aMSmM8SM8aOxcb-hLWwjg%40mail.gmail.com%3E
>  for the conversation. Can you guys fix this?
> 
> Thanks,
> - Nitay


signature.asc
Description: Digital signature

[jira] [Commented] (HIVE-3672) Support altering partition column type in Hive

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584635#comment-13584635
 ] 

Hudson commented on HIVE-3672:
--

Integrated in hive-trunk-hadoop1 #98 (See 
[https://builds.apache.org/job/hive-trunk-hadoop1/98/])
HIVE-3672 Support altering partition column type in Hive
(Jingwei Lu via namit) (Revision 1449109)

 Result = ABORTED
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1449109
Files : 
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableAlterPartDesc.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java
* 
/hive/trunk/ql/src/test/queries/clientnegative/alter_partition_coltype_2columns.q
* 
/hive/trunk/ql/src/test/queries/clientnegative/alter_partition_coltype_invalidcolname.q
* 
/hive/trunk/ql/src/test/queries/clientnegative/alter_partition_coltype_invalidtype.q
* /hive/trunk/ql/src/test/queries/clientpositive/alter_partition_coltype.q
* 
/hive/trunk/ql/src/test/results/clientnegative/alter_partition_coltype_2columns.q.out
* 
/hive/trunk/ql/src/test/results/clientnegative/alter_partition_coltype_invalidcolname.q.out
* 
/hive/trunk/ql/src/test/results/clientnegative/alter_partition_coltype_invalidtype.q.out
* /hive/trunk/ql/src/test/results/clientpositive/alter_partition_coltype.q.out


> Support altering partition column type in Hive
> --
>
> Key: HIVE-3672
> URL: https://issues.apache.org/jira/browse/HIVE-3672
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI, SQL
>Reporter: Jingwei Lu
>Assignee: Jingwei Lu
>  Labels: features
> Fix For: 0.11.0
>
> Attachments: HIVE-3672.1.patch.txt, HIVE-3672.2.patch.txt, 
> HIVE-3672.3.patch.txt, HIVE-3672.4.patch.txt, HIVE-3672.5.patch.txt, 
> HIVE-3672.6.patch.txt, HIVE-3672.6.patch.txt, HIVE-3672.7.patch.txt, 
> HIVE-3672.8.patch.txt, HIVE-3672.9.patch.txt
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> Currently, Hive does not allow altering partition column types.  As we've 
> discouraged users from using non-string partition column types, this presents 
> a problem for users who want to change there partition columns to be strings, 
> they have to rename their table, create a new table, and copy all the data 
> over.
> To support this via the CLI, adding a command like ALTER TABLE  
> PARTITION COLUMN ( );

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4065) Broken documentation links on Hive website

[Carl Steinbach (JIRA)]

Carl Steinbach created HIVE-4065:


 Summary: Broken documentation links on Hive website
 Key: HIVE-4065
 URL: https://issues.apache.org/jira/browse/HIVE-4065
 Project: Hive
  Issue Type: Bug
  Components: Website
Reporter: Carl Steinbach


Many of the documentation links on the Hive website (hive.apache.org) are 
broken. Specifically, links for the following versions don't work: 0.8.0, 
0.8.1, 0.9.0, 0.10.0

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4065) Broken documentation links on Hive website

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584708#comment-13584708
 ] 

Carl Steinbach commented on HIVE-4065:
--

Lefty recently finished copying all of the old xdocs over to the wiki. I'm in 
favor of making the wiki the official location for Hive documentation and 
removing the version specific links from the website. We also currently provide 
links to javadocs for non-public internal APIs. I think these should be removed.

> Broken documentation links on Hive website
> --
>
> Key: HIVE-4065
> URL: https://issues.apache.org/jira/browse/HIVE-4065
> Project: Hive
>  Issue Type: Bug
>  Components: Website
>Reporter: Carl Steinbach
>
> Many of the documentation links on the Hive website (hive.apache.org) are 
> broken. Specifically, links for the following versions don't work: 0.8.0, 
> 0.8.1, 0.9.0, 0.10.0

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4066) New website for Hive

[Carl Steinbach (JIRA)]

Carl Steinbach created HIVE-4066:


 Summary: New website for Hive
 Key: HIVE-4066
 URL: https://issues.apache.org/jira/browse/HIVE-4066
 Project: Hive
  Issue Type: Bug
  Components: Website
Reporter: Carl Steinbach


The current design of the Hive website (hive.apache.org) was inherited from the 
Hadoop project. I'm not a big fan, and apparently [other 
people|https://twitter.com/Prinzhorn/status/294516153475280898] feel the same 
way.

I propose we make the website look more like 
[Mahout's|http://mahout.apache.org/].

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-4067) Followup to HIVE-701: reduce ambiguity in grammar

[Samuel Yuan (JIRA)]

Samuel Yuan created HIVE-4067:
-

 Summary: Followup to HIVE-701: reduce ambiguity in grammar
 Key: HIVE-4067
 URL: https://issues.apache.org/jira/browse/HIVE-4067
 Project: Hive
  Issue Type: Bug
  Components: Query Processor
Reporter: Samuel Yuan
Assignee: Samuel Yuan
Priority: Minor


After HIVE-701 the grammar has become much more ambiguous, and the compilation 
generates a large number of warnings. Making FROM, DISTINCT, PRESERVE, COLUMN, 
ALL, AND, OR, and NOT reserved keywords again reduces the number of warnings to 
134, up from the original 81 warnings but down from the 565 after HIVE-701. 
Most of the remaining ambiguity is trivial, an example being "KW_ELEM_TYPE | 
KW_KEY_TYPE | KW_VALUE_TYPE | identifier", and they are all correctly handled 
by ANTLR.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-701) lots of reserved keywords in hive

[Samuel Yuan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584786#comment-13584786
 ] 

Samuel Yuan commented on HIVE-701:
--

Removing FROM, DISTINCT, PRESERVE, COLUMN, ALL, AND, OR, and NOT brings the 
number of warnings down to 134. Before this change it was at 81. Most of these 
other cases are trivial, an example being "KW_ELEM_TYPE | KW_KEY_TYPE | 
KW_VALUE_TYPE | identifier", and they are all correctly handled by ANTLR.

I have created a followup JIRA: HIVE-4067.

> lots of reserved keywords in hive
> -
>
> Key: HIVE-701
> URL: https://issues.apache.org/jira/browse/HIVE-701
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Samuel Yuan
> Fix For: 0.11.0
>
> Attachments: HIVE-701.1.patch.txt, HIVE-701.2.patch.txt, 
> HIVE-701.D8397.1.patch, HIVE-701.HIVE-701.D8397.2.patch, 
> HIVE-701.HIVE-701.D8397.3.patch
>
>
> There is a problem if we want to use some reserved keywords:
> for example, creating a function of name left/right ? left/right is already a 
> reserved keyword.
> The other way around should also be possible - if we want to add a 'show 
> tables status' and some applications already use status as a column name, 
> they should not break

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4036) remove use of FunctionRegistry during PTF Op initialization

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584799#comment-13584799
 ] 

Ashutosh Chauhan commented on HIVE-4036:


+1 looks good.

> remove use of FunctionRegistry during PTF Op initialization
> ---
>
> Key: HIVE-4036
> URL: https://issues.apache.org/jira/browse/HIVE-4036
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-4036.1.patch.txt
>
>
> current way of initializing WindowFnDefs breaks down for dynamic UDAFs

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hive-0.10.0-SNAPSHOT-h0.20.1 #73

[Apache Jenkins Server]

See 

--
[...truncated 42091 lines...]
[junit] Hadoop job information for null: number of mappers: 0; number of 
reducers: 0
[junit] 2013-02-22 15:47:30,705 null map = 100%,  reduce = 100%
[junit] Ended Job = job_local_0001
[junit] Execution completed successfully
[junit] Mapred Local Task Succeeded . Convert the Join into MapJoin
[junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/build/service/localscratchdir/hive_2013-02-22_15-47-27_501_5875832662408526477/-mr-1
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/build/service/tmp/hive_job_log_jenkins_201302221547_2147466235.txt
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] Copying file: 
file:/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/data/files/kv1.txt
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: load data local inpath 
'/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/data/files/kv1.txt'
 into table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] Copying data from 
file:/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/data/files/kv1.txt
[junit] Loading data to table default.testhivedrivertable
[junit] Table default.testhivedrivertable stats: [num_partitions: 0, 
num_files: 1, num_rows: 0, total_size: 5812, raw_data_size: 0]
[junit] POSTHOOK: query: load data local inpath 
'/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/data/files/kv1.txt'
 into table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: select * from testhivedrivertable limit 10
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: 
file:/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/build/service/localscratchdir/hive_2013-02-22_15-47-32_070_2056824481544820018/-mr-1
[junit] POSTHOOK: query: select * from testhivedrivertable limit 10
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/build/service/localscratchdir/hive_2013-02-22_15-47-32_070_2056824481544820018/-mr-1
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=/x1/jenkins/jenkins-slave/workspace/Hive-0.10.0-SNAPSHOT-h0.20.1/hive/build/service/tmp/hive_job_log_jenkins_201302221547_548824098.txt
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[j

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21 #300

[Apache Jenkins Server]

See 

--
[...truncated 36424 lines...]
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/tmp/jenkins/hive_2013-02-22_16-50-30_623_3701555483962271599/-mr-1
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] Copying file: 

[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: load data local inpath 
'
 into table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] Copying data from 

[junit] Loading data to table default.testhivedrivertable
[junit] POSTHOOK: query: load data local inpath 
'
 into table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: select * from testhivedrivertable limit 10
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: 
file:/tmp/jenkins/hive_2013-02-22_16-50-35_322_1250979079699882365/-mr-1
[junit] POSTHOOK: query: select * from testhivedrivertable limit 10
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 
file:/tmp/jenkins/hive_2013-02-22_16-50-35_322_1250979079699882365/-mr-1
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] Copying file: 

Merging HCatalog into Hive

[Alan Gates]

Alright, our vote has passed, it's time to get on with merging HCatalog into 
Hive.  Here's the things I can think of we need to deal with.  Please add 
additional issues I've missed:

1) Moving the code
2) Dealing with domain names in the code
3) The mailing lists
4) The JIRA
5) The website
6) Committer rights
7) Make a proposal for how HCat is released going forward
8) Publish an FAQ 

Proposals for how we handle these:
Below I propose an approach for how to handle each of these.  Feedback welcome.

1) Moving the code
I propose that HCat move into a subdirectory of Hive.  This fits nicely into 
Hive's structure since it already has metastore, ql, etc.  We'd just add 
'hcatalog' as a new directory.  This directory would contain hcatalog as it is 
today.  It does not follow Hive's standard build model so we'd need to do some 
work to make it so that building Hive also builds HCat, but this should be 
minimal.

2) Dealing with domain names
HCat code currently is under org.apache.hcatalog.  Do we want to change it?  In 
time we probably should change it to match the rest of Hive 
(org.apache.hadoop.hive.hcatalog).  We need to do this in a backward compatible 
way.  I propose we leave it as is for now and if we decide to in the future we 
can move the actual code to org.apache.hadoop.hive.hcatalog and create shell 
classes under org.apache.hcatalog.

3) The mailing lists
Given that our goal is to merge the projects and not create a subproject we 
should merge the mailing lists rather than keep hcat specific lists.  We can 
ask infra to remove hcatalog-*@incubator.apache.org and forward any new mail to 
the appropriate Hive lists.  We need to find out if they can auto-subscribe 
people from the hcat lists to the hive lists.  Given that traffic on the Hive 
lists is an order of magnitude higher we should warn people before we 
auto-subscribe them and allow them a chance to get off.

4) JIRA
We can create an hcatalog component in Hive's JIRA.  All new HCat issues could 
be filed there.  I don't know if there's a way to upload existing JIRAs into 
Hive's JIRA, but I think it would be better to leave them where they are.  We 
should see if infra can turn off the ability to create new JIRAs in hcatalog.

5) Website
We will need to integrate HCatalog's website with Hive's.  This should be easy 
except for the documentation.  HCat uses forrest for docs, Hive uses wiki.  We 
will need to put links under 'Documentation' for older versions of HCat docs so 
users can find them.  As far as how docs are handled for the next version of 
HCatalog, I think that depends on the answer to question 7 (next release of 
HCat), but I propose that HCat needs to conform to the way Hive does docs on 
wiki.  Though I would strongly encourage the HCat docs to be version specific 
(that is, have a set of wiki pages for each version).  
incubator.apache.org/hcatalog should be changed to forward to hive.apache.org.

6) Committer rights
Carl will need to set up committer rights for all the new HCat committers.  
Based on our discussion of making active HCat committers Hive submodule 
committers this would add the following set:  Alan, Sushanth, Francis, Daniel, 
Vandana, Travis, and Mithun.  Ashutosh and Paul are already Hive committers, 
and neither Devaraj nor Mac have been active in HCat in over a year.

7) Future releases
We need to discuss how future releases will happen, as I think this will help 
developers and users know how to respond to the merge.  I propose that HCat 
will simply become part of future Hive releases.  Thus Hive 0.11 (or whatever 
the next major release is) will include HCatalog.  If there are issues found we 
may need to make HCatalog 0.5.x releases from Hive, which should be fine.  But 
I propose there would not be an HCat 0.6.  To be clear I am not proposing that 
HCat functionality would be subsumed into Hive jars.  Just that the existing 
hcat jars would become part of Hive's release.

8) Communicate all of this
We should put up an FAQ page that has this information, as well as tracks our 
progress while we work on getting these things done.  

Alan.

[jira] [Work started] (HIVE-4056) Extend rcfilecat to support (un)compressed size and no. of row

[Gang Tim Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-4056 started by Gang Tim Liu.

> Extend rcfilecat to support (un)compressed size and no. of row
> --
>
> Key: HIVE-4056
> URL: https://issues.apache.org/jira/browse/HIVE-4056
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-4056.patch.1
>
>
> rcfilecat supports data and metadata:
> https://cwiki.apache.org/Hive/rcfilecat.html
> In metadata, it supports column statistics.
> It will be natural to extend metadata support to 
> 1. no. of rows 
> 2. uncompressed size for the file
> 3. compressed size for the file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4056) Extend rcfilecat to support (un)compressed size and no. of row

[Gang Tim Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584937#comment-13584937
 ] 

Gang Tim Liu commented on HIVE-4056:


https://reviews.facebook.net/D8847

> Extend rcfilecat to support (un)compressed size and no. of row
> --
>
> Key: HIVE-4056
> URL: https://issues.apache.org/jira/browse/HIVE-4056
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-4056.patch.1
>
>
> rcfilecat supports data and metadata:
> https://cwiki.apache.org/Hive/rcfilecat.html
> In metadata, it supports column statistics.
> It will be natural to extend metadata support to 
> 1. no. of rows 
> 2. uncompressed size for the file
> 3. compressed size for the file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4056) Extend rcfilecat to support (un)compressed size and no. of row

[Gang Tim Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Tim Liu updated HIVE-4056:
---

Attachment: HIVE-4056.patch.1

> Extend rcfilecat to support (un)compressed size and no. of row
> --
>
> Key: HIVE-4056
> URL: https://issues.apache.org/jira/browse/HIVE-4056
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-4056.patch.1
>
>
> rcfilecat supports data and metadata:
> https://cwiki.apache.org/Hive/rcfilecat.html
> In metadata, it supports column statistics.
> It will be natural to extend metadata support to 
> 1. no. of rows 
> 2. uncompressed size for the file
> 3. compressed size for the file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4056) Extend rcfilecat to support (un)compressed size and no. of row

[Gang Tim Liu (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584938#comment-13584938
 ] 

Gang Tim Liu commented on HIVE-4056:


patch is available on both attachment and phabricator.

> Extend rcfilecat to support (un)compressed size and no. of row
> --
>
> Key: HIVE-4056
> URL: https://issues.apache.org/jira/browse/HIVE-4056
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-4056.patch.1
>
>
> rcfilecat supports data and metadata:
> https://cwiki.apache.org/Hive/rcfilecat.html
> In metadata, it supports column statistics.
> It will be natural to extend metadata support to 
> 1. no. of rows 
> 2. uncompressed size for the file
> 3. compressed size for the file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4056) Extend rcfilecat to support (un)compressed size and no. of row

[Gang Tim Liu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gang Tim Liu updated HIVE-4056:
---

Status: Patch Available  (was: In Progress)

> Extend rcfilecat to support (un)compressed size and no. of row
> --
>
> Key: HIVE-4056
> URL: https://issues.apache.org/jira/browse/HIVE-4056
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-4056.patch.1
>
>
> rcfilecat supports data and metadata:
> https://cwiki.apache.org/Hive/rcfilecat.html
> In metadata, it supports column statistics.
> It will be natural to extend metadata support to 
> 1. no. of rows 
> 2. uncompressed size for the file
> 3. compressed size for the file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3672) Support altering partition column type in Hive

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13584997#comment-13584997
 ] 

Hudson commented on HIVE-3672:
--

Integrated in Hive-trunk-h0.21 #1983 (See 
[https://builds.apache.org/job/Hive-trunk-h0.21/1983/])
HIVE-3672 Support altering partition column type in Hive
(Jingwei Lu via namit) (Revision 1449109)

 Result = SUCCESS
namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1449109
Files : 
* 
/hive/trunk/metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/HiveParser.g
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java
* 
/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableAlterPartDesc.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/AlterTableDesc.java
* /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DDLWork.java
* 
/hive/trunk/ql/src/test/queries/clientnegative/alter_partition_coltype_2columns.q
* 
/hive/trunk/ql/src/test/queries/clientnegative/alter_partition_coltype_invalidcolname.q
* 
/hive/trunk/ql/src/test/queries/clientnegative/alter_partition_coltype_invalidtype.q
* /hive/trunk/ql/src/test/queries/clientpositive/alter_partition_coltype.q
* 
/hive/trunk/ql/src/test/results/clientnegative/alter_partition_coltype_2columns.q.out
* 
/hive/trunk/ql/src/test/results/clientnegative/alter_partition_coltype_invalidcolname.q.out
* 
/hive/trunk/ql/src/test/results/clientnegative/alter_partition_coltype_invalidtype.q.out
* /hive/trunk/ql/src/test/results/clientpositive/alter_partition_coltype.q.out


> Support altering partition column type in Hive
> --
>
> Key: HIVE-3672
> URL: https://issues.apache.org/jira/browse/HIVE-3672
> Project: Hive
>  Issue Type: Improvement
>  Components: CLI, SQL
>Reporter: Jingwei Lu
>Assignee: Jingwei Lu
>  Labels: features
> Fix For: 0.11.0
>
> Attachments: HIVE-3672.1.patch.txt, HIVE-3672.2.patch.txt, 
> HIVE-3672.3.patch.txt, HIVE-3672.4.patch.txt, HIVE-3672.5.patch.txt, 
> HIVE-3672.6.patch.txt, HIVE-3672.6.patch.txt, HIVE-3672.7.patch.txt, 
> HIVE-3672.8.patch.txt, HIVE-3672.9.patch.txt
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> Currently, Hive does not allow altering partition column types.  As we've 
> discouraged users from using non-string partition column types, this presents 
> a problem for users who want to change there partition columns to be strings, 
> they have to rename their table, create a new table, and copy all the data 
> over.
> To support this via the CLI, adding a command like ALTER TABLE  
> PARTITION COLUMN ( );

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4053) Add support for phonetic algorithms in Hive

[Krishna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krishna updated HIVE-4053:
--

Description: 
Following phonetic algorithms should be considered, which are very useful in 
search:
Soundex
Refined Soundex
Daitch–Mokotoff Soundex
Metaphone and Double Metaphone
New York State Identification and Intelligence System (NYSIIS)
Caverphone


  was:
Following phonetic algorithms should be considered, which are very useful in 
search:
Soundex
Daitch–Mokotoff Soundex
Metaphone and Double Metaphone
New York State Identification and Intelligence System (NYSIIS)
Caverphone



> Add support for phonetic algorithms in Hive
> ---
>
> Key: HIVE-4053
> URL: https://issues.apache.org/jira/browse/HIVE-4053
> Project: Hive
>  Issue Type: New Feature
>  Components: UDF
>Reporter: Krishna
>
> Following phonetic algorithms should be considered, which are very useful in 
> search:
> Soundex
> Refined Soundex
> Daitch–Mokotoff Soundex
> Metaphone and Double Metaphone
> New York State Identification and Intelligence System (NYSIIS)
> Caverphone

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4042) ignore mapjoin hint

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585006#comment-13585006
 ] 

Namit Jain commented on HIVE-4042:
--

I agree - that is fairly difficult to find out at the semantic analysis time 
before any pruning.
The hint is to help deployments where there are a lot of queries running with 
mapjoin hint, and so they will fail.

I can add logging while the hint is being ignored, it leaves the burden on the 
client to change the query.

> ignore mapjoin hint
> ---
>
> Key: HIVE-4042
> URL: https://issues.apache.org/jira/browse/HIVE-4042
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.4042.1.patch, hive.4042.2.patch
>
>
> After HIVE-3784, in a production environment, it can become difficult to
> deploy since a lot of production queries can break.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4056) Extend rcfilecat to support (un)compressed size and no. of row

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585007#comment-13585007
 ] 

Namit Jain commented on HIVE-4056:
--

+1

> Extend rcfilecat to support (un)compressed size and no. of row
> --
>
> Key: HIVE-4056
> URL: https://issues.apache.org/jira/browse/HIVE-4056
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-4056.patch.1
>
>
> rcfilecat supports data and metadata:
> https://cwiki.apache.org/Hive/rcfilecat.html
> In metadata, it supports column statistics.
> It will be natural to extend metadata support to 
> 1. no. of rows 
> 2. uncompressed size for the file
> 3. compressed size for the file

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4042) ignore mapjoin hint

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-4042:
-

Attachment: hive.4042.3.patch

> ignore mapjoin hint
> ---
>
> Key: HIVE-4042
> URL: https://issues.apache.org/jira/browse/HIVE-4042
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.4042.1.patch, hive.4042.2.patch, hive.4042.3.patch
>
>
> After HIVE-3784, in a production environment, it can become difficult to
> deploy since a lot of production queries can break.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-4042) ignore mapjoin hint

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4042?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585022#comment-13585022
 ] 

Namit Jain commented on HIVE-4042:
--

I have added logging in the above patch.
The assumption is that the client is monitoring these, and will change the 
queries over time.

> ignore mapjoin hint
> ---
>
> Key: HIVE-4042
> URL: https://issues.apache.org/jira/browse/HIVE-4042
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.4042.1.patch, hive.4042.2.patch, hive.4042.3.patch
>
>
> After HIVE-3784, in a production environment, it can become difficult to
> deploy since a lot of production queries can break.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

HIVE-4053 | Review request

[Krishna]

Hi,

I've implemented 'Refined Soundex' algorithm using a GenericUDF and would
like to share it for a review by experts as I'm a newbie.

Change Details:
A new java class is created: GenericUDFRefinedSoundex.java
Add a entry to FunctionRegistry.java: registerGenericUDF("soundex_ref",
GenericUDFRefinedSoundex.class);

Both files are attached to the email.

I'm planning to implement other phonetic algorithms and submit all as a
single patch. I understand there are many other steps that I need to finish
before a patch is ready but for now, if you could review the attached code
and provide feedback, it'll be great.

Here are the details of Refined Soundex algorithm:
First letter is stored
Subsequent letters are replaced by numbers as defined below-
 * B, P => 1
 * F, V => 2
 * C, K, S => 3
 * G, J => 4
 * Q, X, Z => 5
 * D, T => 6
 * L => 7
 * M, N => 8
 * R => 9
 * Other letters => 0
Consecutive letters belonging to the same group are replaced by one letter

Example:
> SELECT soundex_ref('Carren') FROM src LIMIT 1;
> C30908

Thanks,
Krishna

[jira] [Commented] (HIVE-4053) Add support for phonetic algorithms in Hive

[Krishna (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13585025#comment-13585025
 ] 

Krishna commented on HIVE-4053:
---

I've implemented 'Refined Soundex' algorithm using a GenericUDF and would like 
to share it for a review by experts as I'm a newbie.

Change Details:
A new java class is created: GenericUDFRefinedSoundex.java
Add a entry to FunctionRegistry.java: registerGenericUDF("soundex_ref", 
GenericUDFRefinedSoundex.class);

Both files are attached to the email.

I'm planning to implement other phonetic algorithms and submit all as a single 
patch. I understand there are many other steps that I need to finish before a 
patch is ready but for now, if you could review the attached code and provide 
feedback, it'll be great.

Here are the details of Refined Soundex algorithm:
First letter is stored
Subsequent letters are replaced by numbers as defined below-
 * B, P => 1
 * F, V => 2
 * C, K, S => 3
 * G, J => 4
 * Q, X, Z => 5
 * D, T => 6
 * L => 7
 * M, N => 8
 * R => 9
 * Other letters => 0
Consecutive letters belonging to the same group are replaced by one letter

Example: 
> SELECT soundex_ref('Carren') FROM src LIMIT 1;
> C30908

> Add support for phonetic algorithms in Hive
> ---
>
> Key: HIVE-4053
> URL: https://issues.apache.org/jira/browse/HIVE-4053
> Project: Hive
>  Issue Type: New Feature
>  Components: UDF
>Reporter: Krishna
> Attachments: FunctionRegistry.java, GenericUDFRefinedSoundex.java
>
>
> Following phonetic algorithms should be considered, which are very useful in 
> search:
> Soundex
> Refined Soundex
> Daitch–Mokotoff Soundex
> Metaphone and Double Metaphone
> New York State Identification and Intelligence System (NYSIIS)
> Caverphone

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4053) Add support for phonetic algorithms in Hive

[Krishna (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4053?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Krishna updated HIVE-4053:
--

Attachment: GenericUDFRefinedSoundex.java
FunctionRegistry.java

> Add support for phonetic algorithms in Hive
> ---
>
> Key: HIVE-4053
> URL: https://issues.apache.org/jira/browse/HIVE-4053
> Project: Hive
>  Issue Type: New Feature
>  Components: UDF
>Reporter: Krishna
> Attachments: FunctionRegistry.java, GenericUDFRefinedSoundex.java
>
>
> Following phonetic algorithms should be considered, which are very useful in 
> search:
> Soundex
> Refined Soundex
> Daitch–Mokotoff Soundex
> Metaphone and Double Metaphone
> New York State Identification and Intelligence System (NYSIIS)
> Caverphone

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-4035) Column Pruner for PTF Op

[Prajakta Kalmegh (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-4035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prajakta Kalmegh updated HIVE-4035:
---

Attachment: HIVE-4035.2.patch.txt

Ashutosh, thanks for the review. We have made the changes as per your comments 
in this patch. 

> Column Pruner for PTF Op
> 
>
> Key: HIVE-4035
> URL: https://issues.apache.org/jira/browse/HIVE-4035
> Project: Hive
>  Issue Type: Bug
>  Components: PTF-Windowing
>Reporter: Harish Butani
>Assignee: Prajakta Kalmegh
> Attachments: HIVE-4035.1.patch.txt, HIVE-4035.2.patch.txt
>
>
> for a PTFOp for Windowing; should prune columns based on its children. 
> Virtual Columns should only be carried forward if needed. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira