[jira] [Updated] (HIVE-3826) Rollbacks and retries of drops cause org.datanucleus.exceptions.NucleusObjectNotFoundException: No such database row)

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3826:
-

  Resolution: Fixed
Hadoop Flags: Reviewed
  Status: Resolved  (was: Patch Available)

Committed. Thanks Kevin

> Rollbacks and retries of drops cause 
> org.datanucleus.exceptions.NucleusObjectNotFoundException: No such database 
> row)
> -
>
> Key: HIVE-3826
> URL: https://issues.apache.org/jira/browse/HIVE-3826
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.11
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-3826.1.patch.txt
>
>
> I'm not sure if this is the only cause of the exception 
> "org.datanucleus.exceptions.NucleusObjectNotFoundException: No such database 
> row)" from the metastore, but one cause seems to be related to a drop command 
> failing, and being retried by the client.
> Based on focusing on a single thread in the metastore with DEBUG level 
> logging, I was seeing the objects that were intended to be dropped remaining 
> in the PersistenceManager cache even after a rollback.  The steps seemed to 
> be as follows:
> 1) First attempt to drop the table, the table is pulled into the 
> PersistenceManager cache for the purposes of dropping
> 2) The drop fails, e.g. due to a lock wait timeout on the SQL backend, this 
> causes a rollback of the transaction
> 3) The drop is retried using a different thread on the metastore Thrift 
> server or a different server and succeeds
> 4) Back on the original thread of the original Thrift server someone tries to 
> perform some write operation which produces a commit.  This causes those 
> detached objects related to the dropped table to attempt to reattach, causing 
> JDO to query the SQL backend for those objects which it can't find.  This 
> causes the exception.
> I was able to reproduce this regularly using the following sequence of 
> commands:
> Hive client 1 (Hive1): connected to a metastore Thrift server running a 
> single thread, I hard coded a RuntimeException into the code to drop a table 
> in the ObjectStore, specifically right before the commit in 
> preDropStorageDescriptor, to induce a rollback.  I also turned off all 
> retries at all layers of the metastore.
> Hive client 2 (Hive2): connected to a separate metastore Thrift server 
> running with standard configs and code
> 1: On Hive1, CREATE TABLE t1 (c STRING);
> 2: On Hive1, DROP TABLE t1; // This failed due to the hard coded exception
> 3: On Hive2, DROP TABLE t1; // Succeeds
> 4: On Hive1, CREATE DATABASE d1; // This database already existed, I'm not 
> sure why this was necessary, but it didn't work without it, it seemed to have 
> an affect on the order objects were committed in the next step
> 5: On Hive1, CREATE DATABASE d2; // This database didn't exist, it would fail 
> with the NucleusObjectNotFoundException
> The object that would cause the exception varied, I saw the MTable, the 
> MSerDeInfo, and MTablePrivilege from the table that attempted to be dropped.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3552) HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3552:
-

Attachment: hive.3552.11.patch

> HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a 
> high number of grouping set keys
> -
>
> Key: HIVE-3552
> URL: https://issues.apache.org/jira/browse/HIVE-3552
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3552.10.patch, hive.3552.11.patch, 
> hive.3552.1.patch, hive.3552.2.patch, hive.3552.3.patch, hive.3552.4.patch, 
> hive.3552.5.patch, hive.3552.6.patch, hive.3552.7.patch, hive.3552.8.patch, 
> hive.3552.9.patch
>
>
> This is a follow up for HIVE-3433.
> Had a offline discussion with Sambavi - she pointed out a scenario where the
> implementation in HIVE-3433 will not scale. Assume that the user is performing
> a cube on many columns, say '8' columns. So, each row would generate 256 rows
> for the hash table, which may kill the current group by implementation.
> A better implementation would be to add an additional mr job - in the first 
> mr job perform the group by assuming there was no cube. Add another mr job, 
> where
> you would perform the cube. The assumption is that the group by would have 
> decreased the output data significantly, and the rows would appear in the 
> order of
> grouping keys which has a higher probability of hitting the hash table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3552) HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a high number of grouping set keys

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3552:
-

Status: Patch Available  (was: Open)

> HIVE-3552 performant manner for performing cubes/rollups/grouping sets for a 
> high number of grouping set keys
> -
>
> Key: HIVE-3552
> URL: https://issues.apache.org/jira/browse/HIVE-3552
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3552.10.patch, hive.3552.11.patch, 
> hive.3552.1.patch, hive.3552.2.patch, hive.3552.3.patch, hive.3552.4.patch, 
> hive.3552.5.patch, hive.3552.6.patch, hive.3552.7.patch, hive.3552.8.patch, 
> hive.3552.9.patch
>
>
> This is a follow up for HIVE-3433.
> Had a offline discussion with Sambavi - she pointed out a scenario where the
> implementation in HIVE-3433 will not scale. Assume that the user is performing
> a cube on many columns, say '8' columns. So, each row would generate 256 rows
> for the hash table, which may kill the current group by implementation.
> A better implementation would be to add an additional mr job - in the first 
> mr job perform the group by assuming there was no cube. Add another mr job, 
> where
> you would perform the cube. The assumption is that the group by would have 
> decreased the output data significantly, and the rows would appear in the 
> order of
> grouping keys which has a higher probability of hitting the hash table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3803) explain dependency should show the dependencies hierarchically in presence of views

[Namit Jain (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Namit Jain updated HIVE-3803:
-

Status: Patch Available  (was: Open)

Comments addressed.

Kevin, can you take a look ? This is a code only patch.
It this looks good, I will file a patch with the log file updates.

> explain dependency should show the dependencies hierarchically in presence of 
> views
> ---
>
> Key: HIVE-3803
> URL: https://issues.apache.org/jira/browse/HIVE-3803
> Project: Hive
>  Issue Type: Bug
>  Components: Query Processor
>Reporter: Namit Jain
>Assignee: Namit Jain
> Attachments: hive.3803.1.patch, hive.3803.2.patch, hive.3803.3.patch, 
> hive.3803.4.patch, hive.3803.5.patch
>
>
> It should also include tables whose partitions are being accessed

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3829) Hive CLI needs UNSET TBLPROPERTY command

[Zhenxiao Luo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538670#comment-13538670
 ] 

Zhenxiao Luo commented on HIVE-3829:


[~kevinwilfong] and [~mgrover]: Thanks a lot for the review. My patch is 
updated for review:
https://reviews.facebook.net/D7593

> Hive CLI needs UNSET TBLPROPERTY command
> 
>
> Key: HIVE-3829
> URL: https://issues.apache.org/jira/browse/HIVE-3829
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3829.1.patch.txt, HIVE-3829.2.patch.txt
>
>
> The Hive CLI currently supports
> ALTER TABLE  SET TBLPROPERTIES ('key1' = 'value1', 'key2' = 'value2', 
> ...);
> To add/change the value of table properties.
> It would be really useful if Hive also supported
> ALTER TABLE  UNSET TBLPROPERTIES ('key1', 'key2', ...);
> Which would remove table properties

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3829) Hive CLI needs UNSET TBLPROPERTY command

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-3829:
---

Status: Patch Available  (was: Open)

> Hive CLI needs UNSET TBLPROPERTY command
> 
>
> Key: HIVE-3829
> URL: https://issues.apache.org/jira/browse/HIVE-3829
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3829.1.patch.txt, HIVE-3829.2.patch.txt
>
>
> The Hive CLI currently supports
> ALTER TABLE  SET TBLPROPERTIES ('key1' = 'value1', 'key2' = 'value2', 
> ...);
> To add/change the value of table properties.
> It would be really useful if Hive also supported
> ALTER TABLE  UNSET TBLPROPERTIES ('key1', 'key2', ...);
> Which would remove table properties

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3829) Hive CLI needs UNSET TBLPROPERTY command

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-3829:
---

Status: Open  (was: Patch Available)

> Hive CLI needs UNSET TBLPROPERTY command
> 
>
> Key: HIVE-3829
> URL: https://issues.apache.org/jira/browse/HIVE-3829
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3829.1.patch.txt, HIVE-3829.2.patch.txt
>
>
> The Hive CLI currently supports
> ALTER TABLE  SET TBLPROPERTIES ('key1' = 'value1', 'key2' = 'value2', 
> ...);
> To add/change the value of table properties.
> It would be really useful if Hive also supported
> ALTER TABLE  UNSET TBLPROPERTIES ('key1', 'key2', ...);
> Which would remove table properties

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3829) Hive CLI needs UNSET TBLPROPERTY command

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-3829:
---

Attachment: HIVE-3829.2.patch.txt

> Hive CLI needs UNSET TBLPROPERTY command
> 
>
> Key: HIVE-3829
> URL: https://issues.apache.org/jira/browse/HIVE-3829
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3829.1.patch.txt, HIVE-3829.2.patch.txt
>
>
> The Hive CLI currently supports
> ALTER TABLE  SET TBLPROPERTIES ('key1' = 'value1', 'key2' = 'value2', 
> ...);
> To add/change the value of table properties.
> It would be really useful if Hive also supported
> ALTER TABLE  UNSET TBLPROPERTIES ('key1', 'key2', ...);
> Which would remove table properties

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3831) Add Command to Turn Sorting Off for a Bucketed Table

[Mark Grover (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538666#comment-13538666
 ] 

Mark Grover commented on HIVE-3831:
---

For #1, I was referring to queries that populate the table.

> Add Command to Turn Sorting Off for a Bucketed Table
> 
>
> Key: HIVE-3831
> URL: https://issues.apache.org/jira/browse/HIVE-3831
> Project: Hive
>  Issue Type: Bug
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
>Priority: Minor
>
> If we have specified a bucketed table as sorted on some columns, there is no 
> Hive command to turn the sorting off for that table. There are scenarios 
> where we need to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [jira] [Commented] (HIVE-3831) Add Command to Turn Sorting Off for a Bucketed Table

[Edward Capriolo]

Hive itself does not actually use the sorting to optimize any queries other
then queries that can be optimized by index ( select min do not short
circuit. They are still a full table scan)

On Fri, Dec 21, 2012 at 11:23 PM, Mark Grover (JIRA) wrote:

>
> [
> https://issues.apache.org/jira/browse/HIVE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538663#comment-13538663]
>
> Mark Grover commented on HIVE-3831:
> ---
>
> Bhushan, I am not sure I understand.
> 2 things:
> 1. If you wanted to disable sorting, can't you just set
> {{hive.enforce.sorting}} to false before your queries.
> 2. If you disable sorting, the metadata of the table would still reflect
> that the table is sorted but the actual data would not be sorted. I can't
> think of a scenario where that would be desirable.
>
> Thoughts?
>
> > Add Command to Turn Sorting Off for a Bucketed Table
> > 
> >
> > Key: HIVE-3831
> > URL: https://issues.apache.org/jira/browse/HIVE-3831
> > Project: Hive
> >  Issue Type: Bug
> >Reporter: Bhushan Mandhani
> >Assignee: Bhushan Mandhani
> >Priority: Minor
> >
> > If we have specified a bucketed table as sorted on some columns, there
> is no Hive command to turn the sorting off for that table. There are
> scenarios where we need to do this.
>
> --
> This message is automatically generated by JIRA.
> If you think it was sent incorrectly, please contact your JIRA
> administrators
> For more information on JIRA, see: http://www.atlassian.com/software/jira
>

[jira] [Commented] (HIVE-3831) Add Command to Turn Sorting Off for a Bucketed Table

[Mark Grover (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538663#comment-13538663
 ] 

Mark Grover commented on HIVE-3831:
---

Bhushan, I am not sure I understand.
2 things:
1. If you wanted to disable sorting, can't you just set 
{{hive.enforce.sorting}} to false before your queries.
2. If you disable sorting, the metadata of the table would still reflect that 
the table is sorted but the actual data would not be sorted. I can't think of a 
scenario where that would be desirable.

Thoughts?

> Add Command to Turn Sorting Off for a Bucketed Table
> 
>
> Key: HIVE-3831
> URL: https://issues.apache.org/jira/browse/HIVE-3831
> Project: Hive
>  Issue Type: Bug
>Reporter: Bhushan Mandhani
>Assignee: Bhushan Mandhani
>Priority: Minor
>
> If we have specified a bucketed table as sorted on some columns, there is no 
> Hive command to turn the sorting off for that table. There are scenarios 
> where we need to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3826) Rollbacks and retries of drops cause org.datanucleus.exceptions.NucleusObjectNotFoundException: No such database row)

[Namit Jain (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538660#comment-13538660
 ] 

Namit Jain commented on HIVE-3826:
--

+1

Great catch - running tests.

> Rollbacks and retries of drops cause 
> org.datanucleus.exceptions.NucleusObjectNotFoundException: No such database 
> row)
> -
>
> Key: HIVE-3826
> URL: https://issues.apache.org/jira/browse/HIVE-3826
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore
>Affects Versions: 0.11
>Reporter: Kevin Wilfong
>Assignee: Kevin Wilfong
> Attachments: HIVE-3826.1.patch.txt
>
>
> I'm not sure if this is the only cause of the exception 
> "org.datanucleus.exceptions.NucleusObjectNotFoundException: No such database 
> row)" from the metastore, but one cause seems to be related to a drop command 
> failing, and being retried by the client.
> Based on focusing on a single thread in the metastore with DEBUG level 
> logging, I was seeing the objects that were intended to be dropped remaining 
> in the PersistenceManager cache even after a rollback.  The steps seemed to 
> be as follows:
> 1) First attempt to drop the table, the table is pulled into the 
> PersistenceManager cache for the purposes of dropping
> 2) The drop fails, e.g. due to a lock wait timeout on the SQL backend, this 
> causes a rollback of the transaction
> 3) The drop is retried using a different thread on the metastore Thrift 
> server or a different server and succeeds
> 4) Back on the original thread of the original Thrift server someone tries to 
> perform some write operation which produces a commit.  This causes those 
> detached objects related to the dropped table to attempt to reattach, causing 
> JDO to query the SQL backend for those objects which it can't find.  This 
> causes the exception.
> I was able to reproduce this regularly using the following sequence of 
> commands:
> Hive client 1 (Hive1): connected to a metastore Thrift server running a 
> single thread, I hard coded a RuntimeException into the code to drop a table 
> in the ObjectStore, specifically right before the commit in 
> preDropStorageDescriptor, to induce a rollback.  I also turned off all 
> retries at all layers of the metastore.
> Hive client 2 (Hive2): connected to a separate metastore Thrift server 
> running with standard configs and code
> 1: On Hive1, CREATE TABLE t1 (c STRING);
> 2: On Hive1, DROP TABLE t1; // This failed due to the hard coded exception
> 3: On Hive2, DROP TABLE t1; // Succeeds
> 4: On Hive1, CREATE DATABASE d1; // This database already existed, I'm not 
> sure why this was necessary, but it didn't work without it, it seemed to have 
> an affect on the order objects were committed in the next step
> 5: On Hive1, CREATE DATABASE d2; // This database didn't exist, it would fail 
> with the NucleusObjectNotFoundException
> The object that would cause the exception varied, I saw the MTable, the 
> MSerDeInfo, and MTablePrivilege from the table that attempted to be dropped.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3831) Add Command to Turn Sorting Off for a Bucketed Table

[Bhushan Mandhani (JIRA)]

Bhushan Mandhani created HIVE-3831:
--

 Summary: Add Command to Turn Sorting Off for a Bucketed Table
 Key: HIVE-3831
 URL: https://issues.apache.org/jira/browse/HIVE-3831
 Project: Hive
  Issue Type: Bug
Reporter: Bhushan Mandhani
Assignee: Bhushan Mandhani
Priority: Minor


If we have specified a bucketed table as sorted on some columns, there is no 
Hive command to turn the sorting off for that table. There are scenarios where 
we need to do this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3829) Hive CLI needs UNSET TBLPROPERTY command

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-3829:
---

Attachment: HIVE-3829.1.patch.txt

> Hive CLI needs UNSET TBLPROPERTY command
> 
>
> Key: HIVE-3829
> URL: https://issues.apache.org/jira/browse/HIVE-3829
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3829.1.patch.txt
>
>
> The Hive CLI currently supports
> ALTER TABLE  SET TBLPROPERTIES ('key1' = 'value1', 'key2' = 'value2', 
> ...);
> To add/change the value of table properties.
> It would be really useful if Hive also supported
> ALTER TABLE  UNSET TBLPROPERTIES ('key1', 'key2', ...);
> Which would remove table properties

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3829) Hive CLI needs UNSET TBLPROPERTY command

[Zhenxiao Luo (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538611#comment-13538611
 ] 

Zhenxiao Luo commented on HIVE-3829:


review request submitted at:
https://reviews.facebook.net/D7593

> Hive CLI needs UNSET TBLPROPERTY command
> 
>
> Key: HIVE-3829
> URL: https://issues.apache.org/jira/browse/HIVE-3829
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3829.1.patch.txt
>
>
> The Hive CLI currently supports
> ALTER TABLE  SET TBLPROPERTIES ('key1' = 'value1', 'key2' = 'value2', 
> ...);
> To add/change the value of table properties.
> It would be really useful if Hive also supported
> ALTER TABLE  UNSET TBLPROPERTIES ('key1', 'key2', ...);
> Which would remove table properties

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-3829) Hive CLI needs UNSET TBLPROPERTY command

[Zhenxiao Luo (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-3829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhenxiao Luo updated HIVE-3829:
---

Status: Patch Available  (was: Open)

> Hive CLI needs UNSET TBLPROPERTY command
> 
>
> Key: HIVE-3829
> URL: https://issues.apache.org/jira/browse/HIVE-3829
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhenxiao Luo
>Assignee: Zhenxiao Luo
> Attachments: HIVE-3829.1.patch.txt
>
>
> The Hive CLI currently supports
> ALTER TABLE  SET TBLPROPERTIES ('key1' = 'value1', 'key2' = 'value2', 
> ...);
> To add/change the value of table properties.
> It would be really useful if Hive also supported
> ALTER TABLE  UNSET TBLPROPERTIES ('key1', 'key2', ...);
> Which would remove table properties

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-446) Implement TRUNCATE

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538608#comment-13538608
 ] 

Phabricator commented on HIVE-446:
--

mgrover has commented on the revision "HIVE-446 [jira] Implement TRUNCATE".

INLINE COMMENTS
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:3912 -rmr is 
deprecated. I think the new hip thing to do is "rm -r". Keep in mind though 
that if hive is being used with really old versions of hadoop, rm -r may not 
work, so some version matching needs to happen here.

  If we leave it the way it is, then users will see a deprecation warning. I 
would prefer that not to be the case (btw, that's presently the case with 
insert overwrite but HIVE-3701 plans to change that)
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:727 Ok, 
we should add a test that verifies that something like this works:
  hive>use default;
  hive>truncate some_other_db.my_table;
  ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java:249 I am a little 
confused here. The code seems to indicate that this only works managed, native 
tables. For example, you can't have a HBase table (using HBase storage handler) 
being truncated with this command, can you?
  If so, should we have make error messages for those too?
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:749 
Check for non-native tables here?
  ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java:3913 Touché! Since we 
preserve the metadata, we are preserving it for partitions as well. So, it 
makes sense to create empty directories for partitions that the metastore 
thinks exist.
  ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java:755 This 
could be a bottleneck if the number of partitions is large. Can't it? Is there 
an alternative to adding each of the partitions individually? Can we still not 
do
  outputs.add(new WriteEntity(table));

  ?

REVISION DETAIL
  https://reviews.facebook.net/D7371

To: JIRA, navis
Cc: njain, mgrover


> Implement TRUNCATE
> --
>
> Key: HIVE-446
> URL: https://issues.apache.org/jira/browse/HIVE-446
> Project: Hive
>  Issue Type: New Feature
>  Components: Query Processor
>Reporter: Prasad Chakka
>Assignee: Navis
> Attachments: HIVE-446.D7371.1.patch, HIVE-446.D7371.2.patch
>
>
> truncate the data but leave the table and metadata intact.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3439) PARTITIONED BY clause in CREATE TABLE is order-dependent

[Istvan Szegedi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538548#comment-13538548
 ] 

Istvan Szegedi commented on HIVE-3439:
--

This is determined by Hive.g grammar file used for ANTLR parser - 
./hive-trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g:

createTableStatement
@init { msgs.push("create table statement"); }
@after { msgs.pop(); }
: KW_CREATE (ext=KW_EXTERNAL)? KW_TABLE ifNotExists? name=tableName
  (  like=KW_LIKE likeName=tableName
 tableLocation?
   | (LPAREN columnNameTypeList RPAREN)?
 tableComment?
 tablePartition?
 tableBuckets?
 tableSkewed?
 tableRowFormat?
 tableFileFormat?
 tableLocation?
 tablePropertiesPrefixed?
 (KW_AS selectStatement)?
  )
-> ^(TOK_CREATETABLE $name $ext? ifNotExists?
 ^(TOK_LIKETABLE $likeName?)
 columnNameTypeList?
 tableComment?
 tablePartition?
 tableBuckets?
 tableSkewed?
 tableRowFormat?
 tableFileFormat?
 tableLocation?
 tablePropertiesPrefixed?
 selectStatement?
)
;


The grammar determines the order for create table statement. And this is 
in-line with the DDL create table documentation:

https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTable


In theory, it could be changed to something like:

( tablePartition? tableLocation?  | tableLocation? tablePartition?) 

but it does not really seem to be a bug.

> PARTITIONED BY clause in CREATE TABLE is order-dependent
> 
>
> Key: HIVE-3439
> URL: https://issues.apache.org/jira/browse/HIVE-3439
> Project: Hive
>  Issue Type: Bug
>Reporter: Jonathan Natkins
>
> hive> create external table foo (a int) location '/user/natty/foo' 
> partitioned by (b int);
> FAILED: Parse Error: line 1:61 mismatched input 'partitioned' expecting EOF 
> near ''/user/natty/foo''
> hive> create external table foo (a int) partitioned by (b int) location 
> '/user/natty/foo';
> OK
> Time taken: 0.051 seconds

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hive-0.10.0-SNAPSHOT-h0.20.1 #10

[Apache Jenkins Server]

See 

--
[...truncated 42298 lines...]
[junit] Hadoop job information for null: number of mappers: 0; number of 
reducers: 0
[junit] 2012-12-21 15:30:30,912 null map = 100%,  reduce = 100%
[junit] Ended Job = job_local_0001
[junit] Execution completed successfully
[junit] Mapred Local Task Succeeded . Convert the Join into MapJoin
[junit] POSTHOOK: query: select count(1) as cnt from testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 

[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] Copying file: 

[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: load data local inpath 
'
 into table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] Copying data from 

[junit] Loading data to table default.testhivedrivertable
[junit] Table default.testhivedrivertable stats: [num_partitions: 0, 
num_files: 1, num_rows: 0, total_size: 5812, raw_data_size: 0]
[junit] POSTHOOK: query: load data local inpath 
'
 into table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: select * from testhivedrivertable limit 10
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: 

[junit] POSTHOOK: query: select * from testhivedrivertable limit 10
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: 

[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedrivertable
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Input: default@testhivedrivertable
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] Hive history 
file=
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: drop table testhivedrivertable
[junit] POSTHOOK: type: DROPTABLE
[junit] OK
[junit] PREHOOK: query: create table testhivedrivertable (num int)
[junit] PREHOOK: type: DROPTABLE
[junit] POSTHOOK: query: create table testhivedrivertable (num int)
[junit] POSTHOOK: type: DROPTABLE
[junit] POSTHOOK: Output: default@testhivedrivertable
[junit] OK
[junit] PREHOOK: query: drop table testhivedrivertable
[junit] PREHOOK: type: DROPTABLE
[junit] PREHOOK: Input: default@testhivedrivertable
[junit] PREHOOK: Output: default@testhivedriverta

Re: [VOTE] Apache Hive 0.10.0 Release Candidate 0

[Carl Steinbach]

Thanks for pointing this out. I filed HIVE-3830 to track this issue.

On Fri, Dec 21, 2012 at 2:25 PM, Alan Gates  wrote:

> +1 (non-binding)
> Checked the check sums and key signatures.  Installed it and ran a few
> queries.  All looked good.
>
> As a note Hive should be offering a src only release and a convenience
> binary rather than two binaries, one with the source and one without.  See
> the thread on general@incubator discussing this:
> http://mail-archives.apache.org/mod_mbox/incubator-general/201203.mbox/%3CCAOFYJNY%3DEjVHrWVvAedR3OKwCv-BkTaCbEu0ufp7OZR_gpCTiA%40mail.gmail.com%3E
>  I think this can be solved later and need not block this release.
>
> Alan.
>
> On Dec 18, 2012, at 10:23 PM, Ashutosh Chauhan wrote:
>
> > Apache Hive 0.10.0 Release Candidate 0 is available here:
> > http://people.apache.org/~hashutosh/hive-0.10.0-rc0/
> >
> > Maven artifacts are available here:
> >
> https://repository.apache.org/content/repositories/orgapachehive-049/org/apache/hive/
> >
> >
> > Release notes are available at:
> >
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12320745&styleName=Text&projectId=12310843&Create=Create&atl_token=A5KQ-2QAV-T4JA-FDED%7C70f39c6dd3cf337eaa0e3a0359687cf608903879%7Clin
> >
> >
> > Voting will conclude in 72 hours.
> >
> > Hive PMC Members: Please test and vote.
> >
> > Thanks,
> >
> > Ashutosh
>
>

[jira] [Created] (HIVE-3830) Hive releases should include source only distribution

[Carl Steinbach (JIRA)]

Carl Steinbach created HIVE-3830:


 Summary: Hive releases should include source only distribution
 Key: HIVE-3830
 URL: https://issues.apache.org/jira/browse/HIVE-3830
 Project: Hive
  Issue Type: Bug
  Components: Build Infrastructure
Affects Versions: 0.10.0
Reporter: Carl Steinbach




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3830) Hive releases should include source only distribution

[Carl Steinbach (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538488#comment-13538488
 ] 

Carl Steinbach commented on HIVE-3830:
--

Alan mentioned this on the dev list:

{quote}
As a note Hive should be offering a src only release and a convenience binary 
rather than two binaries, one with the source and one without.  See the thread 
on general@incubator discussing this:  
http://mail-archives.apache.org/mod_mbox/incubator-general/201203.mbox/%3CCAOFYJNY%3DEjVHrWVvAedR3OKwCv-BkTaCbEu0ufp7OZR_gpCTiA%40mail.gmail.com%3E
  I think this can be solved later and need not block this release.
{quote}

> Hive releases should include source only distribution
> -
>
> Key: HIVE-3830
> URL: https://issues.apache.org/jira/browse/HIVE-3830
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.10.0
>Reporter: Carl Steinbach
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Jenkins build is back to normal : Hive-0.9.1-SNAPSHOT-h0.21 #236

[Apache Jenkins Server]

See 

Re: [VOTE] Apache Hive 0.10.0 Release Candidate 0

[Alan Gates]

+1 (non-binding)
Checked the check sums and key signatures.  Installed it and ran a few queries. 
 All looked good.

As a note Hive should be offering a src only release and a convenience binary 
rather than two binaries, one with the source and one without.  See the thread 
on general@incubator discussing this:  
http://mail-archives.apache.org/mod_mbox/incubator-general/201203.mbox/%3CCAOFYJNY%3DEjVHrWVvAedR3OKwCv-BkTaCbEu0ufp7OZR_gpCTiA%40mail.gmail.com%3E
  I think this can be solved later and need not block this release.

Alan.

On Dec 18, 2012, at 10:23 PM, Ashutosh Chauhan wrote:

> Apache Hive 0.10.0 Release Candidate 0 is available here:
> http://people.apache.org/~hashutosh/hive-0.10.0-rc0/
> 
> Maven artifacts are available here:
> https://repository.apache.org/content/repositories/orgapachehive-049/org/apache/hive/
> 
> 
> Release notes are available at:
> https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12320745&styleName=Text&projectId=12310843&Create=Create&atl_token=A5KQ-2QAV-T4JA-FDED%7C70f39c6dd3cf337eaa0e3a0359687cf608903879%7Clin
> 
> 
> Voting will conclude in 72 hours.
> 
> Hive PMC Members: Please test and vote.
> 
> Thanks,
> 
> Ashutosh

Re: Review Request: float and double calculation is inaccurate in Hive

[Mark Grover]

> On Dec. 18, 2012, 12:38 a.m., Mark Grover wrote:
> > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFOPDivide.java,
> >  line 50
> > 
> >
> > 10 seems to be a rather arbitrary number for scale. Any particular 
> > reason you are using it? Maybe we should invoke the method where no scale 
> > needs to be specified.
> 
> Johnny Zhang wrote:
> Hi, Mark, thanks for reviewing it. The reason using 10 is because it is 
> the same as mysql default precision setting. Just want to make the 
> calculation result identical to mysql's
> 
> Johnny Zhang wrote:
> I think I did tried without specify scale, and the result is different 
> from mysql. I agree hard coding the scale is not a good way. Open to other 
> suggestions.

Fair enough. Thanks


- Mark


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8653/#review14625
---


On Dec. 18, 2012, 12:37 a.m., Johnny Zhang wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8653/
> ---
> 
> (Updated Dec. 18, 2012, 12:37 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Description
> ---
> 
> I found this during debug the e2e test failures. I found Hive miss calculate 
> the float and double value. Take float calculation as an example:
> hive> select f from all100k limit 1;
> 48308.98
> hive> select f/10 from all100k limit 1;
> 4830.898046875 <--added 04875 in the end
> hive> select f*1.01 from all100k limit 1;
> 48792.0702734375 <--should be 48792.0698
> It might be essentially the same problem as 
> http://effbot.org/pyfaq/why-are-floating-point-calculations-so-inaccurate.htm 
> But since e2e test compare the results with mysql and seems mysql does it 
> right, so it is worthy fixing it in Hive.
> 
> 
> This addresses bug HIVE-3715.
> https://issues.apache.org/jira/browse/HIVE-3715
> 
> 
> Diffs
> -
> 
>   
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFOPDivide.java
>  1423224 
>   
> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFOPMultiply.java
>  1423224 
> 
> Diff: https://reviews.apache.org/r/8653/diff/
> 
> 
> Testing
> ---
> 
> I did test to compare the result with mysql default float precision setting, 
> the result is identical.
> 
> query:  select f, f*1.01, f/10 from all100k limit 1;
> mysql result:   48309   48792.07027343754830.898046875
> hive result:48308.9848792.0702734375  4830.898046875
> 
> 
> I apply this patch and run the hive e2e test, and the tests all pass (without 
> this patch, 5 related failures)
> 
> 
> Thanks,
> 
> Johnny Zhang
> 
>

Re: Review Request: float and double calculation is inaccurate in Hive

[Mark Grover]

Bharath,
I am interesting in hearing more as well. Could you please comment on
https://issues.apache.org/jira/browse/HIVE-2693
?

Thanks in advance!

On Tue, Dec 18, 2012 at 12:51 AM, Bharath Mundlapudi
 wrote:
> We have solved this issue recently. It is not just a problem in Hive.
> Contact me offline if you need more details.
>
> -Bharath
>
> 
> From: Johnny Zhang 
> To: Johnny Zhang ; Mark Grover
> ; hive 
> Sent: Monday, December 17, 2012 5:13 PM
> Subject: Re: Review Request: float and double calculation is inaccurate in
> Hive
>
>
>
>> On Dec. 18, 2012, 12:38 a.m., Mark Grover wrote:
>> >
>> > http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFOPDivide.java,
>> > line 50
>> > 
>
>> >
>> >10 seems to be a rather arbitrary number for scale. Any particular
>> > reason you are using it? Maybe we should invoke the method where no scale
>> > needs to be specified.
>>
>> Johnny Zhang wrote:
>>Hi, Mark, thanks for reviewing it. The reason using 10 is because it is
>> the same as mysql default precision setting. Just want to make the
>> calculation result identical to mysql's
>
> I think I did tried without specify scale, and the result is different from
> mysql. I agree hard coding the scale is not a good way. Open to other
> suggestions.
>
>
> - Johnny
>
>
> ---
>
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8653/#review14625
> ---
>
>
> On Dec. 18, 2012, 12:37 a.m., Johnny Zhang wrote:
>>
>> ---
>
>> This is an automatically generated e-mail. To reply, visit:
>> https://reviews.apache.org/r/8653/
>> ---
>>
>> (Updated Dec. 18, 2012, 12:37 a.m.)
>
>>
>>
>> Review request for hive.
>>
>>
>> Description
>> ---
>>
>> I found this during debug the e2e test failures. I found Hive miss
>> calculate the float and double value. Take float calculation as an example:
>> hive> select f from all100k limit 1;
>> 48308.98
>> hive> select f/10 from all100k limit 1;
>> 4830.898046875 <--added 04875 in the end
>> hive> select f*1.01 from all100k limit 1;
>> 48792.0702734375 <--should be 48792.0698
>> It might be essentially the same problem as
>> http://effbot.org/pyfaq/why-are-floating-point-calculations-so-inaccurate.htm
>> But since e2e test compare the results with mysql and seems mysql does it
>> right, so it is worthy fixing it in Hive.
>>
>>
>> This addresses bug HIVE-3715.
>>https://issues.apache.org/jira/browse/HIVE-3715
>>
>>
>> Diffs
>> -
>>
>>
>> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFOPDivide.java
>> 1423224
>>
>> http://svn.apache.org/repos/asf/hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/udf/UDFOPMultiply.java
>> 1423224
>>
>> Diff: https://reviews.apache.org/r/8653/diff/
>>
>>
>> Testing
>> ---
>>
>> I did test to compare the result with mysql default float precision
>> setting, the result is identical.
>>
>> query:  select f, f*1.01, f/10 from all100k limit 1;
>> mysql result:  48309  48792.07027343754830.898046875
>> hive result:48308.9848792.07027343754830.898046875
>>
>>
>> I apply this patch and run the hive e2e test, and the tests all pass
>> (without this patch, 5 related failures)
>>
>>
>> Thanks,
>>
>> Johnny Zhang
>>
>>
>
>
>

[jira] [Commented] (HIVE-3828) insert overwrite fails with stored-as-dir in cluster

[Kevin Wilfong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538357#comment-13538357
 ] 

Kevin Wilfong commented on HIVE-3828:
-

+1

> insert overwrite fails with stored-as-dir in cluster
> 
>
> Key: HIVE-3828
> URL: https://issues.apache.org/jira/browse/HIVE-3828
> Project: Hive
>  Issue Type: Bug
>  Components: Import/Export
>Reporter: Gang Tim Liu
>Assignee: Gang Tim Liu
> Attachments: HIVE-3828.patch.1
>
>
> The following query works fine in hive TestCliDriver test suite but not 
> minimr because different Hadoop file system is used.
> The error is
> {code}
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Unable to rename 
> output from: .../_task_tmp.-ext-10002/key=103/_tmp.00_0 to: 
> .../_tmp.-ext-10002/key=103/00_0
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (HIVE-3829) Hive CLI needs UNSET TBLPROPERTY command

[Zhenxiao Luo (JIRA)]

Zhenxiao Luo created HIVE-3829:
--

 Summary: Hive CLI needs UNSET TBLPROPERTY command
 Key: HIVE-3829
 URL: https://issues.apache.org/jira/browse/HIVE-3829
 Project: Hive
  Issue Type: Bug
Reporter: Zhenxiao Luo
Assignee: Zhenxiao Luo


The Hive CLI currently supports

ALTER TABLE  SET TBLPROPERTIES ('key1' = 'value1', 'key2' = 'value2', 
...);

To add/change the value of table properties.

It would be really useful if Hive also supported
ALTER TABLE  UNSET TBLPROPERTIES ('key1', 'key2', ...);

Which would remove table properties

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (HIVE-3772) Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by Reynold Xin)

[Phabricator (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-3772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13538354#comment-13538354
 ] 

Phabricator commented on HIVE-3772:
---

zshao has requested changes to the revision "[jira] [HIVE-3772] Fix a 
concurrency bug in LazyBinaryUtils due to a static field (patch by Reynold 
Xin)".

  Please take this opportunity to fix all static variables in this file.

  By the way, a better way to fix is to use ThreadLocal.  That's more 
effiicient because then we don't need to call new in these low-level functions.


INLINE COMMENTS
  
serde/src/java/org/apache/hadoop/hive/serde2/lazybinary/LazyBinaryUtils.java:126
 Is this a problem for multi-threaded access as well?

REVISION DETAIL
  https://reviews.facebook.net/D7155

BRANCH
  HIVE-3772-static-vLongBytes

To: ashutoshc, njain, raghotham, JIRA, zshao, heyongqiang, nzhang, jsichi, 
pauly, amareshwarisr, cwsteinbach, mbautin


> Fix a concurrency bug in LazyBinaryUtils due to a static field (patch by 
> Reynold Xin)
> -
>
> Key: HIVE-3772
> URL: https://issues.apache.org/jira/browse/HIVE-3772
> Project: Hive
>  Issue Type: Bug
>Reporter: Mikhail Bautin
> Attachments: D7155.1.patch, HIVE-3772-2012-12-04.patch
>
>
> Creating a JIRA for [~rxin]'s patch needed by the Shark project. 
> https://github.com/amplab/hive/commit/17e1c3dd2f6d8eca767115dc46d5a880aed8c765
> writeVLong should not use a static field due to concurrency concerns.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Build failed in Jenkins: Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false #236

[Apache Jenkins Server]

See 


--
[...truncated 9916 lines...]

compile-test:
 [echo] Project: serde
[javac] Compiling 26 source files to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/serde/test/classes
[javac] Note: Some input files use or override a deprecated API.
[javac] Note: Recompile with -Xlint:deprecation for details.
[javac] Note: Some input files use unchecked or unsafe operations.
[javac] Note: Recompile with -Xlint:unchecked for details.

create-dirs:
 [echo] Project: service
 [copy] Warning: 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/service/src/test/resources
 does not exist.

init:
 [echo] Project: service

ivy-init-settings:
 [echo] Project: service

ivy-resolve:
 [echo] Project: service
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml
[ivy:report] Processing 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-service-default.xml
 to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-service-default.html

ivy-retrieve:
 [echo] Project: service

compile:
 [echo] Project: service

ivy-resolve-test:
 [echo] Project: service

ivy-retrieve-test:
 [echo] Project: service

compile-test:
 [echo] Project: service
[javac] Compiling 2 source files to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/service/test/classes

test:
 [echo] Project: hive

test-shims:
 [echo] Project: hive

test-conditions:
 [echo] Project: shims

gen-test:
 [echo] Project: shims

create-dirs:
 [echo] Project: shims
 [copy] Warning: 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/test/resources
 does not exist.

init:
 [echo] Project: shims

ivy-init-settings:
 [echo] Project: shims

ivy-resolve:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml
[ivy:report] Processing 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/resolution-cache/org.apache.hive-hive-shims-default.xml
 to 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/ivy/report/org.apache.hive-hive-shims-default.html

ivy-retrieve:
 [echo] Project: shims

compile:
 [echo] Project: shims
 [echo] Building shims 0.20

build_shims:
 [echo] Project: shims
 [echo] Compiling 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20/java
 against hadoop 0.20.2 
(/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hadoopcore/hadoop-0.20.2)

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml

ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.20S

build_shims:
 [echo] Project: shims
 [echo] Compiling 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common-secure/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.20S/java
 against hadoop 1.0.0 
(/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hadoopcore/hadoop-1.0.0)

ivy-init-settings:
 [echo] Project: shims

ivy-resolve-hadoop-shim:
 [echo] Project: shims
[ivy:resolve] :: loading settings :: file = 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/ivy/ivysettings.xml

ivy-retrieve-hadoop-shim:
 [echo] Project: shims
 [echo] Building shims 0.23

build_shims:
 [echo] Project: shims
 [echo] Compiling 
/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/common-secure/java;/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/shims/src/0.23/java
 against hadoop 0.23.3 
(/x1/jenkins/jenkins-slave/workspace/Hive-0.9.1-SNAPSHOT-h0.21-keepgoing=false/hive/build/hadoopcore/hadoop-0.23.3)

ivy-init-settings:
 [echo] Project: shim