[jira] [Updated] (HIVE-2721) ability to select a view qualified by the database / schema name

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2721?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2721:


Status: Patch Available  (was: Open)

 ability to select a view qualified by the database / schema name
 

 Key: HIVE-2721
 URL: https://issues.apache.org/jira/browse/HIVE-2721
 Project: Hive
  Issue Type: Bug
  Components: Database/Schema, Query Processor
Affects Versions: 0.8.0, 0.7.1, 0.7.0
Reporter: Robert Morton
Assignee: Martin Traverso
Priority: Blocker
 Attachments: HIVE-2721.patch


 HIVE-1517 added support for selecting tables from different databases (aka 
 schemas) by qualifying the tables with the database name. The feature work 
 did not however extend this support to views. Note that this point came up in 
 the earlier JIRA, but was not addressed. See the following two comments:
 https://issues.apache.org/jira/browse/HIVE-1517?focusedCommentId=12996641page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12996641
 https://issues.apache.org/jira/browse/HIVE-1517?focusedCommentId=12996679page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-12996679

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2961) Remove need for storage descriptors for view partitions

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2961:


Status: Patch Available  (was: Open)

 Remove need for storage descriptors for view partitions
 ---

 Key: HIVE-2961
 URL: https://issues.apache.org/jira/browse/HIVE-2961
 Project: Hive
  Issue Type: Improvement
Affects Versions: 0.9.0
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2961.D2877.1.patch


 Storage descriptors were introduced for view partitions as part of HIVE-2795. 
  This was to allow view partitions to have the concept of a region as well as 
 to fix a NPE that resulted from calling describe formatted on them.
 Since regions are no longer necessary for view partitions and the NPE can be 
 fixed by not displaying storage information for view partitions (or 
 displaying the view's storage information if this is preferred, although, 
 since a view partition is purely metadata, this does not seem necessary), 
 these are no longer needed.
 This also means the Python script added which retroactively adds storage 
 descriptors to existing view partitions can be removed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2959) TestRemoteHiveMetaStoreIpAddress always uses the same port

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2959:


Status: Patch Available  (was: Open)

 TestRemoteHiveMetaStoreIpAddress always uses the same port
 --

 Key: HIVE-2959
 URL: https://issues.apache.org/jira/browse/HIVE-2959
 Project: Hive
  Issue Type: Test
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2959.D2841.1.patch


 TestRemoteHiveMetaStoreIpAddress always uses the same port, meaning that if 
 another process happens to be using that port, the tests cannot succeed.
 There seems to be a standard way of finding a free port using Java's 
 ServerSocket class, this should be used instead.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2960) Stop testing concat of partitions containing control characters.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2960:


Attachment: escape2.q.out

 Stop testing concat of partitions containing control characters.
 

 Key: HIVE-2960
 URL: https://issues.apache.org/jira/browse/HIVE-2960
 Project: Hive
  Issue Type: Test
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2960.D2865.1.patch, escape2.q.out


 We have been, for a short while, testing to make sure that concatenation 
 commands work with partitions that contain ASCII control characters.  This 
 happened to work up until recently due to a happy coincidence in the way the 
 Hive object's HiveConf was updated.  Namely, it was updated often enough that 
 it got configs set by the user, but not so often that it got the value for 
 hive.query.string.  With some recent changes, it now needs to be updated more 
 often, see https://issues.apache.org/jira/browse/HIVE-2918
 This breaks the process of launching a job to merge partitions that contain 
 ASCII control characters.  The job conf is constructed using the updated Hive 
 conf containing the value of hive.query.string which contains ASCII control 
 characters.  When the job conf is converted to XML it fails because these 
 characters are illegal.  Given that any query has, even prior to this change, 
 failed when that query contained ASCII control characters, and hence these 
 partitions cannot be queried directly, it seems reasonable to no longer 
 support concatenating them either (which this change will allow for).

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2918) Hive Dynamic Partition Insert - move task not considering 'hive.exec.max.dynamic.partitions' from CLI

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2918:


Status: Open  (was: Patch Available)

 Hive Dynamic Partition Insert - move task not considering 
 'hive.exec.max.dynamic.partitions' from CLI
 -

 Key: HIVE-2918
 URL: https://issues.apache.org/jira/browse/HIVE-2918
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.8.1, 0.8.0, 0.7.1
 Environment: Cent OS 64 bit
Reporter: Bejoy KS
Assignee: Carl Steinbach
 Attachments: HIVE-2918.D2703.1.patch


 Dynamic Partition insert showing an error with the number of partitions 
 created even after the default value of 'hive.exec.max.dynamic.partitions' is 
 bumped high to 2000.
 Error Message:
 Failed with exception Number of dynamic partitions created is 1413, which is 
 more than 1000. To solve this try to set hive.exec.max.dynamic.partitions to 
 at least 1413.
 These are the following properties set on hive CLI
 hive set hive.exec.dynamic.partition=true;
 hive set hive.exec.dynamic.partition.mode=nonstrict;
 hive set hive.exec.max.dynamic.partitions=2000;
 hive set hive.exec.max.dynamic.partitions.pernode=2000;
 This is the query with console error log
 hive 
  INSERT OVERWRITE TABLE partn_dyn Partition (pobox)
  SELECT country,state,pobox FROM non_partn_dyn;
 Total MapReduce jobs = 2
 Launching Job 1 out of 2
 Number of reduce tasks is set to 0 since there's no reduce operator
 Starting Job = job_201204021529_0002, Tracking URL = 
 http://0.0.0.0:50030/jobdetails.jsp?jobid=job_201204021529_0002
 Kill Command = /usr/lib/hadoop/bin/hadoop job  
 -Dmapred.job.tracker=0.0.0.0:8021 -kill job_201204021529_0002
 2012-04-02 16:05:28,619 Stage-1 map = 0%,  reduce = 0%
 2012-04-02 16:05:39,701 Stage-1 map = 100%,  reduce = 0%
 2012-04-02 16:05:50,800 Stage-1 map = 100%,  reduce = 100%
 Ended Job = job_201204021529_0002
 Ended Job = 248865587, job is filtered out (removed at runtime).
 Moving data to: 
 hdfs://0.0.0.0/tmp/hive-cloudera/hive_2012-04-02_16-05-24_919_5976014408587784412/-ext-1
 Loading data to table default.partn_dyn partition (pobox=null)
 Failed with exception Number of dynamic partitions created is 1413, which is 
 more than 1000. To solve this try to set hive.exec.max.dynamic.partitions to 
 at least 1413.
 FAILED: Execution Error, return code 1 from 
 org.apache.hadoop.hive.ql.exec.MoveTask
 I checked the job.xml of the first map only job, there the value 
 hive.exec.max.dynamic.partitions=2000 is reflected but the move task is 
 taking the default value from hive-site.xml . If I change the value in 
 hive-site.xml then the job completes successfully. Bottom line,the property 
 'hive.exec.max.dynamic.partitions'set on CLI is not being considered by move 
 task

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2937) TestHiveServerSessions hangs when executed directly

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2937:


Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed, thanks Navis.

 TestHiveServerSessions hangs when executed directly
 ---

 Key: HIVE-2937
 URL: https://issues.apache.org/jira/browse/HIVE-2937
 Project: Hive
  Issue Type: Test
Reporter: Navis
Assignee: Navis
Priority: Trivial
 Attachments: HIVE-2937.D2697.1.patch, HIVE-2937.D2697.2.patch


 {code}
 ant test -Doffline=true -Dtestcase=TestHiveServerSessions
 {code}
 Hangs infinitely.
 I couldn't imagine exact cause of the problem, but found that by adding 'new 
 HiveServer.HiveServerHandler();' in setup(), test resulted to success.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2952) escape1.q and escape2.q failing in trunk

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2952?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2952:


Issue Type: Test  (was: Bug)

 escape1.q and escape2.q failing in trunk
 

 Key: HIVE-2952
 URL: https://issues.apache.org/jira/browse/HIVE-2952
 Project: Hive
  Issue Type: Test
 Environment: Mac OSX Lion
Reporter: Kevin Wilfong
Priority: Critical

 escape1.q and escape2.q have started failing in at least the Mac OS, but they 
 succeed in Linux
 The last time I saw them succeed in Mac was on 4/11 night

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2942) substr on string containing UTF-8 characters produces StringIndexOutOfBoundsException

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2942:


Status: Patch Available  (was: Open)

 substr on string containing UTF-8 characters produces 
 StringIndexOutOfBoundsException
 -

 Key: HIVE-2942
 URL: https://issues.apache.org/jira/browse/HIVE-2942
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2942.D2727.1.patch


 After HIVE-2792, the substr function produces a 
 StringIndexOutOfBoundsException when called on a string containing UTF-8 
 characters without the length argument being present.
 E.g.
 select substr(str, 1) from table1;
 now fails with that exception if str contains a UTF-8 character for any row 
 in the table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2858) Cache remote map reduce job stack traces for additional logging

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2858:


Status: Patch Available  (was: Open)

 Cache remote map reduce job stack traces for additional logging
 ---

 Key: HIVE-2858
 URL: https://issues.apache.org/jira/browse/HIVE-2858
 Project: Hive
  Issue Type: Improvement
  Components: Logging
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2858.D2223.1.patch, HIVE-2858.D2223.2.patch


 Currently we are parsing the task logs for failed jobs for information to 
 display to the user in the CLI.  In addition, we could parse those logs for 
 stack traces and store e them in the SessionState.  This way, when we log 
 failed queries, these will give us a decent idea of why those queries failed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2908) Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2908:


Affects Version/s: 0.9.0

 Hive: Extend ALTER TABLE DROP PARTITION syntax to use all comparators
 -

 Key: HIVE-2908
 URL: https://issues.apache.org/jira/browse/HIVE-2908
 Project: Hive
  Issue Type: Improvement
  Components: Query Processor
Affects Versions: 0.9.0
Reporter: Sambavi Muthukrishnan
Assignee: Sambavi Muthukrishnan
Priority: Minor
 Attachments: HIVE-2908.D2523.1.patch, HIVE-2908.D2523.2.patch, 
 HIVE-2908.final.patch.txt

   Original Estimate: 96h
  Remaining Estimate: 96h

 To drop a partition from a Hive table, this works:
 ALTER TABLE foo DROP PARTITION(ds = 'date')
 ...but it should also work to drop all partitions prior to date.
 ALTER TABLE foo DROP PARTITION(ds  'date')
 This task is to implement ALTER TABLE DROP PARTITION for all of the 
 comparators,   = =  = != instead of just for =.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2518) pull junit jar from maven repos via ivy

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2518:


Status: Patch Available  (was: Open)

 pull junit jar from maven repos via ivy
 ---

 Key: HIVE-2518
 URL: https://issues.apache.org/jira/browse/HIVE-2518
 Project: Hive
  Issue Type: Improvement
Reporter: He Yongqiang
Assignee: Kevin Wilfong
 Attachments: HIVE-2518.D2493.1.patch


 see https://issues.apache.org/jira/browse/HIVE-2505

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2797:


Attachment: HIVE-2797.7.patch

 Make the IP address of a Thrift client available to HMSHandler.
 ---

 Key: HIVE-2797
 URL: https://issues.apache.org/jira/browse/HIVE-2797
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2797.7.patch, HIVE-2797.D1701.1.patch, 
 HIVE-2797.D1701.2.patch, HIVE-2797.D1701.3.patch, HIVE-2797.D1701.4.patch, 
 HIVE-2797.D1701.5.patch, HIVE-2797.D1701.6.patch


 Currently, in unsecured mode, metastore Thrift calls are, from the 
 HMSHandler's point of view, anonymous.  If we expose the IP address of the 
 Thrift client to the HMSHandler from the Processor, this will help to give 
 some context, in particular for audit logging, of where the call is coming 
 from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2875) Renaming partition changes partition location prefix

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2875:


Status: Patch Available  (was: Open)

 Renaming partition changes partition location prefix
 

 Key: HIVE-2875
 URL: https://issues.apache.org/jira/browse/HIVE-2875
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2875.D2349.1.patch, HIVE-2875.D2349.2.patch


 Renaming a partition changes the location of the partition to the default 
 location of the table, followed by the partition specification.  It should 
 just change the partition specification of the path.
 If the path does not end with the old partition specification, we should 
 probably throw an exception because renaming a partition should not change 
 the path so dramatically, and not changing the path to reflect the new 
 partition name could leave the partition in a very confusing state.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2797:


Status: Patch Available  (was: Open)

 Make the IP address of a Thrift client available to HMSHandler.
 ---

 Key: HIVE-2797
 URL: https://issues.apache.org/jira/browse/HIVE-2797
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2797.7.patch, HIVE-2797.D1701.1.patch, 
 HIVE-2797.D1701.2.patch, HIVE-2797.D1701.3.patch, HIVE-2797.D1701.4.patch, 
 HIVE-2797.D1701.5.patch, HIVE-2797.D1701.6.patch


 Currently, in unsecured mode, metastore Thrift calls are, from the 
 HMSHandler's point of view, anonymous.  If we expose the IP address of the 
 Thrift client to the HMSHandler from the Processor, this will help to give 
 some context, in particular for audit logging, of where the call is coming 
 from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2797:


Status: Patch Available  (was: Open)

 Make the IP address of a Thrift client available to HMSHandler.
 ---

 Key: HIVE-2797
 URL: https://issues.apache.org/jira/browse/HIVE-2797
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2797.D1701.1.patch, HIVE-2797.D1701.2.patch, 
 HIVE-2797.D1701.3.patch, HIVE-2797.D1701.4.patch, HIVE-2797.D1701.5.patch


 Currently, in unsecured mode, metastore Thrift calls are, from the 
 HMSHandler's point of view, anonymous.  If we expose the IP address of the 
 Thrift client to the HMSHandler from the Processor, this will help to give 
 some context, in particular for audit logging, of where the call is coming 
 from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2797:


Status: Open  (was: Patch Available)

 Make the IP address of a Thrift client available to HMSHandler.
 ---

 Key: HIVE-2797
 URL: https://issues.apache.org/jira/browse/HIVE-2797
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2797.D1701.1.patch, HIVE-2797.D1701.2.patch, 
 HIVE-2797.D1701.3.patch, HIVE-2797.D1701.4.patch


 Currently, in unsecured mode, metastore Thrift calls are, from the 
 HMSHandler's point of view, anonymous.  If we expose the IP address of the 
 Thrift client to the HMSHandler from the Processor, this will help to give 
 some context, in particular for audit logging, of where the call is coming 
 from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2471) Add timestamp column to the partition stats table.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2471:


Description: Occasionally, when entries are added to the partition stats 
table the program is halted before it can delete those entries, by an 
exception, keyboard interrupt, etc.  These build up to the point where the 
table gets very large, and it hurts the performance of the update statement 
which is often called.  In order to fix this, I am adding a column to the table 
which is auto-populated with the current timestamp.  I am also adding an index 
on this column.  This will allow us to create scripts that go through 
periodically and clean out old entries from the table.  (was: Occasionally, 
when entries are added to the partition stats table the program is halted 
before it can delete those entries, by an exception, keyboard interrupt, etc.  
These build up to the point where the table gets very large, and it hurts the 
performance of the update statement which is often called.  In order to fix 
this, I am adding a column to the table which is auto-populated with the 
current timestamp.  I am also adding an index on this column.  This will allow 
us to create scripts that go through periodically and clean out old entries 
from the table.  The index will help to keep the runtime of these scripts 
short, and hence reduce the amount of time they need to lock the table/indexes 
for.)
Summary: Add timestamp column to the partition stats table.  (was: Add 
timestamp column with index to the partition stats table.)

 Add timestamp column to the partition stats table.
 --

 Key: HIVE-2471
 URL: https://issues.apache.org/jira/browse/HIVE-2471
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2471.1.patch.txt, HIVE-2471.D2367.1.patch


 Occasionally, when entries are added to the partition stats table the program 
 is halted before it can delete those entries, by an exception, keyboard 
 interrupt, etc.  These build up to the point where the table gets very large, 
 and it hurts the performance of the update statement which is often called.  
 In order to fix this, I am adding a column to the table which is 
 auto-populated with the current timestamp.  I am also adding an index on this 
 column.  This will allow us to create scripts that go through periodically 
 and clean out old entries from the table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2471) Add timestamp column to the partition stats table.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2471:


Description: Occasionally, when entries are added to the partition stats 
table the program is halted before it can delete those entries, by an 
exception, keyboard interrupt, etc.  These build up to the point where the 
table gets very large, and it hurts the performance of the update statement 
which is often called.  In order to fix this, I am adding a column to the table 
which is auto-populated with the current timestamp.  This will allow us to 
create scripts that go through periodically and clean out old entries from the 
table.  (was: Occasionally, when entries are added to the partition stats table 
the program is halted before it can delete those entries, by an exception, 
keyboard interrupt, etc.  These build up to the point where the table gets very 
large, and it hurts the performance of the update statement which is often 
called.  In order to fix this, I am adding a column to the table which is 
auto-populated with the current timestamp.  I am also adding an index on this 
column.  This will allow us to create scripts that go through periodically and 
clean out old entries from the table.)

 Add timestamp column to the partition stats table.
 --

 Key: HIVE-2471
 URL: https://issues.apache.org/jira/browse/HIVE-2471
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2471.1.patch.txt, HIVE-2471.D2367.1.patch


 Occasionally, when entries are added to the partition stats table the program 
 is halted before it can delete those entries, by an exception, keyboard 
 interrupt, etc.  These build up to the point where the table gets very large, 
 and it hurts the performance of the update statement which is often called.  
 In order to fix this, I am adding a column to the table which is 
 auto-populated with the current timestamp.  This will allow us to create 
 scripts that go through periodically and clean out old entries from the table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2471) Add timestamp column to the partition stats table.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2471:


Status: Patch Available  (was: Open)

 Add timestamp column to the partition stats table.
 --

 Key: HIVE-2471
 URL: https://issues.apache.org/jira/browse/HIVE-2471
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2471.1.patch.txt, HIVE-2471.D2367.1.patch


 Occasionally, when entries are added to the partition stats table the program 
 is halted before it can delete those entries, by an exception, keyboard 
 interrupt, etc.  These build up to the point where the table gets very large, 
 and it hurts the performance of the update statement which is often called.  
 In order to fix this, I am adding a column to the table which is 
 auto-populated with the current timestamp.  This will allow us to create 
 scripts that go through periodically and clean out old entries from the table.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2871) Add a new hook to run at the beginning and end of the Driver.run method

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2871:


Status: Patch Available  (was: Open)

 Add a new hook to run at the beginning and end of the Driver.run method
 ---

 Key: HIVE-2871
 URL: https://issues.apache.org/jira/browse/HIVE-2871
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2871.D2331.1.patch


 Driver.run is the highest level method which all queries go through, whether 
 they come from Hive Server, the CLI, or any other entry.  We also do not have 
 any hooks before the compilation method is called, and having hooks in 
 Driver.run would provide this.  Having hooks in Driver.run will allow, for 
 example, being able to overwrite config values used throughout query 
 processing, including compilation, and at the other end, cleaning up any 
 resources/logging any final values just before returning to the user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2872) Store which configs the user has explicitly changed

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2872:


Status: Patch Available  (was: Open)

 Store which configs the user has explicitly changed
 ---

 Key: HIVE-2872
 URL: https://issues.apache.org/jira/browse/HIVE-2872
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2872.D2337.1.patch, HIVE-2872.D2337.2.patch


 It would be useful to keep track of which config variables the user has 
 explicitly changed from the values which are either default or loaded from 
 hive-site.xml.  These include config variables set using the hiveconf 
 argument to the CLI, and via the SET command.  This could be used to prevent 
 Hive from changing a config variable which has been explicitly set by the 
 user, and also potentially for logging to help with later debugging of failed 
 queries.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2871) Add a new hook to run at the beginning and end of the Driver.run method

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2871:


Status: Patch Available  (was: Open)

 Add a new hook to run at the beginning and end of the Driver.run method
 ---

 Key: HIVE-2871
 URL: https://issues.apache.org/jira/browse/HIVE-2871
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2871.D2331.1.patch, HIVE-2871.D2331.2.patch


 Driver.run is the highest level method which all queries go through, whether 
 they come from Hive Server, the CLI, or any other entry.  We also do not have 
 any hooks before the compilation method is called, and having hooks in 
 Driver.run would provide this.  Having hooks in Driver.run will allow, for 
 example, being able to overwrite config values used throughout query 
 processing, including compilation, and at the other end, cleaning up any 
 resources/logging any final values just before returning to the user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2871) Add a new hook to run at the beginning and end of the Driver.run method

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2871:


Status: Patch Available  (was: Open)

 Add a new hook to run at the beginning and end of the Driver.run method
 ---

 Key: HIVE-2871
 URL: https://issues.apache.org/jira/browse/HIVE-2871
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2871.D2331.1.patch, HIVE-2871.D2331.2.patch, 
 HIVE-2871.D2331.3.patch


 Driver.run is the highest level method which all queries go through, whether 
 they come from Hive Server, the CLI, or any other entry.  We also do not have 
 any hooks before the compilation method is called, and having hooks in 
 Driver.run would provide this.  Having hooks in Driver.run will allow, for 
 example, being able to overwrite config values used throughout query 
 processing, including compilation, and at the other end, cleaning up any 
 resources/logging any final values just before returning to the user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2875) Renaming partition changes partition location prefix

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2875:


Status: Patch Available  (was: Open)

 Renaming partition changes partition location prefix
 

 Key: HIVE-2875
 URL: https://issues.apache.org/jira/browse/HIVE-2875
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2875.D2349.1.patch


 Renaming a partition changes the location of the partition to the default 
 location of the table, followed by the partition specification.  It should 
 just change the partition specification of the path.
 If the path does not end with the old partition specification, we should 
 probably throw an exception because renaming a partition should not change 
 the path so dramatically, and not changing the path to reflect the new 
 partition name could leave the partition in a very confusing state.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2864) If hive history file's directory doesn't exist don't crash

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2864:


Status: Patch Available  (was: Open)

 If hive history file's directory doesn't exist don't crash
 --

 Key: HIVE-2864
 URL: https://issues.apache.org/jira/browse/HIVE-2864
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2864.D2265.1.patch


 Currently, if the history file's directory does not exist the Hive client 
 crashes.  Instead, since this is not a vital feature, it should just display 
 a warning to the user and continue without it.
 This will become more important once the directory becomes configurable, see:
 https://issues.apache.org/jira/browse/HIVE-1708

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2866) Cache local map reduce job errors for additional logging

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2866:


Status: Patch Available  (was: Open)

 Cache local map reduce job errors for additional logging
 

 Key: HIVE-2866
 URL: https://issues.apache.org/jira/browse/HIVE-2866
 Project: Hive
  Issue Type: Improvement
  Components: Logging
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2866.D2277.1.patch


 Using the CachingPrintStream we are storing errors that occur locally in 
 Hive, but because local map reduce jobs are run in a separate JVM we are not 
 storing that occur for these. We can use this same construct to store errors 
 written to the subprocesses error stream.  This way, when we log failed 
 queries, these will give us a decent idea of why those queries failed.
 See related issues:
 https://issues.apache.org/jira/browse/HIVE-2832
 https://issues.apache.org/jira/browse/HIVE-2858

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2714) Lots of special characters are not handled in LIKE

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2714:


Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed.  Thanks Jonathan.

 Lots of special characters are not handled in LIKE
 --

 Key: HIVE-2714
 URL: https://issues.apache.org/jira/browse/HIVE-2714
 Project: Hive
  Issue Type: Bug
Reporter: Jonathan Chang
Assignee: Jonathan Chang
 Attachments: HIVE-2714.D1239.1.patch, HIVE-2714.D1239.1.patch


 Currently LIKE converts the string to a regular expression.  It tries to 
 handle special characters but many are not correctly handled, e.g. +, ?, |, 
 etc.  That code should just use Pattern.quote.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2837) insert into external tables should not be allowed

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2837:


Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed, thanks Namit.

 insert into external tables should not be allowed
 -

 Key: HIVE-2837
 URL: https://issues.apache.org/jira/browse/HIVE-2837
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
Assignee: Namit Jain
 Attachments: HIVE-2837.D2211.1.patch, HIVE-2837.D2211.2.patch


 This is a very risky thing to allow. 
 Since, the external tables can point to any user location, which can 
 potentially corrupt some other tables.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2805) Move metastore upgrade scripts labeled 0.10.0 into scripts labeled 0.9.0

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2805:


Status: Patch Available  (was: Open)

 Move metastore upgrade scripts labeled 0.10.0 into scripts labeled 0.9.0
 

 Key: HIVE-2805
 URL: https://issues.apache.org/jira/browse/HIVE-2805
 Project: Hive
  Issue Type: Task
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2805.D1743.1.patch


 Move contents of upgrade-0.9.0-to-0.10.0.mysql.sql, 
 upgrade-0.9.0-to-0.10.0.derby.sql into upgrade-0.8.0-to-0.9.0.mysql.sql, 
 upgrade-0.8.0-to-0.9.0.derby.sql
 Rename hive-schema-0.10.0.derby.sql, hive-schema-0.10.0.mysql.sql to 
 hive-schema-0.9.0.derby.sql, hive-schema-0.9.0.mysql.sql

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2858) Cache remote map reduce job stack traces for additional logging

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2858:


Status: Patch Available  (was: Open)

 Cache remote map reduce job stack traces for additional logging
 ---

 Key: HIVE-2858
 URL: https://issues.apache.org/jira/browse/HIVE-2858
 Project: Hive
  Issue Type: Improvement
  Components: Logging
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2858.D2223.1.patch


 Currently we are parsing the task logs for failed jobs for information to 
 display to the user in the CLI.  In addition, we could parse those logs for 
 stack traces and store e them in the SessionState.  This way, when we log 
 failed queries, these will give us a decent idea of why those queries failed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2838) cleanup readentity/writeentity

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2838:


   Resolution: Fixed
Fix Version/s: 0.9.0
   Status: Resolved  (was: Patch Available)

Committed, thanks Namit.

 cleanup readentity/writeentity
 --

 Key: HIVE-2838
 URL: https://issues.apache.org/jira/browse/HIVE-2838
 Project: Hive
  Issue Type: Bug
Reporter: Namit Jain
Assignee: Namit Jain
 Fix For: 0.9.0

 Attachments: HIVE-2838.D2193.1.patch, HIVE-2838.D2193.2.patch


 Ideally, there should be one common entity instead of readentity/writeentity.
 Unfortunately, that would be a backward incompatible change since users os 
 hive might have written
 there own hooks, where they are using readentity/writeentity.
 We should atleast create a common class, and then we can deprecate read/write 
 entity later, for a new release.
 For now, I propose to make a backward compatible change.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2853) Add pre event listeners to metastore

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2853:


Status: Patch Available  (was: Open)

 Add pre event listeners to metastore
 

 Key: HIVE-2853
 URL: https://issues.apache.org/jira/browse/HIVE-2853
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2853.D2175.1.patch


 Currently there are event listeners in the metastore which run after the 
 completion of a method.  It would be useful to have similar hooks which run 
 before the metastore method is executed.  These can be used to make 
 validating names, locations, etc. customizable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2427) Warn user that precision is lost when bigint is implicitly cast to double in joins and unions.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2427:


Status: Patch Available  (was: Open)

 Warn user that precision is lost when bigint is implicitly cast to double in 
 joins and unions.
 --

 Key: HIVE-2427
 URL: https://issues.apache.org/jira/browse/HIVE-2427
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2427.1.patch.txt, HIVE-2427.2.patch.txt, 
 HIVE-2427.3.patch.txt, HIVE-2427.D2181.1.patch


 When a bigint is implicitly cast to a double (when a bigint is involved in an 
 equality expression with a string or double as part of a join condition, or 
 when a bigint column is to be combined with a string or double column as part 
 of a union) precision may be lost, resulting in unexpected behavior. Until we 
 fix the underlying issue we should throw an error in strict mode, and a 
 warning in nonstrict mode alerting the user about this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2797:


Status: Open  (was: Patch Available)

 Make the IP address of a Thrift client available to HMSHandler.
 ---

 Key: HIVE-2797
 URL: https://issues.apache.org/jira/browse/HIVE-2797
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2797.D1701.1.patch


 Currently, in unsecured mode, metastore Thrift calls are, from the 
 HMSHandler's point of view, anonymous.  If we expose the IP address of the 
 Thrift client to the HMSHandler from the Processor, this will help to give 
 some context, in particular for audit logging, of where the call is coming 
 from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2428) ADD ARCHIVE has no effect when query runs in local mode

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2428:


Attachment: add_archive_local.zip

 ADD ARCHIVE has no effect when query runs in local mode
 ---

 Key: HIVE-2428
 URL: https://issues.apache.org/jira/browse/HIVE-2428
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2428.1.patch.txt, HIVE-2428.D2121.1.patch, 
 add_archive_local.zip


 When you ADD ARCHIVE a file and the query runs in local mode, the file is not 
 available at all.
 For example running the commands
 echo '#!/bin/bash\nls -l 12;read'  foo.sh
 hive -e 'ADD FILE foo.sh; ADD ARCHIVE /tmp/mapreduce_test.zip; SELECT 
 TRANSFORM(*) USING sh foo.sh FROM src'
 when the query is allowed to run in local mode, only information for foo.sh 
 is returned.
 However, if the query is not allowed to run in local mode, information for 
 both foo.sh and mapreduce_test.zip is returned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2428) ADD ARCHIVE has no effect when query runs in local mode

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2428:


Status: Patch Available  (was: Open)

 ADD ARCHIVE has no effect when query runs in local mode
 ---

 Key: HIVE-2428
 URL: https://issues.apache.org/jira/browse/HIVE-2428
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2428.1.patch.txt, HIVE-2428.D2121.1.patch, 
 add_archive_local.zip


 When you ADD ARCHIVE a file and the query runs in local mode, the file is not 
 available at all.
 For example running the commands
 echo '#!/bin/bash\nls -l 12;read'  foo.sh
 hive -e 'ADD FILE foo.sh; ADD ARCHIVE /tmp/mapreduce_test.zip; SELECT 
 TRANSFORM(*) USING sh foo.sh FROM src'
 when the query is allowed to run in local mode, only information for foo.sh 
 is returned.
 However, if the query is not allowed to run in local mode, information for 
 both foo.sh and mapreduce_test.zip is returned.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2833) Fix test failures caused by HIVE-2716

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2833:


Attachment: HIVE-2716.D2055.1.patch

 Fix test failures caused by HIVE-2716
 -

 Key: HIVE-2833
 URL: https://issues.apache.org/jira/browse/HIVE-2833
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Carl Steinbach
Assignee: Kevin Wilfong
 Attachments: HIVE-2716.D2055.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2833) Fix test failures caused by HIVE-2716

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2833:


Status: Patch Available  (was: Open)

 Fix test failures caused by HIVE-2716
 -

 Key: HIVE-2833
 URL: https://issues.apache.org/jira/browse/HIVE-2833
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Carl Steinbach
Assignee: Kevin Wilfong
 Attachments: HIVE-2716.D2055.1.patch




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2832) Cache error messages for additional logging

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2832:


Assignee: Kevin Wilfong
  Status: Patch Available  (was: Open)

 Cache error messages for additional logging
 ---

 Key: HIVE-2832
 URL: https://issues.apache.org/jira/browse/HIVE-2832
 Project: Hive
  Issue Type: Improvement
  Components: Logging
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2832.D2025.1.patch


 It would be good if we could cache logs written to SessionState.err so that 
 they could be exposed to hooks for additional logging.  This would allow 
 logging of error messages with the queries that failed in a central location.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2823) Add logging of total run time of Driver

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2823?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2823:


Status: Patch Available  (was: Open)

 Add logging of total run time of Driver
 ---

 Key: HIVE-2823
 URL: https://issues.apache.org/jira/browse/HIVE-2823
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2823.D1977.1.patch, HIVE-2823.D1977.2.patch


 We are currently logging the run time of several components of the Driver, 
 e.g. compilation and map reduce execution time.  It would be helpful to log 
 the total run time of the Driver's run method as well.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2825) Concatenating a partition does not inherit location from table

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2825:


Status: Patch Available  (was: Open)

 Concatenating a partition does not inherit location from table
 --

 Key: HIVE-2825
 URL: https://issues.apache.org/jira/browse/HIVE-2825
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2825.D1995.1.patch


 When a table is created in one dfs, a partition is added to that table, the 
 table's dfs is changed, and the partitioned is concatenated, the partitions 
 location remains exactly the same.  Instead, it should inherit its location 
 from the table, and be updated accordingly.
 See https://issues.apache.org/jira/browse/HIVE-1707 for an analogous change 
 to insert overwrite.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2490) Add reset operation and average time attribute to Metrics MBean.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2490:


Attachment: HIVE-2490.1.patch.txt

 Add reset operation and average time attribute to Metrics MBean.
 

 Key: HIVE-2490
 URL: https://issues.apache.org/jira/browse/HIVE-2490
 Project: Hive
  Issue Type: New Feature
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2490.1.patch.txt


 We should add a reset operation to the Metrics MBean, which will set all the 
 counters to 0.
 Note: Deleting the counters from the map of attributes was not suggested 
 because that could break any scripts that get the list of attributes from the 
 bean and then the values of each attribute.  Also, 0 is unlikely to be an 
 actual value for any counter, and it will not break the increment 
 functionality. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2490) Add reset operation and average time attribute to Metrics MBean.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2490:


Status: Patch Available  (was: Open)

 Add reset operation and average time attribute to Metrics MBean.
 

 Key: HIVE-2490
 URL: https://issues.apache.org/jira/browse/HIVE-2490
 Project: Hive
  Issue Type: New Feature
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2490.1.patch.txt


 We should add a reset operation to the Metrics MBean, which will set all the 
 counters to 0.
 Note: Deleting the counters from the map of attributes was not suggested 
 because that could break any scripts that get the list of attributes from the 
 bean and then the values of each attribute.  Also, 0 is unlikely to be an 
 actual value for any counter, and it will not break the increment 
 functionality. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2812) Hive multi group by single reducer optimization fails when aggregation with no keys followed by query with no aggregations

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2812:


Status: Patch Available  (was: Open)

 Hive multi group by single reducer optimization fails when aggregation with 
 no keys followed by query with no aggregations
 --

 Key: HIVE-2812
 URL: https://issues.apache.org/jira/browse/HIVE-2812
 Project: Hive
  Issue Type: Bug
Affects Versions: 0.9.0
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2812.D1821.1.patch


 In multi insert queries where one subquery involves an aggregation with no 
 distinct or group by keys and is followed by a query without any 
 aggregations, like the following, Hive will attempt to add a group by 
 operator for the query without aggregations, causing semantic analysis to 
 fail.
 FROM src
 INSERT OVERWRITE TABLE table1 SELECT count(*)
 INSERT OVERWRITE TABLE table2 SELECT key;

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2797:


Component/s: Metastore

 Make the IP address of a Thrift client available to HMSHandler.
 ---

 Key: HIVE-2797
 URL: https://issues.apache.org/jira/browse/HIVE-2797
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Kevin Wilfong
 Attachments: HIVE-2797.D1701.1.patch


 Currently, in unsecured mode, metastore Thrift calls are, from the 
 HMSHandler's point of view, anonymous.  If we expose the IP address of the 
 Thrift client to the HMSHandler from the Processor, this will help to give 
 some context, in particular for audit logging, of where the call is coming 
 from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2797) Make the IP address of a Thrift client available to HMSHandler.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2797:


Assignee: Kevin Wilfong
  Status: Patch Available  (was: Open)

 Make the IP address of a Thrift client available to HMSHandler.
 ---

 Key: HIVE-2797
 URL: https://issues.apache.org/jira/browse/HIVE-2797
 Project: Hive
  Issue Type: Improvement
  Components: Metastore
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2797.D1701.1.patch


 Currently, in unsecured mode, metastore Thrift calls are, from the 
 HMSHandler's point of view, anonymous.  If we expose the IP address of the 
 Thrift client to the HMSHandler from the Processor, this will help to give 
 some context, in particular for audit logging, of where the call is coming 
 from.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2612:


Attachment: HIVE-2612.6.patch.txt

 support hive table/partitions exists in more than one region
 

 Key: HIVE-2612
 URL: https://issues.apache.org/jira/browse/HIVE-2612
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: He Yongqiang
Assignee: Namit Jain
 Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
 HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
 HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, HIVE-2612.D1569.3.patch, 
 HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, HIVE-2612.D1569.6.patch, 
 HIVE-2612.D1569.7.patch, hive.2612.5.patch


 1) add region object into hive metastore
 2) each partition/table has a primary region and a list of living regions, 
 and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2612:


Attachment: HIVE-2612.7.patch.txt

 support hive table/partitions exists in more than one region
 

 Key: HIVE-2612
 URL: https://issues.apache.org/jira/browse/HIVE-2612
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: He Yongqiang
Assignee: Namit Jain
 Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
 HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
 HIVE-2612.7.patch.txt, HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, 
 HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, 
 HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch, hive.2612.5.patch


 1) add region object into hive metastore
 2) each partition/table has a primary region and a list of living regions, 
 and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2612) support hive table/partitions exists in more than one region

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2612:


Status: Patch Available  (was: Open)

 support hive table/partitions exists in more than one region
 

 Key: HIVE-2612
 URL: https://issues.apache.org/jira/browse/HIVE-2612
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: He Yongqiang
Assignee: Kevin Wilfong
 Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
 HIVE-2612.3.patch.txt, HIVE-2612.4.patch.txt, HIVE-2612.6.patch.txt, 
 HIVE-2612.7.patch.txt, HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, 
 HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, 
 HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch, HIVE-2612.D1707.1.patch, 
 hive.2612.5.patch


 1) add region object into hive metastore
 2) each partition/table has a primary region and a list of living regions, 
 and also data location in each region

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2795) View partitions do not have a storage descriptor

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2795:


Description: 
Besides being an inconsistency, it causes errors.

Calling describe formatted on a view partition throws an exception

java.lang.NullPointerException
  at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505)  
  at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570)

because it does not have a column descriptor, which is part of the storage 
descriptor.

 View partitions do not have a storage descriptor
 

 Key: HIVE-2795
 URL: https://issues.apache.org/jira/browse/HIVE-2795
 Project: Hive
  Issue Type: Improvement
Reporter: Namit Jain
Assignee: Kevin Wilfong

 Besides being an inconsistency, it causes errors.
 Calling describe formatted on a view partition throws an exception
 java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) 
  
   at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570)
 because it does not have a column descriptor, which is part of the storage 
 descriptor.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2795) View partitions do not have a storage descriptor

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2795?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2795:


Status: Patch Available  (was: Open)

 View partitions do not have a storage descriptor
 

 Key: HIVE-2795
 URL: https://issues.apache.org/jira/browse/HIVE-2795
 Project: Hive
  Issue Type: Improvement
Reporter: Namit Jain
Assignee: Kevin Wilfong
 Attachments: HIVE-2795.D1683.1.patch


 Besides being an inconsistency, it causes errors.
 Calling describe formatted on a view partition throws an exception
 java.lang.NullPointerException
   at org.apache.hadoop.hive.ql.metadata.Partition.getCols(Partition.java:505) 
  
   at org.apache.hadoop.hive.ql.exec.DDLTask.describeTable(DDLTask.java:2570)
 because it does not have a column descriptor, which is part of the storage 
 descriptor.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2612) support hive table/partitions coexistes in more than one clusters

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2612:


Attachment: HIVE-2612.2.patch.txt

 support hive table/partitions coexistes in more than one clusters
 -

 Key: HIVE-2612
 URL: https://issues.apache.org/jira/browse/HIVE-2612
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: He Yongqiang
Assignee: Namit Jain
 Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
 HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, HIVE-2612.D1569.3.patch, 
 HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch


 1) add cluster object into hive metastore
 2) each partition/table has a creation cluster and a list of living clusters, 
 and also data location in each cluster

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2612) support hive table/partitions coexistes in more than one clusters

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2612:


Attachment: HIVE-2612.3.patch.txt

 support hive table/partitions coexistes in more than one clusters
 -

 Key: HIVE-2612
 URL: https://issues.apache.org/jira/browse/HIVE-2612
 Project: Hive
  Issue Type: New Feature
  Components: Metastore
Reporter: He Yongqiang
Assignee: Namit Jain
 Attachments: HIVE-2612.1.patch, HIVE-2612.2.patch.txt, 
 HIVE-2612.3.patch.txt, HIVE-2612.D1569.1.patch, HIVE-2612.D1569.2.patch, 
 HIVE-2612.D1569.3.patch, HIVE-2612.D1569.4.patch, HIVE-2612.D1569.5.patch, 
 HIVE-2612.D1569.6.patch, HIVE-2612.D1569.7.patch


 1) add cluster object into hive metastore
 2) each partition/table has a creation cluster and a list of living clusters, 
 and also data location in each cluster

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2756) Views should be added to the inputs of queries.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2756:


Description: 
Views, in addition to the underlying tables, should be included in the inputs 
of a command which queries a view.

This is useful when we want to know exactly what inputs need to be present in 
order to run the query run by a user.

  was:Views, in addition to the underlying tables, should be included in the 
inputs of a command which queries a view.


 Views should be added to the inputs of queries.
 ---

 Key: HIVE-2756
 URL: https://issues.apache.org/jira/browse/HIVE-2756
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong

 Views, in addition to the underlying tables, should be included in the inputs 
 of a command which queries a view.
 This is useful when we want to know exactly what inputs need to be present in 
 order to run the query run by a user.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2758) Metastore is caching too aggressively

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2758:


   Resolution: Fixed
Fix Version/s: 0.9.0
   Status: Resolved  (was: Patch Available)

 Metastore is caching too aggressively
 -

 Key: HIVE-2758
 URL: https://issues.apache.org/jira/browse/HIVE-2758
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Fix For: 0.9.0

 Attachments: HIVE-2758.D1491.1.patch, HIVE-2758.D1491.2.patch


 The metastore is caching values, like table names and locations too 
 aggressively, leading to inconsistencies across Hive clients and metastore 
 Thrift servers.
 For example, open two Hive clients, in each call
 DESCRIBE FORMATTED table_foo;
 Then in one of those clients, execute
 ALTER TABLE table_foo RENAME TO table_bar;
 Then in both clients call
 DESCRIBE FORMATTED table_bar;
 In the client that executed the alter command, the location is correct, 
 however, in the other Hive client, it will still show the original location 
 of table_foo.
 A similar experiment can be done using metastore Thrift servers, substituting 
 get_table for DESCRIBE FORMATTED and alter_table for ALTER TABLE ... RENAME 
 TO.
 On the Thrift server you can see that the one which did not execute the alter 
 command, not only returns the wrong location, despite calling 
 get_table('table_bar') it will return a table that still has the name 
 table_foo.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2762:


Attachment: HIVE-2762.2.patch.txt

 Alter Table Partition Concatenate Fails On Certain Characters
 -

 Key: HIVE-2762
 URL: https://issues.apache.org/jira/browse/HIVE-2762
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2762.1.patch.txt, HIVE-2762.2.patch.txt, 
 HIVE-2762.D1533.1.patch, HIVE-2762.D1533.2.patch, HIVE-2762.D1533.3.patch


 Alter table partition concatenate creates a Java URI object for the location 
 of a partition.  If the partition name contains certain characters, such as } 
 or space ' ', the object constructor fails, causing the query to fail. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2762:


Status: Patch Available  (was: Open)

 Alter Table Partition Concatenate Fails On Certain Characters
 -

 Key: HIVE-2762
 URL: https://issues.apache.org/jira/browse/HIVE-2762
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2762.1.patch.txt, HIVE-2762.2.patch.txt, 
 HIVE-2762.D1533.1.patch, HIVE-2762.D1533.2.patch, HIVE-2762.D1533.3.patch


 Alter table partition concatenate creates a Java URI object for the location 
 of a partition.  If the partition name contains certain characters, such as } 
 or space ' ', the object constructor fails, causing the query to fail. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2758) Metastore is caching too aggressively

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2758:


Status: Patch Available  (was: Open)

 Metastore is caching too aggressively
 -

 Key: HIVE-2758
 URL: https://issues.apache.org/jira/browse/HIVE-2758
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2758.D1491.1.patch, HIVE-2758.D1491.2.patch


 The metastore is caching values, like table names and locations too 
 aggressively, leading to inconsistencies across Hive clients and metastore 
 Thrift servers.
 For example, open two Hive clients, in each call
 DESCRIBE FORMATTED table_foo;
 Then in one of those clients, execute
 ALTER TABLE table_foo RENAME TO table_bar;
 Then in both clients call
 DESCRIBE FORMATTED table_bar;
 In the client that executed the alter command, the location is correct, 
 however, in the other Hive client, it will still show the original location 
 of table_foo.
 A similar experiment can be done using metastore Thrift servers, substituting 
 get_table for DESCRIBE FORMATTED and alter_table for ALTER TABLE ... RENAME 
 TO.
 On the Thrift server you can see that the one which did not execute the alter 
 command, not only returns the wrong location, despite calling 
 get_table('table_bar') it will return a table that still has the name 
 table_foo.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2762:


Attachment: HIVE-2762.1.patch.txt

 Alter Table Partition Concatenate Fails On Certain Characters
 -

 Key: HIVE-2762
 URL: https://issues.apache.org/jira/browse/HIVE-2762
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2762.1.patch.txt, HIVE-2762.D1533.1.patch, 
 HIVE-2762.D1533.2.patch


 Alter table partition concatenate creates a Java URI object for the location 
 of a partition.  If the partition name contains certain characters, such as } 
 or space ' ', the object constructor fails, causing the query to fail. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2762) Alter Table Partition Concatenate Fails On Certain Characters

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2762:


Status: Patch Available  (was: Open)

 Alter Table Partition Concatenate Fails On Certain Characters
 -

 Key: HIVE-2762
 URL: https://issues.apache.org/jira/browse/HIVE-2762
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2762.1.patch.txt, HIVE-2762.D1533.1.patch, 
 HIVE-2762.D1533.2.patch


 Alter table partition concatenate creates a Java URI object for the location 
 of a partition.  If the partition name contains certain characters, such as } 
 or space ' ', the object constructor fails, causing the query to fail. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2758) Metastore is caching too aggressively

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2758:


Status: Patch Available  (was: Open)

 Metastore is caching too aggressively
 -

 Key: HIVE-2758
 URL: https://issues.apache.org/jira/browse/HIVE-2758
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2758.D1491.1.patch


 The metastore is caching values, like table names and locations too 
 aggressively, leading to inconsistencies across Hive clients and metastore 
 Thrift servers.
 For example, open two Hive clients, in each call
 DESCRIBE FORMATTED table_foo;
 Then in one of those clients, execute
 ALTER TABLE table_foo RENAME TO table_bar;
 Then in both clients call
 DESCRIBE FORMATTED table_bar;
 In the client that executed the alter command, the location is correct, 
 however, in the other Hive client, it will still show the original location 
 of table_foo.
 A similar experiment can be done using metastore Thrift servers, substituting 
 get_table for DESCRIBE FORMATTED and alter_table for ALTER TABLE ... RENAME 
 TO.
 On the Thrift server you can see that the one which did not execute the alter 
 command, not only returns the wrong location, despite calling 
 get_table('table_bar') it will return a table that still has the name 
 table_foo.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2758) Metastore is caching too aggressively

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2758:


Status: Open  (was: Patch Available)

 Metastore is caching too aggressively
 -

 Key: HIVE-2758
 URL: https://issues.apache.org/jira/browse/HIVE-2758
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2758.D1491.1.patch


 The metastore is caching values, like table names and locations too 
 aggressively, leading to inconsistencies across Hive clients and metastore 
 Thrift servers.
 For example, open two Hive clients, in each call
 DESCRIBE FORMATTED table_foo;
 Then in one of those clients, execute
 ALTER TABLE table_foo RENAME TO table_bar;
 Then in both clients call
 DESCRIBE FORMATTED table_bar;
 In the client that executed the alter command, the location is correct, 
 however, in the other Hive client, it will still show the original location 
 of table_foo.
 A similar experiment can be done using metastore Thrift servers, substituting 
 get_table for DESCRIBE FORMATTED and alter_table for ALTER TABLE ... RENAME 
 TO.
 On the Thrift server you can see that the one which did not execute the alter 
 command, not only returns the wrong location, despite calling 
 get_table('table_bar') it will return a table that still has the name 
 table_foo.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2750) Hive multi group by single reducer optimization causes invalid column reference error

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2750:


Status: Patch Available  (was: Open)

 Hive multi group by single reducer optimization causes invalid column 
 reference error
 -

 Key: HIVE-2750
 URL: https://issues.apache.org/jira/browse/HIVE-2750
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2750.D1455.1.patch


 After the optimization, if two query blocks have the same distinct clause and 
 the same group by keys, but the first query block does not reference all the 
 rows the second query block does, an invalid column reference error is raised 
 for the columns unreferenced in the first query block.
 E.g.
 FROM src
 INSERT OVERWRITE TABLE dest_g2 SELECT substr(src.key,1,1), count(DISTINCT 
 src.key) WHERE substr(src.key,1,1) = 5 GROUP BY substr(src.key,1,1)
 INSERT OVERWRITE TABLE dest_g3 SELECT substr(src.key,1,1), count(DISTINCT 
 src.key), count(src.value) WHERE substr(src.key,1,1)  5 GROUP BY 
 substr(src.key,1,1);
 This results in an invalid column reference error on src.value

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2674) get_partitions_ps throws TApplicationException if table doesn't exist

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2674:


Status: Patch Available  (was: Open)

 get_partitions_ps throws TApplicationException if table doesn't exist
 -

 Key: HIVE-2674
 URL: https://issues.apache.org/jira/browse/HIVE-2674
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2674.D987.1.patch, HIVE-2674.D987.2.patch


 If the table passed to get_partition_ps doesn't exist, a NPE is thrown by 
 getPartitionPsQueryResults.  There should be a check here, which throws a 
 NoSuchObjectException if the table doesn't exist.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2674) get_partitions_ps throws TApplicationException if table doesn't exist

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2674:


Status: Patch Available  (was: Open)

 get_partitions_ps throws TApplicationException if table doesn't exist
 -

 Key: HIVE-2674
 URL: https://issues.apache.org/jira/browse/HIVE-2674
 Project: Hive
  Issue Type: Bug
  Components: Metastore
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2674.D987.1.patch


 If the table passed to get_partition_ps doesn't exist, a NPE is thrown by 
 getPartitionPsQueryResults.  There should be a check here, which throws a 
 NoSuchObjectException if the table doesn't exist.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2611) Make index table output of create index command if index is table based

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2611:


Status: Patch Available  (was: Open)

 Make index table output of create index command if index is table based
 ---

 Key: HIVE-2611
 URL: https://issues.apache.org/jira/browse/HIVE-2611
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2611.1.patch.txt, HIVE-2611.D705.1.patch, 
 HIVE-2611.D705.2.patch, HIVE-2611.D705.3.patch


 If an index is table based, when that index is created a table is created to 
 contain that index.  This should be listed in the output of the command.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2640) Add alterPartition to AlterHandler interface

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2640:


Status: Patch Available  (was: Open)

 Add alterPartition to AlterHandler interface
 

 Key: HIVE-2640
 URL: https://issues.apache.org/jira/browse/HIVE-2640
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2640.D699.1.patch, HIVE-2640.D699.2.patch


 Adding alterPartition to the AlterHandler interface would allow for 
 customized functionality to be executed as part of altering a partition, much 
 like it is already allowed for alterTable.
 Based on the name of the interface, and a comment in the AlterHandler code, 
 it looks like alterPartition was meant to be included along with alterTable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2640) Add alterPartition to AlterHandler interface

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2640?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2640:


Status: Patch Available  (was: Open)

 Add alterPartition to AlterHandler interface
 

 Key: HIVE-2640
 URL: https://issues.apache.org/jira/browse/HIVE-2640
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
 Attachments: HIVE-2640.D699.1.patch


 Adding alterPartition to the AlterHandler interface would allow for 
 customized functionality to be executed as part of altering a partition, much 
 like it is already allowed for alterTable.
 Based on the name of the interface, and a comment in the AlterHandler code, 
 it looks like alterPartition was meant to be included along with alterTable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2374) Make compression used between map reduce tasks configurable.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2374:


Status: Patch Available  (was: Open)

 Make compression used between map reduce tasks configurable.
 

 Key: HIVE-2374
 URL: https://issues.apache.org/jira/browse/HIVE-2374
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2374.1.patch.txt, HIVE-2374.2.patch.txt


 We want to allow the compression between map reduce tasks to be configurable, 
 similar to the way it is between the map and reduce jobs is configurable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2374) Make compression used between map reduce tasks configurable.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2374:


Status: Open  (was: Patch Available)

 Make compression used between map reduce tasks configurable.
 

 Key: HIVE-2374
 URL: https://issues.apache.org/jira/browse/HIVE-2374
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2374.1.patch.txt, HIVE-2374.2.patch.txt


 We want to allow the compression between map reduce tasks to be configurable, 
 similar to the way it is between the map and reduce jobs is configurable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2621) Allow multiple group bys with the same input data and spray keys to be run on the same reducer.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2621:


Status: Patch Available  (was: Open)

 Allow multiple group bys with the same input data and spray keys to be run on 
 the same reducer.
 ---

 Key: HIVE-2621
 URL: https://issues.apache.org/jira/browse/HIVE-2621
 Project: Hive
  Issue Type: New Feature
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2621.1.patch.txt, HIVE-2621.D567.1.patch


 Currently, when a user runs a query, such as a multi-insert, where each 
 insertion subclause consists of a simple query followed by a group by, the 
 group bys for each clause are run on a separate reducer.  This requires 
 writing the data for each group by clause to an intermediate file, and then 
 reading it back.  This uses a significant amount of the total CPU consumed by 
 the query for an otherwise simple query.
 If the subclauses are grouped by their distinct expressions and group by 
 keys, with all of the group by expressions for a group of subclauses run on a 
 single reducer, this would reduce the amount of reading/writing to 
 intermediate files for some queries.
 To do this, for each group of subclauses, in the mapper we would execute a 
 the filters for each subclause 'or'd together (provided each subclause has a 
 filter) followed by a reduce sink.  In the reducer, the child operators would 
 be each subclauses filter followed by the group by and any subsequent 
 operations.
 Note that this would require turning off map aggregation, so we would need to 
 make using this type of plan configurable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2611) Make index table output of create index command if index is table based

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2611:


Status: Patch Available  (was: Open)

 Make index table output of create index command if index is table based
 ---

 Key: HIVE-2611
 URL: https://issues.apache.org/jira/browse/HIVE-2611
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2611.1.patch.txt


 If an index is table based, when that index is created a table is created to 
 contain that index.  This should be listed in the output of the command.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2621) Allow multiple group bys with the same input data and spray keys to be run on the same reducer.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2621:


Attachment: HIVE-2621.1.patch.txt

 Allow multiple group bys with the same input data and spray keys to be run on 
 the same reducer.
 ---

 Key: HIVE-2621
 URL: https://issues.apache.org/jira/browse/HIVE-2621
 Project: Hive
  Issue Type: New Feature
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2621.1.patch.txt, HIVE-2621.D567.1.patch


 Currently, when a user runs a query, such as a multi-insert, where each 
 insertion subclause consists of a simple query followed by a group by, the 
 group bys for each clause are run on a separate reducer.  This requires 
 writing the data for each group by clause to an intermediate file, and then 
 reading it back.  This uses a significant amount of the total CPU consumed by 
 the query for an otherwise simple query.
 If the subclauses are grouped by their distinct expressions and group by 
 keys, with all of the group by expressions for a group of subclauses run on a 
 single reducer, this would reduce the amount of reading/writing to 
 intermediate files for some queries.
 To do this, for each group of subclauses, in the mapper we would execute a 
 the filters for each subclause 'or'd together (provided each subclause has a 
 filter) followed by a reduce sink.  In the reducer, the child operators would 
 be each subclauses filter followed by the group by and any subsequent 
 operations.
 Note that this would require turning off map aggregation, so we would need to 
 make using this type of plan configurable.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2607) Add caching to json_tuple

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2607:


Attachment: HIVE-2607.1.patch.txt

 Add caching to json_tuple
 -

 Key: HIVE-2607
 URL: https://issues.apache.org/jira/browse/HIVE-2607
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2607.1.patch.txt, HIVE-2607.D543.1.patch


 get_json_object uses a variety of caches to improve its performance.  
 json_tuple could benefit from having a similar cache from JSON string to 
 JSONObject.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2611) Make index table output of create index command if index is table based

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2611?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2611:


Attachment: HIVE-2611.1.patch.txt

 Make index table output of create index command if index is table based
 ---

 Key: HIVE-2611
 URL: https://issues.apache.org/jira/browse/HIVE-2611
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2611.1.patch.txt


 If an index is table based, when that index is created a table is created to 
 contain that index.  This should be listed in the output of the command.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2535) Use sorted nature of compact indexes

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2535:


Attachment: HIVE-2535.4.patch.txt

 Use sorted nature of compact indexes
 

 Key: HIVE-2535
 URL: https://issues.apache.org/jira/browse/HIVE-2535
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2535.1.patch.txt, HIVE-2535.2.patch.txt, 
 HIVE-2535.3.patch.txt, HIVE-2535.4.patch.txt


 Compact indexes are sorted based on the indexed columns, but we are not using 
 this fact when we access the index.
 To start with, if the index is stored as an RC file, and if the predicate 
 being used to access the index consists of only one non-partition condition 
 using one of the operators ,=,,=,= we could use a binary search (if 
 necessary) to find the block to begin scanning for unfiltered rows, and we 
 could use the result of comparing the value in the column with the constant 
 (this is necessarily the form of a predicate which is optimized using an 
 index) to determine when we have found all the rows which will be unfiltered.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2427) Warn user that precision is lost when bigint is implicitly cast to double in joins and unions.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2427:


Attachment: HIVE-2427.3.patch.txt

 Warn user that precision is lost when bigint is implicitly cast to double in 
 joins and unions.
 --

 Key: HIVE-2427
 URL: https://issues.apache.org/jira/browse/HIVE-2427
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2427.1.patch.txt, HIVE-2427.2.patch.txt, 
 HIVE-2427.3.patch.txt


 When a bigint is implicitly cast to a double (when a bigint is involved in an 
 equality expression with a string or double as part of a join condition, or 
 when a bigint column is to be combined with a string or double column as part 
 of a union) precision may be lost, resulting in unexpected behavior. Until we 
 fix the underlying issue we should throw an error in strict mode, and a 
 warning in nonstrict mode alerting the user about this.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2535) Use sorted nature of compact indexes

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2535:


Attachment: HIVE-2535.3.patch.txt

 Use sorted nature of compact indexes
 

 Key: HIVE-2535
 URL: https://issues.apache.org/jira/browse/HIVE-2535
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2535.1.patch.txt, HIVE-2535.2.patch.txt, 
 HIVE-2535.3.patch.txt


 Compact indexes are sorted based on the indexed columns, but we are not using 
 this fact when we access the index.
 To start with, if the index is stored as an RC file, and if the predicate 
 being used to access the index consists of only one non-partition condition 
 using one of the operators ,=,,=,= we could use a binary search (if 
 necessary) to find the block to begin scanning for unfiltered rows, and we 
 could use the result of comparing the value in the column with the constant 
 (this is necessarily the form of a predicate which is optimized using an 
 index) to determine when we have found all the rows which will be unfiltered.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2535) Use sorted nature of compact indexes

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2535:


Attachment: HIVE-2535.2.patch.txt

 Use sorted nature of compact indexes
 

 Key: HIVE-2535
 URL: https://issues.apache.org/jira/browse/HIVE-2535
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2535.1.patch.txt, HIVE-2535.2.patch.txt


 Compact indexes are sorted based on the indexed columns, but we are not using 
 this fact when we access the index.
 To start with, if the index is stored as an RC file, and if the predicate 
 being used to access the index consists of only one non-partition condition 
 using one of the operators ,=,,=,= we could use a binary search (if 
 necessary) to find the block to begin scanning for unfiltered rows, and we 
 could use the result of comparing the value in the column with the constant 
 (this is necessarily the form of a predicate which is optimized using an 
 index) to determine when we have found all the rows which will be unfiltered.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2543) Compact index table's files merged in creation

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2543?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2543:


Attachment: HIVE-2543.1.patch.txt

 Compact index table's files merged in creation
 --

 Key: HIVE-2543
 URL: https://issues.apache.org/jira/browse/HIVE-2543
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2543.1.patch.txt


 When a compact index is built there is the possibility of a merge task at the 
 end of the task tree.  If this happens, the index table's files will no 
 longer be sorted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2534) HiveIndexResult creation fails due to file system issue

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2534:


Attachment: HIVE-2534.1.patch.txt

 HiveIndexResult creation fails due to file system issue
 ---

 Key: HIVE-2534
 URL: https://issues.apache.org/jira/browse/HIVE-2534
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2534.1.patch.txt


 If the file system for temp files differs from the default file system in the 
 conf, HiveIndexResult's constructor fails.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2535) Use sorted nature of compact indexes

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2535:


Attachment: HIVE-2535.1.patch.txt

 Use sorted nature of compact indexes
 

 Key: HIVE-2535
 URL: https://issues.apache.org/jira/browse/HIVE-2535
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2535.1.patch.txt


 Compact indexes are sorted based on the indexed columns, but we are not using 
 this fact when we access the index.
 To start with, if the index is stored as an RC file, and if the predicate 
 being used to access the index consists of only one non-partition condition 
 using one of the operators ,=,,=,= we could use a binary search (if 
 necessary) to find the block to begin scanning for unfiltered rows, and we 
 could use the result of comparing the value in the column with the constant 
 (this is necessarily the form of a predicate which is optimized using an 
 index) to determine when we have found all the rows which will be unfiltered.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2505) Update junit jar in testlibs

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2505:


Attachment: junit-4.10.jar

 Update junit jar in testlibs
 

 Key: HIVE-2505
 URL: https://issues.apache.org/jira/browse/HIVE-2505
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2505.1.patch.txt, junit-4.10.jar




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2502) Add whitelist for hosts used in table/partition locations

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2502:


Attachment: HIVE-2502.3.patch.txt

 Add whitelist for hosts used in table/partition locations
 -

 Key: HIVE-2502
 URL: https://issues.apache.org/jira/browse/HIVE-2502
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2502.1.patch.txt, HIVE-2502.2.patch.txt, 
 HIVE-2502.3.patch.txt


 Add a whitelist of host names that can be checked before creating/altering a 
 table/partition to verify that the location is acceptable.  The whitelist 
 should be empty by default, and should be configurable.  The check should 
 default to pass if there is no host in the location, or the whitelist is 
 empty.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2397) Support with rollup option for group by

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2397?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2397:


Attachment: HIVE-2397.5.patch.txt

 Support with rollup option for group by
 ---

 Key: HIVE-2397
 URL: https://issues.apache.org/jira/browse/HIVE-2397
 Project: Hive
  Issue Type: New Feature
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2397.2.patch.txt, HIVE-2397.3.patch.txt, 
 HIVE-2397.4.patch.txt, HIVE-2397.5.patch.txt


 We should support the ROLLUP operator similar to the way MySQL is 
 implemented. 
 Exerted from MySQL documents:
 mysql SELECT year, country, product, SUM(profit)
 - FROM sales
 - GROUP BY year, country, product WITH ROLLUP;
 +--+-++-+
 | year | country | product| SUM(profit) |
 +--+-++-+
 | 2000 | Finland | Computer   |1500 |
 | 2000 | Finland | Phone  | 100 |
 | 2000 | Finland | NULL   |1600 |
 | 2000 | India   | Calculator | 150 |
 | 2000 | India   | Computer   |1200 |
 | 2000 | India   | NULL   |1350 |
 | 2000 | USA | Calculator |  75 |
 | 2000 | USA | Computer   |1500 |
 | 2000 | USA | NULL   |1575 |
 | 2000 | NULL| NULL   |4525 |
 | 2001 | Finland | Phone  |  10 |
 | 2001 | Finland | NULL   |  10 |
 | 2001 | USA | Calculator |  50 |
 | 2001 | USA | Computer   |2700 |
 | 2001 | USA | TV | 250 |
 | 2001 | USA | NULL   |3000 |
 | 2001 | NULL| NULL   |3010 |
 | NULL | NULL| NULL   |7535 |
 +--+-++-+
 http://dev.mysql.com/doc/refman/5.0/en/group-by-modifiers.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2505) Update junit jar in testlibs

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2505:


Attachment: HIVE-2505.1.patch.txt

 Update junit jar in testlibs
 

 Key: HIVE-2505
 URL: https://issues.apache.org/jira/browse/HIVE-2505
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2505.1.patch.txt




--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2502) Add whitelist for hosts used in table/partition locations

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2502?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2502:


Attachment: HIVE-2502.1.patch.txt

 Add whitelist for hosts used in table/partition locations
 -

 Key: HIVE-2502
 URL: https://issues.apache.org/jira/browse/HIVE-2502
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2502.1.patch.txt


 Add a whitelist of host names that can be checked before creating/altering a 
 table/partition to verify that the location is acceptable.  The whitelist 
 should be empty by default, and should be configurable.  The check should 
 default to pass if there is no host in the location, or the whitelist is 
 empty.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2490) Add reset operation and average time attribute to Metrics MBean.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2490:


Summary: Add reset operation and average time attribute to Metrics MBean.  
(was: Add reset operation to Metrics MBean.)

 Add reset operation and average time attribute to Metrics MBean.
 

 Key: HIVE-2490
 URL: https://issues.apache.org/jira/browse/HIVE-2490
 Project: Hive
  Issue Type: New Feature
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong

 We should add a reset operation to the Metrics MBean, which will set all the 
 counters to 0.
 Note: Deleting the counters from the map of attributes was not suggested 
 because that could break any scripts that get the list of attributes from the 
 bean and then the values of each attribute.  Also, 0 is unlikely to be an 
 actual value for any counter, and it will not break the increment 
 functionality. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2479) Log more Hadoop task counter values in the MapRedStats class.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2479:


Attachment: HIVE-2479.3.patch.txt

 Log more Hadoop task counter values in the MapRedStats class.
 -

 Key: HIVE-2479
 URL: https://issues.apache.org/jira/browse/HIVE-2479
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2479.1.patch.txt, HIVE-2479.2.patch.txt, 
 HIVE-2479.3.patch.txt


 We should log more of the Hadoop task tracker counters in the MapRedStats 
 class, in order to make them available to hooks and improve logging.
 We should make the counters that are logged configurable, so that if 
 different Hadoop versions are used, different counters can be collected.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2479) Log more Hadoop task counter values in the MapRedStats class.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2479:


Attachment: HIVE-2479.4.patch.txt

 Log more Hadoop task counter values in the MapRedStats class.
 -

 Key: HIVE-2479
 URL: https://issues.apache.org/jira/browse/HIVE-2479
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2479.1.patch.txt, HIVE-2479.2.patch.txt, 
 HIVE-2479.3.patch.txt, HIVE-2479.4.patch.txt


 We should log more of the Hadoop task tracker counters in the MapRedStats 
 class, in order to make them available to hooks and improve logging.
 We should make the counters that are logged configurable, so that if 
 different Hadoop versions are used, different counters can be collected.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2479) Log more Hadoop task counter values in the MapRedStats class.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2479:


Description: 
We should log more of the Hadoop task tracker counters in the MapRedStats 
class, in order to make them available to hooks and improve logging.  
Specifically these are the counters we should add:

MAP_SPILL_CPU,
MAP_SPILL_WALLCLOCK,
MAP_SPILL_NUMBER,
MAP_SPILL_BYTES,
MAP_MEM_SORT_CPU,
MAP_MEM_SORT_WALLCLOCK,
MAP_MERGE_CPU,
MAP_MERGE_WALLCLOCK,
REDUCE_SHUFFLE_BYTES,
REDUCE_COPY_WALLCLOCK,
REDUCE_COPY_CPU,
REDUCE_SORT_WALLCLOCK,
REDUCE_SORT_CPU,
MAP_TASK_WALLCLOCK,
REDUCE_TASK_WALLCLOCK,
MAP_INPUT_BYTES

  was:
We should log more of the Hadoop task tracker counters in the MapRedStats 
object, in order to make them available to hooks and improve logging.  
Specifically these are the counters we should add:

MAP_SPILL_CPU,
MAP_SPILL_WALLCLOCK,
MAP_SPILL_NUMBER,
MAP_SPILL_BYTES,
MAP_MEM_SORT_CPU,
MAP_MEM_SORT_WALLCLOCK,
MAP_MERGE_CPU,
MAP_MERGE_WALLCLOCK,
REDUCE_SHUFFLE_BYTES,
REDUCE_COPY_WALLCLOCK,
REDUCE_COPY_CPU,
REDUCE_SORT_WALLCLOCK,
REDUCE_SORT_CPU,
MAP_TASK_WALLCLOCK,
REDUCE_TASK_WALLCLOCK,
MAP_INPUT_BYTES

Summary: Log more Hadoop task counter values in the MapRedStats class.  
(was: Log more Hadoop task counter values in the MapRedStats object.)

 Log more Hadoop task counter values in the MapRedStats class.
 -

 Key: HIVE-2479
 URL: https://issues.apache.org/jira/browse/HIVE-2479
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong

 We should log more of the Hadoop task tracker counters in the MapRedStats 
 class, in order to make them available to hooks and improve logging.  
 Specifically these are the counters we should add:
 MAP_SPILL_CPU,
 MAP_SPILL_WALLCLOCK,
 MAP_SPILL_NUMBER,
 MAP_SPILL_BYTES,
 MAP_MEM_SORT_CPU,
 MAP_MEM_SORT_WALLCLOCK,
 MAP_MERGE_CPU,
 MAP_MERGE_WALLCLOCK,
 REDUCE_SHUFFLE_BYTES,
 REDUCE_COPY_WALLCLOCK,
 REDUCE_COPY_CPU,
 REDUCE_SORT_WALLCLOCK,
 REDUCE_SORT_CPU,
 MAP_TASK_WALLCLOCK,
 REDUCE_TASK_WALLCLOCK,
 MAP_INPUT_BYTES

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2479) Log more Hadoop task counter values in the MapRedStats class.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2479:


Attachment: HIVE-2479.1.patch.txt

 Log more Hadoop task counter values in the MapRedStats class.
 -

 Key: HIVE-2479
 URL: https://issues.apache.org/jira/browse/HIVE-2479
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2479.1.patch.txt


 We should log more of the Hadoop task tracker counters in the MapRedStats 
 class, in order to make them available to hooks and improve logging.  
 Specifically these are the counters we should add:
 MAP_SPILL_CPU,
 MAP_SPILL_WALLCLOCK,
 MAP_SPILL_NUMBER,
 MAP_SPILL_BYTES,
 MAP_MEM_SORT_CPU,
 MAP_MEM_SORT_WALLCLOCK,
 MAP_MERGE_CPU,
 MAP_MERGE_WALLCLOCK,
 REDUCE_SHUFFLE_BYTES,
 REDUCE_COPY_WALLCLOCK,
 REDUCE_COPY_CPU,
 REDUCE_SORT_WALLCLOCK,
 REDUCE_SORT_CPU,
 MAP_TASK_WALLCLOCK,
 REDUCE_TASK_WALLCLOCK,
 MAP_INPUT_BYTES

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2479) Log more Hadoop task counter values in the MapRedStats class.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2479:


Attachment: HIVE-2479.2.patch.txt

 Log more Hadoop task counter values in the MapRedStats class.
 -

 Key: HIVE-2479
 URL: https://issues.apache.org/jira/browse/HIVE-2479
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2479.1.patch.txt, HIVE-2479.2.patch.txt


 We should log more of the Hadoop task tracker counters in the MapRedStats 
 class, in order to make them available to hooks and improve logging.  
 Specifically these are the counters we should add:
 MAP_SPILL_CPU,
 MAP_SPILL_WALLCLOCK,
 MAP_SPILL_NUMBER,
 MAP_SPILL_BYTES,
 MAP_MEM_SORT_CPU,
 MAP_MEM_SORT_WALLCLOCK,
 MAP_MERGE_CPU,
 MAP_MERGE_WALLCLOCK,
 REDUCE_SHUFFLE_BYTES,
 REDUCE_COPY_WALLCLOCK,
 REDUCE_COPY_CPU,
 REDUCE_SORT_WALLCLOCK,
 REDUCE_SORT_CPU,
 MAP_TASK_WALLCLOCK,
 REDUCE_TASK_WALLCLOCK,
 MAP_INPUT_BYTES

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2479) Log more Hadoop task counter values in the MapRedStats class.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2479?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2479:


Description: 
We should log more of the Hadoop task tracker counters in the MapRedStats 
class, in order to make them available to hooks and improve logging.

We should make the counters that are logged configurable, so that if different 
Hadoop versions are used, different counters can be collected.

  was:
We should log more of the Hadoop task tracker counters in the MapRedStats 
class, in order to make them available to hooks and improve logging.  
Specifically these are the counters we should add:

MAP_SPILL_CPU,
MAP_SPILL_WALLCLOCK,
MAP_SPILL_NUMBER,
MAP_SPILL_BYTES,
MAP_MEM_SORT_CPU,
MAP_MEM_SORT_WALLCLOCK,
MAP_MERGE_CPU,
MAP_MERGE_WALLCLOCK,
REDUCE_SHUFFLE_BYTES,
REDUCE_COPY_WALLCLOCK,
REDUCE_COPY_CPU,
REDUCE_SORT_WALLCLOCK,
REDUCE_SORT_CPU,
MAP_TASK_WALLCLOCK,
REDUCE_TASK_WALLCLOCK,
MAP_INPUT_BYTES


 Log more Hadoop task counter values in the MapRedStats class.
 -

 Key: HIVE-2479
 URL: https://issues.apache.org/jira/browse/HIVE-2479
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2479.1.patch.txt, HIVE-2479.2.patch.txt


 We should log more of the Hadoop task tracker counters in the MapRedStats 
 class, in order to make them available to hooks and improve logging.
 We should make the counters that are logged configurable, so that if 
 different Hadoop versions are used, different counters can be collected.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2472) Metastore statistics are not being updated for CTAS queries.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2472?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2472:


Attachment: HIVE-2472.1.patch.txt

 Metastore statistics are not being updated for CTAS queries.
 

 Key: HIVE-2472
 URL: https://issues.apache.org/jira/browse/HIVE-2472
 Project: Hive
  Issue Type: Bug
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2472.1.patch.txt


 We need to add a Statistics task at the end of a CTAS query in order to 
 update the metastore statistics for the table being created.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (HIVE-2471) Add timestamp column with index to the partition stats table.

[Kevin Wilfong (Updated) (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-2471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Kevin Wilfong updated HIVE-2471:


Attachment: HIVE-2471.1.patch.txt

 Add timestamp column with index to the partition stats table.
 -

 Key: HIVE-2471
 URL: https://issues.apache.org/jira/browse/HIVE-2471
 Project: Hive
  Issue Type: Improvement
Reporter: Kevin Wilfong
Assignee: Kevin Wilfong
 Attachments: HIVE-2471.1.patch.txt


 Occasionally, when entries are added to the partition stats table the program 
 is halted before it can delete those entries, by an exception, keyboard 
 interrupt, etc.  These build up to the point where the table gets very large, 
 and it hurts the performance of the update statement which is often called.  
 In order to fix this, I am adding a column to the table which is 
 auto-populated with the current timestamp.  I am also adding an index on this 
 column.  This will allow us to create scripts that go through periodically 
 and clean out old entries from the table.  The index will help to keep the 
 runtime of these scripts short, and hence reduce the amount of time they need 
 to lock the table/indexes for.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: 
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira