[jira] [Updated] (HIVE-5690) Support subquery for single sourced multi query

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-5690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-5690:

Attachment: HIVE-5690.12.patch.txt

Another rebasing on trunk

> Support subquery for single sourced multi query
> ---
>
> Key: HIVE-5690
> URL: https://issues.apache.org/jira/browse/HIVE-5690
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
> Attachments: D13791.1.patch, HIVE-5690.10.patch.txt, 
> HIVE-5690.11.patch.txt, HIVE-5690.12.patch.txt, HIVE-5690.2.patch.txt, 
> HIVE-5690.3.patch.txt, HIVE-5690.4.patch.txt, HIVE-5690.5.patch.txt, 
> HIVE-5690.6.patch.txt, HIVE-5690.7.patch.txt, HIVE-5690.8.patch.txt, 
> HIVE-5690.9.patch.txt
>
>
> Single sourced multi (insert) query is very useful for various ETL processes 
> but it does not allow subqueries included. For example, 
> {noformat}
> explain from src 
> insert overwrite table x1 select * from (select distinct key,value) b order 
> by key
> insert overwrite table x2 select * from (select distinct key,value) c order 
> by value;
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (HIVE-7858) Parquet compression should be configurable via table property

[Ferdinand Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-7858 started by Ferdinand Xu.
--
> Parquet compression should be configurable via table property
> -
>
> Key: HIVE-7858
> URL: https://issues.apache.org/jira/browse/HIVE-7858
> Project: Hive
>  Issue Type: Improvement
>Reporter: Brock Noland
>Assignee: Ferdinand Xu
>
> ORC supports the orc.compress table property:
> https://cwiki.apache.org/confluence/display/Hive/LanguageManual+ORC
> {noformat}
> create table Addresses (
>   name string,
>   street string,
>   city string,
>   state string,
>   zip int
> ) stored as orc tblproperties ("orc.compress"="NONE");
> {noformat}
> I think it'd be great to support the same for Parquet.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-8071) hive shell tries to write hive-exec.jar for each run

[Rajesh Balamohan (JIRA)]

Rajesh Balamohan created HIVE-8071:
--

 Summary: hive shell tries to write hive-exec.jar for each run
 Key: HIVE-8071
 URL: https://issues.apache.org/jira/browse/HIVE-8071
 Project: Hive
  Issue Type: Bug
Reporter: Rajesh Balamohan
Assignee: Rajesh Balamohan




For every run of the hive CLI there is a delay for the shell startup

14/07/31 23:07:19 INFO Configuration.deprecation: fs.default.name is 
deprecated. Instead, use fs.defaultFS
14/07/31 23:07:19 INFO tez.DagUtils: Hive jar directory is 
hdfs://mac-10:8020/user/gopal/apps/2014-Jul-31/hive/
14/07/31 23:07:19 INFO tez.DagUtils: Localizing resource because it does not 
exist: 
file:/home/gopal/tez-autobuild/dist/hive/lib/hive-exec-0.14.0-SNAPSHOT.jar to 
dest: 
hdfs://mac-10:8020/user/gopal/apps/2014-Jul-31/hive/hive-exec-0.14.0-SNAPSHOTde1f82f0b5561d3db9e3080dfb2897210a3bda4ca5e7b14e881e381115837fd8.
jar
14/07/31 23:07:19 INFO tez.DagUtils: Looks like another thread is writing the 
same file will wait.
14/07/31 23:07:19 INFO tez.DagUtils: Number of wait attempts: 5. Wait interval: 
5000
14/07/31 23:07:19 INFO tez.DagUtils: Resource modification time: 1406870512963
14/07/31 23:07:20 INFO tez.TezSessionState: Opening new Tez Session (id: 
02d6b558-44cc-4182-b2f2-6a37ffdd25d2, scratch dir: 
hdfs://mac-10:8020/tmp/hive-gopal/_tez_session_dir/02d6b558-44cc-4182-b2f2-6a37ffdd25d2)

Traced this to a method which does PRIVATE LRs - this is marked as PRIVATE even 
if it is from a common install dir.
{code}
 public LocalResource localizeResource(Path src, Path dest, Configuration conf)
throws IOException {

return createLocalResource(destFS, dest, LocalResourceType.FILE,
LocalResourceVisibility.PRIVATE);
  }
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7981) alias of compound aggregation functions fails in having clause

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7981:

Attachment: HIVE-7981.2.patch.txt

> alias of compound aggregation functions fails in having clause
> --
>
> Key: HIVE-7981
> URL: https://issues.apache.org/jira/browse/HIVE-7981
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.13.0
>Reporter: eyal gruss
>Assignee: Navis
>Priority: Minor
> Attachments: HIVE-7981.1.patch.txt, HIVE-7981.2.patch.txt
>
>
> hive> select max(time)-min(time) as span from mytable group by name having 
> span>0;
> FAILED: SemanticException [Error 10025]: Line 1:92 Expression not in GROUP BY 
> key '0'



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8062) Stats collection for columns fails on a partitioned table with null values in partitioning column

[Pengcheng Xiong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131174#comment-14131174
 ] 

Pengcheng Xiong commented on HIVE-8062:
---

+1

> Stats collection for columns fails on a partitioned table with null values in 
> partitioning column
> -
>
> Key: HIVE-8062
> URL: https://issues.apache.org/jira/browse/HIVE-8062
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Affects Versions: 0.14.0
>Reporter: Deepesh Khandelwal
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8062.patch
>
>
> Steps to reproduce:
> 1. Create a data file abc.txt with the following contents:
> {noformat}
> a,1
> b,
> {noformat}
> 2. Use the Hive CLI to create and load the partitioned table:
> {noformat}
> hive> create table abc(a string, b int);
> OK
> Time taken: 0.272 seconds
> hive> load data local inpath 'abc.txt' into table abc;
> Loading data to table default.abc
> Table default.abc stats: [numFiles=1, numRows=0, totalSize=7, rawDataSize=0]
> OK
> Time taken: 0.463 seconds
> hive> create table abc1(a string) partitioned by (b int);
> OK
> Time taken: 0.098 seconds
> hive> set hive.exec.dynamic.partition.mode=nonstrict;
> hive> insert overwrite table abc1 partition (b) select a, b from abc;
> Query ID = hrt_qa_20140911210909_1200fae7-1e18-4e0d-b74f-040453c27cff
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: Executing on YARN cluster with App id 
> application_1410457588978_0063)
> Map 1: -/-Reducer 2: 0/1
> Map 1: 0/1Reducer 2: 0/1
> Map 1: 0(+1)/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 0(+1)/1
> Map 1: 1/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 1/1
> Status: Finished successfully
> Loading data to table default.abc1 partition (b=null)
>   Loading partition {b=__HIVE_DEFAULT_PARTITION__}
> Partition default.abc1{b=__HIVE_DEFAULT_PARTITION__} stats: [numFiles=1, 
> numRows=2, totalSize=7, rawDataSize=5]
> OK
> Time taken: 7.49 seconds
> {noformat}
> 3. Now run the analyze statistics command for columns:
> {noformat}
> hive> analyze table abc1 partition (b) compute statistics for columns;
> Query ID = hrt_qa_20140911211010_440bdb4a-6a0d-496b-9d2e-5fc84db3d0ee
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: Executing on YARN cluster with App id 
> application_1410457588978_0063)
> Map 1: 0(+1)/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 0(+1)/1
> Map 1: 1/1Reducer 2: 1/1
> Status: Finished successfully
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.ColumnStatsTask
> {noformat}
> The analyze statistics for columns fails.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8070) TestHWIServer failed due to wrong references to war and properties file

[Bing Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bing Li updated HIVE-8070:
--
Attachment: HIVE-8070.1.patch

> TestHWIServer failed due to wrong references to war and properties file
> ---
>
> Key: HIVE-8070
> URL: https://issues.apache.org/jira/browse/HIVE-8070
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 0.13.1
>Reporter: Bing Li
>Assignee: Bing Li
> Fix For: 0.14.0
>
> Attachments: HIVE-8070.1.patch
>
>
> In testServerInit() method of that test class, it's still using 
> build.properties to retrieve the version # for the war file name



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Work started] (HIVE-8070) TestHWIServer failed due to wrong references to war and properties file

[Bing Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Work on HIVE-8070 started by Bing Li.
-
> TestHWIServer failed due to wrong references to war and properties file
> ---
>
> Key: HIVE-8070
> URL: https://issues.apache.org/jira/browse/HIVE-8070
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 0.13.1
>Reporter: Bing Li
>Assignee: Bing Li
> Fix For: 0.14.0
>
> Attachments: HIVE-8070.1.patch
>
>
> In testServerInit() method of that test class, it's still using 
> build.properties to retrieve the version # for the war file name



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8070) TestHWIServer failed due to wrong references to war and properties file

[Bing Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8070?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Bing Li updated HIVE-8070:
--
Status: Patch Available  (was: In Progress)

The patch is generated for trunk

> TestHWIServer failed due to wrong references to war and properties file
> ---
>
> Key: HIVE-8070
> URL: https://issues.apache.org/jira/browse/HIVE-8070
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 0.13.1
>Reporter: Bing Li
>Assignee: Bing Li
> Fix For: 0.14.0
>
> Attachments: HIVE-8070.1.patch
>
>
> In testServerInit() method of that test class, it's still using 
> build.properties to retrieve the version # for the war file name



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8062) Stats collection for columns fails on a partitioned table with null values in partitioning column

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8062?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131169#comment-14131169
 ] 

Gunther Hagleitner commented on HIVE-8062:
--

LGTM +1

> Stats collection for columns fails on a partitioned table with null values in 
> partitioning column
> -
>
> Key: HIVE-8062
> URL: https://issues.apache.org/jira/browse/HIVE-8062
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Affects Versions: 0.14.0
>Reporter: Deepesh Khandelwal
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8062.patch
>
>
> Steps to reproduce:
> 1. Create a data file abc.txt with the following contents:
> {noformat}
> a,1
> b,
> {noformat}
> 2. Use the Hive CLI to create and load the partitioned table:
> {noformat}
> hive> create table abc(a string, b int);
> OK
> Time taken: 0.272 seconds
> hive> load data local inpath 'abc.txt' into table abc;
> Loading data to table default.abc
> Table default.abc stats: [numFiles=1, numRows=0, totalSize=7, rawDataSize=0]
> OK
> Time taken: 0.463 seconds
> hive> create table abc1(a string) partitioned by (b int);
> OK
> Time taken: 0.098 seconds
> hive> set hive.exec.dynamic.partition.mode=nonstrict;
> hive> insert overwrite table abc1 partition (b) select a, b from abc;
> Query ID = hrt_qa_20140911210909_1200fae7-1e18-4e0d-b74f-040453c27cff
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: Executing on YARN cluster with App id 
> application_1410457588978_0063)
> Map 1: -/-Reducer 2: 0/1
> Map 1: 0/1Reducer 2: 0/1
> Map 1: 0(+1)/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 0(+1)/1
> Map 1: 1/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 1/1
> Status: Finished successfully
> Loading data to table default.abc1 partition (b=null)
>   Loading partition {b=__HIVE_DEFAULT_PARTITION__}
> Partition default.abc1{b=__HIVE_DEFAULT_PARTITION__} stats: [numFiles=1, 
> numRows=2, totalSize=7, rawDataSize=5]
> OK
> Time taken: 7.49 seconds
> {noformat}
> 3. Now run the analyze statistics command for columns:
> {noformat}
> hive> analyze table abc1 partition (b) compute statistics for columns;
> Query ID = hrt_qa_20140911211010_440bdb4a-6a0d-496b-9d2e-5fc84db3d0ee
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: Executing on YARN cluster with App id 
> application_1410457588978_0063)
> Map 1: 0(+1)/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 0(+1)/1
> Map 1: 1/1Reducer 2: 1/1
> Status: Finished successfully
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.ColumnStatsTask
> {noformat}
> The analyze statistics for columns fails.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7868) AvroSerDe error handling could be improved

[Ferdinand Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ferdinand Xu updated HIVE-7868:
---
Attachment: HIVE-7868.1.patch

> AvroSerDe error handling could be improved
> --
>
> Key: HIVE-7868
> URL: https://issues.apache.org/jira/browse/HIVE-7868
> Project: Hive
>  Issue Type: Improvement
>Reporter: Brock Noland
>Assignee: Ferdinand Xu
> Attachments: HIVE-7868.1.patch
>
>
> When an Avro schema is invalid, AvroSerDe returns an error message instead of 
> throwing an exception. This is described in 
> {{AvroSerdeUtils.determineSchemaOrReturnErrorSchema}}:
> {noformat}
>   /**
>* Attempt to determine the schema via the usual means, but do not throw
>* an exception if we fail.  Instead, signal failure via a special
>* schema.  This is used because Hive calls init on the serde during
>* any call, including calls to update the serde properties, meaning
>* if the serde is in a bad state, there is no way to update that state.
>*/
> {noformat}
> I believe we should find a way to provide a better experience to our users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8070) TestHWIServer failed due to wrong references to war and properties file

[Bing Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131159#comment-14131159
 ] 

Bing Li commented on HIVE-8070:
---

This JIRA is blocked by HIVE-7233

> TestHWIServer failed due to wrong references to war and properties file
> ---
>
> Key: HIVE-8070
> URL: https://issues.apache.org/jira/browse/HIVE-8070
> Project: Hive
>  Issue Type: Test
>  Components: Tests
>Affects Versions: 0.13.1
>Reporter: Bing Li
>Assignee: Bing Li
> Fix For: 0.14.0
>
>
> In testServerInit() method of that test class, it's still using 
> build.properties to retrieve the version # for the war file name



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7868) AvroSerDe error handling could be improved

[Ferdinand Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ferdinand Xu updated HIVE-7868:
---
Attachment: (was: HIVE-7868.patch)

> AvroSerDe error handling could be improved
> --
>
> Key: HIVE-7868
> URL: https://issues.apache.org/jira/browse/HIVE-7868
> Project: Hive
>  Issue Type: Improvement
>Reporter: Brock Noland
>Assignee: Ferdinand Xu
>
> When an Avro schema is invalid, AvroSerDe returns an error message instead of 
> throwing an exception. This is described in 
> {{AvroSerdeUtils.determineSchemaOrReturnErrorSchema}}:
> {noformat}
>   /**
>* Attempt to determine the schema via the usual means, but do not throw
>* an exception if we fail.  Instead, signal failure via a special
>* schema.  This is used because Hive calls init on the serde during
>* any call, including calls to update the serde properties, meaning
>* if the serde is in a bad state, there is no way to update that state.
>*/
> {noformat}
> I believe we should find a way to provide a better experience to our users.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8052) Vectorization: min() on TimeStamp datatype fails with error "Vector aggregate not implemented: min for type: TIMESTAMP"

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131157#comment-14131157
 ] 

Hive QA commented on HIVE-8052:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668198/HIVE-8052.02.patch

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6197 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_dynpart_sort_opt_vectorization
org.apache.hive.hcatalog.pig.TestOrcHCatLoader.testReadDataPrimitiveTypes
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/753/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/753/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-753/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668198

> Vectorization: min() on TimeStamp datatype fails with error "Vector aggregate 
> not implemented: min for type: TIMESTAMP"
> ---
>
> Key: HIVE-8052
> URL: https://issues.apache.org/jira/browse/HIVE-8052
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8052.01.patch, HIVE-8052.02.patch
>
>
> Changes in HIVE-5760 to make explicit when timestamp and date can be 
> vectorized as Long were accidentally to strict for min, max, count, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-8070) TestHWIServer failed due to wrong references to war and properties file

[Bing Li (JIRA)]

Bing Li created HIVE-8070:
-

 Summary: TestHWIServer failed due to wrong references to war and 
properties file
 Key: HIVE-8070
 URL: https://issues.apache.org/jira/browse/HIVE-8070
 Project: Hive
  Issue Type: Test
  Components: Tests
Affects Versions: 0.13.1
Reporter: Bing Li
Assignee: Bing Li
 Fix For: 0.14.0


In testServerInit() method of that test class, it's still using 
build.properties to retrieve the version # for the war file name



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8069) CBO: RowResolver after SubQuery predicate handling should be reset to outer query block RR

[Laljo John Pullokkaran (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131153#comment-14131153
 ] 

Laljo John Pullokkaran commented on HIVE-8069:
--

+1

> CBO: RowResolver after SubQuery predicate handling should be reset to outer 
> query block RR
> --
>
> Key: HIVE-8069
> URL: https://issues.apache.org/jira/browse/HIVE-8069
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-8069.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8067) set default table permissions for table owner to have all privileges

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131152#comment-14131152
 ] 

Lefty Leverenz commented on HIVE-8067:
--

Agreed, and the description should list all the possible values.  

Patch 1 adds defaults INSERT, SELECT, UPDATE, and DELETE but the old 
description says 'An example like "select,drop" will grant select and drop 
privilege to the owner of the table' and the wiki lists DROP instead of DELETE.

Is the wiki list complete and accurate? 

* ALL, ALTER, UPDATE, CREATE (irrelevant here), DROP, INDEX (not implemented), 
LOCK, SELECT, SHOW_DATABASE (irrelevant here)
* [Hive Default Authorization (Legacy Mode) -- Privileges | 
https://cwiki.apache.org/confluence/display/Hive/Hive+Default+Authorization+-+Legacy+Mode#HiveDefaultAuthorization-LegacyMode-Privileges]

> set default table permissions for table owner to have all privileges
> 
>
> Key: HIVE-8067
> URL: https://issues.apache.org/jira/browse/HIVE-8067
> Project: Hive
>  Issue Type: Bug
>  Components: Authorization, SQLStandardAuthorization
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Attachments: HIVE-8067.1.patch
>
>
> When tables are created using without SQLStandards based authorization being 
> enabled, the table owner does not have any privileges on the table.
> It makes sense to set the default privileges to be compatible with sql 
> standard mode's expected default privileges for the owner of the table, 
> instead of setting no privileges at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8069) CBO: RowResolver after SubQuery predicate handling should be reset to outer query block RR

[Harish Butani (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8069?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Harish Butani updated HIVE-8069:

Attachment: HIVE-8069.1.patch

> CBO: RowResolver after SubQuery predicate handling should be reset to outer 
> query block RR
> --
>
> Key: HIVE-8069
> URL: https://issues.apache.org/jira/browse/HIVE-8069
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Harish Butani
>Assignee: Harish Butani
> Attachments: HIVE-8069.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-8069) CBO: RowResolver after SubQuery predicate handling should be reset to outer query block RR

[Harish Butani (JIRA)]

Harish Butani created HIVE-8069:
---

 Summary: CBO: RowResolver after SubQuery predicate handling should 
be reset to outer query block RR
 Key: HIVE-8069
 URL: https://issues.apache.org/jira/browse/HIVE-8069
 Project: Hive
  Issue Type: Sub-task
Reporter: Harish Butani
Assignee: Harish Butani






--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Query regarding Hive configuration on windows OS and make connection through asp.net

[Kapil Khare]

Hi ,

I want to configure Hive-0.13.1 on my windows7 machine ,But some error appear 
during  configuration .I already install hadoop-2.5.0 and cygwin64 terminal on 
my machine. Both are working fine.
For hive configuration on windows there is no specific blog or post available 
on internet. So I need your help or some steps how we can configuring 
Hive-0.13.1 on windows machine and how will we make ODBC connection between 
Hive to asp.net application for both query and Data Cubs .

Please suggest me the steps.

Thanks,
Kapil Khare
Team Lead
Helm360
Phone: +91-120-499 3300
Mobile: +91- 9718012939
A-16, Sector 16 | Noida, UP, India 201 301
kkh...@helm360.com | 
www.helm360.com


[Description: Description: Description: logox80]

Query regarding Hive configuration on windows OS and make connection through asp.net

[Kapil Khare]

Hi ,

I want to configure Hive-0.13.1 on my windows7 machine ,But some error appear 
during  configuration .I already install hadoop-2.5.0 and cygwin64 terminal on 
my machine. Both are working fine.
For hive configuration on windows there is no specific blog or post available 
on internet. So I need your help or some steps how we can configuring 
Hive-0.13.1 on windows machine and how will we make ODBC connection between 
Hive to asp.net application for both query and Data Cubs .

Please suggest me the steps.

Thanks,
Kapil Khare
Team Lead
Helm360
Phone: +91-120-499 3300
Mobile: +91- 9718012939
A-16, Sector 16 | Noida, UP, India 201 301
kkh...@helm360.com | 
www.helm360.com


[Description: Description: Description: logox80]

[jira] [Updated] (HIVE-8017) Use HiveKey instead of BytesWritable as key type of the pair RDD [Spark Branch]

[Rui Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rui Li updated HIVE-8017:
-
Attachment: HIVE-8017.5-spark.patch

Update the golden file for union_remove_25

> Use HiveKey instead of BytesWritable as key type of the pair RDD [Spark 
> Branch]
> ---
>
> Key: HIVE-8017
> URL: https://issues.apache.org/jira/browse/HIVE-8017
> Project: Hive
>  Issue Type: Sub-task
>  Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-8017-spark.patch, HIVE-8017.2-spark.patch, 
> HIVE-8017.3-spark.patch, HIVE-8017.4-spark.patch, HIVE-8017.5-spark.patch
>
>
> HiveKey should be used as the key type because it holds the hash code for 
> partitioning. While BytesWritable serves partitioning well for simple cases, 
> we have to use {{HiveKey.hashCode}} for more complicated ones, e.g. join, 
> bucketed table, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Review Request 25571: Stats collection for columns fails on a partitioned table with null values in partitioning column

[Ashutosh Chauhan]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25571/
---

Review request for hive.


Bugs: HIVE-8062
https://issues.apache.org/jira/browse/HIVE-8062


Repository: hive-git


Description
---

Stats collection for columns fails on a partitioned table with null values in 
partitioning column


Diffs
-

  ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 176a593 
  ql/src/test/queries/clientpositive/stats_only_null.q b47bc48 
  ql/src/test/results/clientpositive/stats_only_null.q.out 063da37 

Diff: https://reviews.apache.org/r/25571/diff/


Testing
---

Added new test.


Thanks,

Ashutosh Chauhan

[jira] [Updated] (HIVE-8062) Stats collection for columns fails on a partitioned table with null values in partitioning column

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-8062:
---
Assignee: Ashutosh Chauhan
  Status: Patch Available  (was: Open)

> Stats collection for columns fails on a partitioned table with null values in 
> partitioning column
> -
>
> Key: HIVE-8062
> URL: https://issues.apache.org/jira/browse/HIVE-8062
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Affects Versions: 0.14.0
>Reporter: Deepesh Khandelwal
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8062.patch
>
>
> Steps to reproduce:
> 1. Create a data file abc.txt with the following contents:
> {noformat}
> a,1
> b,
> {noformat}
> 2. Use the Hive CLI to create and load the partitioned table:
> {noformat}
> hive> create table abc(a string, b int);
> OK
> Time taken: 0.272 seconds
> hive> load data local inpath 'abc.txt' into table abc;
> Loading data to table default.abc
> Table default.abc stats: [numFiles=1, numRows=0, totalSize=7, rawDataSize=0]
> OK
> Time taken: 0.463 seconds
> hive> create table abc1(a string) partitioned by (b int);
> OK
> Time taken: 0.098 seconds
> hive> set hive.exec.dynamic.partition.mode=nonstrict;
> hive> insert overwrite table abc1 partition (b) select a, b from abc;
> Query ID = hrt_qa_20140911210909_1200fae7-1e18-4e0d-b74f-040453c27cff
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: Executing on YARN cluster with App id 
> application_1410457588978_0063)
> Map 1: -/-Reducer 2: 0/1
> Map 1: 0/1Reducer 2: 0/1
> Map 1: 0(+1)/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 0(+1)/1
> Map 1: 1/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 1/1
> Status: Finished successfully
> Loading data to table default.abc1 partition (b=null)
>   Loading partition {b=__HIVE_DEFAULT_PARTITION__}
> Partition default.abc1{b=__HIVE_DEFAULT_PARTITION__} stats: [numFiles=1, 
> numRows=2, totalSize=7, rawDataSize=5]
> OK
> Time taken: 7.49 seconds
> {noformat}
> 3. Now run the analyze statistics command for columns:
> {noformat}
> hive> analyze table abc1 partition (b) compute statistics for columns;
> Query ID = hrt_qa_20140911211010_440bdb4a-6a0d-496b-9d2e-5fc84db3d0ee
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: Executing on YARN cluster with App id 
> application_1410457588978_0063)
> Map 1: 0(+1)/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 0(+1)/1
> Map 1: 1/1Reducer 2: 1/1
> Status: Finished successfully
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.ColumnStatsTask
> {noformat}
> The analyze statistics for columns fails.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8062) Stats collection for columns fails on a partitioned table with null values in partitioning column

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8062?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-8062:
---
Attachment: HIVE-8062.patch

> Stats collection for columns fails on a partitioned table with null values in 
> partitioning column
> -
>
> Key: HIVE-8062
> URL: https://issues.apache.org/jira/browse/HIVE-8062
> Project: Hive
>  Issue Type: Bug
>  Components: Statistics
>Affects Versions: 0.14.0
>Reporter: Deepesh Khandelwal
> Attachments: HIVE-8062.patch
>
>
> Steps to reproduce:
> 1. Create a data file abc.txt with the following contents:
> {noformat}
> a,1
> b,
> {noformat}
> 2. Use the Hive CLI to create and load the partitioned table:
> {noformat}
> hive> create table abc(a string, b int);
> OK
> Time taken: 0.272 seconds
> hive> load data local inpath 'abc.txt' into table abc;
> Loading data to table default.abc
> Table default.abc stats: [numFiles=1, numRows=0, totalSize=7, rawDataSize=0]
> OK
> Time taken: 0.463 seconds
> hive> create table abc1(a string) partitioned by (b int);
> OK
> Time taken: 0.098 seconds
> hive> set hive.exec.dynamic.partition.mode=nonstrict;
> hive> insert overwrite table abc1 partition (b) select a, b from abc;
> Query ID = hrt_qa_20140911210909_1200fae7-1e18-4e0d-b74f-040453c27cff
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: Executing on YARN cluster with App id 
> application_1410457588978_0063)
> Map 1: -/-Reducer 2: 0/1
> Map 1: 0/1Reducer 2: 0/1
> Map 1: 0(+1)/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 0(+1)/1
> Map 1: 1/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 1/1
> Status: Finished successfully
> Loading data to table default.abc1 partition (b=null)
>   Loading partition {b=__HIVE_DEFAULT_PARTITION__}
> Partition default.abc1{b=__HIVE_DEFAULT_PARTITION__} stats: [numFiles=1, 
> numRows=2, totalSize=7, rawDataSize=5]
> OK
> Time taken: 7.49 seconds
> {noformat}
> 3. Now run the analyze statistics command for columns:
> {noformat}
> hive> analyze table abc1 partition (b) compute statistics for columns;
> Query ID = hrt_qa_20140911211010_440bdb4a-6a0d-496b-9d2e-5fc84db3d0ee
> Total jobs = 1
> Launching Job 1 out of 1
> Status: Running (application id: Executing on YARN cluster with App id 
> application_1410457588978_0063)
> Map 1: 0(+1)/1Reducer 2: 0/1
> Map 1: 1/1Reducer 2: 0(+1)/1
> Map 1: 1/1Reducer 2: 1/1
> Status: Finished successfully
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.ColumnStatsTask
> {noformat}
> The analyze statistics for columns fails.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HIVE-8043) Support merging small files [Spark Branch]

[Rui Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8043?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rui Li reassigned HIVE-8043:


Assignee: Rui Li

> Support merging small files [Spark Branch]
> --
>
> Key: HIVE-8043
> URL: https://issues.apache.org/jira/browse/HIVE-8043
> Project: Hive
>  Issue Type: Task
>  Components: Spark
>Reporter: Xuefu Zhang
>Assignee: Rui Li
>  Labels: Spark-M1
>
> Hive currently supports merging small files with MR as the execution engine. 
> There are options available for this, such as 
> {code}
> hive.merge.mapfiles
> hive.merge.mapredfiles
> {code}
> Hive.merge.sparkfiles is already introduced in HIVE-7810. To make it work, we 
> might need a little more research and design on this.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-649) [UDF] now() for getting current time

[Navis (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-649?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131129#comment-14131129
 ] 

Navis commented on HIVE-649:


[~lars_francke]  For the query below, should it be the same value?
{noformat}
select now(), now();
{noformat}

> [UDF] now() for getting current time
> 
>
> Key: HIVE-649
> URL: https://issues.apache.org/jira/browse/HIVE-649
> Project: Hive
>  Issue Type: New Feature
>  Components: UDF
>Reporter: Min Zhou
>Assignee: Lars Francke
> Attachments: HIVE-649.2.patch, HIVE-649.patch
>
>
> http://dev.mysql.com/doc/refman/5.1/en/date-and-time-functions.html#function_now



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8017) Use HiveKey instead of BytesWritable as key type of the pair RDD [Spark Branch]

[Rui Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131126#comment-14131126
 ] 

Rui Li commented on HIVE-8017:
--

[~xuefuz] OK, I'll do that.
BTW, do you think we need a JIRA to track this difference so we can find the 
cause when we have time?

> Use HiveKey instead of BytesWritable as key type of the pair RDD [Spark 
> Branch]
> ---
>
> Key: HIVE-8017
> URL: https://issues.apache.org/jira/browse/HIVE-8017
> Project: Hive
>  Issue Type: Sub-task
>  Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-8017-spark.patch, HIVE-8017.2-spark.patch, 
> HIVE-8017.3-spark.patch, HIVE-8017.4-spark.patch
>
>
> HiveKey should be used as the key type because it holds the hash code for 
> partitioning. While BytesWritable serves partitioning well for simple cases, 
> we have to use {{HiveKey.hashCode}} for more complicated ones, e.g. join, 
> bucketed table, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7325) Support non-constant expressions for MAP type indices.

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7325:

Attachment: HIVE-7325.3.patch.txt

> Support non-constant expressions for MAP type indices.
> --
>
> Key: HIVE-7325
> URL: https://issues.apache.org/jira/browse/HIVE-7325
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.13.1
>Reporter: Mala Chikka Kempanna
>Assignee: Navis
> Fix For: 0.14.0
>
> Attachments: HIVE-7325.1.patch.txt, HIVE-7325.2.patch.txt, 
> HIVE-7325.3.patch.txt
>
>
> Here is my sample:
> {code}
> CREATE TABLE RECORD(RecordID string, BatchDate string, Country string) 
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' 
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,D:BatchDate,D:Country") 
> TBLPROPERTIES ("hbase.table.name" = "RECORD"); 
> CREATE TABLE KEY_RECORD(KeyValue String, RecordId map) 
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' 
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key, K:") 
> TBLPROPERTIES ("hbase.table.name" = "KEY_RECORD"); 
> {code}
> The following join statement doesn't work. 
> {code}
> SELECT a.*, b.* from KEY_RECORD a join RECORD b 
> WHERE a.RecordId[b.RecordID] is not null;
> {code}
> FAILED: SemanticException 2:16 Non-constant expression for map indexes not 
> supported. Error encountered near token 'RecordID' 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8017) Use HiveKey instead of BytesWritable as key type of the pair RDD [Spark Branch]

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8017?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131121#comment-14131121
 ] 

Xuefu Zhang commented on HIVE-8017:
---

[~ruili] I think it might be better to update union_remove_25.q, so we will see 
one less failure every time the test runs. What do you think?

> Use HiveKey instead of BytesWritable as key type of the pair RDD [Spark 
> Branch]
> ---
>
> Key: HIVE-8017
> URL: https://issues.apache.org/jira/browse/HIVE-8017
> Project: Hive
>  Issue Type: Sub-task
>  Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
> Attachments: HIVE-8017-spark.patch, HIVE-8017.2-spark.patch, 
> HIVE-8017.3-spark.patch, HIVE-8017.4-spark.patch
>
>
> HiveKey should be used as the key type because it holds the hash code for 
> partitioning. While BytesWritable serves partitioning well for simple cases, 
> we have to use {{HiveKey.hashCode}} for more complicated ones, e.g. join, 
> bucketed table, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 23352: Support non-constant expressions for MAP type indices.

[Navis Ryu]

> On Sept. 9, 2014, 6:23 p.m., Jason Dere wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java, line 
> > 768
> > 
> >
> > could we also use implicitConvertable() here?

Will introduce more work to be done in index udf. I'll update that.


- Navis


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23352/#review52751
---


On July 9, 2014, 6:57 a.m., Navis Ryu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/23352/
> ---
> 
> (Updated July 9, 2014, 6:57 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-7325
> https://issues.apache.org/jira/browse/HIVE-7325
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> Here is my sample:
> {code}
> CREATE TABLE RECORD(RecordID string, BatchDate string, Country string) 
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' 
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key,D:BatchDate,D:Country") 
> TBLPROPERTIES ("hbase.table.name" = "RECORD"); 
> 
> 
> CREATE TABLE KEY_RECORD(KeyValue String, RecordId map) 
> STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' 
> WITH SERDEPROPERTIES ("hbase.columns.mapping" = ":key, K:") 
> TBLPROPERTIES ("hbase.table.name" = "KEY_RECORD"); 
> {code}
> The following join statement doesn't work. 
> {code}
> SELECT a.*, b.* from KEY_RECORD a join RECORD b 
> WHERE a.RecordId[b.RecordID] is not null;
> {code}
> FAILED: SemanticException 2:16 Non-constant expression for map indexes not 
> supported. Error encountered near token 'RecordID' 
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/ErrorMsg.java 9889cfe 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/TypeCheckProcFactory.java 
> e44f5ae 
>   ql/src/test/queries/clientpositive/array_map_access_nonconstant.q 
> PRE-CREATION 
>   ql/src/test/queries/negative/invalid_list_index.q c40f079 
>   ql/src/test/queries/negative/invalid_list_index2.q 99d0b3d 
>   ql/src/test/queries/negative/invalid_map_index2.q 5828f07 
>   ql/src/test/results/clientpositive/array_map_access_nonconstant.q.out 
> PRE-CREATION 
>   ql/src/test/results/compiler/errors/invalid_list_index.q.out a4179cd 
>   ql/src/test/results/compiler/errors/invalid_list_index2.q.out aaa9455 
>   ql/src/test/results/compiler/errors/invalid_map_index2.q.out edc9bda 
>   
> serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/PrimitiveObjectInspectorUtils.java
>  5ccacf1 
> 
> Diff: https://reviews.apache.org/r/23352/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Navis Ryu
> 
>

[jira] [Commented] (HIVE-7156) Group-By operator stat-annotation only uses distinct approx to generate rollups

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-7156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131099#comment-14131099
 ] 

Hive QA commented on HIVE-7156:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668171/HIVE-7156.3.patch

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 6198 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver_dynamic_partition_pruning
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/752/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/752/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-752/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668171

> Group-By operator stat-annotation only uses distinct approx to generate 
> rollups
> ---
>
> Key: HIVE-7156
> URL: https://issues.apache.org/jira/browse/HIVE-7156
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 0.14.0
>Reporter: Gopal V
>Assignee: Prasanth J
> Attachments: HIVE-7156.1.patch, HIVE-7156.2.patch, HIVE-7156.3.patch
>
>
> The stats annotation for a group-by only annotates the reduce-side row-count 
> with the distinct values.
> The map-side gets the row-count as the rows output instead of distinct * 
> parallelism, while the reducer side gets the correct parallelism.
> {code}
> hive> explain select distinct L_SHIPDATE from lineitem;
>   Vertices:
> Map 1 
> Map Operator Tree:
> TableScan
>   alias: lineitem
>   Statistics: Num rows: 589709 Data size: 4745677733354 
> Basic stats: COMPLETE Column stats: COMPLETE
>   Select Operator
> expressions: l_shipdate (type: string)
> outputColumnNames: l_shipdate
> Statistics: Num rows: 589709 Data size: 4745677733354 
> Basic stats: COMPLETE Column stats: COMPLETE
> Group By Operator
>   keys: l_shipdate (type: string)
>   mode: hash
>   outputColumnNames: _col0
>   Statistics: Num rows: 589709 Data size: 
> 563999032646 Basic stats: COMPLETE Column stats: COMPLETE
>   Reduce Output Operator
> key expressions: _col0 (type: string)
> sort order: +
> Map-reduce partition columns: _col0 (type: string)
> Statistics: Num rows: 589709 Data size: 
> 563999032646 Basic stats: COMPLETE Column stats: COMPLETE
> Execution mode: vectorized
> Reducer 2 
> Reduce Operator Tree:
>   Group By Operator
> keys: KEY._col0 (type: string)
> mode: mergepartial
> outputColumnNames: _col0
> Statistics: Num rows: 1955 Data size: 183770 Basic stats: 
> COMPLETE Column stats: COMPLETE
> Select Operator
>   expressions: _col0 (type: string)
>   outputColumnNames: _col0
>   Statistics: Num rows: 1955 Data size: 183770 Basic stats: 
> COMPLETE Column stats: COMPLETE
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8040) Commit for HIVE-7925 breaks hadoop-1 build

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-8040:

Attachment: HIVE-8040.2.patch.txt

> Commit for HIVE-7925 breaks hadoop-1 build
> --
>
> Key: HIVE-8040
> URL: https://issues.apache.org/jira/browse/HIVE-8040
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8040.1.patch.txt, HIVE-8040.2.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-metastore: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:[45,37]
>  package org.apache.commons.math3.stat does not exist
> [ERROR] -> [Help 1]
> {code}
> Missing pom file changes maybe?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8068) Dynamic partition insert overwrite does not overwrite files, but instead appends file to partition

[Chris Kudelka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Kudelka updated HIVE-8068:

Summary: Dynamic partition insert overwrite does not overwrite files, but 
instead appends file to partition  (was: Dynamic partition insert overwrite 
does not overwrite files, but instead appends.)

> Dynamic partition insert overwrite does not overwrite files, but instead 
> appends file to partition
> --
>
> Key: HIVE-8068
> URL: https://issues.apache.org/jira/browse/HIVE-8068
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 0.13.1
> Environment: Centos 6.2; Amazon S3 as DFS
>Reporter: Chris Kudelka
>
> -- using a reference table `one_row` with contents:
> ||dummy_field||
> |dummy_value|
> -- create test table
> create table if not exists test_table (
>   line string
> ) partitioned by (
>   my_part string
> )
> -- run first time
> insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> -- run again
> insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
> totalSize=20, rawDataSize=1]
> select * from test_table where my_part = 'partVal';
> Expected result:
> 1 row
> Actual result:
> 2 rows
> 
> -- compare to non-dynamic partition insert, which overwrites as expected
> 
> -- drop table and recreate with sme definition
> insert overwrite table test_table partition (my_part='partVal') select 'a' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> -- run again
> insert overwrite table test_table partition (my_part='partVal') select 'a' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> select * from test_table where my_part = 'partVal';
> Expected result:
> 1 row
> Actual result:
> 1 row



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8068) Dynamic partition insert overwrite does not overwrite files, but instead appends.

[Chris Kudelka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Kudelka updated HIVE-8068:

Description: 
-- using a reference table `one_row` with contents:
||dummy_field||
|dummy_value|

{{-- create test table
create table if not exists test_table (
  line string
) partitioned by (
  my_part string
)}}

-- run first time
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
totalSize=20, rawDataSize=1]

select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
2 rows


-- compare to non-dynamic partition insert, which overwrites as expected


-- drop table and recreate with sme definition
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]
select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
1 row

  was:
-- using a reference table `one_row` with contents:
||dummy_field||
|dummy_value|

-- create test table
create table if not exists test_table (
  line string
) partitioned by (
  my_part string
)

-- run first time
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
totalSize=20, rawDataSize=1]

select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
2 rows


-- compare to non-dynamic partition insert, which overwrites as expected


-- drop table and recreate with sme definition
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]
select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
1 row


> Dynamic partition insert overwrite does not overwrite files, but instead 
> appends.
> -
>
> Key: HIVE-8068
> URL: https://issues.apache.org/jira/browse/HIVE-8068
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 0.13.1
> Environment: Centos 6.2; Amazon S3 as DFS
>Reporter: Chris Kudelka
>
> -- using a reference table `one_row` with contents:
> ||dummy_field||
> |dummy_value|
> {{-- create test table
> create table if not exists test_table (
>   line string
> ) partitioned by (
>   my_part string
> )}}
> -- run first time
> insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> -- run again
> insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
> totalSize=20, rawDataSize=1]
> select * from test_table where my_part = 'partVal';
> Expected result:
> 1 row
> Actual result:
> 2 rows
> 
> -- compare to non-dynamic partition insert, which overwrites as expected
> 
> -- drop table and recreate with sme definition
> insert overwrite table test_table partition (my_part='partVal') select 'a' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> -- run again
> insert overwrite table test_table partition (my_part='partVal') select 'a' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> select * from test_table where my_part = 'partVal';
> Expected result:
> 1 row
> Actual result:
> 1 row



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8068) Dynamic partition insert overwrite does not overwrite files, but instead appends.

[Chris Kudelka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Kudelka updated HIVE-8068:

Description: 
-- using a reference table `one_row` with contents:
||dummy_field||
|dummy_value|

{{-- create test table}}
create table if not exists test_table (
  line string
) partitioned by (
  my_part string
)

-- run first time
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
totalSize=20, rawDataSize=1]

select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
2 rows


-- compare to non-dynamic partition insert, which overwrites as expected


-- drop table and recreate with sme definition
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]
select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
1 row

  was:
-- using a reference table `one_row` with contents:
||dummy_field||
|dummy_value|

{{-- create test table
create table if not exists test_table (
  line string
) partitioned by (
  my_part string
)}}

-- run first time
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
totalSize=20, rawDataSize=1]

select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
2 rows


-- compare to non-dynamic partition insert, which overwrites as expected


-- drop table and recreate with sme definition
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]
select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
1 row


> Dynamic partition insert overwrite does not overwrite files, but instead 
> appends.
> -
>
> Key: HIVE-8068
> URL: https://issues.apache.org/jira/browse/HIVE-8068
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 0.13.1
> Environment: Centos 6.2; Amazon S3 as DFS
>Reporter: Chris Kudelka
>
> -- using a reference table `one_row` with contents:
> ||dummy_field||
> |dummy_value|
> {{-- create test table}}
> create table if not exists test_table (
>   line string
> ) partitioned by (
>   my_part string
> )
> -- run first time
> insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> -- run again
> insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
> totalSize=20, rawDataSize=1]
> select * from test_table where my_part = 'partVal';
> Expected result:
> 1 row
> Actual result:
> 2 rows
> 
> -- compare to non-dynamic partition insert, which overwrites as expected
> 
> -- drop table and recreate with sme definition
> insert overwrite table test_table partition (my_part='partVal') select 'a' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> -- run again
> insert overwrite table test_table partition (my_part='partVal') select 'a' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> select * from test_table where my_part = 'partVal';
> Expected result:
> 1 row
> Actual result:
> 1 row



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8068) Dynamic partition insert overwrite does not overwrite files, but instead appends.

[Chris Kudelka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Kudelka updated HIVE-8068:

Description: 
-- using a reference table `one_row` with contents:
||dummy_field||
|dummy_value|

-- create test table
create table if not exists test_table (
  line string
) partitioned by (
  my_part string
)

-- run first time
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
totalSize=20, rawDataSize=1]

select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
2 rows


-- compare to non-dynamic partition insert, which overwrites as expected


-- drop table and recreate with sme definition
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]
select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
1 row

  was:
-- using a reference table `one_row` with contents:
||dummy_field||
|dummy_value|

{{-- create test table}}
create table if not exists test_table (
  line string
) partitioned by (
  my_part string
)

-- run first time
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
totalSize=20, rawDataSize=1]

select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
2 rows


-- compare to non-dynamic partition insert, which overwrites as expected


-- drop table and recreate with sme definition
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]
select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
1 row


> Dynamic partition insert overwrite does not overwrite files, but instead 
> appends.
> -
>
> Key: HIVE-8068
> URL: https://issues.apache.org/jira/browse/HIVE-8068
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 0.13.1
> Environment: Centos 6.2; Amazon S3 as DFS
>Reporter: Chris Kudelka
>
> -- using a reference table `one_row` with contents:
> ||dummy_field||
> |dummy_value|
> -- create test table
> create table if not exists test_table (
>   line string
> ) partitioned by (
>   my_part string
> )
> -- run first time
> insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> -- run again
> insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
> totalSize=20, rawDataSize=1]
> select * from test_table where my_part = 'partVal';
> Expected result:
> 1 row
> Actual result:
> 2 rows
> 
> -- compare to non-dynamic partition insert, which overwrites as expected
> 
> -- drop table and recreate with sme definition
> insert overwrite table test_table partition (my_part='partVal') select 'a' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> -- run again
> insert overwrite table test_table partition (my_part='partVal') select 'a' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> select * from test_table where my_part = 'partVal';
> Expected result:
> 1 row
> Actual result:
> 1 row



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8068) Dynamic partition insert overwrite does not overwrite files, but instead appends.

[Chris Kudelka (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8068?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chris Kudelka updated HIVE-8068:

Description: 
-- using a reference table `one_row` with contents:
||dummy_field||
|dummy_value|

-- create test table
create table if not exists test_table (
  line string
) partitioned by (
  my_part string
)

-- run first time
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
totalSize=20, rawDataSize=1]

select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
2 rows


-- compare to non-dynamic partition insert, which overwrites as expected


-- drop table and recreate with sme definition
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]
select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
1 row

  was:
-- using a reference table `one_row` with contents:
|dummy_field|
 dummy_value

-- create test table
create table if not exists test_table (
  line string
) partitioned by (
  my_part string
)

-- run first time
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
totalSize=20, rawDataSize=1]

select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
2 rows


-- compare to non-dynamic partition insert, which overwrites as expected


-- drop table and recreate with sme definition
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]
select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
1 row


> Dynamic partition insert overwrite does not overwrite files, but instead 
> appends.
> -
>
> Key: HIVE-8068
> URL: https://issues.apache.org/jira/browse/HIVE-8068
> Project: Hive
>  Issue Type: Bug
>  Components: SQL
>Affects Versions: 0.13.1
> Environment: Centos 6.2; Amazon S3 as DFS
>Reporter: Chris Kudelka
>
> -- using a reference table `one_row` with contents:
> ||dummy_field||
> |dummy_value|
> -- create test table
> create table if not exists test_table (
>   line string
> ) partitioned by (
>   my_part string
> )
> -- run first time
> insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> -- run again
> insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
> totalSize=20, rawDataSize=1]
> select * from test_table where my_part = 'partVal';
> Expected result:
> 1 row
> Actual result:
> 2 rows
> 
> -- compare to non-dynamic partition insert, which overwrites as expected
> 
> -- drop table and recreate with sme definition
> insert overwrite table test_table partition (my_part='partVal') select 'a' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> -- run again
> insert overwrite table test_table partition (my_part='partVal') select 'a' 
> from one_row;
> Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
> totalSize=10, rawDataSize=1]
> select * from test_table where my_part = 'partVal';
> Expected result:
> 1 row
> Actual result:
> 1 row



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-8068) Dynamic partition insert overwrite does not overwrite files, but instead appends.

[Chris Kudelka (JIRA)]

Chris Kudelka created HIVE-8068:
---

 Summary: Dynamic partition insert overwrite does not overwrite 
files, but instead appends.
 Key: HIVE-8068
 URL: https://issues.apache.org/jira/browse/HIVE-8068
 Project: Hive
  Issue Type: Bug
  Components: SQL
Affects Versions: 0.13.1
 Environment: Centos 6.2; Amazon S3 as DFS
Reporter: Chris Kudelka


-- using a reference table `one_row` with contents:
|dummy_field|
 dummy_value

-- create test table
create table if not exists test_table (
  line string
) partitioned by (
  my_part string
)

-- run first time
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part) select 'a', 'partVal' 
from one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=2, numRows=1, 
totalSize=20, rawDataSize=1]

select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
2 rows


-- compare to non-dynamic partition insert, which overwrites as expected


-- drop table and recreate with sme definition
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]

-- run again
insert overwrite table test_table partition (my_part='partVal') select 'a' from 
one_row;
Partition test_db.test_table{my_part=partVal} stats: [numFiles=1, numRows=1, 
totalSize=10, rawDataSize=1]
select * from test_table where my_part = 'partVal';

Expected result:
1 row

Actual result:
1 row



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 25557: improve the speed of col stats update speed

[pengcheng xiong]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25557/
---

(Updated Sept. 12, 2014, 3:53 a.m.)


Review request for hive.


Changes
---

rebase to trunk


Repository: hive-git


Description
---

Major improvement
(1) All the partition status update/insert is now done in one transaction.
(2) Rather than to use a query to update per col per partition (total query = 
#col * # part),
now we use 1 query to delete everything and then use 1 query to insert 
everything. The transaction makes sure that this happens in ACID mode.


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 9df6656 
  metastore/src/java/org/apache/hadoop/hive/metastore/HiveMetaStore.java 
33745e4 
  metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java 
5a8591a 
  metastore/src/java/org/apache/hadoop/hive/metastore/ObjectStore.java 637a39a 
  metastore/src/java/org/apache/hadoop/hive/metastore/RawStore.java 5c5ed7f 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreControlledCommit.java
 5905efe 
  
metastore/src/test/org/apache/hadoop/hive/metastore/DummyRawStoreForJdoConnection.java
 88b0791 
  ql/src/test/queries/clientpositive/analyze_tbl_part.q 9040bd4 
  ql/src/test/results/clientpositive/analyze_tbl_part.q.out 40b926c 

Diff: https://reviews.apache.org/r/25557/diff/


Testing
---


Thanks,

pengcheng xiong

[jira] [Updated] (HIVE-8061) improve the partition col stats update speed

[Pengcheng Xiong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pengcheng Xiong updated HIVE-8061:
--
Status: Open  (was: Patch Available)

> improve the partition col stats update speed
> 
>
> Key: HIVE-8061
> URL: https://issues.apache.org/jira/browse/HIVE-8061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>Priority: Minor
> Attachments: HIVE-8061.1.patch, HIVE-8061.2.patch
>
>
> We worked hard towards faster update stats for columns of a partition of a 
> table previously HIVE-7736
> and HIVE-7876
> Although there is some improvement, it is only correct in the first run. 
> There will be duplicate column stats later. Thanks to Eugene Koifman 's 
> comments.
> We fixed this in HIVE-7944 by reversing the patch.
> This JIRA ticket is my another try to improve the speed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8061) improve the partition col stats update speed

[Pengcheng Xiong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pengcheng Xiong updated HIVE-8061:
--
Status: Patch Available  (was: Open)

> improve the partition col stats update speed
> 
>
> Key: HIVE-8061
> URL: https://issues.apache.org/jira/browse/HIVE-8061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>Priority: Minor
> Attachments: HIVE-8061.1.patch, HIVE-8061.2.patch
>
>
> We worked hard towards faster update stats for columns of a partition of a 
> table previously HIVE-7736
> and HIVE-7876
> Although there is some improvement, it is only correct in the first run. 
> There will be duplicate column stats later. Thanks to Eugene Koifman 's 
> comments.
> We fixed this in HIVE-7944 by reversing the patch.
> This JIRA ticket is my another try to improve the speed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8061) improve the partition col stats update speed

[Pengcheng Xiong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pengcheng Xiong updated HIVE-8061:
--
Attachment: HIVE-8061.2.patch

rebase the patch to trunk

> improve the partition col stats update speed
> 
>
> Key: HIVE-8061
> URL: https://issues.apache.org/jira/browse/HIVE-8061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>Priority: Minor
> Attachments: HIVE-8061.1.patch, HIVE-8061.2.patch
>
>
> We worked hard towards faster update stats for columns of a partition of a 
> table previously HIVE-7736
> and HIVE-7876
> Although there is some improvement, it is only correct in the first run. 
> There will be duplicate column stats later. Thanks to Eugene Koifman 's 
> comments.
> We fixed this in HIVE-7944 by reversing the patch.
> This JIRA ticket is my another try to improve the speed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8019) Missing commit from trunk : `export/import statement update`

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131063#comment-14131063
 ] 

Hive QA commented on HIVE-8019:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668168/HIVE-8019.3.patch

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6199 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.txn.compactor.TestCompactor.testStatsAfterCompactionPartTbl
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/751/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/751/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-751/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668168

> Missing commit from trunk : `export/import statement update`
> 
>
> Key: HIVE-8019
> URL: https://issues.apache.org/jira/browse/HIVE-8019
> Project: Hive
>  Issue Type: Bug
>  Components: Import/Export
>Affects Versions: 0.14.0
>Reporter: Mohit Sabharwal
>Assignee: Thejas M Nair
>Priority: Blocker
> Attachments: HIVE-8019.1.patch, HIVE-8019.2.patch, HIVE-8019.3.patch
>
>
> Noticed that commit 1882de7810fc55a2466dd4cbe74ed67bb41cb667 exists in 0.13 
> branch, but not it trunk. 
> https://github.com/apache/hive/commit/1882de7810fc55a2466dd4cbe74ed67bb41cb667
> {code}
> (trunk) $ git branch -a --contains 1882de7810fc55a2466dd4cbe74ed67bb41cb667
> remotes/origin/branch-0.13
> {code}
> I looked through some of the changes in this commit and don't see those in 
> trunk.  Nor do I see a commit that reverts these changes in trunk.
> [~thejas], should we port this over to trunk ? 
> Thanks.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Comment Edited] (HIVE-8040) Commit for HIVE-7925 breaks hadoop-1 build

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131061#comment-14131061
 ] 

Xuefu Zhang edited comment on HIVE-8040 at 9/12/14 3:25 AM:


Build seems still broken even after this commit, with a different error (caused 
by HIVE-7223 maybe):

{code}
ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile 
(default-testCompile) on project hive-metastore: Compilation failure: 
Compilation failure:
[ERROR] 
/home/xzhang/apache/hive7/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStorePartitionSpecs.java:[14,30]
 cannot find symbol
[ERROR] symbol  : class ExitUtil
[ERROR] location: package org.apache.hadoop.util
[ERROR] 
/home/xzhang/apache/hive7/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStorePartitionSpecs.java:[55,25]
 package ExitUtil does not exist
{code}


was (Author: xuefuz):
Build seems still broken even after this commit, with a different error:

{code}
ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile 
(default-testCompile) on project hive-metastore: Compilation failure: 
Compilation failure:
[ERROR] 
/home/xzhang/apache/hive7/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStorePartitionSpecs.java:[14,30]
 cannot find symbol
[ERROR] symbol  : class ExitUtil
[ERROR] location: package org.apache.hadoop.util
[ERROR] 
/home/xzhang/apache/hive7/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStorePartitionSpecs.java:[55,25]
 package ExitUtil does not exist
{code}

> Commit for HIVE-7925 breaks hadoop-1 build
> --
>
> Key: HIVE-8040
> URL: https://issues.apache.org/jira/browse/HIVE-8040
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8040.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-metastore: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:[45,37]
>  package org.apache.commons.math3.stat does not exist
> [ERROR] -> [Help 1]
> {code}
> Missing pom file changes maybe?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8040) Commit for HIVE-7925 breaks hadoop-1 build

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131061#comment-14131061
 ] 

Xuefu Zhang commented on HIVE-8040:
---

Build seems still broken even after this commit, with a different error:

{code}
ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-compiler-plugin:3.1:testCompile 
(default-testCompile) on project hive-metastore: Compilation failure: 
Compilation failure:
[ERROR] 
/home/xzhang/apache/hive7/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStorePartitionSpecs.java:[14,30]
 cannot find symbol
[ERROR] symbol  : class ExitUtil
[ERROR] location: package org.apache.hadoop.util
[ERROR] 
/home/xzhang/apache/hive7/metastore/src/test/org/apache/hadoop/hive/metastore/TestHiveMetaStorePartitionSpecs.java:[55,25]
 package ExitUtil does not exist
{code}

> Commit for HIVE-7925 breaks hadoop-1 build
> --
>
> Key: HIVE-8040
> URL: https://issues.apache.org/jira/browse/HIVE-8040
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8040.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-metastore: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:[45,37]
>  package org.apache.commons.math3.stat does not exist
> [ERROR] -> [Help 1]
> {code}
> Missing pom file changes maybe?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 25524: HIVE-8047 Lazy char/varchar are not using escape char defined in serde params

[Jason Dere]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25524/
---

(Updated Sept. 12, 2014, 3:22 a.m.)


Review request for hive and Thejas Nair.


Changes
---

Allow the lazy char/varchar object inspectors to be cached like the rest of the 
lazy primitive OIs are.


Bugs: HIVE-8047
https://issues.apache.org/jira/browse/HIVE-8047


Repository: hive-git


Description
---

Update lazy char/varchar to support escape character


Diffs (updated)
-

  data/files/data_with_escape.txt PRE-CREATION 
  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java 
ae128a9 
  ql/src/test/queries/clientpositive/escape3.q PRE-CREATION 
  ql/src/test/results/clientpositive/escape3.q.out PRE-CREATION 
  serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveChar.java ef469eb 
  serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyHiveVarchar.java 
bc8d41e 
  serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyString.java 28b3f86 
  serde/src/java/org/apache/hadoop/hive/serde2/lazy/LazyUtils.java 1d62422 
  
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveCharObjectInspector.java
 65fb1ab 
  
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyHiveVarcharObjectInspector.java
 c802ed0 
  
serde/src/java/org/apache/hadoop/hive/serde2/lazy/objectinspector/primitive/LazyPrimitiveObjectInspectorFactory.java
 734b9d8 
  
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveCharObjectInspector.java
 c340dcb 
  
serde/src/java/org/apache/hadoop/hive/serde2/objectinspector/primitive/WritableHiveVarcharObjectInspector.java
 7bbfd1f 

Diff: https://reviews.apache.org/r/25524/diff/


Testing
---

New q file test


Thanks,

Jason Dere

[jira] [Updated] (HIVE-8047) Lazy char/varchar are not using escape char defined in serde params

[Jason Dere (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere updated HIVE-8047:
-
Attachment: HIVE-8047.3.patch

The lazy object inspectors are supposed to be cached, and the previous patches 
caused lazy char/varchar object inspectors to not be cached. Adding the caching 
in v3.

> Lazy char/varchar are not using escape char defined in serde params
> ---
>
> Key: HIVE-8047
> URL: https://issues.apache.org/jira/browse/HIVE-8047
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-8047.1.patch, HIVE-8047.2.patch, HIVE-8047.3.patch
>
>
> A table with char/varchar columns that specifies an escape character for 
> string data still has the escape characters in the char/varchar columns.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Assigned] (HIVE-8041) Hadoop-2 build is broken with JDK6

[Xuefu Zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Xuefu Zhang reassigned HIVE-8041:
-

Assignee: Navis

> Hadoop-2 build is broken with JDK6
> --
>
> Key: HIVE-8041
> URL: https://issues.apache.org/jira/browse/HIVE-8041
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
>Assignee: Navis
> Attachments: HIVE-8041.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-exec: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java:[81,1]
>  illegal start of expression
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8041) Hadoop-2 build is broken with JDK6

[Xuefu Zhang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131053#comment-14131053
 ] 

Xuefu Zhang commented on HIVE-8041:
---

+1. build passes with the patch. Thanks, Navis.

> Hadoop-2 build is broken with JDK6
> --
>
> Key: HIVE-8041
> URL: https://issues.apache.org/jira/browse/HIVE-8041
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8041.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-exec: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java:[81,1]
>  illegal start of expression
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7777) add CSV support for Serde

[Ferdinand Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ferdinand Xu updated HIVE-:
---
Attachment: HIVE-.2.patch

add query test

> add CSV support for Serde
> -
>
> Key: HIVE-
> URL: https://issues.apache.org/jira/browse/HIVE-
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Ferdinand Xu
>Assignee: Ferdinand Xu
> Attachments: HIVE-.1.patch, HIVE-.2.patch, HIVE-.patch, 
> csv-serde-master.zip
>
>
> There is no official support for csvSerde for hive while there is an open 
> source project in github(https://github.com/ogrodnek/csv-serde). CSV is of 
> high frequency in use as a data format.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8067) set default table permissions for table owner to have all privileges

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-8067:

Status: Patch Available  (was: Open)

> set default table permissions for table owner to have all privileges
> 
>
> Key: HIVE-8067
> URL: https://issues.apache.org/jira/browse/HIVE-8067
> Project: Hive
>  Issue Type: Bug
>  Components: Authorization, SQLStandardAuthorization
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Attachments: HIVE-8067.1.patch
>
>
> When tables are created using without SQLStandards based authorization being 
> enabled, the table owner does not have any privileges on the table.
> It makes sense to set the default privileges to be compatible with sql 
> standard mode's expected default privileges for the owner of the table, 
> instead of setting no privileges at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8067) set default table permissions for table owner to have all privileges

[Thejas M Nair (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8067?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Thejas M Nair updated HIVE-8067:

Attachment: HIVE-8067.1.patch

> set default table permissions for table owner to have all privileges
> 
>
> Key: HIVE-8067
> URL: https://issues.apache.org/jira/browse/HIVE-8067
> Project: Hive
>  Issue Type: Bug
>  Components: Authorization, SQLStandardAuthorization
>Reporter: Thejas M Nair
>Assignee: Thejas M Nair
> Attachments: HIVE-8067.1.patch
>
>
> When tables are created using without SQLStandards based authorization being 
> enabled, the table owner does not have any privileges on the table.
> It makes sense to set the default privileges to be compatible with sql 
> standard mode's expected default privileges for the owner of the table, 
> instead of setting no privileges at all.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-8067) set default table permissions for table owner to have all privileges

[Thejas M Nair (JIRA)]

Thejas M Nair created HIVE-8067:
---

 Summary: set default table permissions for table owner to have all 
privileges
 Key: HIVE-8067
 URL: https://issues.apache.org/jira/browse/HIVE-8067
 Project: Hive
  Issue Type: Bug
  Components: Authorization, SQLStandardAuthorization
Reporter: Thejas M Nair
Assignee: Thejas M Nair
 Attachments: HIVE-8067.1.patch

When tables are created using without SQLStandards based authorization being 
enabled, the table owner does not have any privileges on the table.

It makes sense to set the default privileges to be compatible with sql standard 
mode's expected default privileges for the owner of the table, instead of 
setting no privileges at all.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8061) improve the partition col stats update speed

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131016#comment-14131016
 ] 

Hive QA commented on HIVE-8061:
---



{color:red}Overall{color}: -1 no tests executed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668165/HIVE-8061.1.patch

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/750/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/750/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-750/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hive-ptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ [[ -n /usr/java/jdk1.7.0_45-cloudera ]]
+ export JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ JAVA_HOME=/usr/java/jdk1.7.0_45-cloudera
+ export 
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ 
PATH=/usr/java/jdk1.7.0_45-cloudera/bin/:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-maven-3.0.5/bin:/usr/local/apache-maven-3.0.5/bin:/usr/java/jdk1.6.0_34/bin:/usr/local/apache-ant-1.9.1/bin:/usr/local/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/home/hiveptest/bin
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost 
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost 
-Dhttp.proxyPort=3128'
+ cd /data/hive-ptest/working/
+ tee /data/hive-ptest/logs/PreCommit-HIVE-TRUNK-Build-750/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ svn = \s\v\n ]]
+ [[ -n '' ]]
+ [[ -d apache-svn-trunk-source ]]
+ [[ ! -d apache-svn-trunk-source/.svn ]]
+ [[ ! -d apache-svn-trunk-source ]]
+ cd apache-svn-trunk-source
+ svn revert -R .
Reverted 'ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java'
++ egrep -v '^X|^Performing status on external'
++ awk '{print $2}'
++ svn status --no-ignore
+ rm -rf target datanucleus.log ant/target shims/target shims/0.20/target 
shims/0.20S/target shims/0.23/target shims/aggregator/target 
shims/common/target shims/common-secure/target packaging/target 
hbase-handler/target testutils/target jdbc/target metastore/target 
itests/target itests/hcatalog-unit/target itests/test-serde/target 
itests/qtest/target itests/hive-unit-hadoop2/target itests/hive-minikdc/target 
itests/hive-unit/target itests/custom-serde/target itests/util/target 
hcatalog/target hcatalog/core/target hcatalog/streaming/target 
hcatalog/server-extensions/target hcatalog/webhcat/svr/target 
hcatalog/webhcat/java-client/target hcatalog/hcatalog-pig-adapter/target 
accumulo-handler/target hwi/target common/target common/src/gen contrib/target 
service/target serde/target beeline/target odbc/target cli/target 
ql/dependency-reduced-pom.xml ql/target
+ svn update
Umetastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java
Aql/src/test/results/clientpositive/tez/correlationoptimizer1.q.out
Uql/src/java/org/apache/hadoop/hive/ql/optimizer/Optimizer.java
Udata/conf/tez/hive-site.xml
Uitests/src/test/resources/testconfiguration.properties

Fetching external item into 'hcatalog/src/test/e2e/harness'
Updated external to revision 1624440.

Updated to revision 1624440.
+ patchCommandPath=/data/hive-ptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hive-ptest/working/scratch/build.patch
+ [[ -f /data/hive-ptest/working/scratch/build.patch ]]
+ chmod +x /data/hive-ptest/working/scratch/smart-apply-patch.sh
+ /data/hive-ptest/working/scratch/smart-apply-patch.sh 
/data/hive-ptest/working/scratch/build.patch
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668165

> improve the partition col stats update speed
> 
>
> Key: HIVE-8061
> URL: https://issues.apache.org/jira/browse/HIVE-8061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>Priority: Minor
> Attachments: HIVE-8061.1.patch
>
>
> We worked hard towards faster update stats for columns of a partition of a 
> table previously HIVE-7736
> and HIVE-7876
> Although there is some improvement, it is only correct in the first run. 
> There will be duplicate column stats later. Thanks to Eugene Koifman 's 
> comm

[jira] [Commented] (HIVE-8056) SessionState.dropSessionPaths should use FileSystem.getLocal(conf) to delete local files

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131012#comment-14131012
 ] 

Hive QA commented on HIVE-8056:
---



{color:red}Overall{color}: -1 at least one tests failed

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668134/HIVE-8056.1.patch

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 6196 tests executed
*Failed tests:*
{noformat}
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation
{noformat}

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/749/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/749/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-749/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668134

> SessionState.dropSessionPaths should use FileSystem.getLocal(conf) to delete 
> local files
> 
>
> Key: HIVE-8056
> URL: https://issues.apache.org/jira/browse/HIVE-8056
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.14.0
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Fix For: 0.14.0
>
> Attachments: HIVE-8056.1.patch
>
>
> SessionState.java uses the incorrect filesystem api call to remove local 
> files. This should be changed to FileSystem.getLocal(conf).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HIVE-8040) Commit for HIVE-7925 breaks hadoop-1 build

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis resolved HIVE-8040.
-
Resolution: Fixed

Committed to trunk. Thanks Ashutosh!

> Commit for HIVE-7925 breaks hadoop-1 build
> --
>
> Key: HIVE-8040
> URL: https://issues.apache.org/jira/browse/HIVE-8040
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8040.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-metastore: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:[45,37]
>  package org.apache.commons.math3.stat does not exist
> [ERROR] -> [Help 1]
> {code}
> Missing pom file changes maybe?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HIVE-7964) CBO Trunk Merge:Handle explode, lateral views

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran resolved HIVE-7964.
--
Resolution: Fixed

> CBO Trunk Merge:Handle explode, lateral views
> -
>
> Key: HIVE-7964
> URL: https://issues.apache.org/jira/browse/HIVE-7964
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7946) CBO: Merge CBO changes to Trunk

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-7946:
-
Status: Open  (was: Patch Available)

> CBO: Merge CBO changes to Trunk
> ---
>
> Key: HIVE-7946
> URL: https://issues.apache.org/jira/browse/HIVE-7946
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-7946.1.patch, HIVE-7946.2.patch, HIVE-7946.3.patch, 
> HIVE-7946.4.patch, HIVE-7946.5.patch, HIVE-7946.6.patch, HIVE-7946.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7946) CBO: Merge CBO changes to Trunk

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-7946:
-
Status: Patch Available  (was: Open)

> CBO: Merge CBO changes to Trunk
> ---
>
> Key: HIVE-7946
> URL: https://issues.apache.org/jira/browse/HIVE-7946
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-7946.1.patch, HIVE-7946.2.patch, HIVE-7946.3.patch, 
> HIVE-7946.4.patch, HIVE-7946.5.patch, HIVE-7946.6.patch, HIVE-7946.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7946) CBO: Merge CBO changes to Trunk

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-7946:
-
Attachment: HIVE-7946.6.patch

> CBO: Merge CBO changes to Trunk
> ---
>
> Key: HIVE-7946
> URL: https://issues.apache.org/jira/browse/HIVE-7946
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-7946.1.patch, HIVE-7946.2.patch, HIVE-7946.3.patch, 
> HIVE-7946.4.patch, HIVE-7946.5.patch, HIVE-7946.6.patch, HIVE-7946.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-5799) session/operation timeout for hiveserver2

[Navis (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-5799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131003#comment-14131003
 ] 

Navis commented on HIVE-5799:
-

[~brocknoland] It's described in HiveConf, 
{noformat}
With positive value, it's checked for operations in terminal state only 
(FINISHED, CANCELED, CLOSED, ERROR).
With negative value, it's checked for all of the operations regardless of state.
{noformat}

> session/operation timeout for hiveserver2
> -
>
> Key: HIVE-5799
> URL: https://issues.apache.org/jira/browse/HIVE-5799
> Project: Hive
>  Issue Type: Improvement
>  Components: HiveServer2
>Reporter: Navis
>Assignee: Navis
>Priority: Minor
>  Labels: TODOC14
> Fix For: 0.14.0
>
> Attachments: HIVE-5799.1.patch.txt, HIVE-5799.10.patch.txt, 
> HIVE-5799.11.patch.txt, HIVE-5799.12.patch.txt, HIVE-5799.13.patch.txt, 
> HIVE-5799.14.patch.txt, HIVE-5799.15.patch.txt, HIVE-5799.16.patch.txt, 
> HIVE-5799.17.patch.txt, HIVE-5799.2.patch.txt, HIVE-5799.3.patch.txt, 
> HIVE-5799.4.patch.txt, HIVE-5799.5.patch.txt, HIVE-5799.6.patch.txt, 
> HIVE-5799.7.patch.txt, HIVE-5799.8.patch.txt, HIVE-5799.9.patch.txt
>
>
> Need some timeout facility for preventing resource leakages from instable  or 
> bad clients.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8014) hive.in.test not set in MiniTezCli tests

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-8014:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks [~vikram.dixit]!

> hive.in.test not set in MiniTezCli tests
> 
>
> Key: HIVE-8014
> URL: https://issues.apache.org/jira/browse/HIVE-8014
> Project: Hive
>  Issue Type: Bug
>  Components: Testing Infrastructure
>Affects Versions: 0.13.1
>Reporter: Alan Gates
>Assignee: Gunther Hagleitner
> Attachments: HIVE-8014.1.patch
>
>
> When the TestCli driver is run, the configuration value hive.in.test is set 
> to true.  However, when TestMiniTezCli driver is run, this value is not set, 
> which results in some classes not realizing they are running in a test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8058) Disable correlation opt for Tez

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-8058:
-
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to trunk. Thanks [~vikram.dixit]!

> Disable correlation opt for Tez
> ---
>
> Key: HIVE-8058
> URL: https://issues.apache.org/jira/browse/HIVE-8058
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-8058.1.patch
>
>
> Correlation optimizer needs more thought on Tez. Stuff like nested tags are 
> at odds with Tez (where we can expose more stages in a single DAG).
> For now: disable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8040) Commit for HIVE-7925 breaks hadoop-1 build

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14131000#comment-14131000
 ] 

Ashutosh Chauhan commented on HIVE-8040:


+1

> Commit for HIVE-7925 breaks hadoop-1 build
> --
>
> Key: HIVE-8040
> URL: https://issues.apache.org/jira/browse/HIVE-8040
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8040.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-metastore: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:[45,37]
>  package org.apache.commons.math3.stat does not exist
> [ERROR] -> [Help 1]
> {code}
> Missing pom file changes maybe?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8040) Commit for HIVE-7925 breaks hadoop-1 build

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8040?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-8040:

Attachment: HIVE-8040.1.patch.txt

Quick fix. I'll commit this shortly after (I'm using hadoop-1 and this makes 
things really annoying).

> Commit for HIVE-7925 breaks hadoop-1 build
> --
>
> Key: HIVE-8040
> URL: https://issues.apache.org/jira/browse/HIVE-8040
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8040.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-metastore: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreDirectSql.java:[45,37]
>  package org.apache.commons.math3.stat does not exist
> [ERROR] -> [Help 1]
> {code}
> Missing pom file changes maybe?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8056) SessionState.dropSessionPaths should use FileSystem.getLocal(conf) to delete local files

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-8056:
---
Affects Version/s: 0.14.0

> SessionState.dropSessionPaths should use FileSystem.getLocal(conf) to delete 
> local files
> 
>
> Key: HIVE-8056
> URL: https://issues.apache.org/jira/browse/HIVE-8056
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.14.0
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Fix For: 0.14.0
>
> Attachments: HIVE-8056.1.patch
>
>
> SessionState.java uses the incorrect filesystem api call to remove local 
> files. This should be changed to FileSystem.getLocal(conf).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8056) SessionState.dropSessionPaths should use FileSystem.getLocal(conf) to delete local files

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-8056:
---
Fix Version/s: 0.14.0

> SessionState.dropSessionPaths should use FileSystem.getLocal(conf) to delete 
> local files
> 
>
> Key: HIVE-8056
> URL: https://issues.apache.org/jira/browse/HIVE-8056
> Project: Hive
>  Issue Type: Bug
>Reporter: Hari Sankar Sivarama Subramaniyan
>Assignee: Hari Sankar Sivarama Subramaniyan
> Fix For: 0.14.0
>
> Attachments: HIVE-8056.1.patch
>
>
> SessionState.java uses the incorrect filesystem api call to remove local 
> files. This should be changed to FileSystem.getLocal(conf).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8041) Hadoop-2 build is broken with JDK6

[Navis (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130995#comment-14130995
 ] 

Navis commented on HIVE-8041:
-

[~xuefuz] [~satish.mittal] Could you check again with the patch attached? 
Seemed caused by trailing comma.

> Hadoop-2 build is broken with JDK6
> --
>
> Key: HIVE-8041
> URL: https://issues.apache.org/jira/browse/HIVE-8041
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8041.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-exec: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java:[81,1]
>  illegal start of expression
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8041) Hadoop-2 build is broken with JDK6

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-8041:

Status: Open  (was: Patch Available)

> Hadoop-2 build is broken with JDK6
> --
>
> Key: HIVE-8041
> URL: https://issues.apache.org/jira/browse/HIVE-8041
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8041.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-exec: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java:[81,1]
>  illegal start of expression
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8041) Hadoop-2 build is broken with JDK6

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-8041:

Status: Patch Available  (was: Open)

> Hadoop-2 build is broken with JDK6
> --
>
> Key: HIVE-8041
> URL: https://issues.apache.org/jira/browse/HIVE-8041
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8041.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-exec: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java:[81,1]
>  illegal start of expression
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8041) Hadoop-2 build is broken with JDK6

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-8041:

Attachment: HIVE-8041.1.patch.txt

> Hadoop-2 build is broken with JDK6
> --
>
> Key: HIVE-8041
> URL: https://issues.apache.org/jira/browse/HIVE-8041
> Project: Hive
>  Issue Type: Bug
>  Components: Build Infrastructure
>Affects Versions: 0.14.0
>Reporter: Xuefu Zhang
> Attachments: HIVE-8041.1.patch.txt
>
>
> {code}
> [ERROR] Failed to execute goal 
> org.apache.maven.plugins:maven-compiler-plugin:3.1:compile (default-compile) 
> on project hive-exec: Compilation failure
> [ERROR] 
> /home/xzhang/apache/hive7/ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDFIf.java:[81,1]
>  illegal start of expression
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Hive contributor

[Navis류승우]

Done!

Thanks,
Navis

2014-09-12 6:42 GMT+09:00 Sebastien Marti :

> Hi all,
>
> Please add me to Hive contributor list
>
> My Jira User name : smarti
>
> Thanks
>
>
> Sebastien
>

Re: Review Request 25059: Ambiguous column reference error on query

[Navis Ryu]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25059/
---

(Updated Sept. 12, 2014, 1:26 a.m.)


Review request for hive.


Bugs: HIVE-7733
https://issues.apache.org/jira/browse/HIVE-7733


Repository: hive-git


Description
---

{noformat}
CREATE TABLE agg1 
  ( 
 col0 INT, 
 col1 STRING, 
 col2 DOUBLE 
  ); 

explain SELECT single_use_subq11.a1 AS a1, 
   single_use_subq11.a2 AS a2 
FROM   (SELECT Sum(agg1.col2) AS a1 
FROM   agg1 
GROUP  BY agg1.col0) single_use_subq12 
   JOIN (SELECT alias.a2 AS a0, 
alias.a1 AS a1, 
alias.a1 AS a2 
 FROM   (SELECT agg1.col1 AS a0, 
'42'  AS a1, 
agg1.col0 AS a2 
 FROM   agg1 
 UNION ALL 
 SELECT agg1.col1 AS a0, 
'41'  AS a1, 
agg1.col0 AS a2 
 FROM   agg1) alias 
 GROUP  BY alias.a2, 
   alias.a1) single_use_subq11 
 ON ( single_use_subq11.a0 = single_use_subq11.a0 );
{noformat}

Gets the following error:
FAILED: SemanticException [Error 10007]: Ambiguous column reference a2
Looks like this query had been working in 0.12 but starting failing with this 
error in 0.13


Diffs (updated)
-

  ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 2e58b80 
  ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 496f6a6 
  ql/src/test/queries/clientpositive/complex_alias.q PRE-CREATION 
  ql/src/test/results/clientnegative/ambiguous_col.q.out 237c21f 
  ql/src/test/results/clientnegative/ambiguous_col0.q.out 237c21f 
  ql/src/test/results/clientnegative/ambiguous_col1.q.out 237c21f 
  ql/src/test/results/clientnegative/ambiguous_col2.q.out 237c21f 
  ql/src/test/results/clientpositive/ambiguous_col.q.out e8760f1 
  ql/src/test/results/clientpositive/complex_alias.q.out PRE-CREATION 

Diff: https://reviews.apache.org/r/25059/diff/


Testing
---


Thanks,

Navis Ryu

Re: Review Request 25059: Ambiguous column reference error on query

[Navis Ryu]

> On Sept. 10, 2014, 11:58 p.m., Ashutosh Chauhan wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java, line 213
> > 
> >
> > I am bit confused here. Shouldn't this be if (tableAlias == null || 
> > tableAlias.equalsIgnoreCase(prev.getTabAlias()))
> > 
> > Note missing !
> > We want to throw an exception if tabAlias = null or if non-null than 
> > equal to prev. Isnt it?
> > 
> > But, I see existing code is doing the same. So, confused here.

!tableAlias.equalsIgnoreCase(prev.getTabAlias()) means the column is from 
different table alias, which makes it an ambiguous alias. This can be done at 
last stage of processing the subquery. I'll update the patch.


- Navis


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25059/#review52982
---


On Aug. 26, 2014, 9:06 a.m., Navis Ryu wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25059/
> ---
> 
> (Updated Aug. 26, 2014, 9:06 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-7733
> https://issues.apache.org/jira/browse/HIVE-7733
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> {noformat}
> CREATE TABLE agg1 
>   ( 
>  col0 INT, 
>  col1 STRING, 
>  col2 DOUBLE 
>   ); 
> 
> explain SELECT single_use_subq11.a1 AS a1, 
>single_use_subq11.a2 AS a2 
> FROM   (SELECT Sum(agg1.col2) AS a1 
> FROM   agg1 
> GROUP  BY agg1.col0) single_use_subq12 
>JOIN (SELECT alias.a2 AS a0, 
> alias.a1 AS a1, 
> alias.a1 AS a2 
>  FROM   (SELECT agg1.col1 AS a0, 
> '42'  AS a1, 
> agg1.col0 AS a2 
>  FROM   agg1 
>  UNION ALL 
>  SELECT agg1.col1 AS a0, 
> '41'  AS a1, 
> agg1.col0 AS a2 
>  FROM   agg1) alias 
>  GROUP  BY alias.a2, 
>alias.a1) single_use_subq11 
>  ON ( single_use_subq11.a0 = single_use_subq11.a0 );
> {noformat}
> 
> Gets the following error:
> FAILED: SemanticException [Error 10007]: Ambiguous column reference a2
> Looks like this query had been working in 0.12 but starting failing with this 
> error in 0.13
> 
> 
> Diffs
> -
> 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/RowResolver.java 33b8a21 
>   ql/src/test/queries/clientpositive/complex_alias.q PRE-CREATION 
>   ql/src/test/results/clientpositive/complex_alias.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/25059/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Navis Ryu
> 
>

Re: Review Request 25245: Support dynamic service discovery for HiveServer2

[Vaibhav Gumashta]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25245/
---

(Updated Sept. 12, 2014, 1:25 a.m.)


Review request for hive, Alan Gates, Navis Ryu, Szehon Ho, and Thejas Nair.


Changes
---

All the feedback changes except changing the ACL on znodes (I'm  figuring out 
the best solution).


Bugs: HIVE-7935
https://issues.apache.org/jira/browse/HIVE-7935


Repository: hive-git


Description
---

https://issues.apache.org/jira/browse/HIVE-7935


Diffs (updated)
-

  common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5d2e6b0 
  itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java 
ae128a9 
  jdbc/pom.xml 1ad13a7 
  jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java cbcfec7 
  jdbc/src/java/org/apache/hive/jdbc/HiveDriver.java 6e248d6 
  jdbc/src/java/org/apache/hive/jdbc/JdbcUriParseException.java PRE-CREATION 
  jdbc/src/java/org/apache/hive/jdbc/Utils.java 58339bf 
  jdbc/src/java/org/apache/hive/jdbc/ZooKeeperHiveClientException.java 
PRE-CREATION 
  jdbc/src/java/org/apache/hive/jdbc/ZooKeeperHiveClientHelper.java 
PRE-CREATION 
  
ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
 0919d2f 
  ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 
PRE-CREATION 
  
ql/src/test/org/apache/hadoop/hive/ql/lockmgr/zookeeper/TestZookeeperLockManager.java
 59294b1 
  service/src/java/org/apache/hive/service/cli/CLIService.java a0bc905 
  service/src/java/org/apache/hive/service/cli/operation/OperationManager.java 
f5a8f27 
  service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
b0bb8be 
  service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
11d25cc 
  
service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java 
2b80adc 
  service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
443c371 
  service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java 
4067106 
  service/src/java/org/apache/hive/service/server/HiveServer2.java 124996c 
  
service/src/test/org/apache/hive/service/cli/session/TestSessionGlobalInitFile.java
 66fc1fc 

Diff: https://reviews.apache.org/r/25245/diff/


Testing
---

Manual testing.


Thanks,

Vaibhav Gumashta

[jira] [Commented] (HIVE-8014) hive.in.test not set in MiniTezCli tests

[Vikram Dixit K (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130991#comment-14130991
 ] 

Vikram Dixit K commented on HIVE-8014:
--

+1

> hive.in.test not set in MiniTezCli tests
> 
>
> Key: HIVE-8014
> URL: https://issues.apache.org/jira/browse/HIVE-8014
> Project: Hive
>  Issue Type: Bug
>  Components: Testing Infrastructure
>Affects Versions: 0.13.1
>Reporter: Alan Gates
>Assignee: Gunther Hagleitner
> Attachments: HIVE-8014.1.patch
>
>
> When the TestCli driver is run, the configuration value hive.in.test is set 
> to true.  However, when TestMiniTezCli driver is run, this value is not set, 
> which results in some classes not realizing they are running in a test.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-7733) Ambiguous column reference error on query

[Navis (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-7733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Navis updated HIVE-7733:

Attachment: HIVE-7733.4.patch.txt

> Ambiguous column reference error on query
> -
>
> Key: HIVE-7733
> URL: https://issues.apache.org/jira/browse/HIVE-7733
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 0.13.0
>Reporter: Jason Dere
>Assignee: Navis
> Attachments: HIVE-7733.1.patch.txt, HIVE-7733.2.patch.txt, 
> HIVE-7733.3.patch.txt, HIVE-7733.4.patch.txt
>
>
> {noformat}
> CREATE TABLE agg1 
>   ( 
>  col0 INT, 
>  col1 STRING, 
>  col2 DOUBLE 
>   ); 
> explain SELECT single_use_subq11.a1 AS a1, 
>single_use_subq11.a2 AS a2 
> FROM   (SELECT Sum(agg1.col2) AS a1 
> FROM   agg1 
> GROUP  BY agg1.col0) single_use_subq12 
>JOIN (SELECT alias.a2 AS a0, 
> alias.a1 AS a1, 
> alias.a1 AS a2 
>  FROM   (SELECT agg1.col1 AS a0, 
> '42'  AS a1, 
> agg1.col0 AS a2 
>  FROM   agg1 
>  UNION ALL 
>  SELECT agg1.col1 AS a0, 
> '41'  AS a1, 
> agg1.col0 AS a2 
>  FROM   agg1) alias 
>  GROUP  BY alias.a2, 
>alias.a1) single_use_subq11 
>  ON ( single_use_subq11.a0 = single_use_subq11.a0 );
> {noformat}
> Gets the following error:
> FAILED: SemanticException [Error 10007]: Ambiguous column reference a2
> Looks like this query had been working in 0.12 but starting failing with this 
> error in 0.13



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Review Request 25245: Support dynamic service discovery for HiveServer2

[Vaibhav Gumashta]

> On Sept. 11, 2014, 7:24 p.m., Thejas Nair wrote:
> > service/src/java/org/apache/hive/service/server/HiveServer2.java, line 114
> > 
> >
> > Should we do a 'mkdir -p' equivalent as done in 
> > ZookeeperTokenStore.ensurePath ?

Yup, that's a cleaner approach. Adding it for the namespace path though as 
server znode will never have path components.


- Vaibhav


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25245/#review53095
---


On Sept. 11, 2014, 1:08 p.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25245/
> ---
> 
> (Updated Sept. 11, 2014, 1:08 p.m.)
> 
> 
> Review request for hive, Alan Gates, Navis Ryu, Szehon Ho, and Thejas Nair.
> 
> 
> Bugs: HIVE-7935
> https://issues.apache.org/jira/browse/HIVE-7935
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-7935
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5d2e6b0 
>   itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java 
> ae128a9 
>   jdbc/pom.xml 1ad13a7 
>   jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java cbcfec7 
>   jdbc/src/java/org/apache/hive/jdbc/HiveDriver.java 6e248d6 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcUriParseException.java PRE-CREATION 
>   jdbc/src/java/org/apache/hive/jdbc/Utils.java 58339bf 
>   jdbc/src/java/org/apache/hive/jdbc/ZooKeeperHiveClientException.java 
> PRE-CREATION 
>   jdbc/src/java/org/apache/hive/jdbc/ZooKeeperHiveClientHelper.java 
> PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
>  0919d2f 
>   ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 
> PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/lockmgr/zookeeper/TestZookeeperLockManager.java
>  59294b1 
>   service/src/java/org/apache/hive/service/cli/CLIService.java a0bc905 
>   
> service/src/java/org/apache/hive/service/cli/operation/OperationManager.java 
> f5a8f27 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
> b0bb8be 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 11d25cc 
>   
> service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java
>  2b80adc 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 443c371 
>   
> service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java 
> 4067106 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 124996c 
>   
> service/src/test/org/apache/hive/service/cli/session/TestSessionGlobalInitFile.java
>  66fc1fc 
> 
> Diff: https://reviews.apache.org/r/25245/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

Re: Review Request 25245: Support dynamic service discovery for HiveServer2

[Vaibhav Gumashta]

> On Sept. 11, 2014, 6:33 p.m., Thejas Nair wrote:
> > service/src/java/org/apache/hive/service/server/HiveServer2.java, line 129
> > 
> >
> > Should we use READ_ACL_UNSAFE here ?

Sorry I meant to use that. But as you mention in the comment below, we'll need 
to construct an ACL which gives write/delete/create to a specific user id and 
read to all.


> On Sept. 11, 2014, 6:33 p.m., Thejas Nair wrote:
> > service/src/java/org/apache/hive/service/server/HiveServer2.java, line 146
> > 
> >
> > I think we should check here as well, if it is OK to shut down the 
> > server.
> > Consider the case of a rolling upgrade late in the night, when the 
> > cluster might not be very active. There might not be any remaining active 
> > connections on this server. And since it is removed from the zookeeper, no 
> > new connections would be established, and server would not come down by 
> > itself.

Good point, will make the change.


- Vaibhav


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/25245/#review53067
---


On Sept. 11, 2014, 1:08 p.m., Vaibhav Gumashta wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/25245/
> ---
> 
> (Updated Sept. 11, 2014, 1:08 p.m.)
> 
> 
> Review request for hive, Alan Gates, Navis Ryu, Szehon Ho, and Thejas Nair.
> 
> 
> Bugs: HIVE-7935
> https://issues.apache.org/jira/browse/HIVE-7935
> 
> 
> Repository: hive-git
> 
> 
> Description
> ---
> 
> https://issues.apache.org/jira/browse/HIVE-7935
> 
> 
> Diffs
> -
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5d2e6b0 
>   itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java 
> ae128a9 
>   jdbc/pom.xml 1ad13a7 
>   jdbc/src/java/org/apache/hive/jdbc/HiveConnection.java cbcfec7 
>   jdbc/src/java/org/apache/hive/jdbc/HiveDriver.java 6e248d6 
>   jdbc/src/java/org/apache/hive/jdbc/JdbcUriParseException.java PRE-CREATION 
>   jdbc/src/java/org/apache/hive/jdbc/Utils.java 58339bf 
>   jdbc/src/java/org/apache/hive/jdbc/ZooKeeperHiveClientException.java 
> PRE-CREATION 
>   jdbc/src/java/org/apache/hive/jdbc/ZooKeeperHiveClientHelper.java 
> PRE-CREATION 
>   
> ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java
>  0919d2f 
>   ql/src/java/org/apache/hadoop/hive/ql/util/ZooKeeperHiveHelper.java 
> PRE-CREATION 
>   
> ql/src/test/org/apache/hadoop/hive/ql/lockmgr/zookeeper/TestZookeeperLockManager.java
>  59294b1 
>   service/src/java/org/apache/hive/service/cli/CLIService.java a0bc905 
>   
> service/src/java/org/apache/hive/service/cli/operation/OperationManager.java 
> f5a8f27 
>   service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java 
> b0bb8be 
>   service/src/java/org/apache/hive/service/cli/session/SessionManager.java 
> 11d25cc 
>   
> service/src/java/org/apache/hive/service/cli/thrift/ThriftBinaryCLIService.java
>  2b80adc 
>   service/src/java/org/apache/hive/service/cli/thrift/ThriftCLIService.java 
> 443c371 
>   
> service/src/java/org/apache/hive/service/cli/thrift/ThriftHttpCLIService.java 
> 4067106 
>   service/src/java/org/apache/hive/service/server/HiveServer2.java 124996c 
>   
> service/src/test/org/apache/hive/service/cli/session/TestSessionGlobalInitFile.java
>  66fc1fc 
> 
> Diff: https://reviews.apache.org/r/25245/diff/
> 
> 
> Testing
> ---
> 
> Manual testing.
> 
> 
> Thanks,
> 
> Vaibhav Gumashta
> 
>

[jira] [Updated] (HIVE-8066) [CBO] Handle charset

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-8066:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to cbo branch.

> [CBO] Handle charset 
> -
>
> Key: HIVE-8066
> URL: https://issues.apache.org/jira/browse/HIVE-8066
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8066.cbo.patch
>
>
> Disable cbo in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8066) [CBO] Handle charset

[Laljo John Pullokkaran (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130937#comment-14130937
 ] 

Laljo John Pullokkaran commented on HIVE-8066:
--

+1

> [CBO] Handle charset 
> -
>
> Key: HIVE-8066
> URL: https://issues.apache.org/jira/browse/HIVE-8066
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8066.cbo.patch
>
>
> Disable cbo in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8066) [CBO] Handle charset

[Laljo John Pullokkaran (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130938#comment-14130938
 ] 

Laljo John Pullokkaran commented on HIVE-8066:
--

Thanks Ashuthosh

> [CBO] Handle charset 
> -
>
> Key: HIVE-8066
> URL: https://issues.apache.org/jira/browse/HIVE-8066
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8066.cbo.patch
>
>
> Disable cbo in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Resolved] (HIVE-8064) CBO: decimal support is broken for some corner cases

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan resolved HIVE-8064.

Resolution: Fixed

committed to cbo branch.

> CBO: decimal support is broken for some corner cases
> 
>
> Key: HIVE-8064
> URL: https://issues.apache.org/jira/browse/HIVE-8064
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Minor
> Attachments: HIVE-8064.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8064) CBO: decimal support is broken for some corner cases

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8064?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130930#comment-14130930
 ] 

Ashutosh Chauhan commented on HIVE-8064:


+1

> CBO: decimal support is broken for some corner cases
> 
>
> Key: HIVE-8064
> URL: https://issues.apache.org/jira/browse/HIVE-8064
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
>Priority: Minor
> Attachments: HIVE-8064.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8057) Preserve Record Schema For top level Query, subquery enable cbo, Pull common predicate elements

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130926#comment-14130926
 ] 

Ashutosh Chauhan commented on HIVE-8057:


+1

> Preserve Record Schema For top level Query, subquery enable cbo, Pull common 
> predicate elements
> ---
>
> Key: HIVE-8057
> URL: https://issues.apache.org/jira/browse/HIVE-8057
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-8057.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8057) Preserve Record Schema For top level Query, subquery enable cbo, Pull common predicate elements

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-8057:
---
Resolution: Fixed
Status: Resolved  (was: Patch Available)

Committed to cbo branch.

> Preserve Record Schema For top level Query, subquery enable cbo, Pull common 
> predicate elements
> ---
>
> Key: HIVE-8057
> URL: https://issues.apache.org/jira/browse/HIVE-8057
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-8057.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-8058) Disable correlation opt for Tez

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-8058?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130925#comment-14130925
 ] 

Hive QA commented on HIVE-8058:
---



{color:green}Overall{color}: +1 all checks pass

Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12668151/HIVE-8058.1.patch

{color:green}SUCCESS:{color} +1 6197 tests passed

Test results: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/748/testReport
Console output: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/748/console
Test logs: 
http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-748/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12668151

> Disable correlation opt for Tez
> ---
>
> Key: HIVE-8058
> URL: https://issues.apache.org/jira/browse/HIVE-8058
> Project: Hive
>  Issue Type: Bug
>Reporter: Gunther Hagleitner
>Assignee: Gunther Hagleitner
> Attachments: HIVE-8058.1.patch
>
>
> Correlation optimizer needs more thought on Tez. Stuff like nested tags are 
> at odds with Tez (where we can expose more stages in a single DAG).
> For now: disable.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8066) [CBO] Handle charset

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-8066:
---
Status: Patch Available  (was: Open)

> [CBO] Handle charset 
> -
>
> Key: HIVE-8066
> URL: https://issues.apache.org/jira/browse/HIVE-8066
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8066.cbo.patch
>
>
> Disable cbo in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8066) [CBO] Handle charset

[Ashutosh Chauhan (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8066?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-8066:
---
Attachment: HIVE-8066.cbo.patch

> [CBO] Handle charset 
> -
>
> Key: HIVE-8066
> URL: https://issues.apache.org/jira/browse/HIVE-8066
> Project: Hive
>  Issue Type: Bug
>  Components: CBO
>Reporter: Ashutosh Chauhan
>Assignee: Ashutosh Chauhan
> Attachments: HIVE-8066.cbo.patch
>
>
> Disable cbo in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-8066) [CBO] Handle charset

[Ashutosh Chauhan (JIRA)]

Ashutosh Chauhan created HIVE-8066:
--

 Summary: [CBO] Handle charset 
 Key: HIVE-8066
 URL: https://issues.apache.org/jira/browse/HIVE-8066
 Project: Hive
  Issue Type: Bug
  Components: CBO
Reporter: Ashutosh Chauhan
Assignee: Ashutosh Chauhan


Disable cbo in such a case.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8057) Preserve Record Schema For top level Query, subquery enable cbo, Pull common predicate elements

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-8057:
-
Status: Patch Available  (was: Open)

> Preserve Record Schema For top level Query, subquery enable cbo, Pull common 
> predicate elements
> ---
>
> Key: HIVE-8057
> URL: https://issues.apache.org/jira/browse/HIVE-8057
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-8057.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8057) Preserve Record Schema For top level Query, subquery enable cbo, Pull common predicate elements

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-8057:
-
Attachment: HIVE-8057.patch

> Preserve Record Schema For top level Query, subquery enable cbo, Pull common 
> predicate elements
> ---
>
> Key: HIVE-8057
> URL: https://issues.apache.org/jira/browse/HIVE-8057
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
> Attachments: HIVE-8057.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8057) Preserve Record Schema For top level Query, subquery enable cbo, Pull common predicate elements

[Laljo John Pullokkaran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Laljo John Pullokkaran updated HIVE-8057:
-
Summary: Preserve Record Schema For top level Query, subquery enable cbo, 
Pull common predicate elements  (was: Preserve Record Schema For top level 
Query, subquery enable cbo)

> Preserve Record Schema For top level Query, subquery enable cbo, Pull common 
> predicate elements
> ---
>
> Key: HIVE-8057
> URL: https://issues.apache.org/jira/browse/HIVE-8057
> Project: Hive
>  Issue Type: Sub-task
>  Components: CBO
>Reporter: Laljo John Pullokkaran
>Assignee: Laljo John Pullokkaran
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (HIVE-5207) Support data encryption for Hive tables

[JIRA]

[ 
https://issues.apache.org/jira/browse/HIVE-5207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14130892#comment-14130892
 ] 

Sergio Peña commented on HIVE-5207:
---

I see this is a full encryption solution which looks prety good, but since HDFS 
encryption is comming soon (HDFS-6134) I'd like to make some improvements to 
make Hive work with it. This is meant to be compatible with HDFS encryption 
only. It still lacks the ability to encrypt tables on fly through Hive 
statements.

See HIVE-8065

> Support data encryption for Hive tables
> ---
>
> Key: HIVE-5207
> URL: https://issues.apache.org/jira/browse/HIVE-5207
> Project: Hive
>  Issue Type: New Feature
>Affects Versions: 0.12.0
>Reporter: Jerry Chen
>  Labels: Rhino
> Attachments: HIVE-5207.patch, HIVE-5207.patch
>
>   Original Estimate: 504h
>  Remaining Estimate: 504h
>
> For sensitive and legally protected data such as personal information, it is 
> a common practice that the data is stored encrypted in the file system. To 
> enable Hive with the ability to store and query the encrypted data is very 
> crucial for Hive data analysis in enterprise. 
>  
> When creating table, user can specify whether a table is an encrypted table 
> or not by specify a property in TBLPROPERTIES. Once an encrypted table is 
> created, query on the encrypted table is transparent as long as the 
> corresponding key management facilities are set in the running environment of 
> query. We can use hadoop crypto provided by HADOOP-9331 for underlying data 
> encryption and decryption. 
>  
> As to key management, we would support several common key management use 
> cases. First, the table key (data key) can be stored in the Hive metastore 
> associated with the table in properties. The table key can be explicit 
> specified or auto generated and will be encrypted with a master key. There 
> are cases that the data being processed is generated by other applications, 
> we need to support externally managed or imported table keys. Also, the data 
> generated by Hive may be consumed by other applications in the system. We 
> need to a tool or command for exporting the table key to a java keystore for 
> using externally.
>  
> To handle versions of Hadoop that do not have crypto support, we can avoid 
> compilation problems by segregating crypto API usage into separate files 
> (shims) to be included only if a flag is defined on the Ant command line 
> (something like –Dcrypto=true).



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (HIVE-8065) Support HDFS encryption functionality on Hive

[JIRA]

Sergio Peña created HIVE-8065:
-

 Summary: Support HDFS encryption functionality on Hive
 Key: HIVE-8065
 URL: https://issues.apache.org/jira/browse/HIVE-8065
 Project: Hive
  Issue Type: Improvement
Affects Versions: 0.13.1
Reporter: Sergio Peña
Assignee: Sergio Peña


The new encryption support on HDFS makes Hive incompatible and unusable when 
this feature is used.

HDFS encryption is designed so that an user can configure different encryption 
zones (or directories) for multi-tenant environments. An encryption zone has an 
exclusive encryption key, such as AES-128 or AES-256. Because of security 
compliance, the HDFS does not allow to move/rename files between encryption 
zones. Renames are allowed only inside the same encryption zone. A copy is 
allowed between encryption zones.

See HDFS-6134 for more details about HDFS encryption design.

Hive currently uses a scratch directory (like /tmp/$user/$random). This scratch 
directory is used for the output of intermediate data (between MR jobs) and for 
the final output of the hive query which is later moved to the table directory 
location.

If Hive tables are in different encryption zones than the scratch directory, 
then Hive won't be able to renames those files/directories, and it will make 
Hive unusable.

To handle this problem, we can change the scratch directory of the 
query/statement to be inside the same encryption zone of the table directory 
location. This way, the renaming process will be successful. 

Also, for statements that move files between encryption zones (i.e. LOAD DATA), 
a copy may be executed instead of a rename. This will cause an overhead when 
copying large data files, but it won't break the encryption on Hive.

Another security thing to consider is when using joins selects. If Hive joins 
different tables with different encryption key strengths, then the results of 
the select might break the security compliance of the tables. Let's say two 
tables with 128 bits and 256 bits encryption are joined, then the temporary 
results might be stored in the 128 bits encryption zone. This will conflict 
with the table encrypted with 256 bits temporary.

To fix this, Hive should be able to select the scratch directory that is more 
secured/encrypted in order to save the intermediate data temporary with no 
compliance issues.

For instance:

{noformat}
SELECT * FROM table-aes128 t1 JOIN table-aes256 t2 WHERE t1.id == t2.id;
{noformat}

- This should use a scratch directory (or staging directory) inside the 
table-aes256 table location.

{noformat}
INSERT OVERWRITE TABLE table-unencrypted SELECT * FROM table-aes1;
{noformat}

- This should use a scratch directory inside the table-aes1 location.


{noformat}
FROM table-unencrypted
INSERT OVERWRITE TABLE table-aes128 SELECT id, name
INSERT OVERWRITE TABLE table-aes256 SELECT id, name
{noformat}

- This should use a scratch directory on each of the tables locations.
- The first SELECT will have its scratch directory on table-aes128 directory.
- The second SELECT will have its scratch directory on table-aes256 directory.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8052) Vectorization: min() on TimeStamp datatype fails with error "Vector aggregate not implemented: min for type: TIMESTAMP"

[Matt McCline (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-8052:
---
Status: Patch Available  (was: In Progress)

> Vectorization: min() on TimeStamp datatype fails with error "Vector aggregate 
> not implemented: min for type: TIMESTAMP"
> ---
>
> Key: HIVE-8052
> URL: https://issues.apache.org/jira/browse/HIVE-8052
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8052.01.patch, HIVE-8052.02.patch
>
>
> Changes in HIVE-5760 to make explicit when timestamp and date can be 
> vectorized as Long were accidentally to strict for min, max, count, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8052) Vectorization: min() on TimeStamp datatype fails with error "Vector aggregate not implemented: min for type: TIMESTAMP"

[Matt McCline (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-8052:
---
Attachment: HIVE-8052.02.patch

> Vectorization: min() on TimeStamp datatype fails with error "Vector aggregate 
> not implemented: min for type: TIMESTAMP"
> ---
>
> Key: HIVE-8052
> URL: https://issues.apache.org/jira/browse/HIVE-8052
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8052.01.patch, HIVE-8052.02.patch
>
>
> Changes in HIVE-5760 to make explicit when timestamp and date can be 
> vectorized as Long were accidentally to strict for min, max, count, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8052) Vectorization: min() on TimeStamp datatype fails with error "Vector aggregate not implemented: min for type: TIMESTAMP"

[Matt McCline (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-8052:
---
Status: In Progress  (was: Patch Available)

> Vectorization: min() on TimeStamp datatype fails with error "Vector aggregate 
> not implemented: min for type: TIMESTAMP"
> ---
>
> Key: HIVE-8052
> URL: https://issues.apache.org/jira/browse/HIVE-8052
> Project: Hive
>  Issue Type: Bug
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-8052.01.patch
>
>
> Changes in HIVE-5760 to make explicit when timestamp and date can be 
> vectorized as Long were accidentally to strict for min, max, count, etc.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8061) improve the partition col stats update speed

[Pengcheng Xiong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pengcheng Xiong updated HIVE-8061:
--
Summary: improve the partition col stats update speed  (was: improve the 
speed of col stats update speed)

> improve the partition col stats update speed
> 
>
> Key: HIVE-8061
> URL: https://issues.apache.org/jira/browse/HIVE-8061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>Priority: Minor
> Attachments: HIVE-8061.1.patch
>
>
> We worked hard towards faster update stats for columns of a partition of a 
> table previously HIVE-7736
> and HIVE-7876
> Although there is some improvement, it is only correct in the first run. 
> There will be duplicate column stats later. Thanks to Eugene Koifman 's 
> comments.
> We fixed this in HIVE-7944 by reversing the patch.
> This JIRA ticket is my another try to improve the speed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Updated] (HIVE-8061) improve the speed of col stats update speed

[Damien Carol (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-8061?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Damien Carol updated HIVE-8061:
---
Description: 
We worked hard towards faster update stats for columns of a partition of a 
table previously HIVE-7736
and HIVE-7876
Although there is some improvement, it is only correct in the first run. There 
will be duplicate column stats later. Thanks to Eugene Koifman 's comments.
We fixed this in HIVE-7944 by reversing the patch.

This JIRA ticket is my another try to improve the speed.

  was:
We worked hard towards faster update stats for columns of a partition of a 
table previously https://issues.apache.org/jira/browse/HIVE-7736
and https://issues.apache.org/jira/browse/HIVE-7876
Although there is some improvement, it is only correct in the first run. There 
will be duplicate column stats later. Thanks to Eugene Koifman 's comments.
We fixed this in https://issues.apache.org/jira/browse/HIVE-7944 by reversing 
the patch.

This JIRA ticket is my another try to improve the speed.


> improve the speed of col stats update speed
> ---
>
> Key: HIVE-8061
> URL: https://issues.apache.org/jira/browse/HIVE-8061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>Priority: Minor
> Attachments: HIVE-8061.1.patch
>
>
> We worked hard towards faster update stats for columns of a partition of a 
> table previously HIVE-7736
> and HIVE-7876
> Although there is some improvement, it is only correct in the first run. 
> There will be duplicate column stats later. Thanks to Eugene Koifman 's 
> comments.
> We fixed this in HIVE-7944 by reversing the patch.
> This JIRA ticket is my another try to improve the speed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)