[jira] [Updated] (HIVE-22197) Common Merge join throwing class cast exception
[ https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22197: --- Attachment: HIVE-22197.03.patch > Common Merge join throwing class cast exception > > > Key: HIVE-22197 > URL: https://issues.apache.org/jira/browse/HIVE-22197 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch, > HIVE-22197.03.patch > > > In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed > and stored in the writable format, but the object inspector is initialized to > default. So when join operator is fetching the data from dummy store > operator, its getting the OI is Long and the row as LongWritable. This is > causing the class cast exception. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22197) Common Merge join throwing class cast exception
[ https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22197: --- Status: Patch Available (was: Open) > Common Merge join throwing class cast exception > > > Key: HIVE-22197 > URL: https://issues.apache.org/jira/browse/HIVE-22197 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch, > HIVE-22197.03.patch > > > In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed > and stored in the writable format, but the object inspector is initialized to > default. So when join operator is fetching the data from dummy store > operator, its getting the OI is Long and the row as LongWritable. This is > causing the class cast exception. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22197) Common Merge join throwing class cast exception
[ https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22197: --- Status: Open (was: Patch Available) > Common Merge join throwing class cast exception > > > Key: HIVE-22197 > URL: https://issues.apache.org/jira/browse/HIVE-22197 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch, > HIVE-22197.03.patch > > > In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed > and stored in the writable format, but the object inspector is initialized to > default. So when join operator is fetching the data from dummy store > operator, its getting the OI is Long and the row as LongWritable. This is > causing the class cast exception. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Comment Edited] (HIVE-22177) Beeline query for special character like £ does not give back result
[
https://issues.apache.org/jira/browse/HIVE-22177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931023#comment-16931023
]
philipse edited comment on HIVE-22177 at 9/17/19 3:24 AM:
--
hi Sanjay
yes, i am using beeline , HDP 2.5.2 and hive 1.1.0
if reflect is forbidden,may be UDF or utf file format(as the comment above
talks) will be helpful.
it seems i have problems uploading screenshots
!image-2019-09-17-10-36-11-509.png!
was (Author: 小郭飞飞刀):
hi Sanjay
yes, i am using beeline , HDP 2.5.2 and hive 1.1.0
if reflect is forbidden,may be UDF or utf file format(as the comment above
talks) will be helpful.
!image-2019-09-17-10-36-11-509.png!
> Beeline query for special character like £ does not give back result
>
>
> Key: HIVE-22177
> URL: https://issues.apache.org/jira/browse/HIVE-22177
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0, 2.3.6
> Environment: Cloudera 5.15.1 Hive 1.1.0
> beeline and Impala
>Reporter: Sanjay Mishra
>Priority: Major
>
> I have a table in hive with a column having special character like £. When I
> try to a query as simple as 'SELECT * FROM TABLE WHERE COL = "£"; ', the
> query does not come back with any result.
> The table data is UTF-8 encoded. The sql file that contains the above sql and
> run via beeline is also UTF-8 encoded.
> Interestingly, if I try to create the run the same query using impala against
> the same table, I get back the columns containing the £ symbol.
> Also when the data from the table is exported (SELECT * FROM THE TABLE and
> save the output to csv file) the column with special character is shown as
> NULL.
>
> Steps to replicate:
> # create test.sql file with the below content:
> #
> {code:java}
> CREATE TABLE T1 (test String);
> INSERT INTO T1 VALUES('£');
> {code}
> # beeline>!run test.sql
> # create another file test_select.sql with the below content:
> #
> {code:java}
> SELECT * FROM T1 WHERE test='£';{code}
> # beeline>!run test_select.sql
> # No results will be returned back
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22177) Beeline query for special character like £ does not give back result
[
https://issues.apache.org/jira/browse/HIVE-22177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16931023#comment-16931023
]
philipse commented on HIVE-22177:
-
hi Sanjay
yes, i am using beeline , HDP 2.5.2 and hive 1.1.0
if reflect is forbidden,may be UDF or utf file format(as the comment above
talks) will be helpful.
!image-2019-09-17-10-36-11-509.png!
> Beeline query for special character like £ does not give back result
>
>
> Key: HIVE-22177
> URL: https://issues.apache.org/jira/browse/HIVE-22177
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0, 2.3.6
> Environment: Cloudera 5.15.1 Hive 1.1.0
> beeline and Impala
>Reporter: Sanjay Mishra
>Priority: Major
>
> I have a table in hive with a column having special character like £. When I
> try to a query as simple as 'SELECT * FROM TABLE WHERE COL = "£"; ', the
> query does not come back with any result.
> The table data is UTF-8 encoded. The sql file that contains the above sql and
> run via beeline is also UTF-8 encoded.
> Interestingly, if I try to create the run the same query using impala against
> the same table, I get back the columns containing the £ symbol.
> Also when the data from the table is exported (SELECT * FROM THE TABLE and
> save the output to csv file) the column with special character is shown as
> NULL.
>
> Steps to replicate:
> # create test.sql file with the below content:
> #
> {code:java}
> CREATE TABLE T1 (test String);
> INSERT INTO T1 VALUES('£');
> {code}
> # beeline>!run test.sql
> # create another file test_select.sql with the below content:
> #
> {code:java}
> SELECT * FROM T1 WHERE test='£';{code}
> # beeline>!run test_select.sql
> # No results will be returned back
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22209) Creating a materialized view with no tables should be handled more gracefully
[
https://issues.apache.org/jira/browse/HIVE-22209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-22209:
--
> Creating a materialized view with no tables should be handled more gracefully
> -
>
> Key: HIVE-22209
> URL: https://issues.apache.org/jira/browse/HIVE-22209
> Project: Hive
> Issue Type: Bug
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: John Sherman
>Priority: Minor
>
> Currently, materialized views without a table reference are not supported.
> However, instead of printing a clear message about it, when a materialized
> view is created without a table reference, we fail with an unclear message.
> {code}
> > create materialized view mv_test1 as select 5;
> (...)
> ERROR : FAILED: Execution Error, return code 1 from
> org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Add request
> failed :
> INSERT INTO MV_TABLES_USED (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) )
> INFO : Completed executing
> command(queryId=hive_20190916203511_b609cccf-f5e3-45dd-abfd-6e869d94e39a);
> Time taken: 10.469 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaExcep
> tion(message:Add request failed : INSERT INTO MV_TABLES_USED
> (MV_CREATION_METADATA_ID,TBL_ID) VALUES (?,?) ) (state=08S01,code=1)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-22042: -- Assignee: John Sherman > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: John Sherman >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-22042: -- Assignee: (was: Jesus Camacho Rodriguez) > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22042) Set hive.exec.dynamic.partition.mode=nonstrict by default
[ https://issues.apache.org/jira/browse/HIVE-22042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-22042: -- Assignee: Jesus Camacho Rodriguez > Set hive.exec.dynamic.partition.mode=nonstrict by default > - > > Key: HIVE-22042 > URL: https://issues.apache.org/jira/browse/HIVE-22042 > Project: Hive > Issue Type: Bug >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez >Priority: Minor > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-20950) Transform OUTER join with condition always true into INNER join
[ https://issues.apache.org/jira/browse/HIVE-20950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez reassigned HIVE-20950: -- Assignee: Steve Carlin > Transform OUTER join with condition always true into INNER join > --- > > Key: HIVE-20950 > URL: https://issues.apache.org/jira/browse/HIVE-20950 > Project: Hive > Issue Type: Improvement > Components: CBO >Reporter: Jesus Camacho Rodriguez >Assignee: Steve Carlin >Priority: Major > > For instance, it may help the join reordering algorithm. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930863#comment-16930863
]
Hive QA commented on HIVE-20983:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980424/HIVE-20983.5.patch
{color:green}SUCCESS:{color} +1 due to 7 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16760 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18618/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18618/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18618/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980424 - PreCommit-HIVE-Build
> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch,
> HIVE-20983.3.patch, HIVE-20983.4.patch, HIVE-20983.5.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in
> place, but an accurate estimate does not always result in a low number of
> collisions.
> The hashtables which contain a very small number of items tend to lose their
> O(1) lookup performance where there are collisions. Since collisions are easy
> to detect within the fast hashtable implementation, a rehashing to a higher
> size will help these small hashtables avoid collisions and go back to O(1)
> perf.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22177) Beeline query for special character like £ does not give back result
[
https://issues.apache.org/jira/browse/HIVE-22177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930821#comment-16930821
]
Sanjay Mishra commented on HIVE-22177:
--
Hi [~小郭飞飞刀], using reflect is blacklisted and I cant really go ahead to use
this approach. Can you please confirm in the above comment example, you used
beeline and not hive CLI?
> Beeline query for special character like £ does not give back result
>
>
> Key: HIVE-22177
> URL: https://issues.apache.org/jira/browse/HIVE-22177
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0, 2.3.6
> Environment: Cloudera 5.15.1 Hive 1.1.0
> beeline and Impala
>Reporter: Sanjay Mishra
>Priority: Major
>
> I have a table in hive with a column having special character like £. When I
> try to a query as simple as 'SELECT * FROM TABLE WHERE COL = "£"; ', the
> query does not come back with any result.
> The table data is UTF-8 encoded. The sql file that contains the above sql and
> run via beeline is also UTF-8 encoded.
> Interestingly, if I try to create the run the same query using impala against
> the same table, I get back the columns containing the £ symbol.
> Also when the data from the table is exported (SELECT * FROM THE TABLE and
> save the output to csv file) the column with special character is shown as
> NULL.
>
> Steps to replicate:
> # create test.sql file with the below content:
> #
> {code:java}
> CREATE TABLE T1 (test String);
> INSERT INTO T1 VALUES('£');
> {code}
> # beeline>!run test.sql
> # create another file test_select.sql with the below content:
> #
> {code:java}
> SELECT * FROM T1 WHERE test='£';{code}
> # beeline>!run test_select.sql
> # No results will be returned back
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[
https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930818#comment-16930818
]
Hive QA commented on HIVE-20983:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
3s{color} | {color:blue} ql in master has 1572 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18618/dev-support/hive-personality.sh
|
| git revision | master / 6ecd190 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18618/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Vectorization: Scale up small hashtables, when collisions are detected
> --
>
> Key: HIVE-20983
> URL: https://issues.apache.org/jira/browse/HIVE-20983
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Mustafa Iman
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch,
> HIVE-20983.3.patch, HIVE-20983.4.patch, HIVE-20983.5.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Hive's hashtable estimates are getting better with HyperLogLog stats in
> place, but an accurate estimate does not always result in a low number of
> collisions.
> The hashtables which contain a very small number of items tend to lose their
> O(1) lookup performance where there are collisions. Since collisions are easy
> to detect within the fast hashtable implementation, a rehashing to a higher
> size will help these small hashtables avoid collisions and go back to O(1)
> perf.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ashutosh Chauhan updated HIVE-22169:
Fix Version/s: 4.0.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master. Thanks, Gopal!
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch,
> HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21875) Implement drop partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930790#comment-16930790
]
Hive QA commented on HIVE-21875:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980417/HIVE-21875.05.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18617/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18617/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18617/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL
https://issues.apache.org/jira/secure/attachment/12980417/HIVE-21875.05.patch
was found in seen patch url's cache and a test was probably run already on it.
Aborting...
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980417 - PreCommit-HIVE-Build
> Implement drop partition related methods on temporary tables
>
>
> Key: HIVE-21875
> URL: https://issues.apache.org/jira/browse/HIVE-21875
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-21875.01.patch, HIVE-21875.02.patch,
> HIVE-21875.03.patch, HIVE-21875.04.patch, HIVE-21875.05.patch
>
>
> IMetaStoreClient exposes the following methods related to dropping partitions:
> {code:java}
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, boolean deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, boolean deleteData);
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, PartitionDropOptions options);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, PartitionDropOptions options);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> boolean dropPartition(String db_name, String tbl_name, String name, boolean
> deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name, String
> name, boolean deleteData){code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21151) Fix support of dot in quoted identifiers
[ https://issues.apache.org/jira/browse/HIVE-21151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thejas M Nair updated HIVE-21151: - Summary: Fix support of dot in quoted identifiers (was: Fix support of dot in quted identifiers) > Fix support of dot in quoted identifiers > > > Key: HIVE-21151 > URL: https://issues.apache.org/jira/browse/HIVE-21151 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Priority: Major > > Dot should be allowed in quoted identifiers; but because there are some > methods which rely on splitting the "dbtable" string by the dot HIVE-16907 > have removed that option. > https://github.com/apache/hive/blob/dfd63d97902b359e1643e955a4d070ac983debd5/ql/src/java/org/apache/hadoop/hive/ql/exec/Utilities.java#L2180 -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[
https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930776#comment-16930776
]
Hive QA commented on HIVE-22202:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980416/HIVE-22202.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16760 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18616/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18616/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18616/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980416 - PreCommit-HIVE-Build
> Remove unnecessary code from ObjectStore.java
> -
>
> Key: HIVE-22202
> URL: https://issues.apache.org/jira/browse/HIVE-22202
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Hankó Gergely
>Assignee: Hankó Gergely
>Priority: Minor
> Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch
>
>
> Remove unnecessary code
> * remove unnecessary/unused variables
> * remove unused private methods
> * remove unnecessary throws declarations
> * remove unnecessary conditionals that are guaranteed to always give the
> same result
> * simplify collection handling
> ** use member sort() instead of Collections.sort()
> ** use addAll() instead of adding one-by-one in a loop
> * remove Javadoc that doesn't give additional information
> * use try with resource where possible
> * remove unnecessary explicit type arguments
> * remove redundant warning suppression
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[
https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930739#comment-16930739
]
Hive QA commented on HIVE-22202:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
22s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
11s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
24s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
22s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 11 new + 356 unchanged - 55 fixed = 367 total (was 411) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
19s{color} | {color:green} standalone-metastore/metastore-server generated 0
new + 170 unchanged - 8 fixed = 170 total (was 178) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18616/dev-support/hive-personality.sh
|
| git revision | master / 1880bc6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18616/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18616/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove unnecessary code from ObjectStore.java
> -
>
> Key: HIVE-22202
> URL: https://issues.apache.org/jira/browse/HIVE-22202
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Hankó Gergely
>Assignee: Hankó Gergely
>Priority: Minor
> Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch
>
>
> Remove unnecessary code
> * remove unnecessary/unused variables
> * remove unused private methods
> * remove unnecessary throws declarations
> * remove unnecessary conditionals that are guaranteed to always give the
> same result
> * simplify collection handling
> ** use member sort() instead of Collections.sort()
> ** use addAll() instead of adding one-by-one in a loop
> * remove Javadoc that doesn't give additional information
> * use try with resource where possible
> * remove unnecessary explicit type arguments
> * remove redundant warning suppression
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (HIVE-19261) Avro SerDe's InstanceCache should not be synchronized on retrieve
[
https://issues.apache.org/jira/browse/HIVE-19261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929610#comment-16929610
]
Mithun Radhakrishnan edited comment on HIVE-19261 at 9/16/19 5:35 PM:
--
Hello, [~shengzhixia], et al.
This looks like a good fix, generally. A couple of suggestions:
# Please consider the rephrasing the ending of {{InstanceCache::retrieve()}}
as follows, for readability:
##
{code:java}
Instance newInstance = makeInstance(hv, seenSchemas);
instance = cache.putIfAbsent(hv, newInstance);
return instance == null? newInstance : instance;{code}
# Please remove the documentation regarding
{{ConcurrentHashMap::computeIfAbsent()}}; it won't work here.
{{makeInstance()}} lands up making reentrant calls to
{{InstanceCache::retrieve()}}. {{computeIfAbsent()}} requires that the
{{cache}} not be modified within a single call to the {{putIfAbsent()}} lambda
(i.e. {{hv -> makeInstance(hv, seenSchemas)}}. As such, this will cause
{{retrieve()}} to hang on the first reentrant call.
was (Author: mithun):
Hello, [~shengzhixia], et al.
This looks like a good fix, generally. A couple of suggestions:
# Please consider the rephrasing the ending of {{InstanceCache::retrieve()}}
as follows, for readability:
##
{code:java}
Instance newInstance = makeInstance(hv, seenSchemas);
instance = cache.putIfAbsent(hv, newInstance);
return instance == null? newInstance : instance;{code}
# Please remove the documentation regarding
{{ConcurrentHashMap::computeIfAbsent()}}; it won't work here.
{{makeInstance()}} lands up making reentrant calls to
{{InstanceCache::retrieve()}}. {{computeIfAbsent()}} requires that the
{{cache}} not be modified within a single call to the {{putIfAbsent()}} lambda
(i.e. {{hv -> makeInstance(hv, seenSchemas)}}. As such, this will cause
{{retrieve()}} to hang on the first reentrant call.
> Avro SerDe's InstanceCache should not be synchronized on retrieve
> -
>
> Key: HIVE-19261
> URL: https://issues.apache.org/jira/browse/HIVE-19261
> Project: Hive
> Issue Type: Improvement
>Reporter: Fangshi Li
>Assignee: Fangshi Li
>Priority: Major
> Attachments: HIVE-19261.1.patch
>
>
> In HIVE-16175, upstream made a patch to fix the thread safety issue in
> AvroSerDe's InstanceCache. This fix made the retrieve method in InstanceCache
> synchronized. While it should make InstanceCache thread-safe, making retrieve
> synchronized for the cache can be expensive in highly concurrent environment
> like Spark, as multiple threads need to be synchronized on entering the
> entire retrieve method.
> We are proposing another way to fix this thread safety issue by making the
> underlying map of InstanceCache as ConcurrentHashMap. Ideally, we can use
> atomic computeIfAbsent in the retrieve method to avoid synchronizing the
> entire method.
> While computeIfAbsent is only available on java 8 and java 7 is still
> supported in Hive,
> we use a pattern to simulate the behavior of computeIfAbsent. In the future,
> we should move to computeIfAbsent when Hive requires java 8.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-21975) Fix incremental compilation
[
https://issues.apache.org/jira/browse/HIVE-21975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Steve Carlin reassigned HIVE-21975:
---
Assignee: Steve Carlin
> Fix incremental compilation
> ---
>
> Key: HIVE-21975
> URL: https://issues.apache.org/jira/browse/HIVE-21975
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Steve Carlin
>Priority: Major
>
> we have an incremental compilation issue around SA ; mostly because of "?
> extends Serializable"
> it could be reproduced with:
> {code}
> git clean -dfx
> mvn install -pl ql -am -DskipTests
> touch `find . -name Sema*A*java` `find . -name Task*Factory.java`
> mvn install -pl ql -DskipTests
> {code}
> error is:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile
> (default-compile) on project hive-exec: Compilation failure: Compilation
> failure:
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[12573,60]
> incompatible types: java.util.List extends java.io.Serializable>> cannot be converted to
> java.util.List>
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[15187,49]
> incompatible types: java.util.List>
> cannot be converted to java.util.List extends java.io.Serializable>>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21975) Fix incremental compilation
[
https://issues.apache.org/jira/browse/HIVE-21975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930726#comment-16930726
]
Steve Carlin commented on HIVE-21975:
-
Ah, yup, that makes perfect sense.
I'm gonna search the code base and get rid of the "extends Serializable" in all
necessary places (excluding, of course, Task.java)
Thanks!
> Fix incremental compilation
> ---
>
> Key: HIVE-21975
> URL: https://issues.apache.org/jira/browse/HIVE-21975
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Priority: Major
>
> we have an incremental compilation issue around SA ; mostly because of "?
> extends Serializable"
> it could be reproduced with:
> {code}
> git clean -dfx
> mvn install -pl ql -am -DskipTests
> touch `find . -name Sema*A*java` `find . -name Task*Factory.java`
> mvn install -pl ql -DskipTests
> {code}
> error is:
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.6.1:compile
> (default-compile) on project hive-exec: Compilation failure: Compilation
> failure:
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[12573,60]
> incompatible types: java.util.List extends java.io.Serializable>> cannot be converted to
> java.util.List>
> [ERROR]
> /mnt/work/hwx/hive/master/ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java:[15187,49]
> incompatible types: java.util.List>
> cannot be converted to java.util.List extends java.io.Serializable>>
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930719#comment-16930719
]
Hive QA commented on HIVE-22195:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980415/HIVE-22195.6.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16763 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18615/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18615/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18615/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980415 - PreCommit-HIVE-Build
> Configure authentication type for Zookeeper when different from the default
> cluster wide
>
>
> Key: HIVE-22195
> URL: https://issues.apache.org/jira/browse/HIVE-22195
> Project: Hive
> Issue Type: Improvement
>Reporter: Denys Kuzmenko
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch,
> HIVE-22195.4.patch, HIVE-22195.5.patch, HIVE-22195.6.patch
>
>
> This could be useful in case when cluster is kerberized, but Zookeeper is not.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-20983: Attachment: HIVE-20983.5.patch > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch, > HIVE-20983.3.patch, HIVE-20983.4.patch, HIVE-20983.5.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-20983: Status: In Progress (was: Patch Available) > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch, > HIVE-20983.3.patch, HIVE-20983.4.patch, HIVE-20983.5.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-20983) Vectorization: Scale up small hashtables, when collisions are detected
[ https://issues.apache.org/jira/browse/HIVE-20983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mustafa Iman updated HIVE-20983: Status: Patch Available (was: In Progress) > Vectorization: Scale up small hashtables, when collisions are detected > -- > > Key: HIVE-20983 > URL: https://issues.apache.org/jira/browse/HIVE-20983 > Project: Hive > Issue Type: Bug >Reporter: Gopal V >Assignee: Mustafa Iman >Priority: Major > Labels: pull-request-available > Attachments: HIVE-20983.1.patch, HIVE-20983.2.patch, > HIVE-20983.3.patch, HIVE-20983.4.patch, HIVE-20983.5.patch > > Time Spent: 10m > Remaining Estimate: 0h > > Hive's hashtable estimates are getting better with HyperLogLog stats in > place, but an accurate estimate does not always result in a low number of > collisions. > The hashtables which contain a very small number of items tend to lose their > O(1) lookup performance where there are collisions. Since collisions are easy > to detect within the fast hashtable implementation, a rehashing to a higher > size will help these small hashtables avoid collisions and go back to O(1) > perf. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[
https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930684#comment-16930684
]
Hive QA commented on HIVE-22195:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
50s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
34s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
25s{color} | {color:blue} llap-client in master has 26 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
7s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
40s{color} | {color:blue} service in master has 49 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red} 0m
22s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red} 0m
23s{color} | {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red} 0m 23s{color}
| {color:red} service in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
11s{color} | {color:red} llap-client: The patch generated 5 new + 49 unchanged
- 1 fixed = 54 total (was 50) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 0m
24s{color} | {color:red} service in the patch failed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18615/dev-support/hive-personality.sh
|
| git revision | master / 1880bc6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| mvninstall |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18615/yetus/patch-mvninstall-service.txt
|
| compile |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18615/yetus/patch-compile-service.txt
|
| javac |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18615/yetus/patch-compile-service.txt
|
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18615/yetus/diff-checkstyle-llap-client.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18615/yetus/patch-findbugs-service.txt
|
| modules | C: common llap-client standalone-metastore/metastore-server service
U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18615/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Configure authentication type for Zookeeper when different from the default
> cluster wide
>
>
> Key: HIVE-22195
>
[jira] [Commented] (HIVE-22205) Upgrade zookeeper and curator versions
[ https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930676#comment-16930676 ] Tamas Penzes commented on HIVE-22205: - [~ngangam] I think you have more than one ZooKeeper on your classpath. * Curator includes 3.5.4-beta which must be excluded * you add 3.5.5 * you get (probably older) ZooKeeper dependencies transitively through e.g. hadoop, which must be excluded > Upgrade zookeeper and curator versions > -- > > Key: HIVE-22205 > URL: https://issues.apache.org/jira/browse/HIVE-22205 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-22205.patch > > > Other components like hadoop have switched to using new ZK versions. So these > jars end up in classpath for hive services and could cause issues due to > in-compatible curator versions that hive uses. > So it makes sense for hive to upgrade the ZK and curator versions to try to > keep up. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21875) Implement drop partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930670#comment-16930670
]
Hive QA commented on HIVE-21875:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980417/HIVE-21875.05.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16832 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18614/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18614/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18614/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980417 - PreCommit-HIVE-Build
> Implement drop partition related methods on temporary tables
>
>
> Key: HIVE-21875
> URL: https://issues.apache.org/jira/browse/HIVE-21875
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-21875.01.patch, HIVE-21875.02.patch,
> HIVE-21875.03.patch, HIVE-21875.04.patch, HIVE-21875.05.patch
>
>
> IMetaStoreClient exposes the following methods related to dropping partitions:
> {code:java}
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, boolean deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, boolean deleteData);
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, PartitionDropOptions options);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, PartitionDropOptions options);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> boolean dropPartition(String db_name, String tbl_name, String name, boolean
> deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name, String
> name, boolean deleteData){code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-18451) Support UDTF Operator in SimpleFetchOptimizer
[
https://issues.apache.org/jira/browse/HIVE-18451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-18451:
--
Assignee: (was: Jesus Camacho Rodriguez)
> Support UDTF Operator in SimpleFetchOptimizer
> -
>
> Key: HIVE-18451
> URL: https://issues.apache.org/jira/browse/HIVE-18451
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-18416, simple insert into table values{} is still resulting in
> launching of job as earlier which can be avoided by optimizing that job to
> FetchTask.
> This issue is to extend SimpleFetchOptimizer, which does not fire on insert
> queries as it does not handle UDTF.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-18451) Support UDTF Operator in SimpleFetchOptimizer
[
https://issues.apache.org/jira/browse/HIVE-18451?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez reassigned HIVE-18451:
--
Assignee: Jesus Camacho Rodriguez
> Support UDTF Operator in SimpleFetchOptimizer
> -
>
> Key: HIVE-18451
> URL: https://issues.apache.org/jira/browse/HIVE-18451
> Project: Hive
> Issue Type: Improvement
> Components: Physical Optimizer
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> After HIVE-18416, simple insert into table values{} is still resulting in
> launching of job as earlier which can be avoided by optimizing that job to
> FetchTask.
> This issue is to extend SimpleFetchOptimizer, which does not fire on insert
> queries as it does not handle UDTF.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21875) Implement drop partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930648#comment-16930648
]
Hive QA commented on HIVE-21875:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
45s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
11s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
13s{color} | {color:blue} ql in master has 1573 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
18s{color} | {color:green} standalone-metastore/metastore-server: The patch
generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 2 new + 17 unchanged - 0 fixed
= 19 total (was 17) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 5m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
25s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18614/dev-support/hive-personality.sh
|
| git revision | master / 1880bc6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18614/yetus/diff-checkstyle-ql.txt
|
| modules | C: standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18614/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Implement drop partition related methods on temporary tables
>
>
> Key: HIVE-21875
> URL: https://issues.apache.org/jira/browse/HIVE-21875
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-21875.01.patch, HIVE-21875.02.patch,
> HIVE-21875.03.patch, HIVE-21875.04.patch, HIVE-21875.05.patch
>
>
> IMetaStoreClient exposes the following methods related to dropping partitions:
> {code:java}
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, boolean deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name,
[jira] [Commented] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[
https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930620#comment-16930620
]
Hive QA commented on HIVE-22202:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980397/HIVE-22202.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16760 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18613/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18613/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18613/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980397 - PreCommit-HIVE-Build
> Remove unnecessary code from ObjectStore.java
> -
>
> Key: HIVE-22202
> URL: https://issues.apache.org/jira/browse/HIVE-22202
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Hankó Gergely
>Assignee: Hankó Gergely
>Priority: Minor
> Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch
>
>
> Remove unnecessary code
> * remove unnecessary/unused variables
> * remove unused private methods
> * remove unnecessary throws declarations
> * remove unnecessary conditionals that are guaranteed to always give the
> same result
> * simplify collection handling
> ** use member sort() instead of Collections.sort()
> ** use addAll() instead of adding one-by-one in a loop
> * remove Javadoc that doesn't give additional information
> * use try with resource where possible
> * remove unnecessary explicit type arguments
> * remove redundant warning suppression
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21875) Implement drop partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-21875:
-
Attachment: HIVE-21875.05.patch
> Implement drop partition related methods on temporary tables
>
>
> Key: HIVE-21875
> URL: https://issues.apache.org/jira/browse/HIVE-21875
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-21875.01.patch, HIVE-21875.02.patch,
> HIVE-21875.03.patch, HIVE-21875.04.patch, HIVE-21875.05.patch
>
>
> IMetaStoreClient exposes the following methods related to dropping partitions:
> {code:java}
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, boolean deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, boolean deleteData);
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, PartitionDropOptions options);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, PartitionDropOptions options);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> boolean dropPartition(String db_name, String tbl_name, String name, boolean
> deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name, String
> name, boolean deleteData){code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely updated HIVE-22202: - Status: Patch Available (was: In Progress) > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch > > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * simplify collection handling > ** use member sort() instead of Collections.sort() > ** use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely updated HIVE-22202: - Attachment: HIVE-22202.2.patch > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch > > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * simplify collection handling > ** use member sort() instead of Collections.sort() > ** use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely updated HIVE-22202: - Status: In Progress (was: Patch Available) > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > Attachments: HIVE-22202.1.patch, HIVE-22202.2.patch > > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * simplify collection handling > ** use member sort() instead of Collections.sort() > ** use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22195) Configure authentication type for Zookeeper when different from the default cluster wide
[ https://issues.apache.org/jira/browse/HIVE-22195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denys Kuzmenko updated HIVE-22195: -- Attachment: HIVE-22195.6.patch > Configure authentication type for Zookeeper when different from the default > cluster wide > > > Key: HIVE-22195 > URL: https://issues.apache.org/jira/browse/HIVE-22195 > Project: Hive > Issue Type: Improvement >Reporter: Denys Kuzmenko >Assignee: Denys Kuzmenko >Priority: Major > Attachments: HIVE-22195.1.patch, HIVE-22195.2.patch, > HIVE-22195.4.patch, HIVE-22195.5.patch, HIVE-22195.6.patch > > > This could be useful in case when cluster is kerberized, but Zookeeper is not. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[
https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930577#comment-16930577
]
Hive QA commented on HIVE-22202:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
17s{color} | {color:blue} standalone-metastore/metastore-server in master has
178 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
21s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 17 new + 367 unchanged - 44 fixed = 384 total (was 411) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
20s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 170 unchanged - 8 fixed = 171 total (was 178) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 1s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | org.apache.hadoop.hive.metastore.ObjectStore$16.run(List) concatenates
strings using + in a loop At ObjectStore.java:in a loop At
ObjectStore.java:[line 8601] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18613/dev-support/hive-personality.sh
|
| git revision | master / 1880bc6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18613/yetus/diff-checkstyle-standalone-metastore_metastore-server.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18613/yetus/new-findbugs-standalone-metastore_metastore-server.html
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18613/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove unnecessary code from ObjectStore.java
> -
>
> Key: HIVE-22202
> URL: https://issues.apache.org/jira/browse/HIVE-22202
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Hankó Gergely
>Assignee: Hankó Gergely
>Priority: Minor
> Attachments: HIVE-22202.1.patch
>
>
> Remove unnecessary code
> * remove unnecessary/unused variables
> * remove unused private methods
> * remove unnecessary throws declarations
> * remove unnecessary conditionals that are guaranteed to always give the
> same result
> * simplify collection handling
> ** use member sort() instead of Collections.sort()
> ** use addAll() instead of adding one-by-one in a loop
> * remove Javadoc that doesn't give additional information
> * use try with resource where
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930533#comment-16930533
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980385/HIVE-21884.10.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18612/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18612/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18612/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2019-09-16 13:23:43.319
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-18612/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-09-16 13:23:43.323
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 1880bc6 HIVE-22189: HMS Translation: Enforce strict locations
for managed vs external tables.(Naveen Gangam, reviewed by Thejas Nair)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 1880bc6 HIVE-22189: HMS Translation: Enforce strict locations
for managed vs external tables.(Naveen Gangam, reviewed by Thejas Nair)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-09-16 13:23:44.499
+ rm -rf ../yetus_PreCommit-HIVE-Build-18612
+ mkdir ../yetus_PreCommit-HIVE-Build-18612
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-18612
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-18612/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java:148
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java' with
conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java:153
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java' with conflicts.
error: patch failed:
standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java:257799
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-common/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/ThriftHiveMetastore.java'
with conflicts.
error: patch failed:
standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py:45035
Falling back to three-way merge...
Applied patch to
'standalone-metastore/metastore-common/src/gen/thrift/gen-py/hive_metastore/ThriftHiveMetastore.py'
with conflicts.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:920: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:930: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:941: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:1501: trailing whitespace.
/data/hiveptest/working/scratch/build.patch:1712: trailing whitespace.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java:148
Falling back to three-way merge...
Applied patch to
'ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzerFactory.java' with
conflicts.
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/plan/HiveOperation.java:153
Falling back to three-way
[jira] [Commented] (HIVE-22197) Common Merge join throwing class cast exception
[
https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930529#comment-16930529
]
Hive QA commented on HIVE-22197:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980383/HIVE-22197.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16761 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge_test_dummy_operator]
(batchId=53)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18611/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18611/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18611/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980383 - PreCommit-HIVE-Build
> Common Merge join throwing class cast exception
>
>
> Key: HIVE-22197
> URL: https://issues.apache.org/jira/browse/HIVE-22197
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch
>
>
> In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed
> and stored in the writable format, but the object inspector is initialized to
> default. So when join operator is fetching the data from dummy store
> operator, its getting the OI is Long and the row as LongWritable. This is
> causing the class cast exception.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22193) Graceful Shutdown HiveServer2
[ https://issues.apache.org/jira/browse/HIVE-22193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930523#comment-16930523 ] Hui An commented on HIVE-22193: --- There is one way that by deleting HS2 info from znode, HS2 will shutdown itself after executing all active querys. But I think it still has some problems. 1. it will not stop any new users from connecting to it, also old living sessions trying to interact with it. 2. how to handle HS2 which does not set hive.server2.support.dynamic.service.discovery to true? > Graceful Shutdown HiveServer2 > - > > Key: HIVE-22193 > URL: https://issues.apache.org/jira/browse/HIVE-22193 > Project: Hive > Issue Type: Improvement > Components: Server Infrastructure >Reporter: chenshiyun >Priority: Major > > We have a lot of HiveSever2 servers deployed on production environment (about > 10 nodes). > However, if we want to change configuration or add patches, we would have to > restart all of them one by one. So all the Hive Sql job running on the server > will be defeated, and there may be some mistakes come up on the jdbc client > occasionally. > In the proposed changes, planning to add Graceful Shutdown HiveSever2 method > to avoid affecting the production environment jobs -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22197) Common Merge join throwing class cast exception
[
https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930506#comment-16930506
]
Hive QA commented on HIVE-22197:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 2m
0s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
24s{color} | {color:blue} shims/0.23 in master has 7 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
6s{color} | {color:blue} ql in master has 1573 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
28s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 1 new + 0 unchanged - 0 fixed
= 1 total (was 0) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 29 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m 9s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18611/dev-support/hive-personality.sh
|
| git revision | master / 1880bc6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18611/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18611/yetus/whitespace-tabs.txt
|
| modules | C: shims/0.23 ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18611/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Common Merge join throwing class cast exception
>
>
> Key: HIVE-22197
> URL: https://issues.apache.org/jira/browse/HIVE-22197
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch
>
>
> In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed
> and stored in the writable format, but the object inspector is initialized to
> default. So when join operator is fetching the data from dummy store
> operator, its getting the OI is Long and the row as LongWritable. This is
> causing the class cast exception.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21875) Implement drop partition related methods on temporary tables
[
https://issues.apache.org/jira/browse/HIVE-21875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-21875:
-
Attachment: HIVE-21875.04.patch
> Implement drop partition related methods on temporary tables
>
>
> Key: HIVE-21875
> URL: https://issues.apache.org/jira/browse/HIVE-21875
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Attachments: HIVE-21875.01.patch, HIVE-21875.02.patch,
> HIVE-21875.03.patch, HIVE-21875.04.patch
>
>
> IMetaStoreClient exposes the following methods related to dropping partitions:
> {code:java}
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, boolean deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, boolean deleteData);
> boolean dropPartition(String db_name, String tbl_name, List
> part_vals, PartitionDropOptions options);
> boolean dropPartition(String catName, String db_name, String tbl_name,
> List part_vals, PartitionDropOptions options);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, boolean deleteData, boolean
> ifExists, boolean needResults);
> List dropPartitions(String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> List dropPartitions(String catName, String dbName, String tblName,
> List> partExprs, PartitionDropOptions options);
> boolean dropPartition(String db_name, String tbl_name, String name, boolean
> deleteData);
> boolean dropPartition(String catName, String db_name, String tbl_name, String
> name, boolean deleteData){code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930474#comment-16930474
]
Hive QA commented on HIVE-22169:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980377/HIVE-22169.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16760 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18610/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18610/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18610/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980377 - PreCommit-HIVE-Build
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch,
> HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely updated HIVE-22202: - Status: Patch Available (was: In Progress) > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > Attachments: HIVE-22202.1.patch > > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * simplify collection handling > ** use member sort() instead of Collections.sort() > ** use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely updated HIVE-22202: - Attachment: HIVE-22202.1.patch > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > Attachments: HIVE-22202.1.patch > > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * simplify collection handling > ** use member sort() instead of Collections.sort() > ** use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930438#comment-16930438
]
Hive QA commented on HIVE-22169:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
0s{color} | {color:blue} ql in master has 1573 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} ql generated 0 new + 1572 unchanged - 1 fixed = 1572
total (was 1573) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 0s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18610/dev-support/hive-personality.sh
|
| git revision | master / 1880bc6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18610/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch,
> HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21884.10.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21844.09.patch, HIVE-21884.01.patch, HIVE-21884.02.patch, > HIVE-21884.03.patch, HIVE-21884.09.patch, HIVE-21884.10.patch, > HIVE-21884.10.patch, Scheduled queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930417#comment-16930417
]
Hive QA commented on HIVE-22163:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980373/HIVE-22163.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 16761 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestObjectStore.catalogs (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDatabaseOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDeprecatedConfigIsOverwritten
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropParitionsCleanup
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSQLDropPartitionsCacheCrossSession
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testDirectSqlErrorMetrics
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testEmptyTrustStoreProps
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMasterKeyOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testMaxEventResponse
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testPartitionOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testQueryCloseOnError
(batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testRoleOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testTableOps (batchId=233)
org.apache.hadoop.hive.metastore.TestObjectStore.testUseSSLProperty
(batchId=233)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18609/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18609/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18609/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980373 - PreCommit-HIVE-Build
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (HIVE-22177) Beeline query for special character like £ does not give back result
[
https://issues.apache.org/jira/browse/HIVE-22177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930171#comment-16930171
]
philipse edited comment on HIVE-22177 at 9/16/19 10:35 AM:
---
[~sanjurm16], [~nareshpr]i am wondering if we have a serialize format like orc
to deal with the un-utf character,we will just need to create the table with
this kind of format if we know we may store un-utf characters ,and then we
can use the data as normal data ,instand of using the reflect function every
time. for i notice there are other issues map also related to this
onehttps://issues.apache.org/jira/browse/HIVE-3677
was (Author: 小郭飞飞刀):
[~sanjurm16] [~nareshpr]i am wondering if we have a serialize format like orc
to deal with the un-utf character,we will just need to create the table with
this kind of format if we know we may store un-utf characters ,and then we
can use the data as normal data ,instand of using the reflect function every
time. for i notice there are other issues map also related to this
one[https://issues.apache.org/jira/browse/HIVE-3677]
> Beeline query for special character like £ does not give back result
>
>
> Key: HIVE-22177
> URL: https://issues.apache.org/jira/browse/HIVE-22177
> Project: Hive
> Issue Type: Bug
> Components: Beeline
>Affects Versions: 1.1.0, 2.3.6
> Environment: Cloudera 5.15.1 Hive 1.1.0
> beeline and Impala
>Reporter: Sanjay Mishra
>Priority: Major
>
> I have a table in hive with a column having special character like £. When I
> try to a query as simple as 'SELECT * FROM TABLE WHERE COL = "£"; ', the
> query does not come back with any result.
> The table data is UTF-8 encoded. The sql file that contains the above sql and
> run via beeline is also UTF-8 encoded.
> Interestingly, if I try to create the run the same query using impala against
> the same table, I get back the columns containing the £ symbol.
> Also when the data from the table is exported (SELECT * FROM THE TABLE and
> save the output to csv file) the column with special character is shown as
> NULL.
>
> Steps to replicate:
> # create test.sql file with the below content:
> #
> {code:java}
> CREATE TABLE T1 (test String);
> INSERT INTO T1 VALUES('£');
> {code}
> # beeline>!run test.sql
> # create another file test_select.sql with the below content:
> #
> {code:java}
> SELECT * FROM T1 WHERE test='£';{code}
> # beeline>!run test_select.sql
> # No results will be returned back
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22197) Common Merge join throwing class cast exception
[ https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930400#comment-16930400 ] mahesh kumar behera commented on HIVE-22197: [~vgarg] I have added a test for the patch. Please check if it's fine. > Common Merge join throwing class cast exception > > > Key: HIVE-22197 > URL: https://issues.apache.org/jira/browse/HIVE-22197 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch > > > In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed > and stored in the writable format, but the object inspector is initialized to > default. So when join operator is fetching the data from dummy store > operator, its getting the OI is Long and the row as LongWritable. This is > causing the class cast exception. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22197) Common Merge join throwing class cast exception
[ https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22197: --- Attachment: HIVE-22197.02.patch > Common Merge join throwing class cast exception > > > Key: HIVE-22197 > URL: https://issues.apache.org/jira/browse/HIVE-22197 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch > > > In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed > and stored in the writable format, but the object inspector is initialized to > default. So when join operator is fetching the data from dummy store > operator, its getting the OI is Long and the row as LongWritable. This is > causing the class cast exception. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22197) Common Merge join throwing class cast exception
[ https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22197: --- Status: Patch Available (was: Open) > Common Merge join throwing class cast exception > > > Key: HIVE-22197 > URL: https://issues.apache.org/jira/browse/HIVE-22197 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22197.01.patch, HIVE-22197.02.patch > > > In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed > and stored in the writable format, but the object inspector is initialized to > default. So when join operator is fetching the data from dummy store > operator, its getting the OI is Long and the row as LongWritable. This is > causing the class cast exception. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22197) Common Merge join throwing class cast exception
[ https://issues.apache.org/jira/browse/HIVE-22197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] mahesh kumar behera updated HIVE-22197: --- Status: Open (was: Patch Available) > Common Merge join throwing class cast exception > > > Key: HIVE-22197 > URL: https://issues.apache.org/jira/browse/HIVE-22197 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 4.0.0 >Reporter: mahesh kumar behera >Assignee: mahesh kumar behera >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-22197.01.patch > > > In DummyStoreOperator the row is cached to fix HIVE-5973. The row is copyed > and stored in the writable format, but the object inspector is initialized to > default. So when join operator is fetching the data from dummy store > operator, its getting the OI is Long and the row as LongWritable. This is > causing the class cast exception. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930386#comment-16930386
]
Hive QA commented on HIVE-22163:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 9m
10s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
45s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
30s{color} | {color:blue} ql in master has 1573 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 13s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18609/dev-support/hive-personality.sh
|
| git revision | master / 1880bc6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18609/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5
[jira] [Updated] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely updated HIVE-22202: - Description: Remove unnecessary code * remove unnecessary/unused variables * remove unused private methods * remove unnecessary throws declarations * remove unnecessary conditionals that are guaranteed to always give the same result * simplify collection handling ** use member sort() instead of Collections.sort() ** use addAll() instead of adding one-by-one in a loop * remove Javadoc that doesn't give additional information * use try with resource where possible * remove unnecessary explicit type arguments * remove redundant warning suppression was: Remove unnecessary code * remove unnecessary/unused variables * remove unused private methods * remove unnecessary throws declarations * remove unnecessary conditionals that are guaranteed to always give the same result * simplify collection handling ** use foreach where possible ** use member sort() instead of Collections.sort() ** use addAll() instead of adding one-by-one in a loop * remove Javadoc that doesn't give additional information * use try with resource where possible * remove unnecessary explicit type arguments * remove redundant warning suppression > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * simplify collection handling > ** use member sort() instead of Collections.sort() > ** use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22083) Values of tag order cannot be null, so it can be "byte" instead of "Byte"
[
https://issues.apache.org/jira/browse/HIVE-22083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930350#comment-16930350
]
Hive QA commented on HIVE-22083:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980369/HIVE-22083.6.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 16760 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query23]
(batchId=299)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query88]
(batchId=299)
org.apache.hive.jdbc.TestActivePassiveHA.testClientConnectionsOnFailover
(batchId=284)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18608/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18608/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18608/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980369 - PreCommit-HIVE-Build
> Values of tag order cannot be null, so it can be "byte" instead of "Byte"
> -
>
> Key: HIVE-22083
> URL: https://issues.apache.org/jira/browse/HIVE-22083
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Ivan Suller
>Assignee: Ivan Suller
>Priority: Minor
> Attachments: HIVE-22083.1.patch, HIVE-22083.2.patch,
> HIVE-22083.3.patch, HIVE-22083.4.patch, HIVE-22083.4.patch,
> HIVE-22083.4.patch, HIVE-22083.5.patch, HIVE-22083.6.patch
>
>
> Values of tag order cannot be null, so it can be "byte" instead of "Byte".
> Switching between Byte and byte is "cheap" - the Byte objects are cached by
> the JVM - but it still costs a bit more memory and CPU usage.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Lavati updated HIVE-22169:
Attachment: HIVE-22169.1.patch
Assignee: Gopal V (was: David Lavati)
Status: Patch Available (was: In Progress)
Resubmitting
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch,
> HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Lavati updated HIVE-22169:
Status: In Progress (was: Patch Available)
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22169) Tez: SplitGenerator tries to look for plan files which won't exist for Tez
[
https://issues.apache.org/jira/browse/HIVE-22169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Lavati reassigned HIVE-22169:
---
Assignee: David Lavati (was: Gopal V)
> Tez: SplitGenerator tries to look for plan files which won't exist for Tez
> --
>
> Key: HIVE-22169
> URL: https://issues.apache.org/jira/browse/HIVE-22169
> Project: Hive
> Issue Type: Bug
>Reporter: Gopal V
>Assignee: David Lavati
>Priority: Major
> Attachments: HIVE-22169.1.patch, HIVE-22169.1.patch
>
>
> {code}
> at
> org.apache.hadoop.hive.ql.exec.Utilities.clearWork(Utilities.java:310)
> at
> org.apache.hadoop.hive.ql.exec.tez.HiveSplitGenerator.initialize(HiveSplitGenerator.java:318)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method
> {code}
> The split generator tries to clear out the work items from HDFS, which will
> never exist for Tez plans.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22083) Values of tag order cannot be null, so it can be "byte" instead of "Byte"
[
https://issues.apache.org/jira/browse/HIVE-22083?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930326#comment-16930326
]
Hive QA commented on HIVE-22083:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
11s{color} | {color:blue} ql in master has 1573 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} ql: The patch generated 0 new + 599 unchanged - 51
fixed = 599 total (was 650) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 4m
29s{color} | {color:red} ql generated 4 new + 1566 unchanged - 7 fixed = 1570
total (was 1573) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 18s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | org.apache.hadoop.hive.ql.plan.HashTableSinkDesc.getTagOrder() may expose
internal representation by returning HashTableSinkDesc.tagOrder At
HashTableSinkDesc.java:by returning HashTableSinkDesc.tagOrder At
HashTableSinkDesc.java:[line 263] |
| | org.apache.hadoop.hive.ql.plan.HashTableSinkDesc.setTagOrder(byte[]) may
expose internal representation by storing an externally mutable object into
HashTableSinkDesc.tagOrder At HashTableSinkDesc.java:by storing an externally
mutable object into HashTableSinkDesc.tagOrder At HashTableSinkDesc.java:[line
268] |
| | org.apache.hadoop.hive.ql.plan.JoinDesc.getTagOrder() may expose internal
representation by returning JoinDesc.tagOrder At JoinDesc.java:by returning
JoinDesc.tagOrder At JoinDesc.java:[line 419] |
| | org.apache.hadoop.hive.ql.plan.JoinDesc.setTagOrder(byte[]) may expose
internal representation by storing an externally mutable object into
JoinDesc.tagOrder At JoinDesc.java:by storing an externally mutable object
into JoinDesc.tagOrder At JoinDesc.java:[line 429] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18608/dev-support/hive-personality.sh
|
| git revision | master / 1880bc6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18608/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18608/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Values of tag order cannot be null, so it can be "byte" instead of "Byte"
> -
>
> Key: HIVE-22083
> URL: https://issues.apache.org/jira/browse/HIVE-22083
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Ivan Suller
>
[jira] [Updated] (HIVE-22202) Remove unnecessary code from ObjectStore.java
[ https://issues.apache.org/jira/browse/HIVE-22202?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hankó Gergely updated HIVE-22202: - Description: Remove unnecessary code * remove unnecessary/unused variables * remove unused private methods * remove unnecessary throws declarations * remove unnecessary conditionals that are guaranteed to always give the same result * simplify collection handling ** use foreach where possible ** use member sort() instead of Collections.sort() ** use addAll() instead of adding one-by-one in a loop * remove Javadoc that doesn't give additional information * use try with resource where possible * remove unnecessary explicit type arguments * remove redundant warning suppression was: Remove unnecessary code * remove unnecessary/unused variables * remove unused private methods * remove unnecessary throws declarations * remove unnecessary conditionals that are guaranteed to always give the same result * remove unnecessary method parameters * remove never used return values * simplify collection handling * use foreach where possible * use member sort() instead of Collections.sort() * use addAll() instead of adding one-by-one in a loop * remove Javadoc that doesn't give additional information * use try with resource where possible * remove unnecessary explicit type arguments * remove redundant warning suppression > Remove unnecessary code from ObjectStore.java > - > > Key: HIVE-22202 > URL: https://issues.apache.org/jira/browse/HIVE-22202 > Project: Hive > Issue Type: Task > Components: Hive >Reporter: Hankó Gergely >Assignee: Hankó Gergely >Priority: Minor > > Remove unnecessary code > * remove unnecessary/unused variables > * remove unused private methods > * remove unnecessary throws declarations > * remove unnecessary conditionals that are guaranteed to always give the > same result > * simplify collection handling > ** use foreach where possible > ** use member sort() instead of Collections.sort() > ** use addAll() instead of adding one-by-one in a loop > * remove Javadoc that doesn't give additional information > * use try with resource where possible > * remove unnecessary explicit type arguments > * remove redundant warning suppression -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available (was: Open)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Open (was: Patch Available)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Attachment: HIVE-22163.5.patch
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22065) Code cleanup around org.apache.hadoop.hive.ql.exec.tez.RecordProcessor
[ https://issues.apache.org/jira/browse/HIVE-22065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930304#comment-16930304 ] Zoltan Haindrich commented on HIVE-22065: - +1 > Code cleanup around org.apache.hadoop.hive.ql.exec.tez.RecordProcessor > -- > > Key: HIVE-22065 > URL: https://issues.apache.org/jira/browse/HIVE-22065 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-22065.1.patch, HIVE-22065.2.patch > > > I addressed the following issues: > - replace explicit generic definitions with diamond operator where applicable > - fix field visibility > - use lambda expression instead of instantiating a custom interface > implementation > - remove unnecessary null checks -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22083) Values of tag order cannot be null, so it can be "byte" instead of "Byte"
[ https://issues.apache.org/jira/browse/HIVE-22083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Suller updated HIVE-22083: --- Attachment: HIVE-22083.6.patch > Values of tag order cannot be null, so it can be "byte" instead of "Byte" > - > > Key: HIVE-22083 > URL: https://issues.apache.org/jira/browse/HIVE-22083 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Ivan Suller >Assignee: Ivan Suller >Priority: Minor > Attachments: HIVE-22083.1.patch, HIVE-22083.2.patch, > HIVE-22083.3.patch, HIVE-22083.4.patch, HIVE-22083.4.patch, > HIVE-22083.4.patch, HIVE-22083.5.patch, HIVE-22083.6.patch > > > Values of tag order cannot be null, so it can be "byte" instead of "Byte". > Switching between Byte and byte is "cheap" - the Byte objects are cached by > the JVM - but it still costs a bit more memory and CPU usage. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930294#comment-16930294
]
Hive QA commented on HIVE-22163:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980367/HIVE-22163.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 16729 tests
executed
*Failed tests:*
{noformat}
TestDataSourceProviderFactory - did not produce a TEST-*.xml file (likely timed
out) (batchId=233)
TestObjectStore - did not produce a TEST-*.xml file (likely timed out)
(batchId=233)
org.apache.hadoop.hive.llap.cache.TestBuddyAllocator.testMTT[2] (batchId=361)
org.apache.hadoop.hive.metastore.TestHiveMetaStoreAlterColumnPar.org.apache.hadoop.hive.metastore.TestHiveMetaStoreAlterColumnPar
(batchId=246)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18607/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18607/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18607/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980367 - PreCommit-HIVE-Build
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22205) Upgrade zookeeper and curator versions
[ https://issues.apache.org/jira/browse/HIVE-22205?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930283#comment-16930283 ] Peter Vary commented on HIVE-22205: --- [~ngangam]: Is there an independent ZooKeeper dependency in standalone-metastore? I kinda remember that the pom is independent there. The same is true for storage-api, but that should not depend on ZooKeeper :) > Upgrade zookeeper and curator versions > -- > > Key: HIVE-22205 > URL: https://issues.apache.org/jira/browse/HIVE-22205 > Project: Hive > Issue Type: Improvement > Components: HiveServer2 >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Minor > Attachments: HIVE-22205.patch > > > Other components like hadoop have switched to using new ZK versions. So these > jars end up in classpath for hive services and could cause issues due to > in-compatible curator versions that hive uses. > So it makes sense for hive to upgrade the ZK and curator versions to try to > keep up. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930262#comment-16930262
]
Hive QA commented on HIVE-22163:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
58s{color} | {color:blue} ql in master has 1573 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
15s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 33s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18607/dev-support/hive-personality.sh
|
| git revision | master / 1880bc6 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18607/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic
[jira] [Updated] (HIVE-22208) Column name with reserved keyword is unescaped when query including join on table with mask column is re-written
[
https://issues.apache.org/jira/browse/HIVE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riju Trivedi updated HIVE-22208:
Description:
Join query involving table with mask column and other having reserved keyword
as column name fails with SemanticException during parsing re-written query :
Original Query :
{code:java}
select a.`date`, b.nm
from sample_keyword a
join sample_mask b
on b.id = a.id;
{code}
Re-written Query :
{code:java}
select a.date, b.nm
from sample_keyword a
join (SELECT `id`, CAST(mask_hash(nm) AS string) AS `nm`,
BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM
`default`.`sample_mask` )`b`
on b.id = a.id;
{code}
Re-written query does not have escape quotes for date column which cause
SemanticException while parsing :
{code:java}
org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize input
near 'a' '.' 'date' in selection target
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(
SemanticAnalyzer.java:12084)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
SemanticAnalyzer.java:12298)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
CalcitePlanner.java:360)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
BaseSemanticAnalyzer.java:289)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
{code}
was:
Join query involving table with mask column and other having reserved keyword
as column name fails with SemanticException during parsing re-written query :
Original Query :
{code:java}
select a.`date`, b.nm
from sample_keyword a
join sample_mask b
on b.id = a.id;
{code}
Re-written Query :
{code:java}
select a.date, b.nm
from sample_keyword a
join (SELECT `id`, CAST(mask_hash(nm) AS string) AS `nm`,
BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM
`default`.`sample_mask` )`b`
on b.id = a.id;
{code}
Re-written query does not have escape quotes for date column which cause
SemanticException while parsing :
{code:java}
org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize input
near 'a' '.' 'date' in selection target
at
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(
SemanticAnalyzer.java:12084)
at org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
SemanticAnalyzer.java:12298)
at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
CalcitePlanner.java:360)
at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
BaseSemanticAnalyzer.java:289)
at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
{code}
> Column name with reserved keyword is unescaped when query including join on
> table with mask column is re-written
>
>
> Key: HIVE-22208
> URL: https://issues.apache.org/jira/browse/HIVE-22208
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Riju Trivedi
>Priority: Critical
>
> Join query involving table with mask column and other having reserved
> keyword as column name fails with SemanticException during parsing re-written
> query :
> Original Query :
> {code:java}
> select a.`date`, b.nm
> from sample_keyword a
> join sample_mask b
> on b.id = a.id;
> {code}
> Re-written Query :
>
> {code:java}
> select a.date, b.nm
> from sample_keyword a
> join (SELECT `id`, CAST(mask_hash(nm) AS string) AS `nm`,
> BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM
> `default`.`sample_mask` )`b`
> on b.id = a.id;
> {code}
> Re-written query does not have escape quotes for date column which cause
> SemanticException while parsing :
> {code:java}
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near 'a' '.' 'date' in selection target
>
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(
> SemanticAnalyzer.java:12084)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
> SemanticAnalyzer.java:12298)
> at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
> CalcitePlanner.java:360)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
>
[jira] [Updated] (HIVE-22208) Column name with reserved keyword is unescaped when query including join on table with mask column is re-written
[
https://issues.apache.org/jira/browse/HIVE-22208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Riju Trivedi updated HIVE-22208:
Summary: Column name with reserved keyword is unescaped when query
including join on table with mask column is re-written (was: Column name with
reserved keyword is unescaped when query includes join on table with mask
column is re-written)
> Column name with reserved keyword is unescaped when query including join on
> table with mask column is re-written
>
>
> Key: HIVE-22208
> URL: https://issues.apache.org/jira/browse/HIVE-22208
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Riju Trivedi
>Priority: Critical
>
> Join query involving table with mask column and other having reserved
> keyword as column name fails with SemanticException during parsing re-written
> query :
> Original Query :
> {code:java}
> select a.`date`, b.nm
> from sample_keyword a
> join sample_mask b
> on b.id = a.id;
> {code}
> Re-written Query :
>
> {code:java}
> select a.date, b.nm
> from sample_keyword a
> join (SELECT `id`, CAST(mask_hash(nm) AS string) AS `nm`,
> BLOCK__OFFSET__INSIDE__FILE, INPUT__FILE__NAME, ROW__ID FROM
> `default`.`sample_mask` )`b`
> on b.id = a.id;
> {code}
> Re-written query does not have escape quotes for date column which cause
> SemanticException while parsing :
>
>
> {code:java}
> org.apache.hadoop.hive.ql.parse.ParseException: line 1:9 cannot recognize
> input near 'a' '.' 'date' in selection target
>
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.rewriteASTWithMaskAndFilter(
> SemanticAnalyzer.java:12084)
> at
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.analyzeInternal(
> SemanticAnalyzer.java:12298)
> at org.apache.hadoop.hive.ql.parse.CalcitePlanner.analyzeInternal(
> CalcitePlanner.java:360)
> at org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer.analyze(
> BaseSemanticAnalyzer.java:289)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:664)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1869)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Attachment: HIVE-22163.5.patch
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Patch Available (was: Open)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22163) CBO: Enabling CBO turns on stats estimation, even when the estimation is disabled
[
https://issues.apache.org/jira/browse/HIVE-22163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Krisztian Kasa updated HIVE-22163:
--
Status: Open (was: Patch Available)
> CBO: Enabling CBO turns on stats estimation, even when the estimation is
> disabled
> -
>
> Key: HIVE-22163
> URL: https://issues.apache.org/jira/browse/HIVE-22163
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Reporter: Gopal V
>Assignee: Krisztian Kasa
>Priority: Major
> Attachments: HIVE-22163.1.patch, HIVE-22163.1.patch,
> HIVE-22163.1.patch, HIVE-22163.2.patch, HIVE-22163.3.patch,
> HIVE-22163.4.patch, HIVE-22163.4.patch, HIVE-22163.5.patch,
> HIVE-22163.5.patch, HIVE-22163.5.patch
>
>
> {code}
> create table claims(claim_rec_id bigint, claim_invoice_num string, typ_c int);
> alter table claims update statistics set
> ('numRows'='1154941534','rawDataSize'='1135307527922');
> set hive.stats.estimate=false;
> explain extended select count(1) from claims where typ_c=3;
> set hive.stats.ndv.estimate.percent=5e-7;
> explain extended select count(1) from claims where typ_c=3;
> {code}
> Expecting the standard /2 for the single filter, but we instead get 5 rows.
> {code}
> 'Map Operator Tree:'
> 'TableScan'
> ' alias: claims'
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 5 Data size: 19 Basic stats:
> COMPLETE Column stats: NONE'
> {code}
> The estimation is in effect, as changing the estimate.percent changes this.
> {code}
> ' filterExpr: (typ_c = 3) (type: boolean)'
> ' Statistics: Num rows: 1154941534 Data size: 4388777832
> Basic stats: COMPLETE Column stats: NONE'
> ' GatherStats: false'
> ' Filter Operator'
> 'isSamplingPred: false'
> 'predicate: (typ_c = 3) (type: boolean)'
> 'Statistics: Num rows: 230988307 Data size: 877755567
> Basic stats: COMPLETE Column stats: NONE'
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
